M4 Core Enhancements

The following changes have been implemented in the b_core_M4 SVN branch. These changes will be moved into trunk on Jan. 29 at 9:00 am ET.

Developers can prepare for the changes in trunk by checking out the zero.core project from the branch and using the workspace resolver. Problems should be reported in the associated bug, if specified; otherwise open a new bug that blocks bug 2391.

Config files

Modified conditions syntax for event handler configuration

bug 2075

Changed conditions syntax for event-handler registration

In M3, the conditions attribute was a list of strings. The elements of the list were conditions; the list implied a logical "AND" of the elements. For example:

/config/handlers += [{
   "events" : "resolveHandlers",
   "handler" : "zero.core.views.ErrorResolver.class",
   "conditions" : ["/event/resolvingEvent matches render", "/request/view matches error"]
}]

The new support introduces operators, so we can now rewrite the example as:

   "conditions" : "(/event/resolvingEvent =~ render) && (/request/view =~ error)"

Specific details of the change:

M3	New
matches	=~
urimatches	==
not matches	!~
not urimatches	!=
"[/app/flag exists"]	"/app/flag"
"[/event/resolvingEvent matches render", "/request/view matches error"]	"(/event/resolvingEvent =~ render) && (/request/view =~ error)"

In the new syntax, conditions are grouped with parentheses. You can negate grouped conditions with the exclamation mark. For example:

   "conditions" : "!((/event/resolvingEvent =~ render) && (/request/view =~ error))"

Further, we've added support for logical "OR" with conditions. This is new; was not supported in M3. Expressed with "||" operator, similar to the "&&".

Removed support for true condition

In M3, the conditions attribute was required. The true condition was specified explicitly, as follows:

/config/handlers += [{
   "events" : ["http"],
   "handler" : "zero.core.cfadapter.ChannelFrameworkAdapter.class",
   "conditions" : "true"
}]

Now the conditions attribute is optional. If not specified, then it's treated as an implicit true condition.

Further, the explicit true condition is no longer supported (error).

Require /config/handlers specified with list syntax

bug 2456

/config/handlers has always been stored in the config zone as a list of maps. However, for M3, we special cased /config/handlers so that you could append maps directly without requiring the list brackets. This was probably not intuitive, so the special case has been removed. You now must specify /config/handlers as a list.

Previously allowed (not valid any more):

/config/handlers += {
   "events" : "render",
   "handler" : "zero.core.views.JSONHandler.class",
   "conditions" : "/request/view =~ JSON"
}

Replaced by the required syntax (note the brackets around the map):

/config/handlers += [{
   "events" : "render",
   "handler" : "zero.core.views.JSONHandler.class",
   "conditions" : "/request/view =~ JSON"
}]

You can also specify multiple registrations within one list:

/config/handlers += [
{
   "events" : "resolveHandlers",
   "handler" : "zero.core.fileserver.FileServerResolver.class",
   "conditions" : "/event/resolvingEvent =~ (GET|POST|DELETE|PUT|HEAD)"
},
{
   "events" : "resolveHandlers",
   "handler" : "zero.core.views.ErrorResolver.class",
   "conditions" : "(/event/resolvingEvent =~ render) && (/request/view =~ error)"
},
{
   "events" : "resolveHandlers",
   "handler" : "zero.core.views.ViewResolver.class",
   "conditions" : "(/event/resolvingEvent =~ render)"
}]

Changed conditions syntax for security-handler registration

To more closely align with how other handlers are registered, we modified the syntax for defining security rules. For instance, here is a form based security rule.

@include "security/rule.config"{
   "conditions" : "(/request/path =~ /formauth/getonly(/.*)?) && (/request/method == GET)",
   "authType" : "Form",
   "groups" : ["ALL_AUTHENTICATED_USERS"]
}

The thought was that security was hiding too much of the URI matching that was being performed and felt that it made more sense to explicitly expose the conditions clause of the event handler for security. In the example above, we are defining a set of conditions that require the request.path and request.method to match to fire this security rule.

Changed conditions syntax for acf-handler registration

To more closely align with how other handlers are registered, we modified the syntax for defining ACF rules. For instance, here is a simple ACF rule.

@include "acf.config"{
   "conditions" : "/request/path == /Yahoo01.html",
   "filterRuleFile" : "acf-custom-config.xml"
}

The thought was similar to security, ACF was hiding too much of the URI matching that was being performed and felt that it made more sense to explicitly expose the conditions clause of the event handler for ACF.

Simplified include syntax

In addition to the fully qualified syntax for includes, you can specify a virtual location for includes. For instance, here is a basic security rule (also note this was used for the example config stanzas for ACF and Security above):

@include "security/rule.config"{
   "conditions" : "/request/path =~ (/|/index.gt(/.*)?)",
   "authType" : "RP",
   "csrfProtect" : false,
   "roles" : ["OPENID_ROLE"]
}

The processing of the includes will search each dependency virtual directory relative to the file that is doing the include. In this example, the zero.config is zero.core.security.tests/config is including a rule.config location in zero.core/config/security named "rule.config". Note first match found wins.

Support comments throughout the config file

bug 2091

Comments may now be placed anywhere within a config file. Comments start with a hash (#) and extend to the end of the line.

For example:

# List
/config/list = ["foo"]
/config/list += ["bar",   # in-line comment
   "can"]

Changed escaped '$'

bug 2229

$ is a reserved character in JSON strings within config files (e.g. ${var} is the syntax for a variable reference). A literal $ can be part of a string by using an escape syntax.

Was \\$; now $$

Added source metadata

Additional metadata is now stored in the config zone:

• /config/_src/{uri} contains the location (file:line:column) from which the entry at {uri} was loaded
• /config/_files contains a list of file names (strings) from which the configuration was loaded

An API is provided for accessing the location information: ConfigLoader.getConfigSource(String uri).

This information might be useful for system-level developers; perhaps not needed by application developers.

Global Context

Added support for nested updates (zput/zpost/zdelete)

bug 2512

Arbitrary value paths are now supported in the update APIs. If the value path doesn't match the stored value, then the update API returns false.

Modified zput and zpost return values

In M3, these APIs returned the stored value that was replaced/modified.

The concept of the "replaced" value is not so intuitive with the new support for nested updates. These APIs now return a boolean, which indicates whether the stored value was modified.

Added zlistAll method

zlistAll(String uri [, boolean includePrefix]) lists all GC URIs that start with uri. zlist(String uri) returns only children of uri.

Throw exception for zlist with value path

zlist support path only. Invoking zlist with a value path results in an IllegalArgumentException.

Modified behavior of zcontains on a zone

In M3, zcontains("/zone") returned true if the zone was activated. That support was inconsistent with the behavior of zcontains for other URIs.

zcontains("/zone") now returns true if there is a value stored at /zone.

Changed the mechanism for "walking" through types/type handlers

In M3, the GC APIs relied solely on the value path to determine when to apply another type handler. Once the last segment of the value path had been "consumed" by a type handler, then the value was complete.

The M3 approach didn't work for Commands and FirstElementLists, since those types handlers can return modified values even without a value path. The new mechanism loops through type/type handler processing until the output matches the input. That is, we continue to walk until the result stops changing.

Modified "no store" response from Command implementations

In M3, Command implementations returned Context.NO_STORE to indicate that the GC should not change the stored value. This was useful, for example, when using a Command that processed requests dynamically.

Commands now throw a zero.core.context.UpdateDeniedException for the same effect; Context.NO_STORE has been removed.

Modified TypeHandler interface

zero.core.context.types.TypeHandler now uses generics. This simplifies implementations because you no longer need to check/cast the stored value to the corresponding type.

public class MapTypeHandler implements TypeHandler<Map> {

    public <V> V get(ParsedURI parsedUri, Map storedMap) {
        ...

Modified security-related GC URIs

bug 2786

Some of the security-related GC URIs were modified, in keeping with the convention of separating GC data and URIs

M3	New
/request/subject#remoteUser	/request/subject/remoteUser
/request/subject#users	/request/subject/users
/request/subject#groups	/request/subject/groups
/request/subject#roles	/request/subject/roles
/request/subject#remoteCodePrincipals	/request/subject/remoteCodePrincipals

Modified ZoneHandler interface

The delete method now operates on one entry per request; the option to also delete children within the zone handler has been removed. This change was required in order to give Command implementations control over whether they can be deleted. That is, deletes are now driven through the type handlers.

The list(String path) method now returns all keys that start with path.

JSON support

Changed JSON APIs

The primary objective of this effort was to improve performance by eliminating the intermediary objects of type JSONObject and JSONArray when converting JSON data to/from Java objects. This should also have the side effect of simplifying the JSON API in Project Zero. The complete list of changes include

• Remove the JSONObject class. Any reference to JSONObject can now be replaced by a java.util.Map.
• Remove the JSONArray class. Any reference to JSONArray can now be replaced by a java.util.List.
• Replace the zero.json.JsonType class with zero.json.Json that has static utility methods for encoding (encode()) Objects as JSON encoded strings and for decoding (decode) JSON encoded strings as Java objects.
• Encoding objects
  • To encode an object as a JSON string use - String jsonStr = Json.encode(obj);
  • Allow for a formatted (embeds and spaces) JSON string - String formattedJsonStr = Json.encode(obj, true);
  • Allow encoding an object directly to an OutputStream or Writer.
  • Encode Maps as JSON objects ("{ ... }").
  • Encode Lists or Arrays as JSON arrays ("[ ... ]").
  • Encode other complex objects as JSON objects using the object's public fields, preferring getters when available.
  • Eliminate circular references to objects using the $jref notatation.
• Decoding JSON strings
  • To decode a JSON string use - Object obj = Json.decode(jsonStr);
  • Allow decoding directly from an InputStream or Reader
  • Simple types are converted automatically to the equivalent Java types (String, Integer ...)
  • A JSON Object ("{...}") is decoded to a Map. Permit preserving the order of elements in the object using a flag.
  • A JSON Array ("[...]") is decoded to a List.
• Provide for custom serialization/encoding by allowing implementations of an (zero.json.converter.)Encoder interface. Encoders are required to participate in the encoding process to eliminate circular references.
• The Converter interface extends the Encoder interface. Registration of Converters in the Global Context has not changed. The fromObject method is no longer applicable and has been removed.

Modified syntax of registered JSON custom converters

bug 2221

The GC allows developers to place information in the key and/or the value. There's no right or wrong here, but consistency helps. In the core, we've moved to a convention where the GC keys are "fixed"; tailored information appears in the value.

Was:

/config/json/converters/java.sql.Time = "zero.json.converters.java.sql.TimeConverter"

Now:

/config/json/converters += {
  "java.sql.Time" : "zero.json.converters.java.sql.TimeConverter",
  ...
}

Removed JsonComparator

This utility API has been removed.

Modified FirstElementList support

Value path support has changed for FirstElementLists. "*" is now required in order to address the full list; otherwise the value path is applied to the first element.

The changes are summarized in the following updated table:

Java API	Groovy API	Description	Can create entry
zput("/request/params/p", list)	request.params.p = list	Create list; list does not exist
zput("/request/params/p#*", list)	request.params.p['*'] = list	Replace list; list must exist
zput("/request/params/p", "bar")	request.params.p = "bar"	Set/replace first element; list must exist
zput("/request/params/p#*/0", "bar") replaces zput("/request/params/p#0", "bar")	request.params.p['*/0']	Set/replace list element; list must exist
zput("/request/params/p#p1", "bar")	request.params.p['p1'] = "bar"	Assumes a map stored at the first element; set/replace the value of "p1" with "bar" in that map
zpost("/request/params/p", list)	zpost("/request/params/p", list)	Create list; list does not exist
zpost("/request/params/p#*", "bar")	zpost("/request/params/p#*", "bar")	Append to list; list must exist
zget("/request/params/p")	x = request.params.p[]	Returns first element
zget("/request/params/p#*/0") replaces zget("/request/params/p#0")	x = request.params.p['*/0']	Returns first element
zget("/request/params/p#*")	x = request.params.p['*']	Returns full list
zdelete("/request/params/p#*")	zdelete("/request/params/p#*")	Deletes list
zdelete("/request/params/p#*/0") replaces zdelete("/request/params/p#0")	zdelete("/request/params/p#*/0")	Removes first entry

Modified Groovy support for GC access

bug 2429

Reduced GPath-like support to just read/write operations

Removed support for append, which was implemented with the left-shift operator (<<). The mechanism worked when there was no value path, but didn't work with value path, so we've removed the (partial) support.

The z* APIs are still available.

Removed the key() method from GCAccessor

The mechanism couldn't support value path, so the support has been removed.

Modified ConfigLoader APIs

bug 2457

zero.core.config.ConfigLoader contains a set of static methods. In M3, these methods were not consistent in argument types for files: sometimes used String, sometimes used File.

All of the APIs now take String for file references.

Added single-shot GC initialization

zero.core.config.ConfigLoader.initialize(String root [, Context context]) consolidates the following steps into one API:

• Load .zeroresolved.properties
• Load all of the config files in order

This API simplifies test cases that use the GC without the ApplicationLauncher.

Added ListFactory and MapFactory utilities

ListFactory

List list = ListFactory.create("p1", "p2");

is equivalent to

List list = new ArrayList();
list.add("p1");
list.add("p2");

MapFactory

Map map = MapFactory.create("p1", "v1", "p2", "v2");

is equivalent to

Map map = new HashMap();
map.put("p1", "v1");
map.put("p2", "v2");