Enhanced HTTP support for Connection API

The default HTTP protocol support behind the zero.core.connection.Connection API is currently limited in 3 key areas:

• HTTP basic authentication is not supported.
• SSL is not supported.
• HTTP proxies are not supported.

Some of these limitations stem from the fact that the default HttpTransport implementation class is currently based on java.net.HttpUrlConnection and we'd need to find thread-safe solutions to these limitations in order to support calling out to many services simultaneously from a single Zero application. In addition, HttpTransport is only registered for the http protocol in the integration fabric; there is currently no default registration for the https protocol.

This article outlines our thoughts on how to improve the HTTP support for the Connection API. To find our more about how transports are implemented for the Connection API see Creating a custom protocol transport.

Status

The enhanced HTTP support is being delivered under bug 691. Completed items are currently available in M3 zero.core builds:

Overview

The integration fabric will treat http and https as independent protocols (albeit with similar behavior and common implementation classes). As today, the http protocol implementation will expect a target beginning http: and be extension the https implementation will expect the target to be a URL starting https:. The usual rule for honoring protocol configuration explicitly set for a request (e.g. the <connection> element) will apply. Thus the implementation will throw an IntegrationException if a request with http configuration has an https: target, or vice versa.

All configuration for http and http will take place using integration fabric (and hence Connection API) protocol parameters with no special treatment for http or https in the protocol-neutral integration fabric engine. Thus, the configuration for http and https may be specified in the following places, in decreasing order of precedence:

• In the integration_fabric.xml configuration for the <destination> (either by explicit use of the <connection> element or programmatically by a configured mediation Step.
• Set by the application for an individual Connection request (using the Connection.setProtocolConfiguration() method).
• By default for all http and https connections in zero.config:
  • [/config/connection/transports/http]
  • [/config/connection/transports/https]

We do not prevent the user from setting authentication parameters in the zero.config although we should steer the user away from doing so, in order to prevent authentication credentials from being indiscriminately sent with all outbound requests. For this reason, we will look into supporting named pieces of reusable configuration in integration_fabric.xml.

Configuration in integration_fabric.xml associated with destinations using a simple wildcard naming scheme. We do not intend to modify this behavior at this time.

Authentication parameters must be set for each outbound request using the mechanisms described above. There will be no automatic propagation of user credentials. (Note: we did talk about whether it would make sense to forward OpenID credentials but on investigation it would appear that if it were appropriate it would be an advanced scenario so is not included here)

Configuration

The current HttpTransport implementation does not recognize any protocol parameters. In order to remove the limitations outlined above it is proposed that the http protocol be given the following configuration parameters:

We clearly need to avoid all these passwords appearing in plain text in zero.config and intergration_fabric.xml. We can probably leverage Zero's obfuscation mechanisms but need to think about what this would look like.

Questions:

• Should the default values for proxyHost and proxyPort be the values of the http.proxyHost and http.proxyPort system properties, rather than none?
• Are proxyUserid and proxyPassword required?

The following configuration parameters are proposed for the https protocol:

The httpsConfig parameter identifies a member of a set of outbound https connection configurations. These are defined in zero.config under the key /config/connection/https, for example:

/config/connection/https += [{
  "name" : "workingConfig1",
  "trustStore" : "config/keystore.p12",
  "trustStorePassword" : "password",
  "trustStoreType" : "PKCS12"
}]

Handling of HTTP status codes