Database access API

Convenience API to accomplish common database query patterns.

The queryFirst methods can return a java.util.Map or a Java bean, while the queryList methods can return a java.util.List of Map or Java beans.

See extending data access to learn how you can customize the return types of query methods.

Handling exceptions

Many of the methods throw the unchecked runtime exception com.ibm.pdq.runtime.exception.DataRuntimeException. You can choose to catch this exception if you want to handle the error condition.

Retrieving a map

The following query returns a Map<String,Object>:

// Java code:
String sql = "SELECT * FROM person WHERE person.name=?";
Map<String,Object> brian = data.queryFirst(sql, "Brian");
System.out.println(brian.get("id"));

// Groovy, with a list of parameters:
def sql = 'SELECT * FROM person WHERE person.name=?'
def brian = data.queryFirst(sql, ['Brian'])
println brian.id

// Groovy, using GString parameters:
def name = 'Brian'
def brian = data.queryFirst("SELECT * FROM person WHERE name=$name")
println brian.id

In this query, the resulting Map's key is the column name, and the values are the column values in the result set. If further operation using this data structure is necessary, you can cast the Object to the proper type. The queryFirst series of methods are useful when working against the first row of data; all other rows are ignored. If you want to return multiple rows, you can obtain a List of Map using queryList:

// Java code:
String sql = "SELECT * FROM person WHERE name LIKE ?";
List<Map<String,Object>> people = data.queryList(sql, "Br%");
for (Map<String, Object>> person : people)
    System.out.println(person.get("id");

// Groovy, with a list of parameters:
def people = data.queryList('SELECT * FROM person WHERE name LIKE ?', ['Br%'])

// Groovy, using GString parameters:
def name = 'Br%' def people = data.queryList("SELECT * FROM person WHERE name LIKE $name")
people.each { item -> println item.id }

This query returns a List of Map or all rows in person in which the name starts with Br. Each Map has column values keyed by column names, just as with queryFirst.

Retrieving a Java bean

To retrieve a Java bean instead of a Map, call the queryFirst method and pass in the desired class as the second parameter:

// Java code:
String sql = "SELECT * FROM person WHERE person.name=?";
Person brian = data.queryFirst(sql, Person.class, "Brian");
System.out.println(brian.getId());

// Groovy, with a list of parameters:
def sql = "SELECT * FROM person WHERE person.name=?"
def brian = data.queryFirst(sql, Person, ['Brian'])
println brian.id

// Groovy, using GString parameters:
def name = 'Brain'
def brian = data.queryFirst('SELECT * FROM person WHERE name=$name', Person)
println brian.id

Similarly, you can use the queryList method to retrieve a List of beans:

// Java code
List<Person> people = data.queryList("SELECT * FROM person", Person.class);
for (Person person : people)
    System.out.println(person.getId());

// Groovy, with a list of parameters:
def sql = 'SELECT * FROM person WHERE name LIKE ?'
def people = data.queryList(sql, Person, ['Br%'])

// Groovy, using GString parameters:
def name = 'Br%'
def people = data.queryList("SELECT * FROM person WHERE name LIKE $name", Person)
people.each { item -> println item.id }

This query also shows that all query methods take zero or more parameters and are therefore optional, depending on the needs of the given SQL string.

Retrieving simple strings

Some queries only return a single column. In this case, you avoid a more complex data structure and just get the scalar value:

// Java code:
String sql = "SELECT name FROM person WHERE name=?";
String name = data.queryFirst(sql, String.class, "Robert");

// Groovy, with a list of parameters:
def sql = 'SELECT name FROM person WHERE name=?'
def name = data.queryFirst(sql, String, 'Robert')

// Groovy, with GString parameters:
def param = 'Robert'
def name = data.queryFirst("SELECT name FROM person WHERE name=$param", String)

All of the above examples can be modified to return a List of String by changing queryFirst to queryList.

Retrieving primitive values

In most cases when the SQL SELECT statement specifies a single column a single Java primitive wrapper or simple Object is the natural representation for the selected data. Simple Object type here means a Java data type that is supported directly by JDBC.

The natural mapping for individual columns of a table to a Java primitive wrapper or simple Object is that specified by the Data Type Conversion Tables in the JDBC standard. Those tables also specify the allowed data type conversions between data types supported by the data source and Java primitive wrapper types and simple objects when data is selected from or inserted into a data source.

An aggregate function and other scenarios return a single primitive data type as shown in the following examples:

// Java code:
String sql = "SELECT COUNT(*) AS numrows FROM person";
int numrows = data.queryFirst(sql, Integer.class);
List<Integer> allIDs = data.queryFirst("SELECT id FROM person", Integer.class);

// Groovy code:
def sql = 'SELECT COUNT(*) AS numrows FROM person'
def numrows = data.queryFirst(sql, Integer)
def allIDs = data.queryFirst('SELECT id FROM person', Integer);

Using named SQL statements

The documentation for configuring zero.data described how to externalize and provide named SQL statements in configuration files to zero.data.Manager. If you provide SQL statements in this manner, you can reference your named SQL statements using the name, or key, in the JSON object in your configuration.

For instance, assume the following is in your configuration file:

/config/db/reviewDB/statements = {
    "SELECT_ALL"      : "SELECT * FROM table",
    "SELECT_WHERE_IN" : "SELECT * FROM table WHERE id IN (?,?,?)",
    "INSERT" : "INSERT INTO table (col1, col2) VALUES (?,?)"
}

With the /config/db/reviewDB/statements set, you can execute queries against the names as if it were the SQL as follows:

import zero.data.groovy.Manager

def data = Manager.create('reviewDB')
def allrows = data.queryList('SELECT_ALL')

Queries using zero.data.Manager API

Queries with Groovy

Groovy supports the parameter styles described in this document. Using zero.data.groovy.Manager, Groovy's GString enables a streamlined way to pass parameters into the query engine yet still retain the efficiency and protection of a PreparedStatement. For example, the following code snippets show a Groovy implementation and then a Java implementation of equivalent queries:

// groovy
def data = zero.data.groovy.Manager.create('mydb')
def city = 'Raleigh'
def hash = [name: 'Robert', age: 24]
def results = data.queryList("SELECT * FROM table WHERE age < ${hash.age} AND city=$city")

// java
Manager data = Manager.create("mydb");
String city = "Raleigh";
Map hash = new HashMap();
hash.put("name", "Robert");
hash.put("age", 24);
List results = data.queryList("SELECT * FROM table WHERE age < ? AND city=?", hash.get("age"), city);

List Expansion

Using the GString version of the zero.data.groovy.Manager query methods, you can let the Manager dynamically expand lists. This is useful with IN predicates. For example, the following code illustrates that without the list expansion shortcut, query code can be fairly ugly:

def list = [1,2,3]
def results = queryList("SELECT * FROM APP.TEST WHERE id IN (?,?,?)", list)

// or
def otherresults = queryList("SELECT * FROM APP.TEST WHERE id IN ($list[0],$[list[1],$list[2])")

In both cases, the SQL is hard coded for three arguments. If the list changes with input from a form, for instance, this SQL would not support changes to the number elements in the list. However, using list expansion, as follows, it is possible to accommodate dynamic sized lists:

def list = [1,2,3]
def results = queryList("SELECT * FROM APP.TEST WHERE id IN $list")

At runtime, the SQL is modified and the $list parameter is expanded to (?, ?, ?). Thus a PreparedStatement is still used and the basic protection they offer for SQL injection is maintained.

Read this if you are using Groovy

In Groovy, a Java String and a GString are different concepts and created differently. Java Strings behave just as expected in Java and are created using single quotes for example: 'I am a String'.

GStrings, on the other hand, allow for variable replacement and other expressions and are created using double quotes. For example: "I am a GString".

There is no reason to use a GString with the zero.data.groovy.Manager unless variable or expression replacement is desired. Otherwise, Groovy will perform unnecessary processing. This can be somewhat confusing at first for Java developers but this is an important distinction because zero.data.groovy.Manager cannot make use of a PreparedStatement unless a GString is passed.

In Groovy, do not concatenate GString. Use multi-line GString instead:

// DO NOT DO THIS
def sql = "SELECT * FROM table"
          + "WHERE id = $id"

// do this...
def sql = """SELECT * from table
             WHERE id = $id"""

The reason concatenation with GString is heavily discouraged is because Groovy evaluates the GString prior to concatenation and will bypass PreparedStatement optimizations and protection.

Query parameters

There are several styles of SQL parameters supported by the API for different situations:

Ordinal parameters
The classic JDBC style of binding values to delimiter placeholders, ?, in the SQL.
Named parameters
Enables you to embed named placeholders in the SQL. For example, :authorId can bind to values from a bean or a map.
Ordinal and named parameters in combination
Enables you to send multiple complex types as parameters in the form of ?2.authorId.

Ordinal parameters - "?"

Ordinal parameters are unnamed, as indicated by the anonymous ? in traditional PreparedStatement fashion. This requires a correlation order between the order of ? and the method parameters. Variable arguments are supported on all methods. However, when accessing from Groovy, a List construction is required as follows:

// java
Person person = data.queryFirst("SELECT * FROM person WHERE firstname=? AND lastname=? AND age > ?", Person.class, "George", "Bush", 65);

// groovy
def person = data.queryFirst('SELECT * FROM person WHERE firstname=? AND lastname=? AND age > ?', Person, ["George", "Bush", 65]);

Named parameters - ":columnName" or "?.columnName"

Named parameters are useful because ordered correlation between SQL parameters and method parameters is not essential. Instead of the PreparedStatement syntax using ? as a parameter placeholder, the symbol :<key> is used. Supported named parameters can be passed in as a bean or a map.

Using a map to pass in SQL parameters

The following example shows a map being used to pass in SQL parameters:

// java
Map paramMap = new HashMap();
paramMap("name","Brian");
Person person = data.queryFirst("SELECT * FROM person WHERE person.name=:name", Person.class, paramMap);

// groovy params as list
def paramMap = [name: 'Brian']
def person = data.queryFirst('SELECT * FROM person WHERE person.name=:name', Person, [paramMap])

// groovy gstring
def paramMap = [name: 'Brian']
def person = data.queryFirst("SELECT * FROM person WHERE person.name=${paramMap.name}", Person)

Before the SQL query is run, the named parameters from the SQL string are bound to values in the map.

Using a bean to pass in SQL parameters

A bean can be used to contain values (just as a map was used in the previous example). The following example uses a bean to pass SQL parameters:

// java
Person person = new Person();
person.setName("Brian");
person = data.queryFirst("SELECT * FROM person WHERE person.name=:name", Person.class, person);

// groovy params as list
def person = data.queryFirst('SELECT * FROM person WHERE person.name=:name', Person, [person]);

// groovy gstring
def person = data.queryFirst("SELECT * FROM person WHERE person.name=${person.name}", Person);

Ordinal named parameters - "?2.columnName"

Parameters for a single query can be bound from several objects. For example, order data might come from a Map where the customer data comes from an instance of Person. To address such use cases, the following syntax is supported to combine named parameters with order:

// java
Map order = // obtained perhaps through JSON HTTP input
Person customer = // obtained somehow
data.update("INSERT INTO orders (orderId, customerId, itemId, quantity, price, address) VALUES(?1.orderNum, ?2.custId, ?1.itemNum, ?1.quantity, ?1.price, ?2.address", order, customer);

// groovy with parameters as list
def order = // obtained perhaps through JSON HTTP input
def customer = // obtained somehow
data.update('INSERT INTO orders (orderId, customerId, itemId, quantity, price, address) VALUES(?1.orderNum, ?2.custId, ?1.itemNum, ?1.quantity, ?1.price, ?2.address', order, customer)

// groovy with gstring
def order = // obtained perhaps through JSON HTTP input
def customer = // obtained somehow
data.update("INSERT INTO orders (orderId, customerId, itemId, quantity, price, address) VALUES(${order.orderNum}, ${customer.custId}, ${order.itemNum}, ${order.quantity}, ${order.price}, ${customer.address}")

The code snippet in the previous example illustrates that you can use ordinal named parameters to pass in both a Person object and a Map and that the SQL references can reference parameters out of order.

Updating the database

The zero.data API has simple methods to manipulate the data in a database.

UPDATE and DELETE statements

The UPDATE and DELETE SQL statements are driven through the update() method, whereas INSERT statements should be run using the insert() method. The update method returns the number of affected rows while the insert method returns a generated key. The parameter passing rules are the same for the query methods, including Groovy's GString parameter passing for zero.data.groovy.Manager, list expansion, and Java's variable arguments and named arguments with beans and maps.

An UPDATE and DELETE example

// java
int rowsAffected = data.update("UPDATE APP.test SET initials=?,name=?", 'bjs', 'Brandon Smith');

// groovy with list parameters
def rowsAffected = data.update('UPDATE APP.test SET initials=?,name=?', ['bjs', 'Brandon Smith'])

// groovy with gstring
def initials = 'bjs'
def name = 'Brandon Smith'
def rowsAffected = data.update("UPDATE APP.test SET initials=$initials,name=$name")

INSERT statement and auto generated keys

The insert method is provided to support a shortcut to obtaining auto generated keys. There are more differences than other portions of the API between zero.data.groovy.Manager and zero.data.Manager so this documentation is separated. Common to both, however, is that the column name for the autogenerated key is passed.

// groovy using GString to pass parameters
def id = data.insert("INSERT INTO person (name, phone, email) VALUES ($name, $phone, $email)", ['id'])

// groovy using List to pass parameters
def id = data.insert('INSERT INTO person (name, phone, email) VALUES (?,?,?)', ['id'], [name, phone, email])

// java int id = data.insert("INSERT INTO person (name, phone, email) VALUES (?,?,?)", int.class, new String[] {"id"}, name, phone, email);

The major difference between zero.data.groovy.Manager and zero.data.Manager is that the Groovy version assumes, for simplicity sake, that a java.lang.Number is always returned. However, in zero.data.Manager, the return value of the method is the key itself cast to a class you define. In the above example, int.class is used to indicate to the engine to return the generated key as a Java int. The subsequent examples illustrate further uses of the zero.data.Manager insert method and assumes the following table:

CREATE TABLE catalog_entry (
    id int NOT NULL GENERATED BY DEFAULT AS IDENTITY (START WITH 1, INCREMENT BY 1),
    name varchar(30)
);

Person bean = new Person();
bean.setInitials("hp");
bean.setName("Henry Proctor");

Example: Retrieve key as int

// java
int id = data.insert("insert into APP.TEST (INITIALS, NAME) values (:initials,:name)", int.class, new String[] { "id" }, bean); assertEquals(1, id);

Example: Retrieve key as Number

// java
Number id = data.insert(sql, Number.class, new String[] { "id" }, bean); assertEquals(1, id.intValue());

Example: Retrieve key as String

// java
String id = data.insert(sql, String.class, new String[] { "id" }, bean); assertEquals("1", id);

Database transactions

The zero.data API only supports local database transactions. This document describes how to use them.

inTransaction()

Local database transactions are supported using the Manager.inTransaction() method. This method provides the convenience to implicitly demarcate the transaction boundaries. The zero.data.Manager version of inTransaction() takes as an argument an implementation of zero.data.LocalTransaction whereas the zero.data.groovy.Manager version simply takes a closure as shown in the following example:

import zero.data.LocalTransaction;
import zero.data.Manager;

// Java code:
Manager voteDB = Manager.create("votingDB");
voteDB.inTransaction(new LocalTransaction() {
    public void execute() {
        voteDB.update("INSERT...");
        voteDB.update("UPDATE...");
        voteDB.update("DELETE...");
    }
});

// Groovy code:
def voteDB = zero.data.groovy.Manager.create('votingDB')
voteDB.inTransaction {
    voteDB.update("INSERT...")
    voteDB.update("UPDATE...")
    voteDB.update("DELETE...")
}

During the inTransaction demarcation, all query calls use the same Connection. If an exception is thrown either from query calls or explicitly by the application, the local transaction is rolled back. Otherwise, the transaction is committed and the Connection is closed. For example, the following code illustrates manually throwing an exception based on a condition:

// Java code:
import zero.data.Manager;
...
Manager voteDB = Manager.create("votingDB");
voteDB.inTransaction(new LocalTransaction() {
    public void execute() {
        voteDB.update("INSERT...");
        voteDB.update("UPDATE...");
        if (/* some condition */)
            throw new Exception("...");
    }
});

// Groovy code:
def voteDB = zero.data.groovy.Manager.create('votingDB')
voteDB.inTransaction {
    voteDB.update("INSERT...") voteDB.update("UPDATE...")
    if (/* some condition */)
        throw new Exception("...");
}

Manual transactions

Transaction demarcation can also be done directly calling transaction methods. The following methods are provided:

beginTransaction()
commitTransaction()
rollbackTransaction()

The implicit local transaction shown in the previous example can also be achieved by doing the following:

data.startTransaction();
try {
    data.update("INSERT...");
    data.update("UPDATE...");
    data.update("DELETE...");
    data.commitTransaction();
} catch (Exception e) {
    data.rollbackTransaction();
} finally {
    data.endTransaction();
}

Advanced query patterns

While zero.data has a simplified API for the most common data access use cases, some situtations require a bit more advanced querying patterns.

Using JoinResultHandler

JPA, Hibernate, and even iBATIS provide functionality to map relational joins into a hierarchy of beans. JoinResultHandler does just that given a constrained set of assumptions about the SELECT statement and beans. If your use case meets these constraints, this handler can save a lot of coding efforts.

Constrained assumptions

The domain classes, or beans, must meet certain criteria in order to be mapped to a joined ResultSet. These rules are listed below:

  1. All beans must meet the common Java Bean requirements. Namely, getters and setters for bean properties using the proper method naming syntax.
  2. Beans may subclass.
  3. Beans must have one or more properties that map to primary key(s).
  4. The primary key properties must be annotated with @com.ibm.pdq.annotation.Id or unexpected results will occur.
  5. By default, column names map identical property names. @com.ibm.pdq.annotation.Column may be used to map a column to a property when they have different names.
  6. If your domain class has a constructor that takes arguments, you must provide a default, no argument constructor.
  7. And, most importantly, the property representing a nested child class must be annotated with @zero.data.handler.Join

One-to-one mapping

Below is an example of a domain class:

import com.ibm.pdq.annotation.Id;
public class Address {
    @Id private int address_id;
    private String name;
    public Address() {}
    // standard getters/setters removed for brevity
}

import com.ibm.pdq.annotation.Id;
import com.ibm.pdq.annotation.Column;
import zero.data.handler.Join;
public class Person {
    @Id private int person_id;
    @Column(name="firstname") private String name;
    @Join private Address address;
    public Person() {}
    // standard getters/setters removed for brevity
}

In this example, Person.person_id maps to the column person_id and is used in the mapping algorithm. Person.name maps to the column firstname because of the @Column annotation. Person.address is a reference to a prepopulated instance of Address. This is an example of a one-to-one mapping.

One-to-many mapping

Taking the above use case between Person and Address and modeling Person to own many Addresses. Below is an example of a domain class:

import com.ibm.pdq.annotation.Id;
public class Address {
    @Id private int address_id;
    private String name;
    public Address() {}
    // standard getters/setters removed for brevity
}

import com.ibm.pdq.annotation.Id;
import com.ibm.pdq.annotation.Column;
import zero.data.handler.Join;
public class Person {
    @Id private int person_id;
    @Column(name="firstname") private String name;
    @Join private List<Address> addresses;
}

Paging query results

They use paging to show the result little by little, usually in multiples of 5. Paging means showing your query result in multiple pages instead of just put them all in one long page. By splitting the query result in multiple pages you can save processing and download time plus you don't have much scrolling to do. Some database vendors support paging natively in their SQL implementation. Some are simple, others a bit verbose. Check your vendor's documentation for more details. Following are examples from more popular databases.

Using IteratorPagingResultHandler

As indicated above, SQL result paging is not consistently implemented for each database. zero.data provides a way to retrieve pages of data with IteratorPagingResultHandler:

data.beginTransaction()
def result = data.query("SELECT * FROM pager", new IteratorPagingResultHandler(new MapRowHandler(), 51, 75))
// work with result which is Iterator<Map<String,Object>> representing rows 51 through 75
data.endTransaction()

Since IteratorPagingResultHandler returns an Iterator, we need to operator and the results of the query before the connection is closed. Thus, we start with calling beginTransaction(), execute the query, work with the result, and then call endTransaction(), which ensures that the connection is closed. This pattern or use of inTransaction is required. IteratorPagingResultHandler returns an Iterator of whatever type the passed in RowHandler implementation returns. In the example above, MapRowHandler returns a Map for each row, as its name implies. To return bean, BeanRowHandler could be used. IteratorPagingResultHandler has two constructor signatures:

IteratorPagingResultHandler(RowHandler handler, int start, int end)
IteratorPagingResultHandler(int pageNumber, int pageSize, RowHandler handler)

The two signatures offer two patterns of dealing with paging. The first of the signatures allows for explicit start and end values. The example above makes use of this. The second signature takes the page number and page size and calculates the actual start and end rows for you. The following example would return the same results, rows 51 to 75, as the example above:

data.beginTransaction()
def result = data.query("SELECT * FROM pager", new IteratorPagingResultHandler(3, 25, new MapRowHandler()))
// work with result which is Iterator<Map<String,Object>> representing rows 51 through 75
data.endTransaction()

Version 1.1.0.0.21442