Package jdk.incubator.sql2

An API for accessing and processing data stored in a data source (usually a relational database) using the Java™ programming language. This API includes a framework whereby different drivers can be installed dynamically to access different data sources. This API is specifically geared for passing SQL statements to a database though it may be used for reading and writing data from/to any data source that has a tabular format.

This API differs from the API in java.sql in several ways.

  • Is asynchronous
  • Is geared toward high-throughput programs
  • Does not attempt to support every database feature
  • Does not attempt to abstract the database
  • Uses the builder pattern
  • Supports the fluent programming style

It is worth emphasizing that this API is an alternate to the java.sql API, not a replacement. There are many programs that can much more readily be written using the java.sql API as it has many features that are not available in this API. For example this API provides almost no mechanism for getting metadata.

This API is not an extension to the java.sql API. It an independent API and is used on its own without reference to java.sql.

Overview

The core feature of this API is that it is asynchronous. No method call will wait for a network operation.

Possibly blocking actions are represented as Operations. An application using the API creates and submits one or more Operations. The implementation executes these Operations asynchronously, reporting their results via CompletionStages. An application can respond to the results via the CompletionStages or via callbacks that can be configured on many of the Operations or both. Creating and submitting Operations is strictly non-blocking. Handling the results of possibly blocking Operations is done asynchronously. No application thread will ever block on a call to a method in this API.

All Operations provide a CompletionStage. The value of that CompletionStage is the value of the Operation, set when the Operation completes. Some Operations provide callbacks for processing the result of the Operation independent of the CompletionStage. Those Operations can be used for executing SQL that returns results of a specific type. For example SQL that returns a row sequence would be executed with a RowOperation. A RowOperation provides callbacks for processing each row and for collecting the results of processing the rows. Other Operations are specialized for SQL that returns a count or that returns out parameters. The choice of Operation is dependent on the result to be processed and is independent of the particular kind of SQL statement.

An OperationGroup encapsulates a group of Operations and executes them using common attributes. An OperationGroup can be unconditional or conditional, sequential or parallel, dependent or independent, or any combination of these. Dependent/independent controls error handling. If one member of a dependent OperationGroup fails the remaining not-yet-executed members are completed exceptionally. If the OperationGroup is independent, the member Operations are executed regardless of whether one or more fails.

A single context in a data source is represented by a Session. A Session is somewhat analogous to a logical java.sql.Connection. A physical java.sql.Connection has no representation in this API; if such an entity exists at all it is strictly internal to an implementation. Within this spec this entity is referred to as a "data source resource".

A Session is itself an OperationGroup and so can be conditional, parallel, or independent, but by default is unconditional, sequential, dependent. While a Session may be created with values other than the defaults, using the defaults is by far the most common case. The API provides convenience methods that support this case. Using these convenience methods is recommended in all but the most unusual circumstances. In particular making the Session parallel introduces some challenges that would require a full understanding of the details of the API. It would almost certainly be better to create a parallel OperationGroup within the Session.

ISSUE: Should we disallow Session.parallel()?

The java.sql API frequently provides many ways to do the same thing. This API makes no attempt to do this. For those capabilities this API supports, it frequently defines exactly one way to do something. Doing things another way, for example calling methods in a non-standard order, frequently results in an IllegalStateException. This approach is intended to make things simpler for both the user and the implementor. Rather than having to understand complicated interactions of many different components and methods executed in any order, the intent is that there is only one way to do things so only one path must be understood or implemented. Anything off that path is an error. While this requires a programmer to write code in one specific way it makes things easier on future maintainers of the code as the code will conform to the standard pattern. Similarly the implementation is simplified as only the standard use pattern is supported.

One way this API simplifies things in to define types as single use. Many types are created, configured, used once, and are then no longer usable. Most configuration methods can be called only once on a given instance. Once an instance is configured it cannot be reconfigured. Once an instance is used it cannot be reused. This simplifies things by eliminating the need to understand and implement arbitrary sequences of method calls that reconfigure and reuse instances. Since objects are single use there is no expectation that an application cache or share Operations.

While the user visible types are single use, it is expected that an implementation will cache and reuse data and Objects that are worth the effort. Rather than attempt to guess what an implementation should reuse and capture that in the API, this API leaves it entirely up to the implementation. Since the API specifies very little reuse, an implementation is free to reuse whatever is appropriate. Since the pattern of use is strictly enforced figuring out how to reuse objects is greatly simplified.

The java.sql API provides many tools for abstracting the database, for enabling the user to write database independent code. This API does not. It is not a goal of this API to enable users to write database independent code. That is not to say it is not possible, just that this API does not provide tools to support such. Abstraction features typically impose performance penalties on some implementations. As this API is geared for high-throughput programs it avoids such abstractions rather than reduce performance.

One such abstraction feature is the JDBC escape sequences. Implementing these features requires parsing the SQL so as to identify the escape sequences and then generating a new String with the vendor specific SQL corresponding to the escape sequence. This is an expensive operation. Further each SQL must be parsed whether it contains an escape sequence or not imposing the cost on all JDBC users, not just the ones who use escape sequences. The same is true of JDBC parameter markers. The SQL accepted by this API is entirely vendor specific, including parameter markers. There is no need for pre-processing prior to SQL execution substantially reducing the amount of work the implementation must do.

Note: It would be a reasonable future project to develop a SQL builder API that creates vendor specific SQL from some more abstract representation.

This API is targeted at high-throughput apps. If a particular feature of this API would have a surprising performance impact for a particular implementation it is recommended that the implementation not implement that feature. It is better that a feature be unsupported as opposed to users investing substantial effort in an app using that feature only to discover in production that the performance is unacceptable. For example, if an implementation can only support Operation.timeout(java.time.Duration) through active polling it would be better for that implementation to throw UnsupportedOperationException if Operation.timeout(java.time.Duration) is called. To this end every type and method is optional except returning a DataSourceFactory in response to a call to DataSourceFactory.newFactory(java.lang.String) with the appropriate name.

Execution Model

This section describes the function of a conforming implementation. It is not necessary for an implementation to be implemented as described only that the behavior be the same.

An Operation has an action and a CompletionStage. Some Operations have some form of result processor.

An Operation is executed by causing the action to be performed, processing the result of the action if there is a result processor, and completing the CompletionStage with the result of the result processor if there is one. If the action or the result processing causes an unhandled error the CompletionStage is completed exceptionally. The CompletionStage is completed asynchronously, as though it were created by calling an async method on CompletionStage.

Performing the action may require one or more interactions with the database. These interactions may be carried out in parallel with processing the result. If the database result is ordered, that result is processed in the order specified by the database.

An OperationGroup has a collection of member Operations and optionally a condition. For a sequential OperationGroup Operations are selected from the collection in the order they were submitted. For a parallel OperationGroup Operations are selected from the collection in any order.

The action of an OperationGroup is performed as follows:

A Session is a distinguished OperationGroup. A Session is executed upon being submitted.

Transactions

This section describes the function of a conforming implementation. It is not necessary for an implementation to be implemented as described only that the behavior be the same.

An implementation has only limited control over transactions. SQL statements can start, commit, and rollback transactions without the implementation having any influence or even being aware. This specification only describes the behavior of those transaction actions that are visible to and controlled by the implementation, i.e. the endTransaction Operation. Transaction actions caused by SQL may interact with actions controlled by the implementation in unexpected ways.

The creation of Operations and the subsequent execution of those Operations are separated in time. It is quite reasonable to determine that a transaction should commit after the Operation that ends the transaction is submitted. But if the execution of the transaction does not result in the expected results it might be necessary to rollback the transaction rather than commit it. This determination depends on the execution of the Operations long after the endTransaction Operation is submitted. To address this mismatch, the endTransaction Operation is conditioned by a TransactionCompletion. By default, a TransactionCompletion will cause an endTransaciton Operation to commit the transaction. At any time before the endTransaction Operation that references it is executed a TransactionCompletion can be set to rollback the transaction .

An endTransaction Operation, like all Operations, is immutable once submitted. But an endTransaction Operation is created with a TransactionCompletion and that TransactionCompletion can be set to commit or rollback. A TransactionCompletion controls the endTransaction Operation created with it. Using this mechanism an error handler, result handler or other code can cause a subsequent endTransaction Operation to rollback instead of the default which is to commit.

 
   TransactionCompletion t = session.getTransactionEnd();
   session.countOperation(updateSql)
       .resultProcessor( count -> { 
           if (count > 1) t.setRollbackOnly(); 
           return null; 
       } )
       .submit();
   session.catchErrors();
   session.commitMaybeRollback(t);
 
 

In this example if the update SQL modifies more than one row the result processor will set the TransactionCompletion to rollback only. When the endTransaction Operation submitted by OperationGroup.commitMaybeRollback(jdk.incubator.sql2.TransactionCompletion) is executed it will cause the transaction to rollback.

Implementation Note

If an implementation exposes any implementation specific types and methods, the implementation is expected to provide covariant overrides for all methods that return the standard super-type of the implementation specific type.

Consider an implementation that adds a method foo() to RowCountOperation. To do that it would have to expose a type FooRowCountOperation extends RowCountOperation. So that an application can transparently access foo, the implementation would also have to expose FooDataSource, FooOperationGroup and FooSession. Further each of these types would have to declare covariant overrides for every method that returns a direct super-type of one of these types.

  • FooDataSourceFactory must override builder to return FooDataSource.Builder
  • FooDataSource.Builder must override url, password, etc to return a FooDataSource.Builder. build must return a FooDataSource.
  • FooDataSource must override builder to return FooSession.Builder
  • FooSession.Builder must override url, password, etc to return a FooSession.Builder. build must return a FooSession
  • FooDataSource must override getSession to return FooSession
  • FooSession must extend FooOperationGroup
  • FooOperationGroup must override rowCountOperation to return FooRowCountOperation
  • FooRowCountOperation must override apply and onError to return FooRowCountOperation

The intent is to transparently expose the vendor extension without use of casts. Example:

 
   FooDataSourceFactory factory = DataSourceFatory.newFactory("com.foo.FooDataSourceFatory");
   FooDataSource dataSource = factory.builder()
       .url("scott/tiger@host:port")
       .build();
   FooSession session = dataSource.getSession();
   CompletionStage<Long> count = session.rowOperation(sql)
       .set("param", value, AdbaType.VARCHAR)
       .foo()
       .submit()
       .getCompletionStage();
 
 

Notice that there are no casts, yet both standard methods an the vendor extension method foo can be referenced. This is possible only if the implementation exposes all the necessary types and provides covariant overrides for every method that returns one of those types. Implementations are expected (required?) to do this.

If an implementation does not expose any implementation specific methods or types, that implementation is not required to provide covariant overrides that return implementation specific types.