Comments and discussion: jigsaw dash dev at openjdk dot java dot net
| Contents | |
|---|---|
| 1 | Fundamentals |
| Module dependences · Resolution in all phases · Fidelity across all phases · Encapsulation · Versioning · Optional modules · Refactoring · Substitution · Module constraints on targets · Target constraints on modules · Native code · Services · Resources · Read-only operation · Platform modularization · Package subsets · Shared class loaders | |
| 2 | Developers |
| Tool chain · Module declarations alongside source code · Exploded modules · Diagnostic messages · Diagnostic tooling · Debug libraries | |
| 3 | Containers |
| Reification · Simultaneous multiple versions | |
| 4 | Migration and interoperation |
| Modular JAR files · JAR files depending upon modules · Modularize JAR files · Maven support · OSGi support · Extensible module declarations · Authoritative module declarations · Manipulable module metadata | |
| 5 | Packaging |
| Java-native module-file format · Signed module files · Java modules as native packages | |
| 6 | Publication |
| Web publication · Exploratory installation | |
| 7 | Performance |
| No performance regressions · Startup-time improvements · Enable future performance improvements | |
| 8 | Open requirements |
| “Friend” constraints · JSR 294 · Multi-platform installations · Predictability · Service declarations · Package-level dependences · Java-like syntax · Source-form metadata in modular JAR files · Compiled module declarations · Non-Java source-file names · Java source-file names | |
| 9 | Non-requirements |
| Fine-grained Java SE modularization | |
| A | Acknowledgements |
| B | Change history |
| C | Commentary on module-metadata formats |
The terms “must,” “must not,” “required,” “shall,” “shall not,” “should,” “should not,” “recommended,” “may,” and “optional” in this document are to be interpreted as described in RFC 2119.
Module dependences — The module system must replace the class path and its explicit enumeration of JAR files with a less error-prone system of named, versioned modules and explicit dependences between them. Where possible this approach should borrow from UNIX shared-library concepts. Most developers and deployers are quite comfortable with this model since it is used not only by shared libraries but also Maven, .NET, and most UNIX/Linux packaging systems.
Resolution in all phases — Build-time, install-time, and run-time module resolution and linking must all be supported. UNIX has all three models and they are all still in use 40 years later.
Fidelity across all phases — The set of modules seen by a library or application must be computed by the same algorithm at build time, install time, and run time.
Encapsulation — If a module contains a type, whether defined therein or imported from some other module, then it must be possible to declare that it makes that type available for use by modules depending upon it, or that it does not do so.
Versioning — The module system must support common version-string schemes such as those defined by OSGi and Maven and also those found in both proprietary and open-source software products. Version strings must be totally ordered but otherwise the module system must not impose any particular structure or semantics upon them. It must be possible to specify a range of allowable versions when declaring a module dependence.
Optional modules — It must be possible to declare that a module depends optionally upon some other module. If that second module is available then it is resolved and linked normally. If it is not available, however, then no exception is thrown and no error is reported by the module system. A simple API must be provided so that a running program can test whether a particular optional module has been resolved and linked.
Refactoring — It must be possible to refactor a module into a set of smaller modules, without breaking existing modules or applications that depend upon the original module (splitting). Likewise, it must be possible to refactor a set of modules so that they appear to be a single module, without breaking existing modules or applications that depend upon the original modules (aggregation).
Substitution — It must be possible to declare that one module can substitute for another. This supports refactoring via aggregation and also allows a module to resolve against any one of a set of equivalent implementation modules. JREs from different vendors, e.g., can all declare that they are substitutes for the “java” module. An application module that requires only standard Java SE APIs can declare that it requires the “java” module rather than some specific JRE module, and that dependence will be satisfied by any one of those JREs.
Module constraints on targets — In a module declaration it must be possible to declare that the module can only be used on target platforms with specific properties, e.g., particular operating systems, processor architectures, or display geometries. The resolution algorithm must ignore any module whose platform constraints are not satisfied by the target platform.
Target constraints on modules — On a particular target platform it must be possible to declare that only modules with specific properties can be installed, e.g., particular authors, publishers, or licenses. The resolution algorithm must ignore any module that does not satisfy the target platform’s module constraints.
Native code — The module system must provide a way to package both Java class files and native libraries in a single artifact and control linking in a similar way for both. The existing System.loadLibrary method and related methods must be enhanced to be module-aware. It must be possible for a native library in a Java module to have a native dependence upon another native library that is either in the same Java module, in some other Java module, or elsewhere in the target platform.
Services — It must be possible to package service providers into modules so that they can be discovered via an enhanced version of the existing java.util.ServiceLoader API.
Resources — It must be possible to package localization data into modules so that it can be discovered via an enhanced version of the existing java.util.ResourceBundle API.
Read-only operation — The module system must be able to function at run time when running from a read-only volume. This is a security requirement in some scenarios and may also be required for some compliance regimes.
Platform modularization — The module system must support a medium-grained modularization of the Java SE API whose minimal configuration defines an API roughly the size of the current Java ME CDC platform. It must provide a safety valve to prevent the JDK from continuing to grow without bound (e.g., a hypothetical java.3d API shouldn’t need to go into the base platform). It must be possible to provide subsets of the platform such as “headless” (no 2d/awt/swing/audio/sound) and “client” (no RMI, CORBA, etc.) commonly requested by developers of embedded SE and EE solutions. Finally, it must be possible to release and update logically-separate portions of the Java API separately.
Package subsets — In support of platform modularization, it must be possible for the types defined in a Java package to be provided by more than one module yet still be loaded by the same class loader at run time. This is required so that CDC-sized subsets of large legacy packages such as java.util can be defined.
Shared class loaders — In support of platform modularization, it must be possible to declare that the types defined in a specific set of modules must be loaded by the same class loader.
Developers are the most important consumers of the module system. Those familiar with Maven, with OSGi bundles, with .NET assemblies, or with UNIX shared libraries should be immediately comfortable and productive with Java modules. The compile-link-debug cycle should be fast and they should be able to diagnose problems quickly. Features that don’t encourage best practices should be omitted.
Tool chain — A command-line tool chain that supports the creation, installation, linking, and removal of modules must be provided. Its usability should compare favorably with that of the corresponding tools in .NET.
Module declarations alongside source code — It must be easy for developers to both write and read module declarations. The Java language syntax must be extended to define declarations of module metadata. The syntax and, to the degree possible, the semantics of module declarations must be specified in the Java Language Specification. A module declaration must be written in a file that resides with the module’s source content, rather than in a side file in some other format or, even worse, a set of command-line arguments baked into a build script.
Exploded modules — During development it must be possible to compile against and run modules that are “exploded” on disk rather than packaged and then installed. In other words, a development-time format analogous to the class directories currently supported by the java and javac commands must be supported.
Diagnostic messages — ClassCastException and related exceptions must be enhanced to provide as much information as possible about the lineage of the classes involved. Errors arising from run-time linkage problems must be clearly defined and thrown at well-specified times.
Diagnostic tooling — The module system must define a public API that allows running code to examine the class-loader and module structure, determine the lineage of a given class, understand the reason for resolution decisions, and so forth.
Debug libraries — There should be a well-understood convention for building and using debug libraries (e.g., those built with javac -g).
Containers such as IDEs, Java EE containers, and applet containers are the second-most important constituency for the module system. They have specific requirements for dynamism and multiple-version support. For Java EE the primary use case should be an assumed future modular WAR standard in which a WAR file contains one or more Java modules.
Reification — The module system must provide a public API for inspecting all of a module’s metadata. Using the API it must be possible to do the following from within a running Java program: Create a new module library, install modules into that library, optionally resolve and link modules in the library, get a class loader for one of these modules, and then reverse all of these operations.
Simultaneous multiple versions — A container must be able to load a group of modules that, as a whole, requires a different version of one or more modules already loaded by the container. A canonical example is that of a web application requiring a different version of the XML parser than that built-in to the web container. Multi-version support should only be enabled when type collisions can be detected and reported as errors. This most likely means that it cannot be enabled for run-time linking since detecting collisions is nontrivial.
In order to be adopted the module system must “play well” with existing file formats, tool chains, build systems, and module systems.
Modular JAR files — For pure, Java-only modules it must be possible to create a single file that can be used both as a JAR on the class path and installed as a module in a module library. This will allow library developers to continue to produce one artifact for both traditional and modular applications. It must be possible to sign such JARs in the usual way.
JAR files depending upon modules — It must be possible to modify a “plain old” JAR file to declare that it depends upon modules of specific names and possibly version ranges. This will support the incremental modularization of existing code bases from the “inside out.”
Modularize JAR files — A tool must be provided that will transform a “plain old” JAR file into a modular JAR file by adding specified metadata or, possibly, inferring it from the JAR file’s name and legacy manifest information.
Maven support — The module system must be able to download, install, resolve, and link a Maven artifact and its transitive dependences as if they were modules. Maven is a popular and widely used modular build and assembly system which most Java developers are familiar with today, so making the module system work really well with it will be important to adoption.
OSGi support — It must be demonstrated by prototype to be feasible to modify an OSGi micro-kernel such that OSGi bundles running in that kernel can depend upon Java modules. The kernel must be able to load Java modules directly and resolve them using its own resolver, except for core system modules. Core system modules can only be loaded using the module system’s reification API.
Extensible module declarations — To support tools and frameworks built on top of the module system the syntax of module-declaration source files must be extensible, allowing arbitrary structured content in addition to the standard declaration clauses. The syntax must place all extended metadata after all standard metadata, with a clear delineation between them.
Authoritative module declarations — A module declaration must completely and authoritatively define its module’s metadata. No additional metadata shall be required in order to produce or consume a module.
Manipulable module metadata — Module metadata must in all phases be easily read, edited, and written by module-management tools and by people, not just by Java compilers.
A primary use case for OSGi support is Eclipse. The adoption of Java modules will be hindered if they cannot easily be used in the context of Eclipse. An Eclipse plug-in for JPA, e.g., must be able to declare dependences upon standard JPA API modules as well as upon specific implementations.
In order to make Java more widely useful, i.e., beyond the usual Java EE and applet/JNLP scenarios, we should make it easy to package modules that contain both Java and native code. (This is also required for the modularization of the JDK itself.) We should also make it possible to package modules in the standard native-package formats of popular operating systems so that Java libraries and applications can be managed in the way most familiar to developers and end users.
Java-native module-file format — A format designed solely for packaging and transporting modules from one system to another must be defined. Unlike JAR files, this format will not also serve as an execution format. The format must support appropriate content-specific compression algorithms, e.g., Pack200 for class files and bzip2 or lzma for native libraries.
Signed module files — Like signed JAR files, module files must be capable of carrying one or more cryptographic signatures and associated certificate chains so that the integrity and authenticity of a module can be verified. The process of verifying signatures must be no more costly than for JAR files, and should be less so. Verification must, when possible, be done as part of the installation process rather than at run time.
Java modules as native packages — A tool for delivering Java modules as native RPM or Solaris IPS packages must be provided. Support for similarly-capable packaging systems on other platforms (e.g., Debian, AIX, and HP-UX) must be feasible.
Maven provides a way to publish Java code for web discovery and download at build time but it doesn’t work for deployers or end users when they need to install libraries and applications for use at run time. To make Java more widely useful it needs to catch up with Perl (which has CPAN), Python (PyPI), Ruby (Gems), and Scala (Bazaars).
Web publication — A convention must be defined for publishing module files on an HTTP server so that modules can be discovered, downloaded, and installed for use with an existing base JRE. The convention should require only static content. The command-line tool for manipulating collections of installed modules must implement this convention. This functionality is necessary for Windows and other operating systems that do not have sufficiently-capable native packaging systems; it is also useful for modules that are installed by end users rather than administrators.
Exploratory installation — It must be possible to determine, from a description of a yet-to-be installed module M, which other modules, if any, must be installed along with M so that all of its dependences will be satisfied, regardless of whether those additional modules are direct or indirect requirements of M.
No performance regressions — The initial release must not regress HelloWorld or any of the other usual JDK performance metrics.
Startup-time improvements — The module system must support the pre-verification of class files and an optimized path for loading a class file given a class name.
Enable future performance improvements — The long-term roadmap should include other optimizations such as class pre-loading and module pre-compilation. The roadmap should show potential for cutting HelloWorld startup time by a factor of 2.
These requirements are not yet settled, pending further investigation. Some of these open requirements conflict with each other, as noted.
“Friend” constraints — It must be possible for a module to specify the set of modules that can depend directly upon it, and to have that restriction be enforced at run time.
JSR 294 — A new Java language keyword must be defined for controlling module-level type and member accessibility.
Multi-platform installations — It must be possible to install multiple variants of a module, each targeted to a different combination of operating system, processor architecture, and possibly other platform properties.
Predictability — The set of modules seen by an application must not depend upon the application’s behavior, or that of any other application, unless the application is interacting directly with the module system to perform run-time resolution and linking.
Service declarations — A module must be able to declare that it requires and/or provides specific services.
Package-level dependences — It must be possible to declare that a module depends upon one or more Java packages, leaving it to the module system to resolve which modules will be linked to provide the types defined in those packages.
Java-like syntax — The syntax of module declarations must be similar to that of declarations of a like nature in the Java programming language.
The following two requirements conflict. Please see Appendix C for commentary.
Source-form metadata in modular JAR files — The module metadata in a modular JAR file must use the same Java syntax as module-declaration source files. This will ease deployment debugging and tool construction.
Compiled module declarations — Module declarations must be compiled by a Java compiler into a standard compact binary form, just as Java class declarations are compiled into class files.
The following two requirements also conflict.
Non-Java source-file names — When a module declaration is stored in a file then the name of the file must, by convention, end with a string other than “.java”.
Java source-file names — When a module declaration is stored in a file then the name of the file must, by convention, end with the string “.java”.
Fine-grained Java SE modularization — The precise requirements on Java API modularization are beyond the scope of this document, but it is worth stating explicitly here that it is not a requirement to be able to modularize the API to the point that it would fit into a traditional CLDC-class device.
This document includes contributions from Alex Buckley, Jonathan Gibbons, Karen Kinnear, and Adam Messinger. It has been improved with the help of comments and suggestions from Roberto Chinnici, Jerome Dochez, Bill Shannon, David Bosschaert, BJ Hargrave, Doug Lea, and Jeff McAffer.
Version 12
Version 11, 2011/4/19
Version 10, 2011/4/5
Version 9, 2011/3/29
Version 8, 2011/2/16
Among those who have reviewed this document there are divergent opinions as to whether module declarations must, or must not, be compiled by a Java compiler into a standard binary form. The arguments of each side are captured here for the record.
If module declarations are not compiled then they must remain in source form across all phases, i.e., the module metadata in a packaged module would be a textual copy of the original module declaration source code.
Arguments for requiring that module declarations not be compiled:
Arguments for requiring that module declarations be compiled: