Clone the project forest.
hg clone http://hg.openjdk.java.net/build-infra/jdk9 build-infra-jdk9
cd build-infra-jdk9 && bash get_source.sh [additional closed url]
The build infra project is constantly on the move. The safest way to get to a working state is by using a tag, e.g. build-infra-beta-01.
bash common/bin/hgforest.sh update -r build-infra-beta-01
Build it. This works just as the with old build.
bash configure && make
The build process is by default much more quiet. If you want to get some
progress information, similar to the old build, use LOG=info
, e.g.:
make hotspot LOG=info
If you want to see the command lines of "important" commands (like
compilations), add "cmdlines" to the LOG arguments, like LOG=info,cmdlines
or
just LOG=cmdlines
.
If you have questions or want to report bugs or enhancement requests, please
direct them to the build-infra project list build-infra-dev@openjdk.java.net
.
Note that this is different from the Build Group mailing list
build-dev@openjdk.java.net
.
The output directory is differently structured than in the old build, to better align with the existing practices in the JDK build. Note that this might change even more further on, with a future tighter integration between hotspot and the rest of the build.
build/$BUILD/hotspot/dist <-- identical to the old dist, the "deliverable" of the hotspot build
build/$BUILD/hotspot/variant-X, Y, ... <-- one directory for every variant, e.g. server, client...
Let's pick variant-server as an example. This is further structured in:
build/$BUILD/hotspot/variant-server/gensrc <-- similar to the old 'generated'
build/$BUILD/hotspot/variant-server/tools <-- build tools (adlc, jvmti generators)
build/$BUILD/hotspot/variant-server/support <-- intermediate files needed during build
build/$BUILD/hotspot/variant-server/libjsig <-- the libjsig.so library (on unix platforms)
build/$BUILD/hotspot/variant-server/libjvm <-- where the action is :-)
(The names are intentionally chosen to be "tab expansion friendly" as far as reasonably possible.)
Looking into the libjvm
directory, finally, gives us:
libjvm.so/jvm.dll <-- obviously :)
libjvm.diz/jvm.diz <-- debug symbols
hotspot <-- the hotspot launcher script
mapfile <-- the mapfile used by the linker
symbols <-- the symbols used to create the mapfile
symbols-object <-- all symbols extracted from the object files
objs/ <-- all *.o files and build support files
Of particular interest in the objs
directory are the foo.o.cmdline
and
foo.o.log
files, which contain the command line that was used to compile
foo.o
, and the output from the compiler during that compilation,
respectively.
In the new hotspot build, the mapfile is generated at build time. This was
sort-of the case previously as well, but in a more roundabout way. The new
build has "symbol lists" as the core entity. These are simple text files with a
symbol name on each line, for every symbol that should be exported from the JVM
shared library. For a typical example, look at
hotspot/makefiles/symbols/symbols-shared
.
Symbol files exists for all platforms (-shared
) or specific build conditions,
such as the target OS. All symbols file that apply for the current build are
concatenated, together with data extracted from the compiled object files, into
the libjvm/symbols
file in the output directory.
Finally, this file is transformed into a format that suits the linker. (The
traditional mapfile for linux/solaris, other formats for macosx and windows.)
This transformed file has the name libjvm/mapfile
.
Notably, the symbols-unix
file contains multiple symbols that seems to really
belong in symbols-shared
. On windows, exporting from the linker complements
and intersects with exports done in the C source files using
__declspec(dllexport)
. The current solution mimics the old build, but it
certainly has potential for improvement.
The new build provides developers with a much higher degree of freedom in selecting what gets built, and how. Not all combinations make sense, though. The guiding principle here has been that the build system should not put a spoke in the wheel if someone wants to build a special combination. Not all setups are regularly tested or guaranteed to work, but fixing them is not helped by having the build system block any build attempts.
Some examples:
Building with dtrace on linux. The default behavior is the same, but with
--enable-dtrace
, you can build dtrace regardless of conditions. (Also,
--disable-dtrace
can be used to disable dtrace regardless, e.g. on
solaris).
Building all variants on all platforms. The build system will not prohibit you to build 'core' on linux-x64 or 'zero' on windows-x86. Not all combinations will compile successfully, nor less make much sense, but they're open for fixing. ;-)
Building an arbitrary selected set of features. (See below on JVM features).
The hotspot build differs from all other libraries in the JDK in that the library is (potentially) built multiple times with different conditions (-D flags, for instance). The most common combination is building both the 'client' and 'server' variant, but other combinations are possible. While this state of affairs is not universally appreciated :-), it still is a use case that we need to support, and it affects the entire build system for hotspot.
The new build compiles each variant completely independent. It means that a few files are generated multiple times (like jvmti.html) but it also creates a distinct operation to build an entire independent variant. All hotspot output is stored in a per-variant directory (see 'Examining the output' above). After all variants are built, the relevant parts are copied to the hotspot/dist directory.
The new build supports the following variants:
Not all variants makes sense on all platforms. Some of the more unusual
combinations does not even compile. (The build-infra project forest contains a
number of patches (mostly trivial, like #ifdef COMPILER2
) to make most
variants compile on most platforms. These are not planned for integration
though, unless requested by the hotspot developers.)
Which variants to build is, just as before, specified by the configure flag
--with-jvm-variants
, e.g. --with-jvm-variants=server,client,minimal
('minimal1' is kept as an alias for 'minimal').
In essence, a variant is a name, a set of JVM features (see below) and a specified output directory. For server, client and minimal, the output directory is the same as the variant name. The remaining variants all use 'server' as the output directory name. This is also the reason that only one of these variants can be built at the same time.
The zero and zeroshark variants behave internally almost (but not fully) like a separate target CPU, and are quite different in that respect. Since they rewrite global CPU variables, they can never be combined with any other variant.
A JVM variant is described by a set of "JVM features". This is a list of tags that uniquely describe how to compile the given JVM. Some features are determined by the JVM variant (e.g. 'compiler1' for client or 'zero' for zero). Other features are determined by platform (e.g. 'dtrace'), or by being enabled by the user in configure (e.g. 'static-build').
The build system translates a list of JVM features into the proper actions required to build a JVM with these features. In the most typical case, this translates into a set of extra files to include, and -D define to pass to the compiler.
The calculated features for each JVM variant to build is printed both at configure time and at build time.
Additional features can be set (for all variants being built) by the flag
--with-jvm-features
. For instance: --with-jvm-features=jvmci,compiler2
--with-jvm-variants=minimal
would build a "minimal" JVM but with JVMCI and C2
enabled as well (that is, "tiered").
Not all features can be set using --with-jvm-features
. Some require
additional checks by the configure script, such as 'dtrace', which require
suitable tools and header files. Such features need to be enabled by other
means (like --enable-dtrace
). The configure script aborts with an error
message if the user is trying to enable the feature without using the correct
flag. (It is technically possible to re-interpret --with-jvm-features=dtrace
as --enable-dtrace
. If there is demand for such functionality, it can be
added to the configure script.)
For experimental purposes, it is also possible to build a 'custom' JVM variant.
This is a baseline variant with an empty initial set of features. You can use
it if you want to try unorthodox feature combinations, e.g.
--with-jvm-features=jvmci --with-jvm-variants=custom
would build something
akin to core but with JVMCI enabled (and nothing else, like minimal). Bear in
mind that you can specify combinations that will not compile, or that will
compile but not work properly. The only thing that the build system enforces
are dependencies between features, e.g. that 'jvmti' depends on 'services'.
Integration will take place in two major, separate steps. First, the new build
system will be integrated, alongside with the old build system. At this point,
the new build system will be used by default, but the build can easily be
switched to use the old build system with --disable-new-hotspot-build
.
This lowers the risk, since any potential build problem that arises can easily be circumvented by reverting to the old build system. It also allows developers that have local modifications to the old build system to migrate them to the new build system in an orderly fashion.
After a reasonable time for testing and migration (initial estimate is about a month), the old build system will be removed. As long as we need to keep the old build system, there is enhancements (such as improved integration with the rest of the build system) and cleanups that cannot be performed. Removing the old build system frees us of these constraints, and the new hotspot build project will not be fully completed until this point.
In the initial integration, the new build system will reside in a
hotspot/makefiles
directory, leaving the old build system unchanged in
hotspot/make
. After the removal of the old build system, the new build system
will move to hotspot/make
. This process is identical to the integration of
the new build system for JDK some years ago.
This document is written by Magnus Ihse Bursie (magnus dot ihse dot bursie at oracle dot com).
It was last updated on 2016-02-12.