# HG changeset patch # User redestad # Date 1542366017 -3600 # Fri Nov 16 12:00:17 2018 +0100 # Node ID 471f8749e0ca1c20f4c2021d3e40a43e5cdb4483 # Parent 2ed42e41e712570d8e917c1ff61bc4f4538818f1 8061281: Microbenchmark suite build support, directory layout and sample benchmarks 8061282: Migrate jmh-jdk-microbenchmarks into the JDK Reviewed-by: ecaspole, mchung, erikj, ihse Contributed-by: magnus.ihse.bursie@oracle.com, erik.joelsson@oracle.com, claes.redestad@oracle.com, sfriberg@kth.se diff --git a/doc/testing.html b/doc/testing.html --- a/doc/testing.html +++ b/doc/testing.html @@ -24,12 +24,14 @@
To be able to run JTReg tests, configure
needs to know where to find the JTReg test framework. If it is not picked up automatically by configure, use the --with-jtreg=<path to jtreg home>
option to point to the JTReg framework. Note that this option should point to the JTReg home, i.e. the top directory, containing lib/jtreg.jar
etc. (An alternative is to set the JT_HOME
environment variable to point to the JTReg home before running configure
.)
To be able to run microbenchmarks, configure
needs to know where to find the JMH dependency. Use --with-jmh=<path to JMH jars>
to point to a directory containing the core JMH and transitive dependencies. The recommended dependencies can be retrieved by running sh make/devkit/createJMHBundle.sh
, after which --with-jmh=build/jmh/jars
should work.
All functionality is available using the test
make target. In this use case, the test or tests to be executed is controlled using the TEST
variable. To speed up subsequent test runs with no source code changes, test-only
can be used instead, which do not depend on the source and test image build.
For some common top-level tests, direct make targets have been generated. This includes all JTReg test groups, the hotspot gtest, and custom tests (if present). This means that make test-tier1
is equivalent to make test TEST="tier1"
, but the latter is more tab-completion friendly. For more complex test runs, the test TEST="x"
solution needs to be used.
Since the Hotspot Gtest suite is so quick, the default is to run all tests. This is specified by just gtest
, or as a fully qualified test descriptor gtest:all
.
If you want, you can single out an individual test or a group of tests, for instance gtest:LogDecorations
or gtest:LogDecorations.level_test_vm
. This can be particularly useful if you want to run a shaky test repeatedly.
For Gtest, there is a separate test suite for each JVM variant. The JVM variant is defined by adding /<variant>
to the test descriptor, e.g. gtest:Log/client
. If you specify no variant, gtest will run once for each JVM variant present (e.g. server, client). So if you only have the server JVM present, then gtest:all
will be equivalent to gtest:all/server
.
Which microbenchmarks to run is selected using a regular expression following the micro:
test descriptor, e.g., micro:java.lang.reflect
. This delegates the test selection to JMH, meaning package name, class name and even benchmark method names can be used to select tests.
Using special characters like |
in the regular expression is possible, but needs to be escaped multiple times: micro:ArrayCopy\\\\\|reflect
.
A handful of odd tests that are not covered by any other testing framework are accessible using the special:
test descriptor. Currently, this includes failure-handler
and make
.
Additional options to the Gtest test framework.
Use GTEST="OPTIONS=--help"
to see all available Gtest options.
Override the number of benchmark forks to spawn. Same as specifying -f <num>
.
Number of measurement iterations per fork. Same as specifying -i <num>
.
Amount of time to spend in each measurement iteration, in seconds. Same as specifying -r <num>
Number of warmup iterations to run before the measurement phase in each fork. Same as specifying -wi <num>
.
Amount of time to spend in each warmup iteration. Same as specifying -w <num>
.
Specify to have the test run save a log of the values. Accepts the same values as -rff
, i.e., text
, csv
, scsv
, json
, or latex
.
Additional VM arguments to provide to forked off VMs. Same as -jvmArgs <args>
Additional arguments to send to JMH.