1 % Testing OpenJDK
2
3 ## Using the run-test framework
4
5 This new way of running tests is developer-centric. It assumes that you have
6 built a jdk locally and want to test it. Running common test targets is simple,
7 and more complex ad-hoc combination of tests is possible. The user interface is
8 forgiving, and clearly report errors it cannot resolve.
9
10 The main target "run-test" uses the jdk-image as the tested product. There is
11 also an alternate target "exploded-run-test" that uses the exploded image
12 instead. Not all tests will run successfully on the exploded image, but using
13 this target can greatly improve rebuild times for certain workflows.
14
15 Some example command-lines:
16
17 $ make run-test-tier1
18 $ make run-test-jdk_lang JTREG="JOBS=8"
19 $ make run-test TEST=jdk_lang
20 $ make run-test-only TEST="gtest:LogTagSet gtest:LogTagSetDescriptions" GTEST="REPEAT=-1"
21 $ make run-test TEST="hotspot:hotspot_gc" JTREG="JOBS=1;TIMEOUT=8;VM_OPTIONS=-XshowSettings -Xlog:gc+ref=debug"
22 $ make run-test TEST="jtreg:test/hotspot:hotspot_gc test/hotspot/jtreg/native_sanity/JniVersion.java"
23 $ make exploded-run-test TEST=tier2
24
25 ### Configuration
26
44 TEST="tier1"`, but the latter is more tab-completion friendly. For more complex
45 test runs, the `run-test TEST="x"` solution needs to be used.
46
47 The test specifications given in `TEST` is parsed into fully qualified test
48 descriptors, which clearly and unambigously show which tests will be run. As an
49 example, `:tier1` will expand to `jtreg:$(TOPDIR)/test/hotspot/jtreg:tier1
50 jtreg:$(TOPDIR)/test/jdk:tier1 jtreg:$(TOPDIR)/test/langtools:tier1
51 jtreg:$(TOPDIR)/test/nashorn:tier1 jtreg:$(TOPDIR)/test/jaxp:tier1`. You can
52 always submit a list of fully qualified test descriptors in the `TEST` variable
53 if you want to shortcut the parser.
54
55 ### JTReg
56
57 JTReg tests can be selected either by picking a JTReg test group, or a selection
58 of files or directories containing JTReg tests.
59
60 JTReg test groups can be specified either without a test root, e.g. `:tier1`
61 (or `tier1`, the initial colon is optional), or with, e.g. `hotspot:tier1`,
62 `test/jdk:jdk_util` or `$(TOPDIR)/test/hotspot/jtreg:hotspot_all`. The test
63 root can be specified either as an absolute path, or a path relative to the
64 OpenJDK top directory, or the `test` directory. For simplicity, the hotspot
65 JTReg test root, which really is `hotspot/jtreg` can be abbreviated as
66 just `hotspot`.
67
68 When specified without a test root, all matching groups from all test roots
69 will be added. Otherwise, only the group from the specified test root will be
70 added.
71
72 Individual JTReg tests or directories containing JTReg tests can also be
73 specified, like `test/hotspot/jtreg/native_sanity/JniVersion.java` or
74 `hotspot/jtreg/native_sanity`. Just like for test root selection, you can
75 either specify an absolute path (which can even point to JTReg tests outside
76 the source tree), or a path relative to either the OpenJDK top directory or the
77 `test` directory. `hotspot` can be used as an alias for `hotspot/jtreg` here as
78 well.
79
80 As long as the test groups or test paths can be uniquely resolved, you do not
81 need to enter the `jtreg:` prefix. If this is not possible, or if you want to
82 use a fully qualified test descriptor, add `jtreg:`, e.g.
83 `jtreg:test/hotspot/jtreg/native_sanity`.
84
85 ### Gtest
86
87 Since the Hotspot Gtest suite is so quick, the default is to run all tests.
88 This is specified by just `gtest`, or as a fully qualified test descriptor
89 `gtest:all`.
90
91 If you want, you can single out an individual test or a group of tests, for
92 instance `gtest:LogDecorations` or `gtest:LogDecorations.level_test_vm`. This
93 can be particularly useful if you want to run a shaky test repeatedly.
94
95 For Gtest, there is a separate test suite for each JVM variant. The JVM variant
96 is defined by adding `/<variant>` to the test descriptor, e.g.
|
1 % Testing the JDK
2
3 ## Using the run-test framework
4
5 This new way of running tests is developer-centric. It assumes that you have
6 built a JDK locally and want to test it. Running common test targets is simple,
7 and more complex ad-hoc combination of tests is possible. The user interface is
8 forgiving, and clearly report errors it cannot resolve.
9
10 The main target "run-test" uses the jdk-image as the tested product. There is
11 also an alternate target "exploded-run-test" that uses the exploded image
12 instead. Not all tests will run successfully on the exploded image, but using
13 this target can greatly improve rebuild times for certain workflows.
14
15 Some example command-lines:
16
17 $ make run-test-tier1
18 $ make run-test-jdk_lang JTREG="JOBS=8"
19 $ make run-test TEST=jdk_lang
20 $ make run-test-only TEST="gtest:LogTagSet gtest:LogTagSetDescriptions" GTEST="REPEAT=-1"
21 $ make run-test TEST="hotspot:hotspot_gc" JTREG="JOBS=1;TIMEOUT=8;VM_OPTIONS=-XshowSettings -Xlog:gc+ref=debug"
22 $ make run-test TEST="jtreg:test/hotspot:hotspot_gc test/hotspot/jtreg/native_sanity/JniVersion.java"
23 $ make exploded-run-test TEST=tier2
24
25 ### Configuration
26
44 TEST="tier1"`, but the latter is more tab-completion friendly. For more complex
45 test runs, the `run-test TEST="x"` solution needs to be used.
46
47 The test specifications given in `TEST` is parsed into fully qualified test
48 descriptors, which clearly and unambigously show which tests will be run. As an
49 example, `:tier1` will expand to `jtreg:$(TOPDIR)/test/hotspot/jtreg:tier1
50 jtreg:$(TOPDIR)/test/jdk:tier1 jtreg:$(TOPDIR)/test/langtools:tier1
51 jtreg:$(TOPDIR)/test/nashorn:tier1 jtreg:$(TOPDIR)/test/jaxp:tier1`. You can
52 always submit a list of fully qualified test descriptors in the `TEST` variable
53 if you want to shortcut the parser.
54
55 ### JTReg
56
57 JTReg tests can be selected either by picking a JTReg test group, or a selection
58 of files or directories containing JTReg tests.
59
60 JTReg test groups can be specified either without a test root, e.g. `:tier1`
61 (or `tier1`, the initial colon is optional), or with, e.g. `hotspot:tier1`,
62 `test/jdk:jdk_util` or `$(TOPDIR)/test/hotspot/jtreg:hotspot_all`. The test
63 root can be specified either as an absolute path, or a path relative to the
64 JDK top directory, or the `test` directory. For simplicity, the hotspot
65 JTReg test root, which really is `hotspot/jtreg` can be abbreviated as
66 just `hotspot`.
67
68 When specified without a test root, all matching groups from all test roots
69 will be added. Otherwise, only the group from the specified test root will be
70 added.
71
72 Individual JTReg tests or directories containing JTReg tests can also be
73 specified, like `test/hotspot/jtreg/native_sanity/JniVersion.java` or
74 `hotspot/jtreg/native_sanity`. Just like for test root selection, you can
75 either specify an absolute path (which can even point to JTReg tests outside
76 the source tree), or a path relative to either the JDK top directory or the
77 `test` directory. `hotspot` can be used as an alias for `hotspot/jtreg` here as
78 well.
79
80 As long as the test groups or test paths can be uniquely resolved, you do not
81 need to enter the `jtreg:` prefix. If this is not possible, or if you want to
82 use a fully qualified test descriptor, add `jtreg:`, e.g.
83 `jtreg:test/hotspot/jtreg/native_sanity`.
84
85 ### Gtest
86
87 Since the Hotspot Gtest suite is so quick, the default is to run all tests.
88 This is specified by just `gtest`, or as a fully qualified test descriptor
89 `gtest:all`.
90
91 If you want, you can single out an individual test or a group of tests, for
92 instance `gtest:LogDecorations` or `gtest:LogDecorations.level_test_vm`. This
93 can be particularly useful if you want to run a shaky test repeatedly.
94
95 For Gtest, there is a separate test suite for each JVM variant. The JVM variant
96 is defined by adding `/<variant>` to the test descriptor, e.g.
|