< prev index next >

doc/testing.html

Print this page
rev 50850 : 8205956: Fix usage of "OpenJDK" in build and test instructions
   1 <!DOCTYPE html>
   2 <html>
   3 <head>
   4   <meta charset="utf-8">
   5   <meta name="generator" content="pandoc">
   6   <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
   7   <title>Testing OpenJDK</title>
   8   <style type="text/css">code{white-space: pre;}</style>





   9   <link rel="stylesheet" href="../make/data/docs-resources/resources/jdk-default.css">
  10   <!--[if lt IE 9]>
  11     <script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv-printshiv.min.js"></script>
  12   <![endif]-->
  13   <style type="text/css">pre, code, tt { color: #1d6ae5; }</style>
  14 </head>
  15 <body>
  16 <header>
  17 <h1 class="title">Testing OpenJDK</h1>
  18 </header>
  19 <nav id="TOC">
  20 <ul>
  21 <li><a href="#using-the-run-test-framework">Using the run-test framework</a><ul>
  22 <li><a href="#configuration">Configuration</a></li>
  23 </ul></li>
  24 <li><a href="#test-selection">Test selection</a><ul>
  25 <li><a href="#jtreg">JTReg</a></li>
  26 <li><a href="#gtest">Gtest</a></li>
  27 </ul></li>
  28 <li><a href="#test-results-and-summary">Test results and summary</a></li>
  29 <li><a href="#test-suite-control">Test suite control</a><ul>
  30 <li><a href="#jtreg-keywords">JTReg keywords</a></li>
  31 <li><a href="#gtest-keywords">Gtest keywords</a></li>
  32 </ul></li>
  33 </ul>
  34 </nav>
  35 <h2 id="using-the-run-test-framework">Using the run-test framework</h2>
  36 <p>This new way of running tests is developer-centric. It assumes that you have built a jdk locally and want to test it. Running common test targets is simple, and more complex ad-hoc combination of tests is possible. The user interface is forgiving, and clearly report errors it cannot resolve.</p>
  37 <p>The main target &quot;run-test&quot; uses the jdk-image as the tested product. There is also an alternate target &quot;exploded-run-test&quot; that uses the exploded image instead. Not all tests will run successfully on the exploded image, but using this target can greatly improve rebuild times for certain workflows.</p>
  38 <p>Some example command-lines:</p>
  39 <pre><code>$ make run-test-tier1
  40 $ make run-test-jdk_lang JTREG=&quot;JOBS=8&quot;
  41 $ make run-test TEST=jdk_lang
  42 $ make run-test-only TEST=&quot;gtest:LogTagSet gtest:LogTagSetDescriptions&quot; GTEST=&quot;REPEAT=-1&quot;
  43 $ make run-test TEST=&quot;hotspot:hotspot_gc&quot; JTREG=&quot;JOBS=1;TIMEOUT=8;VM_OPTIONS=-XshowSettings -Xlog:gc+ref=debug&quot;
  44 $ make run-test TEST=&quot;jtreg:test/hotspot:hotspot_gc test/hotspot/jtreg/native_sanity/JniVersion.java&quot;
  45 $ make exploded-run-test TEST=tier2</code></pre>
  46 <h3 id="configuration">Configuration</h3>
  47 <p>To be able to run JTReg tests, <code>configure</code> needs to know where to find the JTReg test framework. If it is not picked up automatically by configure, use the <code>--with-jtreg=&lt;path to jtreg home&gt;</code> option to point to the JTReg framework. Note that this option should point to the JTReg home, i.e. the top directory, containing <code>lib/jtreg.jar</code> etc. (An alternative is to set the <code>JT_HOME</code> environment variable to point to the JTReg home before running <code>configure</code>.)</p>
  48 <h2 id="test-selection">Test selection</h2>
  49 <p>All functionality is available using the run-test make target. In this use case, the test or tests to be executed is controlled using the <code>TEST</code> variable. To speed up subsequent test runs with no source code changes, run-test-only can be used instead, which do not depend on the source and test image build.</p>
  50 <p>For some common top-level tests, direct make targets have been generated. This includes all JTReg test groups, the hotspot gtest, and custom tests (if present). This means that <code>make run-test-tier1</code> is equivalent to <code>make run-test TEST=&quot;tier1&quot;</code>, but the latter is more tab-completion friendly. For more complex test runs, the <code>run-test TEST=&quot;x&quot;</code> solution needs to be used.</p>
  51 <p>The test specifications given in <code>TEST</code> is parsed into fully qualified test descriptors, which clearly and unambigously show which tests will be run. As an example, <code>:tier1</code> will expand to <code>jtreg:$(TOPDIR)/test/hotspot/jtreg:tier1 jtreg:$(TOPDIR)/test/jdk:tier1 jtreg:$(TOPDIR)/test/langtools:tier1 jtreg:$(TOPDIR)/test/nashorn:tier1 jtreg:$(TOPDIR)/test/jaxp:tier1</code>. You can always submit a list of fully qualified test descriptors in the <code>TEST</code> variable if you want to shortcut the parser.</p>
  52 <h3 id="jtreg">JTReg</h3>
  53 <p>JTReg tests can be selected either by picking a JTReg test group, or a selection of files or directories containing JTReg tests.</p>
  54 <p>JTReg test groups can be specified either without a test root, e.g. <code>:tier1</code> (or <code>tier1</code>, the initial colon is optional), or with, e.g. <code>hotspot:tier1</code>, <code>test/jdk:jdk_util</code> or <code>$(TOPDIR)/test/hotspot/jtreg:hotspot_all</code>. The test root can be specified either as an absolute path, or a path relative to the OpenJDK top directory, or the <code>test</code> directory. For simplicity, the hotspot JTReg test root, which really is <code>hotspot/jtreg</code> can be abbreviated as just <code>hotspot</code>.</p>
  55 <p>When specified without a test root, all matching groups from all test roots will be added. Otherwise, only the group from the specified test root will be added.</p>
  56 <p>Individual JTReg tests or directories containing JTReg tests can also be specified, like <code>test/hotspot/jtreg/native_sanity/JniVersion.java</code> or <code>hotspot/jtreg/native_sanity</code>. Just like for test root selection, you can either specify an absolute path (which can even point to JTReg tests outside the source tree), or a path relative to either the OpenJDK top directory or the <code>test</code> directory. <code>hotspot</code> can be used as an alias for <code>hotspot/jtreg</code> here as well.</p>
  57 <p>As long as the test groups or test paths can be uniquely resolved, you do not need to enter the <code>jtreg:</code> prefix. If this is not possible, or if you want to use a fully qualified test descriptor, add <code>jtreg:</code>, e.g. <code>jtreg:test/hotspot/jtreg/native_sanity</code>.</p>
  58 <h3 id="gtest">Gtest</h3>
  59 <p>Since the Hotspot Gtest suite is so quick, the default is to run all tests. This is specified by just <code>gtest</code>, or as a fully qualified test descriptor <code>gtest:all</code>.</p>
  60 <p>If you want, you can single out an individual test or a group of tests, for instance <code>gtest:LogDecorations</code> or <code>gtest:LogDecorations.level_test_vm</code>. This can be particularly useful if you want to run a shaky test repeatedly.</p>
  61 <p>For Gtest, there is a separate test suite for each JVM variant. The JVM variant is defined by adding <code>/&lt;variant&gt;</code> to the test descriptor, e.g. <code>gtest:Log/client</code>. If you specify no variant, gtest will run once for each JVM variant present (e.g. server, client). So if you only have the server JVM present, then <code>gtest:all</code> will be equivalent to <code>gtest:all/server</code>.</p>
  62 <h2 id="test-results-and-summary">Test results and summary</h2>
  63 <p>At the end of the test run, a summary of all tests run will be presented. This will have a consistent look, regardless of what test suites were used. This is a sample summary:</p>
  64 <pre><code>==============================
  65 Test summary
  66 ==============================
  67    TEST                                          TOTAL  PASS  FAIL ERROR
  68 &gt;&gt; jtreg:jdk/test:tier1                           1867  1865     2     0 &lt;&lt;
  69    jtreg:langtools/test:tier1                     4711  4711     0     0
  70    jtreg:nashorn/test:tier1                        133   133     0     0
  71 ==============================
  72 TEST FAILURE</code></pre>
  73 <p>Tests where the number of TOTAL tests does not equal the number of PASSed tests will be considered a test failure. These are marked with the <code>&gt;&gt; ... &lt;&lt;</code> marker for easy identification.</p>
  74 <p>The classification of non-passed tests differs a bit between test suites. In the summary, ERROR is used as a catch-all for tests that neither passed nor are classified as failed by the framework. This might indicate test framework error, timeout or other problems.</p>
  75 <p>In case of test failures, <code>make run-test</code> will exit with a non-zero exit value.</p>
  76 <p>All tests have their result stored in <code>build/$BUILD/test-results/$TEST_ID</code>, where TEST_ID is a path-safe conversion from the fully qualified test descriptor, e.g. for <code>jtreg:jdk/test:tier1</code> the TEST_ID is <code>jtreg_jdk_test_tier1</code>. This path is also printed in the log at the end of the test run.</p>
  77 <p>Additional work data is stored in <code>build/$BUILD/test-support/$TEST_ID</code>. For some frameworks, this directory might contain information that is useful in determining the cause of a failed test.</p>
  78 <h2 id="test-suite-control">Test suite control</h2>
  79 <p>It is possible to control various aspects of the test suites using make control variables.</p>
  80 <p>These variables use a keyword=value approach to allow multiple values to be set. So, for instance, <code>JTREG=&quot;JOBS=1;TIMEOUT=8&quot;</code> will set the JTReg concurrency level to 1 and the timeout factor to 8. This is equivalent to setting <code>JTREG_JOBS=1 JTREG_TIMEOUT=8</code>, but using the keyword format means that the <code>JTREG</code> variable is parsed and verified for correctness, so <code>JTREG=&quot;TMIEOUT=8&quot;</code> would give an error, while <code>JTREG_TMIEOUT=8</code> would just pass unnoticed.</p>
  81 <p>To separate multiple keyword=value pairs, use <code>;</code> (semicolon). Since the shell normally eats <code>;</code>, the recommended usage is to write the assignment inside qoutes, e.g. <code>JTREG=&quot;...;...&quot;</code>. This will also make sure spaces are preserved, as in <code>JTREG=&quot;VM_OPTIONS=-XshowSettings -Xlog:gc+ref=debug&quot;</code>.</p>
  82 <p>(Other ways are possible, e.g. using backslash: <code>JTREG=JOBS=1\;TIMEOUT=8</code>. Also, as a special technique, the string <code>%20</code> will be replaced with space for certain options, e.g. <code>JTREG=VM_OPTIONS=-XshowSettings%20-Xlog:gc+ref=debug</code>. This can be useful if you have layers of scripts and have trouble getting proper quoting of command line arguments through.)</p>
  83 <p>As far as possible, the names of the keywords have been standardized between test suites.</p>
  84 <h3 id="jtreg-keywords">JTReg keywords</h3>
  85 <h4 id="jobs">JOBS</h4>
  86 <p>The test concurrency (<code>-concurrency</code>).</p>
  87 <p>Defaults to TEST_JOBS (if set by <code>--with-test-jobs=</code>), otherwise it defaults to JOBS, except for Hotspot, where the default is <em>number of CPU cores/2</em>, but never more than 12.</p>
  88 <h4 id="timeout">TIMEOUT</h4>
  89 <p>The timeout factor (<code>-timeoutFactor</code>).</p>
  90 <p>Defaults to 4.</p>
  91 <h4 id="test_mode">TEST_MODE</h4>
  92 <p>The test mode (<code>-agentvm</code>, <code>-samevm</code> or <code>-othervm</code>).</p>
  93 <p>Defaults to <code>-agentvm</code>.</p>
  94 <h4 id="assert">ASSERT</h4>
  95 <p>Enable asserts (<code>-ea -esa</code>, or none).</p>
  96 <p>Set to <code>true</code> or <code>false</code>. If true, adds <code>-ea -esa</code>. Defaults to true, except for hotspot.</p>
  97 <h4 id="verbose">VERBOSE</h4>
  98 <p>The verbosity level (<code>-verbose</code>).</p>
  99 <p>Defaults to <code>fail,error,summary</code>.</p>
 100 <h4 id="retain">RETAIN</h4>
 101 <p>What test data to retain (<code>-retain</code>).</p>
 102 <p>Defaults to <code>fail,error</code>.</p>
   1 <!DOCTYPE html>
   2 <html xmlns="http://www.w3.org/1999/xhtml" lang="" xml:lang="">
   3 <head>
   4   <meta charset="utf-8" />
   5   <meta name="generator" content="pandoc" />
   6   <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
   7   <title>Testing the JDK</title>
   8   <style type="text/css">
   9       code{white-space: pre-wrap;}
  10       span.smallcaps{font-variant: small-caps;}
  11       span.underline{text-decoration: underline;}
  12       div.column{display: inline-block; vertical-align: top; width: 50%;}
  13   </style>
  14   <link rel="stylesheet" href="../make/data/docs-resources/resources/jdk-default.css">
  15   <!--[if lt IE 9]>
  16     <script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv-printshiv.min.js"></script>
  17   <![endif]-->
  18   <style type="text/css">pre, code, tt { color: #1d6ae5; }</style>
  19 </head>
  20 <body>
  21 <header>
  22 <h1 class="title">Testing the JDK</h1>
  23 </header>
  24 <nav id="TOC">
  25 <ul>
  26 <li><a href="#using-the-run-test-framework">Using the run-test framework</a><ul>
  27 <li><a href="#configuration">Configuration</a></li>
  28 </ul></li>
  29 <li><a href="#test-selection">Test selection</a><ul>
  30 <li><a href="#jtreg">JTReg</a></li>
  31 <li><a href="#gtest">Gtest</a></li>
  32 </ul></li>
  33 <li><a href="#test-results-and-summary">Test results and summary</a></li>
  34 <li><a href="#test-suite-control">Test suite control</a><ul>
  35 <li><a href="#jtreg-keywords">JTReg keywords</a></li>
  36 <li><a href="#gtest-keywords">Gtest keywords</a></li>
  37 </ul></li>
  38 </ul>
  39 </nav>
  40 <h2 id="using-the-run-test-framework">Using the run-test framework</h2>
  41 <p>This new way of running tests is developer-centric. It assumes that you have built a JDK locally and want to test it. Running common test targets is simple, and more complex ad-hoc combination of tests is possible. The user interface is forgiving, and clearly report errors it cannot resolve.</p>
  42 <p>The main target “run-test” uses the jdk-image as the tested product. There is also an alternate target “exploded-run-test” that uses the exploded image instead. Not all tests will run successfully on the exploded image, but using this target can greatly improve rebuild times for certain workflows.</p>
  43 <p>Some example command-lines:</p>
  44 <pre><code>$ make run-test-tier1
  45 $ make run-test-jdk_lang JTREG=&quot;JOBS=8&quot;
  46 $ make run-test TEST=jdk_lang
  47 $ make run-test-only TEST=&quot;gtest:LogTagSet gtest:LogTagSetDescriptions&quot; GTEST=&quot;REPEAT=-1&quot;
  48 $ make run-test TEST=&quot;hotspot:hotspot_gc&quot; JTREG=&quot;JOBS=1;TIMEOUT=8;VM_OPTIONS=-XshowSettings -Xlog:gc+ref=debug&quot;
  49 $ make run-test TEST=&quot;jtreg:test/hotspot:hotspot_gc test/hotspot/jtreg/native_sanity/JniVersion.java&quot;
  50 $ make exploded-run-test TEST=tier2</code></pre>
  51 <h3 id="configuration">Configuration</h3>
  52 <p>To be able to run JTReg tests, <code>configure</code> needs to know where to find the JTReg test framework. If it is not picked up automatically by configure, use the <code>--with-jtreg=&lt;path to jtreg home&gt;</code> option to point to the JTReg framework. Note that this option should point to the JTReg home, i.e. the top directory, containing <code>lib/jtreg.jar</code> etc. (An alternative is to set the <code>JT_HOME</code> environment variable to point to the JTReg home before running <code>configure</code>.)</p>
  53 <h2 id="test-selection">Test selection</h2>
  54 <p>All functionality is available using the run-test make target. In this use case, the test or tests to be executed is controlled using the <code>TEST</code> variable. To speed up subsequent test runs with no source code changes, run-test-only can be used instead, which do not depend on the source and test image build.</p>
  55 <p>For some common top-level tests, direct make targets have been generated. This includes all JTReg test groups, the hotspot gtest, and custom tests (if present). This means that <code>make run-test-tier1</code> is equivalent to <code>make run-test TEST=&quot;tier1&quot;</code>, but the latter is more tab-completion friendly. For more complex test runs, the <code>run-test TEST=&quot;x&quot;</code> solution needs to be used.</p>
  56 <p>The test specifications given in <code>TEST</code> is parsed into fully qualified test descriptors, which clearly and unambigously show which tests will be run. As an example, <code>:tier1</code> will expand to <code>jtreg:$(TOPDIR)/test/hotspot/jtreg:tier1 jtreg:$(TOPDIR)/test/jdk:tier1 jtreg:$(TOPDIR)/test/langtools:tier1 jtreg:$(TOPDIR)/test/nashorn:tier1 jtreg:$(TOPDIR)/test/jaxp:tier1</code>. You can always submit a list of fully qualified test descriptors in the <code>TEST</code> variable if you want to shortcut the parser.</p>
  57 <h3 id="jtreg">JTReg</h3>
  58 <p>JTReg tests can be selected either by picking a JTReg test group, or a selection of files or directories containing JTReg tests.</p>
  59 <p>JTReg test groups can be specified either without a test root, e.g. <code>:tier1</code> (or <code>tier1</code>, the initial colon is optional), or with, e.g. <code>hotspot:tier1</code>, <code>test/jdk:jdk_util</code> or <code>$(TOPDIR)/test/hotspot/jtreg:hotspot_all</code>. The test root can be specified either as an absolute path, or a path relative to the JDK top directory, or the <code>test</code> directory. For simplicity, the hotspot JTReg test root, which really is <code>hotspot/jtreg</code> can be abbreviated as just <code>hotspot</code>.</p>
  60 <p>When specified without a test root, all matching groups from all test roots will be added. Otherwise, only the group from the specified test root will be added.</p>
  61 <p>Individual JTReg tests or directories containing JTReg tests can also be specified, like <code>test/hotspot/jtreg/native_sanity/JniVersion.java</code> or <code>hotspot/jtreg/native_sanity</code>. Just like for test root selection, you can either specify an absolute path (which can even point to JTReg tests outside the source tree), or a path relative to either the JDK top directory or the <code>test</code> directory. <code>hotspot</code> can be used as an alias for <code>hotspot/jtreg</code> here as well.</p>
  62 <p>As long as the test groups or test paths can be uniquely resolved, you do not need to enter the <code>jtreg:</code> prefix. If this is not possible, or if you want to use a fully qualified test descriptor, add <code>jtreg:</code>, e.g. <code>jtreg:test/hotspot/jtreg/native_sanity</code>.</p>
  63 <h3 id="gtest">Gtest</h3>
  64 <p>Since the Hotspot Gtest suite is so quick, the default is to run all tests. This is specified by just <code>gtest</code>, or as a fully qualified test descriptor <code>gtest:all</code>.</p>
  65 <p>If you want, you can single out an individual test or a group of tests, for instance <code>gtest:LogDecorations</code> or <code>gtest:LogDecorations.level_test_vm</code>. This can be particularly useful if you want to run a shaky test repeatedly.</p>
  66 <p>For Gtest, there is a separate test suite for each JVM variant. The JVM variant is defined by adding <code>/&lt;variant&gt;</code> to the test descriptor, e.g. <code>gtest:Log/client</code>. If you specify no variant, gtest will run once for each JVM variant present (e.g. server, client). So if you only have the server JVM present, then <code>gtest:all</code> will be equivalent to <code>gtest:all/server</code>.</p>
  67 <h2 id="test-results-and-summary">Test results and summary</h2>
  68 <p>At the end of the test run, a summary of all tests run will be presented. This will have a consistent look, regardless of what test suites were used. This is a sample summary:</p>
  69 <pre><code>==============================
  70 Test summary
  71 ==============================
  72    TEST                                          TOTAL  PASS  FAIL ERROR
  73 &gt;&gt; jtreg:jdk/test:tier1                           1867  1865     2     0 &lt;&lt;
  74    jtreg:langtools/test:tier1                     4711  4711     0     0
  75    jtreg:nashorn/test:tier1                        133   133     0     0
  76 ==============================
  77 TEST FAILURE</code></pre>
  78 <p>Tests where the number of TOTAL tests does not equal the number of PASSed tests will be considered a test failure. These are marked with the <code>&gt;&gt; ... &lt;&lt;</code> marker for easy identification.</p>
  79 <p>The classification of non-passed tests differs a bit between test suites. In the summary, ERROR is used as a catch-all for tests that neither passed nor are classified as failed by the framework. This might indicate test framework error, timeout or other problems.</p>
  80 <p>In case of test failures, <code>make run-test</code> will exit with a non-zero exit value.</p>
  81 <p>All tests have their result stored in <code>build/$BUILD/test-results/$TEST_ID</code>, where TEST_ID is a path-safe conversion from the fully qualified test descriptor, e.g. for <code>jtreg:jdk/test:tier1</code> the TEST_ID is <code>jtreg_jdk_test_tier1</code>. This path is also printed in the log at the end of the test run.</p>
  82 <p>Additional work data is stored in <code>build/$BUILD/test-support/$TEST_ID</code>. For some frameworks, this directory might contain information that is useful in determining the cause of a failed test.</p>
  83 <h2 id="test-suite-control">Test suite control</h2>
  84 <p>It is possible to control various aspects of the test suites using make control variables.</p>
  85 <p>These variables use a keyword=value approach to allow multiple values to be set. So, for instance, <code>JTREG=&quot;JOBS=1;TIMEOUT=8&quot;</code> will set the JTReg concurrency level to 1 and the timeout factor to 8. This is equivalent to setting <code>JTREG_JOBS=1 JTREG_TIMEOUT=8</code>, but using the keyword format means that the <code>JTREG</code> variable is parsed and verified for correctness, so <code>JTREG=&quot;TMIEOUT=8&quot;</code> would give an error, while <code>JTREG_TMIEOUT=8</code> would just pass unnoticed.</p>
  86 <p>To separate multiple keyword=value pairs, use <code>;</code> (semicolon). Since the shell normally eats <code>;</code>, the recommended usage is to write the assignment inside qoutes, e.g. <code>JTREG=&quot;...;...&quot;</code>. This will also make sure spaces are preserved, as in <code>JTREG=&quot;VM_OPTIONS=-XshowSettings -Xlog:gc+ref=debug&quot;</code>.</p>
  87 <p>(Other ways are possible, e.g. using backslash: <code>JTREG=JOBS=1\;TIMEOUT=8</code>. Also, as a special technique, the string <code>%20</code> will be replaced with space for certain options, e.g. <code>JTREG=VM_OPTIONS=-XshowSettings%20-Xlog:gc+ref=debug</code>. This can be useful if you have layers of scripts and have trouble getting proper quoting of command line arguments through.)</p>
  88 <p>As far as possible, the names of the keywords have been standardized between test suites.</p>
  89 <h3 id="jtreg-keywords">JTReg keywords</h3>
  90 <h4 id="jobs">JOBS</h4>
  91 <p>The test concurrency (<code>-concurrency</code>).</p>
  92 <p>Defaults to TEST_JOBS (if set by <code>--with-test-jobs=</code>), otherwise it defaults to JOBS, except for Hotspot, where the default is <em>number of CPU cores/2</em>, but never more than 12.</p>
  93 <h4 id="timeout">TIMEOUT</h4>
  94 <p>The timeout factor (<code>-timeoutFactor</code>).</p>
  95 <p>Defaults to 4.</p>
  96 <h4 id="test_mode">TEST_MODE</h4>
  97 <p>The test mode (<code>-agentvm</code>, <code>-samevm</code> or <code>-othervm</code>).</p>
  98 <p>Defaults to <code>-agentvm</code>.</p>
  99 <h4 id="assert">ASSERT</h4>
 100 <p>Enable asserts (<code>-ea -esa</code>, or none).</p>
 101 <p>Set to <code>true</code> or <code>false</code>. If true, adds <code>-ea -esa</code>. Defaults to true, except for hotspot.</p>
 102 <h4 id="verbose">VERBOSE</h4>
 103 <p>The verbosity level (<code>-verbose</code>).</p>
 104 <p>Defaults to <code>fail,error,summary</code>.</p>
 105 <h4 id="retain">RETAIN</h4>
 106 <p>What test data to retain (<code>-retain</code>).</p>
 107 <p>Defaults to <code>fail,error</code>.</p>
< prev index next >