New jdk/make/netbeans/README

1 Working on OpenJDK using NetBeans
2 This note describes how to work on the OpenJDK from NetBeans. We've
3 provided several NetBeans projects as starting points. Below we'll
4 describe how to use them, as well as how to create your own.
5
6 Getting Started
7 In addition to the source bundle for Open JDK, you'll need to download
8 and install copies of the JDK and of NetBeans 6. And if you want to run
9 tests on the JDK (you do want to run tests, right?), you'll need to
10 install the jtreg test harness.
11
12 In this note, when pathnames are not fully specified, they should be
13 interpreted as being relative to the directory containing this README
14 and the NetBeans projects themselves.
15
16 The JDK build process is largely make-based, and is not
17 exceptionally tolerant of pathnames with spaces in them (such as
18 "Program Files". Please be sure to install everything in a
19 directories whose paths don't have any spaces!
20
21 Downloading the JDK
22 You've probably done this a million times. Download and install it
23 from http://java.sun.com/javase
24
25 Downloading the OpenJDK sources
26 Since you're reading this, d you've already downloaded the OpenJDK
27 source bundle. Later in this document we'll refer to the location
28 where you installed the Open JDK sources as *install-dir*.
29
30 Downloading a pre-built, JDK 7
31 This will be necessary to do builds of some of the projects. In
32 general, you want to download and install a pre-built JDK that
33 corresponds to the OpenJDK sources you download. Building the entire
34 OpenJDK depends on a few parts of the pre-built JDK. Get this from
35 http://download.java.net/jdk7/binaries
36
37 Note: For working on certain projects, like JMX and JConsole, you
38 may find convenient to use a pre-built version of JDK 7 (or
39 OpenJDK) rather than building your own. This will allow you
40 to build only that part of the OpenJDK sources which correspond
41 to that project.
42
43 NetBeans 6
44 Yep, NetBeans *6*. Nope, not FCS'd yet. We're on the edge here,
45 enjoy it! Get the latest working development build of NetBeans 6
46 from http://netbeans.org
47
48 jtreg
49 "jtreg" is the test harness for running OpenJDK's regression tests.
50 Get it from http://openjdk.java.net/jtreg
51
52 Ant
53 NetBeans comes with ant, but if you use a separately-installed copy
54 please make sure that it is at least version 1.7.0.
55
56 Configuring
57 Building OpenJDK is hard and complex. No, strike that. While it's not
58 exactly "easy", we've got it down to *relatively* small set of
59 properties you need to set.
60
61 The NetBeans projects provided here share a fair amount of common
62 structure. They share properties values where it makes sense. Each
63 project loads properties from these properties files, in this order
64
65 ${basedir}/nbproject/private/build.properties
66 $HOME/.openjdk/${ant.project.name}-build.properties
67 $HOME/.openjdk/build.properties
68 ${basedir}/build.properties
69
70 (${basedir} refers to the directory containing a particular NetBeans
71 project.) The first time a property defined determines value: it is
72 *not* overridden if it is read from properties files read later. The net
73 result is that by carefully choosing where to define a property, you can
74 have it for a specific project, all uses of a specific project (useful
75 if you work on multiple copies of the OpenJDK sources), all projects, or
76 only projects in a specific sandbox.
77
78 With that in mind, please set the following properties. Presuming you
79 want the same values for all your work, set them in
80 $HOME/.openjdk/build.properties.
81
82 * bootstrap.jdk
83 Set to the location where you installed JDK 7.
84
85 * jtreg.home
86 Set to the location where you installed jtreg.
87
88 * make.options
89 Some of the projects invoke "make", since they compile native code.
90 The make.options property is for passing information about what you
91 installed where to make. Change the paths to fit your particular
92 situation:
93
94 make.options=\
95 ALT_BOOTDIR=/home/me/bin/jdk1.6.0 \
96 ALT_JDK_IMPORT_PATH=/home/me/bin/jdk1.7.0 \
97 OPENJDK=true
98
99 The trailing '\' are important, so that make gets the above as a
100 single set of options.
101
102 You might want to add additional additional options: see the README
103 for the project you're using for more information. And see
104 *install-dir*/jdk/make/README-builds.html
105 to read much more about building the JDK.
106
107 Windows-specific configuration
108 First, please note that the entire JDK cannot currently be built on
109 Windows platforms. This will likely limit your ability to build
110 make-based projects. See
111 *install-dir*/jdk/make/README-builds.html
112 for full information on issues with building on the Windows platform.
113
114 That said, there are two ways to work with the Windows-required settings
115 for the Microsoft tools. Either:
116
117 * Set environment variables values in Windows
118 Doing so means accessing the System control panel in Windows, and
119 setting the environment variables there.
120
121 By doing so, you can launch NetBeans by double-clicking its icon,
122 and the environment variable values will be available.
123
124 * Set environment variable values in a shell
125 Doing so means adding the settings to an init file (e.g. .bashrc,
126 .cshrc, etc.) or a file that you source before running NetBeans. In
127 this case, you'll have to launch NetBeans from the command line in a
128 shell in which you've set the environment variables.
129
130 In either case, the end result should be that the settings are available
131 to the make-based build process when it runs from within NetBeans.
132
133 The make-based builds presumes that you're using cygwin, and expects to
134 find "make" in c:\cygwin\bin\make. If you've installed cygwin elsewhere,
135 set "make" in a properties file.
136
137 Configuring Project Properties
138 A note of caution is in order: These are NetBeans *freeform* projects.
139 If you use the NetBeans GUI to examine them, things are likely to not
140 look "right". Please don't edit them there, please instead use a text
141 editor.
142
143 Locale Requirements
144 To build the Open JDK sources, be certain that you are using the "C"
145 locale on Unix (R) platforms, or "English (United States)" locale on
146 Windows.
147
148 Platforms and architectures, oh my!
149 The Open JDK can be built for a variety of operating system platforms
150 and hardware architectures. The resulting builds are always placed in a
151 directory which contains the platform and architecture as part of the
152 pathname, as in *platform*-*arch*. For example, if you build the jdk
153 project on a Linux platform running on x86 hardware, the resulting build
154 will be in:
155
156 *install-dir*/jdk/build/linux-i586
157
158 We've provided support for some platforms and architectures in
159 common/architectures. Add another, if your needs require it.
160
161 Provided NetBeans projects
162 This section describes the NetBeans projects that help you work on
163 particular parts of the JDK. While they're largely similar in structure
164 and should work the way you expect NetBeans projects to work: edit,
165 build, test, etc. But there are some differences. They don't all support
166 the same targets (e.g., there's nothing to run in jarzip project).
167
168 Some projects are built by invoking make, since they involve compilation
169 of native code or other activities that cannot be done by javac. We call
170 these "make-based", and call all others "ant-based".
171
172 They all are configured by way of a build.properties file, which
173 specifies what subdirectories of the JDK sources they manipulate, what
174 directories contain their tests, whether they use make or ant, etc.
175
176 The very first time you open any one of these projects on set of Open
177 JDK sources, NetBeans will scan the entire set of sources, not just
178 those for the project you opened. This will take a few minutes, but will
179 ensure that Go To Type, Go To Source, and so on work as expected. Later,
180 when you open other projects on the same Open JDK sources, there will be
181 at most a slight delay.
182
183 There's a README accompanying each project. Most are text files, which
184 you can Open in NetBeans, some are HTML files, in which case unless you
185 enjoy reading raw HTML, you're better off choosing the *View* menu item
186 from the context menu, which will display the README in your web
187 browser.
188
189 Finally, note that these projects were all created by different people,
190 and are while some attempt has been made to make them look and behave
191 the same, they are maintained separately and will vary somewhat.
192
193 The projects currently provided are:
194
195 jdk (directory "jdk")
196 A convenient starting point for the other projects, and from which
197 you can build the entire OpenJDK. Please note that depending on your
198 hardware, this could take a *very* long time. The results of the
199 build are in *install-dir*/jdk/build/*platform*-*arch*.
200
201 world (directory "world")
202 This project builds both the Hotspot VM and all of JavaSE. Please
203 note that pretty much regardless of your hardware, this *will* take
204 a long time, and use *lots* of disk space (more than 3GB). The
205 results of the build are in
206 *install-dir*/build/*platform*-*arch* and
207 *install-dir*/build/*platform*-*arch*-fastdebug.
208
209 Consult the project's README file for details.
210
211 AWT & Java2d (directory "awt2d")
212 For working on AWT and Java2d. Supports running the Font2DTest demo.
213
214 This is a make-based project: In order to build this project, you
215 should build the jdk project first, since AWT and Java2d include
216 native code.
217
218 JConsole (directory "jconsole")
219 For working on JConsole. Creates ../dist/lib/jconsole.jar. Supports
220 running and debugging JConsole.
221
222 This ant-based project does *not* require that you build the jdk
223 project first, provided that you use a pre-built version of JDK 7.
224
225 Java (TM) Management Extensions (JMX(TM)) API (directory "jmx")
226 For working on JMX source code. Creates ../dist/lib/jmx.jar.
227
228 This ant-based project does *not* require that you build the jdk
229 project first, provided that you use a pre-built version of JDK 7.
230
231 Jar & Zip (directory "jarzip")
232 For working on jar & zip. It builds the zip library (including
233 native code), the jar library, and the jar tool. Creates an
234 executable jar program in ../build/*platform*-*arch*/bin/jar.
235
236 This is a make-based project: In order to build this project, you
237 should build the jdk project first, since AWT and Java2d include
238 native code.
239
240 Swing (directory "swing")
241 For working on Swing. Creates ../dist/lib/swing.jar. Supports
242 running and debugging the SampleTree demo.
243
244 This ant-based project does *not* require that you build the jdk
245 project first, provided that you use a pre-built version of JDK 7.
246
247 In addition, there are projects for building the compiler, javadoc,
248 and related tools, in the OpenJDK langtools component. These
249 projects are separate from those described here, and have their
250 own set of guidelines and conventions. For more details, see the
251 README files in make/netbeans in the OpenJDK langtools component.
252
253 Running Tests
254 We use the jtreg test harness, described more fully at
255 http://openjdk.java.net/jtreg
256
257 The OpenJDK tests are in the default Java package, are public classes,
258 and have a "static void main(String[] args)" with which they are
259 invoked. Some tests are actually shell scripts, which might compile
260 code, etc. jtreg is quite flexible.
261
262 To run tests for a project, use *Test Project* from NetBeans. From the
263 command line, you can invoke "ant jtreg" on any individual project's
264 build.xml file.
265
266 In either NetBeans of on the command line, jtreg prints summary output
267 about the pass/fail nature of each test. An HTML report of the entire
268 test run is
269
270 ../build/*platform*-*arch*/jtreg/*ant-project-name*/JTreport/report.html
271
272 In that same JTreport directory are also individual HTML files
273 summarizing the test environment, test passes and failures, etc.
274
275 More detail on any individual test is under
276
277 ../build/*platform*-*arch*/jtreg/*ant-project-name*/JTwork.
278
279 For example, details about the awt/Modal/SupportedTest/SupportedTest
280 test are under the JTwork directory at the same pathname as the test
281 itself in a ".jtr" file. For example:
282
283 ../build/*platform*-*arch*/jtreg/*ant-project-name*/JTwork/awt/Modal/SupportedTest/SupportedTest.jtr
284
285 Sometimes you will see that running jtreg has resulted in a failure.
286 This does not always mean that a test has an error in it. Jtreg
287 distinguishes between these two cases. There are a number of tests that
288 are "ignored", and not run, and these are reported as failures.
289
290 You can run a single test by right clicking on it and choosing *Run
291 File* from the context menu. Similarly, you can debug a single test by
292 choosing *Debug File*.
293
294 Debugging
295 Debugging is enabled by default in ant-based projects, as if
296 "-g:lines,vars,source" were given. You can alter these settings via
297 entries in one of the configuration properties files. For example:
298
299 javac.debug=false
300 javac.debuglevel=<debug level options>
301
302 To debug a project or test, use NetBeans in the normal way, with *Debug
303 Project* or *Debug File*. Note that not all projects provide a target
304 that can be debugged, but tests can be debugged.
305
306 Creating Javadoc
307 You can create Javadoc for any of the projects: just choose *Generate
308 Javadoc for Project* from the NetBeans menu. Your default browser will
309 open up, displaying the just-generated javadoc.
310
311 Javadoc gets generated into a separate subdirectory for each project.
312 For example, the Jar & Zip project's javadoc gets generated in
313
314 ../build/*platform*-*arch*/jtreg/*ant-project-name*/javadoc/jarzip
315
316 Cleaning projects
317 Each project can of course be cleaned. Make-based and ant-based projects
318 differ a little in what exactly gets cleaned. In both cases, all jtreg
319 results and javadoc are removed.
320
321 In ant-based projects, project-specific files as determined by the
322 project's build.properties file are removed from the classes and gensrc
323 directories that are under ../build/*platform*-*arch*.
324
325 In make-based projects, "make clean" is run in the same directories as
326 "make all" is run when building the project.
327
328 Please note that the jdk project is "special" with respect to
329 cleaning: in this case, the entire ../build directory is removed.
330 Similar for the world project.
331
332 Creating your own NetBeans project
333 The project's we've provided are hopefully a useful starting point, but
334 chances are that you want to work on something else. This section will
335 describe how to select an existing project, and then adapt it to your
336 needs.
337
338 Considerations
339 The first consideration is whether or not the code in which you're
340 interested needs anything beyond javac and copying of resources to
341 build. If so, then you'll need to create a make-based project. If not,
342 an ant-based project is possible. See the project descriptions above to
343 learn which are make-based, and which are ant-based.
344
345 The second consideration is to consider the files that you'll need. Each
346 project is defined by 3 files:
347
348 * build.xml
349 This is the ant build script. For a make-based project, they tend to
350 have a target for "make clean" and another for "make all", each of
351 which invokes "make-run" in the same set of directories. Take a look
352 at jarzip/build.xml for an example.
353
354 For an ant-based project, there might be nothing, with all the work
355 done via the declaration of properties in the build.properties file.
356 Take a look at jconsole/build.xml for an example, and notice how it
357 overrides the -pre-compile and -post-compile targets that are
358 defined in common/shared.xml (where they are defined to do nothing).
359
360 * build.properties
361 This file defines the directories (and possibly files) that are
362 included in and excluded from. Basically, a file is considered to be
363 in a project if it is mentioned in the includes list, or is
364 contained under a directory mentioned in that list, *unless* it is
365 explicitly excluded or is contained under a directory that is
366 excluded. Take a look awt2d/build.properties for an example.
367
368 * nbproject/project.xml
369 This file defines a project for NetBeans for a "freeform" project.
370 Each declares several entity references, which are used later in the
371 project. For an example, see javadoc/nbproject/project.xml, which is
372 an ant-based project. Compare that with
373 jarzip/nbproject/project.xml, which is make-based. Not much
374 difference! That's because while the jarzip project is make-based,
375 it does not have any platform-specifc native code. Contrast that
376 with awt2d/nbproject/project.xml, which does have native code;
377 notice that it uses platform-specific entity references.
378
379 In summary, we recommend exploring the given projects, and choosing one
380 that most closely suits our needs.
381
382 Example: A project for working on collections
383 Let's create a project to work with on the collections classes. There's no native
384 code here, so an ant-based project will do. Therefore, the jconsole
385 project is a reasonable project to use as a starting point.
386
387 Clone the existing project
388 Make a directory for the collections project next to the existing projects:
389
390 % mkdir -p collections/nbproject
391
392 Copy files from the jconsole project:
393
394 % cp jconsole/build.properties collections
395 % cp jconsole/build.xml collections
396 % cp jconsole/nbproject/project.xml collections/nbproject
397
398 Change the set of files included in the project
399 The collections sources are all under one directory, and we want to include
400 them all. The same is true of the tests. So edit
401 collections/build.properties so that it contains these lines:
402
403 includes=\
404 java/util/
405 excludes=\
406 java/util/Calendar.java,\
407 java/util/jar/,\
408 java/util/logging/,\
409 java/util/prefs/,\
410 java/util/regex/,\
411 java/util/spi/,\
412 java/util/zip/,\
413 **/*-XLocales.java.template
414 jtreg.tests=\
415 java/util/**/*Collection/ \
416 java/util/**/*Map/ \
417 java/util/**/*Set/ \
418 java/util/**/*List/
419
420 Notice the trailing "/" in some of those pathnames: that tells NetBeans to
421 treat the path as a directory and include (or exclude) everything beneath
422 it in the hierarchy. Note also how we include java/util, but then exclude
423 several directories under that which are not related to collections.
424
425 The build.xml for collections is about as simple as can be. First, change the
426 name of the project:
427
428 <project name="collections" default="build" basedir=".">
429
430 Then remove the -pre-compile target from the build.xml. Change the
431 -post-compile target to create collections.jar without any manifest, and
432 to only contain the collections-related classes. The jar task now looks
433 like this:
434
435 <jar destfile="${dist.dir}/lib/collections.jar">
436 <fileset dir="${classes.dir}">
437 <include name="java/util/*.class"/>
438 <exclude name="java/util/Calendar*.class"/>
439 </fileset>
440 </jar>
441
442 Also, change the clean target to remove collections.jar instead of
443 jconsole.jar.
444
445 Now edit project.xml file. NetBeans uses an internal name and a
446 user-visible name, both of which should be changed:
447
448 <name>Collections</name> 
449
450 <property name="name">collections</property> 
451
452 Inside of <ide-actions>, you'll see actions defined for "run" and
453 "debug". The Open JDK sources don't include any interesting Collections
454 demos, but leave these here for now: Chances are you'll find or create
455 some collections app of your own, and want to run and or debug it.
456
457 Now, open the Collections project in NetBeans. You'll find that it operates
458 just like all the other projects.
459
460 If/when you want to have this project run a collections demo, change the run
461 target in collections/build.xml to invoke it in whatever manner is appropriate
462 for the app. From NetBeans, you should be able to run and debug the app,
463 including setting breakpoints in collections code.
464
465 Appendix 1: Customizations
466 There are several ways to customize NetBeans projects. These projects
467 share a common structure, based on common/shared.xml and
468 common/make.xml. Because of that sharing, some mechanisms described
469 below apply to most any project.
470
471 Several properties can be user-defined (and several should not be
472 user-defined!). There are different properties files read. Some default
473 targets can be overridden.
474
475 Property files
476 When projects are started, and when when ant runs (whether from NetBeans
477 or the command line), these properties files are loaded in the order
478 shown:
479
480 ${basedir}/nbproject/private/build.properties
481 $HOME/.openjdk/${ant.project.name}-build.properties
482 $HOME/.openjdk/build.properties
483 ${basedir}/build.properties
484
485 Recall that with ant, once a property is defined, its value cannot be
486 changed, so it's "first one wins".
487
488 To set or change a property for all your projects, put the change into
489 $HOME/.openjdk/build.properties. This will affect all projects,
490 regardless of how many copies of the Open JDK sources you have
491 installed.
492
493 Let's say you have 2 copies of the Open JDK sources installed on your
494 machine. To set or change a property for only the jconsole projects, but
495 for both of them, make the change in
496 $HOME/.openjdk/${ant.project.name}-build.properties. If you wanted to
497 make the change for only one of them, do it in that project's
498 ${basedir}/build.properties or
499 ${basedir}/nbproject/private/build.properties.
500
501 Note that the ${basedir}/build.properties file is provided as part of
502 the Open JDK sources. If you want to make a change for a particular
503 project, you can do so there. To be sure that you don't ever
504 accidentally check it in to the Open JDK sources, you might prefer to
505 change it in ${basedir}/nbproject/private/build.properties.
506
507 User-definable Properties
508 You can provide your own definitions for the properties listed below. We
509 don't recommend overriding the definitions of other properties.
510
511 The following two properties should be set before you try to use the
512 projects with NetBeans or ant:
513
514 * bootstrap.jdk
515 Default: None. Please set this, normally in
516 $HOME/.openjdk/build.properties.
517
518 * jtreg.home
519 Default: None. Please set this, normally in
520 $HOME/.openjdk/build.properties.
521
522 These options are for configuring the behavior of make:
523
524 * use.make
525 Default: Not set. Set this, normally in ${basedir}/build.properties,
526 for a project which is make-based.
527
528 * make
529 Default: The right make for the platform, at the normal location, set
530 in *install-dir*/jdk/make/netbeans/common/make.xml
531
532 * make.options
533 Default: Empty string. Set this to any options you want to pass to
534 make, normally in ${basedir}/build.properties.
535
536 The remaining options are for use at your discretion:
537
538 * javac.options
539 Default: -Xlint
540
541 * javac.debug
542 Default: true
543
544 * javac.debuglevel
545 Default: lines,vars,source
546
547 * javadoc.options
548 Default: Empty string. Some projects will need to set this to
549 increase the heap for running javadoc. For example, see the jconsole
550 project.
551
552 * javadoc.packagenames
553 Default: "none". Set this only if your project has packages that
554 should be javadoc'd which are outside of those listed in the javadoc
555 target's packageset. See the jconsole project for an example.
556
557 * jtreg.tests
558 Default: None. Set this to a list of tests and/or directories
559 containing regression tests, normally in
560 ${basedir}/build.properties.
561
562 * jtreg.options
563 Default: Empty string. See http://openjdk.java.net/jtreg
564
565 * jtreg.vm.options
566 Default: Empty string. See http://openjdk.java.net/jtreg
567
568 * jtreg.samevm
569 Default: false. See http://openjdk.java.net/jtreg
570
571 User-overridable Targets
572 The following targets are provided for your convenience in customizing
573 various standard actions of the build process. The default action for
574 each one is to do nothing.
575
576 These come in pairs, allowing your scripts to take some action before or
577 after a standard action.
578
579 * -pre-init
580 Runs before any other initialization has been done.
581
582 * -post-init
583 Runs before after all other initialization has been done.
584
585 * -pre-compile
586 Runs before compilation, whether via ant or make. Note that in the
587 case of make, it is before the -build-make target has run, not after
588 each individual make-run has run.
589
590 * -post-compile
591 Runs after compilation, whether via ant or make.
592
593 * -pre-jtreg
594 Runs before regression tests are run.
595
596 * -post-jtreg
597 Runs before after regression tests are run.
598
599 In a make-based project, you should override these targets to do the
600 build and clean actions required of your project.
601
602 * -build-make
603 * -clean-make
604
605 Known Issues
606 Tests won't run: waiting for lock
607 Occasionally when running tests, there will be a delay, followed by a
608 message like this:
609 Waiting to lock test result cache for
610 /tmp/jdk/build/linux-i586/jtreg/jconsole/JTwork for 20 seconds
611 The workaround is to stop the tests, rm -rf the offending jtreg/<project>
612 directory by hand, and re-run the tests.
613
614 Can't run nor debug a single test in the JConsole test
615 In most projects, you can run a single test by opening it in the editor,
616 and choosing Run File from the context menu. If you try this with the a
617 JConsole test, instead you'll see that *all* tests from *all* projects
618 are run. The workaround is to not try to run a single JConsole test.
619 Debugging is similarly problematic (both running and debugging use the
620 same underlying infrastructure).
621
622 If you do Run File a JConsole tests, you can always stop them by pressing
623 the stop button in the NetBeans output window. But you'll be surprised to
624 learn that they are actually still running in the background. The only
625 way out of this situation is to exit NetBeans. A few more tests will run,
626 but after restarting NetBeans things will be OK.
627
628 Attribution
629 UNIX is a registered trademark in the United States and other countries,
630 exclusively licensed through X/Open Company, Ltd.
631