rev 8961 : [mq]: diff-shenandoah.patch
rev 8830 : 8131682: C1 should use multibyte nops everywhere
Reviewed-by: dlong, goetz, adinn, aph, vlivanov
rev 8413 : 8079792: GC directory structure cleanup
Reviewed-by: brutisso, stefank, david
rev 7965 : 7143664: Clean up OrderAccess implementations and usage
Summary: Clarify and correct the abstract model for memory barriers provided by the orderAccess class. Refactor the implementations using template specialization to allow the bulk of the code to be shared, with platform specific customizations applied as needed.
Reviewed-by: acorn, dcubed, dholmes, dlong, goetz, kbarrett, sgehwolf
Contributed-by: Erik Osterlund <erik.osterlund@lnu.se>
rev 7844 : 8068977: Remove unused sun.misc.Unsafe prefetch intrinsic support
Reviewed-by: kvn, vlivanov
rev 7331 : 8062370: Various minor code improvements
Summary: A lot of fixes useful to improve the code quality.
Reviewed-by: coleenp, dholmes
rev 6602 : 8046684: sharedRuntime.cpp...assert(((nmethod*)cb)->is_at_poll_or_poll_return(pc)) failed: safepoint polling: type must be poll
Summary: Emit a poll relocation for the testl in is_polling_page_far() on x86
Reviewed-by: vlivanov, roland
rev 6307 : 8032410: compiler/uncommontrap/TestStackBangRbp.java times out on Solaris-Sparc V9
Summary: make compiled code bang the stack by the worst case size of the interpreter frame at deoptimization points.
Reviewed-by: twisti, kvn
rev 6221 : 8039043: Implicit null check is in the wrong place in C1 -UseCompressedOops
Summary: Null check is placed in a wrong place when storing a null to an object field on x64 with compressed oops off
Reviewed-by: roland, vlivanov, kvn
rev 6181 : 8031203: remove SafepointPollOffset
Reviewed-by: kvn, roland
rev 5813 : 8028764: dtrace/hotspot_jni/ALL/ALL001 crashes the vm on Solaris-amd64, SIGSEGV in MarkSweep::follow_stack()+0x8a
Summary: C1 generates code to encode compressed oop into tmp register before runtime call for patching where GC may happen
Reviewed-by: iveresov, twisti, kvn
Contributed-by: mgerdin <mikael.gerdin@oracle.com>
rev 5559 : 8026495: JVM Crashes when started with -XX:+DTraceMethodProbes on Solaris x86_64
Summary: Fix wrong calling convention in LIR_Assembler::emit_unwind_handler(), T_METADATA support in calling convention generator, C1 register allocator
Reviewed-by: twisti, jrose
rev 5545 : 8008242: VerifyOops is broken on SPARC
Summary: Fixed displacement issues in SPARC macroassembler and ensure that getClass intrinsic temporary result is T_METADATA
Reviewed-by: kvn, twisti
rev 5479 : 8023657: New type profiling points: arguments to call
Summary: x86 interpreter and c1 type profiling for arguments at calls
Reviewed-by: kvn, twisti
rev 5302 : Merge
rev 5301 : 8023542: Test java/io/File/CheckPermission.java fails due to unfinished recursion (java.lang.StackOverflowError) when JIT'ed code (-client,-server) is running
Summary: Move null check before klass reference materialization in checkcast
Reviewed-by: kvn, roland
rev 5259 : 8015107: NPG: Use consistent naming for metaspace concepts
Reviewed-by: coleenp, mgerdin, hseigel
rev 5193 : 7199175: JSR 292: C1 needs patching when invokedynamic/invokehandle call site is not linked
Summary: Do patching rather bailing out for unlinked call with appendix
Reviewed-by: twisti, kvn
rev 4918 : 7088419: Use x86 Hardware CRC32 Instruction with java.util.zip.CRC32
Summary: add intrinsics using new instruction to interpreter, C1, C2, for suitable x86; add test
Reviewed-by: kvn, twisti
rev 4425 : 7153771: array bound check elimination for c1
Summary: when possible optimize out array bound checks, inserting predicates when needed.
Reviewed-by: never, kvn, twisti
Contributed-by: thomaswue <thomas.wuerthinger@oracle.com>
rev 3883 : 8003240: x86: move MacroAssembler into separate file
Reviewed-by: kvn
rev 3729 : Merge
rev 3724 : 7054512: Compress class pointers after perm gen removal
Summary: support of compress class pointers in the compilers.
Reviewed-by: kvn, twisti
rev 3707 : 8000213: NPG: Should have renamed arrayKlass and typeArrayKlass
Summary: Capitalize these metadata types (and objArrayKlass)
Reviewed-by: stefank, twisti, kvn
rev 3671 : 7023898: Intrinsify AtomicLongFieldUpdater.getAndIncrement()
Summary: use shorter instruction sequences for atomic add and atomic exchange when possible.
Reviewed-by: kvn, jrose
rev 3616 : 7195816: NPG: Crash in c1_ValueType - ShouldNotReachHere
Summary: C1 needs knowledge of T_METADATA at the LIR level.
Reviewed-by: kvn, coleenp
rev 3602 : 6964458: Reimplement class meta-data storage to use native memory
Summary: Remove PermGen, allocate meta-data in metaspace linked to class loaders, rewrite GC walking, rewrite and rename metadata to be C++ classes
Reviewed-by: jmasa, stefank, never, coleenp, kvn, brutisso, mgerdin, dholmes, jrose, twisti, roland
Contributed-by: jmasa <jon.masamitsu@oracle.com>, stefank <stefan.karlsson@oracle.com>, mgerdin <mikael.gerdin@oracle.com>, never <tom.rodriguez@oracle.com>
rev 3534 : 7023639: JSR 292 method handle invocation needs a fast path for compiled code
6984705: JSR 292 method handle creation should not go through JNI
Summary: remove assembly code for JDK 7 chained method handles
Reviewed-by: jrose, twisti, kvn, mhaupt
Contributed-by: John Rose <john.r.rose@oracle.com>, Christian Thalinger <christian.thalinger@oracle.com>, Michael Haupt <michael.haupt@oracle.com>
rev 3413 : 7174928: JSR 292: unresolved invokedynamic call sites deopt and osr infinitely
Reviewed-by: kvn
rev 3352 : 7133857: exp() and pow() should use the x87 ISA on x86
Summary: use x87 instructions to implement exp() and pow() in interpreter/c1/c2.
Reviewed-by: kvn, never, twisti
rev 3325 : 6924259: Remove String.count/String.offset
Summary: Allow a version of String class that doesn't have count and offset fields.
Reviewed-by: never, coleenp
rev 3309 : 7160539: JDeveloper crashes on 64-bit Windows
Summary: x64 C1 needs to zero upper 32bits when doing l2i conversion
Reviewed-by: never, kvn
rev 3157 : 7120481: storeStore barrier in constructor with final field
Summary: Issue storestore barrier before constructor return if the constructor write final field.
Reviewed-by: dholmes, jrose, roland, coleenp
Contributed-by: Jiangli Zhou <jiangli.zhou@oracle.com>
rev 3000 : 7131288: COMPILE SKIPPED: deopt handler overflow (retry at different tier)
Summary: Fix exception handler stub size, enable guarantees to check for the correct deopt and exception stub sizes in the future
Reviewed-by: kvn, never, twisti
rev 2965 : Merge
rev 2956 : 7118863: Move sizeof(klassOopDesc) into the *Klass::*_offset_in_bytes() functions
Summary: Moved sizeof(klassOopDesc), changed the return type to ByteSize and removed the _in_bytes suffix.
Reviewed-by: never, bdelsart, coleenp, jrose
rev 2933 : 7117052: instanceKlass::_init_state can be u1 type
Summary: Change instanceKlass::_init_state field to u1 type.
Reviewed-by: bdelsart, coleenp, dholmes, phh, never
Contributed-by: Jiangli Zhou <jiangli.zhou@oracle.com>
rev 2721 : 7089790: integrate bsd-port changes
Reviewed-by: kvn, twisti, jrose
Contributed-by: Kurt Miller <kurt@intricatesoftware.com>, Greg Lewis <glewis@eyesbeyond.com>, Jung-uk Kim <jkim@freebsd.org>, Christos Zoulas <christos@zoulas.com>, Landon Fuller <landonf@plausible.coop>, The FreeBSD Foundation <board@freebsdfoundation.org>, Michael Franz <mvfranz@gmail.com>, Roger Hoover <rhoover@apple.com>, Alexander Strange <astrange@apple.com>
rev 2664 : 7083786: dead various dead chunks of code
Reviewed-by: iveresov, kvn
rev 2501 : 7047491: C1: registers saved incorrectly when calling checkcast_arraycopy stub
Summary: Save and restore the argument registers around the call to checkcast_arraycopy
Reviewed-by: never, roland
rev 2497 : 7046893: LP64 problem with double_quadword in c1_LIRAssembler_x86.cpp
Summary: Fixed invalid casts in address computation
Reviewed-by: kvn, never
Contributed-by: thomas.salter@unisys.com
rev 2326 : 7035713: 3DNow Prefetch Instruction Support
Summary: The upcoming processors from AMD are the first that support 3dnow prefetch without supporting the 3dnow instruction set.
Reviewed-by: kvn
Contributed-by: tom.deneau@amd.com
rev 2293 : 7033154: Improve C1 arraycopy performance
Summary: better static analysis. Take advantage of array copy stubs.
Reviewed-by: never
rev 2262 : 7032388: guarantee(VM_Version::supports_cmov()) failed: illegal instruction on i586 after 6919934
Summary: 6919934 added some unguarded cmov instructions which hit a guarantee on older hardware.
Reviewed-by: never, iveresov, kvn, phh
rev 2251 : 6964776: c2 should ensure the polling page is reachable on 64 bit
Summary: Materialize the pointer to the polling page in a register instead of using rip-relative addressing when the distance from the code cache is larger than disp32.
Reviewed-by: never, kvn
rev 2168 : 7012914: JSR 292 MethodHandlesTest C1: frame::verify_return_pc(return_address) failed: must be a return pc
Reviewed-by: never, bdelsart
rev 1997 : 7010618: C1: array length should be treated at int on 64bit during array allocation
Summary: Sign-extend the length argument during array allocation
Reviewed-by: never, kvn
rev 1977 : 7009231: C1: Incorrect CAS code for longs on SPARC 32bit
Summary: Fix CAS of longs on SPARC 32bit and cmove on SPARC 64bit.
Reviewed-by: kvn
rev 1920 : 7005241: C1: SEGV in java.util.concurrent.LinkedTransferQueue.xfer() with compressed oops
Summary: Implementation of the CAS primitive for x64 compressed oops was incorrect. It kills rscratch2 register (r11), which is allocatable in C1. Also, we don't need to restore cmpval as it's never used after that, so we need only one temporary register, which can be scratch1.
Reviewed-by: kvn, never
rev 1914 : 7003554: (tiered) assert(is_null_object() || handle() != NULL) failed: cannot embed null pointer
Summary: C1 with profiling doesn't check whether the MDO has been really allocated, which can silently fail if the perm gen is full. The solution is to check if the allocation failed and bailout out of inlining or compilation.
Reviewed-by: kvn, never
rev 1909 : 6985015: C1 needs to support compressed oops
Summary: This change implements compressed oops for C1 for x64 and sparc. The changes are mostly on the codegen level, with a few exceptions when we do access things outside of the heap that are uncompressed from the IR. Compressed oops are now also enabled with tiered.
Reviewed-by: twisti, kvn, never, phh
rev 1879 : 6989984: Use standard include model for Hospot
Summary: Replaced MakeDeps and the includeDB files with more standardized solutions.
Reviewed-by: coleenp, kvn, kamg
rev 1768 : 6991512: G1 barriers fail with 64bit C1
Summary: Fix compare-and-swap intrinsic problem with G1 post-barriers and issue with branch ranges in G1 stubs on sparc
Reviewed-by: never, kvn
rev 1752 : Merge
rev 1750 : 6988018: dtrace/hotspot/MethodInvocation/MethodInvocation002 crashes with client compiler
Reviewed-by: iveresov, kvn, kamg
rev 1728 : 6988779: c1_LIRAssembler_x86.cpp crashes VS2010 compiler
Summary: The workaround changes the scope of the variable
Reviewed-by: phh, ysr, kvn
rev 1711 : 6919069: client compiler needs to capture more profile information for tiered work
Summary: Added profiling of instanceof and aastore.
Reviewed-by: kvn, jrose, never
rev 1710 : 6984056: C1: incorrect code for integer constant addition on x64
Summary: Fix add/sub of constants to ints on x64
Reviewed-by: kvn
rev 1703 : 6953144: Tiered compilation
Summary: Infrastructure for tiered compilation support (interpreter + c1 + c2) for 32 and 64 bit. Simple tiered policy implementation.
Reviewed-by: kvn, never, phh, twisti
rev 1499 : Merge
rev 1492 : 6955349: C1: Make G1 barriers work with x64
Summary: This fixes G1 barriers in c1 on x64.
Reviewed-by: never
rev 1484 : 6930772: JSR 292 needs to support SPARC C1
Summary: C1 for SPARC needs to support JSR 292.
Reviewed-by: never, jrose
rev 1472 : 6941466: Oracle rebranding changes for Hotspot repositories
Summary: Change all the Sun copyrights to Oracle copyright
Reviewed-by: ohair
rev 1411 : Merge
rev 1409 : 6888953: some calls to function-like macros are missing semicolons
Reviewed-by: pbk, kvn
rev 1398 : 6946892: c1 shouldn't sign-extend to upper 32bits on x64
Summary: c1 does sign-extension when it loads ints and shorts from memory to 64-bit registers. This causes problems for c2 because it relies on the fact the int passed in a 64-bit register is zero-extended.
Reviewed-by: never
rev 1378 : 6939930: exception unwind changes in 6919934 hurts compilation speed
Reviewed-by: twisti
rev 1369 : 6942223: c1 64 bit fixes
Summary: This fixes lir_cmp_l2i on x64 and sparc 64bit, and the debug info generation.
Reviewed-by: never
rev 1301 : 6934966: JSR 292 add C1 logic for saved SP over MethodHandle calls
Summary: The logic for x86 C1 to save the SP over MH calls is pretty straight forward but SPARC handles that differently.
Reviewed-by: never, jrose
rev 1297 : 6932496: c1: deoptimization of jsr subroutine fails on sparcv9
Summary: store jsr ret bci as intptr constant in c1 debug info
Reviewed-by: never
rev 1295 : 6919934: JSR 292 needs to support x86 C1
Summary: This implements JSR 292 support for C1 x86.
Reviewed-by: never, jrose, kvn
rev 1257 : Merge
rev 1251 : 6923002: assert(false,"this call site should not be polymorphic")
Summary: Clear the total count when a receiver information is cleared.
Reviewed-by: never, jrose
rev 1216 : Merge
rev 1213 : 6902182: 4/4 Starting with jdwp agent should not incur performance penalty
Summary: Rename can_post_exceptions support to can_post_on_exceptions. Add support for should_post_on_exceptions flag to permit per JavaThread optimizations.
Reviewed-by: never, kvn, dcubed
Contributed-by: tom.deneau@amd.com
rev 1206 : 6614597: Performance variability in jvm2008 xml.validation
Summary: Fix incorrect marking of methods as not compilable.
Reviewed-by: never
rev 1204 : 6921352: JSR 292 needs its own deopt handler
Summary: We need to introduce a new MH deopt handler so we can easily determine if the deopt happened at a MH call site or not.
Reviewed-by: never, jrose
rev 1201 : 6921339: backout 6917766
Reviewed-by: mr
rev 1200 : 6917766: JSR 292 needs its own deopt handler
Summary: We need to introduce a new MH deopt handler so we can easily determine if the deopt happened at a MH call site or not.
Reviewed-by: never, jrose
rev 1060 : 6769124: various 64-bit fixes for c1
Reviewed-by: never
rev 989 : 6863023: need non-perm oops in code cache for JSR 292
Summary: Make a special root-list for those few nmethods which might contain non-perm oops.
Reviewed-by: twisti, kvn, never, jmasa, ysr
rev 780 : 6788527: Server vm intermittently fails with assertion "live value must not be garbage" with fastdebug bits
Summary: Cache Jvmti and DTrace flags used by Compiler.
Reviewed-by: never
rev 647 : Merge
rev 644 : 6813212: factor duplicated assembly code for general subclass check (for 6655638)
Summary: Code in interp_masm, stubGenerator, c1_LIRAssembler, and AD files moved into MacroAssembler.
Reviewed-by: kvn
rev 628 : Merge
rev 622 : 6812678: macro assembler needs delayed binding of a few constants (for 6655638)
Summary: minor assembler enhancements preparing for method handles
Reviewed-by: kvn
rev 579 : 6814575: Update copyright year
Summary: Update copyright for files that have been modified in 2009, up to 03/09
Reviewed-by: katleman, tbell, ohair
rev 512 : 6787106: Hotspot 32 bit build fails on platforms having different definitions for intptr_t & int32_t
Summary: Avoid casting between int32_t and intptr_t specifically for MasmAssembler::movptr in 32 bit platforms.
Reviewed-by: jrose, kvn
rev 321 : 6746320: Hotspot regression test for 6512111 fails in -Xmixed mode
Reviewed-by: kvn
rev 304 : 5108146: Merge i486 and amd64 cpu directories
6459804: Want client (c1) compiler for x86_64 (amd64) for faster start-up
Reviewed-by: kvn
rev 196 : 6719955: Update copyright year
Summary: Update copyright year for files that have been modified in 2008
Reviewed-by: ohair, tbell
rev 29 : 6633953: type2aelembytes{T_ADDRESS} should be 8 bytes in 64 bit VM
Summary: T_ADDRESS size is defined as 'int' size (4 bytes) but C2 use it for raw pointers and as memory type for StoreP and LoadP nodes.
Reviewed-by: jrose
rev 0 : Initial load
15 * You should have received a copy of the GNU General Public License version
16 * 2 along with this work; if not, write to the Free Software Foundation,
17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
18 *
19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
20 * or visit www.oracle.com if you need additional information or have any
21 * questions.
22 *
23 */
24
25 #include "precompiled.hpp"
26 #include "asm/macroAssembler.hpp"
27 #include "asm/macroAssembler.inline.hpp"
28 #include "c1/c1_Compilation.hpp"
29 #include "c1/c1_LIRAssembler.hpp"
30 #include "c1/c1_MacroAssembler.hpp"
31 #include "c1/c1_Runtime1.hpp"
32 #include "c1/c1_ValueStack.hpp"
33 #include "ci/ciArrayKlass.hpp"
34 #include "ci/ciInstance.hpp"
35 #include "gc/shared/barrierSet.hpp"
36 #include "gc/shared/cardTableModRefBS.hpp"
37 #include "gc/shared/collectedHeap.hpp"
38 #include "nativeInst_x86.hpp"
39 #include "oops/objArrayKlass.hpp"
40 #include "runtime/sharedRuntime.hpp"
41 #include "vmreg_x86.inline.hpp"
42
43
44 // These masks are used to provide 128-bit aligned bitmasks to the XMM
45 // instructions, to allow sign-masking or sign-bit flipping. They allow
46 // fast versions of NegF/NegD and AbsF/AbsD.
47
48 // Note: 'double' and 'long long' have 32-bits alignment on x86.
49 static jlong* double_quadword(jlong *adr, jlong lo, jlong hi) {
50 // Use the expression (adr)&(~0xF) to provide 128-bits aligned address
51 // of 128-bits operands for SSE instructions.
52 jlong *operand = (jlong*)(((intptr_t)adr) & ((intptr_t)(~0xF)));
53 // Store the value to a 128-bits operand.
54 operand[0] = lo;
1449 case lir_cond_greater: acond = Assembler::above; break;
1450 default: ShouldNotReachHere();
1451 }
1452 } else {
1453 switch (op->cond()) {
1454 case lir_cond_equal: acond = Assembler::equal; break;
1455 case lir_cond_notEqual: acond = Assembler::notEqual; break;
1456 case lir_cond_less: acond = Assembler::less; break;
1457 case lir_cond_lessEqual: acond = Assembler::lessEqual; break;
1458 case lir_cond_greaterEqual: acond = Assembler::greaterEqual;break;
1459 case lir_cond_greater: acond = Assembler::greater; break;
1460 case lir_cond_belowEqual: acond = Assembler::belowEqual; break;
1461 case lir_cond_aboveEqual: acond = Assembler::aboveEqual; break;
1462 default: ShouldNotReachHere();
1463 }
1464 }
1465 __ jcc(acond,*(op->label()));
1466 }
1467 }
1468
1469 void LIR_Assembler::emit_opConvert(LIR_OpConvert* op) {
1470 LIR_Opr src = op->in_opr();
1471 LIR_Opr dest = op->result_opr();
1472
1473 switch (op->bytecode()) {
1474 case Bytecodes::_i2l:
1475 #ifdef _LP64
1476 __ movl2ptr(dest->as_register_lo(), src->as_register());
1477 #else
1478 move_regs(src->as_register(), dest->as_register_lo());
1479 move_regs(src->as_register(), dest->as_register_hi());
1480 __ sarl(dest->as_register_hi(), 31);
1481 #endif // LP64
1482 break;
1483
1484 case Bytecodes::_l2i:
1485 #ifdef _LP64
1486 __ movl(dest->as_register(), src->as_register_lo());
1487 #else
1488 move_regs(src->as_register_lo(), dest->as_register());
1945 assert(cmpval == rax, "wrong register");
1946 assert(newval != NULL, "new val must be register");
1947 assert(cmpval != newval, "cmp and new values must be in different registers");
1948 assert(cmpval != addr, "cmp and addr must be in different registers");
1949 assert(newval != addr, "new value and addr must be in different registers");
1950
1951 if ( op->code() == lir_cas_obj) {
1952 #ifdef _LP64
1953 if (UseCompressedOops) {
1954 __ encode_heap_oop(cmpval);
1955 __ mov(rscratch1, newval);
1956 __ encode_heap_oop(rscratch1);
1957 if (os::is_MP()) {
1958 __ lock();
1959 }
1960 // cmpval (rax) is implicitly used by this instruction
1961 __ cmpxchgl(rscratch1, Address(addr, 0));
1962 } else
1963 #endif
1964 {
1965 if (os::is_MP()) {
1966 __ lock();
1967 }
1968 __ cmpxchgptr(newval, Address(addr, 0));
1969 }
1970 } else {
1971 assert(op->code() == lir_cas_int, "lir_cas_int expected");
1972 if (os::is_MP()) {
1973 __ lock();
1974 }
1975 __ cmpxchgl(newval, Address(addr, 0));
1976 }
1977 #ifdef _LP64
1978 } else if (op->code() == lir_cas_long) {
1979 Register addr = (op->addr()->is_single_cpu() ? op->addr()->as_register() : op->addr()->as_register_lo());
1980 Register newval = op->new_value()->as_register_lo();
1981 Register cmpval = op->cmp_value()->as_register_lo();
1982 assert(cmpval == rax, "wrong register");
1983 assert(newval != NULL, "new val must be register");
1984 assert(cmpval != newval, "cmp and new values must be in different registers");
1985 assert(cmpval != addr, "cmp and addr must be in different registers");
1986 assert(newval != addr, "new value and addr must be in different registers");
1987 if (os::is_MP()) {
1988 __ lock();
|
15 * You should have received a copy of the GNU General Public License version
16 * 2 along with this work; if not, write to the Free Software Foundation,
17 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
18 *
19 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
20 * or visit www.oracle.com if you need additional information or have any
21 * questions.
22 *
23 */
24
25 #include "precompiled.hpp"
26 #include "asm/macroAssembler.hpp"
27 #include "asm/macroAssembler.inline.hpp"
28 #include "c1/c1_Compilation.hpp"
29 #include "c1/c1_LIRAssembler.hpp"
30 #include "c1/c1_MacroAssembler.hpp"
31 #include "c1/c1_Runtime1.hpp"
32 #include "c1/c1_ValueStack.hpp"
33 #include "ci/ciArrayKlass.hpp"
34 #include "ci/ciInstance.hpp"
35 #include "gc/shenandoah/shenandoahHeap.hpp"
36 #include "gc/shared/barrierSet.hpp"
37 #include "gc/shared/cardTableModRefBS.hpp"
38 #include "gc/shared/collectedHeap.hpp"
39 #include "nativeInst_x86.hpp"
40 #include "oops/objArrayKlass.hpp"
41 #include "runtime/sharedRuntime.hpp"
42 #include "vmreg_x86.inline.hpp"
43
44
45 // These masks are used to provide 128-bit aligned bitmasks to the XMM
46 // instructions, to allow sign-masking or sign-bit flipping. They allow
47 // fast versions of NegF/NegD and AbsF/AbsD.
48
49 // Note: 'double' and 'long long' have 32-bits alignment on x86.
50 static jlong* double_quadword(jlong *adr, jlong lo, jlong hi) {
51 // Use the expression (adr)&(~0xF) to provide 128-bits aligned address
52 // of 128-bits operands for SSE instructions.
53 jlong *operand = (jlong*)(((intptr_t)adr) & ((intptr_t)(~0xF)));
54 // Store the value to a 128-bits operand.
55 operand[0] = lo;
1450 case lir_cond_greater: acond = Assembler::above; break;
1451 default: ShouldNotReachHere();
1452 }
1453 } else {
1454 switch (op->cond()) {
1455 case lir_cond_equal: acond = Assembler::equal; break;
1456 case lir_cond_notEqual: acond = Assembler::notEqual; break;
1457 case lir_cond_less: acond = Assembler::less; break;
1458 case lir_cond_lessEqual: acond = Assembler::lessEqual; break;
1459 case lir_cond_greaterEqual: acond = Assembler::greaterEqual;break;
1460 case lir_cond_greater: acond = Assembler::greater; break;
1461 case lir_cond_belowEqual: acond = Assembler::belowEqual; break;
1462 case lir_cond_aboveEqual: acond = Assembler::aboveEqual; break;
1463 default: ShouldNotReachHere();
1464 }
1465 }
1466 __ jcc(acond,*(op->label()));
1467 }
1468 }
1469
1470 void LIR_Assembler::emit_opShenandoahWriteBarrier(LIR_OpShenandoahWriteBarrier* op) {
1471 Label done;
1472 Register obj = op->in_opr()->as_register();
1473 Register res = op->result_opr()->as_register();
1474 Register tmp1 = op->tmp1_opr()->as_register();
1475 Register tmp2 = op->tmp2_opr()->as_register();
1476 assert_different_registers(res, tmp1, tmp2);
1477
1478 if (res != obj) {
1479 __ mov(res, obj);
1480 }
1481
1482 // Check for null.
1483 if (op->need_null_check()) {
1484 __ testptr(res, res);
1485 __ jcc(Assembler::zero, done);
1486 }
1487
1488 // Check for evacuation-in-progress
1489 Address evacuation_in_progress = Address(r15_thread, in_bytes(JavaThread::evacuation_in_progress_offset()));
1490 __ cmpb(evacuation_in_progress, 0);
1491
1492 // The read-barrier.
1493 __ movptr(res, Address(res, -8));
1494
1495 __ jcc(Assembler::equal, done);
1496
1497 // Check for object in collection set.
1498 __ movptr(tmp1, res);
1499 __ shrptr(tmp1, ShenandoahHeapRegion::RegionSizeShift);
1500 __ movptr(tmp2, (intptr_t) ShenandoahHeap::in_cset_fast_test_addr());
1501 __ movbool(tmp2, Address(tmp2, tmp1, Address::times_1));
1502 __ testb(tmp2, 0x1);
1503 __ jcc(Assembler::zero, done);
1504
1505 if (res != rax) {
1506 __ xchgptr(res, rax); // Move obj into rax and save rax into obj.
1507 }
1508
1509 __ call(RuntimeAddress(Runtime1::entry_for(Runtime1::shenandoah_write_barrier_slow_id)));
1510
1511 if (res != rax) {
1512 __ xchgptr(rax, res); // Swap back obj with rax.
1513 }
1514
1515 __ bind(done);
1516
1517 }
1518
1519 void LIR_Assembler::emit_opConvert(LIR_OpConvert* op) {
1520 LIR_Opr src = op->in_opr();
1521 LIR_Opr dest = op->result_opr();
1522
1523 switch (op->bytecode()) {
1524 case Bytecodes::_i2l:
1525 #ifdef _LP64
1526 __ movl2ptr(dest->as_register_lo(), src->as_register());
1527 #else
1528 move_regs(src->as_register(), dest->as_register_lo());
1529 move_regs(src->as_register(), dest->as_register_hi());
1530 __ sarl(dest->as_register_hi(), 31);
1531 #endif // LP64
1532 break;
1533
1534 case Bytecodes::_l2i:
1535 #ifdef _LP64
1536 __ movl(dest->as_register(), src->as_register_lo());
1537 #else
1538 move_regs(src->as_register_lo(), dest->as_register());
1995 assert(cmpval == rax, "wrong register");
1996 assert(newval != NULL, "new val must be register");
1997 assert(cmpval != newval, "cmp and new values must be in different registers");
1998 assert(cmpval != addr, "cmp and addr must be in different registers");
1999 assert(newval != addr, "new value and addr must be in different registers");
2000
2001 if ( op->code() == lir_cas_obj) {
2002 #ifdef _LP64
2003 if (UseCompressedOops) {
2004 __ encode_heap_oop(cmpval);
2005 __ mov(rscratch1, newval);
2006 __ encode_heap_oop(rscratch1);
2007 if (os::is_MP()) {
2008 __ lock();
2009 }
2010 // cmpval (rax) is implicitly used by this instruction
2011 __ cmpxchgl(rscratch1, Address(addr, 0));
2012 } else
2013 #endif
2014 {
2015 if (UseShenandoahGC) {
2016 Label done;
2017 Label retry;
2018
2019 __ bind(retry);
2020
2021 // Save original cmp-value into tmp1, before following cas destroys it.
2022 __ movptr(op->tmp1()->as_register(), op->cmp_value()->as_register());
2023
2024 if (os::is_MP()) {
2025 __ lock();
2026 }
2027 __ cmpxchgptr(newval, Address(addr, 0));
2028
2029 // If the cmpxchg succeeded, then we're done.
2030 __ jcc(Assembler::equal, done);
2031
2032 // Resolve the original cmp value.
2033 oopDesc::bs()->interpreter_read_barrier(masm(), op->tmp1()->as_register());
2034 // Resolve the old value at address. We get the old value in cmp/rax
2035 // when the comparison in cmpxchg failed.
2036 __ movptr(op->tmp2()->as_register(), cmpval);
2037 oopDesc::bs()->interpreter_read_barrier(masm(), op->tmp2()->as_register());
2038
2039 // We're done if the expected/cmp value is not the same as old. It's a valid
2040 // cmpxchg failure then. Otherwise we need special treatment for Shenandoah
2041 // to prevent false positives.
2042 __ cmpptr(op->tmp1()->as_register(), op->tmp2()->as_register());
2043 __ jcc(Assembler::equal, retry);
2044
2045 __ bind(done);
2046 } else {
2047 if (os::is_MP()) {
2048 __ lock();
2049 }
2050 __ cmpxchgptr(newval, Address(addr, 0));
2051 }
2052 }
2053 } else {
2054 assert(op->code() == lir_cas_int, "lir_cas_int expected");
2055 if (os::is_MP()) {
2056 __ lock();
2057 }
2058 __ cmpxchgl(newval, Address(addr, 0));
2059 }
2060 #ifdef _LP64
2061 } else if (op->code() == lir_cas_long) {
2062 Register addr = (op->addr()->is_single_cpu() ? op->addr()->as_register() : op->addr()->as_register_lo());
2063 Register newval = op->new_value()->as_register_lo();
2064 Register cmpval = op->cmp_value()->as_register_lo();
2065 assert(cmpval == rax, "wrong register");
2066 assert(newval != NULL, "new val must be register");
2067 assert(cmpval != newval, "cmp and new values must be in different registers");
2068 assert(cmpval != addr, "cmp and addr must be in different registers");
2069 assert(newval != addr, "new value and addr must be in different registers");
2070 if (os::is_MP()) {
2071 __ lock();
|