Java Adds Foreign Functions and Memory API

Java 20 introduced the Foreign Functions & Memory API in its second preview. Foreign functions in this context refers to functions or code outside the Java Virtual Machine (JVM), while foreign memory refers to memory not managed by the JVM. Below we explore what problems the new API solves.

Problem with Foreign Memory Management

How can a Java developer access data stored in memory that is non-Java, implying that it is outside the Java runtime? Performance critical applications such as machine learning applications based on TensorFlow cannot completely depend on on-heap memory. “On-heap” memory is the memory allocated on JVM’s heap. On-heap memory has some shortcomings that make it unsuitable for some use cases. On-heap memory depends on JVM’s garbage collection for deallocation, and garbage collection incurs a performance overhead; the known performance issues are fully explored in the Garbage Collection Performance section. Off-heap memory access becomes necessary, especially for performance critical applications.

Two APIs already available (ByteBuffer API & Unsafe API) to provide access to off-heap memory have some limitations that make them unsuitable for systems programming in Java.

Solution

The new Foreign Functions & Memory API provides an efficient access to off-heap memory. The off-heap memory allocation and deallocation is predictable and can be fully managed by a Java developer. The new API introduces several interfaces for allocating, deallocating, and accessing foreign memory. This new API differentiates between on-heap memory and off-heap memory and can be used with different types (native, persistent, and managed heap) of foreign memory. “Native” implies data stored on off-heap memory.

The java.lang.foreign.MemorySegment represents a memory segment, or a contiguous region of memory. The new Foreign Functions & Memory API is located in the java.lang.foreign package. A memory segment could be a heap segment, which implies that it provides access to a memory region on the JVM's heap. Or, a memory segment could be a native segment, which implies that it provides access to an off-heap memory region. In addition to being able to obtain a native segment using a factory method allocateNative(), native segments can be obtained by mapping a file onto an off-heap region. Such segments are called mapped segments and can be persisted/loaded to-and-from the file and memory that maps the file. An object of type of the new SegmentAllocator interface can be used to allocate memory segments. The MemoryLayout interface represents the memory layout that describes the contents of a memory segment. The abstract class VarHandle represents a handle to off-heap data structures. The SegmentScope interface represents a memory segment's scope—global or automatic. The memory segments with global scope are always available to all threads and the associated backing memory regions are never deallocated. The automatic segment scope is automatically managed by the garbage collector.

Another benefit of the new API not hitherto mentioned is that it provides efficient access to non-volatile memory. Non-volatile memory is able to store data even when the power is shut down.

Problem with Foreign Functions Management

The Java Native Interface (JNI) that is already available since version 1.1 does provide a foreign function interface to let Java code call native code (C, C++, and assembly) and vice versa, but JNI has the following limitations:

• A developer has to go through a toolchain of platform-specific artifacts to interact with native code.
• JNI provides cross-functionality with only a few languages—C and C++.
• JNI does not provide a dynamic mapping of Java types to native code types by using some sort of wrapper functions, but instead depends on a developer to provide the glue code.
• In summary, JNI is slow, clunky, and expects a lot from a developer.

Solution

The new Foreign Functions & Memory API overcomes these limitations of the JNI to provide efficient interoperation with native code. The new java.lang.foreign.Linker function provides to-and-from interaction between Java code and foreign functions. The main benefits of the Linker interface are:

• It allows both downcalls and upcalls. Downcalls are calls from Java code to native code, and upcalls are calls from native code to Java code.
• It creates a linker specific for the ABI (Application Binary Interface) of the underlying native platform. An ABI is a set of calling conventions and data types associated with the compiler, OS, and processor used by the native library.

The new FunctionDescriptor interface is provided to model the behavior of the foreign functions. And the new SymbolLookup interface is provided to locate and lookup symbols in native libraries, which is essential to be able to interoperate with native libraries.