JNI/Windows Gotchas: UnsatisfiedLinkError

The Short Story

This post describes a solution that has worked for me when I get a an UnsatisfiedLinkError on Windows using MingW, while developing a JNI library.

The Long Story

Java Native Interface (JNI) development on Windows is extremely annoying. There are all kinds of land mines strewn randomly about the landscape to snare the wary and unwary alike. Any one of these issues can require hours of hair-tearing in order to get things to a working state.

After dealing with enough of these, I decided to post some of the solutions I've found in order to save others from premature baldness. For today's entertainment, I have selected the infameous UnsatisfiedLinkError.

java.lang.UnsatisfiedLinkError: no found in java.library.path

This is caused by not having the DLL that contains your C/C++ code in the path that Java is using to find it. If you have this problem, try this:

• Bring up the properties for the project containing the native declarations
• From the properties dialog, bring up the build path for the project.
• From the build path properties, edit the native libraries location for the project.
• From the resulting dialog, select the folder where your DLLs live

Bring up the properties for the project containing the native declarations

For example:

For the CLIPC project, this is clipc-java. You can be sure by selecting the Java project that contains the .java files with declarations like this one:

From the properties dialog, bring up the build path for the project.

For example:

From the build path properties, edit the native libraries location for the project

For example:

From the resulting dialog, select the folder where your DLLs live

For example, this tells Eclipse to look in the workspace:

Then this tells Eclipse where to look for the DLL:

Conclusion

If you have an UnsatisfiedLinkError, you may need to tell Eclipse where your DLLs live. This post gives a step-by-step on how to do that.

Introducing CLIPC

The Short Story

CLIPC is a new open-source java library for IPC. It provides new IPC primitives like semaphores and shared queues, and it makes existing primitives like shared memory easier to use. CLIPC currently supports the Windows and Linux platforms.

The Long Story

CLIPC is the com.lts.ipc library that I wrote because I could not find a Java library that did some Inter-Process Communications (IPC) functions that I was interested in. CLIPC has been the topic of several talks that I have given recently at the Boulder Java Users Group (BJUG) and the Denver Open Source Users Group (DOSUG).

Over the next couple of weeks, I am hoping to create some blog entries about CLIPC and some of the trials and tribulations I went through to write the library. This will be of interest to people who are interested in IPC and also those are are interested in the Java Native Interface (JNI).

CLIPC makes use of JNI because Java does not support certain IPC concepts like Semaphores and FIFOs. This requires the use of JNI to perform the required system calls and whatnot through C, a language that the two platforms that CLIPC currently supports uses.

Another aspect of CLIPC/JNI is that it requires the creation of a consistent interface across multiple platforms. Both Windows and Linux support First-In, First-Out messaging (FIFOs), but the similarities pretty much end there.

For example, on Windows there are Named Pipes. These are always bi-directionaly and function in a lot of ways like very fast TCP/IP connections. Named pipes are represented with files in a special directory.

Linux also supports FIFOs, but on that platform they are uni-directional. They appear to be files on the file system in that there are no naming requirements and they can appear anywhere that regular files can. Linux FIFOs require no special system calls in order to connect to, though they do require special calls to create.

How do you reconcile these differences in order to create a uniform interface to FIFOs? Should they really be more like the named pipes of Windows and allow bi-directional data flow or should they be uni-directional?

I don't know if I made the best decisions possible for CLIPC, but I can talk about why I made the decisions that I did. This will be the topic of future blog entries.

ByteBuffer is Fast Enough

The Short Story

• ByteBuffer.get/put is fast enough
• Concentrate data into chunks of 256+ bytes

The Long Story

Following up my posting about how ByteBuffer.get/put appeared to be about 3 times slower than simple array accessing, I did some testing.

ByteBuffer is Slow? Not so Fast!

ByteBuffer.put(byte[] b) is actually faster than using the array operator (e.g., b[0], b[1], etc.) ; for array sizes larger than 256 bytes. If you are moving around data with a ByteBuffer in chunks of less than 256 bytes, arrays may be faster. ByteBuffer can be much faster as the size of the array approaches and exceeds 4kB.

ByteBuffer becomes as fast as regular arrays at a buffer size of about 256 bytes

The Java array operator appears to have a bandwidth of 300MB/sec. This is the amount of time that it takes to read 1 byte from an array location and then write a value to the location. Put another way, the turn-around time for Java arrays is less than 5 nsec.

Performing the same operations with ByteBuffer.get/put at 1 byte per operation, the bandwidth could be as low as 7MB/sec; giving a turn-around time of less than 150 nsec. At first glance this is a huge difference: it suggests that ByteBuffer is 30 times slower than array access!

The bandwidth of ByteBuffer increases exponentially with the buffer size. When you hit about 256 bytes, array access and ByteBuffer are equivalent. ByteBuffer's performance continues to increase until you hit about 4KB in size --- at which point you are looking at over 1GB/sec.

Size Matters

The moral of the story is: try to concentrate your data. If you can put all the commonly accessed stuff into a block of 256+ bytes, access will be efficient. If you are in the situation where your data is small and scattered, things may get dicey.

Time Will Tell

Another important consideration is time. Ask yourself "do I really need a 5nsec response time?" The CPU cache is around 1 nsec and main memory is around 10 nsec. If you really need to turn one or two operations around that fast, you are pushing the performance boundaries of the platform.

If the answer is "yes, it really needs to be that fast," then it may be better to do this sort of thing entirely in JNI. That way Java index boundary checking can be avoided. Since we are talking about time scales where that could be a factor, it could make a difference.

If the answer is "no, 1 usec is easily fast enough," then ByteBuffer is probably the way to go. It will be much simpler and when you are trying to debug something, you can be much more confident that the problem is not in how shared memory is being read from or written to.

Your Mileage Will Vary

If you try the tests I performed on your own system, you will get different results. The nature of the tests makes them very sensitive to things like cache size, front-side bus speed, etc. The values I mention here like array access times of 300MB/sec are approximate.

Test Code

For those who are interested, you can find the test code here. The source is included in the executable JAR.

JRE and Shared Memory Overhead

Shared memory access in Java appears to be about 3 times slower than it could be.

In another post, I mentioned that ByteBuffer and its related classes provide a way for Java developers to access shared memory without having to resort to JNI. The problems come in when you try to perform operations on that segment.

One approach is to get a reference to an array of bytes that represents the data in the segment and then use the regular array syntax to access the data. The ByteBuffer.array() method appears to be the the way to do this, but unfortunately it does work.

Here is an example of what I mean:

// DOES NOT WORK!!
MappedByteBuffer mbb;
// code to initialize mbb omitted
byte[] ba = mbb.array(); // throws UnsupportedOperationException

After looking around a bit, I came to the conclusion that this was the intention of the original developers --- if you want to mess with the data in the segment, then you are supposed to use ByteBuffer.get/put.

This would be fine if get/put were about the same cost as using a straight byte array, but they appear to be 2 to 3 times slower. Here is a simple program that highlights the issue I'm running into. The basic difference is that one version uses:

b1 = bb.get(0);
bb.put(0, b2);

And the other that uses

b1 = bb[0];
bb[0] = b2;

The program performs these operations millions of times and then prints out the time (in milliseconds) they took to run. An example output:

Using get/put: 11578
Using array access: 3234

One thing this shows is that I really need to upgrade my system.

The basic point is that using get/put is a lot slower than using simple arrays. A program that reads and writes a lot of data in shared memory would be a lot faster if it could simply use an array rather than get/put.

Is there a way around this?