recipe № 017

handling exit calls from external library called via JNI (-Dexit)

Sometimes you have an access to surce code of the library that you want to use. In that case, you can take slightly different approach when it comes to exit call (man 3 exit).
Instead of installing atexit handler that is application wide, you can provide a specific handler for the library. All you have to do is to use “-Dexit=new_handler_name” option. This option will make all calls to exit to be replaced with calls to new_handler_name.
In this sample, I will show how to utilize this approach.
After you are finished with the tutorial, your tree structure for the project should look like this:

In this project we have few files that compose the project. First of all, we have two Java classes that call JNI code. First one, SigTerm.java, call JNI code that, in turn, call library compiled with “-Dexit” switch. Second, SigTermExtLib.java, calls the JNI code, that makes a call to some externaly provided function that we have no influence on (we don’t have sources).
Of course, in this case we have sources for both libs, but we will pretend that we haven’t compiled second library.
So, let’s take a look at the sources.
SigTerm class has just a one purpose – it calls callExitCode(). Inside this function, there is a call to exit (eventually). To see how the implementation was done, take a look at extLibCompiled.c and Makefile.

SigTermExtLib class, has slightly different purpose. It call JNI code that will break JVM. This is the result of the call to exit function that was compiled inside shared library we have no access to. We can’t compile this library, this is why it uses exit from the system implementation. In case of handling this kind of codes, take a look at recipeNo016.

In addition to Java classes we need few more things. First of all, we need the implementation of stopExit function that is used as a new handler for the exit function.

Note that we jump, somewhere, in the function stopExit. The point here is, that we have to make sure that each code that can potentially call exit function must use sigsetjmp before calling the code from the library. Otherwise, we will jump to some random place.

Next piece of the puzzles is the call to exit itself. So, we have two code. First one, calls the exit but is compiled without “-Dexit” switch

Second one, does the same thing. The only difference is that it will be compiled with “-Dexit” switch. Can you see that the code is the same code as previous one? At least in terms of the execution.

The last piece of the solution is the JNI code itself. Here, we have two calls. Both are wrapped with sigsetjmp call and both call very similar functions. The difference here is that first call will prevent exit from killing whole JVM, while second call, will actually close the whole JVM.

When we compile and execute the code, you can see the different behavior of JVM.

Now, take a look at execution itself.

As you can see, it is possible to prevent JNI library, contating call to exit, from killing the whole JVM.