Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Enabling debug output from Java
04-09-2012, 01:20 PM
Post: #1
Enabling debug output from Java
I noticed a recent commit to provide support in the C# bindings for setting the debug level for log output. I'd like to provide a patch to implement this for Java, if that's OK.

To make debug logging work properly for Java, I would also need to include code to enable the LogOutput callback in the Java version of InitParams. Is there a reason why the Java version of InitParams has all the callbacks disabled? If not, I might be tempted to enable a few more of these. Smile
Find all posts by this user
04-09-2012, 01:38 PM
Post: #2
RE: Enabling debug output from Java
(04-09-2012 01:20 PM)simoncn Wrote:  I noticed a recent commit to provide support in the C# bindings for setting the debug level for log output. I'd like to provide a patch to implement this for Java, if that's OK.

Thanks, I'd be very happy to integrate any patches like this.

(04-09-2012 01:20 PM)simoncn Wrote:  To make debug logging work properly for Java, I would also need to include code to enable the LogOutput callback in the Java version of InitParams. Is there a reason why the Java version of InitParams has all the callbacks disabled? If not, I might be tempted to enable a few more of these. Smile

There's no good reason why the callbacks are disabled. We just hadn't got round to filling them in. As above, any patches in this area would be gratefully received!
Find all posts by this user
15-01-2013, 11:31 PM
Post: #3
RE: Enabling debug output from Java
I've been making good progress with this, and I should have a patch ready in the next couple of days. This will allow the Java bindings to set the logging level and also set callbacks for the log output and the fatal error handler.
Find all posts by this user
20-01-2013, 08:35 PM
Post: #4
RE: Enabling debug output from Java
I'm attaching a patch to the Java bindings that enables setting the debug level, setting a log output callback and setting a fatal error handler callback.

The patch includes two new files:

DebugLevel.java - a Java enumeration class that defines the supported debug levels
IMessageListener.java - a Java interface to be implemented by the log output callback and the fatal error handler callback

the patch also includes the following diff files:

Common.mak.diff
CpDeviceList.c.diff
CpProxy.c.diff
DvDevice.c.diff
DvDeviceStandard.c.diff
DvProvider.c.diff
InitParams.c.diff
InitParams.h.diff
InitParams.java.diff
Library.c.diff
Library.h.diff
Library.java.diff
Property.c.diff

The contents of these should be mostly self-explanatory, but there's one change that deserves further explanation.

JNI provides two APIs for attaching a native thread to the JVM:
1) AttachCurrentThread
2) AttachCurrentThreadAsDaemon

The Java bindings have been written with AttachCurrentThread calls. When using this API, every time a native thread calls Java code, the JVM creates a new Java Thread object for the duration of the call to the JVM. When the Java method returns, the native code calls the DetachCurrentThread API, which destroys the Java Thread object and detaches the native thread from the JVM.

This approach causes major problems when native threads call Java methods frequently, as they do when the log output callback is enabled. From the perspective of the Java code, every method call from native code appears to come from a different thread, so it's impossible for Java code to correlate log messages to the native threads that produced them. This also applies to other method calls from native code to Java code, although the impact of this is less severe.

This patch replaces the AttachCurrentThread calls with AttachCurrentThreadAsDaemon calls. It also removes the DetachCurrentThread calls. This means that a Java Thread object is created the first time a native thread calls a Java method. Any further calls from the same native thread to Java methods continue to use the same Java Thread object, which means that Java code is able to correctly identify these calls as all coming from the same native thread. This can be important for problem determination.

Please let me know if you have any questions about this change or about any other aspects of this patch.


Attached File(s)
.zip  debuglog.zip (Size: 5.63 KB / Downloads: 2)
Find all posts by this user
21-01-2013, 05:13 PM
Post: #5
RE: Enabling debug output from Java
(20-01-2013 08:35 PM)simoncn Wrote:  I'm attaching a patch to the Java bindings that enables setting the debug level, setting a log output callback and setting a fatal error handler callback.

Thanks again. The code should be available on github now.
Find all posts by this user
08-10-2013, 12:29 PM (This post was last modified: 08-10-2013 01:02 PM by simoncn.)
Post: #6
RE: Enabling debug output from Java
(20-01-2013 08:35 PM)simoncn Wrote:  This patch replaces the AttachCurrentThread calls with AttachCurrentThreadAsDaemon calls. It also removes the DetachCurrentThread calls. This means that a Java Thread object is created the first time a native thread calls a Java method. Any further calls from the same native thread to Java methods continue to use the same Java Thread object, which means that Java code is able to correctly identify these calls as all coming from the same native thread. This can be important for problem determination.

I've discovered a problem with this change. It causes problems when the ohNet library is closed, because the native ohNet threads are still attached to the JVM and holding resources. This is causing problems for MinimServer when the same Java process attempts to restart the ohNet library.

To fix this, it would be neccessary to arrange for each thread that was attached to the JVM by the Java bindings to issue a DetachCurrentThread JNI call just before it exits. This call must be issued from native code running in the thread that is to be detached. I can think of two approaches for doing this:

1) The Java bindings would register a pthreads cleanup handler (or the Windows equivalent) that would be called automatically by the OS when the thread exits. The cleanup code would be a routine in the Java bindings that calls DetachCurrentThread. This approach would need some changes in Os.c to support registration of thread cleanup handlers.

2) Add a new ohNet API to set a callback routine for thread termination. The callback would be made by Thread.cpp when a thread exits. The Java bindings would set this callback to a routine in the Java bindings that calls DetachCurrentThread.

Within approach 2), there are two sub-options:

2a) The callback would be thread-specific, so that it's only called for threads that have requested it.

2b) The callback would be global, so that it's called for all threads.

I think approach 2) is cleaner and simpler than approach 1). Sub-option 2a) provides more flexibility, but is more complex to implement as it would require additions to the current TLS data. Sub-option 2b) would be adequate to solve the current problem, because it would be OK for a thread to call DetachCurrentThread even if it wasn't attached previously.

If you are OK with this, I will produce a patch based on approach 2b). Any thoughts or suggestions would be much appreciated.

Edit: On reflection, 2a) isn't as hard to implement as I thought. The callback information could be stored as Thread class member data, so there shouldn't need to be any change to the TLS data. Given this, I think it would be worth going for 2a).
Find all posts by this user
08-10-2013, 03:00 PM
Post: #7
RE: Enabling debug output from Java
Approach 2(b) sounds good. It'd be great if you could propose a patch; I'm happy to do this (possibly more slowly) if you prefer.

Will it matter that we'll only be able to run a callback for threads which ohNet created? Any system threads which run managed code that choose to call into ohNet either won't be shutdown or will close in ways that are hidden from ohNet. The most common example of this is probably the thread which first initialises ohNet.
Find all posts by this user
08-10-2013, 04:14 PM
Post: #8
RE: Enabling debug output from Java
(08-10-2013 03:00 PM)simonc Wrote:  Approach 2(b) sounds good. It'd be great if you could propose a patch; I'm happy to do this (possibly more slowly) if you prefer.

Will it matter that we'll only be able to run a callback for threads which ohNet created? Any system threads which run managed code that choose to call into ohNet either won't be shutdown or will close in ways that are hidden from ohNet. The most common example of this is probably the thread which first initialises ohNet.

Thanks for the quick reply!

I'm interested in why you think 2(b) is better than 2(a). Presumably such an API would follow the pattern of other callbacks, such as the log output callback and the fatal error handler callback. Is it the similarity to existing ohNet APIs that makes you prefer this approach?

It's only the threads created by ohNet that need this callback to enable them to be detached from the JVM when the ohNet library is closed. For other threads, Java provides ways for applications to manage their lifecycle.
Find all posts by this user
08-10-2013, 04:25 PM
Post: #9
RE: Enabling debug output from Java
(08-10-2013 04:14 PM)simoncn Wrote:  I'm interested in why you think 2(b) is better than 2(a). Presumably such an API would follow the pattern of other callbacks, such as the log output callback and the fatal error handler callback. Is it the similarity to existing ohNet APIs that makes you prefer this approach?
It seems simpler/neater to me. I thought 2(a) was offering a custom callback per-thread. This would presumably have required us to offer callbacks informing clients whenever a thread was created plus the ability to run multiple callbacks for a single thread exit. This started to feel like a lot of work and I couldn't immediately think of a use case that'd require this complexity.

(08-10-2013 04:14 PM)simoncn Wrote:  It's only the threads created by ohNet that need this callback to enable them to be detached from the JVM when the ohNet library is closed. For other threads, Java provides ways for applications to manage their lifecycle.
Great, thanks for confirming this.
Find all posts by this user
08-10-2013, 04:55 PM
Post: #10
RE: Enabling debug output from Java
(08-10-2013 04:25 PM)simonc Wrote:  It seems simpler/neater to me. I thought 2(a) was offering a custom callback per-thread. This would presumably have required us to offer callbacks informing clients whenever a thread was created plus the ability to run multiple callbacks for a single thread exit. This started to feel like a lot of work and I couldn't immediately think of a use case that'd require this complexity.

I was thinking that each thread would be able to set a single callback (by an API call made on that thread) which would be specifically for thread termination only. The callback function pointer and void* data to be passed to the callback function would be held in the Thread object. The API that implements this function would use the TLS pointer to the Thread object to find the right Thread object to store the callback pointer and data. If a second call to this API were to be made from the same thread, the callback pointer and data would be replaced, rather than adding an additional callback.

The benefit of this approach would be that only those threads that need a termination callback (i.e., threads that have been attached to the JVM) would receive the callback. However, in the use case that is motivating this API for me, there should be no problem with the termination callback being stored as global data in InitParams and called for all threads. I'll go ahead and create a patch for this simpler global approach, as I should be able to do this more quickly than the thread-specific version.
Find all posts by this user


Forum Jump: