Java Tablet Events for Mac OS X
Java provides no support for digitizing tablets, such as those produced by Wacom and others. On Windows you can use JWinTab, but there's currently no solution for Mac OS X, and this makes us Mac users feel bad as Windows users laugh and kick sand in our face. This page shows one method for retrieving tablet event data, as well as a couple of failed ideas, and shows one way in which you might dispatch tablet events to listeners. This only works for Java 1.4+ on OS X. It's possible to do something different on 1.3, but why bother? There's downloadable source and Xcode project at the bottom of the page.
Disclaimer
The source code on this page is in the public domain. Basically, this means that you are free to do whatever you like with this code, including commercial use, but it's not my fault if your satellite/nuclear power station/missile system fails as a result.
Talking to the Tablet
On OS X, tablet data comes in via the event queue, either Carbon events for Carbon applications or NSEvents for Cocoa applications. We need to get hold of these events. Java on OS X is a hybrid of Carbon and Cocoa, so we could try some different tacks. Here are some things which don't work:
Carbon Events
Why not register a Carbon event handler for the tablet events using JNI? Unfortunately, this doesn't work. I know, I've tried it. I do the experiments so you don't have to. In Java 1.4, the menu bar is handled by Carbon while the windows are Cocoa, so what happens is that you can receive the events if the user clicks in the menu bar, but not in a window. Back to the drawing board on that one.
Cocoa Events in Java
Here's something which sounds like it might work, and even better - it involves no JNI! You can use the Java/Cocoa bridge to get hold of the Cocoa aplication object and ask it for the current event, like this:
import com.apple.cocoa.application.*; NSEvent event = NSApplication.sharedApplication().currentEvent();
This almost works, you can call this from your mouse event handler and get the tablet data from the NSEvent in pure Java. Unfortunately, it's not threadsafe and your application will eventually crash. We can only call these methods from Thread-0, not the AWT event thread.
Cocoa Events Using JNI
We're going to have to delve into JNI, I'm afraid. Somehow we need to get hold of the events that NSApplication is dispatching. It would be nice if there were a delegate, observer or listener or some such thing that we could add to the application to gt a peek at the events, but no such luck. In Cocoa, every event is dispatched by the sendEvent method of NSApplication. We need to subclass NSApplication and override the sendEvent method. At first sight this looks very nasty - how can we subclass NSApplication when we don't create the application instance?
Luckily Objective-C comes to the rescue. We can use the poseAsClass method in order to dynamically subclass an already-existing object. This gives us a handle onto the problem, we just need to write a Java class with a native method which overrides NSApplication's sendEvent method. In that method we can do whatever we like with the tablet data. If we're only interested in, say, the pen pressure, we could just stuff the pressure into a global variable and provide an accessor function for it. If we want to dispatch events back to Java, we can just call back into Java and post simulated events to the event queue or send them directly to listeners.
Update: Unfortunately, the poseAsClass method has been deprecated in later versions of Mac OS X and this method doesn't work any more. Luckily Objective-C comes to the rescue again. This involves something called "method swizzling" where we can exchange the implementation of one method for another. In Snow Leopard, there is built-in support for doing this, but in earlier versions we're still stuck. Luckily, Jonathan 'Wolf' Rentzsch has written JRSwizzle which fills the gap. The downloadable source code below has been updated to use this.
Enough theory. Here's the source for the first style - just making the pressure data available:
First the JNI implementation.
#include "TabletWrapper.h" #includestatic float pressure; @interface CustomApplication : NSApplication @end @implementation CustomApplication - (void) sendEvent:(NSEvent *)event { switch ( [event type] ) { case NSLeftMouseDown: case NSLeftMouseUp: case NSLeftMouseDragged: pressure = [event pressure]; break; default: break; } [super sendEvent: event]; } @end JNIEXPORT void JNICALL Java_TabletWrapper_startup(JNIEnv *env, jobject this) { [CustomApplication poseAsClass: [NSApplication class]]; } JNIEXPORT jfloat JNICALL Java_TabletWrapper_getPressure(JNIEnv *env, jobject this) { return pressure; }
and the matching Java:
public class TabletWrapper {
static {
System.loadLibrary("JNITablet");
}
native void startup();
native float getPressure();
}
Well, that's fairly painless and works well. Just call the getPressure() method from your mouse event handlers and all will be OK. You can obviously extend this method to provide other tablet data. Note that we get the pressure before dispatching the original tablet/mouse event so we're guaranteed to have a valid pressure before your application receives any mouse events. We don't, however have a guarantee that the pressure matches the current event due to threading delays. This may or may not be a problem for you depending on how precise your application needs to be - after any tablet data is better than none. However, we can do better by dispatching our own events. Read on.
Dispatching Events
It would be nice to receive tablet events directly rather than having to rely on mouse events. That way, we'll be able to handle tablet proximity events as well.
To dispatch events, we'll need to call back into Java. This isn't too hard. First we'll define the Java method we'll call when we get a tablet event, and an extra method to free resources when we've finished:
private void postEvent(
final int type,
final float x, final float y,
final int absoluteX, final int absoluteY, final int absoluteZ,
final int buttonMask,
final float pressure, final float rotation,
final float tiltX, final float tiltY,
final float tangentialPressure,
final float vendorDefined1,
final float vendorDefined2,
final float vendorDefined3
) {
}
public void finalize() {
shutdown();
}
native void startup();
native void shutdown();
And now the JNI C code:
#include#include "TabletWrapper.h" #include static JNIEnv *g_env; static jobject g_object; static jclass g_class; static jmethodID g_methodID; @interface CustomApplication : NSApplication @end @implementation CustomApplication - (void) sendEvent:(NSEvent *)event { switch ( [event type] ) { case NSLeftMouseDown: case NSLeftMouseUp: case NSLeftMouseDragged: if ( g_env ) { NSPoint tilt = [event tilt]; NSPoint location = [event locationInWindow]; (*g_env)->CallVoidMethod( g_env, g_object, g_methodID, [event type], location.x, location.y, [event absoluteX], [event absoluteY], [event absoluteZ], [event buttonMask], [event pressure], [event rotation], tilt.x, tilt.y, [event tangentialPressure], 0.0f, 0.0f, 0.0f ); } break; default: break; } [super sendEvent: event]; } @end JNIEXPORT void JNICALL Java_TabletWrapper_startup(JNIEnv *env, jobject this) { [CustomApplication poseAsClass: [NSApplication class]]; g_object = this; g_object = (*env)->NewGlobalRef( env, g_object ); g_class = (*env)->GetObjectClass( env, this ); g_class = (*env)->NewGlobalRef( env, g_class ); if ( g_class != (jclass)0 ) g_methodID = (*env)->GetMethodID( env, g_class, "postEvent", "(IFFIIIIFFFFFFFF)V" ); g_env = env; } JNIEXPORT void JNICALL Java_TabletWrapper_shutdown(JNIEnv *env, jobject this) { if ( g_object ) (*env)->DeleteGlobalRef( env, g_object ); if ( g_class ) (*env)->DeleteGlobalRef( env, g_class ); g_object = NULL; g_class = NULL; }
Now the Java postEvent method will be called whenever we receive a tablet event. We can do the same for tablet proximity events, and while we're at it, why not add in those other useful Mac events that we don't normally receive, such as application activated/deactivated? There's not space here, so I'll leave these as an exercise.
All we need to do now is to dispatch the events to our listeners. There are a couple of things we can do here: The first is to allow listeners to be added to our TabletWrapper class and just call the listener methods when we get an event. But be careful! We are running on some thread which is probably unknown to Java, so we should make sure to dispatch the events on the proper thread. We can do this using SwingUtilities.invokeLater():
SwingUtilities.invokeLater(
new Runnable() {
public void run() {
for ( Iterator it = listeners.iterator(); it.hasNext; ) {
TabletListener listener = (TabletListener)it.next();
listener.tabletEvent(
new TabletEvent( target,
type,
x, y,
absoluteX, absoluteY, absoluteZ,
buttonMask,
pressure, rotation,
tiltX, tiltY,
tangentialPressure,
vendorDefined1, vendorDefined2, vendorDefined3
)
);
}
}
}
);
I've missed out the definition of TabletEvent and TabletListener, because they're fairly obvious.
Better Dispatching
This is quite good, but it would be nice to have tablet events treated the same way as mouse events and dispatched to the component the tablet is over. This is very nearly possible, but not quite, as far as I can see. All you need to do is find the Window at (x,y) and call findComponentAt() on it to determine the Component to dispatch to, adjusting the tablet coordinates to the target component. The snag is with the first part of this. I'm not aware of any way to do it. The nearest thing is to get the focused window from KeyboardFocusManager, but this isn't valid on the activation click on a window. Here's the code anyway. Perhaps you can make it work:
// Find out which component should receive the event
Window w = KeyboardFocusManager.getCurrentKeyboardFocusManager().getFocusedWindow();
if ( w != null ) {
int ix = (int)x, iy = w.getHeight()-(int)y; // The Y coordinate is inverted in Cocoa
Component target = w.findComponentAt( ix, iy );
Component c = target;
// Adjust the tablet event coordinates
if ( c != w && c != null ) {
while ( c != w ) {
Point p = c.getLocation();
ix -= p.x;
iy -= p.y;
c = c.getParent();
}
}
// Dispatch the event as before, but to the component
}
The only trouble with this is that the TabletEvents are bypassing the EventQueue, it would be nice to send the events through the event queue and get them dispatched at the correct time. I haven't investigated this much. If you do, let me now and I'll add the information here.
Summary
This scheme for receiving tablet events works pretty well. There may be other ways to do this such as talking to the HID manager or using IOKit to talk directly to the tablet driver, but this mechanism is fairly high level and very simple so I prefer it.
Download
You can download the source code and an Xcode project for building the whole thing here. The code is released under the BSD 2-clause license so you can do anything you want with it. I'm happy for it to be used in any projects you happen to have. If you do something cool with it, please let me know.