Community
Participate
Working Groups
Windows 7 and Snow Leopard (10.6) have API now for multi-touch events. We should come up with some kind of SWT API to receive these events.
*** Bug 293307 has been marked as a duplicate of this bug. ***
Related to this, I created this project a while ago: http://eclipsemultitch.sourceforge.net So far, the plug-in has only a low-level API (Touch, TouchListener), and recognized gestures are directly translated into actions or commands. It would be easily possible to add event notification for higher level events such as "three-finger swipe down", etc. The code is already licensed under EPL, so I'm happy to contribute it to Eclipse if there is interest.
Not for 3.6 at this point, as we are past API freeze.
+1 for this in 3.7
*** Bug 315735 has been marked as a duplicate of this bug. ***
(In reply to comment #2) > The code is already licensed under EPL, so I'm happy to contribute it to > Eclipse if there is interest. Is the code available from sourceforge? I couldn't find a download link. I'm investigating possible APIs for this, and wouldn't mind seeing what you've done so far.
(In reply to comment #6) > Is the code available from sourceforge? I couldn't find a download link. I'm > investigating possible APIs for this, and wouldn't mind seeing what you've done > so far. Code is accessible from: http://sourceforge.net/projects/eclipsemultitch/develop you can also browse it from:http://eclipsemultitch.git.sourceforge.net/git/gitweb-index.cgi
The main API so far is in package net.sf.multitouch.api (http://eclipsemultitch.git.sourceforge.net/git/gitweb.cgi?p=eclipsemultitch/eclipsemultitch;a=tree;f=net.sf.eclipsemultitouch/src/java/net/sf/eclipsemultitouch/api). It's more of a low-level API for receiving raw data from a Multi-Touch device, not what we'd probably would want to expose at application level. I'm currently working on more flexible gesture recognition, though. The way I envision it, the gesture recognition would use something like net.sf.multitouch.api (which itself would not be Eclipse API) and expose some higher-level "GestureListener" style of API. If you look at net.sf.eclipsemultitouch.MultiTouchListener (the actual listener implementation that adds Multi-Touch functionalities to Eclipse), you can get an idea what sorts of higher-level events we might want to support (threeFingerSwipeLeft, threeFingerSwipeRight, threeFingerControlClick, etc.). We'd probably start out with a selected set of standard gestures, but eventually it would be nice if users could hook up their own custom "gesture recognizers".
(In reply to comment #8) > The way I envision it, the gesture recognition would use something like > net.sf.multitouch.api (which itself would not be Eclipse API) and expose some > higher-level "GestureListener" style of API. If you look at > net.sf.eclipsemultitouch.MultiTouchListener (the actual listener implementation > that adds Multi-Touch functionalities to Eclipse), you can get an idea what > sorts of higher-level events we might want to support (threeFingerSwipeLeft, > threeFingerSwipeRight, threeFingerControlClick, etc.). We'd probably start out > with a selected set of standard gestures, but eventually it would be nice if > users could hook up their own custom "gesture recognizers". Gestures are interesting. We definitely want to handle all of the system-supplied gestures, but I don't know if we want programmatically defined gestures. I want to read up on that a bit more. I've set up a wiki page at http://wiki.eclipse.org/SWT/Multitouch_support so we can gather the discussion in one place -- design by bugzilla is painful. Let's keep the discussion going there.Thanks!
*** Bug 303208 has been marked as a duplicate of this bug. ***
The eclipsemultitch projects implements it's own low level handling of the touches, it would me much easier to just implement NSEvent Apple's API like Firefox did even tho it is more limited it guarantees compatibility http://developer.apple.com/mac/library/documentation/cocoa/reference/ApplicationKit/Classes/NSEvent_Class/Reference/Reference.html "Support for touch and gesture events masks have been added to NSEvent in Mac OS X v10.6. Magnify (pinch), swipe, and rotate masks are supported, as are more generic gesture masks. Using touchesMatchingPhase:inView: method a view can get all of the touch events associated with a gesture without overriding the individual touch responder methods defined in NSResponder."
It's possible that NSEvent might be the better choice for implementing this feature on the Mac. At any rate, we need to make sure that the multi-touch support is designed in such a fashion that different implementations are easily interchangeable. We might need an internal API that could then be implemented by different solutions based on different low-level APIs.
I am working to get multi-touch support to eSWT on mobile devices. Ideally we will use the same APIs in eSWT that SWT implements. Looking at the wiki page I am OK with the direction that this is taking, couple of questions though. Is touch events planned to be available for all Controls all the time? What about a style flag to enable them for a particular Control? Has there been any work started related to this. Some code that I can look at and work on it on eSWT context?
(In reply to comment #13) > I am working to get multi-touch support to eSWT on mobile devices. Ideally we > will use the same APIs in eSWT that SWT implements. Looking at the wiki page I > am OK with the direction that this is taking, couple of questions though. > > Is touch events planned to be available for all Controls all the time? What > about a style flag to enable them for a particular Control? Currently I'm planning to add a method setTouchEventsEnabled() that turns them on or off for a given control. > Has there been any work started related to this. Some code that I can look at > and work on it on eSWT context? I have a working implementation up and running on Mac OS X, and I wrote a Win32 implementation, but I'm waiting for the go-ahead to get hardware to test for the Win32 support. Unfortunately there are no trackpads that do true Windows 7-style gestures; you need a monitor. Fortunately, the API appears to be holding up pretty well on Mac and Windows, so I'll try to post my work in progress later today.
Created attachment 179350 [details] work in progress Work in progress with the Mac implementation.
I will work to implement this on Qt. Will let you know how that goes. Did you plan an API for checking if the multi touch capability is available? Applications may want to adjust their behavior for multi touch.
(In reply to comment #16) > Did you plan an API for checking if the multi touch capability is available? > Applications may want to adjust their behavior for multi touch. I hadn't but that does seem like a good idea. Right now this will only work on Cocoa (10.5 and 10.6) and Windows 7. I would like to implement a full-blown capabilities API for things like this so we don't have to rely on things like checking to see if an API succeeds or not, which isn't sustainable.
I saw this post. It seems there is some work to get multi touch API on Ubuntu 10.10 (which is suppose to be released soon). We should see what is the state of the API. http://www.markshuttleworth.com/archives/455 We should also take a look at: http://www.tuio.org/
For Windows 7, this project looks like it will get me a bit further along without a touch monitor. I was able to install it today on my BootCamp partition and it's working well. http://multitouchvista.codeplex.com/ The monitor has been requested -- I'm waiting for the approvals to get through the system.
(In reply to comment #19) > For Windows 7, this project looks like it will get me a bit further along > without a touch monitor. I was able to install it today on my BootCamp > partition and it's working well. > > http://multitouchvista.codeplex.com/ hmm.. It was working well until I rebooted, and now it's not working again. So much for progress on this today.
This presentation describes the proposed GTK api for touch events: http://www.slideshare.net/fscons/multitouching-your-apps but says that gestures are still to come. Canonical has a gesture interface right now, described here: https://edge.launchpad.net/canonical-multitouch/utouch-geis It will map well to the current proposed SWT API, but of course, would only work on Ubuntu, assuming I'm reading it properly.
Created attachment 184198 [details] work in progress - common files Latest update: -- TouchState class rewritten -- Touch event replaces TouchesBegin, TouchesMoved, TouchesEnded, etc.
Created attachment 184199 [details] work in progress - Mac OS X Mac implementation of touch and gestures
Created attachment 184200 [details] work in progress - Win32 Win32 implementation
Latest work based on HEAD as of 11/30/2010. Here's what works: -- Magnify, rotate, and pan gestures on Win32 and Mac -- Swipe gesture on Mac only -- Touch events on Win32 and Mac - Primary touch, screen X and Y, normalized X and Y on Win32 and Mac. - Mac implements 'screenX' and 'screenY' for a touch as relative to the cursor location and the primary touch point on the trackpad. - Win32 implements 'normalizedX' and 'normalizedY' as a percentage of the screen location relative to the Control's monitor. So, a touch at 500, 500 on a 1024x768 monitor would have a normalizedX of 0.48828 and normalizedY of 0.65104 - The equivalent of Windows' 2-fingered pan gesture on the Mac is a scroll-wheel operation, so when a gesture-begin is detected and subsequent scroll wheel events occur, I treat it as a pan gesture and fill in the xDirection and yDirection fields as appropriate. Still planned: -- Swipe gesture on Win32 -- Linux support (Community contributions greatly appreciated) Not planned: -- Contact area of the touch. This is provided in the TOUCHPOINT structure on Windows 7, but you have to hack it in on Mac OS X via some reverse engineering of a private framework. It's also not clear if we can get this information in Linux.
If you are following this bug, presumably you have an application in mind that would take advantage of the new API. If so, please take a moment to review the API and make sure it will meet your needs. We are planning to check this in for the M5 milestone, so there's time to adjust the API before the freeze.
Created attachment 184827 [details] work in progress - Win32 Win32 update: rotation now works; I was calling SetGestureConfig with the wrong values. Swipe is implemented as a flick -- this is temporary and may not be in the final version. Touch events now report if they are from a direct or indirect input source.
Created attachment 184828 [details] work in progress - common files New constants for touch input source types.
Created attachment 184829 [details] work in progress - Mac OS X -- Lock rotate and magnify gestures so they don 't interleave. Once a rotate or magnify gesture starts, events of the other type will not be sent until the first gesture is finished. -- Swipe and pan directions are now consistent. Left or up is negative, down or right is positive.
Created attachment 184914 [details] work in progress - Win32 Swipe on Win32 implemented with tablet flicks. Flick up, down, left, or right generates a SWT.Gesture with GESTURE_SWIPE detail and xDirection and yDirection set accordingly. Added Display.isTouchEnabled(), which returns true if a touch device is attached, ready, and generating multi-touch events. On Mac this always returns true if the OS version is greater than 10.5.3 (when gesture support was added to MacBooks.)
Created attachment 184915 [details] work in progress - Mac OS X Last change to Mac to get back in sync with Win32. With this change I'm feature complete on Win32 and Mac.
Created attachment 185101 [details] Gestures only for Win32 Win32 implementation of gestures. This also has the PI calls for touch, but we're working out details in the API. Felipe, I cleaned up the natives so it builds with the Windows Server 2003 SDK -- can you please apply it and make sure it builds for you before I check it in?
Created attachment 185155 [details] Gestures only for Win32 Cleanup of previous attempt, with WinCE defines.
Created attachment 185192 [details] Gestures only for Win32 Changes after code review. GestureBegin and GestureEnd were removed and moved to the event detail field. GESTURE_BEGIN and GESTURE_END are now set when GESTUREINFO reports it in the flags.
Created attachment 185582 [details] Gestures only for Win32 Should be the last iteration for Win32 gestures. Mac patch shortly.
Created attachment 185586 [details] Gestures only for Mac
Created attachment 185587 [details] Common gesture code Common bits for gestures. Use this plus the Mac-only attachment for the Mac. Win32 has the platform code plus the common bits.
Created attachment 185600 [details] Common gesture code Forgot to take out EndGesture, and the touch API isn't ready yet.
Created attachment 185601 [details] Gestures only for Mac Left behind some touch API in Display.
Gesture code checked in for all platforms (stubs on all but Win32 and Mac OS X.) Touch is being reviewed; we're looking into a new object to represent the input source.
Created attachment 186057 [details] Touch code for Mac Latest touch support for Mac diff. Also has a bug fix or two for gestures.
Created attachment 186058 [details] Touch code for Win32 Touch for Win32. Has small gesture fixes as well.
Created attachment 186059 [details] Touch common code TouchEvent, TouchAdapter, TouchListener, plus touch state-related classes.
We should not have the adapter classes (TouchAdapter and GestureAdapter). Both listener have only on method, the adapter is not useful.
Created attachment 186448 [details] Touch common code
Created attachment 186449 [details] Touch code for Win32
Created attachment 186450 [details] Touch code for Mac
Some comments: - Do not forget to delete the GestureAdpter class. - TouchEvent.touches need java doc - Sorry I am changing my mind. Let's add back the primary boolean in Touch and remove TOUCHSTATE_PRIMARY. The state represents what changed in a given Touch from the last TouchEvent. So having the primary state would make the application code harder because it would have to mask TOUCHSTATE_PRIMARY out all the time to determine what changed. - We should not have the TOUCHSTATE_STATIONARY constant at all. Without TOUCHSTATE_PRIMARY, it is easy to test state==0 to determine that there is no change in the state of the Touch since the last TouchEvent. This info can be added in the java doc of Touch.state. - Control.setTouchEventsEnabled() on windows does not have the @since 3.7 tag - I believe the values Touch.x/y should not be in screen coordinates. They should be in touch source coordinates. On Windows, the current patch is correct because the touch source is the screen. On Mac, the x,y would be calculated as follows: NSPoint normalizedPos = touch.normalizedPosition(); double normalizedX = normalizedPos.x; double normalizedY = 1 - normalizedPos.y; int x = (int) (normalizedX * deviceSize.width); int y = (int) (normalizedY * deviceSize.height); If the application needs screenX,screenY, it can calculate it using the device x,y and the pointer position when the first touch (primary) happened. - The constructor of TouchSource should not be public. Clients are not allowed to create these objects. - TouchSource should not have public fields because anyone could change their values and this would affect future touch events since we are always returning the same object (hashtable). We should add getters and remove the public modifiers of the fields. TouchSource.getBounds() TouchSource.isDirect() - Touch does not need serialVersionUID since it is not Serializable (events are). - We need the stubs for the other platforms
No argument on the other comments. > - I believe the values Touch.x/y should not be in screen coordinates. They > should be in touch source coordinates. On Windows, the current patch is correct > because the touch source is the screen. On Mac, the x,y would be calculated as > follows: > > NSPoint normalizedPos = touch.normalizedPosition(); > double normalizedX = normalizedPos.x; > double normalizedY = 1 - normalizedPos.y; > int x = (int) (normalizedX * deviceSize.width); > int y = (int) (normalizedY * deviceSize.height); > > If the application needs screenX,screenY, it can calculate it using the device > x,y and the pointer position when the first touch (primary) happened. Actually, the notion of coming up with a screen x and y value on the Mac always felt like a hack to me. It was an idea I saw in Qt, and while I understood it how it was calculated I wasn't sure how you would go about using it. You really have no need for it on the Mac -- deltas are what matter. Making x and y be touch device coordinates makes more sense. It's consistent and works on all platforms. I'm building a test app right now with the API and haven't found the need to worry about the absolute position of the touch yet.
Created attachment 186761 [details] Touch code for Mac Hopefully final Mac touch code
Created attachment 186762 [details] Touch code for Win32
Created attachment 186763 [details] Touch stubs Stubs for other platforms.
Created attachment 186764 [details] Touch common code Hopefully last common changes
At long last, done! Touch and gesture API is now in HEAD > 20110117.
.
Is it possible to simulate multi-touch events/gestures via display.postEvent(...)? I can't see any changes on that method.
(In reply to comment #56) > Is it possible to simulate multi-touch events/gestures via > display.postEvent(...)? I can't see any changes on that method. No, not with the current API. To be honest, we hadn't considered the possibility of sending them via Display.post(), but I suspect it's not possible on Windows 7 and I'm pretty sure it can't be done right now on OS X. There are no low-level equivalents for touch events in CoreGraphics in OS X 10.6.
Filed bug 482018 for the missing implementation on GTK. Added a warning to the APIs: http://git.eclipse.org/c/platform/eclipse.platform.swt.git/commit/?id=638f934232d5f4888b370b7f5d90cae4c955117c
