11/20/12 – goCAD!
This blog’s primary purpose is to document the status of our Kinect-based CAD program, called Gesture Operated CAD, or goCAD. We’ll explain the goals, methods, setbacks, and any resolutions of the program. Questions and advice are always encouraged. First, a video!
The goCAD project has two goals. The first is to create a stand-alone application used to view, manipulate, and assemble objects in 3D using hand gestures. Additionally, the program will use head-tracking to display the object in 3D. The second stage of the project will involve creating a plugin for Google Sketchup, allowing for advanced drawing applications, as well as the natural view control implemented in the stand-alone app.
The previous video shows the first generation prototype of the goCAD program. It uses Candescent hand tracking library and Microsoft’s SDK to control a cube rendered in DirectX10. The Candescent library has modules for multiple hand and finger objects. The library begins the hand detection routine by checking for pixels within a specified distance range from the sensor. It then creates clusters from those pixels contained within edges detected using (working on getting the method) . If the number of pixels in a cluster are within a threshold, it’s considered a hand. The “hand” edge is then traced in 2D using Graham’s scan to locate fingers. The process seems to work well enough for our demo. However, it does comes with some drawbacks…
First, the fingers are only traced after a 2D edge-map is created from the depth stream. This means, if the user tilts their hand perpendicular to the display, the fingers are lost. Second, the library only discriminates objects from hands based on size and whether they are in the specified range. Third, the library requires the sensor is run in “Near Mode,” which doesn’t allow for skeleton tracking. Finally, the entire depth stream is scanned, wasting resources. To compensate for this, the code pixelates the input, so that it is easier on the processor, decreasing effective resolution.