Finally, something concrete to be released. I've created my own Kinect Unity plugin and some utilities, and this prototype will act as a starting point for Leo Holsti, a circus artist / programmer who will join our group in June. A big thanks to Aki Martikainen for the comments on the visuals!
Lessons learned from the cartwheel training prototype:
- Clear numerical feedback indeed seems to motivate training, but as always, gamification can lead to exploitation. The simple flatness scoring can be gamed to some extend by keeping one's feet tucked in and not straight. The straightness of feet should be included in the score, but at the moment it's difficult to implement because the OpenNI/NITE tracking breaks when both feet are off the ground. The flatness scoring works without the tracker because it's computed directly from the user pixels (fortunately, the OpenNI/NITE user blob extraction doesn't break during a cartwheel.) I hope I have time to implement my own tracker or find some other tracker with an easy to use pipeline for defining the range of tracked moves. Totally unlimited tracking/motion capture using a single depth sensor is still a difficult problem even offline, let alone in real time.
- Floor graphics and shadows are more clear than separate windows/views and can be perceived while also watching one's movement. We are of course tuned to sense our surroundings based on shadow and lighting. Projecting graphics on the floor in the real world would probably be even better than the virtual floor graphics, but they would make downloading and trying out the software much more tedious. I tried having a bird's eye camera as a separate window, but it felt confusing, especially since the user mesh is actually just a thin layer lacking the back of the body.
- I also tried making the experience more gamelike by having walls between which you must roll with the cartwheel, however the lack of tactile feedback and stereo vision made it almost impossible to estimate one's precise position in relation to the walls.
You can download the PC prototype (CartwheelTrainer.zip) from my Google Drive. If you don't have Kinect stuff installed, also run the OpenNI, NITE, sensor and SensorKinect installers (.msi) from the same folder. I admit it's quite a hassle, but I don't have the time to learn to make a proper installer right now.
Edit 5th June 2012: embedded the video mirrors video as well.