Johnny Chung Lee of Carnegie Mellon University came to the Stanford Electrical Engineering Computer Systems Colloquium to present “Interaction Techniques Using the Wii Remote”. Notes after the break.

As of December 2007, Nintendo has sold over 20 million Wii consoles worldwide. This significantly exceeds the number of tablet PCs used today according to even the most generous estimates of tablet PC sales. This makes the Nintendo Wii remote one of the most common input devices in the world.

4 Applications Publically Described so far:

1. Tracking Your Fingers with the Wiimote

• put IR emitters on your fingertips
• implement all the classic multitouch

2. Interactive Whiteboard

• Aim WIImote CCD obliquely at a projection
• Use an IR source in a Stylus
• Do a 4 point touch screen style calibration
• Multitouch capable
• Can be used on laptop screen
• Max 1024×768 @ 100 Hz  resolution

3. Head Tracking for Desktop VR

• Put IR emitters on a set of safety glasses
• Aim WIImote out from screen
• Use to track head and present a  3D environment through window of display
• Possible for user to hold 2nd WIImote, play games
• Studies indicate Motion Parallax is more important than stereoscopic vision
• Off-Center Perspective accurately represents motion parallax; part of opengl
• limitations
  • only works for 1 person
  • limited tracking volume
  • can’t touch objects
  • conflicting stereo depth cues

4. Projected Augmented Reality

•  high resolution and speed of Wiimote is well suited to application
• use to track projection surface
• can track a moving and shape changing surface
  • unfurling scroll
  • umbrella
  • paper fan

What’s Next?

• 3D motion tracking - extension of finger tracking, using 2 or more remotes
• Tracking with ID
• IR Glyphs - like RFID
• Laser Tag - Instrument each Wii remote with IR emitter
• Gesture Recognition

Conclusions

• Wii’s are cheap and ubiquitous, use them!
• Document your work and spread the word
  • Youtube vs Academic Papers

Highlights of Q&A

• Need to spend more time actually using these techniques for day to day productivity to refine
• Potential for full body tracking; hasn’t been implemented yet
• Nintendo used existing sensor chip; capable of tracking 4 points
• Difficult to integrate IR emitters for head tracking into headphones because of occlusion with turning head
• Some of the hardware is exclusively licensed to Nintendo
• Many possibilities for multiplexing - time, polarization, etc.

Also, see Johnny Lee’s Wiimote Forum