Handheld projector, projected interface consisting of on and off buttons, and target devices (lamps)
Controlling two lamps with a handheld projector

In everyday environments, we are increasingly surrounded by a growing number of electric devices. At the same time, many of these devices are shrinking in physical size. Both developments come along with their own challenges when it comes to controlling these devices. On the one hand, a plethora of devices imply that not all of them will be in physical reach when needed. Therefore, many people may prefer control at a distance for convenience. On the other hand, small devices are less able to accommodate control interfaces on their surfaces.

Driven by these challenges, we explore PICOntrol, a new interaction approach that employs an off-the-shelf pico projector as the handheld control device for users to directly operate various physical devices in the environment. To use PICOntrol, the user points the handheld projector directly at the device to be controlled (implicit and intuitive selection), and casts a projected graphical user interface (enlarged display capability) directly over a photo sensor unit integrated with the device (no divided attention) to perform a variety of GUI-style or gestural operations (enriched interactions) in order to control the device.

Related Publications

  • Schmidt, D., Molyneaux, D., and Cao, X.
    PICOntrol: Using a Handheld Projector for Direct Control of
    Physical Devices through Visible Light

    In Proceedings of UIST 2012, pp. 379-388 (best paper nominee)
    PDF| YouTube
  • Patent application. Cao, X., Schmidt, D., and Molyneaux, D. “Controlling a Device with Visible Light”

In the Press