HoloDesk is a novel interactive system combining an optical
see through display and Kinect camera to create the illusion
that users are directly interacting with 3D graphics. A virtual
image of a 3D scene is rendered through a half silvered mir-
ror and spatially aligned with the real-world for the viewer.
Users easily reach into an interaction volume displaying the
virtual image. This allows the user to literally get their hands
into the virtual display. A novel real-time algorithm for rep-
resenting hands and other physical objects, which are sensed
by the Kinect inside this volume, allows physically realistic
interaction between real and virtual 3D objects.