HybridDesktop

Hybrid Desktop is a new studying environment which aims to make good use of the advantages of both physical and electronic desktops by using the Mixed Reality technology. Students can use their smart phone and a simple VR headset to get involved in this new world where digital study materials are integrated with a physical desktop and physical interactions are supported. Students are able to view their electronic lecture notes, control the lecture notes by using a Leap Motion (a device which senses the hand and fingers information), annotate their documents, check their to-do list, etc.

This dissertation presents a literature review of existing Mixed Reality workspace applications and relevant research in Education, outlines the requirements and designs, and details the implementation of an Unity Project to build the Augmented Reality Scene, a C# program to recognize the gestures and convert Leap Motion positions to readable information, a socket program for transferring information between Leap Motion and iPhone, a web-based system for making electronic annotations and managing task lists, and a Python Web Crawler to get the latest weather information.

A study performed at the end, found that most students would like to study in such an environment in the future. They reflected that the Hybrid Desktop concept is promising, but also highlighted some current limitations. For example, the ges- ture controls should be more flexible, and a comfortable headset is needed, which can provide a broader and clearer view. Based on the feedbacks from university students, there is a discussion of some further improvements and potential implementations.