Simplifying the programming of intelligent environments

In the future, computers will be virtually everywhere: carried by everyone and integrated into the environment. The increased computation and communication capabilities will enable intelligent environments that react to occupants through automated decision-making. Devices (sensors and actuators) are the key to making intelligent environments a reality. We believe that devices must be made more approachable for average users. Existing approaches to application development for intelligent environments require detailed knowledge about devices and their low-leveling programming interfaces, which greatly limits the number of potential users. Instead of limiting users, we must enable everyone to program the devices around them. Intelligent environments will not be commonplace until average people can set up and manage the hardware and software necessary for their personalized applications.

In simplifying the programming of intelligent environments, we first made sensors and actuators accessible to average programmers then extended our work to end-users. We term the former contribution Sensor Enablement for Average Programmers (SEAP); the latter work is Sensor Enablement for End-Users (SEEU). In our experience, devices’ disparate, niche programming languages and communication protocols presented great difficulty in developing intelligent environments. To ease the development effort for average programmers, we abstracted and standardized complex sensor and actuator interactions, allowing users to instead think in terms of well-understood web applications. Users have said that SEAP is easy-to-use and exciting. But what about average people, end-users?

We found that end-users are incredibly interested in intelligent environments. By engaging end-users we can create intelligent environments even faster and allow domain experts to tailor their environment. This dissertation’s second contribution, Sensor Enablement for End-Users (SEEU) provides a visual programming interface that allows users to create personalized automated behaviors given available devices and data. We performed several user studies to uncover people’s desires for intelligent environments and determine the best interface for managing an intelligent environment. SEEU combines an intuitive interface with the power and flexibility of SEAP. SEEU is a usable end-user programming framework that allows average people to create useful applications for their intelligent environments.

With SEEU and SEAP, we simplified the development of intelligent environments, reducing barriers to adoption of emerging sensing and actuation technologies. We demonstrated the feasability with a series of user studies.