...
The Equator Research challenges.


The Adaptive Computing challenges.

What common distributed infrastructure is required for the merging of physical and digital worlds?

This challenge focuses on an adaptive infrastructure to support "smart spaces" that span both the digital and the physical. Such spaces will naturally be composed of a heterogeneous mix of communications technologies, information services and devices, including collaborative virtual environments, streamed real-time media, distributed object services, mobile and wearable devices and tracking and sensing technologies. Applications that combine these technologies will require support from a highly flexible middleware that provides a uniform model of location and context, realised through a set of common services. A key feature of this middleware will be the need to adapt to dynamic changes in physical and virtual context. The Equator research will focus on three broad classes of common service.

  • Linking services:
    The transition from hypertext to hypermedia involved extending the notion of links between fragments of text to links between different media types. A similar shift is required to support applications that link digital objects to physical objects. Equator will explore the notion of these kinds of hyper-physical links. In so doing, it will address a number of issues. How are such links created? Are they permanent or temporary? Can they be fully symmetrical (for example how can interaction with virtual objects affect physical objects and vice versa)? In addressing these issues, Equator will build upon previous work at Southampton in link services for hypermedia and at Glasgow in distributed information models.

  • Communication services:
    New session management mechanisms will be required to dynamically connect users and objects as they move between different physical and virtual contexts. Management of quality of service will also be highly dynamic, especially as users swap devices, for example moving from a CAVE connected to a high-speed network to a wearable connected to a mobile network. Equator will extend previous work at Nottingham and UCL on communication services for collaborative virtual environments (e.g., the MASSIVE and COVEN projects), at Lancaster and UCL on communication services for mobile devices (e.g., the GUIDE project), and at Bristol on communication services for wearables.

  • Interface services:
    User interfaces will need to recognise and adapt to their context of use, for example recognising requirements for privacy and differing styles of interaction as users move between home, work and leisure. User interface services will also need to adapt themselves to run across a heterogeneous collection of devices. In addressing these issues, Equator will extend previous work at Lancaster on adaptive interface frameworks, at Bristol on interfaces for wearables and at UCL on guaranteed rendering virtual environments.
Equator will also integrate these services into a common framework, a common platform to provide run-time support for applications, as well as toolkits to support their development.

The understanding interaction challenge.


What new methods are required to support the merging of physical and digital worlds?

While there exists considerable human factors guidance for designing various kinds of interface (e.g., graphical displays, direct manipulation interfaces, multimedia systems and Web-sites), this has largely been aimed at representing information for desktop computer displays. We need to ask whether current approaches are appropriate to the design of interfaces where the relationship between the virtual and the physical extends beyond placing a monitor, keyboard and mouse in a room. In particular, do they remain appropriate when computing functionality becomes embedded into the surrounding environment, into specific physical objects or may be worn? The preliminary research of the Equator partners suggests that this is not the case, and that new conceptual models, theories and guidelines are needed for explaining and informing the design of these emerging technologies.
The understanding interaction challenge will establish new models of interactivity that explain how best to conceptualise virtual representations in relation to how they are to be visualised and interacted with in different physical environments. It will develop guidelines to inform the design of new applications, which involve different configurations of virtual representations and physical environments. It will consider when is it best to synthesise, integrate, juxtapose, link or keep distinct different virtual/physical representations? When is it desirable not to do so? What are the implications for designing applications and novel user experiences? What kinds of virtual representations are best to use and how do we decide what is the optimal way of interacting with them in different contexts? These questions will be answered through two strands of research:

  • Design methods:
    This strand will extend current techniques, adapted from sociology and psychology, to include new techniques from art and design. Techniques from sociology include the use of ethnography to inform co-operative systems design. This approach involves observing different social settings to identify subtle and tacit working practices, and is a theme of research at both Lancaster and Sussex. Techniques adapted from psychology have resulted in interface design guidelines to promote system usability, a theme of research at Nottingham and Sussex. Techniques from art and design include the use of 'cultural probes' to discover people's desires and aspirations and evoke the imagination of designers, a theme of research at the RCA. The long term goal of Equator will be to blend these into a common armoury of design methods to help design mixed reality applications that are usable, stimulating, engaging and account for actual social practice.

  • Evaluation methods:
    This strand will focus on how to evaluate applications that span the physical and digital in order to generate new insights for the technical research challenges. Experimental techniques adapted from psychology will be used to test specific design features and hypotheses in controlled laboratory settings. These will be complemented by ethnographically informed techniques adapted from sociology that observe the use of new technologies in use in everyday environments. Finally, techniques from art and design, such as the use of focus groups, audience discussions and critical review, will be used to gauge people's reactions to, and feelings about, new technologies.
Integration across these strands will involve considering the relationships between these different techniques and the extent to which they fit into an overall design and evaluation methodology. This will include an ongoing reflection onto the relationship between the disciplines in Equator (computer science, electronics, sociology, psychology and art and design) including documentation of success stories and difficulties from the user experience projects.

The embedded devices challenge.


Creating display and interaction technologies that are embedded in our surroundings

Embedded devices are those that are integrated into our surroundings. The combination of the physical and digital offers the possibility to integrate devices into the physical fabric of everyday environments in the form of public external displays, specialised sensors, video walls, ambient displays and active furniture. In the most extreme case of a fully immersive CAVE, the device actually becomes our entire surroundings.
The goal of this challenge is to understand how best to integrate devices into our surrounding environment and how these may be used. While we have some understanding about the physical construction of these devices, little is known about how these will be used and the appropriate software and hardware arrangements needed to support interaction with these devices. Examples of the issues to be addressed by this challenge include:

  • Group interaction with embedded devices:
    Nearly all physically environments may be shared and so embedded devices will be shared too. How can a group of users collectively interact with such a device? Equator will develop new interaction techniques to capture and analyse group gestures, for example, by extending video tracking techniques. In turn, group interaction with embedded devices suggests the need for group embodiments within digital worlds. Equator will explore how group activity can be represented so that other participants can make sense of it.

  • Embedded devices as boundaries:
    It is now a common idea that people can use projected displays to create a window into a virtual world. But what do the people in the virtual world see? Equator will consider how embedded devices can act as symmetrical boundaries between the physical and digital so that the occupants of each can see into the other. In the extreme, these boundaries will be made traversable so that people and objects can appear to physically pass from physical to digital and vice versa (extending the conventional idea of immersion where participants apparently enter a virtual world, but without leaving the physical world).
Integration across these ideas will involve developing common services and infrastructure for embedded displays that may be used across a range of advanced projection and immersive facilities within the project including UCL's CAVE and wide-screen multi-projector interfaces at Nottingham and Bristol.

The information appliances challenge.


Display and interaction technologies that are realised as dedicated information appliances

Information appliances are dedicated artefacts to fulfil a specific purpose (rather than being general-purpose devices). They combine a specifically designed physical artefact with computing and communications functionality. Information appliances typically aim to support aspects of everyday life seldom addressed by conventional computers, investigating new aesthetics and new functionality in support of a wider range of personal and social values.
Whereas some embedded devices are now commercially available, information appliances generally are not. A key aspect of this research challenge will therefore be to fabricate a number of information appliances for the user experience projects and to develop an infrastructure to make their fabrication easier as Equator progresses. In addition, the information appliances research challenge will address the following research issues.

  • Common toolkits for dedicated appliances:
    Although each information appliance is designed to be dedicated to a specific purpose, it is necessary to consider to what extent they can share a common infrastructure. We will establish a common base of hardware and software to support rapid design and prototyping of information appliances while still allowing each appliance to be purpose designed and built.

  • Context adaptive appliances:
    Information appliances may be moved between different contexts or the details of their current context may change. We will explore techniques that allow information appliances to adapt themselves to changes in context.

The wearable devices challenge.


Display and interaction technologies that are worn

Wearable devices are display and interaction technologies that are worn by users and that are therefore carried about with them as they move from place to place. They may provide users with continual and open access to digital worlds wherever they are and may also support activities in the open-air, outside of traditional office environments or the home.
Like information appliances, wearable technologies are currently at a very early stage of development, with one of the main centres for this research in the UK being Bristol. Consequently, much of this research challenge will be concerned with the core integration of wearable hardware, software and mobile networking so as to provide a resource of wearable computers for user experience projects. A core activity will be the continual redesign of wearable prototypes so that they are convenient to wear, to use, and robust to be used for prolonged time. In addition to providing a common resource for other projects, the wearable devices challenge will also address the following issues:

  • Tailoring wearables for different uses:
    We will consider the design and tailoring of wearables for disparate communities including children, emergence service workers and the elderly as well as the development of applications to promote different forms of interaction at different distances and across different communities.

  • Communication among wearables:
    We will explore how people using mobile devices and wearable computers can communicate with one another in social settings. Current wearable computers hardly communicate (with the exception being multi-user games on Nokia phones), but in a situation where the wearable computer is a commodity object, wearables will interact all the time. We will therefore establish concepts and models to support a community of wearables.
  • Interacting with wearables:
    There are as yet, no satisfactory models and techniques for interacting with wearables. We will explore usability issues such as comfort, reach and awkwardness. We will also consider the importance of the visibility of interactions with wearables to others. To what extent can other people make sense of a user's interaction with their wearable and how can we design this interaction so that it is socially appropriate and does not make the wearer appear foolish?

The EQUATOR Experience Projects
The EQUATOR Vision Statement

...