Abstract

This paper describes an ongoing project in the UK to establish a framework for distributed virtual reality applications. The paper describes the aims of the project, the three applications (physics teaching, collaborative information browsing, and geometric design), the methodology for evaluation, and the network infrastructure.

Keywords

Collaborative virtual environments, distributed virtual environments, virtual reality, virtual classroom, information retrieval, geometrical modeling.

1. Introduction

Collaborative Virtual Environments (CVEs) involve the use of distributed virtual reality to support the work (and play) of groups of people. The concept of CVEs has emerged from two threads of research. First, the virtual reality community has begun to explore multi-participant VR, either as an extension to single user systems which exploit distributed processing architectures or for supporting specific collaborative activities such as multi-player games and battle simulations. Second, the Computer Supported Cooperative Work community has been developing notions of shared space through technology such as media spaces which have raised issues of social interaction and mutual awareness in computer systems.

This paper describes the work of the project known as DEVRL, Distributed Extensible Virtual Reality Laboratory. The major purpose of this project is to investigate the formation of a distributed virtual reality (VR) laboratory to support research into CVEs. The principle objectives of the distributed laboratory are :-

• the development of a distributed hardware, software and networking infrastructure for constructing and evaluating CVEs over wide area networks;
  
• to demonstrate the potential of a variety of CVE applications;
  
• to conduct experiments with three initial CVE applications in order to explore the issue of multi-participant presence and the underlying network requirements of CVEs and also the relationship between the two (i.e. how does network performance influence the sense of multi-participant presence).

The paper presents a snapshot of DEVRL one year into the project and discusses progress towards each of these three objectives. Section two provides a brief overview of the current DEVRL infrastructure. Section three then describes each of our three demonstration applications: the virtual classroom, collaborative information retrieval and shared geometric design. Finally, section four introduces the issues of multi-participant presence and network requirements and outlines proposed experimental work.

2. The DEVRL infrastructure

We begin with a brief overview of the DEVRL infrastructure. At the core of DEVRL are three universities: University College London, The University of Nottingham and Lancaster University, each with an existing local VR laboratory. These three sites are separated by distances of several hundred kilometres. They are all connected via the UK's SuperJANET research network. At present, SuperJANET provides bandwidths of up to 10 Mbs-1 using the SMDS protocol, although migration to ATM is planned within the next eighteen months, providing bandwidths of 55 Mbs-1 and upwards.

Between them, the four sites provide access to a number of VR workstations including a ProVision100 VPX, one SGI ONYX RE2 and several Indigos and Indies. At present, it is possible to conduct experiments with over ten simultaneous participants spread across the three sites. The sites support immersive access (enabling three participant immersive applications to be run over the wide area) and one supports a projection interface. The software infrastructure is provided by a number of VR platforms including Division's dVS, DIVE and MASSIVE.

Three CVE applications are currently under development (see below). However, some initial testing of the infrastructure has been carried out using the MASSIVE VR-teleconferencing system. MASSIVE has been used to hold a number of project meetings in distributed VR with several participants engaged in simultaneous graphical, audio and textual interaction. Some of these meetings have included participants from other non-core sites; the most notable having spanned five organisations in three countries (The UK, Sweden and Germany). The results of these early experiments with MASSIVE have been reported in [GREEN 95].

3. The DEVRL applications

Next, we describe the three CVE applications which are being developed for DEVRL. From the outset the DEVRL project has aimed to be informed by the practical difficulties encountered in constructing applications. We can characterise such applications on three dimensions:

• The number of participants simultaneously engaged in the application as low (less than 10), medium (in the 10s) or high (in the 100s or more).
  
• The complexity of the objects and their behaviours as low (representation of relatively static data), medium (representation of objects which change in response to a user interaction) or high (representation of objects which can dynamically change of their own volition).
  
• The degree of interaction between participants as low (they see each other and can exchange information), medium (as low, but they can be engaged in complex activities that must be visible to others), and high (as medium, but users may engage in synchronised activities relating to high complexity objects in order to achieve a common task).

DEVRL is constructing three different CVE applications. These are the virtual classroom, a collaborative simulation for learning physics; collaborative information retrieval, a 3-D information visualisation which supports data sharing and chance encounters with other people; and geometric modelling, which supports collaborative and interactive design of complex geometric shapes. We may then approximately characterise these three applications as shown in Table 1. We now describe each of the three applications in turn.

3.1. The Virtual Classroom

The virtual classroom provides access to a number of interactive simulations of basic physical laws. A major advantage of virtual reality based simulation is that it is possible to support physical experiments which cannot be readily reproduced in real classrooms, such as the change in gravitational force exhibited by an object when its mass in changed. Another advantage is the possibility of providing users with viewpoints that are not normally possible in the real world, such as viewing the path of a projectile from the projectile's point of view. Each simulation is different in terms of both its functional and intended cooperative semantics. Physical simulations currently under investigation include gravitational, linear momentum and rotational momentum based experiments. The following paragraphs briefly describe the two of these which have been implemented to date, both as applications of DIVE.

Gravitational simulation. A projectile application exists where a virtual cannon fires a virtual cannon ball into free space. The cannon ball is acted upon by a simulated uniform gravitational field, which pulls the flying ball down as it travels. During its flight the cannonball leaves a trail denoting its path. Users may alter the initial velocity of the cannon ball and the angle of elevation of the cannon from which it is fired. The experimental task involves two participants who must co-operate to hit a target using the cannon. One user may alter the angle of the cannon's barrel and the initial speed of the cannon ball, the target is obscured from this user's view by a large wall, which the cannon ball must clear. The other is 'strapped' to the cannon ball, but may freely move around the cannonball to obtain different perspectives about it, while the ball (and the user) is (are) in flight. It is the task of the moving user to tell the controlling user how far away from the target the cannonball landed.

Rotational momentum simulation. A 3D pivot application allows a number of spheres with differing mass to be moved around a hinged plane. This plane rotates about its centre in the X and Z dimensions. The plane automatically pivots to represent the sum of the moments exerted by each of the masses placed onto it. The aim of the experiment is to balance the plane so that it is flat. This application may be used by any number of participants in many scenarios. For example, each participating user may only move their allocated object and must work co-operatively to balance the plane.

3.2. Collaborative Information Retrieval

Our second application involves the construction of a shared 3-D information visualisation to allow users to browse, search and share on-line document repositories. Given the rapid spread of the World Wide Web, coupled with the recent emergence of the Virtual Reality Modelling Language (VRML), this application is being constructed as a front end to WWW. However, unlike VRML which currently only supports single users navigating relatively static 3-D scenes, our application provides a number of interactive and multi-user visualisations. At the time of writing, the following components have been developed as applications of the DIVE system with embedded links into the Web.

Map tool. The map tool supports browsing of the WWW through the construction of 3-D graphs of a given region of the Web as defined by a starting node and an adjacency distance (i.e. a radius from this node expressed in terms of a number of links). The tool explores the Web within the defined region and then draws a 3-D graph using the Force Directed Placement algorithm [BENF 95]. Users may then navigate the resulting graph, selecting nodes in order to see summary details of the contents or further selecting them in order to launch Mosaic.

Search tool. This tool is based on the previously reported VR-VIBE visualisation [BENF 95b] and supports interactive searching of a document store through the manipulation and comparison of multiple search queries. A number of queries can be defined each consisting of several text keywords. These are positioned in a virtual space to form a spatial framework. Document icons are positioned within this framework according to the strengths of their relative attractions to each query (i.e. the more strongly an individual document matches an individual query, the closer it is placed to it). The size and shade of document icons also shows their overall attraction to all of the queries. Users may dynamically interact with the visualisation in a number of ways: selecting documents displays summary details or launches Mosaic to view the document; raising a relevance filter removes all documents whose overall score falls blow a threshold value from the display; grabbing and dropping queries dynamically deforms the space; switching queries on and off also changes the space and, finally, new queries may be defined dynamically. As with the map tool, the visualisation in DIVE provides links for retrieving actual WWW documents.

Awareness and communication support. In addition to DIVE's standard multi-user facilities , we have introduced a number of further communication mechanisms. First, both visualisations represent the presence of non-VR users as they wander across WWW information. Thus, a Mosaic user who happens to be accessing some of the pages that appear in either the map and search tools will be shown as a simple embodiment located next to the relevant document icon and their changes in location will be animated as they wander over the pages being visualised. Second, additional mechanisms are provided to request meetings with other people of to send them email. For example, on coming across some interesting information, it is possible to invite its author into the visualisation in order to discuss it as part of a virtual meeting.

3.3. Geometric modelling

Out third application builds on previous work in geometrical modelling in VR for single participants [SLAT 94/5]. A single designer has the problem of constructing initial shapes, modifying them and seaming them together. In the context of an environment shared by several designers, each may design a part of the final product, and then merge the parts together. Designers and clients may evaluate the product and engage in collaborative modification of the combined shape.

The underlying model uses a new method for deformable B-Splines based on minimising an energy functional [VASS 96]. This allows the application of forces to deform the shape very precisely and rapidly. Our specific approach is based on the notion of "body centred interaction" [SLAT 94a]. This builds on the notion that the match between sensory data and proprioception enhances the sense of personal presence. Therefore actions are based on appropriate mobilisations of the participant's whole body, rather than on interactive techniques borrowed from 2D display systems, or alternatively, a large number of individual hand gestures. This is based on the belief that immersive systems require their own repertoire of interaction techniques, and a new interaction paradigm.

In a multi-participant environment there are difficult problems to overcome - if two designers have each grabbed a corner of a shape, does this signify a contest for control of the shape or a desire for them to simultaneously stretch (or even tear) it? At the time of writing single designers may create shapes which may be observed by others, but the collaborative aspects are not yet implemented. This application is discussed in the companion paper [USOH 96].

3.4. DEVRL Town

In order to promote awareness of our work and research into CVEs in general, we are constructing a project wide virtual environment called DEVRL Town. Eventually, several versions of DEVRL Town will be realised in DIVE, dVS, MASSIVE and even VRML (at least a limited single user version for the latter). DEVRL Town is obviously based on the metaphor of a virtual town and will provide a general source of project related information as well as a common project entry point for accessing the applications (as buildings within the town). We wish to encourage other researchers and projects to establish their own presence in DEVRL Town.

4. Experimental work

So far, we have described the DEVRL infrastructure and applications. We conclude the paper by previewing the experimental work to be carried out in the later stages of the project. Clearly, there are no results to report at present. Instead, we concentrate of a detailed description of the issues to be explored and the underlying theory that will be driving this work. There are three components to our experimentation:

(1) developing and validating a theory of multi-participant presence - i.e. understanding the factors which affect people's sense of shared presence.

(2) developing and validating a model of network performance - i.e. understanding the kinds of network traffic generated by our applications and, conversely, predicting the effects of bandwidth and latency limitations on application performance.

(3) exploring the relationship between (1) and (2). More specifically, understanding how network and hence system performance affect notions of shared presence and also how users' actions (presumably influenced by the sense of shared presence) affect the underlying system performance.

The following sections touch on each of these issues in turn.

4.1. Multi-participant presence

First we consider the notion of presence as applied to CVEs .

(a) Categories of Presence

In a shared virtual environment there are two related but conceptually different forms of presence: personal presence and shared presence. The first relates to the sense of "being there" in the VE, and has been explored in [HEET 92, HELD 92, LOOM 92, SHER 92, BARF 93, SLAT 94b]. Personal presence itself has two manifestations: subjective presence relating to the individual's state of mind, which can be elicited to some extent through questionnaires, and interviews. The second is behavioural presence, where the individual acts as if they were present in the environment, and exhibits behaviour concomitant with this. Again subjective and objective presence are logically orthogonal, but related in practice.

Shared presence, to our knowledge not yet discussed in the literature, similarly has two aspects. For each individual: first, the sense of the presence of other individuals in the VE; and second, the sense of being part of an entity and a process which is more than just the "sum of the individuals", i.e., being present in a group and in the process which the group is unfolding during the course of the group meeting. Once again, we can separate the subjective and objective aspects of each of these: the subjective relating to each individual's state of mind, and the objective relating to the observable behaviour of each member of the group, and the overall group behaviour. By "overall group behaviour" we mean such phenomena as the group as a whole gradually drifting spatially across a virtual room, without this being the conscious decision of any particular individual.

(b) Theories of Presence

Although there are no well-established fully worked out theories of presence, having some theoretical framework is essential in order to carry out meaningful experiments and take useful measurements. In previous work we have developed an approach to individual presence which is maybe the beginnings of an theory with some empirical backing, and a theory that leads to insights about interaction techniques within immersive virtual environments. This theory (most fully explained in [SLAT 95]) is based on the notion of immersion, as a description of a technology, leading to a potentially quantifiable measure of the degree of immersion offered by a system, and the match between proprioception and sensory data.

We postulate that personal presence is a prerequisite for shared presence. The following additional factors seem relevant:

• The notion of a Virtual Body is perhaps even more important for shared presence than for personal presence. An individual requires some spatial, acoustic, and ideally tactile information to establish and maintain the presence of other individuals in the environment.
  
• The static existence of others is almost certainly not enough, there must be a sense of the possibility of interaction and the exchange of information.
  
• The representation of others is again crucial: It has been found that different individuals respond differently to the different modalities: visual, auditory, kinaesthetic. Some might find it easy to maintain the sense of presence of others with just crude visual representations of Virtual Bodies, and text interaction only. Others might require fully functioning Virtual Bodies.
  
• Immersion: Almost all of our previous studies have been based on a visually immersive system. In this project with the use of the MASSIVE and DIVE systems we have the possibility of exploring non-immersive environments.

(c) Experimental approach

Progress was made in earlier work on understanding the factors that enhance personal presence by choosing a small number of parameters that were measured subjectively using experimentation: the sense of "being there", the sense of having been in the place specified by the VE rather than having just seen images depicting a place; and the extent to which the participant "forgets" that s/he is really in a laboratory wearing a HMD in favour of the virtual world [SLAT 95]. There is an intention to develop a similar set of parameters for "subjective shared presence". The simplest types of questions to elicit this form of presence that we are currently exploring, and that will form part of our experimental strategy are of the form:

• To what extent did you have a sense that you were in the same place as [person X] during the course of these events?
  
• To what extent did you have a sense that [person X] was in the same place as you during the course of these events?
  
• To what extent did you have a sense of the emergence of a group/community during the course of these events?
  
• To what extent did you have a sense of being "part of the group"?

There is similarly a need for a set of observable behaviours that can be compared as between the "real" and "virtual" worlds.

• It is well known that in repeated real world meetings, people tend to arrange themselves around a desk, or spatially in a room, in approximately the same places, meeting after meeting. Does this recur with virtual meetings - do they arrange themselves in the same place relatively as in the real meetings, and also in subsequent virtual meetings?
  
• Do virtual meetings unfold in the same way as real meetings? The same people speaking, in responding to the same kinds of events and information as tends to happen in real meetings?
  
• In the course of events that require motor skills - passing objects around, playing physical games, etc., do the events unfold in the same kinds of ways?
  
• Is there observable group phenomena which occurs whether or not the environment is virtual - for example where the group as a whole acts under the influence of some social gravity, and physically drifts spatially, without this being the conscious intention of any individual?

We are currently designing a series of experiments to explore these parameters.

4.2. The network requirements of CVEs

Next we consider the networking issues raised by CVEs. DEVRL addresses two major networking issues: scale and synchronisation. The following paragraphs discuss each of these in turn, touching on some of the technical approaches that have been adopted by current distributed VR systems.

(a) Scale

As the number of simultaneous inhabitants of CVEs grows beyond a few tens towards hundreds or thousands of people, so issues of scale will become paramount. We identify four distinct dimensions of scale:

• Bandwidth - coping with increasing network traffic as the population and complexity of virtual worlds increases;
  
• Computational - even if the network can deliver the information, techniques need to be developed to ease the computational load of processing it (especially rendering) .
  
• Perceptual - even if the information can be displayed, techniques need to be developed to help participants manage the cognitive load of understanding it (e.g. how would one follow a conversation involving several hundred people speaking all at once?).
  
• Geographical - physical distance introduces its own constraints. In particular, the speed of light imposes a lower limit on network latency which may become significant over wide areas (even without the further delays introduced by switching and transmission technologies).

A number of solutions have been proposed to deal with these issues. The use of multi-cast protocols has been widely discussed as a means of minimising network traffic (e.g. [MAC 94]). In addition, various spatial scoping mechanisms have been implemented in order to reduce both computational and network load by limiting mutual knowledge between objects to specific regions of space. These include the aura mechanism from the DIVE [CARL 93] and current MASSIVE [GREEN 95] systems and the cellular spatial sub-division technique proposed for future versions of NPSNET and MASSIVE. Considering perceptual scale, distancing techniques provide a means of filtering out the detail of more distant and therefore less interesting objects. Of particular note is the generalised spatial model of interaction as implemented in the MASSIVE system, where the notion of mutual awareness, controlled through the further concepts of focus and nimbus, allows for flexible and extensible distancing between objects across media such as graphics, sound and text.

(b) Synchronisation

The issue of synchronisation concerns the degree to which different participants' versions of a shared virtual world need to be kept consistent and the mechanisms by which this can be achieved. This problem becomes apparent to end users when significant latencies occur in the system. However, it is important to be aware that the synchronisation issue is in fact always present. Indeed, relativity tells us that there is no absolute notion of synchronicity in the real world even if we don't perceive the consequences of this for everyday interactions. At the heart of the synchronisation issue is whether to enforce synchronisation or whether to allow different participants' world states to diverge under certain circumstances. Systems which take the former approach may be based on a centralised client-server model or may employ a distributed database locking model to keep different world databases in step with one another (see the DIVE system for an example of the latter). The impact of latency on such systems is likely to be an overall reduction in system performance and an increased perception of lag (in essence, everyone perceives the world at the rate of the slowest person). Such approaches may not work well over wide areas or in highly heterogeneous systems involving machines with radically different capabilities.

An alternative approach involves the use of predictive techniques. Instead of transmitting changes in position, objects exchange higher level models of behaviour which allow their positions and representations to be calculated independently at different nodes of the distributed system. Such techniques seem particularly suited to environments which contain objects whose behaviours are both constrained and predictable (e.g. the path of a missile or vehicle) and have been widely used in battle simulation systems (e.g. NPSNET). However, it is not clear what overhead might be incurred for less predictable environments.

(c) Experimental approach

The overall aim of the network level experimentation is therefore to construct and validate a predictive model of network traffic. Two factors need to be considered. First, the network traffic generated will be application dependent. Second, the network traffic generated will be closely tied to the number of simultaneous users and their on-going actions (e.g. how often to people move, talk etc.). As a result, we propose that network evaluation should proceed as follows:

1. Each application and underlying system needs to be profiled. This involves conducting a formal analysis of communication protocols, resulting in a list of all possible application events and associated network messages combined with a discussion of the amount of traffic generated for each.

2. User behaviour needs to be profiled. This means gathering statistics about patterns of usage allowing us to confidently predict the relative frequencies of the different events described in (1). This requires the construction and use of event logging tools for each application.

The network traffic model arises as a combination of (1) and (2). Specifically,

Once constructed, the model can be validated by comparing predicted traffic against actual measured traffic (using network monitoring tools) for different numbers of users.

5. Summary

This paper has provided an overview of the UK's Distributed Extensible Virtual Reality Laboratory Project (DEVRL). The paper has described initial results in relation to all three of the project's objectives, namely establishing a distributed infrastructure for testing CVE applications; constructing three examples of such applications; and conducting experiments with these applications in order to explore the issue of multi-participant presence and the effects of network latency and bandwidth constraints on their operation.

At the time of writing the infrastructure has been established and tested through a series of virtual meetings and the three applications (the virtual classroom, collaborative information retrieval and geometric modelling) are under development. The next stages of the project will involve experimentation with these applications.

DEVRL is open to new participants who might want to test their own applications and systems over its infrastructure, take part in experiments or establish a presence in DEVRL Town. Please contact: devrl@cs.nott.ac.uk for more details.

References

[BARF 93] Barfield, W. and S. Weghorst (1993) The Sense of Presence Within Virtual Environments: A Conceptual Framework, in Human-Computer Interaction: Software and Hardware Interfaces, Vol B, edited by G. Salvendy and M. Smith, ElsevierPublisher, 699-704, 1993.

[BENF 95b] Steve Benford, Dave Snowdon, Chris Greenhalgh, Rob Ingram, Ian Knox and Chris Brown, VR-VIBE: A Virtual Environment for Co-operative Information Retrieval, in Proc. Eurographics '95, Maastricht, The Netherlands, September, 1995, North-Holland.

[BENF 95] Benford, S., Snowdon, D. and Mariani, J., Populated Information Terrains: First Steps, in Virtual Reality Applications, pp. 27-39, Academic Press Ltd, 1995.

[CARL 93] Carlsson, C. and Hagsand, O., DIVE: A Platform for Multi-User Virtual Environment, Computer & Graphics Vol 17, No. 6, 1993, pp. 663-669.

[GREEN 95] Chris Greenhalgh and Steve Benford, MASSIVE: A Virtual Reality System for Tele-conferencing, ACM Transactions on Computer Human Interfaces (TOCHI), ACM Press (in press).

[HEET 92] Heeter, C. (1992) Being There: The Subjective Experience of Presence, Telepresence, Presence: Teleoperators and Virtual Environments, 1(2), spring 1992, MIT Press, 262-271.

[HELD 92] Held, R.M. and N.I. Durlach (1992) Telepresence, Presence: Teleoperators and Virtual Environments, 1, winter 1992, MIT Press, 109-112.

[LOOM 92] Loomis, J.M. (1992) Presence and Distal Attribution: Phenomenology, determinants, and assessment, SPIE 1666 Human Vision, Visual Processing and Digital Display III, 590-594.

[MAC 94] Macedonia, M. R., Zyda, M. J., Pratt, D. R., Barham, P. T. and Zeswitz, S., NPSNET: a network software architecture for large scale virtual environments, Presence, 3(4), MIT Press, 1994.

[SHER 92] Sheridan, T.B. (1992) Musings on Telepresence and Virtual Presence, Telepresence, Presence: Teleoperators and Virtual Environments, 1, winter 1992, MIT Press,120-126.

[SLAT 94/5] Slater, M. and Usoh, M. (1994) Modeling in Immersive Virtual Environments: A Case for the Science of VR, International Conference on Applications of Virtual Reality, 7-9 June, 1994, Leeds, UK, and in Huw Jones, Rae Earnshaw, John Vince, Virtual Reality Applications Academic Press, 1995.

[SLAT 94a] Slater, M., M. Usoh (1994) Body Centred Interaction in Immersive Virtual Environments, in N. Magnenat Thalmann and D. Thalmann (eds.) Artificial Life and Virtual Reality, John Wiley and Sons, 125-148.

[SLAT 94b] Slater, M., M. Usoh, A. Steed (1994b) Depth of Presence in Immersive Virtual Environments, Presence: Teleoperators and Virtual Environments, MIT Press 3(2), 130-144.

[SLAT 95] Slater, M., M. Usoh, A. Steed (1995) Taking Steps: The Influence of a Walking Metaphor on Presence in Virtual Reality, ACM Transactions on Computer Human Interaction (TOCHI), September.

[USOH 96] Usoh, M., M. Slater, T.I. Vassilev (1996) Collaborative Geometrical Modeling in Immersive Virtual Environments, 3rd Eurographics Workshop on Virtual Environments, Martin Goebel ed., Monte Carlo, 21-23rd February, 1996.

[VASS 96] Vassilev, T.I. (1996) Fair Interpolation and Approximation by Energy Minimization and Points Insertion, in press.