Congestion Pricing Experiments 1 Aim and Research questions The overall aim of this set of experiments is to investigate the question: How acceptable to end-users is congestion-based charging as a network charging scheme? Dynamic usage-based charging must reflect actual resource usage. Resource usage depends on 2 characteristics: 1. The statistical characteristics of traffic. 2. End-users QoS requirements. Congestion-based charging is an attempt to tailor users' QoS requirements with the statistical characteristics of traffic so that the demand for resources correlates with these characteristics. 2 Web Charging Experiment We will compare the acceptance of congestion-based charging to other charging scenarios (see below for a list of proposed measurements): * TBC: Time Based Charging (The amount paid depends on the time spent in a charging area) * CBC: Congestion Based Charging (Charging only occurs when the user uses congested links) * FCC: Fixed Cordon Charging (A fee is paid when the user chooses to pass into a charging area) * Control condition (no charging scheme) 2.1 Conceptual Models This experiment will also be used to test the results of qualitative research into users' mental models of the Internet. These results suggest that charging schemes are conceptualized as most similar to time-based charging. We therefore ask: Do users rate TBC higher than any other charging scheme? If so, is this motivated by their mental model of the Internet? 2.2 Network Feedback Feedback will inform users of congestion associated with certain paths. In situations where feedback about network traffic is good, users must make trade-offs between trying a path that is perceived to be of less QoS, and choosing a path where they enter a charging area. In this situation we ask: Under what circumstances will users choose a path that incurs a charge? In situations where users are not provided with feedback about network traffic the additional question is: Once users have chosen a path that turns out to incur a charge, under what circumstances do they quit that path and choose another...if at all? And comparatively we ask: How does the presence of feedback about network traffic effect users decisions to choose a path that incurs a charge? 2.3 Participants and Task 120 participants will be used in the experiment. All should be familiar with Web-usage. All should have made on-line purchases. The tendency of users to pay for information clearly depends on its criticality. For this reason we should choose a task where there are several paths to completion. The proposed task is "Locating information in an on-line library". This task has the following favorable properties: 1. It is a common task. 2. It is made up of sub-tasks that can each be associated with different charging schemes and feedback. 3. It is made up of sub-tasks which may maintain users' interest and sense of achievement. 4. It is easily manipulated to have several paths to completion. User will be asked to book flights, hotel and car. Each sub-task should proceed from a higher to a lower level of operational granularity. For example, the first page should contain 8 possible links, the second page 6, the third 4 and so-forth. This is to enable users to make alternative choices that incur different charging schemes within the overall task. Each of the sub-tasks should follow the same pattern of high to low number of links available from Web-pages. 2.4 Independent/dependent variables We will vary: * The charging scheme (selected by users depending on what path they take to the goal) * The presence of predictive feedback (presented according to the path users take to the goal) We will measure: * Task completion: Was the user able to complete the booking? The interaction between the charging scheme and network feedback is critical to task completion (The Task Performance Measure-TPM). For example, users may be inclined to quit a path that involves FCC, or to accept CBC when a sub-task has become time-critical to overall task completion. * Perceived QoS, defined as levels of satisfaction. This will be measured dynamically via software 'scales' and verbal protocols, and generally by the same scale, and interviews, administered at the end of each session. * Users experience of charging schemes. We will use verbal protocols and post-session semi-structured interviews to gather qualitative data. * Users' physiological ratings. The inclusion of this DV would have the benefit of adding another objective measure on top of TPMs. Each condition should take approximately 15 minutes, but the user is free to choose different paths through the site. All conditions use the same task. Semi-structure interviews will be conducted for 15 minutes after interaction. Total time for each participant therefore 45minutes. Web latencies should be generated in the range of 11 seconds or less and should follow prespecified patterns (see Section 5). Table1 shows the conditions in the Web experiment (see Section 5 for metrics corresponding to the different patterns of QoS). Table 1: Summary of conditions for Web experiments Condition Participants Attributes 1 120 (All) Control: No charging scheme, no feedback, random QoS 2 30a no feedback, increasing QoS 3 30b feedback, decreasing QoS 4 30c no feedback, random QoS 5 30d feedback, sin-wave QoS 3 Multimedia charging experiment Users should interact with a system where the video frame rate and resolution has been pre-programmed. We will therefore be using streaming video. In this experiment we give users an option of 2 'channels'. Each channel represents the same multimedia clip operating under different charging schemes. In both schemes users are given a quota. In the face of not being able to ask users to spend their own cash, this quota represents a financial resource. This quota will be displayed to users at all times in the form of a 'reservoir' and numerical value. As MMC is dynamic users can't make isolated choices associated with actions, as they can on the Web. Therefore the following channels should be offered explicitly: a) CBC: Allow expenditure to fluctuate according to congestion (a commitment to pay more in the face of congestion). The important aspect here is: Do users opt for FCC when they their quota is at a certain level? b) FCC: Pay a specified amount once to get the channel and suffer the consequences of poor QoS (not linked to congestion). Here we ask: Do users opt for congestion based charging when levels of QoS drop below a certain threshold? Is this because it allows them to maintain that QoS at an acceptable level? 3.1 Participants and Task There should be 120 participants in this experiment. Each should be have a basic familiarity with networks (i.e. be regular users of email and the Web). In this experiment, like the Web experiment, we need to recognize that urgency of information will critically effect users willingness to pay. However, we are interested in comparative measures, within each condition, not on measuring urgency. The task chosen for the MMC experiment is access an important lecture. Participants will be asked specific questions about visual aspects associated with the lecture, and to make an overall judgement of how emploring they consider the lecturer. This task has the following favorable characteristics: * Users have to use social cues to make a judgement. This is know to focus their attention on the video. * It is visually oriented, as users are asked visually-oriented questions. * It is likely to be a scenario that participants have had experience with in the real-world. 3.2 Independent and Dependent Variables We will vary the frame-rate of each clip. In another experiment we will vary the image resolution. The frame-rate and resolution is identical irrespective of the user's choice of charging scheme. We will measure: * Users choice of charging scheme, including their reviews of this choice. * Task completion: Can participants answer the questions given at the end of the interaction? * Perceived QoS, defined as levels of satisfaction. This will be measured dynamically via software 'scales' and verbal protocols and generally by the same scale, and interviews, administered at the end of each session. * We will use verbal protocols and post-session semi-structured interviews to gather qualitative data on users experience of the charging schemes. * Users' physiological ratings. The inclusion of this DV would have the benefit of adding another objective measure on top of TPMs. Table 2 summarizes the conditions for this experiment. Table 2: Conditions for MMC experiment Condition Participants Attributes 1 120 (all) Control: No charging scheme, frame-rate maximum, resolution maximum 2 60a Frame-rate 5-10-16 3 60b Frame-rate 16-10-5 4 60a Resolution low - high 5 60b Resolution high - low 4 Proposed Timescale May: Plan and refine experimental proposals June: Build system and conduct pilot trials Early July: Conduct main trials Mid-late July: Analyze results Early August: Write up 5 Implementation 1 Web experiment We need: * Software to be implemented to capture users satisfaction dynamically, this should either be via an adapted version of the QUASS software, in which case an unlabelled scale will be used post-interaction - or new software scales identical to those used post-session. * Software interface showing users' remaining quota. * Pre-programmed Web-latency associated with access to each Web-page (the software ETEWatch can be used (http://www.candle.com/etewatch). * Preprogrammed feedback and charging scheme associated with certain paths through the task. * The HTML front-end to represent the on-line libraries , with links to pages with pre-programmed latency. * Logs of timestamp, page access name, access number i.e. how many clicks has the user made to get here?, associated charging scheme, feedback given (if any). * Logs of timestamp, users dynamic QoS evaluation. 1.1 Paths through the task In Figure 1 the Web-page display is in bold, instructions to participants in italics. Figure 1 shows that the user should have the option to return to the home page from any other page. Start sub-task 1: Answer specific questions on topic X Operation 1 2 3 4 Start sub-task 2 : Answer specific questions on topic Y Start sub-task 3: Answer specific questions on topic Z 1.2 Operations 1.2.1 Feedback conditions Operation 1- 4: pop-up window feedback on links warning of charging mechanism if CBC then display prediction of congestion on link (i.e. empty, busy, very busy). If: FCC then debit quota on click, update quota UI. If CBC then debit quota on click according to predicted congestion (i.e. debit for 'empty' third debit for 'very busy'), update quota UI. If TBC then debit quota 0.2% per second, continuously update quota UI 1.2.2 Non-feedback conditions Operation 1- 4: pop-up window feedback stating the way the user has been/is being charged Repeat for other sub-tasks 1.3 Patterns of latency 1. Increasing QoS: 11sec-0 sec in 500ms decrements. Then hold QoS at 0sec latency for subsequent accesses. 2. Decreasing QoS: 0sec-11sec in 500ms increments. Then hold QoS at 11sec latency for subsequent accesses. 3. Sin Wave QoS: 11sec-7.5sec in 500ms decrements. Then hold QoS at 7.5sec latency for 6 accesses. Then increase delay from 7.5sec- 11sec in 500ms increments. 4. Random QoS: Generate Web latency from random function in range 0-11sec. Note: Web latency here refers to the time it takes from the request from the old page to the first piece of information to appear on the next page. The old page should be visible in the browser until the next page is ready to be loaded in entirety. 2 Multimedia experiment We need: * Prerecorded frame-rate and resolution on a video stream of a networked lecture (fr range 5-16fps). * Software to be implemented to capture users satisfaction dynamically, this should either be via an adapted version of the QUASS software, in which case an unlabelled scale will be used post-interaction - or new software scales identical to those used post-session. * Software interface showing users' remaining quota. * Ability for the quota UI to dynamically update according to our simulation of network congestion. Note: this update will just be a display, so can be implemented and recorded prior to interaction, and triggered on the user's selection of this charging scheme. * An options UI displaying choices of FCC and CBC, ability for user to stop the lecture on which the options UI should automatically appear to allow charging scheme reselection. * Logs of timestamp, chosen charging scheme, if/when user reviews charging scheme choice. * Logs of timestamp, users' dynamic QoS evaluation. 1