Porta-Person: Telepresence for the Connected Conference Room

Porta-Person: Telepresence for the Connected Conference Room Nicole Yankelovich 1 Network Drive Burlington, MA 01803 USA nicole.yankelovich@sun.com Jonathan Kaplan 1 Network Drive Burlington, MA 01803 USA Jonathan.kaplan@sun.com Nigel Simpson 15395 SE 30th Place, Suite 120 Bellevue, WA 98007 USA nigel.simpson@sun.com Joe Provino 1 Network Drive Burlington, MA 01803 USA joe.provino@sun.com Abstract This paper describes a telepresence device called Porta- Person. This is the first project in a larger initiative known as the Connected Conference Room, which aims to improve the user experience for remote people connected to meetings taking place in conference rooms. The Porta-Person is designed to enhance a sense of social presence for remote meeting participants. It does this by providing a high-fidelity audio connection and a remotely controlled telepresence display with video or animation. Keywords Telepresence, distributed meetings, video surrogates. ACM Classification Keywords H.5.3 [Group and Organization Interfaces]: collaborative computing, synchronous interaction, computer-supported cooperative work, organizational design. Figure 1. Porta-Person with video display. Copyright 2007, Inc. CHI 2007, April 28 May 3, 2007, San Jose, California, USA. ACM 978-1-59593-642-4/07/0004. Introduction If you have ever dialed-in to a meeting taking place in a conference room, you probably know what it feels like to be a second-class citizen. The Porta-Person is designed to address the problems with these sorts of hybrid meetings. In contrast to fully distributed meetings or ones where everyone is in a conference room, hybrid meetings involve a mixture of conference rooms and remote individuals. The disparity between the remote and local experience in hybrid meetings

2 Audio Problems Poor quality speakerphones Too much background noise Multiple people speaking at the same time can be difficult or impossible to hear People speaking too far from microphones Other Remote Attendee Problems Inability to conduct side conversations or participate in informal interactions In-room attendees forget about remote people Challenging to break into lively conversations Difficult to detect in-room speaker changes [10] Hard to know the identity of the people currently in the room Hard to know the identity of every speaker Difficult or impossible to participate in brainstorming Cannot see in-room demonstrations or artifacts Conference Room Problems Local people more emotionally salient than remote participants Easy to forget about remote participants Often do not know who is still connected Figure 2. Problems with hybrid meetings. makes them fundamentally different from other types of distributed meetings. The Laboratories Connected Conference Room initiative is focused on addressing the problems inherent in hybrid meetings. The Porta-Person is the first project in this umbrella initiative to improve the user experience for people connected to meetings that are taking place in conference rooms. The Connected Conference Room initiative was motivated by user research we conducted in 2004 to understand the shortcomings of distributed meetings [9]. At that time, we created the Sun TM Labs Meeting Suite (a.k.a. Meeting Central) to address the problems of fully distributed meetings. We knew from our data, however, that only 30% of distributed meetings in our company are fully distributed. The other 70% involve at least one conference room. In fact, at the time of the study, 60% of all distributed meetings involved a single conference room plus remote individuals. The other 10% involved two or more conference rooms, with about half of these including remote individuals. In our study we also learned that remote attendees suffer more problems than local attendees. The 43% of people who were on their own for distributed meetings reported significantly lower overall meeting effectiveness (p<.0l) as compared to people who were with others. Problems with Hybrid Meetings While hybrid meetings suffer from almost all the same problems as fully distributed meetings, their two-tiered composition results in a host of additional problems. Based on data from that original study as well as information gathered in follow-up interviews, focus groups, and feedback over the past two years from our almost 1,000 Meeting Suite users, we compiled a list of problems with distributed meetings that involve conference rooms. The Porta-Person project addresses the problems marked with a in Figure 2. Taken together, these problems boil down to a lack of social presence, or the feeling of being together with another [1]. This is a key issue for both local and remote participants. Between not being able to hear well and not being able to participate in informal conversations that are important for forming relationships and trust, remote individuals often feel disenfranchised. Because it can be difficult to break into a conversation, the remote attendees struggle to stay engaged in the meeting after missing opportunities to speak and noticing that the meeting moderator has forgotten to solicit input from them. While remote people do not feel a strong sense of social presence with the people in the room, the people in the room have an even poorer sense of being with the remote people. Because the physical presence of the live people is so much more salient than the low quality, disembodied voices coming over the speakerphone, it is natural to devote more emotional energy to the people who are physically present. When in-room attendees do try to engage remote participants, they frequently complain that the remote people are not paying adequate attention. This, unfortunately, is often an accurate assessment. Because they cannot hear well and because they feel like second-class citizens in the meeting, remote attendees attention can wander. So as not to feel as if their time is being entirely wasted, they may also start

3 Figure 3. Porta-Person with an animated display being used during an actual presentation (a). This Porta- Person s (b) camera captured the video panorama shown in Figure 4. to multi-task. And the vicious cycle repeats. The more the remote people seem disengaged, the more they are ignored by the in-room attendees and the more they are ignored, the more likely the remote attendees are to disengage. Resolving the Problems In the Connected Conference Room initiative, we hope to eventually tackle all of the problems mentioned in Figure 2. Our goals are to improve a sense of social presence for both in-room and remote attendees, improve the audio quality for remote attendees, identify by name who is in the conference room and who is currently speaking, provide gaze awareness, and improve all types of data sharing that occurs during meetings. We designed the Porta-Person prototype to primarily address the social presence and audio quality problems highlighted in Figure 2. Related Work The Porta-Person was inspired by Bill Buxton s seminal work on the Hydra project. Hydras, or separate video displays representing each remote participant, were based on Buxton s concept of a video surrogate. He writes that video surrogates are based on the notion that the physical location of participants has an important influence on social interactions in face-toface meetings [2]. Another project that directly targets hybrid meetings is Hermes. This project used multiple television monitors around a conference table to convey a sense of physical location [5]. MIT s Zero Bandwidth Teleconferencing project was another early attempt to enhance social presence for remote conference attendees by experimenting with a variety of animation techniques [6]. Telepresence robots, of which Pebbles [8] is one of many examples, also address some of the same problems of presence awareness. Porta-Person The Porta-Person is a telepresence device that sits in a conference room on or near the table (Figure 3a). The device is made up of a display screen, a video camera, stereo speakers, stereo microphones, a custom-built stereo echo canceller, and a servo motor attached to a rotating platform (Figure 3b). The device can rotate approximately 180 degrees. As it rotates, the Porta- Person captures and transmits video of the room, resulting in a panorama display on the remote user s screen. The panorama is made up of one live video segment, shown in a white box, embedded into still images of the rest of the room (Figure 4). By clicking on different segments of the panorama (or by dragging the white box), the remote user can rotate the Porta- Person in order to see different portions of the room, face people who are talking, or address a question to a particular person. When the remote person connects to the Porta-Person, their video image (Figure 1) or animated representation (Figure 3b) appears on the screen, roughly life-size and at head height. We provide the option for animation since users working from home may not have a camera, the bandwidth, or the desire to share video. When the remote person speaks into the software phone built into the Porta-Person software, their voice comes out of the device's speakers. Depending on the available bandwidth, the fidelity of the audio can be up to CD quality. The psychological power of a person s voice coming from a point source in the room was one of Buxton s most compelling insights. It enables parallel conversations [2]. As with the Hydra, the Porta-Person

4 allows remote participants to have side conversations with people in the room. The Porta-Person, however, improves on the Hydra and other related systems user experience in several important ways guided by our design focus on hybrid meetings. These include the advanced audio capabilities, remote control, device sharing, and collaborative video. The audio capabilities of the device set it apart from other telepresence experiments. First, the high-fidelity audio quality allows local people to hear the remote people clearly, almost as if their voices were in the room with them. The benefits are more dramatic for the remote attendees. The stereo microphones embedded in the Porta-Person allow remote users to hear where local people are situated in the conference room. The voices come from the direction of the people speaking relative to the location of the Porta-Person in the room, giving the remote person a much better sense of being present in the room. As we demonstrated in our recent audio quality study [10], the stereo input also makes it easier for remote attendees to differentiate voices and to distinguish voices from background noise. Additionally, we found that the combination of stereo and high fidelity audio helps remote users to better understand heavily accented speech. High fidelity, stereo audio also helps with understanding speech when multiple local people are talking at once, a problem DiMicco found to be common in face-to-face meetings [3]. Comprehending multiple simultaneous audio streams is almost impossible over a regular telephone line and is the reason why remote people can rarely participate in the informal but socially significant conversations that take place just before and just after conference room meetings. It is during these informal encounters that colleagues establish and maintain trusting relationships. A significant advantage of the high fidelity, stereo audio combined with the remote control is that the remote person can turn to face a local person and have a conversation with that person while other conversations are taking place in the room. This makes it a lot easier to hear the local person since the act of rotating the device repositions the microphones to face the speaker. Figure 4. Video Panorama captured during the presentation shown in Figure 3a. Live video plays in the area surrounded by the white box (right). The rest of the image is made up of still frames that are updated when the Porta- Person rotates. Any user connected to the Porta-Person can see the live video and take control of rotating the device. They can also create, move, and edit labels. The labels created by users in this image indicate the location ( Burlington, MA ) and the name of the presenter ( Daniel Koo ).

5 Figure 5. Face Tool display showing three remote people connected to a single Porta-Person. Nicole s video is showing because she is currently speaking. The box around her name indicates that she also has remote control of the device. Not only does this ability to point the microphones improve audio quality, but the act of moving helps to make the Porta-Person surrogate almost as salient as the live people in the room, giving local participants a much greater sense of the remote person s presence. Our software and audio system are designed to allow any number of Porta-Person devices to be set up in a single room. Ideally there would be one for every remote meeting attendee, but realistically there are likely to be more remote people than available devices. We have therefore designed the Porta-Person as a collaborative device, allowing multiple people to connect at the same time. When this happens, the Porta-Person software, known as the Face Tool, displays either the video or the animation of the remote person who is currently speaking or who last spoke. A list of the names of all the remote people connected to a single Porta-Person is displayed at the bottom of the screen. well as create, edit, and move labels. This allows the remote people who know the identity of the local people to label the video for the benefit of the others connected to the same device. In fact, someone in the conference room could also connect to the video panorama for the purpose of annotating the video image with meeting attendee names. Preliminary Findings and Future Work We currently only have one fragile working Porta- Person prototype constructed out of foam core. In thinking about building additional units in order to run a field trial, cost was immediately a factor. The original design embeds an expensive tablet PC for use as a display. We have therefore redesigned the Porta-Person to use a removable laptop (Figure 6). This significantly reduces the cost of each unit, and makes it considerably more portable. To use this new model, a meeting attendee places a laptop on the Porta-Person device that provides power, audio, and networking support for the laptop. The name of the person currently speaking is marked with a speaking indicator and the name of the remote person who is currently controlling the device is highlighted (Figure 5). From the point of the view of the remote people sharing a Porta-Person, they all connect to the same Video Panorama collaboration (Figure 4). This software has a "Take Control" feature, allowing different users to control the movement of the Porta-Person. While only a single person can rotate the device at one time, any number of users may simultaneously see the video as The need to custom-cut foam core as well as the fragility of the material makes the original prototype impractical to reproduce. Our new design involves all standard parts that can be ordered from Internet sources. This, coupled with a new wiring scheme, should make it relatively easy for others in our company to build their own Porta-Person, which we hope will stand up to real use. Because of the fragility of the one working device, only our own project team and our extended staff have experienced the Porta-Person in the context of real meetings. One of our home-based employees has been using the Porta-Person for several months to connect to

6 Figure 6. Revised Porta-Person design with removable laptop showing animation and video. staff meetings. For the first time I feel like I'm an equal participant in meetings, he reports. I can hear everyone clearly and people pay much more attention to me. Another of our early users spent two days connected to a Porta-Person to remotely attend an event. I felt as if I was really in that conference room, she said. People would see me through the window and come in to say hello. It s the best remote work experience I have ever had. Preliminary user experience indicates that in addition to solving other audio problems, the Porta-Person largely solves the problem of people speaking too far from the microphones. We believe this is due to a number of factors. The microphones themselves are high quality and the audio does not cut out as happens with some speakerphones. There are also two microphones that are pointed like ears to pick up sound from the left and right rather than from a single point source, and the audio is transmitted at a higher quality, up to CDquality stereo audio compared to telephone-quality mono audio. As soon as we finalize the design of the removable laptop version, we plan to deploy a handful of Porta- Person devices around our company to assess how well it performs in terms of enhancing a sense of social presence and improving audio quality. The next step in the Connected Conference Room initiative is to move on to solving the problem of identifying individuals in the conference room and determining which of those people is currently speaking. We are also launching a project within this initiative to explore ideas around mixed-reality conferencing such that whiteboards and other data sources can be more easily shared between virtual and physical conference rooms. References [1] Biocca, F., Burgoon, J., Harms, C., and Stoner, M. Criteria and scope conditions for a theory and measure of social presence, Presence 2001 (2001). [2] Buxton, B. Scientific Director s Report: Living in Augmented Reality, in the Ontario Telepresence Project Final Report, Information Technology Research Centre, Telecommunications Research Institute of Ontario (1995), 18-34. [3] DiMicco, J. M, Hollenbach, K., and Bender, W. Using visualizations to review a group s interaction dynamics, Proc. CHI 2006, ACM Press (2006), 706-711. [4] Hauber, J., Regenbrecht, H., Billinghurst, M., and Cockburn, A. Spatiality in videoconferencing: trade-offs between efficiency and social presence. Proc. CSCW 2006, ACM Press, 413-422. [5] Inoue, T., Okada, K., and Matsushita, Y. Integration of face-to-face and video-mediated meetings: HERMES, Proc. GROUP '97, ACM Press, 405-414. [6] MIT Architecture Machine Group. Zero Bandwidth Teleconferencing - video, (1982). http://www.media.mit.edu/speech/sig_videos.html [7] Sellen, A. J. Speech patterns in video-mediated conversations. Proc. CHI 92. ACM Press (1992), 49-59. [8] Telbotics. Pebbles. http://www.telbotics.com/features.htm [9] Yankelovich, N., Walker, W., Roberts, P., Wessler, M., Kaplan, J., and Provino, J. Meeting Central: Making distributed meetings more effective. Proc. CSCW 2004, ACM Press, 419-42. [10] Yankelovich, N., Kaplan, J., Wessler, M, Provino, J., and DiMicco, J. Improving Audio Conferencing: Are two ears better than one. Proc. CSCW 2006, ACM Press, 333-342.