Second Life as a research platform
Svetlana Stoyanchev
SLTC Newsletter, January 2010
Second Life (SL) is a virtual social environment with over 11 million registered users. SL is now also becoming a platform for research in dialogue, gestures, social behaviour, and information presentation. It provides a resource for creating and using virtual spaces, designing and manipulating virtual characters, and gives access to a large pool of potential users for evaluating new technology. This article describes some of the projects that use Second Life environment for research.
Embodied Agents
Embodied conversational agents (ECA) are virtual characters in multimodal environments that normally have abilities such as speaking, moving, gesturing, changing eye gaze, and facial expressions. Virtual characters often have application-specific reasoning abilities and can answer a user's questions or guide a user through a task. For example, virtual characters in research projects have played roles such as real estate agents [1] or museum guides [2]. Researchers model dialogue, social, and visual communication behaviour using virtual characters and study how the virtual character's behaviour, such gestures, facial expressions, eye gaze, physical appearance, and positioning of a virtual agent affect human-agent interaction.
Virtual agent behaviour, like spoken dialogue, is difficult to evaluate automatically because there is no gold standard for gestures in a particular situation or a predefined answer to an unexpected question. The improvement of behavioural and dialogue models rely mainly on human evaluation. Systems implementing virtual agents can benefit from access to a large pool of users.
Description of Second Life
Second Life is a virtual world accessible via internet using a free client program. Membership is free and users create and customize their character. SL has grown to be a virtual collaborative community with over 11 million registered users. Users can buy land, and build elaborate virtual spaces for art, music events, business, and educational resources. Universities have a significant presence in Second Life as both users and researchers. Universities create virtual spaces to host meetings in Second Life, provide educational resources for students and visitors, and hold virtual courses, as well as study social interaction and user behaviour in SL environment. For example, here is a video of a space in SL created at the Open University. The space is now open for the use of researchers and educators.
Researchers at AT&T Labs have introduced services such as telephone messaging and web publishing that allow communication between SL users and the real world. This technology was demonstrated and tested by the attendees of the WWW 2008 conference with a SL site constructed for this purpose [3].
Use of 3d Environments in Research
The Second Life virtual environment is becoming a platform for conducting research. In addition to SL 'residents' who are human-controlled avatars, automatically controlled avatars (bots) can enter the environment and communicate with the residents.
Second Life provides a rich infrastructure with graphics, interaction, social network, making ECA research much more accessible. The availability of virtual spaces and potential users makes Second Life a promising environment for research on virtual agents, dialogue, models of communication, or information presentation.
For example, Breitfuss and Prendinger [4] evaluate automatic algorithm for gestures and eye gaze generation using second-life characters:
The AI lab in Bielefeld University has transferred a virtual agent to Second Life [5] for research on modelling social skills.
Jan et al. [6] describe a virtual guide character that shows visitors around the Army island in Second Life. The agent detects a user's presence, offers a tour, and leads the user to different destinations. He stops when user falls behind and can hold a dialogue about the island. To try this tour in Second Life, search for location named " Army Information Center " and teleport there. Once teleported, look for 'StaffDutyOfficer Moleno' who will give you a tour around the island.
Some of the Tools for Creating Second Life Bots
OpenMetaverse foundation provides an open-source solution for creating bots in Second Life.
At NII lab researchers have also developed MPML3D, a 3D markup language for virtual characters. The script allows the manipulation of verbal and non-verbal behaviour of agents in Second Life. The script's interpreter runs as a server and is publicly available.
Advantages of Second Life
Second Life provides an environment where 3D characters can be created, used, and tested. Second Life is already populated with 11 million users providing a rich subject pool for running research studies on virtual characters and testing dialogue and multimodal applications. SL's environment is multi-modal, multi-user, and situated. An additional bonus is that text-based conversations are common in SL, allowing for natural dialogues through text interface that bypass Automatic Speech Recognition. Users may be attracted to the research project by advertisement. Research participants may be paid with Linden Dollars (250L$ = 1US$ accordingly to the official SL L$ exchange). However, if an experiment involves a game, research participants may be attracted to participate without monetary compensation. Some examples of such experiments that were not done in Second Life include a treasure hunt game used evaluate Natural Language Generation of instructions for the players of the game [7], the ESP game in which the players aim at achieving consensus on image description, and Restaurant Game that captures behaviour and dialogue from human-human interactions.
Researchers venturing into SL need to take care of ethical issues of research on human subjects and compliance with the Institutional Review Board (in the US). Bots that invade privacy, or annoy other users raise questions in the user community.
Second Life is a social and gaming environment which at the same time provides a resource for researchers for designing and implementing experiments with virtual characters. It also makes research on virtual agents much more accessible to researchers and gives an access to a pool of potential users for evaluating experimental models. Second Life is only one of many virtual worlds and the future of virtual environments as experimental platforms is promising.
Thanks to Antonio Roque and Giuseppe Di Fabbrizio for providing extensive input to this article.
References:
- J. Cassell, M. Stone and H. Yan Coordination and context-dependence in the generation of embodied conversation , INLG (2000)
- S. Koppet al. A conversational agent as museum guide - design and evaluation of a real-world application. In: IVA. Volume 3661 of LNCS., Springer (2005)
- R. Jana et al. Bridging Communication Services – Connecting Virtual Worlds to Real World: a Service Provider’s Perspective In WWW 2008
- E. Weitnauer et al. Intelligent agents living in social virtual environments--bringing Max into Second Life , Intelligent Virtual Agents (IVA08)
- W. Breitfuss et al. Automatic generation of non-verbal behavior for agents in virtual worlds: A system for supporting multimodal conversations of bots and avatars. Proc HCI International 2009
- D. Jan et al. A Virtual Tour Guide for Virtual Worlds, Intelligent Virtual Agents 2009
- D. Byron et al. Report on the First NLG Challenge on Generating Instructions in Virtual Environments (GIVE)
If you have comments, corrections, or additions to this article, please contact the author: Svetlana Stoyanchev, s.stoyanchev [at] open [dot] ac [dot] uk.
Svetlana Stoyanchev is Research Associate at the Open University. Her interests are in dialog and information presentation.
Add A Comment