Steels, Robotics, and the Evolution of Language

Antonio Roque

SLTC Newsletter, April 2010

As research in language technology continues to develop, insights are contributed from interdisciplinary topics such as robotics and computer investigations into the evolution of language.

One researcher pursuing such directions is Luc Steels, Professor of Computer Science at the Free University of Brussels and Director of the Sony Computer Science Laboratory in Paris. His research takes a number of directions, generally using agent-based computer simulations, analytical models and experiments with robots [4] to investigate the development of the cognitive mechanisms needed to manage the complexities of language use. For example, he may use simulations to examine the ways in which a population of agents might self-organize a set of perceptually grounded categories, focusing on the specific example of categorization of colors [7].

"Language is an open system," he said in an interview conducted at IJCAI-09. "So it's always changing, always evolving. So we need to build adaptive language systems, as opposed to [developing] a database of examples, and applying some machine learning techniques to get a language system, and then you deploy it; this is how most systems are now built. But I think part of the reason why they don't work very well is because dialogue is really a matter of give and take. So all my research almost can be seen as a way to figure out how to do that."

The goal is a practical one: study ways in which robots and agents might learn and teach each other the various aspects of language [2]. For example, investigating the ways that robots could perform lexical acquisition using representations that link perception, body, action, and language [8], or studying issues that arise from embodient when conducting color-learning experiments with robots [1].

"So we're trying to understand very basic things like color or space or time or articles," Professor Steels said. "So I think we need to do a lot more basic research in this area to really go to the bottom of all these different surrounding domains, syntactic domains, and so that's what we are doing... I see our role more in delving deep, and then you realize actually how difficult it is, even if you take something as simple as color language, or language about space. Like, if you say left and right, you immediately get into issues of perspective reversal and how to express things with a minimal complexity, and robustness, and all those kinds of things. So I think what we do is basic research."

A useful tool in investigating these issues is the Fluid Construction Grammar [3]. "The idea of Fluid Construction Grammar is that the grammar is not fixed," said Professor Steels. "So the rules of the grammar are potentially changing at all times, so this means, among other things, that there's not a single way of doing things, there are always multiple possibilities which are stored in one agent, we always have a multi-agent system. So there are multiple possibilities and there's a preference, and so there's a score with every rule that's in your grammar, and these scores are being adapted as the agent is interacting [with other agents]. So in a sense it's a way of dealing with variation which is clearly there in a population of language users. So that's where the fluidity comes from. And also the fact that you can deal with ungrammatical sentences. So you're parsing not to check whether the sentence you read is grammatically correct, but you're parsing to get as much as you can out of the sentence.... And then you use the real world in your understanding process to come to an interpretation. And then once you do that, you can potentially get more feedback from the listener, or from the effect of the action that you execute as a consequence of what you said... So then you update, you change, you adjust the rules, you change the scores, you do lots of lots of things after every interaction."

Pursuing this and related research has led to advances and proposed future directions in some of the most fundamental questions in AI and natural language processing, such as investigating issues in meaningful symbols use [6], the social aspects of language use [5], and more.

For more information, see:

April 2010 Newsletter Home

From the SLTC Chair: Time to Get Affiliated?

IEEE Signal Processing Society Newsletter

Calls for papers, proposals, and participation

Job advertisements

Book announcements

"Hot speech summer" in Brno

Normalizing txts into text for speech synthesis

Following Global Events with IBM Translingual Automatic Language Exploration System (TALES)

ASRU 2011

Steels, Robotics, and the Evolution of Language

SLaTE - the ISCA Special Interest Group on Speech and Language Technology in Education

Overview of Acoustic Modeling Techniques from ICASSP 2010

Speech scientists are dead. Interaction designers are dead. Who is next?

The AT&T Statistical Dialog Toolkit V1.0

Question Generation Challenge

Subscribe to the newsletter

SLTC Home