Book review: "Symbols and Embodiment: Debates on meaning and cognition"

Antonio Roque

SLTC Newsletter, July 2011

This book review discusses "Symbols and Embodiment: Debates on meaning and cognition", edited by Manuel de Vega, Arthur Glenberg, and Arthur Graesser, which contains many ideas useful to language technologists.

One reason that working with language and computers is so interesting is that it leads to notoriously difficult problems. Trying to make robust interactive systems, especially in situated environments, leads the researcher to consider the problem of meaningful language, and the relationship between symbol systems, cognition, and language use.

"Symbols and Embodiment: Debates on meaning and cognition" is a good example of how such difficult research problems can be productively addressed by an interdisciplinary group. The book was developed as a follow-up to a workshop of computer scientists, neuroscientists, experimental psychologists, and other cognitive scientists. The main chapters of the book are expansions of presentations that were given at the workshop, and each chapter includes a transcription of the discussion that occurred after the presentation.

An introductory chapter presents the central questions of the book. The first set of questions considers a symbol system being used by an intelligent agent. To create meaningful behavior (such as language use), is it enough for this symbol system to be "arbitrary, abstract, and amodal"? Or should the symbols be embodied in perceptual systems, in the same way that empirical data suggests that humans use sensorimotor cognitive resources for symbol processing? Also (or alternately?) should the symbol systems be grounded by relating them to physical objects in the agent's world? Second, to what extent do the results showing the activation of brain regions for perception and action during language processing in humans actually suggest that meaningful language use in humans is embodied? Third, how do the first two questions relate to meaningful language use in computers: are humans and computers completely different, are they analogous, can computers only be used to simulate meaningful human language use, or can they be meaningful in the same way? Finally, what are the best ways to explore these questions?

The participants propose a number of different answers to these questions. Latent semantic analysis is discussed as an example of statistical symbolism (as opposed to less data-driven symbolic approaches) and in this context, meaning is based on word co-occurrence. This is supported by evidence that covariation reflects world knowledge, for example. Proponents of embodied cognition discuss empirical data supporting their view, and discuss theoretical models such as neuronal resonance and affordances as being central to meaning. Neuroscientific approaches describe both language-specific and meaning-specific structures in the brain, although the data is "currently fragmentary and partially contradictory" and thus unable to provide an entire answer by itself. AI approaches from the domains of robotics and intelligent tutoring systems show that interactive systems can be valuable without grounded and embodied representations, although they are less valuable as complete explanations of linguistic meaning; and furthermore, conversational robots benefit from grounded representations. These, and more, are the types of issues discussed in the book: informed by experiments, data, and implemented systems, yet highly theoretical.

Although the various authors of the book spend time defining terms such as "symbols", "representations", "grounding", and "embodied", they are (thankfully) happy enough to provide the definitions they have used in their own research, and proceed with their arguments. Perhaps because of the enormity of the problems of investigating meaningful language use, the authors find plenty of space to describe the research questions they are pursuing and how they differ from the others, rather than producing minute critiques of other researchers' ideas. For example, statistical symbolists point out the difficulty of handling abstract meaning in embodied approaches, and embodied theorists argue that statistical symbolism only reflects world knowledge because words correlated with experience, but all participants seem to sense that statistical-symbolist and embodied approaches are describing different parts of a much larger mystery.

In all, the book is a fascinating overview of how various disciplines are approaching the problem of meaningful language use, and how interdisciplinary teams can find common ground while disagreeing on many details. Perhaps this is because of a common acknowledgment of how little humans actually understand about meaningful language use, and the vast amount of research there is yet to be done on the topic. Even in this fairly wide-ranging collection there are perspectives that could have contributed; for example "grounding" in this collection refers exclusively to linking symbols to objects in a physical world, while other researchers have explored the importance of societal grounding and dialogue grounding in meaningful language use. Nevertheless, this is still a book worth examining for anyone interested in the topics discussed above.

For more information, see:

• "Symbols and Embodiment: Debates on meaning and cognition" edited by Manuel de Vega, Arthur Glenberg, and Arthur Graesser. More information is on the publisher page or Google Books
• Details about the workshop on which the book was based, including a pre-press version of the first chapter (pdf) of the book which was distributed before the workshop.