Graphical language games: Representational form and communicative use.

Dr Pat Healey

This talk will develop the claim that that design for Human-Human Interaction and design for Human-System interaction are distinct. It will draw on work under the ESRC/EPSRC PACCIT initiative project MAGIC (Multimodality and Graphics in Interactive Communication) which is investigating the use of graphics as a mode of human-human interaction. I will describe a series of experiments using 'pictionary'-like tasks which involve exclusively graphical interaction. The results of these studies show several parallels between verbal and graphical dialogue. Specifically, co-ordination of 'graphical referring expressions' and the emergence of community-specific graphical 'sub-languages'. These experiments also provide evidence that communicative use directly conditions representational form - independently of cognitive or 'semantic' constraints. Some of the implications of these findings for theories of representation and abstraction will be discussed.