Transcribing and visualizing language, time, action, and the body: an experimental project

Lecture series, Science, Roma

Ways of Seeing





Lecture series, Science

Ways of Seeing

Transcribing is a key practice in several scientific domains in which the methodology crucially relies on the gathering of data through recordings – such as video materials – and on their representation, in a textual and visual form. For example, the tradition of conversation analysis bases its studies on audio and video recordings of social interactions, which are then transcribed according to various conventions. Whereas talk is often transcribed in a rather standardized way (e.g., Jefferson 2004), for embodied conduct – including gesture, gaze, head movements, facial expressions, body postures, movements of the entire body – there is still a lack of standardized conventions. Multimodal transcription constitutes an attempt to offer a representation of the temporal details of action and interaction, through the hybridization of textualizations and visualizations of video recorded images. This workshop aims at exploring analytical and technical aspects of textual and visual practices in the fabrication of transcripts. For so doing, researchers practicing multimodal transcription and artists interested in the representation of movement, body, and action in time will collaborate together. The aim is to produce new forms of visualizations and experiment their advantages and limitations for the analysis and the representation of social interaction.


With: Guelfo Carbone, Clélia Dumas, Ryan Hamill, Katariina Harjunpää, Lorenzo Gigotti, Sara Keel, Eric Laurier, Lorenza Mondada, David Tomás Monteiro, Philippe Sormani, Hanna Svensson, Kimmo Svinhufvud, Veronika Timashkova


Evening lecture, 13 April 2016:
Prof. Lorenza Mondada (Universities of Basel and Helsinki)
A visual turn in linguistics? Uses of video for studying language

Full video of the evening lecture

Lorenza Mondada
I was born on the shores of Lago Maggiore, in Ticino, the Italian-speaking, southern part of Switzerland, in 1963.

After studying at the University of Fribourg (1982-1986), I worked as a teaching and research assistant at the universities of Lausanne, Fribourg and Neuchâtel. I spent a year in Paris at EHESS (1991-1992) and I completed my PhD in January 1994 (Univ. of Lausanne). In my PhD I investigate the description of space in travelogues. This led to my interest in practices of space formulation in spoken language and talk, as I increasingly became involved in researching naturally occurring recorded social interactions.

In 1996, I spent a sabbatical semester in the Anthropology Department at the University of California Berkeley. From 1996 to 2001, I was appointed Assistant Orofessor at the University of Basel, where I obtained my Habilitation in 2000. While there, I directed a research project on the interactional production of knowledge in plurilingual research teams, based on extensive ethnographic fieldwork and data collection in various scientific and medical departments. During this time, I became interested in talk at work and in workplace practices, as well as in the ways in which grammar is both a resource for interaction and emerges within its sequential organization.

In 2001 I was awarded the National Latsis Prize – a prize awarded annually in Switzerland to the top researcher under 40.

I moved to Lyon in 2001 to take up a position as Professor in the Department of Linguistics at the University of Lyon 2. In January 2007, I was appointed Director of the ICAR Research Lab (a unit supported by CNRS, University of Lyon 2, and Ecole Nationale Supérieure). In Lyon, I have developed analyses of the linguistic and multimodal resources for the sequential organization of interaction, along with new projects on talk at work.

Over the years, I have had the occasion to spend fruitful research periods at Universidade Estadual de Campinas (1996, 2001, 2006), Universität Bielefeld (1999-2000), Instituto Nacional de Antropologia e Historia Mexico (2003), King’s College London (2004), Universitat Pompeu Fabra and Universitat Autonoma Barcelona (2004, 2006), Max Planck Institute for Psycholinguistics Nijmegen (2005), Helsingin Yliopisto (2006), the Freiburg Institute of Advanced Studies (2009-2010) and Linköping University (2010, 2011). In 2010, I was elected as senior member of the Institut Universitaire de France (IUF).

In January 2012, I have been appointed as Chair in General Linguistics and French Linguistics at the University of Basel. From January 2015 I am also Finnish Distinguished Professor at the University of Helsinki, where I develop a major research project, called “Revisiting Language through Multimodality in Interaction”.


Wednesday, 13 April

9.45     Welcome, Michele Luminati and Philippe Sormani (Istituto Svizzero di Roma)

10.00   Roundtable, presentations

10.30   The integration of temporal and visual aspects in transcripts
Lorenza Mondada (Universities of Basel and Helsinki)

11.30   The graphic transcript and Bogen’s complaint
Eric Laurier (University of Edinburgh)

14.30   Data sessions on the excerpts chosen

17.00   Tools and technologies: artists’ presentations and transcriptional implications
Clélia Dumas (Bern University of Arts), Ryan Hamill (Edinburgh) and Veronika Timashkova (Dutch Art Institute, Arnhem)

19.15    Evening lecture
A visual turn in linguistics? Uses of video for studying language
Lorenza Mondada (Universities of Basel and Helsinki)


Thursday, 14 April

9.00-12.00    Parallel working groups on Modalities, Media, and Trajectories (1)

12.00-14.00   Plenum: presentation and discussion

16.00-19.00   Parallel working groups – Reprise

20.00     Exhibition and discussion of the results


Friday, 15 April

9.00-12.00    Parallel working groups on Modalities, Media, and Trajectories (2)

12.00-13.00    Presentation of NERO Publishing (Lorenzo Micheli Gigotti)

15.00-18.00    Exhibition with workshop results and Plenum discussion

18.00-20.00    Roundtable – Conclusionary remarks and relaunch