Gesture as content

Bodily communication is incredibly significant, indeed more significant than we had ever assumed. Bodily communication does not just reveal our emotions and how we feel about another person. It also reveals our hidden thoughts. Hand gestures actually embody our thinking through bodily action. Such behaviours provide us with a glimpse of our hidden unarticulated thoughts. Movements of the hands and arms can act as a window on the human mind. They make thought visible.

Gesture can sometimes be considered as content in addition to the linguistic stream. Here is some analysis of gestures, using Beattie’s (2016) framework, produced by Andrew Marr while speaking on The Jonathon Ross show.

Gesture 1 – ‘huge amounts’ (0:26)

Consider carefully the gesture made starting at 26 seconds. Notice the preparation phase (P1), the stroke phase (S1) and the return phase (R1)

AM: I’d been making the history of the world
AM: which means P1
AM: (.) [S1 huge amounts of] R1 flying around the world

• Preparatory phase: Hand anticipates what is about to be said. It moves to be ready.
• Stroke phase: Hand timed with speech [huge amounts]
• Return phase: Hand returns to resting position

Gesture 2 – ‘really really’ (0:31)

JR: (a big topic)
AM: P2 [S2 really really] R2 knackered

Gesture 3 – ‘writing a book’ (0:33)

AM: [S3 writing a book] R3 and all the rest of it

Note, there is no real preparatory stroke that can be identified here.

Gesture 4 – ‘the thing that happened’ (0:35)

AM: and then (.) ye-
AM: [S4 the thing that happened at the same time]

Gesture 5 – ‘as you say’ (0:38)

AM: R4 was the I used to run
AM: [S5 as you say about twenty five] 25 miles every week

Gestures as content is sometimes termed gesticulation. Not all the above examples are gesticulation. But when it is used it is:

• (1) Synchronous with speech
• (2) Co-expressive with speech and context-sensitive
• (3) Perform the same semantic and pragmatic functions as speech (Kendon, 1980; McNeill 1992);
• (4) The stroke of the gesture, the part with meaning frequently occurs with prosodic peaks (Nobe, 2000); and
• (5) The gesture itself frequently occurs with elements high in communicative dynamism,
  • i.e., contrastive, new, or focused information

•It seems that speakers pick up on crucial semantic aspects of a story and (in certain instances) encode that information into the gestural modality which operates alongside the linguistic modality. This would suggest that gesture, as opposed to speech, is significantly more selective in the information it encodes, tending to restrict itself to information of greater importance. One might tentatively suggest, therefore, that in order to really understand what is important to any speaker, it should be the hands that we attend to rather than the speech, since the gestures they produce seem much more sensitive to the salience of individual elements. (Beattie 2016)

Remaining transcript
erm erm but I knackered my knees
so I had to stop
and as a result
I took up new kinds of exercise
including lethally a rowing machine
keep away from rowing machines everybody
because the great thing about rowing machines
is you can work your body
as hard as you possibly can
and nothing goes pop
until my a- artery went pop
I was trying to hit a really really hard target
in terms of number of miles 
I was rowing a certain amount of time
and I hit the target 
I thought great
and the minute I hit the target
I thought
I’ve done something really really stupid
and then went back
served up the family meal
JR: so hold on
so you so you’d felt something had happened
but you didn’t know it was that serious at the time

I’d no idea what it was 
it just felt this weird sense
of that was really stupid
and I’ve done something silly
but I couldn’t tell what it was
in fact I’d torn my artery into the brain
erm I went back in to the house
served up the family meal
which I normally do
didn’t feel hungry
I had a kind of kaleidoscopic headache

“I will argue that such movements are not part of some system of communication completely divorced from speech, some system of ‘body language’ vs ‘verbal language’, as many psychologists of the past seem to have assumed; rather these bodily movements are intimately connected with speaking and with thinking. … However, this new theory argues for the other side of the argument as well; it suggests that hand gesture communicates semantic information (the traditionally accepted domain of verbal language), acting alongside the speech itself. This new theory argues that these movements of the hands and arms reflect our thinking, like language but in a completely different manner, using a different sort of system of communication with very different properties. … I will attempt to persuade you that such behaviours provide us with a glimpse of our hidden unarticulated thoughts. Movements of the hands and arms act as a window on the human mind; they make thought visible. The fact that they do this without our conscious awareness makes them particularly interesting for both psychologists and the general public in their everyday lives. … The starting point of the book is really the very simple observation that when human beings talk, they make many bodily movements, but in particular they make frequent, and I suggest largely unconscious, movements of the hands and arms. They do this in every possible situation – in face-to-face communication, on the telephone, even when the hands are below a desk and thus out of sight of their interlocutor (I have many recordings of these and similar occurrences). It is as if human beings are neuro biologically programmed to make these movements whilst they talk, that these movements are so important, and they would seem to be (in evolutionary terms) a good deal more primitive than speech itself, with language evolving on the back of these visible movements (or alongside these movements according to McNeill 2012). People who have been blind from birth still gesture even though they have never actually seen gestures themselves, and they continue to gesture even when conversing with other blind people that they know are blind (Iverson and Goldin-Meadow 1997). These gestures are often imagistic in form, and the resultant images are closely integrated in time with the speech itself (on other occasions, however, the movements are simpler than this and appear to be timed with the stress points in the speech). … But on occasion, even when the speech and the hands refer to the same basic concept or idea, they do not match and when there is a clash of this kind (or a ‘mismatch’ as it is called) I will argue that the idea expressed in the hand movement tends to be a much more accurate reflection of our underlying thoughts than the speech itself. And this occurs for one very simple reason, namely that the ‘unconscious meaning unconsciously expressed’ has not been controlled or edited by the speaker to send a particular type of message. This is a recurrent theme of this book. … I noticed, for example, how speakers averted their eye gaze during hesitations in speech when they seemed to be planning what they were going to say, and showed more direct eye gaze when they got to the more fluent sections of their speech (Beattie 1983). It seems to be very difficult to plan what you are going to say and maintain direct eye contact with your interlocutor. It just overloads the cognitive system, the mental planning of the content of speech and the distractions caused by those fast fleeting signals coming back at you, competing for the same cognitive resources. Indeed, I found in my early research that false starts in speech, which are one type of speech error where somebody would start saying something and then stop abruptly, were five times more common in speech when speakers did not avert their eye gaze from their interlocutor when they were planning what they were going to say (Beattie 1983: 64). I had read a lot about the importance of eye gaze and eye contact as a body language signal and what it might mean, but few of the authors of these books started with the premise that if you want to really understand eye gaze as a signal, then you need to recognise that eye gaze is also a channel for receiving information and that because it is a channel it coordinates with the planning requirements of the speech stream. We avert our eye gaze when we are planning speech because otherwise we overload our cognitive capacity. It is simply too demanding to plan fresh ideas in speech and monitor and interpret complex nonverbal signals at the same time (Beattie 1981). So any serious analysis of eye gaze and eye contact should perhaps start with how it is affected by what is being said, in other words the relationship between speech and eye gaze. … p15 The above example supports some of the observations made by some former colleagues at the University of Manchester (Rowbotham et al. 2012) who analysed people talking about a variety of recent pain experiences. They found that gestures were produced at a rate of 7.7 gestures per 100 words, which tells us something about how common these are in talk of this kind. In their sample of 757 gestures that they scored as carrying some meaning, they found that the gestures accounted for 57 per cent of all the information conveyed (that is to say, more than half). Significantly more information about the location and the size of the pain (‘small’, ‘localised pain’, ‘more widespread pain’, etc.) was represented in the gestures than in the speech, whereas the speech tended to represent information about other dimensions, including the intensity, the effects, the duration, the cause and the awareness of the pain. The only category of information that was represented significantly more in gesture and speech together than in either modality alone was the quality of the pain, suggesting in the words of the authors that ‘neither modality is sufficiently able to provide a complete representation of the information’”

Beattie 2016

Beattie, G. (2016) Rethinking Body Language: How Hand Movements Reveal Hidden Thoughts. Routledge

Rowbotham, S., Holler, J., Lloyd, D. and Wearden, A. (2012). How do we communicate about pain? A systematic analysis of the semantic contribution of co-speech gestures in pain-focused conversations. Journal of Nonverbal Behavior 36: 1–21.

Backup video