“Language has to be realized through, and comes in the company of, other semiotic modes” (Gunther Kress and Theo Van Leeuwen, 1998, p. 186)

A new path for linguists

Printed material, such as advertisements, manuals, instruction books, maps, graphics, and cartoons usually depend on the interaction between verbal and pictorial/visual modes to convey messages and information. The interaction of those different semiotic modes to make meaning is described as multimodality, and hence a multimodal theory of communication has been established. While the emphasis on the verbal-visual interaction is not new, its contribution to the field of linguistics has been recently developed. A linguist whose concern has for a long time been on verbal language, either written or spoken, is now better able to analyse the language of advertisements, the meaning of which is also communicated through visual features; and examine a news text accompanied by an image or a photograph. However, Jewitt (2009) provided a definition of multimodality that focused on the role of different semiotic modes (verbal, visual, and audio/visual) in achieving meaningful communication:

Multimodality describes approaches that understand communication and representation to be more than about language, and which attend to the full range of communicational forms people use – image, gesture, gaze, posture and so on – and the relationships between them).

The following is a good example where visual and verbal elements form part of the cartoon’s meaning:

In the above cartoon, a couple is depicted as offering a gift for each other while saying the same expression. However, the material value of this same expression ‘All this 4 u’ differs in terms of the associated visual choices. It is important to consider the meaning of the predeterminer ‘All’, the whole of something, to grasp the intended meaning of the cartoon. The man’s offer which is represented visually as ‘a small tree’ does not match the presumed meaning of ‘All’ in his utterance if it is compared to ‘the Earth’ in his right hand. Moreover, the choice of the tree to represent the man’s offer stands for the proportion of his sacrifice which is very little in comparison to the woman’s sacrifice ‘the Earth’. In other words, the wife has approached the real meaning of ‘All’ by sacrifying the whole whilst the man failed to approach that meaning. The verbal-visual interaction guides readers to come up with contrasts such as, man’s sacrifice vs. woman’s sacrifice, or man’s giving vs. woman’s giving.

Multimodal metaphor

The multimodal theory has inspired other scholars to introduce new terms describing phenomena manifested in our daily life. In texts, such as advertisements and billboards, metaphor may appear not only visually, but also multimodally, that is, a combination of both the pictorial/visual and the verbal mode. Metaphors generally consist of two major elements: the target and the source. The target or abstract concept is understood in terms of the source which is often a concrete object or element. In verbal or visual metaphors the target and the source are represented either only verbally or only visually while in multimodal metaphors the target and the source are exclusively and predominantly represented in two different modes: the visual and the verbal mode.

Like advertisements, cartoons are rich in metaphors through which cartoonists can explain complicated political situations, or sum up and comment on current events and social issues in a particularly humorous manner. Cartoons are a good data source to explore multimodal metaphors since cartoons often rely on visual and verbal modes and metaphor is a common technique frequently used by cartoonists.

The funny side

As we have talked about multimodal metaphors in cartoons, we should bear in mind the humour aspect which is often the ultimate purpose of many cartoonists. Combining both the pictorial and verbal modes, cartoons may be more appealing and even more attractive in conveying the cartoonists’ humorous message. However, there is scholarly agreement that humour appreciation depends on the broader socio-cultural context in which cartoons appear: what seems to be funny and humorous to one person might be viewed as unfunny, or even offensive to another person.

The study

For my PhD project I have collected 202 cartoons from three English-language Saudi newspapers: Arab News, Saudi Gazette, and Al Riyadh, and perform a qualitative multimodal analysis of the cartoons. I have thematically classified the cartoons into four major categories: gender, social phenomena, education and the impact of technology, and economy and prices. The analysis aims to show how visual and verbal elements and choices interact to make humorous meaning, and how themes can affect the cartoonists’ choice of a particular technique in multimodal texts such as cartoons.

For example, the cartoons related to economy and prices have shown a notable use of multimodal metaphors to depict the rise in prices, the cost of living, loan traps, and the stock market. In 28 out of 39 cartoons related to this category, multimodal metaphors were employed creatively. The creativity of those metaphors not only relies on being totally innovative, but also on the cartoonists’ ability to exploit the existing idiomatic or metaphorical expressions used in daily communication (such as figurative expressions and phrasal verbs). Those expressions are visualised in a particularly humorous manner and are accompanied with verbal clues and labels to help readers or viewers recognise the reference to a certain popular expression, or ‘intertextuality’.

Intertextuality in cartoons refers to the cartoonist’s borrowing or quoting from previously existing texts, and to the viewers’ interpreting of the cartoons in the light of those texts. Here are some examples to show how our daily speech and expressions may act as an inspiring source for cartoonists to create humorous multimodal metaphors.

Creating Multimodal Metaphor

Bills absorb ones blood

In the above cartoon, the word ‘Bills’ is significant in identifying the target of the metaphor in the cartoon. It is a multimodal metaphor because the source of the metaphor ‘the mosquito and the blood being absorbed’ is represented visually while the target of the metaphor ‘Bills’ is represented verbally. The word ‘Bills’ implies the money needed to pay bills. Accordingly, the multimodal metaphor can be verbalised as: BILLS CONSUMING ONE’S MONEY IS LIKE A MOSQUITO ABSORBING ONE’S BLOOD. Humorously, the shared victim in both cases is the man.

A reader who is familiar with everyday speech in the Saudi society would have no difficulty accessing the daily metaphoric expression used by the society members to complain about the high bills paid for services: bills absorb one’s blood. The cartoonist relied on the metaphoric expression and his imagination to design the above image. He inserted one essential verbal element ‘bills’ and another essential visual element ‘blood’ as a means to help familiar readers recall the original expression.

Drowning in loans

The above cartoon depicts a man in the middle of the sea putting on a snorkeling apparatus for breathing. He is not dressed in swimming clothes, but in his torn thobe as a symbol of his financial problems. The word ‘loans’ is written above the seawater. The cartoon includes a multimodal metaphor as the source ‘sea’ is represented visually while the target ‘loans’ are represented verbally. Again, the man is the victim in both situations: in case of the sea and in case of the loans. The basic metaphor LOANS ARE LIKE A DEEP SEA may also suggest a metaphorical scene based on the image and the associated word: HAVING LOANS IS LIKE DROWNING IN THE SEA.

However, the imaginary scene of ‘the man under the water’ accompanied with the word ‘loans’ evokes the Arabic idiomatic expression ‘to be drowning in debts/loans’ which is used to describe someone involved in debts and bank loans. In other words, the cartoonist exploited the literal meaning of the word ‘drowning’ to create the metaphorical scene while referring to the idiomatic expression through giving the verbal clue ‘loans’. The expression has an English equivalent ‘drown in something’ which means ‘to have more of something than you are able to deal with’. A typical example is ‘I’m drowning in unpaid bills’ (Cambridge Online Dictionary).

Big fish eats small fish

The above cartoon shows a hierarchical order of eight colourful fish: the big ones eat the little ones. It is a typical scene of marine life where the weak creatures fall victims to large creatures. The words written on the four fish are economic terms in the world of trading. Those economic terms are represented in a hierarchical order corresponding to the size of the fish. The cartoon includes multimodal metaphor since the source ‘big and small fish’ is depicted visually while the target is represented verbally through the words ‘supplier’, ‘wholesaler’, ‘retailer’, and ‘consumer’. THE SUPPLIER, THE WHOLESALER, THE RETAILER, AND THE CONSUMER ARE LIKE FISH IN THE SEA WHERE BIG ONES EAT LITTLE ONES.

The hierarchical order suggests that the supplier gains profits from the wholesaler who gains profits from the retailer who finally gains profits from the consumer. The consumer is the victim and the weakest of the four who gets no benefits, but pays heaps of money instead. Without the words, we would neither know the metaphor, nor understand the intended meaning of the cartoon. However, the metaphor is based on a common metaphorical expression ‘BIG FISH EATS SMALL FISH’ used in daily communication to describe the world of trading and stock market as well.

Prices are like fire

The cartoon displays a scene in which some green notes are depicted as a moving creature jumping into flames of fire. As the flames of fire are identified as ‘Ramadan prices’, the green notes stand for money. However, the cartoon includes a multimodal metaphor as the source ‘fire’ is depicted visually whereas the target ‘Ramadan prices’ is represented verbally. The metaphor can be verbalised as RAMADAN PRICES ARE LIKE FIRE. It suggests that the high prices during Ramadan consume money in the same way as fire burns objects. The month of Ramadan is a profitable season for merchants. During that month, prices reach their peak. Generally speaking, familiar readers can easily recall the very common metaphoric expression PRICES ARE FIRE which is used in daily life to describe the rise in prices. While the cartoon is based on a conventional verbal metaphor, we cannot overlook the cartoonist’s ability to make it humorous by adding imaginary visual elements, such as ‘depicting money as a flying creature jumping into the flaming fire’.

Multimodal metaphors between reality and imagination

Whenever I come across a cartoon that evokes a particular expression in my memory bank, I can feel the power of our daily communication. Our everyday language can either affect our language production as in the case of cartoonists relying on their linguistic repertoire to create multimodal metaphors, or influence our reception as in the case of readers who interpret those metaphors in the light of already existing texts. Such a conclusion draws on Bakhtin’s and Kristeva’s theory about intertextuality: authors do not completely depend on their own minds to create their texts, but rather build them from already existing texts. So, texts cannot be separated from the social and cultural textuality out of which they were born.

While cartoonists rely on the available metaphorical/idiomatic expressions in creating their cartoons, their imagination is a major ingredient in making the cartoon appealing and creative. They use their imagination to add balanced visual and verbal elements to end up creating a multimodal metaphor that might also be appealing to readers who are not familiar with the original expression. However, readers who share a similar social and cultural background often rely on the verbal clues associated with the image not only to laugh at the funny metaphors, but also to enjoy sharing the same source of inspiration with the cartoonist.

Conclusion

On a larger scale, my study falls into two research areas: multimodal research and humour research. It contributes to the growing literature on multimodality in the field of Applied Linguistics. It not only provides insights into how visual and verbal modes interact to make meaning, but also shows how those semiotic modes cannot be isolated from the broader social and cultural context of a particular society. What is more interesting about the above examples is that familiar readers can ‘see what is said’: readers are invited to a kind of visual-verbal communication in which they are unconsciously involved in a process of searching their linguistic repertoire or memory to recall the expression that match the available visual and verbal elements in a cartoon. The study moreover contributes to the body of literature on humour research, particularly in the Middle East since the cartoons are collected from Saudi online newspapers. While we can talk about humour studies in the Arab world, contribution to humour research in the Arabic, particularly the Saudi, context is still urgent.

My thanks to my superb supervisor Dr. Martin Howard for his continuous support, encouragement, and academic guidance.