Many Minds
A new picture of language
Episode guests
Podcast summary created with Snipd AI
Quick takeaways
- Language is enhanced by a multimodal approach, integrating vocal, bodily, and graphic modalities for richer communication.
- Dr. Neil Cohn asserts that graphic systems like comics deserve recognition as full languages with their own structures and meanings.
- The belief in an inability to draw reflects a lack of exposure to graphic languages, not an innate lack of talent.
- Cohn's critique of traditional language evolution emphasizes the importance of recognizing all modalities in understanding human communication.
Deep dives
The Nature of Language and Its Modalities
Language is characterized as a system for conveying meaning through various modalities, including vocal, bodily, and graphic forms. Traditional views, particularly influenced by 19th-century linguistics, suggest that the relationship between words and their meanings is arbitrary, a perspective that excludes the rich capabilities found in graphic and gestural communication. This blog challenges that long-standing view, proposing that each modality serves as a full linguistic system in itself, capable of expressing complex ideas and meanings. The assertion emphasizes that a multimodal approach is essential for understanding the complete spectrum of human language.
The Case for Visual Language
Dr. Neil Cohn argues that graphic systems, such as comics, function as full-blown languages with their own grammatical structures and expressive capabilities, challenging traditional assumptions in linguistic theory. This viewpoint is reinforced by the argument that different modalities are optimized for various types of meaning-making, with graphic modalities particularly effective for iconicity. By examining cultural artifacts and systems that have evolved over time, Cohn demonstrates that graphics can parallel spoken language in their complexity and structure. This perspective elevates the importance of visual languages alongside more recognized forms of communication.
Dissecting the 'I Can't Draw' Phenomenon
When people assert they cannot draw, it often reflects a misunderstanding of what drawing truly entails. Rather than an innate talent, drawing is presented as a skill that involves acquiring a visual vocabulary akin to learning a spoken language. Cohn challenges the notion that drawing is merely about replicating visuals from the mind, suggesting it instead involves systematic patterns and configurations. He posits that the belief in an inherent inability to draw stems from not having learned these graphic languages, much like individuals may struggle with sign languages if they have not formally studied them.
Cross-Cultural Insights in Graphic Communication
The discussion spans various graphic traditions, exploring cross-cultural similarities and differences in visual storytelling through comics and other mediums. Cohn highlights how comic systems have transformed over the past century, reflecting evolving societal norms and technological advancements. He notes that different cultures exhibit distinct styles while still adhering to universal principles of visual language, underscoring the complexity and richness of graphic systems globally. This exploration emphasizes the need for a global perspective in understanding how graphic communication develops and changes.
The Interplay of Sign Language and Gestures
Sign languages are acknowledged as full-fledged languages, playing a crucial role in the broader understanding of human communication. Cohn emphasizes the significance of bodily gestures in conveying meaning alongside spoken language, revealing how these modalities exist in a complementary relationship. The concept of homesign systems exemplifies how individuals can develop their visual communication styles in the absence of formal sign language exposure. Such insights reinforce the idea that every modality—be it gesture, speech, or graphics—plays a vital role in the intricate tapestry of human expression.
Reflections on the Evolution of Language
Cohn critiques traditional evolutionary theories of language that often prioritize speech as the primary modality, arguing that this perspective overlooks the rich evolution of multimodal communication. The assertion is made that language evolution should incorporate all modalities, recognizing their interdependence and transformation throughout human history. By proposing that the development of various communication systems has operated concurrently, Cohn posits a more inclusive view that embraces the natural evolution of human expressive capacity across multiple frameworks. His approach challenges long-held assumptions and invites a reevaluation of how we think about language across time.
Understanding the Cognitive Frameworks Behind Language
The discussion includes exploring the cognitive frameworks that underlie how humans process and produce language, whether spoken, signed, or graphic. Cohn indicates that language operates not merely through symbolic means but also through indexical and iconic representations, enriching communication's complexity. The examination of iconic drawing techniques reveals the innate abilities humans possess, urging the recognition of graphic modalities as fundamental components of language production. Embracing these frameworks promotes a more multifaceted understanding of linguistic cognition and communication.
Implications for Future Language Studies
Cohn's work underlines the vital necessity for linguists and cognitive scientists to adopt a multimodal perspective when studying language. By treating every expressive modality as integral, future studies can benefit from a richer understanding of human communication patterns. This shift encourages collaborative efforts between disciplines such as linguistics, cognitive science, and visual studies to deepen insights into how languages evolve and function. Overall, Cohn's arguments advocate for a reassessment of language's role in human society, pushing for broader inclusion of varied communicative forms.
If you've taken Linguistics 101, you know what language is. It's a system for conveying meaning through speech. We build words out of sounds, and then complex ideas out of those words. Remarkably, the relationship between the sounds and the meanings they convey is purely arbitrary. Human language consists, in other words, of abstract symbols. Now, of course, there are also sign languages, but these operate in the same way, just in a different medium. This, anyway, is the view of language that has dominated and defined linguistics for many decades. But some think it gets some pretty fundamental things pretty wrong. Some think we need a new picture of language altogether.
My guest today is Dr. Neil Cohn. Neil is Associate Professor at the Tilburg Center for Cognition and Communication, in the Netherlands; he is also the director of the Visual Language Lab at Tilburg. For about two decades, Neil has been studying the rich properties of graphic systems—especially comics—and has built an argument that some constitute full-blown languages. His latest book, co-authored with, Joost Schilperoord, is titled A Multimodal Language Faculty. It challenges that longstanding, deeply held view of what language is. Instead, the book argues that the human language capacity combines three different modalities—the vocal modality (as in speech), the bodily modality (as in gesture), and the graphic modality (as in comics and other visual narratives). And each of these modalities is naturally able to support full-blown languages.
Here, Neil and I talk about the basic assumptions of modern linguistics and where those assumptions come from. We discuss the idea that there are three expressive modalities that come naturally to humans, with each modality optimized for certain kinds of meaning. We talk about Neil's career, not only as an academic, but as an illustrator. We discuss cross-cultural differences and similarities in comics, and how comics have changed over the last century. And, finally, we consider how Neil's framework challenges current theorizing about the evolution of language. Along the way, Neil and I touch on sign languages and homesign systems, visual style vs visual language, Peircean semiotics, animal tracks, cave art, emoji, upfixes, sand drawing, Manga, the refrain "I can't draw," and the idea that the graphic modality is the only one that's truly unique to our species.
After this episode we'll be taking a bit of a summer break, but we'll be posting some old favorites to tide you over. Alright friends, hope you enjoy this one. On to my conversation with Neil Cohn. Enjoy!
A transcript of this episode is available here.
Notes and links
3:30 – An earlier paper by Dr. Cohn on the well-worn refrain “I can’t draw.” His more recent Twitter thread covering the topic.
9:00 – An overview of research on homesign systems. For a broader discussion of differences between gesture, homesign systems, and established sign languages, see here.
15:00 – A comic, ‘Chinese Room,’ commissioned by the philosopher Dan Dennett and drawn by Dr. Cohn.
19:30 – The webpage of Dr. Cohn’s graduate mentor, Ray Jackendoff.
25:00 – A brief overview paper by Dr. Cohn and Dr. Schilperoord on the need to “reimagine language.”
25:30 – The classic book, based on lecture notes, by Ferdinand de Saussure, ‘Course in General Linguistics.'
44:00 – For an overview of “bimodal bilingualism,” see here.
50:00 – A study by Dr. Cohn and colleagues on the processing of emoji substituted for words.
56:00 – A recent study by Dr. Cohn and colleagues on anaphora in visual narratives.
58:30 – For our previous audio essay on animal (and human) tracks, see here.
1:01:30 – For examples of scholarship on non-Western methods of visual storytelling, including Aboriginal Australian sand drawing, see Dr. Cohn’s earlier edited volume here. For a deeper dive into sand drawing, see the monograph by Jenny Green here.
1:03:00 – Dr. Cohn also recently published a book on cross-cultural aspects of comics, The Patterns of Comics. The book is the fruit of his lab’s TINTIN project.
1:11:00 – For a video of Aboriginal Australian sand drawing, see here.
1:13:00 – See Dr. Cohn’s earlier book, Who Understands Comics?
1:15:00 – A study on “upfixes” by Dr. Cohn and a colleague.
1:22:00 – A popular article by Dr. Cohn on the linguistic status of emoji.
1:31:00 – For a deep dive into Peircean semiotics, see here.
1:36:00 – For my own general-audience treatment of “gesture first” theories of language evolution and the “modality transition” problem, see here.
1:37:00 – A paper by Dr. Jackendoff and Eva Wittenberg outlining their “complexity hierarchy.”
1:50:00 – For the Getty museum exhibit associated with Dr. Cohn’s lecture, see here.
Recommendations
The Texture of the Lexicon, by Jenny Audring and Ray Jackendoff
Battle in the Mind Fields, by John Goldsmith and Bernard Laks
History and Philosophy of the Language Sciences podcast, hosted by James McElvenny
Many Minds is a project of the Diverse Intelligences Summer Institute, which is made possible by a generous grant from the Templeton World Charity Foundation to UCLA. It is hosted and produced by Kensy Cooperrider, with help from Assistant Producer Urte Laukaityte and with creative support from DISI Directors Erica Cartmill and Jacob Foster. Our artwork is by Ben Oldroyd. Our transcripts are created by Sarah Dopierala.
Subscribe to Many Minds on Apple, Stitcher, Spotify, Pocket Casts, Google Play, or wherever you listen to podcasts. You can also now subscribe to the Many Minds newsletter here!
We welcome your comments, questions, and suggestions. Feel free to email us at: manymindspodcast@gmail.com.
For updates about the show, visit our website or follow us on Twitter: @ManyMindsPod.
Remember Everything You Learn from Podcasts
