Visual prosody supports reading aloud expressively for deaf readers

Authors

  • Maarten Renckens
  • Leo De Raeve University College Leuven-Limburg
  • Erik Nuyts University College PXL and the University of Hasselt
  • María Pérez Mena PXL-MAD School of Arts and Hasselt University
  • Ann Bessemans PXL-MAD School of Arts and Hasselt University

DOI:

https://doi.org/10.34314/vl.v55i1.4603

Keywords:

type design, visual prosody, prosody, deaf readers, expressive reading, reading comprehension

Abstract

Type is a wonderful tool to represent speech visually. Therefore, it can provide deaf individuals the information that they miss auditorily. Still, type does not represent all the information available in speech: it misses an exact indication of prosody. Prosody is the motor of expressive speech through speech variations in loudness, duration, and pitch. The speech of deaf readers
is often less expressive because deafness impedes the perception and production of prosody. Support can be provided by visual cues that provide information about prosody—visual prosody—supporting both the training of speech variations and expressive reading.

We will describe the influence of visual prosody on the reading expressiveness of deaf readers between age 7 and 18 (in this study, ‘deaf readers’ means persons with any kind of hearing loss, with or without hearing devices, who still developed legible speech). A total of seven cues visualize speech variations: a thicker/thinner font corresponds with a louder/quieter voice; a wider/narrower font relates to a lower/faster speed; a font raised above/lowered below the baseline suggests a higher/lower pitch; wider spaces between words suggest longer pauses.

We evaluated the seven cues with questionnaires and a reading aloud test. Deaf readers relate most cues to the intended
speech variation and read most of them aloud correctly. Only the raised cue is di#cult to connect to the intended speech variation at first, and a faster speed and lower pitch prove challenging to vocalize. Despite those two difficulties, this approach to visual prosody is elective in supporting speech prosody. The applied materials can form an example for typographers, type designers, graphic designers, teachers, speech therapists, and researchers developing expressive reading materials.

Author Biographies

  • Maarten Renckens

    Maarten Renckens is a teacher and design researcher with a love for letters and a heart for people. Dealing with a reading diffculty himself, he is very interested in the reading process. His projects include the typeface 'Schrijfmethode Bosch' (Writing Method Bosch) that learns children how to write and typefaces to encourage beginner readers and readers with hearing loss to read more expressively. With a background in architectural engineering, he is used to approach concepts technically and mathematically. He applies this technical knowledge to unravel letterforms, in order to determine the effects of different letterforms on the reading process.

  • Leo De Raeve, University College Leuven-Limburg

    Leo De Raeve PhD has 3 professions: he is a Doctor in Medical Sciences, psychologist and teacher of the deaf. He is founding director of ONICI, the Independent Information and Research Center on Cochlear Implants, is lecturer at University College Leuven-Limburg and scientific advisor of the
    European Users Association of Cochlear Implant (EURO-CIU).

  • Erik Nuyts, University College PXL and the University of Hasselt

    Prof. Dr. Erik Nuyts is researcher and lecturer at the University College PXL and associate professor at the University of Hasselt. He got a master degree in mathematics, and afterwards a PhD in biology.

    Since his specialty is research methodology and analysis, his working area is not limited to one speci$c $eld. His experiences in research, therefore, vary from mathematics to biology, tra#c engineering and credit risks, health, physical education, (interior) architecture and typography.

    His responsibilities both at the University College PXL as at the University of Hasselt involve preparation of research methodology, data collection and statistical analyses in many di%erent projects. He is responsible for courses in research design, statistics, and mathematics.

  • María Pérez Mena, PXL-MAD School of Arts and Hasselt University

    Dr. María Pérez Mena is an award-winning graphic and type designer. She is postdoctoral researcher at the legibility research group READSEARCH at PXL-MAD School of Arts and Hasselt University. María teaches typography and type design in the BA in Graphic Design at PXL-MAD and is lecturer in the International Master program ‘Reading Type & Typography’ and the Master program ‘Graphic Design’ at the same institution. She received her PhD “with the highest distinction” from University of Basque Country and is
    a member of the Data Science Institute UHasselt.

  • Ann Bessemans, PXL-MAD School of Arts and Hasselt University

    Prof. Dr. Ann Bessemans is a legibility expert and award-winning graphic and type designer. She founded the READSEARCH legibility research group at the PXL-MAD School of Arts and Hasselt University where she teaches typography and type design. Ann is the program director of the international Master program ‘Reading Type & Typography’. Ann received her PhD from Leiden University and Hasselt University under the supervision of Prof. Dr. Gerard Unger. She is a member of the Data Science Institute UHasselt, the Young Academy of Belgium and lecturer at the Plantin Institute of Typography.

Downloads

Published

2021-09-24

Issue

Vol. 55 No. 1 (2021): April 2021

Section

Journal Article