A preliminary study exploring the relation between visual prosody and the prosodic components in sign language

Authors

  • Maaten Renckens
  • Leo De Raeve University College Leuven-Limburg
  • Erik Nuyts University College PXL and University of Hasselt
  • María Pérez Mena PXL-MAD School of Arts and Hasselt University
  • Ann Beesemans PXL-MAD School of Arts and Hasselt University

DOI:

https://doi.org/10.34314/vl.v55i1.4604

Keywords:

type design, visual prosody, deaf readers, sign language, expressiveness

Abstract

Type enriched with visual prosody is a powerful tool to encourage expressive reading. Visual prosody adds cues to text to guide vocal variations in loud-ness, duration, and pitch. More vocal variations result in a less monotonous voice and thus more expression. A positive e!ect of visual prosody is known on the voice of normal hearing readers and of signed bilingual deaf readers who developed signed language and spoken language. These deaf readers rely on speech as well as sign language and both modalities can be used interchangeably to compensate each other.

This preliminary study explores visual prosody in text in relation to Flemish Sign Language to see if sign language can be used to explain prosody. We asked deaf readers between 7 and 18 to relate prosodic cues to videos presenting prosodic components of Flemish Sign Language. We found that those readers connect the prosodic cues with the components in Flemish Sign Language as intended. Larger word-spacing cor-relates with a pause between signs, a wider font with a sign with ‘longer du-ration’, a thicker font with more ‘displacement’ in the sign, a raised font with a ‘faster velocity’ in the sign. However, some confusion occurred as participants seemed to extract only two prosodic components in the sign language: both the ‘faster velocity’ and ‘longer duration’ were referred to in terms of 'speed' and were not perceived as separate prosodic components. Participants were confused about why there were three cues in the text. Therefore, it is advised to re-evaluate and to re-design visual prosody for sign language with only ‘displacement’ and ‘speed’ in mind.

Author Biographies

  • Maaten Renckens

    Maarten Renckens is a teacher and design researcher with a love for letters and a heart for people. Dealing with a reading diffculty himself, he is very interested in the reading process. His projects include the typeface 'Schrijfmethode Bosch' (Writing Method Bosch) that learns children how to write and typefaces to encourage beginner readers and readers with hearing loss to read more expressively. With a background in architectural engineering, he is used to approach concepts technically and mathematically. He applies this technical knowledge to unravel letterforms, in order to determine the effects of different letterforms on the reading process.

  • Leo De Raeve, University College Leuven-Limburg

    Leo De Raeve PhD has 3 professions: he is a Doctor in Medical Sciences, psychologist and teacher of the deaf. He is founding director of ONICI, the Independent Information and Research Center on Cochlear Implants, is lecturer at University College Leuven-Limburg and scientific advisor of the European Users Association of Cochlear Implant (EURO-CIU).

  • Erik Nuyts, University College PXL and University of Hasselt

    Prof. Dr. Erik Nuyts is researcher and lecturer at the University College PXL and associate professor at the University of Hasselt. He got a master degree in mathematics, and afterwards a PhD in biology.

    Since his specialty is research methodology and analysis, his working area is not limited to one speci$c $eld. His experiences in research, therefore, vary from mathematics to biology, tra%c engineering and credit risks, health, physical education, (interior) architecture and typography.

    His responsibilities both at the University College PXL as at the University of Hasselt involve preparation of research methodol-ogy, data collection and statistical analyses in many di!erent projects. He is responsible for courses in research design, statistics, and mathematics.

  • María Pérez Mena, PXL-MAD School of Arts and Hasselt University

    Dr. María Pérez Mena is an award-winning graphic and type designer. She is postdoctoral researcher at the legibility research group READSEARCH at PXL-MAD School of Arts and Hasselt University. María teaches typography and type design in the BA in Graphic Design at PXL-MAD and is lecturer in the International Master program ‘Reading Type & Typography’ and the Master program ‘Graphic Design’ at the same institution. She received her PhD “with the highest distinction” from University of Basque Country and is a member of the Data Science Institute UHasselt.

  • Ann Beesemans, PXL-MAD School of Arts and Hasselt University

    Prof. Dr. Ann Bessemans is a legibility expert and award-winning graphic and type designer. She founded the READSEARCH legibility research group at the PXL-MAD School of Arts and Hasselt University where she teaches typography and type design. Ann is the program director of the international Master program ‘Reading Type & Typography’. Ann received her PhD from Leiden University and Hasselt University under the supervision of Prof. Dr. Gerard Unger. She is a member of the Data Science Institute UHasselt, the Young Academy of Belgium and lecturer at the Plantin Institute of Typography.

Downloads

Published

2021-09-24

Issue

Vol. 55 No. 1 (2021): April 2021

Section

Journal Article