Research Article: Sensory substitution: The affordance of passability, body-scaled perception, and exploratory movements

Date Published: March 27, 2019

Publisher: Public Library of Science

Author(s): Carlos de Paz, David Travieso, Jorge Ibáñez-Gijón, Miguel Bravo, Lorena Lobo, David M. Jacobs, Thomas A. Stoffregen.


The theory of affordances states that perception is of environmental properties that are relevant to action-capabilities of perceivers. The present study illustrates how concepts and methodological tools from the theory of affordances may help to advance research in the field of sensory substitution. The sensory substitution device (SSD) that was used consisted of two horizontal rows of 12 coin motors that each vibrated as a function of the distance to the nearest object. Sixty blindfolded participants used the SSD to explore virtual horizontal apertures with different widths. They were asked to judge the passability of the apertures. Participants with narrow shoulders judged narrower apertures as passable than participants with wide shoulders. This difference disappeared when aperture width was scaled to shoulder width, demonstrating that perception was body scaled. The actual aperture width was closely related to aspects of the exploratory movements and to aspects of the vibrotactile stimulation that was obtained with the exploratory movements. This implies that the exploratory movements themselves and the vibrotactile stimulation were both informative about the aperture width, and hence that the perception of passability may have been based on either of them or on a global variable that spans vibrotactile as well as kinaesthetic stimulation. Similar performance was observed for participants who accomplished the 7-trial familiarization phase with or without vision, meaning that practice with vision is not indispensable to learn to use the SSD.

Partial Text

Sensory substitution devices (SSDs) are devices that substitute one perceptual modality (usually vision) with another one (usually hearing or touch) [1–7]. A substantial number of SSDs and experiments with SSDs have been oriented toward the recognition of objects and the perception of properties of objects such as their shape or size [8–13]. Our research is more aligned with SSD-based studies that address action-related tasks such as locomotion toward targets [14, 15], collision avoidance [16, 17], stepping on ground-level objects [18], or the perception of the direction and location of objects [19–21]. Relatedly, we believe that it would be fruitful to base research in the field of sensory substitution on the inherently action-related theory of affordances. Affordances are defined as properties of the environment that provide possibilities for action to a given organism [22–25]. The concept of affordance is a cornerstone of the ecological approach to perception and action, which was initiated by Gibson [22]. The ecological approach claims that, rather than abstract, organism independent properties of the environment, individuals perceive affordances, which is to say, they perceive what actions the environment affords them.

The present study applied the theory of affordances in the field of sensory substitution. Our first purpose was to demonstrate the body-scaled nature of SSD-based perception, using the perception of passability as task. We analyzed two groups of individuals, with narrow and wide shoulders. Those with narrow shoulders judged narrower apertures as passable than those with wide shoulders. This difference disappeared when aperture width was scaled to shoulder width. The critical value of the dimensionless number, πc, defined as the value that leads to 50% of affirmative judgments, did not differ for the groups with narrow and wide shoulders. Furthermore, the overall πc value of 1.22 that we obtained for SSD-based perception was not significantly different from the value of 1.16 that Warren and Whang [35] reported for the visual perception of passability. The standard interpretation of such findings is that perception is scaled to the anthropometric dimensions of perceivers (e.g., [26, 29, 31–34, 43]).