Syndicate content

Some People Hear What They See—"Visual Ear” May Occur in Up to 20% of All People, More Common in Musicians Than Others; Disinhibition of Pre-Existing Neural Cross-Connections May Explain Why Some Hear the “Skipping Pylon” GIF

A synesthesia-like effect in which people “hear” silent flashes or movement, such as in popular “noisy GIFs” and memes, could be due to a reduction of inhibition of signals that travel between visual and auditory areas of the brain, according to a new study led by researchers at City, University of London. (Synesthesia is the production of a sense impression relating to one sense or part of the body by stimulation of another sense or part of the body.) The new study is the first to provide insight into the brain mechanisms underpinning such auditory sensations also known as a “visually-evoked auditory response” (also known as vEAR or “visual ear”). Whilst one theory is that areas of the brain responsible for visual and auditory processing normally compete, this research suggests that they may actually cooperate in people who report “visual ear.” It was also found that musicians taking part in the study were significantly more likely to report experiencing visual ear than non-musician participants. This could be because musical training may promote joint attention to both the sound of music and the sight of the coordinated movements of the conductor or other musicians. Dr Elliot Freeman, Principal Investigator on the study and a Senior Lecturer in Psychology at the University said: "We already knew that some people hear what they see. Car indicator lights, flashing neon shop signs, and people's movements as they walk may all trigger an auditory sensation. Our latest study reveals normally-occurring individual differences in how our senses of vision and hearing interact. We found that people with 'visual ears' can use both senses together to see and also 'hear' silent motion, while, for others, hearing is inhibited when watching such visual sequences."

Some neuroscientists believe “visual ear” may be a type of synesthesia, with other examples including music, letters, or numbers that can evoke perceptions of color. However, “visual ear” appears to be the most prevalent, with as many as 20% of people reporting some experiences of it compared to 4.4% for other types.

The condition has received more attention due to the recent, viral popularity of the “skipping pylon GIF” (https://twitter.com/i/status/937105553968566272) and other “noisy GIFs” depicting silent motion, which, in some people, evoke very vivid “visual ear” sensations.

To shed light on what may be going on in the brain when people view such content, the researchers applied a weak alternating current to participants' scalps, using a technique called transcranial Alternating Current Stimulation (tACS), to explore how the visual and auditory parts of the brain interact in those who experience “visual ear” and those who don't.

The first experiment of the study included 36 healthy participants, including 16 classical musicians from the London Royal College of Music. All were shown auditory and visual “Morse code” sequences, while electrical simulation (tACS) was applied to either the back of the head (visual areas of the brain) or the sides (auditory areas) using “alpha-frequency” tACS stimulation. Participants were then classified as “visual” or “non-visual” ear, depending on whether they reported “hearing” the silent flashes.

VISUAL & AUDITORY BRAIN AREAS SEEM TO COMPETE IN THOSE WITH “NON-VISUAL EAR”

The researchers found that in “non-visual ear” participants, alpha-frequency stimulation to auditory areas significantly reduced auditory performance, but improved visual performance, while the opposite pattern was found for the same frequency of stimulation to visual areas (poorer vision, better audition).

This reciprocal pattern suggests a competitive interaction between visual and auditory brain areas, with each normally inhibiting the performance of the other.

VISUAL & AUDITORY AREAS SEEM TO COOPERATE IN THOSE WITH “VISUAL EAR”

However, these interactions were strikingly absent in “visual ear” participants, suggesting that their auditory and visual areas were not competing, but cooperating, with each other.

SECOND EXPERIMENT

A second experiment was conducted to see whether even people without conscious awareness of “visual ear” sometimes use their auditory brain areas to make purely visual judgements. The results of this experiment suggested that this might indeed be the case for some, where stimulation to auditory areas of the brain affected accuracy of visual judgements almost as much as stimulating visual areas.

RESULTS SUGGEST DISINHIBITION OF PRE-EXISTITING NEURAL CROSS-CONNECTIONS MAY BE KEY

Taken together, the results of these experiments support a popular theory that some kinds of synaesthesia may depend on a disinhibition of pre-existing neural cross-connections between sensory brain areas that are normally inactive. When these connections are disinhibited this may result in conscious awareness of “visual ear” and other synesthetic phenomena.

BETTER TASK PERFORMANCE BY THOSE WITH “VISUAL EAR,” POSSIBLE BENEFITS FOR MUSICAL PERFORMANCE

Dr Freeman said: "We were also interested to find that, on average, participants with ‘visual ear’ performed better on both visual and auditory tasks than those without. Perhaps their audio-visual cooperation benefits performance because more of the brain is engaged in processing visual stimuli.

"Such cooperation might also benefit musical performance, explaining why so many of the musicians we tested reported experiencing ‘visual ear.’"

ARTICLE DESCRIPTION

The article describing this new research was published online on March 18, 2019 in the Journal of Cognitive Neuroscience. The article is titled “Hearing Through Your Eyes: Neural Basis of Audiovisual Cross-activation, Revealed by Transcranial Alternating Current Stimulation.”

QUESTIONNAIRE AVAILABLE FOR FURTHER RESEARCH

Dr. Freeman is continuing his research in this field with a new online questionnaire, which anyone can complete as a participant (https://cityunilondon.eu.qualtrics.com/jfe/form/SV_1XrRAjC74ScbklT).

[Press release] [Journal of Cognitive Science abstract]

[Watch the "Skipping Pylon" GIF] [Watch the "Bam Ba, from the Flintstones" GIF]