Scientists are turning data into sound to listen to the whispers of the universe (and more)

GettyImages/ Mark Newman

Data sonification converts data into sound. It has powerful applications in research, education and outreach, and it also enables blind and visually impaired communities to understand plots, images and other data.

We often think of astronomy as a visual science with beautiful images of the universe. However, astronomers use a wide range of analysis tools beyond images to understand nature at a deeper level.

Data sonification is the process of converting data into sound. It has powerful applications in research, education and outreach, and also enables blind and visually impaired communities to understand plots, images and other data.

Its use as a tool in science is still in its early stages – but astronomy groups are leading the way.

In a paper published in Nature Astronomy, my colleagues and I discuss the current state of data sonification in astronomy and other fields, provide an overview of 100 sound-based projects and explore its future directions.

The cocktail party effect

Imagine this scene: you’re at a crowded party that’s quite noisy. You don’t know anyone and they’re all speaking a language you can’t understand – not good. Then you hear bits of a conversation in a far corner in your language. You focus on it and head over to introduce yourself.

While you may have never experienced such a party, the thought of hearing a recognizable voice or language in a noisy room is familiar. The ability of the human ear and brain to filter out undesired sounds and retrieve desired sounds is called the “cocktail party effect”.

Similarly, science is always pushing the boundaries of what can be detected, which often requires extracting very faint signals from noisy data. In astronomy we often push to find the faintest, farthest or most fleeting of signals. Data sonification helps us to push these boundaries further.

The video below provides examples of how sonification can help researchers discern faint signals in data. It features the sonification of nine bursts from a repeating fast radio burst called FRB121102.

Casey Law/Youtube.

Fast radio bursts are millisecond bursts of radio emission that can be detected halfway across the universe. We don’t yet know what causes them. Detecting them in other wavelengths is the key to understanding their nature.

Too much of a good thing

When we explore the universe with telescopes, we find it’s full of cataclysmic explosions including the supernova deaths of stars, mergers of black holes and neutron stars that create gravitational waves, and fast radio bursts.

Here you can listen to the merger of two black holes.

LIGO/YouTube.

And the merger of two neutron stars.

LIGO/YouTube.

These events allow us to understand extreme physics at the highest-known energies and densities. They help us to measure the expansion rate of the universe and how much matter it contains, and to determine where and how the elements were created, among other things.

Upcoming facilities such as the Rubin Observatory and the Square Kilometre Array will detect tens of millions of these events each night. We employ computers and artificial intelligence to deal with these massive numbers of detections.

However, the majority of these events are faint bursts, and computers are only so good at finding them. A computer can pick out a faint burst if it’s given a template of the “desired” signal. But if signals depart from this expected behavior, they become lost.

And it’s often these very events that are the most interesting and yield the biggest insight into the nature of the universe. Using data sonification to verify these signals and identify outliers can be powerful.

More than meets the eye

Data sonification is useful for interpreting science because humans interpret audio information faster than visual information. Also, the ear can discern more pitch levels than the eye can discern levels of colour (and over a wider range).

Another direction we’re exploring for data sonification is multi-dimensional data analysis – which involves understanding the relationships between many different features or properties in sound.

Plotting data in ten or more dimensions simultaneously is too complex, and interpreting it is too confusing. However, the same data can be comprehended much more easily through sonification.

As it turns out, the human ear can tell the difference between the sound of a trumpet and flute immediately, even if they play the same note (frequency) at the same loudness and duration.

Why? Because each sound includes higher-order harmonics that help determine the sound quality, or timbre. The different strengths of the higher-order harmonics enable the listener to quickly identify the instrument.

Now imagine placing information – different properties of data – as different strengths of higher-order harmonics. Each object studied would have a unique tone, or belong to a class of tones, depending on its overall properties.

With a bit of training, a person could almost instantly hear and recognise all of the object’s properties, or its classification, from a single tone.

Beyond research

Sonification also has great uses in education (Sonokids) and outreach (for example, SYSTEM Sounds and STRAUSS), and has widespread applications in areas including medicine, finance and more.

But perhaps its greatest power is to enable blind and visually impaired communities to understand images and plots to help with everyday life.

It can also enable meaningful scientific research, and do so quantitatively, as sonification research tools provide numerical values on command.

This capability can help promote STEM careers among blind and visually impaired people. And in doing so, we can tap into a massive pool of brilliant scientists and critical thinkers who may otherwise not have envisioned a path towards science.

What we need now is government and industry support in developing sonification tools further, to improve access and usability, and to help establish sonification standards.

With the growing number of tools available, and the growing need in research and the community, the future of data sonification sounds bright!

The Conversation

Jeffrey Cooke, Professor, Centre for Astrophysics and Supercomputing, Swinburne University of Technology

This article is republished from The Conversation under a Creative Commons license. Read the original article.

NEXT STORY: Smart cities grow up

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.