SonarSnoop Acoustic Side-Channel Attack Can Steal Touchscreen Interactions

[nir]

A new academic study shows that a regular smartphone can act as a sonar system and steal sensitive information based on the victim's finger movement on the screen.

 

Researchers from Lancaster and Linköping University set out to capture the unlock pattern of an Android phone (Samsung S4) using the principle of a sonar: emitting sound waves and catching their echoes produced as they bounce off nearby objects.

Phone works as a sonar

Named SonarSnoop, the framework they developed relies on the phone's speakers to issue acoustic signals and its microphones to catch the reflections. In this regard, SonarSnoop is the first active acoustic side-channel attack because it does not wait for the victim to generate the acoustic signal.

 

The researchers explain that the speakers send an orthogonal frequency-division multiplexing (OFDM) signal at a frequency inaudible to most humans (18-20kHz), so the user remains unaware of the audio activity.

 

When the objects are static, all returning echoes arrive at the same time, and a difference is observed when the finger moves on the screen.

 

"The received signals are represented by a so-called echo profile matrix which visualizes this shift and allows us to observe movement. Combining observed movement from multiple microphones allows us to estimate strokes and inflections," the study clarifies.

Analyzing movement on the screen

The next stage after generating the signal and collecting the data is is to process the signal. This step takes into consideration the position of the microphones on the device and aims to clear the artifacts that may interfere with data analysis.

 

The data collected and processed this way can be interpreted into a meaningful unlock pattern. The methods used by the researchers involved analyzing the direction and range of each stroke to create the right sequence.

Strokes with different direction information

Results of the test

The team used 12 unlock patterns in their tests, with 15 unique strokes. All the data collected from the 10 volunteers involved in the study was then fed into a machine learning model for classification of each stroke.

 

As expected, the value for the classification accuracy was higher with input from both microphones.

 

However, the purpose of the experiment was to combine the strokes into the unlock pattern traced by the volunteer.

 

Sample patterns

It is important to note that the SonarSnoop framework provides candidates for the pattern rather than the exact drawing.

 

Using multiple methods of analysis, the researchers managed to reduce the average number of correct candidates to 3.6 patterns. In some instances, the analysis eliminated all guesses and revealed the correct pattern.

The study is purposefully limited in scope, as the researchers are aware that the framework can be improved to take into consideration additional variables.

 

"For convenience and simplicity, we do not implement the system to cope with different users interaction speeds. We use a fixed column width of the echo profile matrix to determine if there is movement," the paper explains.

 

Also, the methods they used to produce a relevant pool of candidate patterns were sufficient to demonstrate the feasibility of an active acoustic side-channel attack; but they could be improved and are not the only methods possible.

Leveraging the sonar capabilities for evil

SonarSnoop opens the door to new attack scenarios that combine different components like sensors and cameras with the active sonar. Tracking other types of information (passwords, messages) on the phone screen is a possibility, as well. 

 

An adversary using this side-channel attack could add the malicious code to an app with permission to access the microphone. This is achievable with an app that offers sound effects or voice control.

 

Exfiltrating the data from the phone should not be a problem, either, since all apps are expected to communicate with a server.

 

For the moment, the framework comes with a limitation, though, which should make it unattractive to cybercriminals: it has to be adapted per phone model.

 

SonarSnoop was inspired by FingerIO, a highly accurate finger tracking solution that turns the space around smartphones and smartwatches into an interactive surface represented on the device's screen.

 

 

Previous studies into the privacy implications of tracking human movement using acoustics have also been used; one example is CovertBand.

 

Full details on how SonarSnoop works are offered in a paper called "SonarSnoop: Active Acoustic Side-Channel Attacks" available here.

Source