LSU Computer Science Professor Taking a “Hands-on” Approach

LSU Computer Science Professor Taking a “Hands-on” Approach

BATON ROUGE, La., March 10, 2022 (GLOBE NEWSWIRE) — As smartphones have become more sophisticated over the years, so have the security measures that come with them. Simple passwords have been replaced by fingerprints and facial recognition. However, these methods do not solve the notification privacy issue.

For example, sharing your phone with a friend, family member, or someone else, or even briefly leaving it on a nearby surface, could expose your privacy in the form of an incoming call, e- email, a reminder or an app notification. Existing iOS Guided Access and Android Multi-Account features have been tested to resolve this issue, but were unsuccessful.

Chen Wang, assistant professor of computer science at LSU, thinks he may have the answer. Specifically, he is working with third-year doctoral student Long Huang on a hand verification method that ensures the correct user is holding the smartphone before viewing potentially sensitive content. Their recent article on this topic was published at Mobicom 2021, the annual international conference on mobile computing and networking. A short demo can be viewed here.

When a notification tone plays, the phone’s microphone records the sound. An AI-based algorithm processes sound and extracts biometric features to match the user’s feature profile or registered grip. If there is a match, the verification is successful and the notification preview is displayed on the screen. Otherwise, only the number of pending notifications is displayed.

“We view this as an attempt at security design to embrace the art,” said Wang, whose expertise is in cybersecurity and privacy, mobile sensing and computing, wireless communications, among other areas. “We find that when we play music with a phone, our hands often feel the beats, which are caused by the vibrations of the surface of the phone. This is a way in which the sound of music conveys information to us. Because the sounds in music are signals, they can be absorbed/attenuated, reflected, or refracted by our hands.

“We then use the phone’s mic to capture the remaining sounds to see how we react to the music. Because people have different hand sizes, finger lengths, holding strengths, and hand shapes, the impacts on sounds are different and can be learned and distinguished by AI. Along the way, we are developing a system to use notification tones to verify gripping hand for notification privacy protection. This is very different from previous acoustic detection work, which all relies on dedicated sounds that are inaudible or distracting to human ears.

The project is one of two supported by the Louisiana Board of Regents that Chen is working on involving users’ smartphones and hands. The other, in collaboration with Ruxin Wang, a second-year doctoral student, and Kailyn Maiden, a computer science graduate, uses the back of the user’s hand to grab the phone for verification purposes in kiosks, such as those used to order food, print tickets and self-checkout at the grocery store. This research will be published as a late-breaking work at the ACM CHI 2022 Conference on Human Factors in Computing Systems. A short demo can be viewed here.

“When a user holds [his or her] phone near the kiosk for NFC or QR code authentication, the back of the user’s hand is captured by a camera on the kiosk,” Wang said. “An AI-based method will process the gripping hand image and compare it to the user’s registered hand image by checking the shape of the gripping hand, skin patterns/colors and the grip gesture. Note here that the user’s identity has been claimed by traditional NFC or QR-code methods when they pass the user’s security token. Thus, here we provide two-factor authentication at the kiosk: the security token and the biometrics of the geometry of the gripping hand.

Wang added that he and the students were improving authentication systems and conducting user studies with more participants and device methods. They also examine factors that impact the practical use of these systems, including ambient noise and lighting conditions. Additionally, they are investigating potential attacks, for example, a fake 3D-printed silicone hand and acoustic replay attacks.

Wang plans to commercialize these techniques in three years.

Like us on Facebook (@lsuengineering) or follow us on Twitter and Instagram (@lsuengineering).​

  • Kiosk System Demo

  • Grip demonstration