Avatar screens border-crossers in Arizona
Connecting state and local government leaders
CBP is testing a system that uses speech recognition and voice anomaly-detection software to detect signs that a traveler might be lying.
U.S. Customs and Border Protection is testing an avatar kiosk on the U.S.-Mexican border that can speak English and Spanish and detect signs that a traveler being interviewed could be lying.
The system, developed by researchers at the University of Arizona in conjunction with the National Center for Border Security and Immigration (BORDERS), is being tested at Nogales, Ariz., as a way to help enroll travelers into CBP’s Trusted Traveler program.
The avatar, dark-haired and wearing a tie, uses speech recognition and voice anomaly-detection software to monitor the content of what travelers say and search for changes and variations in their vocal characteristics, according to a report in Scientific American.
Related story:
TSA may have a solution for virtual-striptease body scans
Border patrol officers monitor the conversations with feedback sent wirelessly from the avatar to an iPad. The avatar flags answers with green, yellow or red color codes, and officers can follow up on any in the red range, the article said. The avatar can't tell definitively is someone’s lying, but can relay possible indicators.
The current system is an upgrade of an earlier one tested in March, with improved speech recognition software that added Spanish and was tuned to allow for interruptions in conversation.
CBP expects to test the system on 1,000 interviewees, Scientific American reported, and could add other features to future additions, such as the ability for travelers to have their passports and other documents scanned, and additional sensors such as video analyzers, eye trackers and infrared cameras.
Although those features aren’t in the version CBP is testing, they have been part of the research. In a 2011 report on Arizona University’s Eller College of Management site, researchers talked about some of the elements that go into the avatar kiosk.
At that time, researchers were working with sensors that tracked body movement, voice, pupils and eye movement. In all, they could track 400 to 500 psychophysiological and behavioral cues, said Arizona professor Jay Nunamaker, who is principal investigator and director of BORDERS.
Out of those 500 cues, that version of the kiosk was looking for 15 that can indicate deception, some of which are involuntary. “Humans can maybe control two or three cues at one time, but not all 15,” Nunamaker said.
He and Doug Derrick, a doctoral student working on the program, tested a prototype of the kiosk in the fall of 2011 with European border guards in Poland. The guards built fake bombs, packed them into suitcases and then tried to get past the avatar kiosk. The avatar flagged all of the “bombers,” but also snagged some false positives, which Derrick said indicated they had more work to do.
For now, CBP will test its system using speech and anomaly-detection software with the goal of not only detecting deception but also just speeding up enrollments in the Trusted Traveler program.
BORDERS is a 14-university consortium, funded by the Homeland Security Department and led by Arizona University, that researches border- and immigration-related technologies and processes.