Meet UA’s AVATAR: It’s good at deception detection

Bradley Dorn shows the hardware of AVATAR, which includes a PC, Microsoft Kinect, infrared sensor and more. (Photo by: Chandler Donald/ Arizona Sonora News)

Scientists are closer than ever to developing technology that can accurately detect whether someone is trying to deceive you. In other words, it’s going to be a lot harder to get away with a lie.

How will the world change when presidents can’t stretch the truth with their supporters, cheaters can’t fib to their lovers and applicants can’t mislead their employers?

At the University of Arizona’s Center for the Management of Information Systems, a team of scientists led by Jay Nunamaker has developed an automated system of deception detection called AVATAR. It was funded by the Department of Defense, National Science Foundation, FRONTEX (EU) and Department of Homeland Security. It stands for Automated Virtual Agent for Truth Assessments in Real-Time, and it was designed for use at the U.S. border.

The virtual interviewer prompts the user to show passport and fingerprints for identification, then proceeds to about a 30-second interview. (Photo by Chandler Donald/ Arizona Sonora News)

The device scans the user’s passport and fingertips and asks a few questions. Equipped with a Microsoft Kinect motion sensor and near-infrared body imaging, along with dozens of other sensors, the system detects whether the user is being deceptive by tracking movement in the eyes, face and body.

“There has been an exponential growth of possibilities with deception detection technology,” said Judee Burgoon, director of research for the center at UA’s Eller College of Management. “All of these aspects of human behavior have been investigated with a lot more depth than in decades past.”

Researchers will tell you there is no device or technique that can definitively determine whether someone is lying. They can only make educated guesses at emotions and reactions. But UA’s MIS lab said AVATAR has an accuracy of 70 to 92 percent. And it’s getting better, thanks to Artificial Intelligence software that collects data, such as nuances in language or body movement, so researchers can see where to make improvements.

Teresa Small, public affairs liaison for the Tucson Field Office of U.S. Customs and Border Protection, said there are no plans to use AVATAR technology at the Mexican border in Arizona. But Frontex and UA have had successful trials at points of entry in Romania.

“There a lot of things impeding technology that have nothing to do with the technology itself,” Burgoon said.

The automation of jobs usually completed by people has a lot of pushback from organizations and unions concerned about labor rights, according to Burgoon.

Some also might be concerned about the effects of false positives with people crossing the border. What if someone is innocent but has a nervous disposition and they are wrongfully detained?

Burgoon said not to worry, because countermeasures are included in the system. For example, baseline questions — those that shouldn’t elicit any deception — are done at the beginning, so that changes in a person’s response to questions can be calculated.

“If you do your technology right, you will rule out those who are innocent and nervous from those who are nervous because they are not innocent,” Burgoon said. “It’s important to stress that these kinds of technology are backed by a lot of research that has been going on for a long time.”

Bradley Dorn, an MIS doctoral student working on AVATAR, said the technology also can decrease the number of people who are wrongfully accused of deception.

“People can be biased, tired or not trained correctly,” Dorn said. “Use of innovation and context like this can complete the same task in a systematic and unbiased way.”

While we likely won’t see AVATAR kiosks at the port of entry any time soon, Dorn and Burgoon say the technology has the potential to solve a huge spectrum of problems. Burgoon said pre-employment screenings, doctors visits and fraud detection are just a few of the possible applications for AVATAR and tech like it.

And while the automated technology still has a ways to go before we see it used for law enforcement, the science behind it is used across the world for forensic investigation.

“There has been a paradigm change in recent years from the controlled question test, or ‘Did you do it?’ questions,” said Peter Rosenfeld, a professor in the Department of Psychology at Northwestern University. “(Those) tend to elicit a confession, to the guilty knowledge test or concealed information test, which asks questions only a guilty party would know the answer to.”

His research deals with the concealment of information for forensic use, which is helping law enforcement agencies investigate suspects by means of cognitive and behavioral psychology.

Rosenfeld said this area of psychology was quite small until the attacks on Sept. 11, 2001. Since then, the need and willingness to invest in this research has taken off.

The main way to measure the concealment of information has been through different polygraph testing, along with concealed information testing. An interviewer might ask about a series of objects, one of which would be a murder weapon. The interviewer can recognize changes in a guilty person’s reaction to a specific object.

Polygraph testing is a bit of a misnomer, according to Rosenfeld. You might think of old movies where the interviewee is attached to a breathing and heart rate monitor, but polygraph simply means that more than one measurement are being taken simultaneously.

Modern polygraphs use different measures, such as voice inflection and movement of the eyes, face and body for AVATAR. In Rosenfeld’s research, electroencephalography or EEG is used to track suspect’s brain waves as they are given the concealed information test.

Japan is the first and only nation to use the concealed information test in police field work. Rosenfeld has worked with the National Research Institute of Police Science in Japan, Osaka University and the Forensic Science Laboratory of Hyogo Prefectural Police Headquarters in Kobe, Japan.

According to “Daily application of the concealed information test: Japan” by Akemi Osugi, over 5,000 of the tests are used annually for police investigation by over 100 specially trained interviewers. Results of these tests can even be used as evidence in court, according to a study titled, “The CIT in the Courtroom: Legal Aspects,” by Gershon Ben-Shakhar and Mordechai Kremnitzer.

Neither the controlled question test nor the concealed information test hold water in the U.S. court system, but Rosenfeld has high hopes for the concealed information test, with false positive rates at nearly 2 percent in Japanese court. There is a potential to use wider use of this technique in the future across a variety of applications.

So whether you’re a good liar or not, the science of deception detection is moving at a rate faster than even the best liars could keep up with. Burgoon, Dorn and Rosenfeld all agreed that people will never stop lying. But in places where it counts — like a job interview, a doctor’s office or a courtroom — the odds of getting caught are increasing.

How to be a pass the test:
Answer everything slowly
In “Detecting Concealed Information and Deception Recent Developments,” edited by Rosenfeld, reaction time when it comes to answering questions can be a big tell if you’re being deceptive. Say you’re asked about cheating on three tests, but you only cheated on one of them. You will take more or less time to answer the question about the test you cheated on if you’re being deceptive. In their research, there were some false-negative, or subjects who got away with it, by answering each question slowly and at the same rate.

Steady breathing
For old-school polygraphs, respiration, or breathing rate, played a major factor in whether or not you were found being deceptive. With more modern polygraph testing this technique may not be quite as useful, but it will help in your speech response time and your heart rate. Fluctuations in either could result in a positive test.

Keep your eyes off the prize
For AVATAR, a major clue as to whether or not you are concealing information is the time your eyes spend on any one thing. In “Countermeasures and Eye Tracking Deception Detection” Ryan Schuetzler of the University of Arizona found that when given a group of objects, the subject will keep their eyes on the object associated with their deception for longer than the others. In this situation, you have the potential to beat the test by spending no more time looking at one thing than anything else.

Don’t show your feelings on your face
In “Exploring the movement dynamics of deception” Nunamaker and other researchers found that movements in the upper face are often involuntary and can paint a picture of the emotional state of the subject. So keep that in mind when you find the need to lie; your face says it all.

Chandler Donald is a reporter for the Arizona Sonora News, a service from the School of Journalism with the University of Arizona. Contact him at

Click here for a word version of the story and high resolution photos. 

Leave a Reply

Your email address will not be published. Required fields are marked *