NIST Guidelines Can Help Organizations Detect Face Photo Morphs, Deter Identity Fraud

Face morphing software can blend photos of different people’s faces into a single synthesized image.

Credit:

NIST

Face morphing software, which can blend photos of different people’s faces into a single synthesized image, can make it easier for bad actors to bypass identity verification systems in buildings, at borders, in airports and in other settings. Morphed photos can deceive face recognition systems into falsely identifying the image as belonging to both original individuals, allowing the first individual to assume the second person’s identity and vice versa.

To address this issue, the National Institute of Standards and Technology (NIST) has released guidelines that can help organizations deploy and use modern detection methods designed to catch morph attacks before they succeed.

The new publication, Face Analysis Technology Evaluation (FATE) MORPH 4B: Considerations for Implementing Morph Detection in Operations (NISTIR 8584), is a lay-language introduction to morphs and discusses ways to deal with them. The document is intended to guide organizations toward effective deployment of tools and practices in situations where morphed photographs might appear, such as in passport application offices or at border crossings. It also covers what organizations could consider doing after a morph detector flags a potentially fake photo.

Morph detection software has improved dramatically in recent years, according to NIST’s Mei Ngan, an author of the publication.

“Some modern morph detection algorithms are good enough that they could be useful in detecting morphs in real-world operational situations,” Ngan said. “Our publication is a set of recommendations that can be tailored to a specific situation.”

Nowadays, software that creates morphed face photos is readily available. Producing a morph is possible using smartphone apps, desktop graphics packages or AI-based tools. Some of these tools are more effective than others. Some morph creation software can leave artifacts in the morphed photo, such as inconsistent skin texture and color, as well as unnatural-looking areas around the iris, nostrils, lips and eyebrows.

Since 2018, NIST has been evaluating the performance of software designed to detect morphed photos.  While the new guidelines touch briefly upon the current state of the art in detection algorithms, they focus mainly on use cases.

“Some modern morph detection algorithms are good enough that they could be useful in detecting morphs in real-world operational situations. Our publication is a set of recommendations that can be tailored to a specific situation.” —Mei Ngan, NIST computer scientist

A key distinction the guidelines draw is between two different detection situations: In one, called single-image morph attack detection, officials possess only the questionable photo, such as processing a new application in a passport office. In the other, differential morph attack detection, they have the questionable photo along with a second image that is known to be genuine, such as a photo taken of the person at a border crossing.

The guidelines discuss the capabilities and limitations of detectors in both scenarios. Single-image detection, in the best cases, can detect morphs as often as 100% of the time (at a false detection rate of 1%) if the detector has been trained on examples from the software that generated the morph.  However, accuracy can degrade to well below 40% on morphs generated with software unfamiliar to the detector. Differential detectors are more consistent in their abilities, in the best cases with accuracy ranging from 72% to 90%, across morphs created using both open-source and closed-source morphing software, but they require an additional genuine photo for comparison.

The bulk of the guidelines concern how one might configure morph attack detection software under different scenarios, and they suggest steps and procedures to help investigators when the detection software flags a suspicious photo. The best procedures for investigating a candidate morph involve a combination of human review, the use of automated tools, and implementation of a process to handle review of suspected morphs.

The most effective defense against the use of morphs in identity fraud is to prevent morphs from getting into operational systems and workflows in the first place, Ngan said. The guidelines discuss methods to achieve this during document application and issuance.

Ngan said the authors considered the likely pressures that reviewing officers might deal with, such as the volume of photos they would regularly encounter and the number of adjudicators available to inspect a candidate morph.

“What we’re trying to do is guide operational staff in determining whether there is a need for investigation and what steps that might take,” Ngan said.

In addition to developing methods for testing morph detection software, the team’s focus is to increase awareness of the existence of morphing attacks.

“It’s important to know that morphing attacks are happening, and there are ways to mitigate them,” she said. “The most effective way is to not allow users the opportunity to submit a manipulated photo for an ID credential in the first place.”