What’s facial recognition technology?

Facial recognition technology is just one phrase that’s become increasingly homogenized in our digital age. In fact, there are some important distinctions to learn about the technology and how it's used for facial authentication and facial comparison. Read on to learn more.

If you’re here, then it’s obvious you’re interested in researching cutting-edge tech. It’s not just a hobby – it’s a way of life.

Like most technology buffs, you’ve likely watched (or at the very least heard about) the show Black Mirror. Without getting too deep into the series, it discusses what happens when humans start to interact with and incorporate technology in their daily lives, and the ensuing pros and cons of its increased use in our digital world (see our other blog The pros and cons of biometrics).

The creator, Charlie Brooker, is a humorist and a satirist that goes to extreme lengths with the show to point out how technology envelopes our lives – a show, which ironically, is hosted on a streaming service that wants you to binge TV. But, one of his best subtle tricks with the series is in the name of the series itself.

"Any TV, any LCD, any iPhone, any iPad — something like that — if you just stare at it, it looks like a 'Black Mirror,' and there's something cold and horrifying about that, and it was such a fitting title for the show," Brooker told The Guardian in 2011.

As we stare into our own ‘Black Mirrors’ -handheld devices, tablets, phones, and computer screens- some of them stare back using Facial Recognition Technology. This technology can authenticate and verify a user by using biometrics before authentication and allowing access. But there’s some important distinctions to make. For this next piece in our Biometric series, we’ll discuss what is facial recognition vs. facial comparison, how they are different, and how it’s used in cutting-edge technology like facial comparison.  

When the Black Mirror stares back – What is facial recognition and facial authentication?

Biometric identity verification technologies like Facial Recognition and Authentication now ubiquitous. A lot of the most popular phones in the world today have replaced a fingerprint as the de facto biometric authenticator and use a person’s face instead. What’s exciting, people are beginning to use this biometric technology (see our What are biometrics - A complete guide) in more ways than just unlocking phones: airport terminals use Facial Recognition technology to speed passengers through long security lines; sharing-economy companies use Facial Recognition technology to conduct background checks on prospective contractors; law enforcement uses Facial Recognition technology to search an image database of known criminals in order to find and compare a match.

But as Facial Recognition technology offered unprecedented convenience and spoof-proof capabilities a few months ago, its efficacy is in the news again. Recently, people find it hard to open their phones while wearing face masks in order to comply with new COVID-19 recommendations from governments. Biometric Update discussed technology reporting and testing standards with Patrick Grother from the National Institute of Standards and Technology (NIST) about their plans for future biometrics testing and Facial Recognition technology as a result of the pandemic: “What happens when you occlude the mouth region with a face mask? And is face recognition undermined by that to any great extent?” Grother asks. “That’s one example of what we’re trying to do to support informed usage of face recognition with more quantitative data.”

As facial recognition technology continues to undergo new iterations and changes as a result of changes people’s everyday lives, it’s important to note key items around phrasing that industry professionals use to describe it. Oftentimes, providers will throw out terms like facial recognition, facial authentication, and facial comparison. Though similar, each phrase refers to different, nuanced ways of using the biometrics-based technology. Conceptually, each describes a use to verify that a biometric -in this case an image of a human face- matches another image that’s stored as a template and supplied by the user. But the practices are different.

What are the differences between facial recognition, facial authentication, and facial comparison?

The thin line between facial recognition and facial authentication

Facial recognition operates using multiple images that are stored either on a personal device, or in a larger server / database. The user will be captured from a source image -in this case a picture of a person’s face- in order for the Facial Recognition system to run an analysis on other stored images and find a match. The delineating factors:

• A user may not control the Facial Recognition device
• A user may not knowingly provide a source image
• A user’s image may not be the only biometric template profile that’s stored
• A user’s own facial biometrics may not be the only profile that it gets matched against

Examples of facial recognition: a Facial Recognition system used in surveillance purposes is looking for a person with brown hair - the Facial Recognition surveillance technology will return multiple hits of people with brown hair. Those hits or matches might not include the intended user or person of interest. Security and law enforcement services will employ Facial Recognition technology to find criminals in large databases housing hundreds or thousands of biometric profiles, and are notified when there are possible matches.

How facial recognition technology is used for facial authentication: Facial Recognition technology is commonly used for security purposes like securing a physical or digital perimeter for surveillance, but it operates only returning “possible matches.” Facial authentication is actively used to admit a qualified and approved person into a secured location via facial biometric scanning. Secure a bank vault that only opens via an Iris scan? Very 007, but that’s facial authentication. Have a phone unlock after the qualified owner’s puts their face in front of? Secured using facial authentication.

Examples of when businesses use facial authentication: Facial images provided by a user are searched and scanned against a company’s database that contains all of its employees’ facial biometrics. A verified employee trying to access a restricted area is only permitted on a successful match with their picture that’s in the digital vault.

Facial comparison – how it’s used to verify and authenticate

Facial comparison is used in real-time, where an image of a person’s face and biometrics are readily available to compare to another, separate source of it. The source doesn’t always have to be a biometric template that’s been uploaded and stored in the cloud or physical server – some APIs and services may not store biometric templates at all. This means a user has to provide another, trusted source of their face in order to verify their identity at the moment of capture.

In order to effectively use facial comparison, there’s some important requirements:

• The user needs to be present
• A user needs to provide a trusted source picture
• The technology needs to confirm the source picture is authentic

When these are met, facial comparison will run an analysis to confirm a match. In layman’s terms and more modern practices: a user provides an identity document (like driver’s license and passport), the system validates the document isn’t a forgery, the facial comparison technology analyzes the biometrics between the user’s face and the picture on the provided document to ensure they’re the same.

How facial comparison technology is used: A common use-case for Facial Comparison is during digital account onboarding. Document verification technology, like Mitek’s Mobile Verify, is used to verify the authenticity of an applicant’s passport, ID card, or driver’s license. Once the authenticity of the ID is approved, the applicant is asked to provide a selfie. Facial comparison technology then compares the captured selfie images with the image from the verified ID to prove the user is both present and is who they say they are.

Here’s how facial comparison might be used in a bank’s digital onboarding platform:

• A person downloads a Bank’s app in order to apply for a loan
• The bank, using a digital identity verification service running facial comparison, asks the user to provide an identity document for authentication
• The user takes a picture of the Identity Document
• An image of the user’s photograph from the ID as well as other important classifiers like UV stamps, barcodes, and font are extracted from the document to determine whether or not the ID legitimate
• If the ID is approved, the user’s picture from the document becomes the trusted source image, or baseline template, for the Facial Comparison
• The software will then ask for a selfie of the user. *Note: more advanced systems use “Liveness Detection” which is like a live selfie to show a person is real
• Facial Comparison takes the trusted source image from the document and… here’s the kicker, COMPARES it with the user in the captured selfie
• If the user’s facial biometrics in the selfie and ID photograph match, the user is approved to get their loan

*Note* that more sophisticated facial comparison technologies incorporate a feature called liveness detection that walks users through a live, selfie-taking process instead of uploading any picture they want. A main positive about liveness detection is it’s nearly spoof-proof capabilities. Liveness detection walks users through a digital capture process to ensure a real person is present by having them do things like blink, turn their head, and smile.

As most companies move to digital platforms, technologies like facial comparison are an effective way to prevent fraudsters or bad actors from circumventing traditional roadblocks on more outdated onboarding channels. Using advanced data processing systems like Artificial Intelligence, Machine Learning, and Computer Vision to extract biometric data from live facial images and legitimate documents, companies that employ facial comparison with prospective users can ensure that masquerading fraudsters aren’t getting on to their platforms to enact digital sabotage. In the same vein, people who use companies that employ facial comparison and liveness detection can take more control over their digital identities. These new and compelling technologies allow real people to be retain autonomy over their digital lives by only allowing certified and approved access.