This is the one feature the iPhone 13 desperately needs
Widespread mask-wearing has rendered Face ID. Apple already has the solution at its fingertips
Earlier this year I purchased an iPhone 12 Pro, a £1,000 smartphone that is, frankly, a device not fit for purpose in 2020.
The iPhone 12, much like every Apple smartphone since 2017’s iPhone X, has just one method of biometric authentication: Face ID. However, as a result of the pandemic sweeping the globe, the majority of us are now wearing masks whenever we leave the house, which has been confounding our smartphones’ built-in facial recognition software. And, despite inbound vaccines, it will be some time before as a society we are mask-free, and even then there is no guarantee we will return to such a culture post Covid.
Face ID uses a TrueDepth camera and infrared technology to project and analyse over 30,000 invisible dots to create a depth map of your face and to study the features around the eye, nose, mouth and ears. Apple’s neural engine transforms the depth map and infrared image into a mathematical representation and compares that representation with the enrolled facial data.
While, according to Apple, “Face ID automatically adapts to changes in your appearance, such as wearing make-up or growing facial hair”, mask-wearing has made the feature redundant.
Although the company has made changes to the system so that your iPhone will now prompt you to enter your passcode if it detects you’re wearing a mask, tasks such as unlocking your smartphone and paying for your shopping using Apple Wallet have become frustrating. The other option is to pull your mask down every time you need to use your phone, but that's neither hygienic nor practical.
It isn’t just Face ID that has been rendered useless as a result of widespread maks-wearing; ongoing testing by the US National Institue of Standards and Technology (NIST) show the efficacy of facial recognition algorithms used in everything from Customs systems and surveillance cameras has declined as a result.
Many are trying to solve this problem. In April, for example, researchers posted an image dataset to GitHub featuring 1,200 pictures culled from Instagram selfies. They tagged the images to identify people wearing medical masks, non-medical masks or no mask to support creative solutions for Covid-challenged facial recognition problems.
Separately, researchers at Wuhan University in China compiled and posted to GitHub a larger dataset that consists of 5,000 cleaned and labelled images of masked faces of 525 different individuals, along with images of 90,000 unmasked faces, in order to train future facial recognition algorithms.
However, it’s unlikely the facial recognition technology we use on our smartphones will be able to adapt – not unless we’re willing to compromise on both performance and our personal security.
“Facial recognition algorithms can be trained using data from individuals wearing masks," says Huseyin Atakan Varol, professor of Nazarbayev University School of Engineering and Digital Sciences, and founding director of the Institute of Smart Systems and Artificial Intelligence. However, due to the increased occlusion of the facial features by the mask, the performance of the face recognition would be worse than the facial recognition conducted with a full set of facial features.
Andrew Bud, founder and CEO at iProov, adds: “It's much harder to assure the genuine presence of somebody wearing a mask because so much information about the face is hidden. A mask has no definable characteristics – it hides so much information that's important for the assurance that this is a genuine person.”
Bud doesn’t think, in future iPhones, this will be a problem “because we're not going to be wearing masks for prolonged periods,” while Ryan Gosling, commercial director at Callsign, suggests phone makers should implement mobile behavioural biometrics as a way to bypass the problem.
“It uses gestures to positively identify a user – such as a swipe across the screen or entering in a One Time Passcode (OTP) – and it can be as passive or interactive as required,” he says. “The user’s behaviour creates a unique profile and can be used to positively identify them in future.
“This is an ideal workaround or replacement for facial biometrics, as behavioural biometrics only requires the user to have their hand available to swipe across their mobile screen or enter an OTP to authenticate themselves.”
There’s a simpler solution, of course: bring back Touch ID. Apple has already shown us it’s capable of doing so without bringing back the home button, as the recently announced iPad Air features Touch ID mapped to its power button. What’s more, under-screen fingerprint recognition technology is now commonplace, featuring on the latest devices from OnePlus and Samsung.
It’s likely the re-implementation of Touch ID simply wasn’t possible with the iPhone 12. Apple typically finalised the design of its next-generation devices 12 to 18 months before they head to production, and with the pandemic not even a year old at the time, it’s unlikely the company had the window to make such as drastic change.
But with the iPhone 13 the company has no excuse. While some people think otherwise, mask-wearing is likely here to stay for the foreseeable future, and technology should be making our lives more convenient, not less.
The iPhone 12, much like every Apple smartphone since 2017’s iPhone X, has just one method of biometric authentication: Face ID. However, as a result of the pandemic sweeping the globe, the majority of us are now wearing masks whenever we leave the house, which has been confounding our smartphones’ built-in facial recognition software. And, despite inbound vaccines, it will be some time before as a society we are mask-free, and even then there is no guarantee we will return to such a culture post Covid.
Face ID uses a TrueDepth camera and infrared technology to project and analyse over 30,000 invisible dots to create a depth map of your face and to study the features around the eye, nose, mouth and ears. Apple’s neural engine transforms the depth map and infrared image into a mathematical representation and compares that representation with the enrolled facial data.
While, according to Apple, “Face ID automatically adapts to changes in your appearance, such as wearing make-up or growing facial hair”, mask-wearing has made the feature redundant.
Although the company has made changes to the system so that your iPhone will now prompt you to enter your passcode if it detects you’re wearing a mask, tasks such as unlocking your smartphone and paying for your shopping using Apple Wallet have become frustrating. The other option is to pull your mask down every time you need to use your phone, but that's neither hygienic nor practical.
It isn’t just Face ID that has been rendered useless as a result of widespread maks-wearing; ongoing testing by the US National Institue of Standards and Technology (NIST) show the efficacy of facial recognition algorithms used in everything from Customs systems and surveillance cameras has declined as a result.
Many are trying to solve this problem. In April, for example, researchers posted an image dataset to GitHub featuring 1,200 pictures culled from Instagram selfies. They tagged the images to identify people wearing medical masks, non-medical masks or no mask to support creative solutions for Covid-challenged facial recognition problems.
Separately, researchers at Wuhan University in China compiled and posted to GitHub a larger dataset that consists of 5,000 cleaned and labelled images of masked faces of 525 different individuals, along with images of 90,000 unmasked faces, in order to train future facial recognition algorithms.
However, it’s unlikely the facial recognition technology we use on our smartphones will be able to adapt – not unless we’re willing to compromise on both performance and our personal security.
“Facial recognition algorithms can be trained using data from individuals wearing masks," says Huseyin Atakan Varol, professor of Nazarbayev University School of Engineering and Digital Sciences, and founding director of the Institute of Smart Systems and Artificial Intelligence. However, due to the increased occlusion of the facial features by the mask, the performance of the face recognition would be worse than the facial recognition conducted with a full set of facial features.
Andrew Bud, founder and CEO at iProov, adds: “It's much harder to assure the genuine presence of somebody wearing a mask because so much information about the face is hidden. A mask has no definable characteristics – it hides so much information that's important for the assurance that this is a genuine person.”
Bud doesn’t think, in future iPhones, this will be a problem “because we're not going to be wearing masks for prolonged periods,” while Ryan Gosling, commercial director at Callsign, suggests phone makers should implement mobile behavioural biometrics as a way to bypass the problem.
“It uses gestures to positively identify a user – such as a swipe across the screen or entering in a One Time Passcode (OTP) – and it can be as passive or interactive as required,” he says. “The user’s behaviour creates a unique profile and can be used to positively identify them in future.
“This is an ideal workaround or replacement for facial biometrics, as behavioural biometrics only requires the user to have their hand available to swipe across their mobile screen or enter an OTP to authenticate themselves.”
There’s a simpler solution, of course: bring back Touch ID. Apple has already shown us it’s capable of doing so without bringing back the home button, as the recently announced iPad Air features Touch ID mapped to its power button. What’s more, under-screen fingerprint recognition technology is now commonplace, featuring on the latest devices from OnePlus and Samsung.
It’s likely the re-implementation of Touch ID simply wasn’t possible with the iPhone 12. Apple typically finalised the design of its next-generation devices 12 to 18 months before they head to production, and with the pandemic not even a year old at the time, it’s unlikely the company had the window to make such as drastic change.
But with the iPhone 13 the company has no excuse. While some people think otherwise, mask-wearing is likely here to stay for the foreseeable future, and technology should be making our lives more convenient, not less.