Temperature scanners being used to check for covid can be wildly inaccurate, researchers find

Temperature-scanning devices that check for fevers in schools, workplaces and public venues distort the results in a way that could overlook the telltale sign of a coronavirus infection, according to new research that casts doubt on the systems’ effectiveness in helping people resume normal life.

The thermal cameras and “temperature tablet” kiosks have been heralded as a critical first line of defense against new pandemic outbreaks. But in a new study of the scanners by the surveillance research organisation IPVM, researchers warn that the tools are dangerously ineffective, raising the risk that infected people could be waved through medical screening checkpoints and go on to spread the virus unchecked.

On Thursday night, shortly after The Washington Post discussed the research findings with the Food and Drug Administration, the agency issued a public alert warning that improper use of the devices could lead to inaccurate measurements and “present potentially serious public health risks.”

The agency also announced that it was sending official “warning letters” to one of the discussed companies, as well as three others, for selling “unapproved, uncleared, and unauthorised thermal imaging systems.”

In the letter to the company, Certify Global, FDA officials said use of the devices carried the risk of incorrect detection, particularly if used to scan multiple people simultaneously, and that a person with an undetected fever may “be less likely to adhere to infection prevention and control guidelines.”

The researchers found that seven widely used scanners attempt to compensate for the imprecisions of lower-cost sensors and the unpredictable factors of real-world tests by “normalising” the readings of people’s temperatures.

But that “compensating algorithm,” they argue, severely undermines the devices’ medical usefulness. A feverish person with a core temperature of 100.4F (38C), their research found, could be assessed by the test devices as having a temperature of 98F (36.6C), well within the healthy range.

“The utility of these devices as fever screeners is now highly questionable, and arguably a risk to public health, because they actively report fevers as normal,” said Conor Healy, the lead researcher of the study set for publication in the Journal of Biomedical Optics.

Representatives of the tested companies Certify, Dahua, Meridian and ZKTeco disputed the findings, saying their systems don’t manipulate temperature readings but in some cases use software techniques to “self-calibrate” to their environment.

“The deviation setting is not intended to ‘distort’ results. It’s designed to allow customers to receive alerts only when actual threats exist,” said Larry Reed, the chief executive of ZKTeco, which makes a “SpeedFace” thermal-imaging system that sells for about $3,800 (£2,700). “On a hot summer day in Arizona, [non-feverish] employees might trigger the device alarm all day if it’s set at 101F (38.3F) and the employees are scanned upon immediately entering the building.”

Peter Plassmann, a thermography expert whose UK-based company Thermetrix designs thermal-imaging systems for medical use, said the research highlights how companies have sought to bolster their business by oversimplifying how well the devices are supposed to work.

“That’s generally the problem with infrared imaging: It’s so deceptively easy,” he said. “You point the camera at somebody, you get a nice colorful image and you get a temperature reading. Great. But in reality, it’s all rubbish. There are so many factors you need to consider.”

Companies have promoted the thermal-imaging systems for their ability to measure temperatures more quickly and at a safer distance than traditional and infrared thermometers that measure temperature from a person’s forehead, ear or mouth.

The scanners use infrared sensors to analyse the heat radiating from a person’s skin - a close but imperfect reflection of their core body temperature - and some systems are advertised as being able to assess multiple people in a passing crowd.

The FDA typically requires thermal scanners and other medical-use devices be tested for safety and effectiveness under a process known as 510(k) clearance. But in April, the agency said it would no longer require premarket reviews or object to unvetted devices that did not “create an undue risk.”

A surge of new thermal scanners followed, said the researchers, who counted more than 200 companies now making or advertising such devices. Many of the companies first jumped into the market last year without any previous experience in thermal-imaging or medical devices.

The FDA has said the change was a necessary move to address fears of device shortages. But the researchers said they worry that such untested systems are now widely distributed across the country and could fuel a “false sense of security” that could imperil public health.

Thermal scanners face a huge flaw in their ability to detect coronavirus infections: Roughly 40 per cent of infected people won’t have a fever at all, the Centers for Disease Control and Prevention estimated last year. And a person’s temperature can shift wildly based on many factors, including if they’re overweight, stressed, menopausal or wearing heavy clothing, or if they recently exercised, stepped out of a hot car, or drank alcohol or caffeine.

But because there is no immediate alternative for detection, many officials have seen the fever scans as perhaps their only way of identifying someone who could spark a new outbreak. Some companies and local governments have spent tens of thousands of dollars per camera to bolster their defenses.

The FDA has worked to adjust Americans’ expectations about how useful the systems can be. The agency says in official guidelines that thermal scanners are “not effective at determining if someone definitively has Covid-19”; that they “have not been shown to be accurate when used to take the temperature of multiple people at the same time”; that their accuracy depends heavily on “careful set-up and operation”; and that “their effectiveness as part of efforts to reduce the spread of disease has been mixed.”

Bill Maisel, the chief medical officer and director of the Office of Device Evaluation at the FDA’s Center for Devices and Radiological Health, said the research identified several systems that “didn’t have the level of accuracy that we would expect.” Even after the premarket review change last year, he said, the FDA still required all devices to meet certain performance expectations. The agency, he added, regularly monitors for problematic systems and works with the companies to address errors when their products are found to underperform.

The devices, he said, should play only a limited role when assessing risks during the pandemic, and they are no replacement for social distancing, mask-wearing or more traditional medical screening.

Thermal-imaging devices are “imperfect, and they are particularly imperfect when screening for Covid,” he said. “The inaccuracy of devices is one of the aspects that contributes to the imperfection.”

The IPVM researchers did not test traditional thermometers, and their findings confirmed that some FDA-cleared devices returned near-perfect accuracy. But all of the thermal scanners they tested appeared to deliberately “normalise” high temperature readings into a healthier, non-fever-like range: Colder readings were pulled higher, while hotter readings were pulled down.

The tested systems relied on lower-cost hardware with dramatically lower precision: Several scanners used a sensor with a resolution of about 1,000 pixels - far more limited than the 76,000-pixel sensor found in one of the FDA-vetted machines they used as a control device.

To get an accurate temperature reading, international guidelines for medical electrical equipment say the systems should only be used in controlled environments with regularly calibrated devices on people who are consistently “prepared”: People walking in from an indoor waiting room, for example, would offer very different results than others who had lined up waiting in the sun.

But the companies, Mr Healy said, appeared to navigate that challenge by making the systems appear to work consistently across a wide range of ever-changing conditions.

The systems, Mr Healy said, were designed “to maintain an appearance of normal function despite poor device capabilities or screening conditions, enabling manufacturers to cover up performance issues while selling to a much wider set of use-cases.”

IPVM is a surveillance research group that examines camera hardware, imaging devices and other technical tools for its members, largely in the security industry. Its researchers examined the systems in a temperature-controlled warehouse laboratory in Pennsylvania.

The tested companies are not household names, but they form the technical backbone of the screening systems used in schools, retail stores and workplaces nationwide. The devices range in price, from $600 (£430) to $13,000 (about £7000), and are promoted as being able to conduct person-by-person entry scans or assess visitors en masse.

Certify, a Maryland-based seller of devices found in casinos and hotels, advertises on its website that it offers “the #1 Fever Detection & Thermal Scanning Solution in the Marketplace” and can “replace manual scanning.”

Certify spokeswoman Jasmine Neisser said the system does not alter high temperatures but does set a minimum temperature level that will return a 96 degree reading if the scan fails. The system, she said, “uses world-class manufactured sensors for thermal applications, which are rigorously tested for accuracy.”

Certify Vice President Tim Goodwin had said in a LinkedIn post that devices like the SnapXT Pro, its $2,000 (about £1500) thermal scanner with an eight-inch touch screen, could be found in “more than 75 per cent of NFL stadiums.” An NFL spokesperson disputed that claim, saying the devices are used in roughly a dozen of the league’s 30 stadiums nationwide. (Ms Neisser said that the comment related to the number of open stadiums when the post was written last year and that at the time Certify devices had been deployed in five of the eight stadiums then open to visitors.)

Mr Plassmann said the medical-equipment industry standards for deploying such systems, which cover everything from the lighting and humidity of screening rooms to how much time people should acclimate before their test, are often ignored in the real world, where unpredictable conditions and public circumstances can vary wildly and skew the results.

He said he has seen an influx of companies that offer glossy marketing materials but little commitment to the best practices for public-health use, and he said he is concerned that their widespread deployment could undermine efforts to guard against new outbreaks.

“It needs to be done properly,” he said, “or it can do more harm than good.”

The Washington Post