Performance of popular pulse oximeters compared with simultaneous arterial oxygen saturation or clinical-grade pulse oximetry: a cross-sectional validation study in intensive care patients

Background The demand for pulse oximeters is high in the current COVID-19 pandemic. Despite their popularity, clinical studies to evaluate the reliability in obtaining information on a patient’s oxygenation status, are lacking. Aim To evaluate the performance of pulse oximeters under clinical conditions, with arterial blood gas measurement (SaO2) as reference standard. Methods We studied the accuracy of ten top-selling pulse oximeters in Europe and the USA in an intensive care population at the Flevoziekenhuis in Almere, the Netherlands. The studied pulse oximeters were: AFAC FS10D, AGPTEK FS10C, ANAPULSE ANP 100, Cocobear, Contec CMS50D1, HYLOGY MD-H37, Mommed YM101, PRCMISEMED F4PRO, PULOX PO-200, and Zacurate Pro Series 500DL. Adult patients in whom an SaO2 measurement was obtained as part of routine care were asked for inclusion. Directly after obtaining the SaO2 blood sample we obtained the pulse oximeter readings (SpO2) of the investigational devices, in random order. Outcomes were bias (SpO2 – SaO2) mean, root mean square difference (ARMS), mean absolute error (MAE), and accuracy in identifying hypoxemia (SaO2≤90%). As a clinical index test, we included a hospital-grade SpO2-monitor. Results In 35 consecutive patients we obtained 2,258 SpO2-readings and 234 SaO2-samples. Mean bias ranged from −0.6 to −4.8. None of the pulse oximeters had an ARMS≤3%, the requirement set by ISO-standards and required for FDA 501(k) clearance. The MAE ranged from 2.3 to 5.1, and 5 out of 10 pulse oximeters met the requirements of ≤3%. For hypoxemia, negative predictive values were 98-99%. Positive predictive values ranged from 11-30%. Highest accuracy (95%CI) were found for Contec CMS50D1; 91% (86-94) and Zacurate Pro Series 500DL; 90% (85-94). The hospital-grade SpO2-monitor had an ARMS of 3.0 and MAE of 1.9, and an accuracy of 95% (91-97%). Conclusion Direct-to-consumer pulse oximeters do not meet ISO-standards required for FDA-clearance, but can accurately rule out hypoxemia. ### Competing Interest Statement The authors have declared no competing interest. ### Clinical Trial This study did not involve a clinical trial ### Funding Statement This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors. ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: The study was approved by the local ethics committee and the board of directors of the Flevoziekenhuis All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable. Yes All data relevant to the study are included in the article or uploaded as an online supplement. The datasets used during the current study are available from the corresponding author on reasonable request.