+1 Recommend
    • Review: found
    Is Open Access

    Review of 'Head to head comparisons in performance of CD4 point–of-care assays: A Bayesian meta-analysis (2000-2013)'

    Head to head comparisons in performance of CD4 point–of-care assays: A Bayesian meta-analysis (2000-2013)Crossref
    A very nice review and meta-analysis that might need some minor revisions
    Average rating:
        Rated 4 of 5.
    Level of importance:
        Rated 4 of 5.
    Level of validity:
        Rated 4 of 5.
    Level of completeness:
        Rated 4 of 5.
    Level of comprehensibility:
        Rated 4 of 5.
    Competing interests:

    Reviewed article

    • Record: found
    • Abstract: found
    • Article: found
    Is Open Access

    Head to head comparisons in performance of CD4 point–of-care assays: A Bayesian meta-analysis (2000-2013)

    Background: Timely detection, staging, treatment initiation are pertinent to controlling HIV Infection. CD4+ cell-based point-of-care (POC) devices offer the potential to rapidly stage patients, and decide on initiating treatment, but a comparative evaluation of their performance has not yet been performed. With this in mind, we conducted a systematic review and meta-analyses. Methods: For the period Jan 2000 to April 2015, 19 databases were systematically searched, 6619 citations retrieved, and 25 articles selected. Diagnostic performance was compared across devices (i.e., PIMA, CyFlow, miniPOC, MBioCD4 System) and across specimens (i.e., capillary blood vs. venous blood). A Bayesian approach was used to meta-analyze the data. The primary outcome, the Bland-Altman (BA) mean bias (which represents agreement between cell counts from POC device and flow cytometry), was analyzed with a Bayesian hierarchical normal model. Findings: We performed a head-to-head comparison of two point-of-care devices, PIMA and PointCareNOW CD4. PIMA appears to perform better vs. PointCareNOW with venous samples (BA mean bias: -9.5 cells/μL; 95% CrI:-37.71 to 18.27 vs. 139.3 cells/μL; 95% CrI:-0.85 to 267.4, mean difference = 148.8, 95% CrI: 11.8, 285.8); however, PIMA’s best performed when used with capillary samples (BA mean bias: 2.2 cells/μL; 95% CrI:-19.32 to 23.6). Sufficient data was available to allow pooling of sensitivity and specificity data only at the 350 cells/μL cutoff. For PIMA device sensitivity 91.6 (84.7 to 95.5) and specificity was 94.8 (90.1 to 97.3) respectively. There was not sufficient data to allow comparisons between any other devices. Conclusions: PIMA device was comparable to flow cytometry. The estimated differences between the CD4+ cell counts of the device and the reference was small and best estimated in capillary blood specimens. As the evidence stands, the PointCareNOW device will need to improve prior to widespread use and more data on MBio and MiniPOC are needed. Findings inform implementation of PIMA and improvements in other CD4 POC device prior to recommending widespread use.

      Review information

      Review text

      The authors conducted a comprehensive systematic review and meta-analysis of published data on the accuracy of rapid, point-of-care CD4 devices. They appropriately used the mean bias of a Bland-Altman analysis as the primary measure of comparison, which is a more appropriate measure than correlation coefficients for comparison of continuous variables with non-normal distributions. They found Alere’s Pima machine to be accurate, and performed as well on capillary blood as compared to use on venous blood samples. In general, the authors have done and excellent study, but they have overstated their findings in several sections (described below).

      The authors imply (in the abstract and text) that the Pima was better on capillary blood as compared to venous blood, as suggested by a simple comparison of the mean bias. However, the CI range encompasses the mean value, so there is no significant difference. The authors should change their terminology to say results from capillary blood ‘were equivalent to’ or ‘not significantly different from’ results from venous blood. This revision should also be made in the second paragraph of the Discussion. The final recommendation – that we should be testing capillary blood given easier operational use – remains unchanged.

      Given that only 2 studies reported data for PointCareNOW (and with very different results), I suggest the authors be more cautious about drawing conclusions of this device (perhaps that one study was poorly executed). Within the Results (Bland-Altman section), rather than say the “PIMA device was superior”, without a direct comparison, the authors would be better served to describe the data – ‘Although data are limited, PointCareNOW had a higher mean bias than Pima.’ The second sentence of the Summary should be revised to ‘based on 2 studies, the PointCareNOW performed poorly in one study and sufficiently well in a second study, which highlights the need for additional data’. In the Conclusion, the paper would be better served to highlight that data for the PointCareNOW were limited and needs more data before being recommended for use (rather than stating it is inferior to Pima).

      In the third paragraph of the Discussion, “current POC devices” should be “PIMA”. There was no clear evidence that the devices other than Pima worked better on capillary blood (perhaps equivalent to, but not better).

      The authors should add to the limitations the variability of these studies. They have likely used different gold standard CD4 reference tests, were conducted in settings with different levels of infrastructure (perhaps some were done in a clinic), machines likely had different calibration procedure before testing, etc.


      Comment on this review