There is a high degree of interrater and intrarater variability when interpreting ultrasound.

Overview

B-line count interrater agreement varies across lung zones. Still need training for maintained POCUS competency and confidence in applying it to clinical work.

In a recent study on interrater agreement (ICC) and intrarater agreement for B-line counts:

• ICC between experts = 0.603
• ICC between expert and novice was 0.593,
• intrarater agreement between expert and self was 0.586-0.676.

Another study reported ICC of 0.82 between experts for B-line counts.

Efforts have been made to develop frameworks for assessing competency in point-of-care ultrasound (POCUS), but more can be done to ensure technical skill and clinical utilization. Furthermore, additional research could be done to obtain evidence for POCUS training effectiveness and longitudinal trainee competency in support of policy guideline development.

‍

Expert vs Expert Agreement

‍

Publication

Interobserver agreement in the evaluation of B-lines using bedside ultrasound.

Key Findings

• The average agreement (ICC) across all lung zones for B-line counts between emergency medicine expert sonographers in patients presenting with undifferentiated dyspnea was 0.603
• Agreement for B-line counts between experts varied based on zone of the lung, with the anterior/superior left lung zone showing highest agreement (ICC 0.891)
• Intrarater agreement (Expert vs self) for B-line counts was ICC 0.586-0.676

Paper Implications

• Variance in B-line counts in POCUS may strongly depend on the operator and pathology

‍

Publication

Interobserver Agreement and Correlation of an Automated Algorithm for B-Line Identification and Quantification With Expert Sonologist Review in a Handheld Ultrasound Device.

Key Findings

• ICC between experts for B-line counts was 0.82 on lung ultrasound clips obtained by novice POCUS sonographers
• ICC between expert and artificial intelligence was 0.56 on lung ultrasound clips obtained by novice POCUS sonographers

Paper Implications

• More research is required to develop AI to be comparable to expert review

‍

Novice vs Expert Agreement

‍

‍Publication

Interobserver agreement in the evaluation of B-lines using bedside ultrasound.

Key Findings

• The average agreement (ICC) across all lung zones for B-line counts between expert and novice in patients presenting with undifferentiated dyspnea was 0.593
• Agreement for B-line counts varied between expert and novice based on zone of the lung, with the highest agreement in the anterior/superior right lung zone (ICC 0.756), with the lowest agreement in the lateral/inferior lung zones

Paper Implications

• Additional attention could be focused on the inferior, lateral lung zones when training for POCUS in B-line counts for undifferentiated dyspnea