Scan Acquisition Competency Metrics
There are many different ways to measure scan acquisition competence, and there is an increasing need for longitudinal monitoring and assessment.
Overview
While POCUS competency assessment has so far been largely left up to individual educational institutions, various organizations have provided guidelines or recommended scales to help inform competency assessment. Notably, consistency can be an important factor in determining learner competency, suggesting a need for longitudinal monitoring by experts.
Comprehensive Frameworks
Publication
Key Findings
"...POCUS competency is likely better assessed with instruments that can objectively evaluate the performance of a learner over time rather than using a threshold number of scans" (p. 2 of 11)
Paper presents a Point-of-care Ultrasound Image Quality (POCUS IQ) scale (p. 6 of 11), assessing Technical skills, Scanning Skills, and Interpretability, as follows:
Technical skills comprise:
- Probe Choice, assessed via "Inadequate for visualization of area of interest", "adequate", and "Ideal"
- Depth, assessed via "Inadequate for visualization of area of interest", "adequate", and "Ideal"
- Gain/Presets, assessed via "Inadequate for visualization of area of interest", "adequate", and "Ideal"
Scanning Skills comprise:
- Probe Control, assessed via "Poor probe control, frequent erratic movements", "Fair probe control", and "Excellent probe control, consistently smooth movements"
- Anatomy/Landmarks, assessed via "Poor demonstration of anatomy/landmarks", "Fair demonstration of anatomy/landmarks", and "Excellent demonstration of anatomy/landmarks"
Interpretability comprises:
- Labeling (e.g., location, orientation), assessed via "Inadequate labelling for image interpretation", "Adequate labelling", and "Ideal labelling OR Labelling not necessary for application"
- Completeness (e.g., appropriate views, modes, measurements), assessed via "Inadequate views for image interpretation", "Adequate views", and "Ideal views"
Paper Implications
- Image acquisition skills may be assessed through quality assurance image or clip review, even remotely and asynchronously
Image Quality Metrics
Publication
Key Findings
In line with Council of Emergency Medicine Residency–Academy of Emergency Ultrasound guidelines and image review sessions, a scale of 1 to 5 for image quality was developed and presented in Figure 1 on page 175 as follows:
- 1 - No recognizable structures, no objective data can be gathered
- 2 - Minimally recognizable structures but insufficient data for diagnosis
- 3 - Minimal criteria met for diagnosis, recognizable structures but with some technical or other flaws
- 4 - Minimal criteria met for diagnosis, all structures imaged well and diagnosis easily supported
- 5 - Minimal criteria met for diagnosis, all structures imaged with excellent image quality and diagnosis completely supported and suitable for educational purposes
A score of 3 or above is required for a diagnosis
Paper Implications
- Ultrasound image quality can be assessed by examiners or image reviewers on a scale of one to five
Publication
Key Findings
The B-QUIET scale on page 1651 for ultrasound image quality assessment includes "Image Anatomy" with scores of 1 to 4 covering "Unacceptable," "Needs Improvement," "Acceptable," and "Optimal", respectively, for:
- Near Field (Top of Screen)
- Receding Edge (Right Side)
- Far Field (Bottom of Screen)
- Leading Edge (Left Side)
A score of 1 indicates "Unclear representation", 2 means "Image partially distorted", 3 means "Adequate visualized anatomy", and 4 means "Optimal anatomy representation"
Paper Implications
- Ultrasound image quality assessment scales still require expert review in order to score image quality
Publication
Key Findings
Using the Assessment of Competency in Thoracic Sonography (ACTS) scale, image quality can be assessed using a scale of 1 to 5, with a score of 0 reserved for "not obtained"
- Score of 1 indicates "Image quality too poor to permit meaningful interpretation"
- Score of 3 indicates "Suboptimal image quality, but basic image interpretation possible"
- Score of 5 indicates "Good image quality, meaningful image interpretation easy"
Most learner improvement happens in the first 25-30 practice studies
Paper Implications
- Quantitative scoring can help in learner assessment
Acquisition Skill Metrics
Publication
Key Findings
According to Figure 1 on page 1651, technical skills for ultrasound image acquisition that can be assessed on a scale of 1 to 4 include:
- Resolution - Application/Focal zone, where 1 = wrong application, wrong focal zone, 2 = correct application, wrong focal zone, 3 = wrong application, correct focal zone, 4 = Correct application, correct focal zone
- Depth - Field of view, where 1 = Image too small (<half of screen), 2 = Image cut off or not maximized at present depth (>4 cm from ideal depth), 3 = Adequate depth (within 1-3 cm), 4 = optimal depth (intended image fills the screen)
- Gain (Segmental/Overall), where 1 = Inappropriate (too white, too dark throughout image), 2 = Poor gain setting in near or far field, 3 = Adequate gain settings (minimal echoes in fluid filled structures, all structures contrasted), 4 = Optimal gain settings and greyscale (No internal echoes in vascular or fluid filled structures, appropriate tones to soft tissue)
Paper Implications
- Technical skills of interest include resolution, depth and gain
Publication
Key Findings
According to Table 2 on page 1027, potential technical competency components for assessment include:
- Probe selection
- Image mode selection (e.g., cardiac, abdominal)
- Proper image orientation marker (e.g., “dot”) selection
- Probe positioning
- Image depth
- Image gain
- Centering of target structure
- Troubleshooting difficult windows
- Demonstrates advanced functions (M-mode, Doppler, image capture)
Paper Implications
- While the paper does not recommend specific metrics for each competency component, this list provides a guideline for institutions to develop their own metrics
Publication
Key Findings
In Table 3, the publication lists an Objective Structured Assessment of Ultrasound Skills (OSAUS). Among the components includes “Image optimization”
- Image optimization means “Consistently ensuring optimal image quality by adjusting gain, depth, focus, frequency, etc.” on a scale of 1 to 5 where 1 = fails to optimize images, 3 = competent image optimization but not done consistently, and 5 = consistent optimization of images
Paper Implications
- Consistency can be a key part of assessing competency of image acquisition, assessing a singular instance of image acquisition is most likely insufficient
Publication
Key Findings
In Table 2 on page 557, containing a list of competency components in a lung ultrasound objective structured assessment of technical skills tool, a section titled “Technical Skills” on scales of 1 to 5 includes:
- Correct placement of patient (e.g., supine when scanning for pneumothorax), where 1 = Wrong placement (e.g., evaluating pleural effusion with patient in Trendelenburg position) and 5 = Optimal placement (e.g., evaluating pleural effusion with patient in seated position)
- Correct choice of transducer, where 1 = Wrong choice of transducer and 5 = Optimal choice of transducer
- Correct depth, where 1 = Wrong depth setting and 5 = Optimal depth setting
- Correct gain, where 1 = Wrong setting of gain and 5 = Optimal setting of gain
- Correct handling of transducer, where 1 = Poor transducer handling and 5 = Optimal transducer handling
Paper Implications
- Notably, the tool uses a numerical scale, but scoring learners within the scale requires subjective judgement - likely, experienced POCUS experts are needed for learner assessment using this tool