Skip to main content


When to incorporate point-of-care ultrasound (POCUS) into the initial assessment of acutely ill patients: a pilot crossover study to compare 2 POCUS-assisted simulation protocols

Article metrics



The purpose of this study was to determine the ideal timing for providers to perform point-of-care ultrasound (POCUS) with the least increase in workload.


We conducted a pilot crossover study to compare 2 POCUS-assisted evaluation protocols for acutely ill patients: sequential (physical examination followed by POCUS) vs parallel (POCUS at the time of physical examination). Participants were randomly assigned to 2 groups according to which POCUS-assisted protocol (sequential vs parallel) was used during simulated scenarios. Subsequently, the groups were crossed over to complete assessment by using the other POCUS-assisted protocol in the same patient scenarios. Providers’ workloads, measured with the National Aeronautics and Space Administration Task Load Index (NASA-TLX) and time to complete patient evaluation, were compared between the 2 protocols.


Seven providers completed 14 assessments (7 sequential and 7 parallel). The median (IQR) total NASA-TLX score was 30 (30–50) in the sequential and 55 (50–65) in the parallel protocol (P = .03), which suggests a significantly lower workload in the sequential protocol. When individual components of the NASA-TLX score were evaluated, mental demand and frustration level were significantly lower in the sequential than in the parallel protocol (40 [IQR, 30–60] vs 50 [IQR, 40–70]; P = .03 and 25 [IQR, 20–35] vs 60 [IQR, 45–85]; P = .02, respectively). The time needed to complete the assessment was similar between the sequential and parallel protocols (8.7 [IQR, 6–9] minutes vs 10.1 [IQR, 7–11] minutes, respectively; P = .30).


A sequential POCUS-assisted protocol posed less workload to POCUS operators than the parallel protocol.


Point-of-care ultrasound (POCUS) is a rapidly evolving diagnostic modality that is performed and interpreted by providers at the bedside [1,2,3,4,5,6,7]. An increasing number of reports have shown that POCUS can modify diagnoses, direct further testing, and change medical therapy [8,9,10,11,12,13,14,15,16,17,18,19]. While the application of POCUS continues to expand, professional societies have published guidelines and recommendations for the use of POCUS specific to their scope of practice. Emergency ultrasound guidelines have identified core emergency ultrasound applications [20], and the Society of Critical Care Medicine recently published guidelines on the use of ultrasonography for critically ill patients [21]. Many clinicians agree that POCUS is complementary to the physical examination. POCUS can be used as a single examination, repeated because of clinical need or deterioration, or used for monitoring physiologic or pathologic changes.

Interestingly, most studies that show clinical utility of POCUS were conducted within 24 h of a patient’s presentation but not always at the time patients were initially assessed. The Focused Assessment with Sonography for Trauma (FAST) examination is one of the very few examples in which POCUS is consistently conducted during the initial resuscitation phase [14, 15, 22]. However, it is still not clear from the original studies if FAST is best conducted in sequence or parallel (or simultaneously) with the primary or secondary survey. In fact, the current guidelines in emergency ultrasound and critical care ultrasonography do not specify the optimal timing of POCUS in relation to the physical examination or other assessments during the initial evaluation of acutely ill patients.

For POCUS to be fully beneficial in patient care, it should be performed and interpreted without interference with the actual workflow during the initial evaluation. Otherwise, POCUS may increase the provider’s workload substantially without adding clinical benefits. We hypothesized that the POCUS-assisted evaluation performed in a sequential manner would pose less workload for the performing providers. To test this hypothesis, we conducted a pilot simulation study to determine the optimal method from the provider’s perspective for integrating POCUS into the initial evaluation of acutely ill patients.


Study setting

We conducted a pilot, unblinded crossover study in which we measured POCUS provider workload to compare 2 POCUS-assisted evaluation approaches: sequential (physical examination followed by multiorgan POCUS) vs parallel (multiorgan POCUS at the time of physical examination). The study was approved by the Mayo Clinic Institutional Review Board (12–007998) and performed from July 13, 2015, through July 26, 2015.

Study participants

Critical care fellows who participated in the Mayo Clinic institutional POCUS workshop were recruited for the study. The other inclusion criterion was certification with Advanced Cardiac Life Support. There were no exclusion criteria. The fellowship program comprises trainees from Critical Care Medicine and Critical Care/Anesthesia. The programs are 1 to 2 years depending on prior training. The background and training level varied among the participants. All of the fellows participated in the POCUS workshop before recruitment, which is offered in July of each academic year. This course is a full day and includes didactic sessions, hands on learning with standardized patients, and competency testing. Verbal consent was obtained from the study participants.

Study design

Participants were randomly assigned to group A or group B, according to which POCUS-assisted protocol (sequential vs parallel) was used during simulated patient scenarios. Group A was asked to complete the assessment of patients using the sequential protocol. Group B was asked to complete the same assessment using the parallel protocol. Subsequently, the groups were crossed over to complete assessment using the other POCUS-assisted protocol in the same patient scenarios: group A used the parallel and group B used the sequential protocol. The simulated scenarios were 1) a 30-year-old man with undifferentiated hypotension and 2) a 36-year-old man with acute respiratory distress. POCUS was performed on standardized patients using a VScan device (General Electric, Boston, Massachusetts USA).

The sequential protocol was performed in the following order (Fig. 1): airway, breathing, circulation, disability, and exposure assessments, followed by thoracic, cardiac, abdominal, and venous POCUS examinations. The parallel protocol was conducted in the following order: airway assessment, breathing assessment, then thoracic POCUS; circulation assessment, then cardiac POCUS; disability assessment, exposure assessment, then abdominal POCUS and venous POCUS (vasculature). The POCUS examinations had 10 required ultrasound examination points and 6 optional points, for a total of 16 potential points. The examination sites for the POCUS protocol are shown in Table 1. Participants were asked to complete a National Aeronautics and Space Administration Task Load Index (NASA-TLX) survey after they finished the 2 simulated patient assessments [13].

Fig. 1

Flowcharts Showing the Sequential and Parallel POCUS Protocols. a, In the sequential protocol, the ABCDE (airway, breathing, circulation, disability, and exposure [and vasculature*]) evaluations are completed, and then the system-specific POCUS examinations are performed. b, In the parallel protocol, the airway and breathing assessments are completed, followed by the thoracic POCUS; then the circulation assessment, followed by cardiac POCUS; then the disability and exposure assessments, followed by the abdominal POCUS and vasculature* (venous) POCUS. POCUS indicates point-of-care ultrasound. * indicates addition to the ABCDE evaluation

Table 1 Ultrasound Transducer Application Sites in the POCUS Protocol

Outcome variables

The NASA-TLX survey was used to measure perceived workload as the primary outcome. NASA-TLX is an integrated measure of overall workload [13]. It consists of 6 domains that represent clustered independent variables: mental demand, physical demand, temporal demand, performance, effort, and frustration [12]. Scores in each domain and overall raw NASA-TLX scores were collected from participants after they completed the 2 patient scenarios. Time to complete the assessment was also recorded. Qualitative feedback from participants about 2 POCUS-assisted protocols was also collected in a survey format.

Statistical analysis

Descriptive statistical terms, such as median and interquartile range (IQR) were used to summarize the outcome variables. Variables were compared between sequential and parallel POCUS-assisted protocols using the Wilcoxon signed rank test. JMP 10.0 software (SAS Institute Inc., Cary, North Carolina USA) was used for the statistical analysis. Because this was a pilot study, sample size calculation was not performed. A P value of <.05 was considered significant.


Seven critical care fellows were enrolled in the study: 4 fellows were assigned to group A (sequential then parallel POCUS-assisted protocol); 3 fellows to group B (parallel then sequential POCUS-assisted protocol). There were 3 trainees from Critical Care Medicine (all postgraduate year [PGY] 4), 1 trainee from critical care emergency medicine (included in the Critical Care Medicine program; PGY 6), and 3 trainees from Critical Care/Anesthesia (all PGY 5). A total of 7 sets of NASA-TLX scores, time to complete patient assessments, and participant surveys were collected for each POCUS-assisted protocol (sequential and parallel). The median NASA-TLX score and time to complete assessment are shown in Table 2. We observed a significant difference in the median (IQR) total NASA-TLX scores between the sequential and the parallel protocols (30 [30–50] versus 55 [50–65], respectively; P = .03), with the sequential protocol being associated with a lower provider workload. When evaluating the individual components of the NASA-TLX scores, we observed a significantly lower median (IQR) mental demand and frustration level for the sequential protocol than the parallel protocol: 40 (30–60) vs 50 (40–70) (P = .03) and 25 (20–35) vs 60 (45–85) (P = .02), respectively. The scores for the other 4 scales (physical demand, temporal demand, performance, and effort) were not significantly different between the 2 protocols.

Table 2 Comparison Between Sequential and Parallel POCUS-assisted Protocols in NASA-TLX Score and Time to Complete Assessment

Fellows commented that maneuvering between the physical examinations with a stethoscope and then an ultrasound transducer in the parallel POCUS-assisted protocol was distracting and more frustrating than completing the physical examination and then performing the POCUS examination in the sequential protocol. Fellows also suggested that the time needed to remove the ultrasound gel between examination components prolonged the time needed to complete the examination. Despite these perceptions, the median (IQR) time needed to complete patient scenarios was not significantly different between the 2 protocols: 8.7 (6–9) minutes in the sequential and 10.1 (7–11) minutes in the parallel protocol (P = .30).


Our study demonstrated that the sequential protocol posed less workload for POCUS operators than the parallel protocol, although the time to complete assessments was not significantly different between the 2 protocols. Mental demand and frustration level were significantly lower in the sequential protocol. To our knowledge, this pilot study is the first to assess the most efficient timing for integrating POCUS with the physical examination by evaluating workload perceived by the POCUS operators.

As POCUS is a relatively new diagnostic and procedural modality, previous literature has focused on investigating its emerging indications and clinical utility, and emphasis was more on demonstrating its advantage over physical examination alone [1,2,3,4,5,6,7,8,9,10,11]. Therefore, less attention was paid to the timing of POCUS and how POCUS is best incorporated in the clinical workflow. For instance, a study in an emergency department found that POCUS conducted immediately after the recognition of undifferentiated hypotension decreased physicians’ diagnostic uncertainty and allowed them to modify treatment plans and determine the need for and place of admission [18]. However, in this study, the physician performing POCUS was not the clinician performing the history and physical examination. Another study performed in an emergency department also showed that POCUS reduced the number of viable diagnoses when it was performed early [16]. In this study, the ultrasound examination was performed after a history, physical examination, and “standard care interventions” were performed. Therefore, in both of these studies, the ultrasound examination was performed after the history and physical examination had been completed. Neither study assessed the integration of POCUS into the initial evaluation phase or the provider’s workload associated with introduction of the POCUS examination.

Studies conducted in intensive care units [10, 11] have had similar issues to those conducted in emergency departments. A study evaluating the role of cardiac POCUS in patients with shock showed that cardiac POCUS influenced fluid and inotropic therapy and that its use was associated with decreased mortality [9]. In this study, cardiac POCUS was performed between 7.5 to 15 h from the time the patient arrived in the emergency department and not at the initial evaluation. Another study evaluated the use of POCUS for determining the cause of pulmonary edema in patients in the intensive care unit [19]. The patients who underwent POCUS immediately after pulmonary edema was diagnosed had the cause identified significantly sooner than a control group. In this study, POCUS was performed after the initial history taking and physical examination, and no discussion on the best timing of POCUS was made [19]. The same limitation applies to a study about the “ICU-sound” protocol, a “head-to-toe” POCUS examination within 12 h of admission that resulted in a modified admitting diagnosis in 26% of the cases [8].

Current guidelines in emergency ultrasound and critical care ultrasonography do not specify the optimal timing of POCUS in relation to the physical examination or other workflow during the initial evaluation of critically ill patients. POCUS is well known for being operator dependent in image acquisition and interpretation. Furthermore, integration of POCUS-obtained information into clinical decision-making is also highly operator dependent. Our study results show that a sequential approach is more practical than a parallel approach for maximizing the benefit of POCUS in patient care without causing a substantial workload for providers.

This pilot study has several limitations. Our sample size was very small. However, we did observe a significant difference in several metrics related to the POCUS operator’s workload. The patient scenarios (undifferentiated hypotension and respiratory distress) were arbitrary, although they are among the most common patient presentations in intensive care units. The 2 protocols were compared in a hypothetical situation, in which the primary provider conducted the initial evaluation and resuscitation. It is not clear if the same conclusion can be drawn regarding a preferred POCUS-assisted protocol in a situation where a provider acts as a team leader while other team members conduct history taking, physical examination, and the POCUS examination.


In summary, a sequential POCUS-assisted protocol posed less workload to POCUS operators than a parallel protocol. For POCUS to be fully beneficial in clinical care, it needs to be integrated into current workflow without a substantial burden to providers. Our study supports the use of a sequential approach when POCUS is incorporated into the initial resuscitation and evaluation phase.



Focused Assessment with Sonography for Trauma


Interquartile range


National Aeronautics and Space Administration Task Load Index


Postgraduate year


Point-of-care ultrasound


  1. 1.

    Ferrada P, Vanguri P, Anand RJ, Whelan J, Duane T, Aboutanos M, Malhotra A, Ivatury R. A, B, C, D, echo: limited transthoracic echocardiogram is a useful tool to guide therapy for hypotension in the trauma bay--a pilot study. J Trauma Acute Care Surg. 2013;74(1):220–3.

  2. 2.

    Hollister N, Bond R, Donovan A, Nicholls B. Saved by focused echo evaluation in resuscitation. Emerg Med J. 2011;28(11):986–9.

  3. 3.

    Gunst M, Sperry J, Ghaemmaghami V, O'Keeffe T, Friese R, Frankel H. Bedside echocardiographic assessment for trauma/critical care: the BEAT exam. J Am Coll Surg. 2008;207(3):e1–3.

  4. 4.

    Pershad J, Myers S, Plouman C, Rosson C, Elam K, Wan J, Chin T. Bedside limited echocardiography by the emergency physician is accurate during evaluation of the critically ill patient. Pediatrics. 2004;114(6):e667–71.

  5. 5.

    Copetti R, Soldati G, Copetti P. Chest sonography: a useful tool to differentiate acute cardiogenic pulmonary edema from acute respiratory distress syndrome. Cardiovasc Ultrasound. 2008;6:16.

  6. 6.

    Chin EJ, Chan CH, Mortazavi R, Anderson CL, Kahn CA, Summers S, Fox JC. A pilot study examining the viability of a prehospital assessment with UltraSound for emergencies (PAUSE) protocol. J Emerg Med. 2013;44(1):142–9.

  7. 7.

    Bagheri-Hariri S, Yekesadat M, Farahmand S, Arbab M, Sedaghat M, Shahlafar N, Takzare A, Seyedhossieni-Davarani S, Nejati A. The impact of using RUSH protocol for diagnosing the type of unknown shock in the emergency department. Emerg Radiol. 2015;22(5):517–20.

  8. 8.

    Manno E, Navarra M, Faccio L, Motevallian M, Bertolaccini L, Mfochive A, Pesce M, Evangelista A. Deep impact of ultrasound in the intensive care unit: the "ICU-sound" protocol. Anesthesiology. 2012;117(4):801–9.

  9. 9.

    Kanji HD, McCallum J, Sirounis D, MacRedmond R, Moss R, Boyd JH. Limited echocardiography-guided therapy in subacute shock is associated with change in management and improved outcomes. J Crit Care. 2014;29(5):700–5.

  10. 10.

    Zieleskiewicz L, Muller L, Lakhal K, Meresse Z, Arbelot C, Bertrand PM, Bouhemad B, Cholley B, Demory D, Duperret S, et al. Point-of-care ultrasound in intensive care units: assessment of 1073 procedures in a multicentric, prospective, observational study. Intensive Care Med. 2015;41(9):1638–47.

  11. 11.

    Bernier-Jean A, Albert M, Shiloh AL, Eisen LA, Williamson D, Beaulieu Y. The diagnostic and therapeutic impact of point-of-care ultrasonography in the intensive care unit. J Intensive Care Med. 2017;32(3):197–203.

  12. 12.

    Hart SG. NASA-task load index (NASA-TLX); 20 years later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 2006;50(9):904–8.

  13. 13.

    Hart SG, Staveland LE. Development of NASA-TLX (task load index): results of empirical and theoretical research. Adv Psychol. 1988;52:139–83.

  14. 14.

    Arrillaga A, Graham R, York JW, Miller RS. Increased efficiency and cost-effectiveness in the evaluation of the blunt abdominal trauma patient with the use of ultrasound. Am Surg. 1999;65(1):31–5.

  15. 15.

    Rose JS, Levitt MA, Porter J, Hutson A, Greenholtz J, Nobay F, Hilty W. Does the presence of ultrasound really affect computed tomographic scan use? A prospective randomized trial of ultrasound in trauma. J Trauma. 2001;51(3):545–50.

  16. 16.

    Jones AE, Tayal VS, Sullivan DM, Kline JA. Randomized, controlled trial of immediate versus delayed goal-directed ultrasound to identify the cause of nontraumatic hypotension in emergency department patients. Crit Care Med. 2004;32(8):1703–8.

  17. 17.

    Scalea TM, Rodriguez A, Chiu WC, Brenneman FD, Fallon WF Jr, Kato K, McKenney MG, Nerlich ML, Ochsner MG, Yoshii H. Focused assessment with sonography for trauma (FAST): results from an international consensus conference. J Trauma. 1999;46(3):466–72.

  18. 18.

    Shokoohi H, Boniface KS, Pourmand A, Liu YT, Davison DL, Hawkins KD, Buhumaid RE, Salimian M, Yadav K. Bedside ultrasound reduces diagnostic uncertainty and guides resuscitation in patients with undifferentiated hypotension. Crit Care Med. 2015;43(12):2562–9.

  19. 19.

    Wang XT, Liu DW, Zhang HM, Chai WZ. Integrated cardiopulmonary sonography: a useful tool for assessment of acute pulmonary edema in the intensive care unit. J Ultrasound Med. 2014;33(7):1231–9.

  20. 20.

    American College of Emergency Physicians. Emergency ultrasound guidelines. Ann Emerg Med. 2009;53(4):550–70.

  21. 21.

    Frankel HL, Kirkpatrick AW, Elbarbary M, Blaivas M, Desai H, Evans D, Summerfield DT, Slonim A, Breitkreutz R, Price S, et al. Guidelines for the appropriate use of bedside general and cardiac ultrasonography in the evaluation of critically ill patients-part I: general ultrasonography. Crit Care Med. 2015;43(11):2479–502.

  22. 22.

    Melniker LA, Leibner E, McKenney MG, Lopez P, Briggs WM, Mancuso CA. Randomized controlled clinical trial of point-of-care, limited ultrasonography for trauma in the emergency department: the first sonography outcomes assessment program trial. Ann Emerg Med. 2006;48(3):227–35.

Download references

Availability of data and materials

Available on request.

Author information

CB designed the study, collected and interpreted the data, wrote and reviewed the manuscript. SS interpreted data and wrote and reviewed the manuscript. NJ participated in study design, interpretation of data, and review of the manuscript. AK participated in study design, interpretation of data, and review of the manuscript. OG participated in study design, contributed expert knowledge, and reviewed the manuscript. HS participated in study design, contributed expert knowledge, interpreted data, and revised the manuscript. All authors reviewed and approved the final version before submission.

Correspondence to Courtney E. Bennett.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the Mayo Clinic Institutional Review Board (12–007998).

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark


  • Critical illness
  • Point-of-care
  • Simulation
  • Ultrasound