Interobserver reliability of echocardiography for prognostication of normotensive patients with pulmonary embolism

Objectives To evaluate the interobserver reliability of echocardiographic findings of right ventricle (RV) dysfunction for prognosticating normotensive patients with pulmonary embolism (PE). Methods A central panel of cardiologists evaluated echocardiographic studies of 75 patients included in the PROTECT study for the following signs: RV diameter, RV/left ventricular (LV) diameter ratio, hypokinesis of the RV free wall, and tricuspid plane systolic excursion (TAPSE). Investigators used intraclass correlation to assess agreement between the measurements of the central panel and each of the local cardiologists. Investigators used the single weighted kappa statistic to test for agreement between readers of interpretation of RV enlargement and RV hypokinesis. Results The two observers had fair agreement (k = 0.45) for RV enlargement assessed by the RV diameter, and good agreement (k = 0.65) for RV enlargement assessed by the RV/LV diameter ratio. The interobserver reliability of the assessment whether hypokinesis of the RV free wall is present was good (к = 0.70), and whether RV dysfunction (assessed by TAPSE measurement) is present was very good (k = 0.86). The intraclass correlation for the RV/LV diameter ratio was fair (0.55; 95% confidence interval [CI], 0.37-0.69), for the RV diameter was good (0.70; 95% CI, 0.56-0.80), and for the TAPSE measurement was very good (0.85; 95% CI, 0.77-0.90). On Bland-Altman analysis, the mean differences for RV diameter, RV/LV diameter ratio and TAPSE measurement were 2.33 (±5.38), 0.06 (±0.23) and 0.08 (±2.20), respectively. Conclusion TAPSE measurement is the least user dependent and most reproducible echocardiographic finding of RV dysfunction in normotensive patients with PE.


Introduction
Acute pulmonary embolism (PE) is a common disease with a 3-month mortality rate of up to 17.4% [1][2][3][4]. Even if PE is properly treated with anticoagulation, the mortality rate in hemodynamically stable patients varies from 8.1% to 15.1% [4,5]. Death is usually caused by acute right heart failure [4][5][6][7][8][9]. Acute PE increases the pressure of the pulmonary arterial system and right ventricle (RV) resulting in RV dysfunction, which may progress to right heart failure and circulatory collapse [5,6]. Patients with RV dysfunction have a higher mortality rate than those without, even if they are initially hemodynamically stable [6,7]. Thus, the presence of RV dysfunction is a marker for adverse clinical outcome in patients with acute PE [6][7][8].
Transthoracic echocardiography (TTE) is the most common first-line examination to diagnose the signs of RV dysfunction [6][7][8][9]. Echocardiography is capable of visualizing the changes occurring in the morphology and function of the right ventricle as a result of acute pressure overload. A variety of different methods for the assessment of RV dysfunction on TTE have been proposed and the literature shows variable results for the prognostic power of TTE signs of RV dysfunction to predict adverse outcomes [10]. This variability may in part be explained by the somewhat subjective nature of diagnosing RV dysfunction on TTE because formal criteria for establishing these signs are not available. It is noteworthy that prior publications on this topic did not report interobserver reproducibility of the findings.
Accordingly, the purpose of our study was to determine the interobserver reproducibility of TTE findings previously described to indicate RV dysfunction with the goal of identifying the most robust, least observer dependent method.

Study design
This was a sub analysis of the first 75 patients enrolled in the PROTECT, a prospective, multicenter observational cohort study designed by the authors (see Appendix) and sponsored by the Institute of Health Carlos III, Spain (NCT00880737) [11]. Local ethics committees approved the study. All patients provided written informed consent.

Patients
Only patients diagnosed with pulmonary embolism by multidetector CT were eligible [12]. Exclusion criteria consisted of treatment with thrombolytics at the time of PE diagnosis, life expectancy less than 3 months, pregnancy, geographic inaccessibility precluding follow-up, age younger than 18 years, renal insufficiency (creatinine clearance < 30 mL/min), inability to complete CT testing (e.g., allergy to intravenous contrast agents, unavailability of CT, patient too ill), or hemodynamic instability at presentation (defined as cardiogenic shock, systolic blood pressure < 90 mmHg, or use of inotropic support). We also excluded patients that did not successfully complete the protocol-required transthoracic echocardiography.

Examinations
The study required that patients undergo echocardiography (i.e., TTE) within 24 hours after diagnosis of PE. Patients underwent testing in the left lateral position. Trained and certified local cardiologists, blinded to the patient's clinical data and laboratory test results, performed and interpreted each echocardiogram. This sub study defined echocardiographic RV dysfunction as the presence of dilatation of the right ventricle (end-diastolic diameter > 30 mm from the parasternal view or the right ventricle appearing larger than the left ventricle from the subcostal or apical view), hypokinesis of the right ventricle free wall (any view), or a tricuspid plane systolic excursion (TAPSE) of 1.6 cm or less. TAPSE was measured as the total displacement of the tricuspid annulus (centimeters) from end-diastole to end-systole, with values representing the average TAPSE of three to five beats [13].
Local cardiologists recorded all examinations on digital format for off-line blinded re-evaluation by one of the echocardiographers from the central panel (S.B., M.R. and H.C.) (25 studies each).
Statistical significance was defined as a two-tailed P-value of <0.05 for all analyses. Analyses were performed using SPSS, version 15.0 for the PC (SPSS, Inc. Chicago, IL, USA).

Results
Transthoracic echocardiography was technically inadequate in 2 of the 75 patients who were enrolled in this

Ratio of the RV to the LV short axis
The mean RV to left ventricle ratios were 0.83 ± 0.28 for local and 0.88 ± 0.20 for central cardiologist, respectively. The intraclass correlation was fair (0.55; 95% CI, 0.37-0.69).
On Bland-Altman analysis of RV/LV ratio measurements, the means and standard deviation (SD) between central and local cardiologists were 0.06 and 0.23, respectively ( Figure 2). For the ratio of the RV to the LV short axis the observers agreed that 52 patients (71%; 95% CI, 61-82%) were free of RV dysfunction. They agreed upon the presence of RV dysfunction in 12 patients (16%; 95% CI, 7.9-25%). Disagreement existed in 9 patients (12%; 95% CI, 4.8-20%) ( Table 3). The interobserver agreement reflecting the presence or absence of RV dysfunction was good with a weighted kappa of 0.65 (95% CI, 0.44-0.86).

Discussion
In this study, we aimed at analyzing the observer dependence of establishing the echocardiographic signs of RV dysfunction that have hitherto been described in the literature to identify the most robust and reproducible method. Our results suggest that considerable interindividual differences exist in the reproducibility of echocardiographic signs of RV dysfunction. TAPSE measurement is the least user dependent and most reproducible. Studies have recognized RV dysfunction as a key determinant of prognosis in PE [16]. Echocardiographic findings suggesting RV dysfunction have been reported to occur in at least 25% of PE patients [17]. A metaanalysis found more than a two-fold increased risk of PE-related mortality in patients with echocardiographic signs of RV dysfunction [18]. Two out of the seven studies included an estimation of risk in normotensive   patients with PE [7,19]. In such patients RV dysfunction had sensitivity of 56-61% and was related to the absolute increase in the early PE-related mortality of 4-5% [18]. Importantly, patients with normal echocardiographic findings had an excellent outcome, with in hospital PErelated mortality less than 1% in most of the reported series [6,7,19]. The most important limitations are the lack of standardization of the echocardiographic criteria [10], and the somewhat subjective nature of diagnosing RV dysfunction on pulmonary echocardiography. Our results show that the interobserver reliability was higher for qualitative abnormalities on transthoracic echocardiography (i.e., hypokinesis of the RV free wall). There are several potential explanations for these findings. Moderate or severe RV dysfunction is usually identified qualitatively, and it is ordinarily readily apparent to observers even with only modest experience [8]. Moreover, RV dimensions are highly dependent on probe rotation by the user, which can result in an underestimation of RV width [20]. For normotensive patients with acute symptomatic PE, TAPSE is independently predictive of survival [21]; however no studies had previously assessed interobserver reliability for this parameter in these patients. In the initial validation study by Kaul et al. [22], TAPSE correlated strongly with radionuclide angiography, with low interobserver variability. For PE patients, our results confirm that interobserver variability for TAPSE measurements was very low and lower than for all other echocardiographic signs of RV dysfunction investigated here.
The findings of this study might have clinical implications. Some authors have proposed that normotensive patients with RV dysfunction on echocardiography should potentially undergo thrombolytic therapy [23]. This study detected only fair reproducibility of the RV end-diastolic diameter and the RV diameter/left ventricular diameter ratio. Thus, findings from this study do not adequately justify use of these parameters to drive decision-making regarding thrombolytic therapy.
Besides the fact that this study is the first study to report upon the clinical implication of interobserver reliability on the measurement of echocardiographic RV dysfunction in normotensive patients with acute PE, this study has other strengths. Particularly, this is the first study that reports on the interobserver reliability of TAPSE. Our study has several limitations. We were not able to assess the interobserver reproducibility of systolic pulmonary pressure and other echocardiographic criteria for RV dysfunction (e.g., systolic excursion velocity of the tricuspid annulus). Though with 3D echocardiography there is less underestimation of RV end-diastolic  and end-systolic volumes and improved test-retest variability compared with 2D echocardiography [24], investigators did not perform volumetric analyses of the RV. Furthermore, our results should be interpreted with caution due to the limited sample size.
In conclusion, this is the first study that systematically assessed the interobserver reliability of echocardiographic findings of RV dysfunction in normotensive patients with acute PE. We found considerable differences in the interobserver reproducibility of these findings. TAPSE measurement is the least user dependent and most reproducible. If these signs are used in clinical practice to make patient management decisions, practitioners should be aware of the variable degree of subjectivity and reproducibility associated with these observations.