This article has Open Peer Review reports available.
Reproducibility of 3-dimensional ultrasound readings of volume of carotid atherosclerotic plaque
© Ludwig et al; licensee BioMed Central Ltd. 2008
Received: 10 July 2008
Accepted: 26 August 2008
Published: 26 August 2008
Non-invasive 3-dimensional (3D) ultrasound (US) has emerged as the predominant approach for evaluating the progression of carotid atherosclerosis and its response to treatment. The aim of this study was to investigate the quality of a central reading procedure concerning plaque volume (PV), measured by 3D US in a multinational US trial.
Two data sets of 45 and 60 3D US patient images of plaques (mean PV, 71.8 and 39.8 μl, respectively) were used. PV was assessed by means of manual planimetry. The intraclass correlation coefficient (ICC) was applied to determine reader variabilities. The repeatability coefficient (RC) and the coefficient of variation (CV) were used to investigate the effect of number of slices (S) in manual planimetry and plaque size on measurement variability.
Intra-reader variability was small as reflected by ICCs of 0.985, 0.967 and 0.969 for 3 appointed readers. The ICC value generated between the 3 readers was 0.964, indicating that inter-reader variability was small, too. Subgroup analyses showed that both intra- and inter-reader variabilities were lower for larger than for smaller plaques. Mean CVs were similar for the 5S- and 10S-methods with a RC of 4.7 μl. The RC between both methods as well as the CVs were comparatively lower for larger plaques.
By implementing standardised central 3D US reading protocols and strict quality control procedures highly reliable ultrasonic re-readings of plaque images can be achieved in large multicentre trials.
Measurement of carotid atherosclerosis burden and progression is an important tool for research and patient management . 2-dimensional (2D) B-mode ultrasound (US) has been shown to be a sensitive and reproducible method to detect pre intrusive thickening of artery walls and to measure intima-media thickness (IMT) in the carotid artery [2, 3]. Ultrasonographically determined IMT has been used as a marker of atherosclerosis elsewhere in the arterial system and randomised imaging studies have examined the effects of blood pressure- and lipid-lowering therapies on carotid IMT changes . The atherogenic nature of IMT, however, is uncertain, since 2D US does not measure medial and intimal thickness separately . An increase in IMT may be the result of an adapted response of medial layer (remodelling) to tensile (hypertensive) stress or intimal thickening reflecting early atherosclerosis . Atherosclerotic plaque volume assessed by 3-dimensional (3D) US  may represent a more reliable measure of atherosclerosis than IMT  and more recently non-invasive 3D US imaging has ermerged as the predominant approach for evaluating the progression of carotid atherosclerosis [1, 9–12]. 3D US provides a precise and reproducible method for determining the change in atherosclerotic plaque volume during treatment [1, 8, 10]. A recent randomised trial in hypertensive patients using carotid 3D US has successfully investigated the effects of two different classes of antihypertensive agents on plaque volume changes and demonstrated the suitability of the 3D US method for tracking progression or regression of plaque volume over time .
3D US measurement of PV within a multicentre clinical trial setting typically involves processing of data obtained at more than one investigational site. It is important to ensure that analysis of the data, and re-reading of the 3D US images of plaque is carried out in a centralised, standardised and reproducible manner. The primary aim of this study was to assess reliability of a central re-reading procedure implemented in a multinational 3D US trial by determining intra- and inter-reader variabilities and major factors influencing reading precision.
Materials and methods
The study was performed in accordance with the principles of the Declaration of Helsinki, and the regulatory requirements of the International Conference on Harmonisation Guidelines on Good Clinical Practice.
The protocol was approved by the appropriate Institutional Review Board or Ethics Committee at each centre involved.
All patients gave written informed consent for publication of this report and accompanying images.
Two data sets of 45 and 60 3D US carotid plaque images were obtained from 45 patients (32 men; mean [SD] age, 62.2 [6.9] years) and 60 patients (50 men; mean [SD] age, 61.3 [8.1] years), respectively, who had been followed up in the ACAT Inhibition Plaque Regression Study (APRES). The APRES was a multicentre European ultrasound trial, and participants were recruited at 31 clinical centers throughout the Czech Republic, Germany, Italy and Poland. The study assessed the effect of an ACAT-inhibitor as compared to placebo on carotid plaque volume (Data on file at Daiichi Sankyo, Munich, Germany). To be eligible for inclusion in the APRES, patients with defined cardiovascular risk had to have an increased common carotid artery (CCA) intima-media thickness (IMT) of > 0.8 mm, and at least one atherosclerotic plaque in the CCA or the carotid bulb without marked mineralisation (plaque volume between 20 and 500 μl).
Ultrasound measurement of plaque volume (PV)
Ultrasound measurements of PV had been carried out by trained and certified sonographers at 14 ultrasound referral centers using a high-resolution Voluson 530 D MT, 2D-/3D CFM-Ultrasound System (Kretz-Technik AG, Zipf, Austria; the equipment is commercially available from General Electric, GE Ultraschall Deutschland GmbH/D-42655 Solingen and named Logiq 7 BT07) equipped with a mechanical 10 MHz sector motor-driven 3D-probe that provided an axial resolution of 0.1 mm. An integrated 6.9 MHz Doppler system was used to determine the grade of possible stenoses.
Quality assessment of central reading
The first set of data consisted of 45 3D US plaque images with PV ranging from 21 to 240 μl. This data set was used to compare the results of plaque volume measurements obtained by 3 trained and certified readers at the European Ultrasound Teaching and Reading Center (EUTARC; Feldafing, Germany) and to investigate intra- and inter-reader variability of the reading process. The second data set consisted of 60 plaque images (PV between 20 and 60 μl) and was used to explore the effect of plaque size and the number of slices (S) (5 vs. 10 S) in manual planimetry on the quality of the central reading procedure by a randomly assigned reader.
Study protocol and statistical analysis
Sonographic images of the 45 plaques of the first data set were recorded on video tapes and MODs, and 6 identical blinded copies were prepared by an independent institution (Medizinische Software GbR [MESO], Mittweida, Germany) and named A to F. Three sets of the 45 MODs were then sent to the Reading Center EUTARC and randomly assigned to 3 appointed readers for subsequent evaluation. After the measurements from the first round of reading procedure had been returned to MESO, the remaining 3 sets of MODs were delivered to the 3 readers for a second round of readings. Thus, each reader performed two independent evaluations of the 45 MODs according to a randomized, pre-defined procedure. The mean of 3 individual readings was used for analysis.
Inter-reader variability was evaluated by comparing the mean of the first and the second measurement of each reader between all 3 readers. Assessment of intra-reader variability was based on comparison between the first and the second set of measurements of each reader. The intraclass correlation coefficient (ICC) was applied to assess inter- and intra-reader variabilities . The ICC is a measure of the proportion of variance that is attributable to the objects of measurements, and has emerged as a universal and widely accepted reliability index . Furthermore, it allows the evaluation of the variability between more than 2 methods and readers, respectively. Calculation of the ICC was carried out by means of analysis of covariance (ANCOVA) using the SAS program (version 9.1). The three ICC values between the first and second set of measurements of each reader provide an estimation of intra-reader variability, and the ICC between all 3 readers is a measure of inter-reader variability. An additional analysis was performed after stratification by plaque size (PV < 60 μl versus PV > 60 μl) as determined in the APRES.
The second data set was used to investigate the effect of the number of slices in manual planimetry on the quality of PV measurement. The results of the 5 S- and 10 S- method were compared by means of the repeatability coefficient (RC) based on the mean of 3 individual readings per plaque. The RC was calculated as follows : RC = 1.96 SDDiff, where SDDiff denotes the standard deviation of the differences in PV measurements between both methods. If the differences follow a normal distribution, approximately 5 percent of differences between measurements are expected to lie outside the limits + RC. Variability of measurements was investigated by calculating the coefficient of variation (CV) for 3 individual assessments of each of the 60 plaques by either method. In order to analyse the effect of plaque size (PV) on the precision of measurement, CVs and RC were calculated separately for smaller plaques (PV between 20 and < 40 μl) and larger plaques (PV between 40 and 60 μl).
Intra- and inter-reader variability
Inter- and intra-reader variabilities for re-reading plaque volume (PV) of 45 plaques
PV reading 1 (μl)
PV reading 2 (μl)
PV both readings (μl)
Intra-class correlation coefficients for re-reading plaque volume (PV) of 21 plaques with PV < 60 μl and 24 plaques with PV ≥ 60 μl
PV < 60 μl
PV ≥ 60 μl
Effect of number of slices
Coefficients of variation (CV) and repeatability coefficients (RC) for 60 plaques comparing the 5S- and 10S- methods
5S vs. 10S
20 to < 40
40 to < 60
20 to < 60
Because carotid plaque progression is not limited to changes in one direction, it is important to measure progression in three dimensions. Plaques grow and regress circumferentially as well as in length and thickness. As a non-invasive technique, 3D imaging allows:
- direct plaque visualization
- quantification of plaque features
- the possibility of investigating volume changes that occur in multiple dimensions, such as plaque surface morphology, plaque geometry, and plaque distribution.
For these reasons 3D-ultrasound is becoming more important in serial monitoring of disease progression or regression. Sample sizes which are required to test the effects of new therapies might be smaller for measurements of plaque volume than for traditional 2D measurements.
The present study aimed to assess the quality of a centralised reading procedure of 3D US recordings of carotid PV measured in a multinational clinical trial. The small intra- and inter-reader variabilities reported in the study validate the reproducibility and reliability of the centralised PV measurement reading technique. Any slight differences observed between the 3 appointed readers may have reflected a minor subjective element due to differences in practice; however, these differences were small and randomly distributed, and would not be expected to have a significant effect on results generated in a clinical trial setting. Individual deviations between the first and second measurement made by the same reader, and between participating readers, were comparable with the occurrence of outliers in other methods of measurement applied in clinical trials, with respect to magnitude and frequency.
In the present study, variability of the re-reading procedure was dependent on plaque size. Both inter- and intra-reader variabilities were lower for larger than for smaller plaques. This finding is in agreement with previous PV variability studies which utilised 3D ultrasound [1, 14, 20, 21] showing that the CV in the measurement of PV decreased with plaque size. Due to irregular shape of some of the atheromatic plaques an increase in the number of slices used for calculation in manual planimetry method by decreasing the distance between slices could possibly increase the accuracy of volume determinations. Therefore, in the present study additional analyses were undertaken on a sample of 60 plaques to assess the variability depending on plaque size and the number of slices used during the determination of plaque volume. Half of plaques were between 20 and < 40 μl and half were between 40 and < 60 μl. The results indicate that the 5S- and the 10S-method provided similar results in volume calculation. Subgroup analyses of the two tracing techniques demonstrated that for the smaller plaques (PV between 20 and < 40 μl), the 10S-method offered slightly greater reliability as indicated by a lower coefficient of variation. In contrast, there was no difference between reliability of the 5S- or 10S-method in the larger plaque group (PV between 40 and < 60 μl). Therefore, the reliability of measurements of smaller plaques (< 40 μl) may be improved by using the 10S-method.
The small variability in 3D US PV measurements and re-readings is an important finding, given the fact that this approach may be useful for evaluating the progression of carotid atherosclerosis and its response to treatment and for targeting preventive therapy . Poor reproducibility could lead to inappropriate clinical management of individual patients, particularly given that the ratio of random measurement error to the variability among progression rates is large, and repeated measures or longer follow-up would be required to sufficiently reduce the contributions of random error to allow individual diagnoses .
It should be noted that a number of limitations can affect the 3D technique for measuring plaque volume. In a small number of cases, plaque boundaries might not be well defined due to dropouts and shadowing owing to attenuation of the US beam, and these might be present in the reconstructed 3D US images. Calcified plaques with marked shadowing are not measureable, and plaque identification at the carotid bifurcation and in areas of poor image resolution may in a few cases also create some difficulty in plaque identification. In addition, there is a higher variability in smaller plaques (< 40 μl), and the reliability of measurements of smaller plaques (< 40 μl) may be improved by using an increased number of slices (10 instead of 5 slices).
In summary, the centralized reading procedure investigated in this study has been shown to be reliable and reproducible. Variability in the reading process increases with decreasing plaque volumes, and the 10S-method may offer greater reproducibility than the 5S-method for volume assessment of small plaques.
The 3D US techniques combined with a well controlled centralised reading procedure described in this study are appropriate tools for monitoring progression of carotid atherosclerosis and its response to treatment. By implementing standardised central 3D US reading protocols and strict quality control procedures highly reliable ultrasonic re-readings of plaques images can be achieved in large multicentre trials.
Daiichi Sankyo, Munich, Germany provided funding for and were involved in the design, development and management of the APRES study. The analyses described in this paper were initiated and carried out independently without funding support from Daiichi Sankyo.
- Landry AM, Spence JD, Fenster A: Measurement of carotid plaque volume by 3-dimensional ultrasound. Stroke. 2004, 35: 864-869.View ArticlePubMedGoogle Scholar
- Graf S, Gariepy J, Massonneau M, Armentano R, Mansour S, Barra JG, Simon A, Levenson J: Experimental and clinical validation of arterial diameter waveform and intimal media thickness obtained from B-mode ultrasound image processing. Ultrasound Med Biol. 1999, 25: 1353-1363.View ArticlePubMedGoogle Scholar
- Ludwig M, Stapff M, Ribeiro A, Fritschka E, Tholl U, Smith RD, Stumpe KO: Comparison of the effects of losartan and atenolol on common carotid artery intima-media thickness in patients with hypertension: results of a 2-year, double-blind, randomized, controlled study. Clin Ther. 2002, 24: 1175-1193.View ArticlePubMedGoogle Scholar
- Lorenz MW, Markus HS, Bots M, Rosvall M, Sitzer M: Prediction of clinical cardiovascular events with carotid intima-media thickness. Circulation. 2007, 115: 459-467.View ArticlePubMedGoogle Scholar
- Persson J, Formgren J, Israelsson B, Berglund G: Ultrasound-determined intima-media thickness and atherosclerosis. Direct and indirect validation. Arterioscler Thromb. 1994, 14: 261-264.View ArticlePubMedGoogle Scholar
- Zanchetti A, Bond MG, Hennig M, Neiss A, Mancia G, Dal Palù C, Hansson L, Magnani B, Rahn KH, Reid JL, Rodicio J, Safar M, Eckes L, Rizzini P, : Calcium antagonist lacidipine slows down progression of asymptomatic carotid atherosclerosis. Principal results of the European Lacidipine Study on Atherosclerosis (ELSA), a randomized, double-blind, long-term trial. Circulation. 2002, 106: 2422-2427.View ArticlePubMedGoogle Scholar
- Fenster A, Downey DB, Cardinal HN: Three-dimensional ultrasound imaging. Phys Med Biol. 2001, 46: 67-99. 10.1088/0031-9155/46/5/201.View ArticleGoogle Scholar
- Mintz GS, Nissen SE, Anderson WD, Bailey SR, Erbel R, Fitzgerald PJ, Pinto FJ, Rosenfield K, Siegel RJ, Tuzcu EM, Yock PG: American College of Cardiology Clinical Expert Consensus Document on Standards for Acquisition, Measurement and Reporting of Intravascular Ultrasound Studies (IVUS): a report of the American College of Cardiology Task Force on Clinical Expert Consensus Documents. J Am Coll Cardiol. 2001, 37: 1478-1492.View ArticlePubMedGoogle Scholar
- Spence JD: Technology Insight: ultrasound measurement of carotid plaque-patient management, genetic research, and therapy evaluation. Nat Clin Pract Neurol. 2006, 2: 611-619.View ArticlePubMedGoogle Scholar
- Ludwig M, Willinek WA, von Buquoy M, Jörger U, Stumpe KO: Limitations of 2-dimensional (D)-ultrasound imaging for the quantitative assessment of common carotid artery atherosclerosis: superiority of high-resolution 3-D-ultrasonography. J Hypertens. 1998, 16 (Suppl 12): S104-Google Scholar
- Schminke U, Motsch L, Griewing B, Gaull M, Kessler C: Three-dimensional power-mode ultrasound for quantification of the progression of carotid artery atherosclerosis. J Neurol. 2000, 247: 106-111.View ArticlePubMedGoogle Scholar
- Spence JD: Ultrasound measurement of carotid plaque as a surrogate outcome for coronary artery disease. Am J Cardiol. discussion 15B–16B, 2002, 89: 10B-15B.View ArticlePubMedGoogle Scholar
- Stumpe KO, Agabiti-Rosei E, Zielinski T, Schremmer D, Scholze J, Laeis P, Schwandt P, Ludwig M: Carotid intima-media thickness and plaque volume changes following 2-year angiotensin II-receptor blockade. The Multicentre Olmesartan atherosclerosis Regression Evaluation (MORE) study. Ther Adv Cardiovasc Dis. 2007, 1: 97-106. 10.1177/1753944707085982.View ArticlePubMedGoogle Scholar
- Ludwig M, Jörger U, Willinek WA, Stumpe KO: Simultaneous quantitative assessment of wall thickness and size and volume of soft plaques in superficial large arteries by a new 3-dimensional ultrasound method. J Hypertens. 1996, 14 (Suppl 1): S263(P1231)-Google Scholar
- Ludwig M: 3D-Sonographie. Klinische Angiologie. 1997, 4.6: 137-175.Google Scholar
- Ludwig M, von Buquoy M, von Petzinger-Kruthoff A, Stumpe KO, Schremmer D: Reproducibility of 3D-ultrasound assessment of carotid plaque in patients with cardiovascular risk. Atherosclerosis. 2007, 8 (Suppl 1): S134-View ArticleGoogle Scholar
- Shrout PE, Fleiss JL: Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin. 1979, 86: 420-428. 10.1037/0033-2909.86.2.420.View ArticlePubMedGoogle Scholar
- Shoukri MM: Reliability for continuous scale measurements. Measures of interobserver agreement. Edited by: Shoukri MM. Boca Raton : Chapman & Hall/CRC, 2004.Google Scholar
- Bland JM, Altman DG: Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986, 1: 307-310.View ArticlePubMedGoogle Scholar
- Fenster A, Blake C, Gyacskov I, Landry A, Spence JD: 3D ultrasound analysis of carotid plaque volume and surface morphology. Ultrasonics. 2006, 44 (Suppl 1): e153-157.View ArticlePubMedGoogle Scholar
- Palombo C, Kozakova M, Morizzo C, Andreuccetti F, Tondini A, Palchetti P, Mirra G, Parenti G, Pandian NG: Ultrafast three-dimensional ultrasound. Application to carotid artery imaging. Stroke. 1998, 29: 1631-1637.View ArticlePubMedGoogle Scholar
- Espeland MA, Craven TE, Riley WA, Corson J, Romont A, Furberg CD, : Reliability of longitudinal ultrasonographic measurements of carotid intima-media thickness. Stroke. 1996, 27: 480-485.View ArticlePubMedGoogle Scholar
- Naghavi M, Falk E, Hecht HS, Jamieson MJ, Kaul S, Berman D, Fayad Z, Budoff MJ, Rumberger J, Naqvi TZ, Shaw LJ, Faergeman O, Cohn J, Bahr R, Koenig W, Demirovic J, Arking D, Herrera VL, Badimon J, Goldstein JA, Rudy Y, Airaksinen J, Schwartz RS, Riley WA, Mendes RA, Douglas P, Shah PK, : From vulnerable plaque to vulnerable patient-part III: executive summary of the Screening for Heart Attack Prevention and Education (SHAPE) Task Force Report. Am J Cardiol. 2006, 98 (Suppl 1): 2-15. 10.1016/j.amjcard.2006.03.002.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.