The reason for doing this is to assess the MRI findings. Objective: To systematically review the association between the location of knee pain and the location of abnormal imaging or arthroscopic findings.

Our patients are incredibly well educated when it comes to pain. Good patient histories are more important than looking at films. Some of the most common surgeries, such as arthroscopy for meniscal tears, are based on MRI findings, which have an increasingly high percentage of false-positive rates as we age.

MRI does not match up well with arthroscopic discoveries in meniscus tears. What did they find? When discussing MRI accuracy or reliability, people will often point to the age of the study.

because there is a belief that technology is moving so rapidly that anything more than a few years, months, or even weeks old is already outdated. That is a misnomer. Look at this recent research and we will bring it current.

A July paper published in the BMJ open quality 22 suggested a problem with overreliance of MRIs in the United Kingdom. As you can see from the associated research presented here, a very similar problem to that seen here in the United States.

In this study the researchers saw that the musculoskeletal injury or pain was the cause for the largest proportion of general practitioner recommended magnetic resonance imaging MRI. In September , doctors at Yale-New Haven Health at Bridgeport Hospital and Boston University School of Medicine write in the journal Radiologic Clinics of North America 3 that:.

In a study doctors writing in the Archives of Physical Medicine and Rehabilitation 4 made this observation: we saw evidence that weight-bearing MRI evaluations based on current imaging protocols are compatible with patients reporting mild to moderate knee osteoarthritis-related pain.

The finding? MRI confirms what you told your doctor, you have knee pain. The MRI is demonstrating confusion. As noted in this MRI report, surgical changes are demonstrated in medial meniscus with smaller than the expected size of the body of medial meniscus altered signal intensity in body and posterior horn of medial is extending to inferior articular surface demonstrates similar appearance to previous outside MRI to see the represents residual changes from prior surgery and meniscus tear or recurrent are persistent from prior examination.

Now, what does this mean? MRI of the knee without contrast noted or changes in the medial meniscus. Even the radiologist cannot determine whether this represents recurrent meniscus tear or is just post-surgical changes.

In July doctors at Queen Elizabeth Hospital in the United Kingdom and the University Medical Centre, Rotterdam, The Netherlands write in the European Journal of Orthopaedic Surgery and Traumatology : 5.

The assessment of a patient with chronic hip pain can be challenging. Magnetic resonance imaging MRI arthrography of the hip has been widely used now for the diagnosis of articular pathology of the hip.

Our study conclusions are MRI arthrogram is a useful investigation tool in detecting hip labral tears , it is also helpful in the diagnosis of femoro-acetabular impingement.

Was the meniscus damage from previous surgery or the continued and new degenerative changes in the knee? Hershey Medical Center published in the journal Clinical medicine insights. Arthritis and musculoskeletal disorders.

As a result, its diagnostic efficacy and reliability come into question. Specifically, in the field of orthopedics, there has been little discussion on the problems many physicians face while using MRIs in practice.

To gauge the perceived limitations of MRI, the researchers designed a study to assess the utility of MRIs and estimate the number of inconclusive MRIs ordered within an orthopedic practice to explore potential alternative avenues of diagnosis.

A survey was given to board-certified practicing orthopedic surgeons asking about the value, reliability, and diagnostic utility of MRIs in preoperative planning in shoulder and knee surgery. Prior surgery would limit the diagnostic accuracy including hardware distortions.

The surgeons also questioned the problems of identifying cartilage defects. In doctors at leading medical universities in South Korea were published in the medical journal Knee Surgery and Related Research. The above research should not be particularly shocking to a patient with chronic knee pain.

We see many patients with MRIs, and for many of those patients, the MRI has failed to come up with a good treatment plan for them. That is why they are visiting us. In other words — the MRI pointed out the obvious, but it had difficulty pointing out the less obvious.

It could not offer help if the image was too difficult to interpret, or were of no use at all for recommending a treatment plan. What we often see when those patients come to our office and we do an evaluation is even though their MRI may be normal we actually can pick up on joint instabilities and ligament laxity is with ultrasound evaluation.

Ultrasound also allows us and the patient to see the problems in real-time. A January review study in the journal Acta Biomedica 9 suggested that despite several limitations typical of MRI, one is that they are static , imaging in non-recumbent position under physiological stress allows detection of load-induced physiological and pathological variations Simply, by standing up you can see how the weight of the patient is causing stress on the knee.

The most relevant results up to date showed that this new diagnostic instrument allows recognizing both the meniscal tear stability and a latent instability, making it possible to correctly guide the orthopedic surgeon towards the treatment management.

This is seen as being helpful to understand how to proceed with surgery. Moreover, upright MRI allows to accurately understand patellofemoral kinematics during painful activities, helping to differentiate mal-trackers from non-maltrackers and to improve treatment for patellofemoral pain.

Three images were taken. Non-weight bearing flat on the back. Standing knee straight Squatting at degree angle.

In a study, 11 University researchers in Turkey published their research calling into question MRI accuracy in ACL readings. They found that ACL damage in degenerated knees was much more difficult to determine than an acute injury and in fact were:.

So as you can see an MRI can sometimes be detrimental to designing a treatment program for the patient, especially a surgical treatment program.

Asking the patient questions about their pain has always been a primary component of our initial consultation. Now it is considered a sound scientific device in recent research comparing taking a patient history to MRI accuracy.

Doctors in Korea writing in BioMed Central Musculoskeletal Disorders 12 examined whether MRI findings are of value in predicting the degree of knee joint laxity as measured using two typical physical examinations, i.

Now here is another study on the problems of surgery selections based on MRIs. This comes from doctors of the British Navy publishing in their Journal of the Royal Naval Medical Service. Getting back to the discussion of MRI for knee osteoarthritis and the subsequent use of MRI imaging to send patients to possible unnecessary knee replacement surgery, we find that a lot of research suggests that the decision to have knee replacement surgery should be made after a physical examination and consultation.

Unfortunately many times the decision is left to the interpretation of a scan or X-ray that may not provide the doctor with an accurate assessment. Before you say to yourself, this study is ten years old, we remind you that the research presented in this article that is supportive of these finding are from the last year or two.

Writing in the medical journal Arthritis , 14 doctors in Turkey wrote:. Semiquantitative assessment this is defined as non-precise — or subject to interpretation of the joints by expert interpreters of MRI data is a powerful tool that can increase our understanding of the natural history of this complex disease.

Several reliable and validated semiquantitative scoring systems now exist and have been applied to large-scale, multicentre, cross-sectional, and longitudinal observational epidemiological studies. Such approaches have advanced our understanding of the associations of different tissue pathologies with pain and improved the definition of joint alterations that lead to disease progression.

Although these new scoring systems offer theoretical advantages over pre-existing systems, whether they offer actual superiority with regard to reliability, responsiveness and validity remain to be seen. Therefore treatment of knee osteoarthritis could be planned according to the clinical features and functional status instead of radiological findings.

In both our opinion and that of certain researchers, physical examination and patient history are superior to the current MRI technology.

In looking at the widespread use of MRI to identify joint disease, the study says that semiquantitative assessment which is a non-precise — subject to interpretation reading of the joints by expert interpreters of MRI data is a powerful tool that can increase understanding of joint disease in osteoarthritis.

Using a tried and testing scoring system for different joint diseases, doctors can precisely diagnose problems of the joint as seen on the MRI. BUT, the researchers warn — it is still not accurate! The entire study points to the use of MRI as a valuable tool until the end, which states, in theory, this should work, but that remains to be seen.

Supporting this finding is another research paper: Incredibly the paper cites that there is not much by way of published literature to help a doctor to have a proper consultation.

Listen to what Arthritis Research United Kingdom Primary Care Centre doctors had to say: The ideal consultation for a patient presenting with possible Osteoarthritis is not known. The aim of the study was to develop the content of a model Osteoarthritis consultation for the assessment and treatment of older adults presenting in general practice with peripheral joint problems.

There is a lack of consensuses of how to perform the proper consultation in determining how to help a patient. Here is what these researchers came up with: The model Osteoarthritis consultation included 25 tasks to be undertaken during the initial consultation between the doctor and the patient presenting with peripheral joint pain.

The 25 tasks provide detailed advice on how the following elements of the consultation should be addressed:.

This study has enabled the priorities of the doctors and patients to be identified for a model Osteoarthritis consultation. A December paper 21 from radiologists at Ain Shams University in Cairo examined and tested the diagnostic reliability of a knee ultrasound for the evaluation of meniscus and collateral ligaments damage.

They then compared these results with a knee MRI. Therefore, ultrasonography has been suggested as an effective rapid alternative in many knee abnormalities, especially after injuries of the meniscus and collateral ligaments.

Doctors at The Steadman Philippon Research Institute wrote in the medical journal Foot and Ankle International examined MRI readings in the ankle. Doctors looked at a group of patients: Since these patients were seeking relief of pain it can be assumed that these were failed ankle surgeries.

One could give an opinion that MRI and surgical confirmation must be balanced and checked by a physical examination in the clinical setting. One could also ask the question — why not just get the physical examination in the first place and put off the MRI and arthroscopic intervention?

In July doctors writing in the journal Knee Surgery, Sports Traumatology, Arthroscopy 18 tried to determine the reliability and validity of preoperative magnetic resonance imaging MRI scan for the detection of additional pathologies in patients with chronic ankle instability compared to arthroscopic findings.

To do this they looked at thirty patients. What did they find in these thirty patients? Chronic ankle instability is associated with a high incidence of additional pathologies.

In some cases, MRI delivers insufficient results, which may lead to misinterpretation of present comorbidities. MRI is a helpful tool for preoperative evaluation, but arthroscopy remains gold standard in the diagnosis of associated lesions in patients with chronic ankle instability.

In our article, Is your MRI sending you to a back surgery you do not need? we seek to offer one simple piece of information. Your MRI may be sending you to a spinal surgery you do not need. We support this simple idea with a lot of research and our observations in the many patients we have seen after failed back surgery syndrome.

Please visit that article for a comprehensive understanding of the role of MRIs in back surgery preparation. For one thing, a doctor has to ask the right questions and do a little investigation. In trying to determine the best testing methods for spinal fusion surgery success prior to the surgery, investigators had this to say:.

Spinal fusion is a common but controversial treatment for chronic low back pain. In an effort to test whether any pre-fusion test could be performed to increase satisfactory surgeries and make fusion less controversial, different diagnostic tests including MRI were examined.

In the end, no tests in patients with chronic low back pain could be identified for whom spinal fusion is a predictable and effective treatment. Best evidence does not support the use of current tests for patient selection in clinical practice.

Please see this article on Failed Back Surgery Risks In this article I explain why MRI readings may lead to INCREASED incidents of failed back surgery syndrome. Our patients are incredibly well educated when it comes to their pain. That is why we think it is better to talk than look at films.

We hope you found this article informative and it helped answer many of the questions you may have surrounding issues with MRIs. If you would like to get more information specific to your challenges please email us: Get help and information from our Caring Medical staff.

Subscribe to our newsletter. Association between knee pain location and abnormal imaging or arthroscopic findings: A systematic review. Annals of Physical and Rehabilitation Medicine. Accuracy of standard magnetic resonance imaging sequences for meniscal and chondral lesions versus knee arthroscopy.

ANZ Journal of Surgery. Imaging in Osteoarthritis. Radiologic Clinics of North America. The validity and accuracy of MRI arthrogram in the assessment of painful articular disorders of the hip.

Upright Magnetic Resonance Imaging Tasks in the Knee Osteoarthritis Population: Relationships Between Knee Flexion Angle, Self-Reported Pain, and Performance. Arch Phys Med Rehabil. doi: Reliability and accuracy of MRI in orthopedics: A survey of its use and perceived limitations.

When they move back into place, they send signals for a computer to analyze. A computer creates the 2D images that your radiologist reads. That information can help your practitioner make a diagnosis and develop your treatment plan.

When used to detect cancer early, an MRI can help you avoid unnecessary or intrusive tests or interventions. Because an MRI machine uses a magnetic field and not ionizing radiation, an MRI is safer than tomography and X-rays. As a radiological modality, MRI findings can also pinpoint the size and location of tumors, lesions, and injuries.

MRI scans outperform CT scans for detecting some cancers such as uterine and prostate cancer and some liver cancers. An MRI also shows brain and bone metastases meaning cancer has spread from the initial site more clearly than a CT scan. Researchers look at three indicators to determine MRI accuracy: sensitivity, specificity, and the positive predictive value PPV.

The increased sensitivity in MRI-assisted cancer staging may lead to more accurate diagnoses, which means more extensive breast surgery. If your MRI indicates additional cancer, talk to your doctor about your treatment options.

A study published in the New England Journal of Medicine rated the sensitivity of three methods of invasive breast cancer detection. In the study, the specificity was However, keep in mind that the higher percentages for clinical exam and mammography are related to the delay in finding invasive cancer.

Dense breast tissue can also return a false-positive in a clinical exam. However, a 3D mammogram finds more cancers than traditional 2D mammograms , including those hidden by dense breast tissue. Research indicates MRI scans may be more accurate than other prostate cancer detection methods, especially biopsies.

This means a prostate MRI may help avoid unneeded biopsies. Urinary tract infection is the most common infectious complication of prostate biopsy, occurring in 1 to 11 percent of patients.

Men with elevated PSA levels who want more information about their prostate before undergoing a biopsy can get a prostate MRI with contrast. It will indicate with better accuracy where any potential cancer may be prior to undergoing a biopsy.

As a biopsy involves random tissue samples, it can miss a cancerous region of the prostate. This means MRI results can also help avoid missed diagnoses.

Beyond offering more accurate cancer detection, MRIs can also better evaluate orthopedic issues than other diagnostic modalities such as X-rays. In addition to being safer than an X-ray because there is no radiation , an MRI is more accurate than an X-ray.

A musculoskeletal MRI is important for assessing the results of orthopedic sometimes orthopaedic surgery for arthroscopic repairs, such as meniscal tears or anterior cruciate ligament injuries.

Having an MRI before a biopsy means radiologists can identify exactly where cancer may be. Marking a suspicious area aids a targeted needle biopsy. Quite possibly, the most significant benefit of MRI is whole-body imaging. The ability to scan head-to-toe means MRIs can detect cancer throughout the body.

A medical provider will often order an MRI scan, but you can also book a scan on your own. Detecting cancer before symptoms occur leads to better treatment plans and a better prognosis. At some sites, you will be provided with a headset and can select a Spotify playlist to help you relax during the scan.

Radiologists and other health care providers discourage having an MRI during the first trimester. If you have had such a response, discuss it with your doctor, technologist, or radiologist, especially if you also have kidney disease. However, rest assured that ezra does not use contrast material for the full body scan.

Early and accurate detection offers peace of mind. Ezra takes safety to heart and has protocols such as thorough cleanings between exams, social distancing in waiting rooms, and providing MRI-compatible masks to wear during scans.

We encourage you to get your annual cancer screening appointments booked. Do you have a loved one who could benefit from an Ezra scan? Purchase one of our Ezra gift cards. You can also schedule a call with our team to learn more. Contact us at or hello ezra.

How It Works.

An MRI could, in fact, even help you avoid an unnecessary biopsy. For example, a prostate MRI can help doctors tell the difference between harmless and aggressive cancers better than a biopsy. MRI accuracy is high when detecting breast cancer as well.

Comparatively, the combined ultrasound and mammography detection rate was just MRI images are often clearer and more detailed than other imaging methods, which makes them a more accurate detection method. An MRI machine is a large, tube-shaped cylinder. The walls of the tube hold powerful magnets involved in the scan.

When they move back into place, they send signals for a computer to analyze. A computer creates the 2D images that your radiologist reads. That information can help your practitioner make a diagnosis and develop your treatment plan.

When used to detect cancer early, an MRI can help you avoid unnecessary or intrusive tests or interventions. Because an MRI machine uses a magnetic field and not ionizing radiation, an MRI is safer than tomography and X-rays. As a radiological modality, MRI findings can also pinpoint the size and location of tumors, lesions, and injuries.

MRI scans outperform CT scans for detecting some cancers such as uterine and prostate cancer and some liver cancers. An MRI also shows brain and bone metastases meaning cancer has spread from the initial site more clearly than a CT scan.

Researchers look at three indicators to determine MRI accuracy: sensitivity, specificity, and the positive predictive value PPV. The increased sensitivity in MRI-assisted cancer staging may lead to more accurate diagnoses, which means more extensive breast surgery.

If your MRI indicates additional cancer, talk to your doctor about your treatment options. A study published in the New England Journal of Medicine rated the sensitivity of three methods of invasive breast cancer detection.

In the study, the specificity was However, keep in mind that the higher percentages for clinical exam and mammography are related to the delay in finding invasive cancer.

Dense breast tissue can also return a false-positive in a clinical exam. However, a 3D mammogram finds more cancers than traditional 2D mammograms , including those hidden by dense breast tissue. Research indicates MRI scans may be more accurate than other prostate cancer detection methods, especially biopsies.

This means a prostate MRI may help avoid unneeded biopsies. Urinary tract infection is the most common infectious complication of prostate biopsy, occurring in 1 to 11 percent of patients.

Men with elevated PSA levels who want more information about their prostate before undergoing a biopsy can get a prostate MRI with contrast. It will indicate with better accuracy where any potential cancer may be prior to undergoing a biopsy.

As a biopsy involves random tissue samples, it can miss a cancerous region of the prostate. This means MRI results can also help avoid missed diagnoses. Beyond offering more accurate cancer detection, MRIs can also better evaluate orthopedic issues than other diagnostic modalities such as X-rays.

In addition to being safer than an X-ray because there is no radiation , an MRI is more accurate than an X-ray. A musculoskeletal MRI is important for assessing the results of orthopedic sometimes orthopaedic surgery for arthroscopic repairs, such as meniscal tears or anterior cruciate ligament injuries.

Having an MRI before a biopsy means radiologists can identify exactly where cancer may be. Marking a suspicious area aids a targeted needle biopsy. Quite possibly, the most significant benefit of MRI is whole-body imaging.

The ability to scan head-to-toe means MRIs can detect cancer throughout the body. A medical provider will often order an MRI scan, but you can also book a scan on your own. Detecting cancer before symptoms occur leads to better treatment plans and a better prognosis.

At some sites, you will be provided with a headset and can select a Spotify playlist to help you relax during the scan.

Radiologists and other health care providers discourage having an MRI during the first trimester. If you have had such a response, discuss it with your doctor, technologist, or radiologist, especially if you also have kidney disease.

However, rest assured that ezra does not use contrast material for the full body scan. Early and accurate detection offers peace of mind.

External validation of DSC showed a lower sensitivity and a higher specificity for the reported cut-off values included in this metaanalysis. Conclusion: A combination of techniques shows the highest diagnostic accuracy differentiating tumor progression from treatment induced abnormalities.

External validation of imaging results is important to better define the reliability of imaging results with the different techniques.

Keywords: Brain metastasis; MRI; Meta-analysis; Pseudoprogression; Treatment response. Copyright © The Author s. Published by Elsevier B.

All rights reserved. Abstract Background: Treatment response assessment in patients with brain metastasis uses contrast enhanced T1-weighted MRI.

These three fields were chosen to meta-analyse as they had the largest numbers of studies with available data. Two hundred twenty-four other studies were included for qualitative synthesis in other medical specialities. Summary estimates of imaging and speciality-specific diagnostic accuracy metrics are described in Table 1.

Units of analysis for each speciality and modality are indicated in Tables 2 — 4. PRISMA preferred reporting items for systematic reviews and meta-analyses flow diagram of included studies. Eighty-two studies with separate patient cohorts reported diagnostic accuracy data for DL in ophthalmology see Table 2 and Supplementary References 1.

Optical coherence tomography OCT and retinal fundus photographs RFP were the two imaging modalities performed in this speciality with four main pathologies being diagnosed—diabetic retinopathy DR , age-related macular degeneration AMD , glaucoma and retinopathy of prematurity ROP.

Only eight studies 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 used prospectively collected data and 29 refs. No studies provided a prespecified sample size calculation.

Twenty-five studies 17 , 28 , 29 , 35 , 37 , 39 , 40 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 compared algorithm performance against healthcare professionals. Reference standards, definitions of disease and threshold for diagnosis varied greatly as did the method of internal validation used.

There was high heterogeneity across all studies see Table 2. Diabetic retinopathy: Twenty-five studies with 48 different patient cohorts reported diagnostic accuracy data for all, referable or vision-threatening DR on RFP. Twelve studies and 16 cohorts reported on diabetic macular oedema DME or early DR on OCT scans.

AUC was 0. Age-related macular degeneration: Twelve studies reported diagnostic accuracy data for features of varying severity of AMD on RFP 14 cohorts and 11 studies in OCT 21 cohorts. Glaucoma: Seventeen studies with 30 patient cohorts reported diagnostic accuracy for features of glaucomatous optic neuropathy, optic discs or suspect glaucoma on RFP and five studies with 6 cohorts on OCT.

One study 34 with six cohorts on RFP provided contingency tables. When averaging across the cohorts, the pooled sensitivity was 0.

The AUC of the summary receiver-operating characteristic SROC curve was 0. Retinopathy of prematurity: Three studies reported diagnostic accuracy for identifying plus diseases in ROP from RFP. Sensitivity was 0. AUC was only reported in two studies so was not pooled. Others: Eight other studies reported on diagnostic accuracy in ophthalmology either using different imaging modalities ocular images and visual fields or for identifying other diagnoses pseudopapilloedema, retinal vein occlusion and retinal detachment.

These studies were not included in the meta-analysis. One hundred and fifteen studies with separate patient cohorts report on diagnostic accuracy of DL on respiratory disease see Table 3 and Supplementary References 2.

Only two studies 62 , 63 used prospectively collected data and 13 refs. Twenty-one 54 , 63 , 64 , 65 , 66 , 67 , 70 , 72 , 76 , 77 , 78 , 79 , 80 , 81 , 82 , 83 , 84 , 85 , 86 , 87 , 88 studies compared algorithm performance against healthcare professionals.

Reference standards varied greatly as did the method of internal validation used. There was high heterogeneity across all studies see Table 3. Lung nodules: Fifty-six studies with 74 separate patient cohorts reported diagnostic accuracy for identifying lung nodules on CT scans on a per lesion basis, compared with nine studies and 14 patient cohorts on CXR.

Seven studies reported on diagnostic accuracy for identifying lung nodules on CT scans on a per scan basis, these were not included in the meta-analysis. Lung cancer or mass: Six studies with nine patient cohorts reported diagnostic accuracy for identifying mass lesions or lung cancer on CT scans compared with eight studies and ten cohorts on CXR.

Abnormal Chest X-ray: Twelve studies reported diagnostic accuracy for abnormal CXR with 13 different patient cohorts. Pneumothorax: Ten studies reported diagnostic accuracy for pneumothorax on CXR with 14 different patient cohorts.

Five patient cohorts from two studies 73 , 89 provided contingency tables with raw diagnostic accuracy. The AUC of the SROC curve was 0. Pneumonia: Ten studies reported diagnostic accuracy for pneumonia on CXR with 15 different patient cohorts. Tuberculosis: Six studies reported diagnostic accuracy for tuberculosis on CXR with 17 different patient cohorts.

Four patient cohorts from one study 90 provided contingency tables with raw diagnostic accuracy. X-ray imaging was also used to identify atelectasis, pleural thickening, fibrosis, emphysema, consolidation, hiatus hernia, pulmonary oedema, infiltration, effusion, mass and cardiomegaly.

CT imaging was also used to diagnose COPD, ground glass opacity and interstitial lung disease, but these were not included in the meta-analysis. Eighty-two studies with separate patient cohorts report on diagnostic accuracy of DL on breast disease see Table 4 and Supplementary References 3.

The four imaging modalities of mammography MMG , digital breast tomosynthesis DBT , ultrasound and magnetic resonance imaging MRI were used to diagnose breast cancer.

No studies used prospectively collected data and eight 91 , 92 , 93 , 94 , 95 , 96 , 97 , 98 studies validated algorithms on external data. Sixteen studies 62 , 91 , 92 , 94 , 97 , 98 , 99 , , , , , , , , compared algorithm performance against healthcare professionals.

There was high heterogeneity across all studies see Table 4. Breast cancer: Forty-eight studies with 59 separate patient cohorts reported diagnostic accuracy for identifying breast cancer on MMG AUC 0.

Our literature search also identified studies in other medical specialities reporting on diagnostic accuracy of DL algorithms to identify disease. A key finding of our review was the large degree of variation in methodology, reference standards, terminology and reporting among studies in all specialities.

The most common variables amongst DL studies in medical imaging include issues with the quality and size of datasets, metrics used to report performance and methods used for validation see Table 5. Only eight studies in ophthalmology imaging 14 , 21 , 32 , 33 , 43 , 55 , , , ten studies in respiratory imaging 64 , 66 , 70 , 72 , 75 , 79 , 82 , 87 , 89 , and six studies in breast imaging 62 , 91 , 97 , , , mentioned adherence to the STARD guidelines or had a STARD flow diagram in the manuscript.

Funnel plots were produced for the diagnostic accuracy outcome measure with the largest number of patient cohorts in each medical speciality, in order to detect bias in the studies included see Supplementary Figs. These demonstrate that there is high risk of bias in studies detecting lung nodules on CT scans and detecting DR on RFP, but not for detecting breast cancer on MMG.

The overall risk of bias and applicability using Quality Assessment of Diagnostic Accuracies Studies 2 QUADAS-2 led to a majority of studies in all specialities being classified as high risk, particularly with major deficiencies in regard to patient selection, flow and timing and applicability of the reference standard see Fig.

These were mostly related to a case—control study design and sampling issues. This was largely due to missing information about patients not receiving the index test or whether all patients received the same reference standard. This was mostly due to reference standard inconsistencies if the index test was validated on external datasets.

Risk of bias and applicability concerns summary about each QUADAS-2 domain presented as percentages across the 82 included studies in ophthalmic imaging a , in respiratory imaging b and 82 in breast imaging c. This study sought to 1 quantify the diagnostic accuracy of DL algorithms to identify specific pathology across distinct radiological modalities, and 2 appraise the variation in study reporting of DL-based radiological diagnosis.

The findings of our speciality-specific meta-analysis suggest that DL algorithms generally have a high and clinically acceptable diagnostic accuracy in identifying disease.

High diagnostic accuracy with analogous DL approaches was identified in all specialities despite different workflows, pathology and imaging modalities, suggesting that DL algorithms can be deployed across different areas in radiology. However, due to high heterogeneity and variance between studies, there is considerable uncertainty around estimates of diagnostic accuracy in this meta-analysis.

In ophthalmology, the findings suggest features of diseases, such as DR, AMD and glaucoma can be identified with a high sensitivity, specificity and AUC, using DL on both RFP and OCT scans.

In general, we found higher sensitivity, specificity, accuracy and AUC with DL on OCT scans over RFP for DR, AMD and glaucoma. Only sensitivity was higher for DR on RFP over OCT. In respiratory medicine, our findings suggest that DL has high sensitivity, specificity and AUC to identify chest pathology on CT scans and CXR.

DL on CT had higher sensitivity and AUC for detecting lung nodules; however, we found a higher specificity, PPV and F 1 score on CXR. For diagnosing cancer or lung mass, DL on CT had a higher sensitivity than CXR.

In breast cancer imaging, our findings suggest that DL generally has a high diagnostic accuracy to identify breast cancer on mammograms, ultrasound and DBT. The performance was found to be very similar for these modalities.

In MRI, however, the diagnostic accuracy was lower; this may be due to small datasets and the use of 2D images. The utilisation of larger databases and multiparametric MRI may increase the diagnostic accuracy Extensive variation in the methodology, data interpretability, terminology and outcome measures could be explained by a lack of consensus in how to conduct and report DL studies.

The STARD checklist , designed for reporting of diagnostic accuracy studies is not fully applicable to clinical DL studies The variation in reporting makes it very difficult to formally evaluate the performance of algorithms.

Furthermore, differences in reference standards, grader capabilities, disease definitions and thresholds for diagnosis make direct comparison between studies and algorithms very difficult.

This can only be improved with well-designed and executed studies that explicitly address questions concerning transparency, reproducibility, ethics and effectiveness and specific reporting standards for AI studies , The QUADAS-2 ref.

Although this tool was not designed for DL diagnostic accuracy studies, the evaluation allowed us to judge that a majority of studies in this field are at risk of bias or concerning for applicability. Of particular concern was the applicability of reference standards and patient selection.

Despite our results demonstrating that DL algorithms have a high diagnostic accuracy in medical imaging, it is currently difficult to determine if they are clinically acceptable or applicable. This is partially due to the extensive variation and risk of bias identified in the literature to date.

Furthermore, the definition of what threshold is acceptable for clinical use and tolerance for errors varies greatly across diseases and clinical scenarios There are broad methodological deficiencies among the included studies.

Most studies were performed using retrospectively collected data, using reference standards and labels that were not intended for the purposes of DL analysis. Minimal prospective studies and only two randomised studies , , evaluating the performance of DL algorithms in clinical settings were identified in the literature.

Proper acquisition of test data is essential to interpret model performance in a real-world clinical setting. Poor quality reference standards may result in the decreased model performance due to suboptimal data labelling in the validation set 28 , which could be a barrier to understanding the true capabilities of the model on the test set.

This is symptomatic of the larger issue that there is a paucity of gold-standard, prospectively collected, representative datasets for the purposes of DL model testing. However, as there are many advantages to using retrospectively collected data, the resourceful use of retrospective or synthetic data with the use of labels of varying modality and quality represent important areas of research in DL Many studies did not undertake external validation of the algorithm in a separate test set and relied upon results from the internal validation data; the same dataset used to train the algorithm initially.

This may lead to an overestimation of the diagnostic accuracy of the algorithm. The problem of overfitting has been well described in relation to machine learning algorithms True demonstration of the performance of these algorithms can only be assumed if they are externally validated on separate test sets with previously unseen data that are representative of the target population.

Surprisingly, few studies compared the diagnostic accuracy of DL algorithms against expert human clinicians for medical imaging. This would provide a more objective standard that would enable better comparison of models across studies. Furthermore, application of the same test dataset for diagnostic performance assessment of DL algorithms versus healthcare professionals was identified in only select studies This methodological deficiency limits the ability to gauge the clinical applicability of these algorithms into clinical practice.

Similarly, this issue can extend to model-versus-model comparisons. Specific methods of model training or model architecture may not be described well enough to permit emulation for comparison Thus, standards for model development and comparison against controls will be needed as DL architectures and techniques continue to develop and are applied in medical contexts.

There was varying terminology and a lack of transparency used in DL studies with regards to the validation or test sets used. Furthermore, the inconsistent terminology led to difficulties understanding whether an independent external test set was used to test diagnostic performance Crucially, we found broad variation in the metrics used as outcomes for the performance of the DL algorithms in the literature.

Very few studies reported true positives, false positives, true negatives and false negatives in a contingency table as should be the minimum for diagnostic accuracy studies Moreover, some studies only reported metrics, such as dice coefficient, F 1 score, competition performance metric and Top-1 accuracy that are often used in computer science, but may be unfamiliar to clinicians Metrics such as AUC, sensitivity, specificity, PPV and NPV should be reported, as these are more widely understood by healthcare professionals.

However, it is noted that NPV and PPV are dependent on the underlying prevalence of disease and as many test sets are artificially constructed or balanced, then reporting the NPV or PPV may not be valid. The wide range of metrics reported also leads to difficulty in comparing the performance of algorithms on similar datasets.

This systematic review and meta-analysis statistically appraises pooled data collected from studies. It is the largest study to date examining the diagnostic accuracy of DL on medical imaging.

However, our findings must be viewed in consideration of several limitations. Firstly, as we believe that many studies have methodological deficiencies or are poorly reported, these studies may not be a reliable source for evaluating diagnostic accuracy.

Consequently, the estimates of diagnostic performance provided in our meta-analysis are uncertain and may represent an over-estimation of the true accuracy. Secondly, we did not conduct a quality assessment for the transparency of reporting in this review.

This was because current guidelines to assess diagnostic accuracy reporting standards STARD were not designed for DL studies and are not fully applicable to the specifics and nuances of DL research Thirdly, due to the nature of DL studies, we were not able to perform classical statistical comparison of measures of diagnostic accuracy between different imaging modalities.

Fourthly, we were unable to separate each imaging modality into different subsets, to enable comparison across subsets and allow the heterogeneity and variance to be broken down. This was because our study aimed to provide an overview of the literature in each specific speciality, and it was beyond the scope of this review to examine each modality individually.

The inherent differences in imaging technology, patient populations, pathologies and study designs meant that attempting to derive common lessons across the board did not always offer easy comparisons. Finally, our review concentrated on DL for speciality-specific medical imaging, and therefore it may not be appropriate to generalise our findings to other forms of medical imaging or AI studies.

For the quality of DL research to flourish in the future, we believe that the adoption of the following recommendations are required as a starting point.

This can be achieved through governmental support and will enable greater reproducibility of DL models Rather than classical trials, novel experimental and quasi-experimental methods to evaluate DL have been proposed and should be evaluated This may include ongoing evaluation of algorithms once in clinical practice, as they continue to learn and adapt to the population that they are implemented in.

A major reason for the difficulties encountered in evaluating the performance of DL on medical imaging are largely due to inconsistent and haphazard reporting. Existing reporting guidelines for diagnostic accuracy studies STARD , prediction models TRIPOD , randomised trials CONSORT and interventional trial protocols SPIRIT do not fully cover DL research due to specific considerations in methodology, data and interpretation required for these studies.

As such, we applaud the recent publication of the CONSORT-AI and SPIRIT-AI guidelines, and await AI-specific amendments of the TRIPOD-AI and STARD-AI statements which we are convening.

We trust that when these are published, studies being conducted will have a framework that enables higher quality and more consistent reporting. An update to the QUADAS-2 tool taking into account the nuances of DL diagnostic accuracy research should be considered.

Outdated policies need to be updated and key questions answered in terms of liability in cases of medical error, doctor and patient understanding, control over algorithms and protection of medical data The World Health Organisation and others have started to develop guidelines and principles to regulate the use of AI.

These regulations will need to be adapted by each country to fit their own political and healthcare context Furthermore, these guidelines will need to proactively and objectively evaluate technology to ensure best practices are developed and implemented in an evidence-based manner DL is a rapidly developing field that has great potential in all aspects of healthcare, particularly radiology.

This systematic review and meta-analysis appraised the quality of the literature and provided pooled diagnostic accuracy for DL techniques in three medical specialities.

While the results demonstrate that DL currently has a high diagnostic accuracy, it is important that these findings are assumed in the presence of poor design, conduct and reporting of studies, which can lead to bias and overestimating the power of these algorithms.

The application of DL can only be improved with standardised guidance around study design and reporting, which could help clarify clinical utility in the future.

There is an immediate need for the development of AI-specific STARD and TRIPOD statements to provide robust guidance around key issues in this field before the potential of DL in diagnostic healthcare is truly realised in clinical practice. Studies that report upon the diagnostic accuracy of DL algorithms to investigate pathology or disease on medical imaging were sought.

The primary outcome was various diagnostic accuracy metrics. Secondary outcomes were study design and quality of reporting. Electronic bibliographic searches were conducted in Medline and EMBASE up to 3rd January For the full search strategy, please see Supplementary Methods 1.

The search included all study designs. Further studies were identified through manual searches of bibliographies and citations until no further relevant studies were identified. Two investigators R. and V. independently screened titles and abstracts, and selected all relevant citations for full-text review.

Disagreement regarding study inclusion was resolved by discussion with a third investigator H. Studies that comprised a diagnostic accuracy assessment of a DL algorithm on medical imaging in human populations were eligible.

Only studies that stated either diagnostic accuracy raw data, or sensitivity, specificity, AUC, NPV, PPV or accuracy data were included in the meta-analysis.

No limitations were placed on the date range and the last search was performed in January Articles were excluded if the article was not written in English. Abstracts, conference articles, pre-prints, reviews and meta-analyses were not considered because an aim of this review was to appraise the methodology, reporting standards and quality of primary research studies being published in peer-reviewed journals.

Studies that investigated the accuracy of image segmentation or predicting disease rather than identification or classification were excluded. independently extracted demographic and diagnostic accuracy data from the studies, using a predefined electronic data extraction spreadsheet.

The data fields were chosen subsequent to an initial scoping review and were, in the opinion of the investigators, sufficient to fulfil the aims of this review. Three investigators R. and GM assessed study methodology using the QUADAS-2 checklist to evaluate the risk of bias and any applicability concerns of the studies A bivariate model for diagnostic meta-analysis was used to calculate summary estimates of sensitivity, specificity and AUC data Independent proportion and their differences were calculated and pooled through DerSimonian and Laird random-effects modelling This considered both between-study and within-study variances that contributed to study weighting.

Where raw diagnostic accuracy data were available, the SROC model was used to evaluate the relationship between sensitivity and specificity We utilised Stata version 15 Stata Corp LP, College Station, TX, USA for all statistical analyses.

We chose to appraise the performance of DL algorithms to identify individual disease or pathology patterns on different imaging modalities in isolation, e. We felt that combining imaging modalities and diagnoses would add heterogeneity and variation to the analysis. Meta-analysis was only performed where there were greater than or equal to three patient cohorts, reporting for each specific pathology and imaging modality.

This study is registered with PROSPERO, CRD Further information on research design is available in the Nature Research Reporting Summary linked to this article. The authors declare that all the data included in this study are available within the paper and its Supplementary Information files.

LeCun, Y. Deep learning. Nature , — Article CAS PubMed Google Scholar. Obermeyer, Z. Predicting the future — big data, machine learning, and clinical medicine.

Article PubMed PubMed Central Google Scholar. Esteva, A. et al. A guide to deep learning in healthcare. Litjens, G. A survey on deep learning in medical image analysis.

Image Anal. Article PubMed Google Scholar. Bluemke, D. Assessing radiology research on artificial intelligence: a brief guide for authors, reviewers, and readers—from the radiology editorial board. Radiology , — Wahl, B. Artificial intelligence AI and global health: how can AI contribute to health in resource-poor settings?

BMJ Glob. Health 3 , e—e Zhang, L. Big data and medical research in China. BMJ , j Nakajima, Y. Radiologist supply and workload: international comparison.

Kelly, C. Key challenges for delivering clinical impact with artificial intelligence. BMC Med. Article PubMed PubMed Central CAS Google Scholar.

Topol, E. High-performance medicine: the convergence of human and artificial intelligence. Benjamens, S.

The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database. npj Digital Med.

Article Google Scholar. Beam, A. Big data and machine learning in health care. JAMA , — Liu, X. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis.

Lancet Digital Health 1 , e—e Abràmoff, M. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. Bellemo, V. Artificial intelligence using deep learning to screen for referable and vision-threatening diabetic retinopathy in Africa: a clinical validation study.

Lancet Digital Health 1 , e35—e44 Christopher, M. Performance of deep learning architectures and transfer learning for detecting glaucomatous optic neuropathy in fundus photographs.

Gulshan, V. Performance of a deep-learning algorithm vs manual grading for detecting diabetic retinopathy in India. JAMA Ophthalmol , — Article PubMed Central PubMed Google Scholar.

Keel, S. Visualizing deep learning models for the detection of referable diabetic retinopathy and glaucoma. JAMA Ophthalmol. Sandhu, H. Automated diagnosis and grading of diabetic retinopathy using optical coherence tomography. Zheng, C. Detecting glaucoma based on spectral domain optical coherence tomography imaging of peripapillary retinal nerve fiber layer: a comparison study between hand-crafted features and deep learning model.

Graefes Arch. Kanagasingam, Y. Evaluation of artificial intelligence-based grading of diabetic retinopathy in primary care.

JAMA Netw. Open 1 , e—e Alqudah, A. AOCT-NET: a convolutional network automated classification of multiclass retinal diseases using spectral-domain optical coherence tomography images.

Asaoka, R. Validation of a deep learning model to screen for glaucoma using images from different fundus cameras and data augmentation.

Glaucoma 2 , — Bhatia, K. Disease classification of macular optical coherence tomography scans using deep learning software: validation on independent, multicenter data. Retina 40 , — Chan, G. Fusing results of several deep learning architectures for automatic classification of normal and diabetic macular edema in optical coherence tomography.

In Conference proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference, Vol.

Gargeya, R. Automated identification of diabetic retinopathy using deep learning. Ophthalmology , — Grassmann, F. A deep learning algorithm for prediction of age-related eye disease study severity scale for age-related macular degeneration from color fundus photography.

Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. Hwang, D. Artificial intelligence-based decision-making for age-related macular degeneration.

Theranostics 9 , — Development and validation of a deep-learning algorithm for the detection of neovascular age-related macular degeneration from colour fundus photographs. Krause, J. Grader variability and the importance of reference standards for evaluating machine learning models for diabetic retinopathy.

Li, F. Automatic detection of diabetic retinopathy in retinal fundus photographs based on deep learning algorithm. Li, Z. An automated grading system for detection of vision-threatening referable diabetic retinopathy on the basis of color fundus photographs.

Diabetes Care 41 , — Liu, H. Development and validation of a deep learning system to detect glaucomatous optic neuropathy using fundus photographs.

Liu, S. A deep learning-based algorithm identifies glaucomatous discs using monoscopic fundus photographs. Glaucoma 1 , 15—22 MacCormick, I. Accurate, fast, data efficient and interpretable glaucoma diagnosis with automated spatial analysis of the whole cup to disc profile.

PLoS ONE 14 , e Article CAS PubMed PubMed Central Google Scholar. Phene, S. Deep learning and glaucoma specialists: the relative importance of optic disc features to predict glaucoma referral in fundus photographs.

Ramachandran, N. Diabetic retinopathy screening using deep neural network. Raumviboonsuk, P. Deep learning versus human graders for classifying diabetic retinopathy severity in a nationwide screening program. Sayres, R. Using a deep learning algorithm and integrated gradients explanation to assist grading for diabetic retinopathy.

Ting, D. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. Deep learning in estimating prevalence and systemic risk factors for diabetic retinopathy: a multi-ethnic study.

Verbraak, F. Diagnostic accuracy of a device for the automated detection of diabetic retinopathy in a primary care setting. Diabetes Care 42 , Van Grinsven, M. Fast convolutional neural network training using selective data sampling: application to hemorrhage detection in color fundus images.

IEEE Trans. Imaging 35 , — Rogers, T. Evaluation of an AI system for the automated detection of glaucoma from stereoscopic optic disc photographs: the European Optic Disc Assessment Study. Eye 33 , — Al-Aswad, L. Evaluation of a deep learning system for identifying glaucomatous optic neuropathy based on color fundus photographs.

Glaucoma 28 , — Brown, J. Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks. Burlina, P. Utility of deep learning methods for referability classification of age-related macular degeneration. Automated grading of age-related macular degeneration from color fundus images using deep convolutional neural networks.

Comparing humans and deep learning performance for grading AMD: a study in using universal deep features and transfer learning for automated AMD analysis. Computers Biol. De Fauw, J. Clinically applicable deep learning for diagnosis and referral in retinal disease.

Article PubMed CAS Google Scholar. Gómez-Valverde, J. Automatic glaucoma classification using color fundus images based on convolutional neural networks and transfer learning.

Express 10 , — Jammal, A. Human versus machine: comparing a deep learning algorithm to human gradings for detecting glaucoma on fundus photographs. Kermany, D. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell , — e Deep learning-based automated detection of retinal diseases using optical coherence tomography images.

Long, E. An artificial intelligence platform for the multihospital collaborative management of congenital cataracts. Matsuba, S. Accuracy of ultra-wide-field fundus ophthalmoscopy-assisted deep learning, a machine-learning technology, for detecting age-related macular degeneration.

Nagasato, D. Automated detection of a nonperfusion area caused by retinal vein occlusion in optical coherence tomography angiography images using deep learning.

Peng, Y. DeepSeeNet: a deep learning model for automated classification of patient-based age-related macular degeneration severity from color fundus photographs. Shibata, N. Development of a deep residual learning algorithm to screen for glaucoma from fundus photography. Zhang, Y. Development of an automated screening system for retinopathy of prematurity using a deep neural network for wide-angle retinal images.

IEEE Access 7 , — Becker, A. Classification of breast cancer in ultrasound imaging using a generic deep learning analysis software: a pilot study.

Google Scholar. Zhang, C. Toward an expert level of lung cancer detection and classification using a deep convolutional neural network. Oncologist 24 , — Ardila, D. End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography. Hwang, E.

Deep learning for chest radiograph diagnosis in the emergency department. Development and validation of a deep learning—based automated detection algorithm for major thoracic diseases on chest radiographs.

Open 2 , e—e Development and validation of a deep learning—based automatic detection algorithm for active pulmonary tuberculosis on chest radiographs.

Liang, C. Identifying pulmonary nodules or masses on chest radiography using deep learning: external validation and strategies to improve clinical practice.

Nam, J. Development and validation of deep learning—based automatic detection algorithm for malignant pulmonary nodules on chest radiographs. Qin, Z. Using artificial intelligence to read chest radiographs for tuberculosis detection: A multi-site evaluation of the diagnostic accuracy of three deep learning systems.

Setio, A. Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. Sim, Y. Deep convolutional neural network—based software improves radiologist detection of malignant lung nodules on chest radiographs.

Taylor, A. Automated detection of moderate and large pneumothorax on frontal chest X-rays using deep convolutional neural networks: a retrospective study.

PLOS Med. Uthoff, J. Machine learning approach for distinguishing malignant and benign lung nodules utilizing standardized perinodular parenchymal features from CT. Zech, J. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study.

Cha, M. Performance of deep learning model in detecting operable lung cancer with chest radiographs. Imaging 34 , 86—91 Chae, K. Ciompi, F. Towards automatic pulmonary nodule management in lung cancer screening with deep learning. Dunnmon, J. Assessment of convolutional neural networks for automated classification of chest radiographs.

Li, X. Deep learning-enabled system for rapid pneumothorax screening on chest CT. Li, L. Evaluating the performance of a deep learning-based computer-aided diagnosis DL-CAD system for detecting and characterizing lung nodules: comparison with the performance of double reading by radiologists.

Cancer 10 , — Majkowska, A. Chest radiograph interpretation with deep learning models: assessment with radiologist-adjudicated reference standards and population-adjusted evaluation. Park, S. Deep learning-based detection system for multiclass lesions on chest radiographs: comparison with observer readings.

ADC values are also known to show intratumoral variation with low ADC values for solid tumor components and high ADC values for necrotic areas, which can be a caveat in drawing regions of interest [ 39 ].

This might be the reason for the different strategies used in the region of interest analyses. Whole tumor volume possibly included necrotic areas [ 22 ].

The studies targeting the most conspicuous area can be assumed to exclude necrosis [ 23 ], while necrosis is certainly excluded for the studies stated to target the most conspicuous area excluding necrosis [ 27 ] or the complete solid component excluding necrosis [ 16 , 29 ].

One study did not provided details about the region of interest analysis, hindering a judgment about the quality [ 21 ]. Despite the variation in thresholds, tumor heterogeneity and different b-values, ADC data still outperformed anatomical MRI techniques.

Because of the limited number of studies we were not able to assess the diagnostic accuracy of ADC and MRI in various threshold subgroups.

However, implementation in clinical practice would benefit from standardized and validated ADC threshold values and region of interest analysis. This lack of standardization and the current high variability also hinders the generation of an advice regarding the best cut-off value to be used in clinical practice.

Nevertheless, this meta-analysis demonstrate what many radiologist experience in daily practice, namely that adding a diffusion sequence to the anatomical sequences enhances treatment evaluation.

Numbers of excluded patients due to susceptibility artefacts in the head and neck area were provided in some studies see Tables 1 and 2. This is a known limitation of DWI sequences, but the current limited data suggest that it is a problem in a minority of the patients.

Small primary tumor size was an exclusion criteria in only two studies [ 23 , 25 ]. The sensitivity and specificity reported in studies excluding tumors smaller than 6 mm, however, did not show a significantly higher accuracy over studies without size limitations.

Other factors, like claustrophobia played a minimal role. Data for perfusion and spectroscopy studies were searched, but were not available yet for inclusion in our meta-analysis. Perfusion is, however, feasible and already shows to be able to predict survival before treatment or predict tumor response early in the treatment [ 9 , 40 ].

The potential value of perfusion is also shown by high diagnostic accuracies in treatment response evaluation in patients with brain tumors [ 41 ]. Spectroscopy is even less studied although its feasibility has been demonstrated in head and neck tumors. However, diagnostic accuracy remains speculative currently [ 10 ].

The main analysis included predominantly posttreatment studies, but also a few intratreatment studies. Combining both was considered to be justified as MRI aims in both to identify viable tumor, although the question differs slightly.

Intratreatment MRI aims at differentiating responders from non-responders to adapt the treatment in non-responders, while posttreatment MRI is used to select patients for addition therapy when tumor is shown.

The overlapping diagnostic accuracy supports the legitimacy of combining intratreatment and posttreatment MRI. Identifying non-responders and responders early after treatment start or even before treatment would be optimal.

The few intratreatment studies in our data suggest a preference for using ADC data over anatomical MRI for this [ 17 , 22 , 29 ]. Predicting treatment response before the start of it also favors ADC for primary and nodal sites [ 34 , 37 , 42 ].

Although good, the performance is until now too variable for wide clinical implication. It might probably benefit from more precise coregistration to anatomical MRI, but also more clinical trials in a large population for validation of DWI early after the start of treatment [ 43 ].

Identifying the non-responders with ADC as a potential biomarker early during treatment may enable treatment tailoring and may avoid possible side-effects of an ineffective and expensive treatment regime [ 44 ]. Prediction of clinical outcome would be of interest as well.

FDG-PET is frequently used for treatment response assessment with high sensitivity but lower specificity [ 45 ]. Compared to FDG-PET, ADC can be performed earlier to assess treatment response. FDG-PET is less reliable in the first months after treatment with false positive results due to inflammation, granulation and scar tissue [ 46 ].

ADC can be performed in this period, but false positive and false negatives are not fully excluded. True restricted diffusion can be seen in an abscess or with inflammation, although central enhancement as shown in tumor would be lacking.

Scar tissue can display low ADC but normally in combination with lack of diffusion restriction. This distinguishes scar tissue from tumor with low values on the ADC map together with diffusion restriction [ 47 ].

Minimal to absent enhancement of scar tissue helps in further differentiation from tumor. Included studies used ADC values only for calculations and therefore likely underestimated the accuracy of diffusion weighted MRI. Combining anatomical MRI with diffusion weighted MRI including b -maps, ADC maps and post contrast images would probably demonstrate even higher diagnostic accuracy in clinical practice.

The higher specificity less false positives of ADC compared to anatomic MRI results in a reduction of unnecessary and costly initiation of treatment in patients with treatment related changes.

It might also reduce the patients that are false interpreted on anatomical MRI as having tumor progression resulting in incorrect continuation of therapy.

Moreover, the higher sensitivity less false negatives of ADC contributes in decreasing the number of missed patients with tumor recurrence. In general, the methodological quality of the included studies was similar, but low. This might also explain the wider confidence interval in some studies [ 18 ]B, but could not provide a convincing explanation for others [ 16 , 19 , 29 ].

The heterogeneity of patient selection, reference standards or relatively small group size might provide additional sources of variation. This is a reflection of the complexity of the field, however this variation is an important limitation of the current study.

Especially the variability in the definition used to identify tumor residual or recurrence compared to treatment effects as shown in Tables 1 and 2 might be a limiting factor.

Furthermore, as discussed above and also displayed in these tables, different b-values and ADC thresholds were used in the different studies. Although it still can be concluded that ADC helps in the differentiation of tumor residual or recurrence and treatment related effects as fibrosis, this variability hinders stronger conclusions and a firm implication in clinical practice.

Further research should also focus on comparing all imaging techniques in the same population using direct comparisons to ensure a higher quality.

In such a study, the same reference standard should be applied in a consecutive large cohort of patients. This would also allow subgroup analyses to search for the sources of heterogeneity in the diagnostic performance of the MRI sequences.

To conclude, a higher diagnostic accuracy of ADC values over anatomical MRI in patients with treated head and neck tumors is demonstrated in this meta-analysis. It is should be kept in mind that this was only statistically significant for the direct comparison of the primary tumor site and not convincing for the direct comparisons of the nodal site.

However, this emphases the relevance to include DWI with ADC for response evaluation of treated head and neck tumor patients. Diagnostic accuracy and the 2x2 table is displayed with true positives TP , false positives FP , false negatives FN and true negative TN.

All authors declare that they have no conflict of interest. No funding was obtained for the current study. We would like to thank Dr. Sanjeev Chawla Hospital of University of Pennsylvania for providing additional data for the paper of Berrak et al.

We would also like to thank all authors who responded to our data request for their efforts to look whether they were able to provide additional data. It has also been presented orally also at the European Head and Neck Society meeting Leiden, The Netherlands, September Conceptualization: AH PL HW.

Data curation: AH HW GH. Investigation: AH PL GH HW. Methodology: GH. Project administration: AH PL GH HW. Supervision: AH HW. Validation: GH. Browse Subject Areas?

Click through the PLOS taxonomy to find articles in your field. Article Authors Metrics Comments Media Coverage Reader Comments Figures.

Abstract Background Novel advanced MRI techniques are investigated in patients treated for head and neck tumors as conventional anatomical MRI is unreliable to differentiate tumor from treatment related imaging changes. Purpose As the diagnostic accuracy of MRI techniques to detect tumor residual or recurrence during or after treatment is variable reported in the literature, we performed a systematic meta-analysis.

Data sources Pubmed, EMBASE and Web of Science were searched from their first record to September 23 th Study selection Studies reporting diagnostic accuracy of anatomical, ADC, perfusion or spectroscopy to identify tumor response confirmed by histology or follow-up in treated patients for head and neck tumors were selected by two authors independently.

Data analysis Two authors independently performed data extraction including true positives, false positives, true negatives, false negatives and general study characteristics. Data synthesis We identified 16 relevant studies with anatomical MRI and ADC.

Limitations Main limitation are the low, but comparable quality of the included studies and the variability between the studies. Conclusions The higher diagnostic accuracy of ADC values over anatomical MRI for the primary tumor location emphases the relevance to include DWI with ADC for response evaluation of treated head and neck tumor patients.

Funding: The authors received no specific funding for this work. Introduction Head and neck tumors are a devastating disease being the seventh leading cancer with respect to incidence, and the eight with respect to mortality rates [ 1 ].

Methods Our systematic review was performed according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses PRISMA, see S1 PRISMA Checklist criteria and the AMSTAR guidelines [ 12 , 13 ].

Data sources and search strategy PubMed, EMBASE and Web of Science were searched by AH and HW in separate sessions using the same search strategy from their first records to September 23 th Selection criteria We searched for studies with patients who were treated for newly diagnosed head and neck tumors.

Study selection Study selection, data extraction and study quality assessment was independently done by two authors AH and HW and discrepancies were resolved by discussion. Data extraction and quality assessment Data extraction was done with the use of a data extraction form.

Results Description of studies Our electronic search revealed a total of unduplicated references, of which 23 references were eligible for inclusion in the meta-analysis Fig 1 ; Tables 1 and 2 [ 16 — 38 ].

Download: PPT. Methodological quality of included studies The methodological quality of the included studies is summarized Fig 2. Fig 2. Risk of bias and applicability concerns summary with for each domain of the QUADAS-2 for each included study. Main findings primary site The forest plot of the anatomical MRI 11 studies with patients for the primary tumor location showed a reasonable homogenous specificity see S1 Fig.

Fig 3. Hierarchical summary receiver operator curves of anatomical MRI and ADC for the primary tumor site. Main findings nodal site The forest plot of the data for the nodal site for anatomical MRI 4 studies with patients showed small overlapping confidence interval for the sensitivity and specificity with exception of the sensitivity of one study [ 29 ] and the specificity of another study [ 16 ] see S1 Fig.

Imaging time point Intratreatment evaluation 2 studies with 79 patients , early posttreatment evaluation 3 studies with patients , and late posttreatment evaluation 8 studies with patients measurements demonstrated similar diagnostic accuracy for the primary tumor locations for the anatomical MRI S2 Fig.

Discussion By using the statistical strategy of a systematic meta-analysis, we were able to demonstrate a benefit of DWI with derived ADC data over anatomical conventional MRI sequences. Conclusions To conclude, a higher diagnostic accuracy of ADC values over anatomical MRI in patients with treated head and neck tumors is demonstrated in this meta-analysis.

Supporting information. S1 Text. Search strategy. s DOCX. S1 PRISMA Checklist. s DOC. S1 Fig. Forest plots with diagnostic accuracy anatomical MRI and ADC for different scan times for the primary tumor site.

s PDF. S2 Fig. See caption S1. Acknowledgments All authors declare that they have no conflict of interest. Author Contributions Conceptualization: AH PL HW. References 1. Ferlay J, Shin HR, Bray F, Forman D, Mathers C, Parkin DM.

Estimates of worldwide burden of cancer in GLOBOCAN Int J Cancer ; 12 — Bray F, Jemal A, Grey N, Forman D. Global cancer transitions according to the Human Development Index — : a population-based study.

Lancet Oncol ;13 8 — Bar-Ad V, Palmer J, Yang H, Cognetti D, Curry J, Lunginbuhl A, Tuluc M, et al. Current management of locally advanced head and neck cancer: the combination of chemotherapy with locoregional treatments. Semin Oncol ;41 6 — Ratko TA, Douglas GW, de Souza JA, Belinson SE, Aronson N.

AHRQ Comparative Effectiveness Reviews. Radiotherapy Treatments for Head and Neck Cancer Update [Internet]. Rockville: Agency for Healthcare Research and Quality US EHCEF Pignon JP, le Maître A, Maillard E, Bourhis J; MACH-NC Collaborative Group.

Meta-analysis of chemotherapy in head and neck cancer MACH-NC : an update on 93 randomised trials and 17, patients. Radiother Oncol ;92 1 :4— Rumboldt Z, Gordon L, Bonsall R, Ackermann S.

Imaging in head and neck cancer. Curr Treat Options Oncol ;7 1 — Al-Shwaiheen FA, Wang SJ, Uzelac A, Yom SS, Ryan WR. The advantages and drawbacks of routine magnetic resonance imaging for long-term posttreatment locoregional surveillance of oral cavity squamous cell carcinoma.

Am J Otolaryngol ;— View Article Google Scholar 8. Maroldi R, Ravanelli M, Farina D. Magnetic resonance for laryngeal cancer. Curr Opin Otolaryngol Head Neck Surg ;— Zheng D, Chen Y, Liu X, Chen Y, Xu L, Ren W, et al.

Early response to chemoradiotherapy for nasopharyngeal carcinoma treatment: Value of dynamic contrast-enhanced 3.

J Magn Reson Imaging ;— Devpura S, Barton KN, Brown SL, Palyvoda O, Kalkanis S, Naik VM, et al. Med Phys ;41 6 View Article Google Scholar Bhatnagar P, Subesinghe M, Patel C, Prestwich R, Scarsbrook AF. Functional imaging for radiation treatment planning, response assessment, and adaptive therapy in head and neck cancer.

Radiographics ;33 7 — Moher D, Liberati A, Tetzlaff J, Altman DG, The PRIAMS group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.

J Clin Epidemiol ;62 10 — Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J, et al. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Whiting PF, Rutjes AWS, Westwood ME, Mallett S, Deeks JJ, Reltsma JB, et al.

QUADAS a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med ; 8 — Reitsma JB, Glas AS, Rutjes AWS, Scholten RJPM, Bossuyt PM, Zwinderman AH.

Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol ;58 10 — Berrak S, Chawla S, Kim S, Quon H, Sherman E, Loevner LA, et al.

Diffusion weighted imaging in predicting progression free survival in patients with squamous cell carcinomas of the head and neck treated with induction chemotherapy.

Acad Radiol ;18 10 — Bhatia KSS, King AD, Yu KH, Vlantis AC, Tse GMK, Mo FKF, et al. Does primary tumor volumetry performed early in the course of definitive concomitant chemoradiotherapy for head and neck squamous cell carcinoma improve prediction of primary site outcome?

Br J Radiol ;83 — Chan SC, Ng SH, Chang JTC, Lin CY, Chen YC, Chang YC, et al. Advantages and pitfalls of 18F-fluorodeoxy-D-glucose positron emission tomography in detecting locally residual or recurrent nasopharyngeal carcinoma: comparison with magnetic resonance imaging.

Eur J Nucl Med Mol Imaging ;33 9 — Chong VFH, Fan YF. Detection of recurrent nasopharyngeal carcinoma: MR imaging versus CT. Radiology ; 2 — Comoretto M, Balestreri L, Borsatti E, Cimitan M, Franchin G, Lise M. Radiology ; 1 — Gouhar GK, El-Hariri MA.

The Eqyptian Journal of Radiology and Nuclear Medicine ;— Hong J, Yao Y, Zhang Y, Tang T, Zhang H, Bao D, et al. Value of magnetic resonance diffusion-weighted imaging for the prediction of radiosensitivity in nasopharyngeal carcinoma.

Otolaryngol Head Neck Surg ; 5 — Hwang I, Choi SH, Kim YJ, Le aL, Yun TJ, Kim JH, et al. Differentiation of recurrent tumor and posttreatment changes in head and neck squamous cell carcinoma: application of high b-value diffusion-weighted imaging. AJNR Am J Neuroradiol ;34 12 — King AD, Keung CK, Mo FKF, Bhatia KS, Yeung DKW, Tse GMK, et al.