Y. Arriaga1,3, R. Hekmat1, K. Draulis1, S. Wang1, W. Felix1, I. Dankwa-Mullan1, K. Rhee1, G. Jackson1,2 1IBM Watson Health,Cambridge, MA, USA 2Vanderbilt University Medical Center,Pediatric Surgery,Nashville, TN, USA 3University Of Texas Southwestern Medical Center,Internal Medicine,Dallas, TX, USA
Introduction: Watson for Oncology (WfO) is an artificial intelligence (AI)-based clinical decision-support system (CDSS) that presents therapeutic options to support oncologists in making treatment decisions for cancer patients. WfO was released in 2015, and it has been deployed in over 200 institutions across the world. This study systematically reviewed the results of concordance studies between WfO therapeutic options and treatment recommendations by multidisciplinary tumor boards (MTBs) for surgical patients with non-metastatic colon, rectal, and breast cancers.
Methods: We conducted a review of the WfO publication database and a PubMed search to identify WfO concordance studies in non-metastatic colon, rectal, and breast cancer patients, published from 01/01/2015 to 06/30/2019. Studies were included in the analysis if patients were eligible for resection of the primary tumor, had resection of the primary tumor, and were eligible for neoadjuvant therapies, adjuvant therapies or surveillance. Studies were excluded if they measured concordance for multiple cancer types. Concordance was defined as agreement between WfO “Recommended” and “For Consideration” treatment options and treatments prescribed by MTBs. Mean concordance rates were calculated as an average, weighted by the number of patients in each study.
Results: Table 1 presents the results of six identified studies reporting the concordance between WfO therapeutic options and MTB treatment decisions in 1,300 non-metastatic colon, rectal, and breast cancer patients from China, India and South Korea. Mean concordance for all studies was 85.9 % (SD 26.4%), with a range from 16.7% in gastric cancer patients to 94.2% in a breast cancer study.
Conclusion: This study is one of the first AI CDSS concordance analyses focused on surgical patients. Overall concordance between the WfO therapeutic options and treatment decisions of MTBs in patients with non-metastatic colon, rectal, and breast cancers was high, demonstrating performance comparable with experts in China, India, and South Korea. Concordance varied across countries and stage, reflecting the need for localization to address regional differences in practice. The low concordance seen in gastric cancer is likely due to differences in cancer biology, management, and the prescription of S-1 chemotherapy by oncologists in the Asian region. With the increasing prevalence of cancer and global shortage of oncologists, such CDSS may have a role in supporting cancer care across the world.
