K. D. Simmons1, L. E. Kuo1, R. L. Hoffman1, E. K. Bartlett1, D. N. Holena1, R. R. Kelz1 1University Of Pennsylvania,Department Of Surgery,Philadelphia, PA, USA
Introduction:
Outcomes research methodology is being used to rank hospital quality. The usability of this information by patients and providers is dependent upon appropriate risk adjustment and transparency of the statistical methods. For surgical outcomes, the timing of the procedure relative to the admission date is seldom considered or described in methods sections. As surgery performed during an admission for another reason is often high-risk, this may be a source of unmeasured confounding, thereby limiting the utility of the ranking data. We aimed to examine the association between surgical timing and outcomes as well as the subsequent effect on hospital rankings in a model of mortality following colectomy.
Methods:
Claims data from three state databases (California, Florida, and New York) were used to analyze in-hospital mortality following colectomy. Two cohorts were identified using different selection criteria: Cohort 1 included all patients who underwent a colectomy at any time during the admission; Cohort 2 was restricted to patients who underwent a colectomy on the day of admission only. Logistic regression was used in each cohort to adjust for patient and hospital characteristics associated with in-patient mortality and to estimate risk-adjusted mortality rates. Bootstrapping was performed to control for differences in sample size. Model performance was evaluated using the C statistic and the Hosmer-Lemeshow goodness-of-fit test. Hospitals were ranked by observed-to-expected (O/E) mortality ratio in each cohort.
Results:
Cohort 1 (all colectomies) included 209,515 colectomies, of which 118,387 (56.5%) were included in Cohort 2 (day of admission only). Overall mortality rates were 6.3% for Cohort 1 and 3.2% for Cohort 2. Models fit to mortality rates in each cohort fit the data well (Hosmer-Lemeshow statistics 105.0 in Cohort 1 and 37.6 in Cohort 2, p<10^-5 for each; C statistics 0.794 in Cohort 1 and 0.834 in Cohort 2).
Using the resulting risk-adjusted mortality rates, we ranked 675 hospitals by O/E ratio in each cohort. The two ranking systems had a correlation coefficient of 0.63 (p<.0001). Nevertheless, 391 hospitals (57.9%) changed rank by at least 10% between the rankings, and 370 hospitals (54.8%) were in different quintiles when ranked by Cohort 1 versus Cohort 2.
Bootstrapping confirmed that the difference in risk-adjusted mortality between Cohort 1 and Cohort 2 was not due to the smaller number of patients in Cohort 2.
Conclusion:
Time between admission and surgery can influence the likelihood of death following colectomy and thereby alter hospital rankings in the absence of real differences in the care provided. For maximally effective quality improvement, it is important that hospitals have information regarding the cohort of patients used to derive the ranking. Moreover, given the influence of surgical timing on outcomes, its use in patient selection should be delineated within the methods sections of any subsequent reports.