Study: Hospital rankings may rely on faulty data; U.S. News responds

Summary: “The October 2016 issue of The Joint Commission Journal on Quality and Patient Safety, features the article ‘Consumer Rankings and Health Care: Toward Validation and Transparency,’ by Bala Hota, MD, MPH, and co-authors, Rush University Medical Center (RUMC), Chicago.” according to a press release on EurekAlert. “After RUMC received a lower than expected ranking for patient safety in the 2015-16 U.S. News & World Report’s (USNWR) ‘Best Hospitals’ rankings, the authors compared the data that USNWR used for their hospital to their own internal data. The authors found that the USNWR data showed many more patient safety events, such as pressure ulcers, almost all of which had actually been present at the patient’s admission. Suspecting a broader problem, Hota et al. analyzed data on a sample of hospitals and found that RUMC was not the only organization with discrepancies in data. False-positive event rates were common among high-transfer and high-volume hospitals. The authors conclude more transparency and validation is needed for consumer-based benchmarking methods. In response to these findings and concerns raised by others, USNWR made changes to its methodology and data sources in 2016. In an accompanying editorial, ‘The Quality Measurement Crisis: An Urgent Need for Methodological Standards and Transparency,’ David M. Shahian, MD, Elizabeth Mort, MD, MPH, and Peter J. Pronovost, MD, PhD, reflect on the Hota et al. article to conclude, ‘Just as health care providers have ethical and moral responsibilities to the public they serve, rating organizations and journalists that grade providers have similar obligations–in their case, to ensure measure validity and methodological transparency.’ RUMC further explored the importance of rating organization’s validity and methodological transparency in the following essay, ‘Hospital Rankings Have Room for Improvement.'” Study: Hospital rankings may rely on faulty data,” EurekAlert! Science News. In response, Ben Harder, head of the U.S. News and Word Report quality ranking efforts, posted a description of how “quality measurement is a journey,” which you can read here.