by Walae Hayek, MHA’s Director of Health Equity

As artificial intelligence-based technologies grow, the healthcare community is confronted with both an opportunity and an obligation. How do we work to understand and evaluate technology’s impact, how do we use it in service of advancing high quality healthcare, and how do we ensure it can close long-standing gaps in health outcomes – rather than widen them?

In collaboration with the Boston Chapter of the National Association of Health Services Executives (NAHSE), MHA recently hosted a webinar featuring expert guest speakers to help local healthcare organizations work through some of these pressing questions.

Trust is Everything

“AI inherits the trust debt of healthcare,” stated Dr. Patel, DSM, Professor and Vice Chair of Innovation at University of Pittsburgh, citing patients’ attitudes when the systems embedded in healthcare fall short of meeting their needs. We cannot discuss the technology without discussing the systems it operates within, the infrastructure it builds from, and the communities it impacts.

In Massachusetts, healthcare systems have committed to closing care disparities, expanding access to care, listening to their communities, and building new processes to establish trust. That responsibility now extends to the use of cutting-edge technology and augmented intelligence, and ensuring their use accounts for the needs of historically underserved patient populations.

An Across-the-Board Commitment

Dr. Tinu Tadese, vice president and chief medical informatics officer at Boston Medical Center (BMC), agreed, adding that the responsibility of large safety net organizations like BMC is to work with vendors to ensure that communities from diverse backgrounds, such as those identifying as Black and Brown, publicly insured, low income, and multilingual, are included in the design, implementation, and evaluation processes.

More importantly, she emphasized, is that developers and vendors of AI-based technologies “be humble” and willing to receive critical feedback at every stage in their process. Reiterative evaluation allows for accountability, and accountability is important to building and maintaining trust between healthcare organizations and the people they serve. Otherwise, there is risk of perpetuating systemic challenges and widening disparities.

Codifying Accountability

Accountability should also be codified. Kade Crockford, director of technology & justice programs at American Civil Liberties Union, Massachusetts (ACLUM), recalled a similar approach in the auto industry, where quality and safety regulations set the standard for what’s acceptable. Years later, we are safer in our cars because of these regulations.

So, instead of waiting until after the harms are identified, how can we work together to be proactive, develop a framework for accountability, and ensure that equity, transparency, quality, and safety are embedded from start to finish, across development phases, and across industries when it comes to AI?

Dr. Christopher Fields, associate research scientist at Yale School of Medicine, is working on just that. His work in developing algorithms for predictive and diagnostic care unravels the information models used to train algorithms that may be perpetuating stereotypes, misinterpreting information, and misattributing demographic differences. These issues have implications when using predictive models in clinical settings and may contribute to missed or late diagnoses in illnesses such as end-stage renal disease, diabetes, hypertension, and more. Data can be misinterpreted, especially when using average data to inform individual decision-making.

“Race is a messy proxy” – it’s not enough to use it as a variable or correctional factor to measure “average performance,” Dr. Fields said. “Trust isn’t just about what model works on average, it’s about whether we understand where it works well and where it might fail.”

The Road Ahead

The discussion only scratched the surface. Remaining questions include: How do we build better models? Where can we ensure open feedback, transparency, and accountability? What is the system of care that we are striving toward?

As Dr. Patel concluded, the first step is to commit to building a foundation for trust:

MHA members share this commitment by creating spaces for shared learning and discussion. With more educational opportunities focused on AI in healthcare and health equity to come, we welcome you to share your ideas and feedback as the journey continues.