Chatbot Care Managers? ACOs Should be Cautious in AI Adoption

Given Artificial Intelligence’s potential to improve patient care and reduce costs, it’s no surprise that AI applications are gaining momentum in health care. As your organization explores the benefits of AI in your journey towards Value-Based Care, however, you need to carefully assess the implications, for better and worse.

Evaluating AI implications can be tricky. Healthcare AI varies widely, with clinical technology paving the way for advancements in diagnostics and treatment. But pressure on ACOs to achieve savings is spurring consideration of machine assistants for customary clinician services. That’s the message of one recent study suggesting that AI chatbots may outperform physicians in communicating with patients, offering higher-quality responses and displaying greater empathy. The study evaluated how chatbots versus physicians responded to 195 patient questions from Reddit’s r/AskDocs. Surprisingly, healthcare professionals who reviewed the responses favored chatbot answers over those from physicians in terms of quality (78.5 to 22.1 percent) and empathy (45.1 to 4.6 percent). Not a great report card for physicians!

But before planning to use chatbots in patient education, navigation, and coaching – especially given staffing shortages in health care – ACOs should closely examine the underlying assumptions. Consider these very human factors: How do patients feel about discussing treatment plans with chatbots? Was the study validated and reviewed by peers?  What were the study’s limitations and biases? Did physicians couch their responses with caution due to liability or clinical concerns? Either could have negatively affected the tone of physician communications.

The rapid adoption of AI technology risks incorporating human biases into algorithms, perpetuating gender and race biases through AI healthcare recommendations. Before jumping on the AI bandwagon, we need a better understanding of the effects on physicians and patients, as well as a thorough evaluation of potential unintended consequences.

There may be an advantage in time and money to using chatbots to assist in patient education and to support—not replace–human roles in medicine. We still need to preserve essential conversations between patients and physicians to maintain trust. Ceding that direct communication to technology could erode the patient-physician relationship. It would also undermine efforts to recruit talent into the healthcare profession, where shortages of skilled clinicians is already a significant issue, especially in rural and poorer communities.

Under pressure to adapt to Risk, many ACO stakeholders may be eager to deploy AI solutions. To resist being swept up by the momentum, carefully consider your options, support your participating clinicians in their clinical AI applications, and explore how you might collaborate.

Here are three guidelines for leveraging AI to strengthen your organization while recognizing potential weaknesses of machine-based systems:

1. Use AI to analyze complex data for risk identification, patterns, and variations in healthcare services and costs. AI’s ability to efficiently analyze diverse datasets aligns well with Value-Based Care. Personalized treatment plans based on multiple patient data points can be developed using AI analysis. For instance, AI algorithms can drive episodes of care, enabling ACOs to compare procedure costs, reduce variations, target patients for clinical review, and identify opportunities for improvement. However, be sure to exercise caution and scrutinize algorithms for potential biases that may impact population groups and health equity.
2. Evaluate the use of AI in creating patient materials for review by clinicians. ACOs have a responsibility to provide patients with factual information, support medical decision-making, promote cost transparency, and engage patients and their families in the process. Chatbot-generated communications, subject to clinical review, can be an efficient way to develop the necessary tools.
3. Defer replacement of direct communications with patients with AI and test the programs first. Pilot AI-driven communication and education tools, such as patient check-ins and self-management programs, with evaluation of changes in outcomes and patient acceptance. The urgency to utilize data wisely will drive ACOs toward AI solutions. Remember that technology is never neutral. Plan carefully for human and non-human resources to ensure that any AI applications benefit your organization and avoid potential, significant harm.

About Theresa Hush

As CEO and Co-founder of Roji Health Intelligence, Theresa Hush is a healthcare strategist and change expert with experience across the healthcare spectrum, including public, non-profit and private sectors. Her accomplishments include leading the transformation of Blue Cross Blue Shield regulations in Illinois, improving access to care as Director of the Illinois Medicaid program, and serving in executive leadership for both private payers and physician organizations. An expert at creating consensus for desired change through education and collaboration, Terry helps organizations take actions that will direct their future through meaningful technology and programs.