Pharmacy AI for Refills: Kinney Drugs Pulls Back

Kinney Drugs implemented the AI virtual assistant for patient communication and refill services in May. Soon after, its customers began to experience problems, they said. ／ According to nearly a dozen Kinney Drugs customers across its stores in Vermont and New York who spoke to VTDigger, the AI rollout has coincided with a range of problems. Delays have grown between physicians ordering drugs and the pharmacy processing them. The system has been unable to locate longtime customer accounts, has ordered incorrect dosages and has failed to notify customers when refills were ready for pickup. ／ Their mounting frustrations toward the AI is leading some customers to leave the pharmacy entirely, they said, and forcing others to continue using a system they distrust due to a lack of pharmaceutical options in rural areas. ／ “I’ve got four bottles of something that I take twice a week sitting here,” Callaghan said. ／ Disclosure: Kinney Drugs is an advertiser with VTDigger. — From the rollout, the problems customers reported, and the advertiser disclosure

Previously, to refill a prescription at Kinney Drugs, a customer would call a prompt-based, touch-tone, phone system that would give them the opportunity to navigate a keypad to reach a pharmacist. Customers who spoke to VTDigger said this old system worked smoothly, and they had no problems. ／ The AI assistant, which Kinney Drugs named Burt, after its founder, was built by third-party AI pharmaceutical company, Synerio. ／ Synerio, the Texas-based technology company that supplies the AI tool to the pharmacy. ／ In the pharmacy’s terms, Kinney Drugs states that “Burt may not be accurate, complete or up-to-date and may be misleading or contain errors and omissions.” ／ “Burt demands that you talk to him,” said Maria Aveni, a Vermont Kinney Drugs customer who has worked in healthcare as an anesthesiologist. “There’s no option to use a keypad to enter what you want to, which has been the case with other telephone interactions I’ve had. So it’s more of a talk to Burt or don’t get your drugs.” ／ According to Aveni and other customers who spoke to VTDigger, Kinney Drugs did not communicate about the tool before implementation. ／ While some customers recalled signing onto a new refill system earlier this summer, others said they didn’t realize the company implemented the new tool until they called for a refill and heard the AI voice. — From the old phone line, the vendor, the pharmacy's terms, the missing keypad option, and the lack of advance notice

A Kinney Drugs representative who did not want to be identified by name, wrote in a statement to VTDigger that the company is “aware that some patients have experienced frustration with Burt, especially during rollout,” and that it is reviewing each concern and attempting to reach each patient. While customers said they were confused over why Kinney Drugs switched to the AI, the company cited a clear reason: to eliminate repetitive tasks and free up time for its pharmacists. ／ Its leaders believe the AI has helped in achieving this goal, with one director saying that “it’s definitely doing the trick and handling those phone calls in such a nice manner that it’s reducing the calls to our pharmacy.” ／ But customers’ experience with the tool has led to an oftentimes opposite effect. As they encounter problems with the AI, many are turning to pharmacists more for help. In Vermont, pharmacists are already overburdened due to a chronic worker shortage. ／ “It’s slowing down productivity of an already stressed staff, and because that staff has treated me for a long time, and I feel loyalty, it isn’t just a financial concern for me, it’s an inhumane way to treat the staff,” Aveni said. — From the stated goal, the director's view, and the customers' turn to pharmacists

“Burt was introduced to improve access, convenience, and responsiveness for patients. Burt is designed to supplement pharmacy teams by handling routine, non-clinical interactions so staff can focus more time on direct patient care, clinical questions, and in-pharmacy services,” a Kinney Drugs representative previously wrote in a statement to VTDigger. — From the company statement quoted in the August 10 story

Any customer who provides their telephone number to Kinney Drugs automatically consents to interacting with the tool, though they may later opt out, according to the pharmacy’s terms. ／ The pharmacy’s privacy policy, released the same month as the AI assistant, discloses, for example, that customers’ protected health information may be used in AI tools to improve the pharmacy’s operations. ／ “Once I went into the store and actually read the privacy policy, I was shocked, and immediately transferred my prescriptions to UVM Hospital,” Tuan Green, a former Vermont Kinney Drugs customer, said. ／ Near, the UVM computer science professor, said that regulations like the new Vermont law are inherently flawed simply because they are based on consumer consent. Kinney Drugs, for instance, spreads its terms and conditions across 11 documents. ／ “I think that people absolutely do not understand what they’re consenting to, and so basing our regulation and what we think is okay on the idea of consent is not a realistic solution to these kinds of privacy concerns nowadays,” he said. ／ Joseph Near, an associate professor of computer science at the University of Vermont, said that, privacy-wise, the AI tool is worse than the pharmacy’s previous refill system because this additional company, Synerio, now also has access to customers’ health data. ／ “If I just interact with my healthcare provider, I’m relying on the security that they have. If they now start interacting with many other organizations, I now have to start worrying about all the security issues with each of them,” Bradley Malin, a professor of biomedical informatics at Vanderbilt University Medical Center, said, referring to Synerio. ／ The Kinney Drugs representative said that the company maintains “multiple layers of security controls,” invests in “information security technologies to help protect sensitive data” and has the protocols in place to respond to potential data breaches. ／ However, as long as the pharmacy has a business associate agreement — the contract under HIPAA that establishes that third-party vendors securely handle protected health information — the companies are contractually obliged to handle data securely. The Kinney Drugs representative declined to share the agreement, citing an inability to disclose private vendor contracts, but wrote that the company “maintains the appropriate agreements.” ／ Synerio did not respond to multiple requests for comment, but also says on its webpages that the tools it creates are built to be compliant with HIPAA. — From the consent terms, the privacy policy, the 11 documents, the vendor's access to health data, and the HIPAA business associate agreement

As AI pops up in healthcare systems across the country, several lawyers told VTDigger that the technology is outpacing regulation, leaving consumer protections thin. ／ In Vermont, a new data privacy bill, S.71, signed by Gov. Phil Scott on June 16, takes effect in two years. That delay creates a gap in compliance to a bill that privacy advocates say is already too weak to protect consumers. ／ “Even if it were in effect, I’m not sure that it would necessarily cover the situation,” said Ryan Kriger, a Vermont lawyer and resident who testified on the bill. ／ While Synerio did not respond to VTDigger’s questions about customer protections, Kinney Drugs’ pharmacy management system, McKesson, acknowledged them in public filings, writing that “reliance on third-party AI tools and solutions may expose us to risks that are outside of our control, including compliance gaps.” — From the observation that regulation lags the technology, and the disclosure about third-party AI risk

The call came as the pharmacy, Kinney Drugs, recently pulled back its AI assistant, John Marraffa, president of the Vermont and New York pharmacy chain, told WCAX on Friday. ／ Privacy experts raised concerns surrounding the security of protected health information, a characterization Kinney Drugs denied. ／ But Marraffa acknowledged in his interview with WCAX that the new technology had been creating issues for customers, saying it did not get customer experience “right,” while defending the AI tool’s security. ／ Marraffa and a Kinney representative did not respond to requests for comment Monday from VTDigger. ／ The scaling back of the AI tool means that the pharmacy has returned to its previous prompt-based, touch-tone phone system for patient calls, Marraffa told WCAX. The AI tool will still be used for prescription refill texts. Now, however, patients must opt in to receiving these texts. ／ Previously, any customer who provided their telephone number to Kinney Drugs automatically consented to interacting with the AI tool ／ When Kathy Callaghan called her local pharmacy for a prescription refill last Wednesday, she expected to hear the usual AI voice that had attempted to assist her for the last three months. Instead, a pharmacist picked up. ／ “The company finally admitted that it was a mistake,” Bellini said. — From the August 10 story on the pullback, the opt-in change, and the president's remarks