As artificial intelligence applications increasingly enter the healthcare sector, significant questions are emerging regarding the robust data security and privacy practices that accompany these new technologies. Companies like OpenAI, Anthropic, and Google have launched AI-powered health offerings, designed to assist users and organizations with health advice, medical record analysis, and diagnostics, raising concerns about the protection of sensitive personal health information.
Hundreds of millions of individuals already utilize platforms like ChatGPT for health-related inquiries, and research suggests artificial intelligence, particularly large language models, can achieve remarkable proficiency in medical diagnostics, with some studies noting capabilities that surpass human doctors. However, beyond standard cybersecurity risks, experts highlight a critical gap in legal protections for personal medical data shared with these AI health applications.
The HIPAA Gap in AI Healthcare
A key concern is that these AI companies are likely not subject to the same stringent federal regulations, such as the Health Insurance Portability and Accountability Act (HIPAA), that govern traditional healthcare providers. This means that the data security and privacy assurances offered by these tech giants may stem from company policy rather than legal mandate.
Sara Geoghegan, senior counsel at the Electronic Privacy Information Center, explained that data protection offered through terms of service agreements is distinct from the legally binding protections afforded by regulated healthcare entities. “On a federal level there are no limitations – generally, comprehensively – on non-HIPAA protected information or consumer information being sold to third parties, to data brokers,” she stated.
The bankruptcy and subsequent sale of genetic testing company 23andMe last year serves as a cautionary example of the risks consumers face when entrusting sensitive health or biometric data to unregulated platforms. The potential for data breaches, leakage, and misuse of information is heightened when entities are not bound by strict data protection laws.
Furthermore, these AI health applications can carry the same inherent risks as other generative AI products, including data leakage, the generation of inaccurate information (“hallucinations”), and vulnerability to prompt injection attacks. The healthcare industry itself has already been grappling with an escalating number of data breaches, even before the current AI surge.
Existing Challenges in Healthcare Cybersecurity
Healthcare organizations are frequent targets for cyberattacks, including hacking, phishing, and ransomware. Despite HIPAA’s legal frameworks for protecting patient data, breaches persist due to factors such as outdated software, reliance on numerous third-party vendors, and the substantial financial and technical resources required to maintain robust cybersecurity defenses.
Carter Groome, CEO of First Health Advisory, a healthcare risk management firm, noted that the security protections offered by AI companies may not extend beyond internal company policies. “They’re not mandated by HIPAA,” Groome said. “Organizations that are building apps, there’s a real gray area for any sort of compliance” with healthcare data privacy laws.
Privacy is paramount in healthcare to protect sensitive medical records and foster trust in the health system. Laws like HIPAA mandate that covered entities and their business associates implement “reasonable and appropriate administrative, physical, and technical safeguards” for protected health information. These regulations also include breach notification rules, requiring prompt reporting to affected individuals and federal authorities.
Understanding HIPAA Applicability
Andrew Crawford, senior counsel at the Center for Democracy and Technology’s Data and Privacy Project, and Groome indicated that companies like OpenAI, Anthropic, and Google are unlikely to be considered covered entities under HIPAA. According to the Department of Health and Human Services (HHS), HIPAA’s security rule applies to health plans, clearinghouses, healthcare providers, and business associates engaged in the transfer of Electronic Protected Health Information (ePHI).
While Anthropic states Claude for Healthcare is “HIPAA-ready” and OpenAI claims its enterprise healthcare products “support” HIPAA compliance, neither company asserts full HIPAA compliance for their direct-to-consumer AI health offerings. Representatives for OpenAI and Anthropic did not respond to requests for comment.
This distinction means that numerous companies not bound by HIPAA’s privacy mandates could be collecting, sharing, and using personal health data. “And since it’s up to each company to set the rules for how health data is collected, used, shared, and stored, inadequate data protections and policies can put sensitive health information in real danger,” Crawford stated.
Company Promises vs. Regulatory Requirements
Companies are emphasizing data security in their AI health product launches. OpenAI has stated its health model incorporates advanced encryption, isolation features for conversations, and multifactor authentication. Data is encrypted at rest and in transit, chats can be deleted within 30 days, and user data is not used for AI training. For medical record uploads, OpenAI partners with b.well, which claims a transparent privacy policy, user control over data sharing, and adherence to standards that aim to meet or exceed HIPAA requirements, though it acknowledges no system is entirely immune to cyber risk.
However, legal experts caution that claims of “HIPAA compliant” or “HIPAA ready” by tech companies may not equate to the legal obligations under HIPAA. Geoghegan noted that some unregulated wellness businesses ambiguously claim “HIPAA-compliant” status to avoid the fact that they are not legally bound by HIPAA regulations. “Generally speaking, a lot of companies say they’re HIPAA compliant, but what they mean is that they’re not a HIPAA regulated entity, therefore they have no obligation,” she said.
Groome views some AI companies’ security assurances as potentially “hyperbolic,” intended to quell privacy concerns, noting the “comical level of how much they say they’re going to protect your information.” The inherent “black box” nature of AI, where even developers may not fully understand its internal workings, adds another layer of uncertainty, particularly concerning sensitive health data.
Despite these risks, AI health apps are expected to remain popular, partly due to the high cost and inaccessibility of traditional American healthcare. While these tools offer convenience and cost-effectiveness, the trade-offs in data privacy and security are significant. As AI continues its integration into healthcare, regulatory bodies and consumers will likely need to closely monitor the evolution of its data handling practices and the effectiveness of its security measures.