What Should Our Board Be Asking About AI Risk?

Key Takeaways

·The Joint Commission and CHAI guidance (September 2025) is voluntary, and one of its plainest recommendations is a mechanism to keep the governing body "updated on uses, outcomes, and potential adverse events."
·AI is already in most hospitals. A federal data brief reports that 71% of hospitals used predictive AI integrated with their EHR in 2024, and 80% of hospitals used predictive AI sourced from their EHR developer.
·In the same 2024 data, 82% of hospitals said they evaluated predictive AI for accuracy and 74% for bias. About one in six answered "don't know" on accuracy, and about one in five on bias.
·We group the board's questions into five areas: inventory, validation, accountability, monitoring and insurance. The grouping is ours.
·Neither the guidance nor CHAI's playbooks set a reporting schedule. Both say regular. We suggest quarterly, with a fuller annual review.
·What the law requires of your board is a question for your counsel. This page is about what to ask management.

The short answer

Ask five things. What AI are we using, and who keeps the list? How do we know each tool works on our patients? Who is the named person accountable for it? How would we find out if a tool was performing badly or unfairly? Has anyone reviewed our insurance with AI in mind? Many organizations will find they can answer some of these with specifics and others only in general terms. The second group is where to start.

Why This Is Reaching the Board

The Joint Commission and CHAI guidance on the responsible use of AI puts the board in its first element. Organizations should establish a governance structure "including a mechanism to keep the hospital's governing body updated on uses, outcomes, and potential adverse events," and "the fiduciary board of the healthcare organization should be regularly updated on AI use and its outcomes in healthcare." The guidance is voluntary. So is the certification the Joint Commission built on it in June 2026, whose standards begin with governance.

The practical reason is that AI is already in the building, often without a purchase the board ever saw. The federal ASTP/ONC data brief on hospital use of predictive AI reports that 71% of hospitals used predictive AI integrated with the EHR in 2024, up from 66% in 2023, and that 80% of hospitals used predictive AI sourced from their EHR developer. CHAI's Risk and Impact Assessments playbook observes that "boards are increasingly asking: 'What are our top AI risks?' and 'Where are they reflected in ERM?'"

Lawyers who advise boards talk about a duty of oversight. What that duty asks of your board depends on your state's law and on whether your organization is nonprofit or for-profit, and it is a question for your counsel. We aren't attorneys, and this page doesn't try to answer it.

Five Areas for Board Questions

These aren't technical questions. They are governance questions about whether the right structures exist and whether accountability is clear, and a director doesn't need to understand model architecture to ask them. The five-part grouping is our own. Where a published source supports a point, we name it.

Inventory and awareness

Does the organization know what AI tools are in use, by whom, and for what purpose? A board can't oversee what nobody has listed. Ask who maintains the list, how often it's updated, and whether it includes AI features that came inside the EHR and other platforms, tools bought at the department level, and consumer AI that staff use without formal approval. The Joint Commission's certification program calls for a centralized registry of governed AI tools, and CHAI's Lifecycle Management playbook recommends logging every AI system in a central inventory. The answer should come from a maintained register.

Validation and performance

How do we know each tool works on our patients? FDA clearance reflects a review at one point in time, and many AI tools in hospitals are never reviewed by FDA at all. In the 2023 American Hospital Association IT survey, 61% of hospitals that used predictive models had evaluated them for accuracy on their own data, and 44% had evaluated them for bias (Nong and colleagues, Health Affairs, January 2025). The federal brief on 2024 reports 82% and 74%, from a differently worded question, with 15% and 21% of hospitals answering "don't know." Ask how tools are checked before they go live, who reviews the results, and what happens when performance after go-live differs from what was expected. CHAI's playbook leaves it to each organization to decide, by risk tier, when local validation is needed.

Governance structure and accountability

Who is accountable? The guidance says governance structures "should include a designated individual(s) with appropriate technology expertise, ideally in AI if available, to lead implementation and use of AI tools." It also says the structure doesn't need to be a standalone team. Our own view goes one step further: there should be a named person with the authority to pause a tool when a safety concern arises. If the answer to "who would explain what happened if this tool contributed to harm?" is "the committee," ask which member.

Risk monitoring and incident detection

Would we find out? A 2019 study in Science examined a widely used commercial algorithm that the authors say affects millions of patients. It found that "at a given risk score, Black patients are considerably sicker than White patients," because the algorithm predicted health care costs in place of illness. The tool had been in wide use before the study identified the problem. Ask what monitoring is in place for drift, bias and safety events, and who sees the results. The federal brief reports that 79% of hospitals conducted post-implementation evaluation or monitoring of predictive AI in 2024. An absence of reported AI incidents may mean the tools are working well, or that nobody is looking.

Insurance

Has anyone reviewed our coverage with AI in mind? Many policies were written before clinical AI was common. Ask whether the organization's malpractice, cyber and directors' and officers' coverage has been reviewed for AI-related events, who did the review, and what questions carriers are asking at renewal. Whether a given policy would respond to a given event is a question for your broker and counsel.

The Question Behind the Questions

Every question above builds toward one. Can management show the board, with documents, that AI is being governed? A maintained inventory, a named accountable executive, validation results for the tools that matter most, monitoring reports and a record of decisions are what that looks like on paper.

Alignment with the Joint Commission and CHAI guidance gives the board a recognized reference point for describing its oversight to anyone who asks, whether that is a regulator, an insurer, a buyer in a transaction or a patient's family. It is also the record your counsel will want to have if something goes wrong. Executive accountability remains with leadership. The board's job is to make sure that accountability is visible.

What Board-Level AI Reporting Could Include

This is our suggestion for a regular report. These are accountability updates that connect AI use to risks the board already watches, and they don't need to be technical.

·The current AI tool inventory with a risk classification for each tool. Tools that influence high-impact clinical decisions warrant closer board attention than administrative automation. CHAI's playbook suggests high-risk AI be visible in the enterprise risk register.
·Validation status for each high-impact tool: dates, method and summary findings.
·A summary of AI-related incident reports, near misses and bias monitoring results from the prior period.
·The named accountable executive, with a short update on governance structure and significant changes since the last report.
·The status of any insurance review for AI-related events, including open questions.

Full Board or a Committee?

Either can work, and we haven't found an authority that settles it for hospitals. A board can route AI through its quality, audit or risk committee, with the full board hearing a summary. What matters more is that the assignment is explicit, that the committee's charter says so, and that the reporting rhythm is set.

One example in CHAI's Organizational Structure playbook lists board work first among its governance activities: "Establish strategic AI priorities with the Board and maintain a regular cadence of updates and education to support informed oversight and decision-making." Education is the easy part to skip. A short annual session on how the organization's main AI tools work will improve every question asked for the rest of the year.

Frequently Asked Questions

Common questions from board members and the executives who report to them.

Does a hospital board have a legal obligation to oversee AI?

That is a legal question, and the answer depends on your state's law and on whether your organization is nonprofit or for-profit. Ask your counsel. What we can say is what the published guidance recommends: the Joint Commission and CHAI guidance says the board should be regularly updated on AI use and its outcomes, and that the governance structure should include a mechanism to keep the governing body informed of uses, outcomes and potential adverse events.

What does the Joint Commission say about boards and AI?

Its guidance with CHAI (September 2025) recommends regular board updates on AI use and outcomes, and a mechanism for keeping the governing body informed. The guidance is voluntary. On June 1, 2026 the Joint Commission opened a voluntary certification built on it, and the certification standards begin with governance. We haven't found a Joint Commission accreditation standard specific to AI.

How often should the board get AI reports?

Neither the Joint Commission and CHAI guidance nor CHAI's playbooks set a schedule. Both say regular. We suggest quarterly reporting to the responsible committee, with a fuller annual review for the full board, and an immediate report when a significant AI-related safety event occurs.

We already have an AI committee. Is that enough?

A committee is a good start, and the guidance doesn't ask for more structure than you need. Check three things: whether a named individual is accountable for each high-impact tool, whether the committee's work reaches the board on a set rhythm, and whether the board has ever seen the AI inventory. Those are the pieces the guidance specifically recommends and that committees often leave implicit.

Should boards ask for a named accountable executive?

We think so. The guidance recommends a designated individual or individuals with appropriate technology expertise to lead implementation and use of AI. Boards can be explicit about asking management to name that person and to say what authority they hold.

Sources

The Joint Commission and the Coalition for Health AI. Guidance on The Responsible Use of AI in Healthcare (RUAIH), Element 1. September 17, 2025. (PDF)
The Joint Commission. Responsible Use of AI in Healthcare certification (program page). Accessed September 20, 2026.
Coalition for Health AI. AI Governance Playbooks: Domain 2, Organizational Structure; Subdomain 4.1, Lifecycle Management; Subdomain 4.2, Risk and Impact Assessments. Released May 27, 2026.
ASTP/ONC. Hospital Trends in the Use, Evaluation, and Governance of Predictive AI, 2023-2024. Data Brief No. 80. September 2025.
Nong P, Adler-Milstein J, Apathy NC, Holmgren AJ, Everson J. Current Use And Evaluation Of Artificial Intelligence And Predictive Models In US Hospitals. Health Affairs. 2025;44(1):90-98.
Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447-453.

What Should Our Board of Directors Be Asking About AI Risk?