How to Create and Maintain an AI Tool Inventory for Your Health System

Key Takeaways

·The AMA's governance toolkit suggests asking each department head to inventory AI tools, including AI features used through enterprise technology. In our experience, that last part is where inventories have their largest blind spot.
·A shared, organization-wide definition of 'AI tool' is the most important step before launching any inventory effort. Without it, departments will report inconsistently and your inventory will be incomplete before you've started.
·The HHS AI Use Case Inventory, the format federal health agencies use to report their own AI, is a useful starting template for the data fields to track. The field list below adapts it for clinical settings.
·Inventory comes first. Data access control, PHI and PII handling, and output monitoring all depend on knowing what is running.
·Annual review cycles are not sufficient. Tie inventory maintenance to vendor contract reviews, procurement decisions, and a standing update process.
·A complete tool inventory answers what AI is running. The governance question it makes possible is which decisions AI influences, at what consequence level, and with what human-in-the-loop status. The inventory is the starting point for that analysis.

The short answer

Start by establishing a shared organization-wide definition of what counts as an AI tool, then ask each department head to inventory their tools, including AI features embedded in platforms already in use. Use the HHS AI Use Case Inventory data fields as a starting template. Categorize what you find by clinical risk level, identify tools with no named owner or validation evidence, and build a continuous update process from there. You can't govern what you haven't counted, and keeping the count current is ongoing work.

Why Most Health Systems Undercount Their AI Footprint

When health system leaders are asked how many AI tools their organization is running, the first estimate is almost always low. Nobody is hiding anything. The question is harder to answer than it sounds.

Standalone AI products that went through a formal procurement process are relatively easy to see. The AI scribe a clinical department has been piloting, the sepsis prediction tool an ICU adopted, the revenue cycle optimization platform a CFO approved. Those tend to be visible because they had a buying decision attached to them.

What gets missed are AI features embedded within enterprise software the organization already runs. An EHR almost certainly has AI-assisted documentation, predictive risk scoring, or AI-enhanced imaging reading built into modules that were licensed years before anyone was asking governance questions about AI. A billing platform may have machine learning-driven claim prioritization running continuously in the background. These aren't new purchases. They arrived through software updates, often without triggering any governance review, and they're rarely top of mind when someone asks about AI use.

The AMA's governance toolkit makes the same point. It suggests the working group "could request each department head to inventory AI tools, including AI features used through enterprise technology."

Inventory as a Governance Foundation

Panelists at a HIMSS26 preconference session on AI operating models listed four things they considered essential for governing AI agents: knowing which agents are out there, knowing what data each one can reach, keeping PHI from reaching an agent unredacted, and watching what the agents do. The last three depend on the first. An organization that doesn't know what AI tools are running can't systematically assess data access, can't ensure PHI handling policies apply to all tools, and has no basis for a monitoring program. Inventory is where governance starts.

Define "AI Tool" Before You Ask Anyone to Report

The most common reason AI inventories fail before they start is definitional ambiguity. If you ask department heads to report their AI tools without giving them a shared definition, you'll get wildly inconsistent results. One department head will list only FDA-cleared clinical decision support tools. Another will include every software product with a chatbot feature. A third will omit the consumer generative AI tools their staff use informally because those weren't officially approved.

Before any survey goes out, your governance working group needs to agree on a definition that department leaders across the organization can apply consistently. A working definition should cover: predictive and statistical models, classical machine learning, generative AI including large language models, AI features embedded within existing enterprise platforms such as EHRs and imaging systems, and consumer AI tools staff may be using in workflows even without formal approval.

This definition doesn't need to be perfect. It needs to be specific enough that the people responding understand what to include and what to leave out. You can refine scope in subsequent inventory cycles once the baseline is established. The AMA toolkit says it is helpful to define "AI tools" so that people across the organization understand what belongs in the inventory, and it suggests feeding the results into a standard inventory management tool.

What to Track for Each Tool

The fields below are adapted from the HHS AI Use Case Inventory, with additions for clinical settings. The CHAI Lifecycle Management playbook separately recommends a centralized inventory that captures intended use, risk tier, monitoring owner, and version tracking. Together they cover what you need for governance, monitoring, and board reporting.

Use case name and description

What the tool does, what problem it's solving, and what clinical or operational workflow it touches.

Responsible department and named owner

Which department or function is accountable for this tool's performance and governance. A tool with no named owner is a governance gap.

Stage in the lifecycle

Pre-deployment, pilot, deployed, or retired. Keep retired tools in the record too, so you can see what was discontinued and why.

Clinical impact classification

Whether the tool directly influences patient care decisions, is one step removed in care coordination or operational workflows, or is primarily administrative. Clinical tools require the most rigorous governance attention.

AI type

Predictive analytics, generative AI, natural language processing, computer vision, or another category. This shapes the relevant governance questions about validation, failure modes, and output review.

System outputs

What the tool produces: a recommendation, an alert, a generated document, an autonomous action, or a score. The output type determines what clinician oversight looks like.

Vendor and build or buy status

Vendor name, version or release information if available, and whether the tool was built internally, procured from a vendor, or is a feature embedded in a larger enterprise platform.

Authorization to operate status

Has this tool been formally reviewed and approved for use? By whom? When? This field captures the governance history of the deployment decision.

PII and PHI involvement

Does the tool use protected health information? Does it use demographic variables such as race, age, or socioeconomic status in its predictions? Both have governance implications.

Training data description

What data was the model trained on? Does it reflect your patient population or a different clinical context? This bears directly on local validation, a recurring theme in the Joint Commission and CHAI guidance.

The goal isn't to fill in every field for every tool on day one. A partially complete inventory is significantly more useful than no inventory. Start with the fields you can answer quickly, flag the gaps, and build a process to fill them over time.

Categorizing What You Find by Risk Level

Once you have an initial catalog, sort it by risk level. Clinical tools that influence patient care decisions call for a different level of governance attention than tools managing scheduling, supply chain, or administrative workflows. A practical three-tier framework.

Clinical (Tier 1)

Tools that directly influence clinical decisions, patient care workflows, or diagnostic outputs. Examples include sepsis prediction models, AI-assisted radiology reads, ambient documentation scribes, and clinical decision support alerts. These require the most rigorous validation, named clinical ownership, active monitoring, and documented oversight. This is the tier where validation and monitoring effort should concentrate.

Operational (Tier 2)

Tools managing care coordination, utilization management, prior authorization support, or patient communication workflows. These deserve real governance attention, particularly around health equity and access to care, but are one step removed from point-of-care decisions. Validation and monitoring are still warranted, though the urgency is lower than Tier 1.

Administrative (Tier 3)

Tools handling revenue cycle, supply chain, HR, facilities management, or other non-clinical functions. Lower direct patient safety risk, but still subject to data governance, privacy, and bias considerations. These shouldn't be invisible in your inventory, but they don't require the same governance intensity as clinical tools.

What to Do with Tools Discovered Without Formal Approval

You will find them. Departments that moved quickly, vendors who rolled out AI features through platform updates, staff using consumer generative AI tools in daily workflows. The inventory process should not be framed as an audit designed to penalize anyone for using technology that was helping them do their jobs.

The AMA toolkit addresses framing directly. The assessment "is not meant to challenge activity already in flight, but to catalog successful-use cases, identify future opportunities, and ensure compliance with applicable law." That framing matters because it determines whether department heads are honest with you or protective of their teams. An inventory effort perceived as punitive will produce an incomplete inventory.

Tools discovered without prior approval should be assigned to an "under review" category. High-risk clinical tools in that category need expedited governance review. Lower-risk administrative tools can move through a standard review queue. What they can't do is remain in limbo indefinitely with no named owner and no review timeline.

EHR-Embedded AI: The Most Common Blind Spot

·EHR vendors have been adding AI capabilities through platform updates for years. Predictive risk scores, ambient documentation features, AI-enhanced coding support, and clinical decision support modules may all be running in your environment without having gone through a governance review.
·The starting point is to formally request from your EHR vendor a complete list of all AI-enabled features currently active in your deployment, and ask for it in writing. Organizations that haven't made that request are operating without a complete picture of their clinical AI footprint.

Keeping the Inventory Current

An AI tool inventory that's twelve months out of date can mislead more than it helps. The AI deployment environment in most health systems is changing faster than annual review cycles can track.

Practical approaches to continuous maintenance include tying inventory updates to vendor contract review cycles, requiring that any new AI tool deployment or vendor-pushed AI capability update triggers a notification to the inventory owner, including AI inventory status as a standing item in department leadership meetings, and assigning a named inventory owner with clear authority to query vendors and department heads when changes are suspected.

At enterprise scale, human-only review breaks down. Panelists at the same HIMSS26 session made the point that governing AI at scale takes technology as well as people, including tools for inventory and monitoring. Organizations building governance infrastructure for larger AI portfolios should be evaluating purpose-built governance tools rather than maintaining a growing inventory in a spreadsheet.

The inventory also matters because of where federal transparency rules stand. ONC's HTI-1 final rule (January 2024) requires developers of certified health IT to give users information about the predictive decision support tools they supply (45 CFR 170.315(b)(11)). In December 2025, ONC proposed in its HTI-5 rule to "remove all requirements related to source attributes" along with the related risk management requirements. The comment period closed February 27, 2026, and as of this page's date ONC's website still lists HTI-5 as a proposed rule. If it is finalized as proposed, less information about embedded AI will reach health systems by default, and more will depend on your own inventory and on what your vendor contracts call for.

What the Inventory Makes Possible

A complete, maintained tool inventory tells you what AI is running in your organization. That's a necessary starting point. It's not a sufficient governance posture on its own.

The governance question the inventory makes possible is which decisions AI influences, at what consequence level, and with what human-in-the-loop status. That is a different question from which tools are running.

In our experience, the gaps that surface after an adverse event are more often about decision accountability than about the inventory. A tool that's properly catalogued but governs a high-stakes clinical decision without named executive oversight is still exposed. The inventory shows the tool exists. It doesn't show whether anyone is responsible for what that tool decides.

The next step, mapping AI by the decisions it influences, is what board reporting calls for. The inventory makes that analysis possible.

Frequently Asked Questions

Common questions from health system leaders building or improving their AI tool inventories.

What counts as an 'AI tool' for inventory purposes?

For inventory purposes, treat any software component using machine learning, statistical prediction, large language models, or algorithmic decision support as an AI tool. This includes standalone AI products, AI features embedded within existing enterprise platforms such as EHRs and imaging systems, and consumer AI tools staff may be using in workflows even without formal approval. The goal is a complete picture of what's running, not just the tools that went through a formal procurement process.

Should we include AI features built into our EHR?

Yes, and this is where most inventories have their largest blind spot. EHR vendors have been adding AI capabilities through platform updates for years. Risk prediction models, ambient documentation features, AI-enhanced coding support, and clinical decision support modules may all be running in your environment. The starting point is to formally request from your EHR vendor a complete list of all AI-enabled features currently active in your deployment, and ask for it in writing. Organizations that haven't made that request are operating without a complete picture.

How often should we update the inventory?

Annual review cycles are not sufficient given how quickly the AI deployment environment is changing. Tie inventory updates to your vendor contract review process, require notification from departments when new AI tools are deployed or when vendors push updates with new AI capabilities, and designate a named inventory owner responsible for maintaining currency. Treat the inventory as a living document. A tool found active in your environment a year after deployment, with no governance review, is a gap your governance committee and your counsel will want closed.

What do we do with AI tools we discover were never formally approved?

Don't treat this as a punitive audit. Assign unapproved tools to an 'under review' category, communicate clearly to department heads that the goal is to catalog and support rather than penalize, and prioritize review based on risk level. Clinical tools with no validation or named owner need expedited governance attention. Administrative tools with lower risk profiles can move through a standard queue. What they can't do is stay in the inventory indefinitely without a resolution path.

Who should own the AI tool inventory?

Ownership typically sits with the AI governance committee or its designated lead, with day-to-day administration delegated to a named role within clinical informatics, IT governance, or the CMO's office depending on organizational structure. What matters more than the specific title is that the owner has clear authority to query department heads and vendors, an established process for surfacing new additions, and organizational backing to escalate when high-risk tools are found without governance documentation.

What's the minimum viable inventory for a smaller hospital?

For a smaller organization just starting, focus first on clinical AI tools in active use: tools that influence patient care decisions in any way. For each, capture what the tool does, which vendor supplies it or whether it was built internally, whether it has been formally reviewed and approved, who in the organization is accountable for it, and whether it uses patient data. That five-field foundation gives you a starting point you can build from without requiring a large governance infrastructure to launch.

How does an AI tool inventory relate to the Joint Commission and CHAI guidance?

The Joint Commission and CHAI guidance on the responsible use of AI in healthcare (September 2025) is voluntary. It recommends governance structures, local validation, and ongoing monitoring for the AI an organization uses, and none of those is possible without knowing which tools are running. On June 1, 2026 the Joint Commission also began offering a voluntary certification based on that guidance, and organizations don't need to be accredited by the Joint Commission to apply. The CHAI governance playbooks released in May 2026 recommend logging every AI system in a centralized inventory. An organization that can produce a current inventory, with a documented authorization decision for each tool, is in a much stronger position than one that has never catalogued its AI.

Sources

American Medical Association. Governance for Augmented Intelligence: Establish a Governance Framework to Implement, Manage, and Scale AI Solutions. AMA STEPS Forward toolkit, developed in collaboration with Manatt Health. 2025, p. 8. AMA overview of the toolkit.
U.S. Department of Health and Human Services. HHS Artificial Intelligence Use Cases Inventory, FY25. Content last reviewed July 16, 2026.
Coalition for Health AI. AI Governance Playbooks, Subdomain 4.1: Lifecycle Management, and Subdomain 4.4: Third Party Management. Released May 27, 2026.
The Joint Commission and the Coalition for Health AI. The Responsible Use of AI in Healthcare (RUAIH). September 17, 2025. (PDF)
The Joint Commission. Voluntary Responsible Use of AI in Healthcare Certification, news release. June 1, 2026.
ASTP/ONC. Health Data, Technology, and Interoperability: Certification Program Updates, Algorithm Transparency, and Information Sharing (HTI-1) Final Rule. 89 FR 1192, January 9, 2024. 45 CFR 170.315(b)(11).
ASTP/ONC. Health Data, Technology, and Interoperability: ASTP/ONC Deregulatory Actions To Unleash Prosperity (HTI-5), Proposed Rule. 90 FR 60970, December 29, 2025.
"The New AI Operating Model for Healthcare," panel, AI in Healthcare Forum, HIMSS26, Las Vegas, March 9, 2026. Authors' notes from attendance.

How Do We Create and Maintain an AI Tool Inventory for Our Health System?