OMB Requirements for AI Acquisition Will Impact Government Contractors

October 4, 2024

WHAT: On September 24, 2024, the Office of Management and Budget (OMB) issued Memorandum M-24-18, which prescribes requirements and guidance on the responsible acquisition of artificial intelligence (AI) in the federal government. M-24-18 focuses generally on agency use of AI, including but not limited to generative AI and what it classifies as “rights-impacting” or “safety-impacting” uses. The guidance centers on three main principles: (1) managing AI risk and performance; (2) establishing cross-functional and interagency collaboration to reflect new AI responsibilities; and (3) promoting a competitive AI market through innovative acquisition methods.

WHAT IT MEANS FOR INDUSTRY: Although the memorandum is directed to federal agencies, M-24-18 will have direct impacts on government contractors, who will start seeing some of the flow-down requirements in provisions in their AI-related contracts. This guidance is an “initial means” for regulating AI acquisitions and will likely be a building block for future regulations, such as updates to the Federal Acquisition Regulation (FAR), more directly applicable to contractors.

As detailed below, M-24-18 requires federal agencies to impose fairly prescriptive requirements on acquisitions of AI – including requirements related to incident reporting, data management, transparency, and testing, as well as heightened requirements for AI-based biometrics and generative AI. An implementation of these new requirements will be on an expedited timeline, with key deadlines for contracts for certain high-risk AI applications in November and December 2024, and deadlines for other contracts for AI in March 2025. Thus, even though updates to the FAR may be years away, contractors will start to see terms prescribed by M-24-18 in solicitations and contracts with greater frequency in the coming months.

OMB’s latest memorandum is part of the White House’s broader efforts to advance safe, secure, and trustworthy AI. M-24-18 is a deliverable under President Biden’s October 2023 AI Executive Order, and it builds upon OMB Memorandum M-24-10, Advancing Governance, Innovation, and Risk Management for Agency Use of Artificial Intelligence.

The M-24-18 memorandum applies not just to Executive departments and agencies, but also to independent regulatory agencies and government corporations, meaning that implementation will play out across a range of frameworks. The memorandum expressly excludes “AI used incidentally by a contractor during performance of a contract,” giving the example of a contractor’s use of AI “when not directed or required to fulfill requirements.” This limitation may cabin the scope of M-24-18’s requirements as contractors (like businesses in all industries) increasingly integrate AI tools into their operations, though it may take years to tell whether and how the memorandum’s requirements end up indirectly factoring into contractor-agency disputes that involve incidental AI use anyway.

M-24-18 sets forth a broad range of new and expanded requirements and expectations, on a very quick timeline:

• By November 1, 2024 (less than a month away), agencies must identify any contracts associated with agency use of “rights-impacting” or “safety-impacting” AI;
• By December 1, 2024, existing and new contracts for rights-impacting and safety-impacting AI must comply with both M-24-10 and M-24-18; and
• Starting March 23, 2025 (180 days from M-24-18’s issuance), M-24-18’s requirements will apply to any new contract awarded based on a solicitation issued after this date, as well as to any option to renew or extend an existing contract exercised after this date.

These new requirements and recommendations include the following, many of which involve concepts that may be familiar to contractors involved in other current areas of government focus such as cybersecurity (e.g., incident reporting) and competition (e.g., system interoperability):

Managing AI Risks and Performance. M-24-18 supplements the growing list of risk management requirements agencies are subject to – and ultimately must pass down to contractors. While many of the requirements imposed by M-24-18 are geared toward agency actions, many are also expected to directly (or indirectly) impact contractors. Below is a sampling of the requirements contractors can expect to see in their forthcoming AI-related contracts:

• Incident Reporting for “Rights-Impacting” and “Safety-Impacting” AI: Identify and disclose to agencies “serious AI incidents and malfunctions of the acquired AI system or service” within 72 hours (“or a timely manner based on the severity of the incident”) after the contractor reasonably believes the incident occurs, a requirement that may add yet another timeline to the reports contractors may be required to make after cyber and other incidents;
• Data Management: Address systems and procedures for data management (e.g., data collection, data analysis, data labeling, data storage, data filtration, data mining, data aggregation, data retention, and data use) and accountability frameworks identifying data handling access and responsibilities;
• Transparency: Submit performance metrics, including real-world performance data for specific subgroups and demographic groups, to identify any discriminatory outcomes;
• Testing: Provide government customer with sufficient access and time to conduct any necessary real-world testing;
• Heightened Requirements for AI-Based Biometrics Systems: Submit documentation, test results, and/or test data to facilitate independent validation of performance of AI-based biometric systems; and
• Heightened Requirements for Generative AI: Ensure that any audio, image, and video outputs of the AI systems that are not readily identifiable as AI-generated use some sort of mechanism, such as a watermark, to signify the AI-source of the media.

Contractors can also expect to negotiate with the government the scope of licensing rights for their AI systems and be required to define the process for monitoring the AI systems.

Ensuring Collaboration Across the Federal Government. Agencies are directed to work together through interagency councils and other concerted efforts to collaborate on how to address the range of AI risks and make strategic acquisition plans. The guidance highlights three areas for potential collaboration efforts:

• Prioritizing AI investments that best serve the agency’s goals;
• Developing the capacity to deploy any sought-after AI systems; and
• Promoting adoption of best practices identified in cross-functional interagency councils.

Contractors can expect to see the fruits of the interagency labors in the form of new contract provisions and performance requirements that flow from previous successful and unsuccessful attempts to acquire AI and issues discovered in procured AI systems. The goal is to avoid reinventing the wheel each time an agency faces a risk or issue associated with AI technology.

Promoting a Competitive AI Market. M-24-18 focuses on expanding the marketplace for AI products and services through efforts such as increasing interoperability and decreasing vendor lock in, lowering barriers of entry for small businesses and non-traditional contractors, and lowering costs. The memorandum states that robust competition can incentivize companies to enhance their security and safety capabilities, which in turn will make the AI systems and services less risky when deployed by the agencies. To meet this goal, M-24-18 encourages agencies to include contract terms committing contractors to increased knowledge transfers (to include open-source licenses to contractors’ AI models and underlying datasets) and more transparent pricing schemes.

Agencies are encouraged to utilize innovative acquisition practices when acquiring AI systems and products, such as:

• Performance-based acquisition techniques that allow agencies to evaluate contractors’ claims about their AI products and services prior to contract award and better monitor performance post-award;
• Addition of expert consultants to acquisition teams or the source selection evaluation board;
• Leverage knowledge-sharing platforms across the federal government that facilitate deeper market research;
• Require contractors to demonstrate technical capabilities through a “show, don’t tell” approach that utilizes oral presentations, trials, prototypes, and other proofs of concept;
• Limit the duration of contracts to ensure frequent reassessment of contractors’ systems and the agency’s needs;
• Utilize contracting vehicles that prioritize the technical capability of contractors instead of defaulting to lowest price technically acceptable solicitations; and
• Increase use of other transactions authorities and pilot programs instead of relying solely on FAR-based agreements.

***

M-24-18 is the latest in a long line of agency actions targeted at AI risk management. While M-24-18 does not immediately impose requirements on contractors, it does foretell that more is to come soon. Contractors should also keep a close eye on their contracts to see which new provisions have been added to address AI risks – and can get a head start now by comparing their existing contractual terms for AI software and systems with the types of terms prescribed by the memorandum. Wiley’s Government Contracts and Artificial Intelligence practices will continue to monitor developments in this area and will be prepared to counsel clients on any new AI laws, policies, and regulations that flow from M-24-18.