ARIA Funding Call for Safeguarded AI (United Kingdom)

Safeguarded AI’s success will depend on showing that the gatekeeper AI actually works in a safety-critical domain. The research teams selected for TA3 will work with other programme teams, global AI experts, academics, and entrepreneurs, in setting the groundwork to deploy Safeguarded AI in one or more areas.

In this first phase of funding, ARIA will allocate an initial £5.4M aimed at eliciting requirements, sourcing datasets, and establishing evaluation benchmarks for relevant cyber-physical domains. A second phase (£8.4m) will launch in approximately 1 year.

About the Programme

• The Safeguarded AI programme is a £59m-backed R&D effort to build out a general-purpose AI workflow for producing domain-specific AI agents or decision-support tools for managing cyber-physical systems with quantitative guarantees which improve upon both performance and robustness compared to existing operations.
• To do this, they will employ a system that includes both state of the art, “frontier AI”, as well as human expertise to construct a gatekeeper system which monitors and ensures safe behaviour of other AI agents. This gatekeeper will consist of a formal world model and safety specifications about the application domain, and several ML components responsible for proposing effective task policies and generating verifiable safety guarantees, among others. The resulting Safeguarded AI system will unlock the raw potential of state of the art machine learning models in a wide array of business-critical or safety-critical applications domains where reliability is key.
• The programme will develop the toolkit for building such a Safeguarded AI workflow, and demonstrate it in a range of applications domains such as energy, transport, telecommunication, healthcare, and more. This would, first, act as a proof of concept, proving that it’s possible to realise the benefits of AI in safety critical applications through quantitative safety guarantees; and second, catalyse further R&D to replicate and scale the results across application areas in the world.
• The Safeguarded AI programme is divided into three main Technical Areas (TAs). This call for proposals is in Technical Area 3 (TA3), where they will fund teams to develop and prototype domain-specific applications of the Safeguarded AI workflow.

Objectives

• The programme’s Technical Area 3 (TA3) aims to challenge the claim that “guaranteed safe AI will provide no additional economic value beyond mainstream AI.”
• TA3 is seeking two types of Creators (i.e. individuals and teams that ARIA will fund and support). Throughout the rest of this document, they will refer to them as Track 1 and Track 2.
  • Track 1 Creators are product developers, i.e. individuals, entrepreneurs and existing organisations (including startups, SMEs, large companies, and nonprofit R&D organisations) interested in leveraging near-future Safeguarded AI technology to develop products tailored to the needs of customers/end users in a variety of specific application domains (see section 4 for more details). As part of track 1, they are also open to funding operators who are looking to build a small team to do this R&D work in-house.
  • Track 2 Creators are customers/end users, i.e. organisations who already want to participate in the co-creation of solutions—not by building an in-house solution R&D team, but by actively participating in the customer discovery processes run by Track 1 Creators whom they may fund in the same sector/domain.
• TA3 will include two phases. In Phase 1 of TA3 (this solicitation), they will cast a wide net, funding a number of “pilot” efforts to deeply understand customer needs, elicit requirements, begin to source datasets and/or simulators, design evaluation suites to validate the performance of predictive models and autonomous and semi-autonomous controllers, etc. in the chosen application areas.
• Later, in Phase 2, TA3 Creators will be able to use the tooling prototypes made available by Creators in TA1 and TA2, in order to accelerate and scale up the creation of models and specifications.

Funding Information

• Programme structure
  • TA3 Phase 1 will be funded with £5.4m (inclusive of VAT where applicable) distributed across approximately 10 teams. These could be part-time efforts depending on the details of the proposal.
  • Following on from Phase 1, Phase 2 will likely consist of further £8.2 million of funding distributed between 2-4 teams full-time, each pursuing a different application area using the domain-general tools developed in TAs 1 and 2.
• Programme duration
  • They expect to fund TA3 Phase 1 projects through the end of March 2026. Application proposals should not exceed this timeframe. They intend to make decisions about Phase 2 in early 2026, based on the outputs from the initial phase. The second phase will focus on building a proof of concept followed by a deployable product. Phase 1 Creators will be asked to submit proposals for Phase 2 funding, and the solicitation will also be open to new applicants to submit their proposals.

Eligibility Criteria

• They welcome applications from across the R&D ecosystem, including individual entrepreneurs, startups, and established companies.
• The primary focus is on funding those who are based in the UK. For the vast majority of applicants, they therefore require the majority of the project work to be conducted in the UK (i.e. >50% of project costs and personnel time).
• However, they can award funding to applicants whose projects will primarily take place outside of the UK, if they believe it can boost the net impact of a programme.
• In these instances, you must outline any proposed plans or commitments in the UK that will contribute to the programme within the project’s duration (note the maximum project duration is 15 months). If you are selected for an award subject to negotiation, these plans will form part of those negotiations and any resultant contract/grant.

For more information, visit Innovate UK.