How Governments Use AI: Cybersecurity Procurement

Cybersecurity, alongside medical diagnostics, appears to be a leading candidate for AI use in the public sector. Based on analysis of over 6,000 contract and tender documents, there's a clear pattern: AI capabilities are now a common requirement within cybersecurity infrastructure rather than as standalone novelties.

Our research shows some very clear shift away from relying on the feeds from researchers that identified threats, to proactive, automated monitoring systems that detect, triage, and respond to threats automatically.

1. The Shift to Automated Response

The old process of handling threats was for software to flag and isolate malicious code or attacks. Modern systems are increasingly capable of autonomous responses and the data shows that buyers value these automatic interventions.

• The Irish Courts Service has tendered for a detection and response solution that uses unsupervised and supervised machine learning. The requirement explicitly states the system must be able to "[automatically] shut down an attack while allowing normal behaviour to continue uninterrupted".
• The UK's Pension Protection Fund has procured Darktrace Network Detection and Response software, specifically requiring the use of AI to detect unpredictable cyber threats and provide proactive defense.
• Peterborough City Council procured an Extended Detection and Response (XDR) application that uses behavioural analytics and machine learning to profile network behaviour and detect anomalies indicative of attacks.

2. Automating Intelligence

Agencies responsible for sector-wide security are procuring data feeds on threats that have been compiled automatically, a significant change from the past where threat lists were hand curated by researchers.

• In the Netherlands, Stichting ZCERT (the cybersecurity centre for the healthcare sector) noted that collecting, analysing, and distributing threat intelligence manually to its healthcare entities is "not scalable". They are procuring structured, machine-readable Cyber Threat Intelligence (CTI) feeds to automate the process.
• Similarly, Norway's Directorate for Public Management and Financial Control (DFØ) is establishing a central cloud-based framework to provide automated cyber threat intelligence to the broader Norwegian public sector.

3. Securing Artificial Intelligence Models

As governments begin to adopt Large Language Models (LLMs) and other AI tools, a new procurement category is emerging focused on testing the vulnerabilities of the algorithms themselves.

• The UK Department for Science, Innovation & Technology (DSIT) is running procurements to build "Cyber Range Scenarios". They are commissioning suppliers to design realistic computer networks to assess a "scaffolded LLM model" at cyber-relevant tasks and test how an AI agent might conduct an offensive cyberattack on Critical National Infrastructure.
• In Germany, research funding has been allocated for "Robust and Secure Machine Learning" (RSML). This project focuses on the research and development of approaches to increase the robustness and security of neural AI models, ensuring they can be safely applied to operational defense and security systems.

In each case governments are clearly adapting to the innovations that the cybersecurity industry has developed to increase the effectiveness of their tools. The purchase of algorithmic vulnerability testing is particularly interesting, however, whilst all of this feels very "bold, new world" there's a question about the vulnerabilities that must exist within these AI tools themselves. We'll be writing on that soon.