2025 was the year AI stopped being an interesting experiment and became an operational reality for both attackers and defenders. Adversaries used generative models to scale social engineering, fabricate identities, and automate parts of intrusion campaigns, while defensive teams brought agentic AI into detection and response workflows to gain speed and scale. That twin lesson is simple and uncomfortable: the same capabilities that speed incident response also widen the attack surface if not governed and hardened from the start.
Adversary tradecraft moved rapidly from proof of concept to production. Multiple threat intelligence vendors documented groups using generative AI to produce highly convincing phishing and vishing content, to create synthetic resumes and deepfaked interviews for insider recruitment, and to help less-skilled operators assemble functional malware and scripts. AI-augmented identity attacks and compromises of agent platforms became recurring themes in the field. These developments changed timelines and priorities for defenders: containment must now assume an adversary that can iterate and adapt at machine speed.
Defenders responded in kind but with a different emphasis. AI-driven detection, orchestration, and automated containment tools reduced human workload and compressed mean time to detect and respond. Security vendors and large operators deployed AI assistants and automated agents to triage alerts, prioritize actions, and execute containment steps when confidence thresholds were met. Those tools materially shortened response windows, but they also introduced new trust and verification challenges around agent identity, provenance of training data, and exposure of sensitive telemetry to third-party models.
Policy and operational guidance matured alongside technology. U.S. defense components released tailoring guidance that integrates cybersecurity risk management into the AI lifecycle, emphasizing early integration of cybersecurity into acquisition, testing, and authorization. At the same time, civilian agencies and cross-sector collaboratives published playbooks and frameworks for sharing AI-specific incident information and for coordinating defensive responses across public and private partners. Industry players and AI providers deepened engagement with national security stakeholders to align expectations around secure deployments. Those moves helped create a baseline for defensible AI adoption in sensitive contexts, but they do not remove operational risk overnight.
Where the defense community must pay attention now is in three practical risk domains.
1) Agent and model identity. AI agents function like privileged non-human identities. They need lifecycle management, credential hygiene, and least-privilege access, just like a service account that can access critical systems. Treat agents as first-class identities: provision them with short lived credentials, instrument them with strong telemetry, and require attestation of their supply chain and model provenance before granting sensitive privileges. Failure to do this turns an agent into an incubator for lateral movement and data exfiltration.
2) Data and model integrity. Data poisoning, prompt injection, and model manipulation are not theoretical risks. Model inputs must be validated and segregated, and sensitive training or operational data should never be exposed to externally hosted models without contractual and technical protections. Defense programs should apply data classification, strict logging, and sandboxed validation pipelines prior to deploying models in operational enclaves. The DoD tailoring guidance and related government playbooks underscore the need for these controls across the AI lifecycle.
3) Human governance and red teaming. AI accelerates both attack and defense. That means governance must accelerate too. Human-in-the-loop constraints, layered approval processes for agent actions, and routine adversarial testing of models and agents must be standard practice. Red teams must be funded and authorized to simulate AI-enabled attack chains that blend social engineering, agent compromise, and cloud identity abuse. Blue teams must exercise automated responses to ensure playbooks do not create destructive steering loops or cascade failures.
Operational recommendations for defense cyber teams
-
Inventory AI assets and agents. Know every model, every agent, and every third-party API that touches mission data. Maintain an authoritative inventory tied to access control and approval workflows.
-
Apply identity-first controls to agents. Use short-lived credentials, hardware-backed keys where possible, and fine-grained authorization for models that call production services.
-
Harden supply chain and vendor contracts. Require provenance attestations, secure development lifecycle evidence, and incident reporting SLAs from AI vendors supporting defense workloads.
-
Adopt model validation and canary deployment. Validate outputs for safety and integrity in a controlled environment before wide release. Use canaries and progressive rollouts with strict observability.
-
Invest in AI-aware telemetry. Log agent actions with context, correlate model inputs with outcomes, and build detections tailored for model manipulation and prompt abuse.
-
Institutionalize adversarial testing. Schedule regular red-team exercises that mimic AI-augmented adversaries and validate that automated playbooks behave as intended.
-
Share actionable intelligence across partnerships. Use existing public-private channels and the emergent AI playbooks to share indicators, tactics, and mitigation guidance rapidly and responsibly. Coordination reduces duplication and prevents avoidable compromise.
A closing caution: technological fixes alone will not solve the problem. 2025 demonstrated that AI can amplify both human ingenuity and human error. For defense organizations that means adopting AI with humility and discipline. Invest in basic cyber hygiene, identity and supply chain controls, and adversarial testing before you expand agent capabilities into mission-critical functions. If you do not, you will inherit a fast, scalable, and opaque attack surface built on the very tools meant to give you advantage.
If there is one practical takeaway from 2025 it is this: accelerate adoption, but build security in at every stage. The cost of a missed control is no longer measured in hours. It is measured in operational reach and strategic exposure.