Key Findings
- Generative AI is assessed as likely to affect all levels of intelligence work, with the earliest impact expected in all-source analysis because of its ability to process large volumes of data.
- The report argues that current generative AI tools can speed and enhance tasks across the analytic workflow, including data discovery, product conceptualization, tradecraft checks, and classification review.
- Workshop participants agreed that current generative AI should not replace human analysts because expert intuition, contextual judgment, and explanation of implications remain beyond present system capabilities.
- The report identifies hallucinations, limited reasoning, bias amplification, and opaque model outputs as major risks that require human oversight, technical safeguards, and standards for explainability.
- Recommended actions urge U.S. and Australian analytic managers to plan for continuous model improvement, automate suitable workflow elements, build human-machine analytic teams, and prepare the workforce for AI adoption.
Overview
Conducted through a collaboration between the Special Competitive Studies Project and the Australian Strategic Policy Institute, this report examines the potential of large language model powered generative AI to enhance all-source intelligence analysis in the U.S. and Australian intelligence communities. Its foundation is a series of workshops held in Canberra and Washington, supplemented by literature review and expert consultations. The report argues that generative AI could first reshape the all-source analytic mission because it can process large bodies of unstructured information and help produce insights at speed. It reviews possible uses across the analytic workflow, including triaging information, organizing research, conceptualizing products, supporting drafts, checking sourcing and tradecraft, and engaging customers. The report also emphasizes limits of current models, including hallucinations, weak reasoning, bias amplification, and opaque outputs that challenge trust, accountability, and compliance with analytic standards. Its recommendations urge leaders to design for continuous model improvement, begin automating suitable workflow elements, build human-machine analytic teams, create workforce training and incentives, and collaborate on shared U.S.-Australian standards and evaluation metrics. The policy relevance centers on safe adoption, allied interoperability, analytic quality, and preparing intelligence organizations for more capable AI systems.
The promise of “augmented intelligence” – seamlessly combining human knowledge and creativity with machine scale and precision to create a system greater than the sum of its parts.
SCSP
Recommendations
- Design for Continuous AI Model Improvements. The ICs should adapt their technology acquisition processes – and budgets – to enable for multiple annual updates.
- Start Automating Portions of the Analytic Workflow Now. Analytic managers should select a few areas for initial focus and insist that units begin utilizing AI to realize early gains and to build their…
- Build Human-Machine Analytic Teams. The ICs would do well to study human analysts’ needs and mission priorities in future AI design and integration.
- Create AI-Ready Training and Incentive Structures for the Analytic Workforce. The ICs will need to invest more in digital acumen, both through the recruitment of highly-trained talent and the upskilling of the existing…
From the report
Documents
Publication
Published by the Special Competitive Studies Project. Co-publishers and external authors, where they exist, are credited in the byline above.