In the movie Avengers: Infinity War (2018), superheroes fight to save the universe from the villain Thanos. As they struggle to find a way to defeat Thanos, Dr. Strange speaks of his time-traveling: “I went forward in time. To view alternate futures. To see all the possible outcomes of the coming conflict.” He glimpses 14,000,605 futures and discovers the one path to victory.
Dr. Strange’s ability to look into the future is no longer just fiction. AI-embedded modeling simulations can do a similar job: While they cannot see the future, they can predict it. Through AI-generated battlefield simulations, the Navy can fight an adversary force more than a million times to find a key to winning the battle. With a vast database of diverse scenarios, the Navy can be poised to counter any attempt to replace the traditional rules-based order at sea.
Computer-based Simulation/Wargames
Traditional computer-based simulation and wargames have a long history in the Department of Defense (DoD). DoD adopted Janus, a conflict simulation model, in the 1980s. Janus was an operational planning tool for Operation Just Cause (the invasion of Panama) in 1989 and Operation Desert Storm a year later.1 After witnessing its impact, DoD expanded its adoption of computer-based simulation, which today includes Lockheed Martin’s Warfighters’ Simulation (WarSim), the Army’s One Semi-Automated Forces (OneSAF), the Marine Corps’ MAGTF Tactical Warfare Simulation (MTWS), and the Air Force’s Advanced Framework for Simulation, Integration, and Modeling (AFSIM).
Nevertheless, modern simulations have limitations. They rely heavily on preestablished scenarios and require extensive human heuristic decision-making and involvement, which constrain attempts to simulate numerous vignettes and scenarios. Advances in AI and significant increases in computing power offer opportunities to mitigate these constraints.
What distinguishes AI-embedded simulation from traditional computer-based simulation is its ability to simulate millions of battles in a short period. Through millions of self-plays, it can generate vignettes autonomously, produce numerous courses of action for given scenarios, and offer decision-makers multiple options. It also can evaluate or generate optimal actions for opposing forces and devise countermeasures to defeat them.
Advances in Simulation and AI and implications for the Navy
AI nonplayer characters. Over the past few decades, the resolution of gaming graphics has improved significantly, allowing users to become fully immersed within a few seconds of gameplay. Another component that makes games fun is nonplayer characters (NPCs), which are “characters within a computer game that are controlled by the computer, rather than the player.”2 Even when gamers are playing alone, these virtual characters make it feel as if they are playing against real people.
But in some games, NPCs do not act very human. They do the same things in the same situations, over and over. These repetitive actions make it easy for players to guess what will happen next, and they might quickly find the game boring.
There are several ways to make NPCs act more like humans. In the past, developers used simple rule-based behavior algorithms. With the advances in neural networks, however, NPCs have become more dynamic and more adaptive to their opponents’ actions. In 2005, three computer scientists at the University of Texas at Austin demonstrated that NPCs embedded with neural networks could be trained in real-time as the user plays.3 This challenges the player to compete against a more human-like, intelligent opponent. One of the most impressive achievements in NPC development was in DeepMind’s work with Atari games. By employing a deep neural network combined with reinforcement learning (a sector of machine learning), the NPC surpassed human performance after just 2,600 iterative self-plays.4 If NPCs embedded with neural networks were applied to military training, they could help train individuals in complex tasks.
Integrating NPCs in military applications opens an innovative avenue for enhancing combat training and operational strategies across diverse naval domains—space, air, surface, undersea, and cyber. For example, by designing NPCs to carry out specific tasks such as island defense, antisurface warfare, and underwater operations on a detailed map at varying levels of difficulty (expert, normal, novice, etc.), ships could prepare for a range of critical scenarios and identify the most effective strategies for accomplishing their objectives by engaging in virtual battles against NPCs set to the appropriate skill levels.
Research is underway to develop these specialized NPCs, with notable efforts coming from several institutions related to the military. The Naval Postgraduate School is researching cognitive AI for NPCs that deal with military features such as hierarchy, fog of war, or specific scenarios within the ATLATL platform. At the University of Southern California’s Institute of Creative Technology, researchers are working on adaptive NPCs within the Rapid Integration & Development Environment, specifically tailored for military training purposes.5 The research at these institutions has potential applications beyond its initial scope, including strategic decision-making processes such as identifying the optimal course of action. In addition, it could pave the way for the development of unmanned vehicles capable of autonomous thinking and decision-making, which could be a game changer on the future battlefield.
Generative AI for simulation/wargames. Generative AI (GenAI) specializes in creating new content that resembles what it learns from. This capability has led to its widespread application across various industries and has become a part of our daily lives, like ChatGPT.
The primary advantage of GenAI is its capability to address the problem of limited operational experience or training data.6 From this perspective, one notable application is in generating vignettes. Given basic parameters such as the enemy navy’s estimated size and ship types and quantities, GenAI can generate numerous realistic scenarios. This enables operational planners to explore and experiment with various vignettes that might be beyond human thinking yet plausible.
GenAI such as COA-GPT, developed by Vinicius Goecks and Nicholas Waytowich, also can suggest courses of action by interacting with command-and-control personnel to support decision-making (see Figure 1 on page 64). The Navy could use this method to generate courses of action from simple scenarios, such as finding the most efficient formation or positioning in the open sea, to complicated scenarios, such as combat situations near coastal regions and islands.
Another aspect of GenAI implementation is scenario generation. From historical experiences and known enemy tactics, objectives, and missions, GenAI can generate synthetic enemy behaviors tailored to the vignettes created. This capability would enable the Navy to prepare for unexpected situations by simulating various potential adversary actions, thereby increasing its readiness for dynamic threats at sea.
Digital twin. A digital twin translates reality into digital form. The current vanguard within the industrial sector is Industry 4.0, which integrates technologies such as cloud computing, the Internet of Things, AI, and digital twins to gather and analyze data produced during the manufacturing process to enhance decision-making. Picture a smart factory in which every machine on the production line has its own sensors, constantly collecting data and sharing it with the whole system. No matter what kind of data it is, AI analyzes it and comes up with ways to make the manufacturing process smoother and more efficient.8
Introduced by NASA in 2010, a digital twin is “an integrated multi-physics, multi-scale, probabilistic simulation of a vehicle or system that uses the best available physical models, sensor updates, fleet history, etc., to mirror the life of its flying twin.”9 The advantage of a digital twin is being able not only to visualize the entire system during the design phase, but also to predict problems, optimize solutions, accelerate prototyping, and facilitate training prior to real-world implementation.10 Core components such as simulation and AI, machine learning (enabling systems to learn from experience without explicit programming), and reinforcement learning (allowing systems to achieve the most rewarding outcomes) are essential to facilitate predictive visibility into future outcomes.11
Digital twins allow decision-makers to see the outcomes of their decisions and revise their choices. They can predict desirable, circumvent predictable undesirable, and lessen the impact of unpredicted undesirable behaviors.12 This dual-layered approach extends beyond prediction and life-cycle management of equipment on a ship; it significantly enhances strategic planning and real-time decision-making in various operational scenarios. By using comprehensive data analysis and simulation, digital twins foster a deeper understanding of maintenance systems, enabling the fleet to optimize ship performance, increase reliability, and fight with greater confidence. Furthermore, digital twins play a crucial role in risk management by providing a safe environment in which to test hypotheses and assess potential interventions without having to physically alter the actual system. Consequently, digital twins are not just advantageous but essential for maintaining competitiveness and achieving operational excellence in today’s rapidly challenging operational maritime situation.
Possible Platforms
Creating a virtualized battlefield requires a significant investment of capital and effort. However, there are some promising platforms that could be adapted for military use. The gaming industry offers numerous military-themed games, some of which feature realistic data inputs or allow users to modify inputs to meet specific requirements. While these game platforms may not currently be equipped for AI and machine learning (ML) applications, they hold potential as a foundation for creating an AI/ML environment. By harnessing the advanced simulation capabilities such as the physics engine or terrain generator functions of these platforms, it is possible to develop sophisticated training and strategic planning tools for military applications without starting from scratch. This approach not only saves resources but also accelerates the development and deployment of advanced virtual battlefield technologies. Examples of such games and simulation platforms include:
Command: Modern Operations (published by Slitherine Ltd): This game offers a multidomain simulation of modern warfare, providing detailed scale military operations across land, sea, and air. Its strength is its intricate scenario editor, which allows users to craft specific operational scenarios. By integrating ML algorithms, the military could enhance the predictive capabilities of these scenarios, improving decision-making and strategic planning in virtual exercises.
Modern Naval Warfare (published by Slitherine Ltd): This platform focuses on high-fidelity simulations of naval operations, including submarine warfare, surface ship engagements, and antiair defense. By adapting it for ML use, the Navy could develop algorithms to simulate and analyze naval strategies, offering unprecedented training opportunities and insights into naval combat tactics and strategy optimization.
Modern Air Combat Environment (developed by BSI): MACE is a highly detailed simulation tool for air combat scenarios, offering realistic models of aircraft, missile systems, and radar tracking. Its ability to simulate complex air engagements makes it an excellent candidate for ML adaptation; algorithms could analyze engagements to offer insights into tactics and strategies, potentially revolutionizing air combat training and planning.
VR Forces (developed by MAK Technologies): VR Forces creates detailed virtual environments for land, air, and sea operations. Its strength is its ability to simulate large-scale military maneuvers and operations. Integrating ML capabilities could enable the platform to provide real-time tactical adjustments and predictions, enhancing the realism and effectiveness of training exercises.
Time to Evolve
The 2022 National Security Strategy notes that “the post-Cold War era is definitively over and a competition is underway between the major powers to shape what comes next.”13 This implies a significant shift from a relatively predictable to an unpredictable battlefield environment. Thus, there is an urgent need to develop and implement cutting-edge technologies to manage and reduce uncertainty effectively. Like Dr. Strange, Navy decision-makers, with the aid of AI, will be able to traverse countless scenarios to find tactics and strategies that could ensure victory on an uncertain battlefield.
Dr. Strange’s visionary superpower is embodied in AI-embedded simulation and wargaming. The journey ahead may present challenges, but with a decisive and careful commitment to innovation, the Navy can elevate itself to the next level.
1. Science and Technology, Lawrence Livermore National Laboratory, “Janus, A Conflict Simulation Model,” st.llnl.gov.
2. Henrik Warpefelt, The Non-Player Character: Exploring the Believability of NPC Presentation and Behavior (Stockholm: Department of Computer and Systems Sciences, Stockholm University, 2016).
3. Kenneth O. Stanley, Bobby D. Bryant, and Risto Miikkulainen, “Evolving Neural Network Agents in the NERO Video Game,” IEEE 2005 Symposium on Computational Intelligence and Games, Essex University, Colchester, Essex, UK, 4–6 April 2005.
4. Volodymyr Mnih et al., Playing Atari with Deep Reinforcement Learning, NIPS Deep Learning Workshop, Cornell University, ArXiv, 19 December 2013.
5. Arno Hartholt et al., “Rapid Prototyping for Simulation and Training with the Rapid Integration & Development Environment (RIDE),” Proceedings of the 2021 Interservice/Industry Training, Simulation, and Education Conference.
6. Kunfeng Wang et al., “Generative Adversarial Networks: Introduction and Outlook,” IEEE/CAA Journal of Automatica Sinica 4, no. 4 (October 2017): 588–98.
7. Vinicius G. Goecks and Nicholas Waytowich, “COA-GPT: Generative Pre-trained Transformers for Accelerated Course of Action Development in Military Operations,” ArXiv, 1 February 2024.
8. Blaž Rodic, “Industry 4.0 and the New Simulation Modeling Paradigm,” Organizacija 50, no. 3 (August 2017): 193–207.
9. Mike Shafto et al., Draft Modeling, Simulation, Information Technology & Processing Roadmap: Technology Area (National Aeronautics and Space Administration, November 2010), TA11-7.
10. Maulshree Singh et al., “Digital Twin: Origin to Future,” Applied System Innovation 4, no. 2 (May 2021): 36.
11. Bahar Biller and Stephan Biller, “Implementing Digital Twins That Learn: AI and Simulation Are at the Core,” Machines 11, no. 4 (March 2023).
12. Michael Grieves and John Vickers, “Digital Twin: Mitigating Unpredictable, Undesirable Emergent Behavior in Complex Systems,” in Transdisciplinary Perspectives on Complex Systems, Franz-Josef Kahlen, Shannon Flumerfelt, and Anabela Alves, eds. (Switzerland: Springer International Publishing, 2017), 85–113.
13. The White House, National Security Strategy (Washington, DC: The White House, October 2022).
