Darkness is fading into dawn across the Pacific, and—at least to all appearances—nothing seems out of the ordinary. A powerful adversary is conducting exercises up and down its coast, and at outlying islands in disputed territory—but that’s something it does routinely.
Except that in many locations, the adversary’s planes and ships are heading in somewhat different directions than usual, and in different numbers and formations. Taken one area or exercise at a time, these variations seem innocuous. It is only when all the pieces are seen as a whole that an alarming picture emerges: the adversary may be about to launch a full-scale surprise attack.
High above the Pacific, a network of DoD surveillance satellites, each equipped with “multi-modal” AI, is building this overall picture in real time, and is alerting the joint forces. And, this multi-modal AI—which combines generative AI and traditional machine learning—is factoring in additional pieces of information that, also when seen separately, may not arouse suspicion. These include intelligence reports that the adversary is repositioning its own satellites over the Pacific, stepping up cyberattacks on U.S. military and civilian ports, and beginning new disinformation campaigns aimed at our allies and partners.
The joint forces may soon have the ability to achieve this level of surveillance and intelligence.
Multi-modal AI on surveillance satellites could not only help provide early warning of an attack, it could give commanders greater situational awareness throughout an actual conflict—and predict what an adversary might do next. The AI might be coming to the same conclusions that human analysts would—but it would be doing it faster, and in situations where a moment lost is a battle lost.
Surveillance Satellites With AI
Commanders can gain a comprehensive view of the Pacific with a mesh network of hundreds or even thou- sands of small surveillance satellites in low-earth orbit. SpaceX, which now has about 5,000 Starlink communications satellites in low-earth orbit, has shown that it is both technologically and economically feasible to put large numbers of small satellites into space.
With a mesh network of small satellites, the entire picture of the Pacific—or any other area—can be seen as a whole. And the network is largely protected against jamming and other attacks on the satellites. While an adversary might be able to disable a handful of the small satellites, others in the network would quickly take over their roles, maintaining the continuous, full view of the Pacific. In this sense, the mesh network is self-healing.
Multi-modal AI on each satellite, networked together, would have the ability to integrate widely disparate surveillance and intelligence information—from any number of sources—that might otherwise be stove-piped. And it could quickly make sense what that data, seen in its entirety, might mean.
A key way the AI makes sense of the data is by taking advantage of modeling and simulation. For example, computers might run tens of thou- sands or even millions of scenarios of how a conflict in the Pacific could begin. If what’s currently happening in the Pacific matches one of those scenarios—such as how the adversary is positioning its ships, planes and satellites—the AI would look at how that scenario played out in modeling and simulation. It might find, for example, that in that same situation the adversary launched an attack 80 percent of the time. The AI would then conclude that there is an 80 percent chance that what the adversary is doing now in the Pacific is actually a prelude to an attack.
Asking Questions of Surveillance Data
Rapid advances in multi-modal AI are improving its ability to help commanders understand and work
with the predictions. In the near future the AI might, for example, explain to commanders—in plain language— why the adversary’s current actions suggest a probable surprise attack. The AI could continually update its predictions in real time, by taking new surveillance data as it comes in, and running it through new modeling and simulation.
At the same time, commanders could interact with the multi-modal AI, asking questions and exploring new scenarios. For example, a commander might ask the AI what would likely happen if the joint forces responded to an adversary’s actions in certain ways, and how the adversary might react.
With conventional AI, such complex questions might require the help of a data scientist, making it time- consuming and cumbersome for commanders to get the answers they need. However, multi-modal AI is gaining the ability to directly run those questions through modeling and simulation, and then to interpret the results for commanders. Before or during a conflict, commanders across the Pacific could continually query the multi-modal AI as conditions rapidly change.
Multi-modal AI on surveillance satellites could also use its data to help create digital twins of operating areas, providing commanders with continuously updated 3-D maps showing the real-time status of adversary and allied forces. As commanders pose questions—what might happen, for example, if they moved a carrier strike group to a certain location—they could use the digital twins to visualize the AI’s predictions of likely outcomes.
Greatly reduced launch costs and other areas of savings are now making a mesh network of small satellites cost-effective. At the same time, the multi-modal AI that would be on these satellites is rapidly gaining in sophistication. Together, these technologies have the potential to provide the joint force with greater situational awareness across the vast expanses of the Pacific, and to help predict an adversary’s next moves.
Michelle Harper ([email protected]), a solutions architect at Booz Allen, is an expert in AI who has worked with intelligence agencies to develop advanced surveillance capabilities. She leads a team that provides the military with software solutions for enterprise integration and other capabilities.
Collin Par An ([email protected]) is an AI architect at Booz Allen who builds large language models for a variety of applications for the Space Force, Navy, Army and Air Force.
Booz Allen subject-matter expert Kevin Koerner contributed to this article.