Use of accelerators is growing in line with the growth of AI. These accelerators are specialized chips that are neither CPUs nor GPUs – they specifically offload and speed up the precise workloads required for AI. Primary producers include Tenstorrent, Groq, Cerebras, Graphcore, and Google.
Neo-clouds are new but increasingly available specialized clouds distinct from the traditional class of hyperscalers such as Azure, Google Cloud and AWS. They are AI-first, often built with accelerators, and well-suited for customers requiring AI training, inference and model building services. Neo-clouds provide massive parallelism, low-latency edge compute, flexible deployment and cheaper or more predictable economics. Example neo-clouds include CoreWeave and Nebius.
Typical AI use of neo-clouds includes training large-scale AI models and running high-throughput AI inference. In the latter, for example, the customer will use the neo-cloud’s accelerator-optimized low latency hardware to ensure rapid response times for in-house developed chatbots.
But accelerators, and therefore neo-clouds, suffer from several security blind spots. Traditional cybersecurity tools have been developed over decades around CPU-centric operating systems. They haven’t kept pace with the emergence of accelerators, and they lack visibility into the accelerators’ high-speed video memory. Current cybersecurity cannot readily detect what is happening within neo-cloud hardware.
Consequently, if a neo-cloud is stealthily compromised by an attacker, neither it nor its customers are guaranteed to see the compromise. The result could be a severe but invisible supply chain threat to all customers, but especially those using the neo-cloud for AI development purposes. If it had been more successful, the recent Januscape malware could have been used in such a manner.
Startup firm Stealthium aims to tackle this threat. It cannot see into the accelerators in the neo cloud, but uses an agent housed within the customers’ infrastructure that is specially trained to detect the subtle hints coming from a compromised neo-cloud.
“Unfortunately, our understanding of the shared model of responsibility for security doesn’t apply here, and certainly security controls and observability aren’t applicable in this space,” comments Chris Hosking, GTM Advisor at Stealthium. “Our go-to thinking has always been that if you cannot see something happening, then nothing is happening.” This is seriously dangerous, especially with accelerators – in cybersecurity, absence of proof is never proof of absence.
“Organizations are increasingly uncomfortable because they lack meaningful security and observability controls, in real time, for that silicon accelerator layer,” he continues. “That’s the challenge that Stealthium exists to solve. We believe that AI needs to be trustworthy. Sovereign AI can only be sovereign if it’s secure and trustworthy. That’s the purpose of Stealthium. We’re a security and observability company for AI accelerated runtime.”
The technology used is not new; it’s just highly specialized. “We deploy an agent that searches the telemetry coming from the neo cloud, looking for hints of compromise.” It’s the hints rather than the technology that are dramatically different.
As an example, Januscape exploited a vulnerability in nested virtualization, enabling the attacker to offer, or sell an environment to a third party. Such an exploit in a neo cloud could lead to cross-tenant leakage, with the third party gaining insights and potential access into legitimate in-house chatbots. Stealthium will detect this by detecting subtle hints in neo cloud telemetry indicating this, or a different, type of attack.
Such supply chain attacks already occur, but incidence is likely to increase in the future. The prize is attractive to both financially motivated cybercriminals and information gathering or influence seeking nation states. “As an attacker who has compromised a neo-cloud node, I can get access to a customer’s AI weights,” explains Hosking. “I can poison and corrupt and change the way that the model operates without anyone noticing. So, for example, I could make it more sympathetic to the cause that I’m trying to promote. I could extort the customer or just use the shared environment to run crypto mining or be my new base of operations.”
Hypothetically, if a nation state were able to influence or change ChatGPT or Gemini, it would be able to influence entire nations. The stakes are very high in a threat vector that is very new with little established security. Stealthium is an early example of a new type of security company, one that seeks visibility into the operation of accelerators. In this instance, it does not look into the hardware concerned but gathers and analyzes the telemetry coming from the hardware, with a specialized and continuously updated agent trained to detect subtle hints of accelerator compromise.
Related: Over 400 NPM Packages Infected in ChainDrop Supply Chain Attack
Related: New GitHub, PyPI Policies Boost Supply Chain Security
Related: Trump Orders Defense Contractors to Map Software, Suppliers Across Supply Chains
Related: North Korean Hackers Target Open Source Developers in Supply Chain Attacks