Cybersecurity experts have found serious security holes in popular AI inference server frameworks used by big companies like Meta, Nvidia, and Microsoft, as well as open-source projects such as vLLM and SGLang. These vulnerabilities could allow hackers to run malicious code remotely, posing a big risk to enterprise AI systems.

A key issue is how developers copied code from one project to another. This practice often spread the same security flaws across multiple platforms. The main root of the problem is the unsafe use of ZeroMQ (ZMQ) for messaging and Python’s pickle for data serialization, which aren’t secure when used improperly.

How the Flaw Was Discovered and Spread

The security team from Oligo Security traced the vulnerabilities back to Meta’s Llama Stack. In that system, a function used ZeroMQ’s “recv-pyobj()” to receive data and then passed it directly to Python’s “pickle.loads()”. This process allowed anyone to send malicious data that could execute code on the server without proper authentication.

Since this pattern was easy to copy, it appeared in other frameworks too. Nvidia’s TensorRT-LLM, vLLM, SGLang, and the Modular Max Server all showed the same risky code. Sometimes, developers even added comments like “Adapted from vLLM,” showing how widespread the reuse was. Oligo calls this pattern “ShadowMQ,” highlighting how a hidden flaw travels from one project to another through copy-paste, rather than starting from scratch.

Because these frameworks are used in many AI systems, a single security gap can affect many downstream projects. This creates a systemic risk, where a single infected component can compromise an entire AI ecosystem.

The Impact on AI Infrastructure and What to Do

These inference servers are the backbone of many enterprise AI setups. They handle sensitive data like prompts, model weights, and customer information. Oligo found thousands of exposed ZeroMQ sockets on the internet, some connected to these vulnerable servers. If exploited, hackers could run malicious code on GPU clusters, take control of systems, steal data, or even install crypto-mining malware.

Some of these frameworks are used by major companies like xAI, AMD, Nvidia, Intel, LinkedIn, and Google Cloud. This widespread adoption makes the security flaws even more concerning.

To fix the issues, developers have updated the frameworks with safer code. Meta patched its Llama Stack in September 2024, replacing pickle with JSON serialization. Nvidia, vLLM, and Modular Max Server also released updates that fix the vulnerabilities, identified as CVE-2024-50050, CVE-2025-30165, CVE-2025-23254, and CVE-2025-60455.

Oligo recommends upgrading to these patched versions and avoiding the use of pickle with untrusted data. They also suggest adding security measures like HMAC and TLS for ZeroMQ communication, and training teams on the risks involved in code reuse. These steps are crucial to prevent malicious attacks and protect sensitive AI infrastructure from systemic failure.

Inspired by

• https://www.infoworld.com/article/4090086/copy-paste-vulnerability-hit-ai-inference-frameworks-at-meta-nvidia-and-microsoft-2.html

Sources

• gmpg.org
• csoonline.com
• oligo.security
• csoonline.com
• csoonline.com