CVE-2026-115913

vLLM is an inference and serving engine for large language models. Prior to 0.32.0, the Phi-4 multimodal audio encoder in phi4mm_audio.py and phi4mm_utils.py uses eager attention by default, allowing a remote client to submit one legal long audio clip that causes MultiHeadedAttention.forward_attention to materialize and retain full attention matrices in self.attn. The scaled dot-product attention branch does not materialize these matrices. The resulting quadratic GPU-memory use can cause an out-of-memory condition and terminate EngineCore for all users. This issue is fixed in version 0.32.0.

vLLM is an inference and serving engine for large language models. Prior to 0.32.0, the Phi-4 multimodal audio encoder in phi4mm_audio.py and phi4mm_utils.py uses eager attention by default, allowing a remote client to submit one legal long audio clip that causes MultiHeadedAttention.forward_attention to materialize and retain full attention matrices in self.attn. The scaled dot-product attention branch does not materialize these matrices. The resulting quadratic GPU-memory use can cause an out-of-memory condition and terminate EngineCore for all users. This issue is fixed in version 0.32.0.

Summary

The Phi-4 multimodal audio encoder (phi4mm_audio.py / phi4mm_utils.py) runs its conformer attention in eager mode by default: the official HF config of microsoft/Phi-4-multimodal-instruct does not set use_pt_scaled_dot_product_attention, so the default False applies (phi4mm_audio.py:166, :876; phi4mm_utils.py:1669). The eager path fully materializes the attention probability matrix of shape [n_chunks, 16, 500, 500] per layer — and then stores it on the module: self.attn = attn (phi4mm_utils.py:1802, a torch.jit.Attribute declared at :1693), keeping a live reference for every one of the 24 encoder layers.

The only gates on audio length are the server-side limits VLLM_MAX_AUDIO_CLIP_FILESIZE_MB=25 and VLLM_MAX_AUDIO_DECODE_DURATION_S=600 (envs.py:83-84) — i.e. a 10-minute clip is a legal input by default. A 600 s clip becomes ~60,000 mel frames, which the encoder unfolds into 500-frame chunks (phi4mm_audio.py:996, max_seq_len = 500), giving n_chunks ≈ 120. Each layer therefore materializes a 120×16×500×500 matrix (~1–2 GiB depending on dtype), and the module attribute keeps all completed layers' matrices alive, so GPU memory climbs monotonically within a single request until an allocation fails.

Verified end-to-end on the official vllm/vllm-openai:v0.31.0 image: one 19.2 MB silent 600 s WAV → worker torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 2.29 GiB ... 501.75 MiB free ... 22.09 GiB allocated by PyTorch inside get_audio_features (phi4mm_audio.py:1279 → :1332 → phi4mm_utils.py:1386) → EngineCore encountered a fatal error → API EngineDeadError → whole server down; /health refuses connections. Total time from request to dead service: ~6 seconds.

A secondary effect: because self.attn persists on the module after the request ends, the matrices of the last audio request stay resident indefinitely (observed +50 MiB pinned after a single 5 s control request); they are only overwritten by the next audio request, so the resident footprint equals the largest audio processed since startup.

Details

1. Root cause. MultiHeadedAttention.forward_attention in eager mode computes attn = masked_softmax(scores, mask) and unconditionally executes self.attn = attn (phi4mm_utils.py:1795-1802). The attribute is a leftover inference/debug hook (used by torch.jit.Attribute(None, Tensor | None) at :1693); in serving it serves no purpose but pins the full [batch·chunks, heads, T, T] tensor per layer past both the layer's forward and the request's lifetime. The SDP attention branch (use_pt_scaled_dot_product_attention=True) never materializes this matrix — but the HF config does not enable it and the code default is False.

2. Why one request suffices. The conformer chunks long audio to max_seq_len = 500 frames (phi4mm_audio.py:996). Duration is bounded only by VLLM_MAX_AUDIO_DECODE_DURATION_S=600, so n_chunks reaches ~120 for a maximal clip. With 16 heads and fp32 scores, one layer's attention tensor is ~1.8 GiB (the observed failing allocation was 2.29 GiB, consistent with the scores+bias materialization). The encoder has num_blocks=24, attention_heads=16, attention_dim=1024, time_reduction=8 (HF config). Weights plus a handful of retained per-layer matrices exhaust the ~2.3 GiB headroom that --gpu-memory-utilization 0.9 leaves on a 24 GB card — mid-forward, on the first request.

4. Deployment reality. --trust-remote-code is the documented, standard way to serve this model (the HF model card instructs it), so the finding applies to default deployments. The crash is deterministic, not load-dependent: scheduler stats at death show a single running request and near-zero KV cache usage.

Impact

Any unauthenticated client that can reach the chat/completions API of a default deployment serving Phi-4-multimodal-instruct (or any model reusing this audio…

vLLM is an inference and serving engine for large language models. Prior to 0.32.0, the Phi-4 multimodal audio encoder in phi4mm_audio.py and phi4mm_utils.py uses eager attention by default, allowing a remote client to submit one legal long audio clip that causes MultiHeadedAttention.forward_attention to materialize and retain full attention matrices in self.attn. The scaled dot-product attention branch does not materialize these matrices. The resulting quadratic GPU-memory use can cause an out-of-memory condition and terminate EngineCore for all users. This issue is fixed in version 0.32.0.

vLLM is an inference and serving engine for large language models. Prior to 0.32.0, the Phi-4 multimodal audio encoder in phi4mm_audio.py and phi4mm_utils.py uses eager attention by default, allowing a remote client to submit one legal long audio clip that causes MultiHeadedAttention.forward_attention to materialize and retain full attention matrices in self.attn. The scaled dot-product attention branch does not materialize these matrices. The resulting quadratic GPU-memory use can cause an out-of-memory condition and terminate EngineCore for all users. This issue is fixed in version 0.32.0.

Summary

The Phi-4 multimodal audio encoder (phi4mm_audio.py / phi4mm_utils.py) runs its conformer attention in eager mode by default: the official HF config of microsoft/Phi-4-multimodal-instruct does not set use_pt_scaled_dot_product_attention, so the default False applies (phi4mm_audio.py:166, :876; phi4mm_utils.py:1669). The eager path fully materializes the attention probability matrix of shape [n_chunks, 16, 500, 500] per layer — and then stores it on the module: self.attn = attn (phi4mm_utils.py:1802, a torch.jit.Attribute declared at :1693), keeping a live reference for every one of the 24 encoder layers.

The only gates on audio length are the server-side limits VLLM_MAX_AUDIO_CLIP_FILESIZE_MB=25 and VLLM_MAX_AUDIO_DECODE_DURATION_S=600 (envs.py:83-84) — i.e. a 10-minute clip is a legal input by default. A 600 s clip becomes ~60,000 mel frames, which the encoder unfolds into 500-frame chunks (phi4mm_audio.py:996, max_seq_len = 500), giving n_chunks ≈ 120. Each layer therefore materializes a 120×16×500×500 matrix (~1–2 GiB depending on dtype), and the module attribute keeps all completed layers' matrices alive, so GPU memory climbs monotonically within a single request until an allocation fails.

Verified end-to-end on the official vllm/vllm-openai:v0.31.0 image: one 19.2 MB silent 600 s WAV → worker torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 2.29 GiB ... 501.75 MiB free ... 22.09 GiB allocated by PyTorch inside get_audio_features (phi4mm_audio.py:1279 → :1332 → phi4mm_utils.py:1386) → EngineCore encountered a fatal error → API EngineDeadError → whole server down; /health refuses connections. Total time from request to dead service: ~6 seconds.

A secondary effect: because self.attn persists on the module after the request ends, the matrices of the last audio request stay resident indefinitely (observed +50 MiB pinned after a single 5 s control request); they are only overwritten by the next audio request, so the resident footprint equals the largest audio processed since startup.

Details

1. Root cause. MultiHeadedAttention.forward_attention in eager mode computes attn = masked_softmax(scores, mask) and unconditionally executes self.attn = attn (phi4mm_utils.py:1795-1802). The attribute is a leftover inference/debug hook (used by torch.jit.Attribute(None, Tensor | None) at :1693); in serving it serves no purpose but pins the full [batch·chunks, heads, T, T] tensor per layer past both the layer's forward and the request's lifetime. The SDP attention branch (use_pt_scaled_dot_product_attention=True) never materializes this matrix — but the HF config does not enable it and the code default is False.

2. Why one request suffices. The conformer chunks long audio to max_seq_len = 500 frames (phi4mm_audio.py:996). Duration is bounded only by VLLM_MAX_AUDIO_DECODE_DURATION_S=600, so n_chunks reaches ~120 for a maximal clip. With 16 heads and fp32 scores, one layer's attention tensor is ~1.8 GiB (the observed failing allocation was 2.29 GiB, consistent with the scores+bias materialization). The encoder has num_blocks=24, attention_heads=16, attention_dim=1024, time_reduction=8 (HF config). Weights plus a handful of retained per-layer matrices exhaust the ~2.3 GiB headroom that --gpu-memory-utilization 0.9 leaves on a 24 GB card — mid-forward, on the first request.

4. Deployment reality. --trust-remote-code is the documented, standard way to serve this model (the HF model card instructs it), so the finding applies to default deployments. The crash is deterministic, not load-dependent: scheduler stats at death show a single running request and near-zero KV cache usage.

Impact

Any unauthenticated client that can reach the chat/completions API of a default deployment serving Phi-4-multimodal-instruct (or any model reusing this audio…