{"id":"CVE-2026-44223","aliases":["EUVD-2026-29800","GHSA-83vm-p52w-f9pw","PYSEC-2026-145"],"status":"published","title":"vLLM: extract_hidden_states speculative decoding crashes server on any request with penalty parameters","description":"vLLM is an inference and serving engine for large language models (LLMs). From 0.18.0 to before 0.20.0, the extract_hidden_states speculative decoding proposer in vLLM returns a tensor with an incorrect shape after the first decode step, causing a RuntimeError that crashes the EngineCore process. The crash is triggered when any request in the batch uses sampling penalty parameters (repetition_penalty, frequency_penalty, or presence_penalty). A single request with a penalty parameter (e.g., \"repetition_penalty\": 1.1) is sufficient to crash the server. This vulnerability is fixed in 0.20.0.","severity":{"version":"3.1","score":6.5,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H","level":"medium","assessor":"cna","from":"cve"},"epss":{"day":"2026-11-11","score":0.00367,"percentile":0.3093,"model":"v2026.06.15"},"facts":{"in_kev":false,"known_exploited":false,"public_exploit":false,"fix_available":true},"cwes":["CWE-131","CWE-704"],"published":"2026-05-06T21:45:51Z","modified":"2026-11-11T20:10:00Z","fixes":[{"ecosystem":"PyPI","package":"vllm","fixed_in":["0.20.0"],"sources":["ghsa","osv"]}],"affected":[{"source":"ghsa","ecosystem":"PyPI","package":"vllm","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0.18.0"},{"fixed":"0.20.0"}]}]},{"source":"osv","ecosystem":"PyPI","package":"vllm","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0.18.0"},{"fixed":"0.20.0"}]}],"versions":["0.18.0","0.18.1","0.19.0","0.19.1"]},{"source":"osv","ecosystem":"PyPI","package":"vllm","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0.18.0"},{"fixed":"0.20.0"}]}],"versions":["0.18.0","0.18.1","0.19.0","0.19.1"]}],"products":[{"vendor":"vllm","product":"vllm"},{"vendor":"vllm-project","product":"vllm"}],"references":[{"url":"https://github.com/vllm-project/vllm/security/advisories/GHSA-83vm-p52w-f9pw","tags":["x_refsource_CONFIRM","exploit","Mitigation","Vendor Advisory","advisory","web"]},{"url":"https://github.com/vllm-project/vllm/pull/38610","tags":["x_refsource_MISC","exploit","Issue Tracking","Patch","fix","web"]},{"url":"https://nvd.nist.gov/vuln/detail/CVE-2026-44223","tags":["advisory"]},{"url":"https://github.com/pypa/advisory-database/tree/main/vulns/vllm/PYSEC-2026-145.yaml","tags":["web"]},{"url":"https://github.com/advisories/GHSA-83vm-p52w-f9pw"},{"url":"https://github.com/CVEProject/cvelistV5/tree/main/cves/2026/44xxx/CVE-2026-44223.json","tags":["advisory"]},{"url":"https://github.com/vllm-project/vllm","tags":["package"]}],"sources":{"cve":[{"id":"CVE-2026-44223","title":"vLLM: extract_hidden_states speculative decoding crashes server on any request with penalty parameters","description":"vLLM is an inference and serving engine for large language models (LLMs). From 0.18.0 to before 0.20.0, the extract_hidden_states speculative decoding proposer in vLLM returns a tensor with an incorrect shape after the first decode step, causing a RuntimeError that crashes the EngineCore process. The crash is triggered when any request in the batch uses sampling penalty parameters (repetition_penalty, frequency_penalty, or presence_penalty). A single request with a penalty parameter (e.g., \"repetition_penalty\": 1.1) is sufficient to crash the server. This vulnerability is fixed in 0.20.0.","status":"published","published":"2026-05-12T19:58:40.862Z","modified":"2026-06-22T21:49:24.277Z","cvss":[{"version":"3.1","score":6.5,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H","level":"medium","assessor":"cna"}],"cwes":["CWE-131","CWE-704"],"updated_at":"2026-11-02T13:43:00.288193Z"}],"euvd":[{"id":"EUVD-2026-29800","description":"vLLM is an inference and serving engine for large language models (LLMs). From 0.18.0 to before 0.20.0, the extract_hidden_states speculative decoding proposer in vLLM returns a tensor with an incorrect shape after the first decode step, causing a RuntimeError that crashes the EngineCore process. The crash is triggered when any request in the batch uses sampling penalty parameters (repetition_penalty, frequency_penalty, or presence_penalty). A single request with a penalty parameter (e.g., \"repetition_penalty\": 1.1) is sufficient to crash the server. This vulnerability is fixed in 0.20.0.","status":"published","published":"2026-05-12T19:58:40Z","modified":"2026-06-22T21:49:24Z","cvss":[{"version":"3.1","score":6.5,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H","level":"medium"}],"updated_at":"2026-11-02T18:20:33.850201Z"}],"ghsa":[{"id":"GHSA-83vm-p52w-f9pw","title":"vLLM: extract_hidden_states speculative decoding crashes server on any request with penalty parameters","description":"### Summary\n\nThe `extract_hidden_states` speculative decoding proposer in vLLM returns a tensor with an incorrect shape after the first decode step, causing a `RuntimeError` that crashes the EngineCore process. The crash is triggered when any request in the batch uses sampling penalty parameters (`repetition_penalty`, `frequency_penalty`, or `presence_penalty`).\n\nA single request with a penalty parameter (e.g., `\"repetition_penalty\": 1.1`) is sufficient to crash the server. The crash is deterministic and immediate — no concurrency, race condition, or special workload is required.\n\n### Details\n\nIn vLLM v0.17.0, the `extract_hidden_states` proposer's `propose()` method returned `sampled_token_ids.unsqueeze(-1)`, producing a tensor of shape `(batch_size, 1)`.\n\nIn [PR #37013](https://github.com/vllm-project/vllm/pull/37013) (first released in v0.18.0), the KV connector interface was refactored out of `propose()`. The return type changed from `tuple[Tensor, KVConnectorOutput | None]` to `Tensor`, and the `.unsqueeze(-1)` call was removed along with the KV connector output:\n\n```python\n# Before (v0.17.0):\nreturn sampled_token_ids.unsqueeze(-1), kv_connector_output  # shape (batch_size, 1)\n\n# After (v0.18.0+):\nreturn sampled_token_ids  # shape (batch_size, 2) after first decode step\n```\n\nThe refactor missed that `sampled_token_ids` changed semantics between the first and subsequent decode steps. After the first decode step, the rejection sampler allocates its output as `(batch_size, max_spec_len + 1)`. With `num_speculative_tokens=1`, this produces shape `(batch_size, 2)` instead of the expected `(batch_size, 1)`, causing a broadcast shape mismatch during penalty application.\n\n### Impact\n\nAny vLLM deployment between v0.18.0 and v0.19.1 (inclusive) configured with `extract_hidden_states` speculative decoding is affected. A single API request containing any penalty parameter immediately and permanently crashes the EngineCore process, resulting in complete loss of service availability.\n\n### Patches\n\nFixed in [PR #38610](https://github.com/vllm-project/vllm/pull/38610), first included in vLLM v0.20.0. The fix slices the return value to `sampled_token_ids[:, :1]`, ensuring the correct `(batch_size, 1)` shape regardless of the rejection sampler's output dimensions.\n\n### Workarounds\n\n- Upgrade to vLLM v0.20.0 or later.\n- If upgrading is not possible, avoid using `extract_hidden_states` as the speculative decoding method on affected versions.\n- Alternatively, reject or strip penalty parameters (`repetition_penalty`, `frequency_penalty`, `presence_penalty`) from incoming requests at an API gateway before they reach vLLM.","status":"published","published":"2026-05-06T21:45:51Z","modified":"2026-06-08T19:52:36Z","cvss":[{"version":"3.1","score":6.5,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H","level":"medium"}],"cwes":["CWE-131","CWE-704"],"updated_at":"2026-11-02T14:40:20.187636Z"}],"nvd":[{"id":"CVE-2026-44223","description":"vLLM is an inference and serving engine for large language models (LLMs). From 0.18.0 to before 0.20.0, the extract_hidden_states speculative decoding proposer in vLLM returns a tensor with an incorrect shape after the first decode step, causing a RuntimeError that crashes the EngineCore process. The crash is triggered when any request in the batch uses sampling penalty parameters (repetition_penalty, frequency_penalty, or presence_penalty). A single request with a penalty parameter (e.g., \"repetition_penalty\": 1.1) is sufficient to crash the server. This vulnerability is fixed in 0.20.0.","status":"published","published":"2026-05-12T20:16:43.293Z","modified":"2026-11-11T20:10:00.273Z","cvss":[{"version":"3.1","score":6.5,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H","level":"medium","assessor":"secondary"}],"cwes":["CWE-131","CWE-704"],"updated_at":"2026-11-11T21:26:15.366289Z"}],"osv":[{"id":"CVE-2026-44223","title":"vLLM: extract_hidden_states speculative decoding crashes server on any request with penalty parameters","description":"vLLM is an inference and serving engine for large language models (LLMs). From 0.18.0 to before 0.20.0, the extract_hidden_states speculative decoding proposer in vLLM returns a tensor with an incorrect shape after the first decode step, causing a RuntimeError that crashes the EngineCore process. The crash is triggered when any request in the batch uses sampling penalty parameters (repetition_penalty, frequency_penalty, or presence_penalty). A single request with a penalty parameter (e.g., \"repetition_penalty\": 1.1) is sufficient to crash the server. This vulnerability is fixed in 0.20.0.","status":"published","published":"2026-05-12T19:58:40.862Z","modified":"2026-10-28T02:30:47.609380789Z","cvss":[{"version":"3.1","score":6.5,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H","level":"medium"}],"cwes":["CWE-131","CWE-704"],"updated_at":"2026-11-02T15:45:04.820169Z"},{"id":"GHSA-83vm-p52w-f9pw","title":"vLLM: extract_hidden_states speculative decoding crashes server on any request with penalty parameters","description":"### Summary\n\nThe `extract_hidden_states` speculative decoding proposer in vLLM returns a tensor with an incorrect shape after the first decode step, causing a `RuntimeError` that crashes the EngineCore process. The crash is triggered when any request in the batch uses sampling penalty parameters (`repetition_penalty`, `frequency_penalty`, or `presence_penalty`).\n\nA single request with a penalty parameter (e.g., `\"repetition_penalty\": 1.1`) is sufficient to crash the server. The crash is deterministic and immediate — no concurrency, race condition, or special workload is required.\n\n### Details\n\nIn vLLM v0.17.0, the `extract_hidden_states` proposer's `propose()` method returned `sampled_token_ids.unsqueeze(-1)`, producing a tensor of shape `(batch_size, 1)`.\n\nIn [PR #37013](https://github.com/vllm-project/vllm/pull/37013) (first released in v0.18.0), the KV connector interface was refactored out of `propose()`. The return type changed from `tuple[Tensor, KVConnectorOutput | None]` to `Tensor`, and the `.unsqueeze(-1)` call was removed along with the KV connector output:\n\n```python\n# Before (v0.17.0):\nreturn sampled_token_ids.unsqueeze(-1), kv_connector_output  # shape (batch_size, 1)\n\n# After (v0.18.0+):\nreturn sampled_token_ids  # shape (batch_size, 2) after first decode step\n```\n\nThe refactor missed that `sampled_token_ids` changed semantics between the first and subsequent decode steps. After the first decode step, the rejection sampler allocates its output as `(batch_size, max_spec_len + 1)`. With `num_speculative_tokens=1`, this produces shape `(batch_size, 2)` instead of the expected `(batch_size, 1)`, causing a broadcast shape mismatch during penalty application.\n\n### Impact\n\nAny vLLM deployment between v0.18.0 and v0.19.1 (inclusive) configured with `extract_hidden_states` speculative decoding is affected. A single API request containing any penalty parameter immediately and permanently crashes the EngineCore process, resulting in complete loss of service availability.\n\n### Patches\n\nFixed in [PR #38610](https://github.com/vllm-project/vllm/pull/38610), first included in vLLM v0.20.0. The fix slices the return value to `sampled_token_ids[:, :1]`, ensuring the correct `(batch_size, 1)` shape regardless of the rejection sampler's output dimensions.\n\n### Workarounds\n\n- Upgrade to vLLM v0.20.0 or later.\n- If upgrading is not possible, avoid using `extract_hidden_states` as the speculative decoding method on affected versions.\n- Alternatively, reject or strip penalty parameters (`repetition_penalty`, `frequency_penalty`, `presence_penalty`) from incoming requests at an API gateway before they reach vLLM.","status":"published","published":"2026-05-06T21:45:51Z","modified":"2026-09-10T03:50:46.257654637Z","cvss":[{"version":"3.1","score":6.5,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H","level":"medium"}],"cwes":["CWE-131","CWE-704"],"updated_at":"2026-11-02T15:46:41.732368Z"},{"id":"PYSEC-2026-145","description":"vLLM is an inference and serving engine for large language models (LLMs). From  to before 0.20.0, the extract_hidden_states speculative decoding proposer in vLLM returns a tensor with an incorrect shape after the first decode step, causing a RuntimeError that crashes the EngineCore process. The crash is triggered when any request in the batch uses sampling penalty parameters (repetition_penalty, frequency_penalty, or presence_penalty). A single request with a penalty parameter (e.g., \"repetition_penalty\": 1.1) is sufficient to crash the server. This vulnerability is fixed in 0.20.0.","status":"published","published":"2026-05-12T20:16:43.293Z","modified":"2026-05-20T09:19:21.596358Z","cvss":[{"version":"3.1","score":6.5,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H","level":"medium"}],"updated_at":"2026-11-02T15:52:54.622769Z"}]}}