GHSA-23j4-mw76-5v7h
Scrapy allows redirect following in protocols other than HTTP
GHSA-23j4-mw76-5v7h is a medium vulnerability (CVSS 6.5) in scrapy, published May 14, 2024. No source lists public exploit code. Fixed in scrapy 2.11.2.
- Severity
- 6.5 MediumCVSS 3.1, GitHub Advisories
- Exploit chance
- UnscoredNo EPSS score yet
- Exploitation
- None knownNot on CISA’s or ENISA’s lists
- Fixed in
2.11.2scrapy +1 more
Description
Impact
Scrapy was following redirects regardless of the URL protocol, so redirects were working for data://, file://, ftp://, s3://, and any other scheme defined in the DOWNLOAD_HANDLERS setting.
However, HTTP redirects should only work between URLs that use the http:// or https:// schemes.
A malicious actor, given write access to the start requests (e.g. ability to define start_urls) of a spider and read access to the spider output, could exploit this vulnerability to:
- Redirect to any local file using the
file://scheme to read its contents. - Redirect to an
ftp://URL of a malicious FTP server to obtain the FTP username and password configured in the spider or project. - Redirect to any
s3://URL to read its content using the S3 credentials configured in the spider or project.
For file:// and s3://, how the spider implements its parsing of input data into an output item determines what data would be vulnerable. A spider that always outputs the entire contents of a response would be completely vulnerable, while a spider that extracted only fragments from the response could significantly limit vulnerable data.
Patches
Upgrade to Scrapy 2.11.2.
Workarounds
Replace the built-in retry middlewares (RedirectMiddleware and MetaRefreshMiddleware) with custom ones that implement the fix from Scrapy 2.11.2, and verify that they work as intended.
References
This security issue was reported by @mvsantos at https://github.com/scrapy/scrapy/issues/457.
Attack profile
How easy it is to exploit
- Attack vector
- Network
- Complexity
- Low
- Privileges needed
- Low
- User interaction
- None
What an attacker can affect
- Confidentiality
- High
- Integrity
- None
- Availability
- None
Affected packages
| Package | Affected | Fixed |
|---|---|---|
scrapyPyPI | before 2.11.2 | 2.11.2 |
scrapyPyPI | before 2.11.2, plus listed versions 0.10.4.23640.12.0.25500.14.10.14.20.14.3 Show 45 moreShow less0.14.40.16.00.16.10.16.20.16.30.16.40.16.50.18.00.18.10.18.20.18.30.18.40.20.00.20.10.20.20.22.00.22.10.22.20.24.00.24.10.24.20.24.30.24.40.24.50.24.60.70.80.91.0.01.0.0rc11.0.0rc21.0.0rc31.0.11.0.21.0.31.0.41.0.51.0.61.0.71.1.01.1.0rc11.1.0rc21.1.0rc31.1.0rc41.1.1 | 2.11.2 |
What each source reports
GitHub Advisories Score used
Scrapy allows redirect following in protocols other than HTTP
Impact
Scrapy was following redirects regardless of the URL protocol, so redirects were working for data://, file://, ftp://, s3://, and any other scheme defined in the DOWNLOAD_HANDLERS setting.
However, HTTP redirects should only work between URLs that use the http:// or https:// schemes.
A malicious actor, given write access to the start requests (e.g. ability to define start_urls) of a spider and read access to the spider output, could exploit this vulnerability to:
- Redirect to any local file using the
file://scheme to read its contents. - Redirect to an
ftp://URL of a malicious FTP server to obtain the FTP username and password configured in the spider or project. - Redirect to any
s3://URL to read its content using the S3 credentials configured in the spider or project.
For file:// and s3://, how the spider implements its parsing of input data into an output item determines what data would be vulnerable. A spider that always outputs the entire contents of a response would be completely vulnerable, while a spider that extracted only fragments from the response could significantly limit vulnerable data.
Patches
Upgrade to Scrapy 2.11.2.
Workarounds
Replace the built-in retry middlewares (RedirectMiddleware and MetaRefreshMiddleware) with custom ones that implement the fix from Scrapy 2.11.2, and verify that they work as intended.
References
This security issue was reported by @mvsantos at https://github.com/scrapy/scrapy/issues/457.
- Record
GHSA-23j4-mw76-5v7h- CVSS 3.1
- 6.5 Medium
CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N - Weakness
- CWE-552
- Updated
- 2026-11-02
OSV
Scrapy allows redirect following in protocols other than HTTP
Impact
Scrapy was following redirects regardless of the URL protocol, so redirects were working for data://, file://, ftp://, s3://, and any other scheme defined in the DOWNLOAD_HANDLERS setting.
However, HTTP redirects should only work between URLs that use the http:// or https:// schemes.
A malicious actor, given write access to the start requests (e.g. ability to define start_urls) of a spider and read access to the spider output, could exploit this vulnerability to:
- Redirect to any local file using the
file://scheme to read its contents. - Redirect to an
ftp://URL of a malicious FTP server to obtain the FTP username and password configured in the spider or project. - Redirect to any
s3://URL to read its content using the S3 credentials configured in the spider or project.
For file:// and s3://, how the spider implements its parsing of input data into an output item determines what data would be vulnerable. A spider that always outputs the entire contents of a response would be completely vulnerable, while a spider that extracted only fragments from the response could significantly limit vulnerable data.
Patches
Upgrade to Scrapy 2.11.2.
Workarounds
Replace the built-in retry middlewares (RedirectMiddleware and MetaRefreshMiddleware) with custom ones that implement the fix from Scrapy 2.11.2, and verify that they work as intended.
References
This security issue was reported by @mvsantos at https://github.com/scrapy/scrapy/issues/457.
- Record
GHSA-23j4-mw76-5v7h- CVSS 3.1
- 6.5 Medium
CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N - Weakness
- CWE-552
- Updated
- 2026-11-02
History
| Recorded | Changed | Sources |
|---|---|---|
| first indexed | GitHub Advisories, OSV |
Dates are when Every CVE recorded each change. Upstream dates may differ.
References
Left out here: 48 versions. The JSON record has them.