{"id":"GHSA-23j4-mw76-5v7h","aliases":[],"status":"published","title":"Scrapy allows redirect following in protocols other than HTTP","description":"### Impact\n\nScrapy was following redirects regardless of the URL protocol, so redirects were working for `data://`, `file://`, `ftp://`, `s3://`, and any other scheme defined in the `DOWNLOAD_HANDLERS` setting.\n\nHowever, HTTP redirects should only work between URLs that use the `http://` or `https://` schemes.\n\nA malicious actor, given write access to the start requests (e.g. ability to define `start_urls`) of a spider and read access to the spider output, could exploit this vulnerability to:\n- Redirect to any local file using the `file://` scheme to read its contents.\n- Redirect to an `ftp://` URL of a malicious FTP server to obtain the FTP username and password configured in the spider or project.\n- Redirect to any `s3://` URL to read its content using the S3 credentials configured in the spider or project.\n\nFor `file://` and `s3://`, how the spider implements its parsing of input data into an output item determines what data would be vulnerable. A spider that always outputs the entire contents of a response would be completely vulnerable, while a spider that extracted only fragments from the response could significantly limit vulnerable data.\n\n### Patches\n\nUpgrade to Scrapy 2.11.2.\n\n### Workarounds\n\nReplace the built-in retry middlewares (`RedirectMiddleware` and `MetaRefreshMiddleware`) with custom ones that implement the fix from Scrapy 2.11.2, and verify that they work as intended.\n\n### References\n\nThis security issue was reported by @mvsantos at https://github.com/scrapy/scrapy/issues/457.","severity":{"version":"3.1","score":6.5,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N","level":"medium","from":"ghsa"},"facts":{"in_kev":false,"known_exploited":false,"public_exploit":false,"fix_available":true},"cwes":["CWE-552"],"published":"2024-05-14T20:14:49Z","modified":"2024-11-28T05:42:50Z","fixes":[{"ecosystem":"PyPI","package":"scrapy","fixed_in":["2.11.2"],"sources":["ghsa","osv"]}],"affected":[{"source":"ghsa","ecosystem":"PyPI","package":"scrapy","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"2.11.2"}]}]},{"source":"osv","ecosystem":"PyPI","package":"scrapy","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"2.11.2"}]}],"versions":["0.10.4.2364","0.12.0.2550","0.14.1","0.14.2","0.14.3","0.14.4","0.16.0","0.16.1","0.16.2","0.16.3","0.16.4","0.16.5","0.18.0","0.18.1","0.18.2","0.18.3","0.18.4","0.20.0","0.20.1","0.20.2","0.22.0","0.22.1","0.22.2","0.24.0","0.24.1","0.24.2","0.24.3","0.24.4","0.24.5","0.24.6","0.7","0.8","0.9","1.0.0","1.0.0rc1","1.0.0rc2","1.0.0rc3","1.0.1","1.0.2","1.0.3","1.0.4","1.0.5","1.0.6","1.0.7","1.1.0","1.1.0rc1","1.1.0rc2","1.1.0rc3","1.1.0rc4","1.1.1","1.1.2","1.1.3","1.1.4","1.2.0","1.2.1","1.2.2","1.2.3","1.3.0","1.3.1","1.3.2","1.3.3","1.4.0","1.5.0","1.5.1","1.5.2","1.6.0","1.7.0","1.7.1","1.7.2","1.7.3","1.7.4","1.8.0","1.8.1","1.8.2","1.8.3","1.8.4","2.0.0","2.0.1","2.1.0","2.10.0","2.10.1","2.11.0","2.11.1","2.2.0","2.2.1","2.3.0","2.4.0","2.4.1","2.5.0","2.5.1","2.6.0","2.6.1","2.6.2","2.6.3","2.7.0","2.7.1","2.8.0","2.9.0"]}],"references":[{"url":"https://github.com/scrapy/scrapy/security/advisories/GHSA-23j4-mw76-5v7h","tags":["web"]},{"url":"https://github.com/scrapy/scrapy/issues/457","tags":["web"]},{"url":"https://github.com/scrapy/scrapy/commit/36287cb665ab4b0c65fd53181c9a0ef04990ada6","tags":["web"]},{"url":"https://github.com/advisories/GHSA-23j4-mw76-5v7h"},{"url":"https://github.com/scrapy/scrapy","tags":["package"]}],"sources":{"ghsa":[{"id":"GHSA-23j4-mw76-5v7h","title":"Scrapy allows redirect following in protocols other than HTTP","description":"### Impact\n\nScrapy was following redirects regardless of the URL protocol, so redirects were working for `data://`, `file://`, `ftp://`, `s3://`, and any other scheme defined in the `DOWNLOAD_HANDLERS` setting.\n\nHowever, HTTP redirects should only work between URLs that use the `http://` or `https://` schemes.\n\nA malicious actor, given write access to the start requests (e.g. ability to define `start_urls`) of a spider and read access to the spider output, could exploit this vulnerability to:\n- Redirect to any local file using the `file://` scheme to read its contents.\n- Redirect to an `ftp://` URL of a malicious FTP server to obtain the FTP username and password configured in the spider or project.\n- Redirect to any `s3://` URL to read its content using the S3 credentials configured in the spider or project.\n\nFor `file://` and `s3://`, how the spider implements its parsing of input data into an output item determines what data would be vulnerable. A spider that always outputs the entire contents of a response would be completely vulnerable, while a spider that extracted only fragments from the response could significantly limit vulnerable data.\n\n### Patches\n\nUpgrade to Scrapy 2.11.2.\n\n### Workarounds\n\nReplace the built-in retry middlewares (`RedirectMiddleware` and `MetaRefreshMiddleware`) with custom ones that implement the fix from Scrapy 2.11.2, and verify that they work as intended.\n\n### References\n\nThis security issue was reported by @mvsantos at https://github.com/scrapy/scrapy/issues/457.\n","status":"published","published":"2024-05-14T20:14:49Z","modified":"2024-05-14T20:14:50Z","cvss":[{"version":"3.1","score":6.5,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N","level":"medium"}],"cwes":["CWE-552"],"updated_at":"2026-11-02T14:37:48.848211Z"}],"osv":[{"id":"GHSA-23j4-mw76-5v7h","title":"Scrapy allows redirect following in protocols other than HTTP","description":"### Impact\n\nScrapy was following redirects regardless of the URL protocol, so redirects were working for `data://`, `file://`, `ftp://`, `s3://`, and any other scheme defined in the `DOWNLOAD_HANDLERS` setting.\n\nHowever, HTTP redirects should only work between URLs that use the `http://` or `https://` schemes.\n\nA malicious actor, given write access to the start requests (e.g. ability to define `start_urls`) of a spider and read access to the spider output, could exploit this vulnerability to:\n- Redirect to any local file using the `file://` scheme to read its contents.\n- Redirect to an `ftp://` URL of a malicious FTP server to obtain the FTP username and password configured in the spider or project.\n- Redirect to any `s3://` URL to read its content using the S3 credentials configured in the spider or project.\n\nFor `file://` and `s3://`, how the spider implements its parsing of input data into an output item determines what data would be vulnerable. A spider that always outputs the entire contents of a response would be completely vulnerable, while a spider that extracted only fragments from the response could significantly limit vulnerable data.\n\n### Patches\n\nUpgrade to Scrapy 2.11.2.\n\n### Workarounds\n\nReplace the built-in retry middlewares (`RedirectMiddleware` and `MetaRefreshMiddleware`) with custom ones that implement the fix from Scrapy 2.11.2, and verify that they work as intended.\n\n### References\n\nThis security issue was reported by @mvsantos at https://github.com/scrapy/scrapy/issues/457.\n","status":"published","published":"2024-05-14T20:14:49Z","modified":"2024-11-28T05:42:50.070417Z","cvss":[{"version":"3.1","score":6.5,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N","level":"medium"}],"cwes":["CWE-552"],"updated_at":"2026-11-02T15:46:33.520522Z"}]}}