{"id":"CVE-2021-41125","aliases":["DEBIAN-CVE-2021-41125","DLA-2950-1","EUVD-2021-0239","GHSA-jwqp-28gf-p498","PYSEC-2021-363","UBUNTU-CVE-2021-41125","USN-7476-1","openSUSE-SU-2024:11558-1"],"status":"published","title":"HTTP authentication credential leak to target websites in scrapy","description":"Scrapy is a high-level web crawling and scraping framework for Python. If you use `HttpAuthMiddleware` (i.e. the `http_user` and `http_pass` spider attributes) for HTTP authentication, all requests will expose your credentials to the request target. This includes requests generated by Scrapy components, such as `robots.txt` requests sent by Scrapy when the `ROBOTSTXT_OBEY` setting is set to `True`, or as requests reached through redirects. Upgrade to Scrapy 2.5.1 and use the new `http_auth_domain` spider attribute to control which domains are allowed to receive the configured HTTP authentication credentials. If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to Scrapy 1.8.1 instead. If you cannot upgrade, set your HTTP authentication credentials on a per-request basis, using for example the `w3lib.http.basic_auth_header` function to convert your credentials into a value that you can assign to the `Authorization` header of your request, instead of defining your credentials globally using `HttpAuthMiddleware`.","severity":{"version":"3.1","score":5.7,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:R/S:U/C:H/I:N/A:N","level":"medium","assessor":"cna","from":"cve"},"epss":{"day":"2026-11-10","score":0.01261,"percentile":0.69877,"model":"v2026.06.15"},"facts":{"in_kev":false,"known_exploited":false,"public_exploit":false,"fix_available":true},"cwes":["CWE-200","CWE-522"],"published":"2021-10-06T17:15:13Z","modified":"2026-10-23T18:03:38Z","fixes":[{"ecosystem":"PyPI","package":"scrapy","fixed_in":["1.8.1","2.5.1"],"sources":["ghsa","osv"]},{"ecosystem":"Debian:12","package":"python-scrapy","fixed_in":["2.5.1-1"],"sources":["osv"]},{"ecosystem":"Debian:13","package":"python-scrapy","fixed_in":["2.5.1-1"],"sources":["osv"]},{"ecosystem":"Debian:14","package":"python-scrapy","fixed_in":["2.5.1-1"],"sources":["osv"]},{"ecosystem":"Ubuntu:Pro:18.04:LTS","package":"python-scrapy","fixed_in":["1.5.0-1ubuntu0.1~esm1"],"sources":["osv"]},{"ecosystem":"Ubuntu:Pro:20.04:LTS","package":"python-scrapy","fixed_in":["1.7.3-1ubuntu0.1~esm1"],"sources":["osv"]},{"ecosystem":"openSUSE:Tumbleweed","package":"python-Scrapy","fixed_in":["2.5.1-1.1"],"sources":["osv"]}],"affected":[{"source":"ghsa","ecosystem":"PyPI","package":"scrapy","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"1.8.1"}]}]},{"source":"ghsa","ecosystem":"PyPI","package":"scrapy","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"2.0.0"},{"fixed":"2.5.1"}]}]},{"source":"osv","ecosystem":"Debian:12","package":"python-scrapy","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"2.5.1-1"}]}]},{"source":"osv","ecosystem":"Debian:13","package":"python-scrapy","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"2.5.1-1"}]}]},{"source":"osv","ecosystem":"Debian:14","package":"python-scrapy","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"2.5.1-1"}]}]},{"source":"osv","ecosystem":"Debian:9","package":"python-scrapy","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"1.0.3-2+deb9u1"}]}],"versions":["1.0.3-2"],"bundled":2},{"source":"osv","ecosystem":"PyPI","package":"scrapy","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"1.8.1"}]}],"versions":["0.10.4.2364","0.12.0.2550","0.14.1","0.14.2","0.14.3","0.14.4","0.16.0","0.16.1","0.16.2","0.16.3","0.16.4","0.16.5","0.18.0","0.18.1","0.18.2","0.18.3","0.18.4","0.20.0","0.20.1","0.20.2","0.22.0","0.22.1","0.22.2","0.24.0","0.24.1","0.24.2","0.24.3","0.24.4","0.24.5","0.24.6","0.7","0.8","0.9","1.0.0","1.0.0rc1","1.0.0rc2","1.0.0rc3","1.0.1","1.0.2","1.0.3","1.0.4","1.0.5","1.0.6","1.0.7","1.1.0","1.1.0rc1","1.1.0rc2","1.1.0rc3","1.1.0rc4","1.1.1","1.1.2","1.1.3","1.1.4","1.2.0","1.2.1","1.2.2","1.2.3","1.3.0","1.3.1","1.3.2","1.3.3","1.4.0","1.5.0","1.5.1","1.5.2","1.6.0","1.7.0","1.7.1","1.7.2","1.7.3","1.7.4","1.8.0"]},{"source":"osv","ecosystem":"PyPI","package":"scrapy","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"2.0.0"},{"fixed":"2.5.1"}]}],"versions":["2.0.0","2.0.1","2.1.0","2.2.0","2.2.1","2.3.0","2.4.0","2.4.1","2.5.0"]},{"source":"osv","ecosystem":"PyPI","package":"scrapy","ranges":[{"type":"GIT","events":[{"introduced":"0"},{"fixed":"b01d69a1bf48060daec8f751368622352d8b85a6"}]},{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"1.8.1"},{"introduced":"2.0.0"},{"fixed":"2.5.1"}]}],"versions":["0.10.4.2364","0.12.0.2550","0.14.1","0.14.2","0.14.3","0.14.4","0.16.0","0.16.1","0.16.2","0.16.3","0.16.4","0.16.5","0.18.0","0.18.1","0.18.2","0.18.3","0.18.4","0.20.0","0.20.1","0.20.2","0.22.0","0.22.1","0.22.2","0.24.0","0.24.1","0.24.2","0.24.3","0.24.4","0.24.5","0.24.6","0.7","0.8","0.9","1.0.0","1.0.0rc1","1.0.0rc2","1.0.0rc3","1.0.1","1.0.2","1.0.3","1.0.4","1.0.5","1.0.6","1.0.7","1.1.0","1.1.0rc1","1.1.0rc2","1.1.0rc3","1.1.0rc4","1.1.1","1.1.2","1.1.3","1.1.4","1.2.0","1.2.1","1.2.2","1.2.3","1.3.0","1.3.1","1.3.2","1.3.3","1.4.0","1.5.0","1.5.1","1.5.2","1.6.0","1.7.0","1.7.1","1.7.2","1.7.3","1.7.4","1.8.0","2.0.0","2.0.1","2.1.0","2.2.0","2.2.1","2.3.0","2.4.0","2.4.1","2.5.0"]},{"source":"osv","ecosystem":"Ubuntu:16.04:LTS","package":"python-scrapy","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"}]}],"versions":["1.0.0-1","1.0.3-1"]},{"source":"osv","ecosystem":"Ubuntu:Pro:18.04:LTS","package":"python-scrapy","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"1.5.0-1ubuntu0.1~esm1"}]}],"versions":["1.3.0-1~exp2","1.4.0-1","1.5.0-1"]},{"source":"osv","ecosystem":"Ubuntu:Pro:20.04:LTS","package":"python-scrapy","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"1.7.3-1ubuntu0.1~esm1"}]}],"versions":["1.7.3-1"]},{"source":"osv","ecosystem":"Ubuntu:Pro:18.04:LTS","package":"python-scrapy","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"1.5.0-1ubuntu0.1~esm1"}]}],"versions":["1.3.0-1~exp2","1.4.0-1","1.5.0-1"],"bundled":6},{"source":"osv","ecosystem":"Ubuntu:Pro:20.04:LTS","package":"python-scrapy","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"1.7.3-1ubuntu0.1~esm1"}]}],"versions":["1.7.3-1"],"bundled":6},{"source":"osv","ecosystem":"Ubuntu:Pro:22.04:LTS","package":"python-scrapy","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"2.5.1-2ubuntu0.1~esm1"}]}],"versions":["2.4.1-2","2.5.1-2"],"bundled":6},{"source":"osv","ecosystem":"Ubuntu:Pro:24.04:LTS","package":"python-scrapy","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"2.11.1-1ubuntu0.1~esm2"}]}],"versions":["2.10.0-1","2.11.0-1","2.11.0-2","2.11.1-1"],"bundled":6},{"source":"osv","ecosystem":"openSUSE:Tumbleweed","package":"python-Scrapy","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"2.5.1-1.1"}]}]}],"products":[{"vendor":"debian","product":"debian_linux"},{"vendor":"scrapy","product":"scrapy"}],"references":[{"url":"https://github.com/scrapy/scrapy/security/advisories/GHSA-jwqp-28gf-p498","tags":["x_refsource_CONFIRM","x_transferred","Third Party Advisory","advisory","web","report"]},{"url":"https://github.com/scrapy/scrapy/commit/b01d69a1bf48060daec8f751368622352d8b85a6","tags":["x_refsource_MISC","x_transferred","Patch","Third Party Advisory","fix","web","report"]},{"url":"https://w3lib.readthedocs.io/en/latest/w3lib.html#w3lib.http.basic_auth_header","tags":["x_refsource_MISC","x_transferred","Third Party Advisory","advisory","web","report"]},{"url":"http://doc.scrapy.org/en/latest/topics/downloader-middleware.html#module-scrapy.downloadermiddlewares.httpauth","tags":["x_refsource_MISC","x_transferred","Vendor Advisory","advisory","web","report"]},{"url":"https://lists.debian.org/debian-lts-announce/2022/03/msg00021.html","tags":["mailing-list","x_refsource_MLIST","x_transferred","Mailing List","Third Party Advisory","advisory","web"]},{"url":"https://nvd.nist.gov/vuln/detail/CVE-2021-41125","tags":["advisory"]},{"url":"https://github.com/pypa/advisory-database/tree/main/vulns/scrapy/PYSEC-2021-363.yaml","tags":["web"]},{"url":"https://github.com/advisories/GHSA-jwqp-28gf-p498"},{"url":"https://security-tracker.debian.org/tracker/CVE-2021-41125","tags":["advisory"]},{"url":"https://github.com/scrapy/scrapy","tags":["package"]},{"url":"https://ubuntu.com/security/CVE-2021-41125","tags":["report"]},{"url":"https://www.cve.org/CVERecord?id=CVE-2021-41125","tags":["report"]},{"url":"https://ubuntu.com/security/notices/USN-7476-1","tags":["advisory"]},{"url":"https://ubuntu.com/security/CVE-2022-0577","tags":["report"]},{"url":"https://ubuntu.com/security/CVE-2024-1892","tags":["report"]},{"url":"https://ubuntu.com/security/CVE-2024-1968","tags":["report"]},{"url":"https://ubuntu.com/security/CVE-2024-3572","tags":["report"]},{"url":"https://ubuntu.com/security/CVE-2024-3574","tags":["report"]},{"url":"","tags":["advisory"]},{"url":"https://www.suse.com/security/cve/CVE-2021-41125","tags":["web"]}],"sources":{"cve":[{"id":"CVE-2021-41125","title":"HTTP authentication credential leak to target websites in scrapy","description":"Scrapy is a high-level web crawling and scraping framework for Python. If you use `HttpAuthMiddleware` (i.e. the `http_user` and `http_pass` spider attributes) for HTTP authentication, all requests will expose your credentials to the request target. This includes requests generated by Scrapy components, such as `robots.txt` requests sent by Scrapy when the `ROBOTSTXT_OBEY` setting is set to `True`, or as requests reached through redirects. Upgrade to Scrapy 2.5.1 and use the new `http_auth_domain` spider attribute to control which domains are allowed to receive the configured HTTP authentication credentials. If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to Scrapy 1.8.1 instead. If you cannot upgrade, set your HTTP authentication credentials on a per-request basis, using for example the `w3lib.http.basic_auth_header` function to convert your credentials into a value that you can assign to the `Authorization` header of your request, instead of defining your credentials globally using `HttpAuthMiddleware`.","status":"published","published":"2021-10-06T17:15:13Z","modified":"2024-08-04T02:59:31.432Z","cvss":[{"version":"3.1","score":5.7,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:R/S:U/C:H/I:N/A:N","level":"medium","assessor":"cna"}],"cwes":["CWE-200"],"updated_at":"2026-11-02T14:32:59.105382Z"}],"euvd":[{"id":"EUVD-2021-0239","description":"Scrapy is a high-level web crawling and scraping framework for Python. If you use `HttpAuthMiddleware` (i.e. the `http_user` and `http_pass` spider attributes) for HTTP authentication, all requests will expose your credentials to the request target. This includes requests generated by Scrapy components, such as `robots.txt` requests sent by Scrapy when the `ROBOTSTXT_OBEY` setting is set to `True`, or as requests reached through redirects. Upgrade to Scrapy 2.5.1 and use the new `http_auth_domain` spider attribute to control which domains are allowed to receive the configured HTTP authentication credentials. If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to Scrapy 1.8.1 instead. If you cannot upgrade, set your HTTP authentication credentials on a per-request basis, using for example the `w3lib.http.basic_auth_header` function to convert your credentials into a value that you can assign to the `Authorization` header of your request, instead of defining your credentials globally using `HttpAuthMiddleware`.","status":"published","published":"2021-10-06T17:15:13Z","modified":"2024-08-04T02:59:31Z","cvss":[{"version":"3.1","score":5.7,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:R/S:U/C:H/I:N/A:N","level":"medium"}],"updated_at":"2026-11-03T03:24:19.242491Z"}],"ghsa":[{"id":"GHSA-jwqp-28gf-p498","title":"Scrapy HTTP authentication credentials potentially leaked to target websites ","description":"### Impact\n\nIf you use [`HttpAuthMiddleware`](http://doc.scrapy.org/en/latest/topics/downloader-middleware.html#module-scrapy.downloadermiddlewares.httpauth) (i.e. the `http_user` and `http_pass` spider attributes) for HTTP authentication, all requests will expose your credentials to the request target.\n\nThis includes requests generated by Scrapy components, such as `robots.txt` requests sent by Scrapy when the `ROBOTSTXT_OBEY` setting is set to `True`, or as requests reached through redirects.\n\n### Patches\n\nUpgrade to Scrapy 2.5.1 and use the new `http_auth_domain` spider attribute to control which domains are allowed to receive the configured HTTP authentication credentials.\n\nIf you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to Scrapy 1.8.1 instead.\n\n### Workarounds\n\nIf you cannot upgrade, set your HTTP authentication credentials on a per-request basis, using for example the [`w3lib.http.basic_auth_header`](https://w3lib.readthedocs.io/en/latest/w3lib.html#w3lib.http.basic_auth_header) function to convert your credentials into a value that you can assign to the `Authorization` header of your request, instead of defining your credentials globally using [`HttpAuthMiddleware`](http://doc.scrapy.org/en/latest/topics/downloader-middleware.html#module-scrapy.downloadermiddlewares.httpauth).\n\n### For more information\nIf you have any questions or comments about this advisory:\n* [Open an issue](https://github.com/scrapy/scrapy/issues)\n* [Email us](mailto:opensource@zyte.com)\n","status":"published","published":"2021-10-06T17:46:22Z","modified":"2024-10-26T22:53:22Z","cvss":[{"version":"4.0","score":6.9,"vector":"CVSS:4.0/AV:N/AC:L/AT:N/PR:L/UI:P/VC:H/VI:N/VA:N/SC:N/SI:N/SA:N","level":"medium"},{"version":"3.1","score":5.7,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:R/S:U/C:H/I:N/A:N","level":"medium"}],"cwes":["CWE-200","CWE-522"],"updated_at":"2026-11-02T14:38:13.90937Z"}],"nvd":[{"id":"CVE-2021-41125","description":"Scrapy is a high-level web crawling and scraping framework for Python. If you use `HttpAuthMiddleware` (i.e. the `http_user` and `http_pass` spider attributes) for HTTP authentication, all requests will expose your credentials to the request target. This includes requests generated by Scrapy components, such as `robots.txt` requests sent by Scrapy when the `ROBOTSTXT_OBEY` setting is set to `True`, or as requests reached through redirects. Upgrade to Scrapy 2.5.1 and use the new `http_auth_domain` spider attribute to control which domains are allowed to receive the configured HTTP authentication credentials. If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to Scrapy 1.8.1 instead. If you cannot upgrade, set your HTTP authentication credentials on a per-request basis, using for example the `w3lib.http.basic_auth_header` function to convert your credentials into a value that you can assign to the `Authorization` header of your request, instead of defining your credentials globally using `HttpAuthMiddleware`.","status":"published","published":"2021-10-06T18:15:10.953Z","modified":"2026-06-17T04:07:55.323Z","cvss":[{"version":"3.1","score":5.7,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:R/S:U/C:H/I:N/A:N","level":"medium","assessor":"secondary"},{"version":"3.1","score":6.5,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N","level":"medium","assessor":"primary"},{"version":"2.0","score":4,"vector":"AV:N/AC:L/Au:S/C:P/I:N/A:N","level":"medium","assessor":"primary"}],"cwes":["CWE-200","CWE-522"],"updated_at":"2026-11-02T14:06:59.314106Z"}],"osv":[{"id":"CVE-2021-41125","description":"Scrapy is a high-level web crawling and scraping framework for Python. If you use `HttpAuthMiddleware` (i.e. the `http_user` and `http_pass` spider attributes) for HTTP authentication, all requests will expose your credentials to the request target. This includes requests generated by Scrapy components, such as `robots.txt` requests sent by Scrapy when the `ROBOTSTXT_OBEY` setting is set to `True`, or as requests reached through redirects. Upgrade to Scrapy 2.5.1 and use the new `http_auth_domain` spider attribute to control which domains are allowed to receive the configured HTTP authentication credentials. If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to Scrapy 1.8.1 instead. If you cannot upgrade, set your HTTP authentication credentials on a per-request basis, using for example the `w3lib.http.basic_auth_header` function to convert your credentials into a value that you can assign to the `Authorization` header of your request, instead of defining your credentials globally using `HttpAuthMiddleware`.","status":"published","published":"2021-10-06T18:15:10.953Z","modified":"2026-10-20T10:54:57.387412343Z","cvss":[{"version":"3.1","score":6.5,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N","level":"medium"}],"updated_at":"2026-11-02T15:44:21.249056Z"},{"id":"DEBIAN-CVE-2021-41125","description":"Scrapy is a high-level web crawling and scraping framework for Python. If you use `HttpAuthMiddleware` (i.e. the `http_user` and `http_pass` spider attributes) for HTTP authentication, all requests will expose your credentials to the request target. This includes requests generated by Scrapy components, such as `robots.txt` requests sent by Scrapy when the `ROBOTSTXT_OBEY` setting is set to `True`, or as requests reached through redirects. Upgrade to Scrapy 2.5.1 and use the new `http_auth_domain` spider attribute to control which domains are allowed to receive the configured HTTP authentication credentials. If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to Scrapy 1.8.1 instead. If you cannot upgrade, set your HTTP authentication credentials on a per-request basis, using for example the `w3lib.http.basic_auth_header` function to convert your credentials into a value that you can assign to the `Authorization` header of your request, instead of defining your credentials globally using `HttpAuthMiddleware`.","status":"published","published":"2021-10-06T18:15:10.953Z","modified":"2026-10-23T18:03:38.600255764Z","cvss":[{"version":"3.1","score":6.5,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N","level":"medium"}],"updated_at":"2026-11-02T15:45:35.521549Z"},{"id":"GHSA-jwqp-28gf-p498","title":"Scrapy HTTP authentication credentials potentially leaked to target websites ","description":"### Impact\n\nIf you use [`HttpAuthMiddleware`](http://doc.scrapy.org/en/latest/topics/downloader-middleware.html#module-scrapy.downloadermiddlewares.httpauth) (i.e. the `http_user` and `http_pass` spider attributes) for HTTP authentication, all requests will expose your credentials to the request target.\n\nThis includes requests generated by Scrapy components, such as `robots.txt` requests sent by Scrapy when the `ROBOTSTXT_OBEY` setting is set to `True`, or as requests reached through redirects.\n\n### Patches\n\nUpgrade to Scrapy 2.5.1 and use the new `http_auth_domain` spider attribute to control which domains are allowed to receive the configured HTTP authentication credentials.\n\nIf you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to Scrapy 1.8.1 instead.\n\n### Workarounds\n\nIf you cannot upgrade, set your HTTP authentication credentials on a per-request basis, using for example the [`w3lib.http.basic_auth_header`](https://w3lib.readthedocs.io/en/latest/w3lib.html#w3lib.http.basic_auth_header) function to convert your credentials into a value that you can assign to the `Authorization` header of your request, instead of defining your credentials globally using [`HttpAuthMiddleware`](http://doc.scrapy.org/en/latest/topics/downloader-middleware.html#module-scrapy.downloadermiddlewares.httpauth).\n\n### For more information\nIf you have any questions or comments about this advisory:\n* [Open an issue](https://github.com/scrapy/scrapy/issues)\n* [Email us](mailto:opensource@zyte.com)\n","status":"published","published":"2021-10-06T17:46:22Z","modified":"2026-07-08T06:29:55.026270559Z","cvss":[{"version":"3.1","score":5.7,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:R/S:U/C:H/I:N/A:N","level":"medium"},{"version":"4.0","score":6.9,"vector":"CVSS:4.0/AV:N/AC:L/AT:N/PR:L/UI:P/VC:H/VI:N/VA:N/SC:N/SI:N/SA:N","level":"medium"}],"cwes":["CWE-200","CWE-522"],"updated_at":"2026-11-02T15:46:51.10002Z"},{"id":"UBUNTU-CVE-2021-41125","description":"Scrapy is a high-level web crawling and scraping framework for Python. If you use `HttpAuthMiddleware` (i.e. the `http_user` and `http_pass` spider attributes) for HTTP authentication, all requests will expose your credentials to the request target. This includes requests generated by Scrapy components, such as `robots.txt` requests sent by Scrapy when the `ROBOTSTXT_OBEY` setting is set to `True`, or as requests reached through redirects. Upgrade to Scrapy 2.5.1 and use the new `http_auth_domain` spider attribute to control which domains are allowed to receive the configured HTTP authentication credentials. If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to Scrapy 1.8.1 instead. If you cannot upgrade, set your HTTP authentication credentials on a per-request basis, using for example the `w3lib.http.basic_auth_header` function to convert your credentials into a value that you can assign to the `Authorization` header of your request, instead of defining your credentials globally using `HttpAuthMiddleware`.","status":"published","published":"2021-10-06T18:15:00Z","modified":"2026-02-04T03:41:09.630405Z","cvss":[{"version":"3.1","score":5.7,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:R/S:U/C:H/I:N/A:N","level":"medium"},{"version":"3.1","score":6.5,"vector":"CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N","level":"medium"}],"updated_at":"2026-11-02T15:54:29.220315Z"},{"id":"DLA-2950-1","title":"python-scrapy - security update","status":"published","published":"2022-03-16T00:00:00Z","modified":"2026-03-09T01:23:33.254476Z","updated_at":"2026-11-02T15:46:11.189256Z"},{"id":"openSUSE-SU-2024:11558-1","title":"python-Scrapy-doc-2.5.1-1.1 on GA media","description":"These are all security issues fixed in the python-Scrapy-doc-2.5.1-1.1 package on the GA media of openSUSE Tumbleweed.","status":"published","published":"2024-06-15T00:00:00Z","modified":"2026-02-04T02:21:52.441901Z","updated_at":"2026-11-02T16:58:35.195147Z"},{"id":"PYSEC-2021-363","description":"Scrapy is a high-level web crawling and scraping framework for Python. If you use `HttpAuthMiddleware` (i.e. the `http_user` and `http_pass` spider attributes) for HTTP authentication, all requests will expose your credentials to the request target. This includes requests generated by Scrapy components, such as `robots.txt` requests sent by Scrapy when the `ROBOTSTXT_OBEY` setting is set to `True`, or as requests reached through redirects. Upgrade to Scrapy 2.5.1 and use the new `http_auth_domain` spider attribute to control which domains are allowed to receive the configured HTTP authentication credentials. If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to Scrapy 1.8.1 instead. If you cannot upgrade, set your HTTP authentication credentials on a per-request basis, using for example the `w3lib.http.basic_auth_header` function to convert your credentials into a value that you can assign to the `Authorization` header of your request, instead of defining your credentials globally using `HttpAuthMiddleware`.","status":"published","published":"2021-10-06T18:15:00Z","modified":"2023-11-08T04:06:52.040115Z","updated_at":"2026-11-02T15:52:52.682031Z"},{"id":"USN-7476-1","title":"python-scrapy vulnerabilities","description":"It was discovered that Scrapy improperly exposed HTTP authentication \ncredentials to request targets, including during redirects. An attacker \ncould use this issue to gain unauthorized access to user accounts. This\nissue only affected Ubuntu 18.04 LTS and Ubuntu 20.04 LTS. (CVE-2021-41125)\n\nIt was discovered that Scrapy did not remove the cookie header during\ncross-domain redirects. An attacker could possibly use this issue to gain\nunauthorized access to user accounts. This issue only affected Ubuntu 18.04\nLTS, Ubuntu 20.04 LTS, and Ubuntu 22.04 LTS. (CVE-2022-0577)\n\nIt was discovered that Scrapy inefficiently parsed XML content. An\nattacker could use this issue to cause a denial of service by sending a\ncrafted XML response. This issue only affected Ubuntu 18.04 LTS,\nUbuntu 20.04 LTS, and Ubuntu 22.04 LTS. (CVE-2024-1892)\n\nIt was discovered that Scrapy did not properly check response size during\ndecompression. An attacker could send a crafted response that would\nexhaust memory and cause a denial of service. This issue only affected\nUbuntu 18.04 LTS, Ubuntu 20.04 LTS, and Ubuntu 22.04 LTS. (CVE-2024-3572)\n\nIt was discovered that Scrapy did not remove the authorization header\nduring cross-domain redirects. An attacker could possibly use this issue\nto gain unauthorized access to user accounts. This issue only affected \nUbuntu 18.04 LTS, Ubuntu 20.04 LTS, and Ubuntu 22.04 LTS. (CVE-2024-3574)\n\nIt was discovered that Scrapy did not remove the authorization header\nduring redirects that change scheme but remain in the same domain. This\nissue could possibly be used by an attacker to expose sensitive \ninformation or to gain unauthorized access to user accounts.\n(CVE-2024-1968)\n\n","status":"published","published":"2025-05-05T16:31:52Z","modified":"2026-04-27T18:02:20.75951352Z","updated_at":"2026-11-02T16:58:19.988474Z"}]}}