I've recently been experimenting with using proxies with Python requests and cannot seem to get them to work. Although the requests go through with the proxies, testing that I have done has lead me to believe the proxies aren't being applied to the request. Even with obviously bad proxies, my requests still go through, which makes me think the proxy was not being used at all. To demonstrate this, I made a simple script (the working proxy has been edited for this post):
import requests
proxy1 = {"http":"http://this:should@not:work"}
proxy2= {"http":"http://this:[email protected]:33128"}
r1 = requests.get("https://google.com", proxies=proxy2)
print(r1.status_code)
#prints 200 as expected
r2 = requests.get("https://google.com", proxies=proxy1)
print(r2.status_code)
#prints 200 which is weird since I was expecting the request to not go through
Does anyone know why this is happening and if the requests actually are being used with the proxies?
1 Answer 1
In both examples you define proxy only for http
proxy1 = {"http": "http://this:should@not:work"}
proxy2 = {"http": "http://this:[email protected]:33128"}
but you use url with https:
https://google.com
so requests doesn't use proxy.
You have to define proxy for https
proxy1 = {"https": "http://this:should@not:work"}
proxy2 = {"https": "http://this:[email protected]:33128"}
Doc: requests: proxy
EDIT:
Using https://httpbin.org/get you can test GET requests and it will send you back all your headers and IP
I took proxy from one of page with free proxies so it may not work for some time
import requests
proxy = {"https": "104.244.75.26:8080"}
r = requests.get("https://httpbin.org/get", proxies=proxy1)
print(r.status_code)
print(r.text)
Result shows IP of proxy
200
{
"args": {},
"headers": {
"Accept": "*/*",
"Accept-Encoding": "gzip, deflate",
"Host": "httpbin.org",
"User-Agent": "python-requests/2.22.0"
},
"origin": "104.244.75.26, 104.244.75.26",
"url": "https://httpbin.org/get"
}