Making post request using urllib

Question

I am trying to make request on API provider

curl "https://api.infermedica.com/dev/parse" \
 -X "POST" \
 -H "App_Id: 4c177c" -H "App_Key: 6852599182ba85d70066986ca2b3" \
 -H "Content-Type: application/json" \
 -d '{"text": "i feel smoach pain but no couoghing today"}' 

This curl request gives response.

But same request when I try to make in code

self.headers = { "App_Id": "4c177c", "App_Key": "6852599182ba85d70066986ca2b3", "Content-Type": "application/json", "User-Agent": "M$
self.url = "https://api.infermedica.com/dev/parse"
data = { "text": text }
json_data = json.dumps(data)
req = urllib2.Request(self.url, json_data.replace(r"\n", "").replace(r"\r", ""), self.headers)
response = urllib2.urlopen(req).read()

It gives

Traceback (most recent call last):
 File "symptoms_infermedia_api.py", line 68, in <module>
 SymptomsInfermedia().getResponse(raw_input("Enter comment"))
 File "symptoms_infermedia_api.py", line 39, in getResponse
 response = urllib2.urlopen(req).read()
 File "/usr/lib/python2.7/urllib2.py", line 127, in urlopen
 return _opener.open(url, data, timeout)
 File "/usr/lib/python2.7/urllib2.py", line 410, in open
 response = meth(req, response)
 File "/usr/lib/python2.7/urllib2.py", line 523, in http_response
 'http', request, response, code, msg, hdrs)
 File "/usr/lib/python2.7/urllib2.py", line 448, in error
 return self._call_chain(*args)
 File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
 result = func(*args)
 File "/usr/lib/python2.7/urllib2.py", line 531, in http_error_default
 raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 403: Forbidden

• Brendan Abel: Are you dead-set on using urllib2? Because requests has a much better interface for making http requests.
• user123: I flexible enough to try requests
• user123: @BrendanAbel: r = requests.post("https://api.infermedica.com/dev/parse",json = { "App_Id": "7247c", "App_Key": "68599182ba85d70066986ca2b3", "Content-Type": "application/json"}) also gives same error message

Answer 1

This would be the equivalent request using the python requests library.

url = "https://api.infermedica.com/dev/parse"
headers = {
 'App_Id': '4c177c',
 'App_Key': '6852599182ba85d70066986ca2b3',
 'Content-Type': 'application/json',
}
data = {'text': 'i feel stomach pain but no coughing today'}
r = requests.post(url, headers=headers, data=json.dumps(data))
print r.status_code
print r.json()

But your real problem is that you're using the wrong header keys for their api. It's App-Id and App-key, not App_Id and App_key. It would look like this:

headers = {
 'App-Id': 'xx', 
 'App-key': 'xxxx', 
 'Accept': 'application/json', 
 'Content-Type': 'application/json',
 'Dev-Mode': 'true'}
data = {'text': 'i feel stomach pain but no coughing today'}
r = requests.post(url, headers=headers, data=json.dumps(data))

Also worth noting, they have a python api that does all this for you.

• user123: @Brendam: Thanks for such sharp obeservation. I will take care this sort of problem in future

Answer 2

json_data = json.dumps(data) is not the correct way to prepare POST data.

You should use urllib.urlencode() to do the job:

import urllib
data = { "text": text }
req = urllib2.Request(self.url, urllib.urlencode(data), self.headers)
response = urllib2.urlopen(req).read()

Docs :

class urllib2.Request(url[, data][, headers][, origin_req_host][, unverifiable]) This class is an abstraction of a URL request.

data may be a string specifying additional data to send to the server, or None if no such data is needed. Currently HTTP requests are the only ones that use data; the HTTP request will be a POST instead of a GET when the data parameter is provided. data should be a buffer in the standard application/x-www-form-urlencoded format. The urllib.urlencode() function takes a mapping or sequence of 2-tuples and returns a string in this format.

• Shane: Have you double checked those parameters in headers? I guess there're some restrictions on server side.
• Shane: If still it doesn't work, you could use a local proxy(e.g. fiddler) to debug this request, and find out the difference between curl and python request