how to post form request on python

Question

I am trying to fill a form like that and submit it automaticly. To do that, I sniffed the packets while logging in.

POST /?pg=ogrgiris HTTP/1.1
Host: xxx.xxx.com
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded
Origin: http://xxx.xxx.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Safari/605.1.15
Referer: http://xxx.xxx.com/?pg=ogrgiris
Upgrade-Insecure-Requests: 1
DNT: 1
Content-Length: 60
Connection: close
seviye=700&ilkodu=34&kurumkodu=317381&ogrencino=40&isim=ahm

I repeated that packet by burp suite and saw works porperly. the response was the html of the member page.

Now I tried to do that on python. The code is below:

 import requests
 url = 'http://xxx.xxx.com/?pg=ogrgiris'
 headers = {'Host':'xxx.xxx.com',
'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Encoding':'gzip, deflate',
'Content-Type':'application/x-www-form-urlencoded',
'Referer':'http://xxx.xxx.com/?pg=ogrgiris',
'Content-Lenght':'60','Connection':'close'}
credentials = {'seviye': '700','ilkodu': '34','kurumkodu': '317381','ogrecino': '40','isim': 'ahm'}
r = requests.post(url,headers=headers, data=credentials)
print(r.content)

the problem is, that code prints the html of the login page even I send all of the credentials enough to log in. How can I get the member page? thanks.

Answer 1

If the POST request displays a page with the content you want, then the problem is only that you are sending data as JSON, not in "form" data format (application/x-www-form-urlencoded).

If a session is created at the request base and you have to make another request for the requested data, then you have to deal with cookies.

Problem with data format:

r = requests.post(url, headers=headers, data=credentials)

Kwarg json = creates a request body as follows:

{"ogrecino": "40", "ilkodu": "34", "isim": "ahm", "kurumkodu": "317381", "seviye": "700"}

While data= creates a request body like this:

seviye=700&ilkodu=34&kurumkodu=317381&ogrencino=40&isim=ahm

You can try https://httpbin.org:

from requests import post
msg = {"a": 1, "b": True}
print(post("https://httpbin.org/post", data=msg).json()) # Data as Form data, look at key `form`, it's object in JSON because it's Form data format
print(post("https://httpbin.org/post", json=msg).json()) # Data as json, look at key `data`, it's string

• user11321600: i updated the code as you told. but still loads the login page html :( could you please check it again?

Answer 2

If your goal is to replicate the sample request, you are missing a lot of the headers; this in particular is very important Content-Type: application/x-www-form-urlencoded because it will tell your HTTP client how to format/encode the payload.

Check the documentation for requests so see how these form posts can work.

• Sergio Pulgarin: stackoverflow.com/questions/12385179/…
• user11321600: i updated the code as you told. added all of the headers in burp suite but didnt work :( could you please check it again?
• Sergio Pulgarin: Try adding an argument files={'file': open( .. )} and see if requests sends the data in the correct format. Try an empty text file or something like that.
• Ivan Vinogradov: Content-Type header should not be set manually in requests. It is being set by lib implicitly based on payload type.
• user11321600: so do i need headers? @Ivan Vinogradov