Submitting Forms with Mechanize (Python)

Question

Well, I am trying to login to a site using Python and mechanize.

I've got the site opened:

site = br.open("http://example.com/login.php")

And I've got a list of the forms (with br.forms).

<GET http://example.com/search.php application/x-www-form-urlencoded
<HiddenControl(search=1) (readonly)>
...
<POST http://example.com/login.php application/x-www-form-urlencoded
<TextControl(username=)>
<PasswordControl(password=)>
<CheckboxControl(stay=[1])>
<SubmitControl(<None>=Log in) (readonly)>>

I've been trying to submit the username and password fields.

I tried doing it like this:

br.select_form(nr=0)
br.form["username"] = 'usernamehere'
br.form["password"] = 'passwordhere'
br.submit()

Then I realised that the forms I were trying to fill in weren't the first on the page, but changing the 0 didn't help with anything. What should I do for selecting the form on a page like this?

However! That is not the only problem.

When I run it, I get this error:

Traceback (most recent call last):
File "C:\Python26\login.py", line 34, in <module>
br.form["username"] = 'usernamehere'
...
ControlNotFoundError: no control matching name 'username'

How can I fix this? D: Or am I doing it totally wrong? If it's the latter, how would I go about doing it?

Answer 1

to select a form using its name you should use:

br.select_form(name="order")

what you are doing here:

br.form["username"] = 'usernamehere'

is trying to set a value to a control under the selected form, so when he can't find it, it throws the exception you are seeing.

• Matthew: br.select_form(name = "username") throws back an error: FormNotFoundError: no form matching name 'username' - same as before, but this time with the line I just added.
• tovmeod: are you sure you have the right name? try for form in br.forms(): print form
• Matthew: Positive. I checked br.forms() and the raw html.
• tovmeod: can you tell me what's the page? I may try having a look at it if you don't mind

Answer 2

You might have several issues

• if the form is generated through javascript, you can't solve it with mechanize - at least not in a straight forward way - in this case I recommend you to try and use selenium - you can try to look at the page HTML source - if you don't have the form there in pure html, it is pretty clear that it is inserted into DOM by javascript. Also, if the form is submitted through javascript, mechanize won't help you

• also, the form might not be on the first page - you might want to set mechanize to follow the redirects

• Matthew: Nope, it's not javascript. And I could access the form fine with urllib... I'd just rather use mechanize because I'm not sure how cookiejar works exactly and I need to stay logged in >_> And login.php is the form location. I am certain of this.