Python mechanize form authentication error

Question

I'm trying to automate form filling on Sharepoint site, but my Python script can't get passed this authentication box that pops up when you type in the url from below.

Auth Box

from base64 import b64encode
import mechanize
url = 'http://moss.micron.com/MFG/ProbeTest/Lists/Manufacturing%20Requests/AllItems.aspx'
username = 'username'
password = 'password'
# I have had to add a carriage return ('%s:%s\n'), but
# you may not have to.
b64login = b64encode('%s:%s' % (username, password))
br = mechanize.Browser()
br.addheaders.append( 
 ('Authorization', 'Basic %s' % b64login, )
)
br.open(url)!

This results in the following error:

Error

EDIT:

Here are the results of running wget on the requested page.

--2013年08月30日 11:16:17-- http://moss.micron.com/MFG/ProbeTest/Lists/Manufacturing%20Requests/AllItems.aspx
Resolving moss.micron.com... 137.201.88.118
Connecting to moss.micron.com|137.201.88.118|:80... connected.
HTTP request sent, awaiting response... 
 HTTP/1.1 401 Unauthorized
 Server: Microsoft-IIS/7.0
 WWW-Authenticate: Negotiate
 WWW-Authenticate: NTLM
 X-Powered-By: ASP.NET
 MicrosoftSharePointTeamServices: 12.0.0.6341
 Date: 2013年8月30日 17:16:17 GMT
 Connection: keep-alive
 Content-Length: 0
Authorization failed.

• László Papp: Please use error output, i.e. text, not screenshot for text error output...
• theAlse: @Mark Kennedy, did you manage to solve this issue? I am facing the exact same problem!

Answer 1

Your browser is respecting the robots.txt on your site disallowing it.

You can set mechanize.Browser to ignore robots.txt, prior to making the request via: br.set_handle_robots(False)

Alternately, edit your robots.txt to allow that sort of connection.

If you set a custom UserAgent header in your mechanize.Browser to allow you to filter for it.

See here for basic info about robots.txt.

• Mark Kennedy: After making that change I get a new error: HTTP Error 401: Unauthorized
• Mark Kennedy: I am certain my username/password combination is correct. Is this because I'm trying to connect to a Sharepoint site?
• DivinusVox: I presume it does have some sort of filtering against browsers attempting to access it without a UserAgent header.
• DivinusVox: Maybe try using the built in function for http auth? br.add_password('http://safe-site.domain', 'username', 'password')
• Mark Kennedy: @DivinuxVox I already tried that before. I also tried adding the following. Still get the same error br.addheaders(['User-agent', 'Mozilla/20.0)')]

Answer 2

If you can get to the site with a PC, download Fiddler2 which will allow you to see the transactions required when you log in.

Edit.. Ok. Obviously you have a PC.

• Mark Kennedy: I've run a wget command. The WWW-Authenticate values seem to jump at me.