urllib2 data sending

Question

I've recently written this with help from SO. Now could someone please tell me how to make it actually log onto the board. It brings up everything just in a non logged in format.

import urllib2, re
import urllib, re
logindata = urllib.urlencode({'username': 'x', 'password': 'y'})
page = urllib2.urlopen("http://www.woarl.com/board/index.php", logindata)
pagesource = page.read()
print pagesource

• S.Lott: What's the status that comes back with the page? The page object is rather complex, and has status codes and headers. What are those?

Answer 1

Someone recently asked the same question you're asking. If you read through the answers to that question you'll see code examples showing you how to stay logged in while browsing a site in a Python script using only stuff in the standard library.

The accepted answer might not be as useful to you as this other answer, since the accepted answer deals with a specific problem involving redirection. However, I recommend reading through all of the answers regardless.

Answer 2

You probably want to look into preserving cookies from the server.

Pycurl or Mechanize will make this much easier for you

Answer 3

If actually look at the page, you see that the login link takes you to http://www.woarl.com/board/ucp.php?mode=login

That page has the login form and submits to http://www.woarl.com/board/ucp.php?mode=login again with POST.

You'll then have to extract the cookies that are probably set, and put those in a CookieJar or similar.

Answer 4

You probably want to create an opener with these handlers and apply it to urllib2. With these applied your cookies are handled and you'll be redirected, if server decides it wants you somewhere else.

# Create handlers
cookieHandler = urllib2.HTTPCookieProcessor() # Needed for cookie handling
redirectionHandler = urllib2.HTTPRedirectHandler() # needed for redirection (not needed for javascript redirect?)
# Create opener
opener = urllib2.build_opener(cookieHandler,redirectionHandler)
# Install the opener
urllib2.install_opener(opener)