homepage

I'm not sure what's the best approach here.
1. Avoid changes in the Lib, and document a work-around, which involves
 installing an opener with the specific User-agent. The draw-back is that it
 modifies the behaviour of urlopen() globally, so that change affects any
 other call to urllib.request.urlopen.
2. Revert to the old way, using an instance of a FancyURLopener (or URLopener),
 in the RobotFileParser class. This requires a modification of the Lib, but
 allows us to modify only the behaviour of that specific instance of
 RobotFileParser. The user could sub-class FancyURLopener, set the appropiate
 version string.
I attach a script, tested against the ``default`` branch of the mercurial
repository. It shows the work around for python3.3.