Python requests sometimes fails with Errno -2

Question

I'm writing a scraper which runs perfectly on my Ubunutu 13.04 based laptop, however when running it on my RaspBMC based pi it throws the error below. Sometimes straight away, sometimes after running/scraping/downloading for an hour or two. I'm using requests to handle opening urls etc. Any help would be massively appreciated!I've included the entire traceback.

Traceback (most recent call last):
 File "trailer-dl.py", line 290, in <module>
 main()
 File "trailer-dl.py", line 282, in main
 if coming_soon_bool : makeSoup(base_url + '/coming-soon/')
 File "trailer-dl.py", line 267, in makeSoup
 makeNewSoup(current_link) # Start again with a new link to see if there is anything to grab
 File "trailer-dl.py", line 247, in makeNewSoup
 checkLink(new_current_link)
 File "trailer-dl.py", line 147, in checkLink
 downloadLink(active_link) 
 File "trailer-dl.py", line 165, in downloadLink
 r = requests.get(url, headers={"User-agent":"Quicktime"}, stream = True) # Identify ourselves as a quicktime player and open url as a stream
 File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 55, in get
 return request('get', url, **kwargs)
 File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
 return session.request(method=method, url=url, **kwargs)
 File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 383, in request
 resp = self.send(prep, **send_kwargs)
 File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 486, in send
 r = adapter.send(request, **kwargs)
 File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 378, in send
 raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='movietrailers.apple.com', port=80): Max retries exceeded with url: /movies/focus_features/badwords/badwords-tlr1_h720p.mov (Caused by <class 'socket.gaierror'>: [Errno -2] Name or service not known)

• Steve Robillard: it would help if you included your code.
• jason: Here is a link to it on github. github.com/nipponbob/XBMC-Trailer-Downloader

Answer 1

Since you are certain that it is a Pi specific problem and that the issue seems to arise from a network issue I'm going to assume a power supply issue.

I have has similar problems where the network IC drops the connection because of a spike in power draw in a (completely unrelated) part of the setup, usually this gives it's self away by the network status LEDs.

Best to ensure that you are using a supply rated for at least 800mA plus the rated load of any peripherals.

• jason: I'm using a 1.5A power supply with a powered usb hub. I have noticed that the pi drops it's wifi connection roughly every 4 to 6 days and requires a reboot. I would be inclined to rule out power supply issues, but I can't be 100% certain! I am going to try trapping the error and moving past it, and see if that produces the same error...
• user8894: What devices are you using with it, and what type of power supply is it, just because it has a sticker that says 1.5A on it does not mean it can deliver it without sufficient voltage drop. Is is possible to use Ethernet on the Pi for a while and see if this problem persists?
• jason: It's a mobile phone psu but you are right, I have just been assuming that it outputs a steady 1.5 amps. I don't have a multimeter handy to test it either... Good idea re the ethernet, I'll give that a try and see if there is any improvement! Thanks!