Scrapy/XPath help

Always Learning cbrowning at ou.edu
Fri Dec 21 16:21:08 EST 2012


Hello all. I'm new to Python, but have been playing around with it for a few weeks now, following tutorials, etc. I've spun off on my own and am trying to do some basic web scraping. I've used Firebug/View XPath in Firefox for some help with the XPaths, however, I still am receiving errors when I try to run this script. If you could help, it would be greatly appreciated!
from scrapy.spider import BaseSpider
from scrapy.selector import HtmlXPathSelector
from cbb_info.items import CbbInfoItem, Field
class GameInfoSpider(BaseSpider):
 name = "game_info"
 allowed_domains = ["www.sbrforum.com"]
 start_urls = [
 'http://www.sbrforum.com/betting-odds/ncaa-basketball/',
 ]
 def parse(self, response):
 hxs = HtmlXPathSelector(response)
 toplevels = hxs.select("//div[@class='eventLine-value']")
 items = []
 for toplevels in toplevels:
 item = CbbInfoItem()
 item ["teams"] = toplevels.select("/span[@class='team-name'/text()").extract()
 item ["lines"] = toplevels.select("/div[@rel='19']").extract()
 item.append(item)
 return items


More information about the Python-list mailing list

AltStyle によって変換されたページ (->オリジナル) /