Python Web Crawling

23 Web Crawling packages and projects

• Scrapy

  9.9 9.4 L4 Python

  Scrapy, a fast high-level web crawling & scraping framework for Python.
  scrapy logo

• pyspider

  9.5 0.0 L3 Python

  DISCONTINUED. A Powerful Spider(Web Crawler) System in Python.

InfluxDB – Built for High-Performance Time Series Workloads

InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

Promo www.influxdata.com

[画像:InfluxDB Logo]

• requests-html

  9.1 0.0 Python

  Pythonic HTML Parsing for HumansTM
  psf logo

• portia

  8.8 0.0 L2 Python

  Visual scraping for Scrapy
  scrapinghub logo

• MechanicalSoup

  7.7 5.6 L4 Python

  A Python library for automating interaction with websites.
  MechanicalSoup logo

• RoboBrowser

  7.2 0.0 L4 Python

  A simple, Pythonic library for browsing the web without a standalone web browser.
  jmcarp logo

• Grab

  6.4 9.2 L3 Python

  Web Scraping Framework
  lorien logo

• PSpider

  6.4 0.0 Python

  简单易用的Python爬虫框架,QQ交流群:597510560
  xianhu logo

• feedparser

  6.3 7.5 L3 Python

  Parse feeds in Python
  kurtmckee logo

• cola

  6.3 0.0 L3 Python

  DISCONTINUED. A high-level distributed crawling framework.
  qinxuye logo

• Scrapely

  6.1 0.0 HTML

  A pure-python HTML screen-scraping library
  scrapy logo

• gain

  6.0 0.0 Python

  Web crawling framework based on asyncio.
  elliotgao2 logo

• Google Search Results in Python

  4.2 0.0 Python

  Google Search Results via SERP API pip Python Package
  serpapi logo

• Sukhoi

  4.2 0.0 Python

  Minimalist and powerful Web Crawler.
  untwisted logo

• MSpider

  4.0 0.0 Python

  Spider
  manning23 logo

• reader

  3.5 9.4 Python

  A Python feed reader library.
  lemon24 logo

• spidy Web Crawler

  3.3 0.0 Python

  The simple, easy to use command line web crawler.
  rivermont logo

• Crawley

  2.7 0.0 Python

  Pythonic Crawling / Scraping Framework based on Non Blocking I/O operations.
  jmg logo

• brownant

  2.6 0.0 Python

  Brownant is a web data extracting framework.
  douban logo

• Demiurge

  2.2 0.0 L5 Python

  PyQuery-based scraping micro-framework.
  matiasb logo

• Pomp

  1.7 0.0 L5 Python

  Screen scraping and web crawling framework
  estin logo

• FastImage

  1.1 0.0 L4 Python

  Python library that finds the size / type of an image given its URI by fetching as little as needed
  bmuller logo

• Mariner

  0.5 0.0 Python

  This a is mirror of Gitlab repository. Open your issues and pull requests there.
  radek-sprta logo

* Code Quality Rankings and insights are calculated and provided by Lumnify.
They vary from L1 to L5 with "L5" being the highest.

Popular Comparisons

• pyspider
  vs
  Scrapy
• MechanicalSoup
  vs
  RoboBrowser
• requests-html
  vs
  Scrapy
• Scrapy
  vs
  MechanicalSoup
• Scrapy
  vs
  RoboBrowser