AI based web-wrapper for web-content-extraction
python data-mining library ai information-extraction web-scraping web-mining web-content-extractor web-data-extraction
-
Updated
Feb 6, 2023 - Python
#
web-data-extraction
Here are 28 public repositories matching this topic...
Language: All
Filter by language
Sort: Most stars
Sort options
The this.url class is designed to fetch and parse URL data, returning an object with structured information that can then be used for machine learning algorithms in a database or other storage.
web-scraping url-parsing metadata-extraction web-data-extraction neurons-me-ecosystem structured-url-data machine-learning-urls data-driven-web-analysis intelligent-link-processing ai-ready-url-processing
-
Updated
Aug 26, 2025 - JavaScript
Using LLMs and AI Browser Automation to Robustly Extract Web Data
nlp markdown crawler etl html-to-markdown data-engineering html-parser openai rss-feed webscraping data-pipeline web-data-extraction ai-agents article-extractor web-extraction rag llm google-gemini llm-scraper llm-extraction
-
Updated
Aug 7, 2025 - TypeScript
Quick guide with code example how to use Java for web scraping
-
Updated
Dec 18, 2024
GNewsScraper is a TypeScript package that scrapes article data from Google News based on a keyword or phrase. It returns the results as an array of JSON objects, making it convenient to access and use the scraped information
typescript web-scraping json-parsing web-crawling google-news data-scraping google-news-scraper web-data-extraction web-automation keyword-search gnews news-scraping gnews-api article-extraction gnews-scraper
-
Updated
Aug 19, 2023 - TypeScript
AI-based web extractor
-
Updated
Feb 25, 2023 - Python
An API wrapper for Scrappey.com written in Node.js (cloudflare bypass & solver)
web-scraping data-extraction web-data-extraction scraping-framework scraping-tool cloudflare-bypass web-scraping-solution cloudflare-solver api-scraping scraping-solution website-data-extraction scraping-library cloudflare-anti-bot scraping-service data-scraping-tool website-scraping-tool turnstile-solver
-
Updated
Jan 10, 2024 - JavaScript
Open-source web crawler
python url html open-source website opensource links web-crawler urls free data-extraction webcrawler web-crawling web-data-extraction urllib web-crawler-python
-
Updated
Jul 21, 2018 - Python
A pipeline to scrape, extract, and analyze book data from web pages to insights.
python books jupyter-notebook eda data-visualization web-scraping data-analysis html-parsing beautifulsoup csv-export data-pipeline web-data-extraction data-science-project project-portfolio book-dataset
-
Updated
Aug 4, 2025 - HTML
The Tableau Web Data Connector for Facebook Insights API
-
Updated
Jun 26, 2017 - JavaScript
Java Framework which is used by the Web Data Commons project to extract Microdata, Microformats and RDFa data, Web graphs, and HTML tables from the web crawls provided by the Common Crawl Foundation.
-
Updated
Dec 13, 2022 - Java
RealShotPDF is a Chrome extension designed to simplify the process of creating PDF documents from web content. The extension allows users to navigate through selected webpages, parse and display links in a tree view, and generate PDFs for the chosen pages. It operates locally without sending any data to external servers.
chrome-extension pdf angular web-scraping knowledgebase browser-extension knowledge-base pdf-generator pdf-generation web-crawling pdf-merger web-data-extraction pdf-downloader data-preservation ai-assistant webpage-to-pdf link-parsing gpt-integration web-content-capture local-data-processing
-
Updated
Mar 1, 2024 - TypeScript
Lightfeed SDK to search and filter web data
crawler etl extract data-engineering business-intelligence data-extraction data-integration structured-data knowledge-base webscraping data-pipeline web-data-extraction ai-agents rag vector-database web-data-management llm embedding-search llm-scraper llm-extraction
-
Updated
Jun 7, 2025 - Python
This repository contains the code and data download links to reproduce the building process of the 2021 Schema.org Table Corpus.
-
Updated
May 12, 2021 - Python
Get and process multiple resources from web, using asyncio (aiohttp) to fetch the data and multiprocessing/multithreading for processing it.
-
Updated
Mar 4, 2021 - Python
A web data extraction library written in golang.
-
Updated
Apr 16, 2025 - Go
This repository contains the source files of the Web Data Commons website and is used to maintain the site. The Web Data Commons project extracts structured data from the Common Crawl
-
Updated
Jan 13, 2025 - HTML
Python-based desktop app for effortless web scraping
desktop-app python cross-platform pyqt5 web-scraping xpath web-data-extraction browsing web-pages user-friendly-interface
-
Updated
Jun 26, 2023 - Python
Metadata extractor for the sprawling web ⚙️
-
Updated
Jan 8, 2023 - TypeScript
Improve this page
Add a description, image, and links to the web-data-extraction topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the web-data-extraction topic, visit your repo's landing page and select "manage topics."