Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

0xlibless/FreeProxyScraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

History

3 Commits

Repository files navigation

FreeProxy scraper

FreeProxy is a Python tool designed to discover, extract, and verify free proxies at scale from GitHub repositories. It is capable of collecting up to +100,000 proxies and verifying them at extreme speeds.

Features

  • Smart Discovery: Automatically scans recent GitHub repositories for updated proxy lists.
  • Anti-Blocking Rotation: Implements a truly random, non-repeating rotation system to bypass GitHub API rate limits.
  • Ultra-Fast Verification: Uses multithreading (+50 parallel workers) to test thousands of proxies in minutes.
  • Data Cleaning: Automatically extracts IP:PORT format from messy text files.
  • Efficient Management: Manages proxy lists in-memory to prevent unnecessary disk I/O and overhead.

Installation

  1. Clone the repository:

    git clone https://github.com/0xlibfree/FreeProxy.git
    cd FreeProxy
  2. Install the required dependencies:

    pip install requests

How tu use?

Simply run the main script:

python main.py

The process works in two phases:

  1. Discovery Phase: The script searches for GitHub repositories created or updated in the last 24 hours and downloads every .txt file containing potential proxies. It stops once the PROXIES_LIMIT is reached.
  2. Verification Phase: It loads the collected list and performs mass-testing using 100 parallel threads. Validated proxies are saved to proxies.txt.

Configuration

You can adjust the following parameters inside main.py:

  • PROXIES_LIMIT: Maximum number of proxies to collect before starting the test phase (default: 100,000).
  • max_workers: Within the ThreadPoolExecutor block, you can adjust the number of parallel threads (default: 100).

About

Scrap github repos to obtain free proxies

Topics

Resources

Stars

Watchers

Forks

Languages

AltStyle によって変換されたページ (->オリジナル) /