ProxyBroker

Proxy [Finder | Checker | Server]. HTTP(S) & SOCKS :performing_arts:

4,023

1,130

4,023

104

View on GitHub

Top Related Projects

proxy-list

1,260

Lists of HTTP, SOCKS4, SOCKS5 proxies with geolocation info. Updated every hour.

PROXY-List

4,458

Get PROXY List that gets updated everyday

Quick Overview

ProxyBroker is an open-source Python tool designed to find and check public proxies from various sources. It can be used as a standalone application or integrated into other Python projects. ProxyBroker helps users find working proxies for tasks such as web scraping, anonymity, or bypassing geographical restrictions.

Pros

Supports multiple proxy protocols (HTTP, HTTPS, SOCKS4, SOCKS5)
Asynchronous architecture for efficient proxy checking
Can be used as both a command-line tool and a Python library
Provides detailed information about each proxy, including geolocation and anonymity level

Cons

Last updated in 2019, which may indicate lack of recent maintenance
Limited documentation and examples for advanced usage
May require additional dependencies for certain features
Performance can vary depending on the number of proxies being checked

Code Examples

Finding and printing proxies:

import asyncio
from proxybroker import Broker

async def show(proxies):
    while True:
        proxy = await proxies.get()
        if proxy is None:
            break
        print('Found proxy:', proxy)

proxies = asyncio.Queue()
broker = Broker(proxies)
tasks = asyncio.gather(
    broker.find(types=['HTTP', 'HTTPS'], limit=5),
    show(proxies)
)

loop = asyncio.get_event_loop()
loop.run_until_complete(tasks)

Using ProxyBroker as a proxy server:

import asyncio
from proxybroker import Broker

async def start_proxy_server():
    broker = Broker()
    await broker.serve(host='127.0.0.1', port=8888, types=['HTTP', 'HTTPS'])

if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(start_proxy_server())

Checking a specific proxy:

import asyncio
from proxybroker import Broker

async def check_proxy(proxy_str):
    broker = Broker()
    proxy = await broker.grab(proxy_str)
    if proxy:
        print(f"Proxy {proxy_str} is working")
    else:
        print(f"Proxy {proxy_str} is not working")

loop = asyncio.get_event_loop()
loop.run_until_complete(check_proxy('123.45.67.89:8080'))

Getting Started

To get started with ProxyBroker, follow these steps:

Install ProxyBroker using pip:
```
pip install proxybroker
```
Import the Broker class in your Python script:
```
from proxybroker import Broker
```

Create a Broker instance and use its methods to find or check proxies:

broker = Broker()
# Use broker methods like find(), serve(), or grab()

Run your script using an asyncio event loop:

import asyncio

async def main():
    # Your ProxyBroker code here

if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

For more detailed usage and examples, refer to the project's documentation on GitHub.

Competitor Comparisons

proxy-list

1,260

Lists of HTTP, SOCKS4, SOCKS5 proxies with geolocation info. Updated every hour.

Pros of proxy-list

Regularly updated list of free proxies
Simple to use, no complex setup required
Lightweight and easy to integrate into existing projects

Cons of proxy-list

Limited functionality compared to ProxyBroker
No built-in proxy checking or filtering capabilities
Lacks advanced features like geolocation or anonymity level detection

Code Comparison

proxy-list:

with open('proxies.txt', 'r') as file:
    proxies = file.readlines()

ProxyBroker:

async def show(proxies):
    while True:
        proxy = await proxies.get()
        if proxy is None: break
        print('Found proxy: %s' % proxy)

proxy-list provides a simple text file with proxy addresses, while ProxyBroker offers a more sophisticated asynchronous approach to proxy handling and filtering.

proxy-list is best suited for projects that need a quick and easy source of proxy addresses, while ProxyBroker is more appropriate for applications requiring advanced proxy management and filtering capabilities.

PROXY-List

4,458

Get PROXY List that gets updated everyday

Pros of PROXY-List

Simpler to use, providing ready-to-use proxy lists
Regularly updated with fresh proxies
Supports multiple proxy types (HTTP, SOCKS4, SOCKS5)

Cons of PROXY-List

Less flexible, as it doesn't offer proxy checking or filtering capabilities
No built-in functionality for integrating proxies into applications

Code Comparison

ProxyBroker:

async with Broker() as broker:
    proxies = await broker.find(types=['HTTP', 'HTTPS'], limit=10)
    async for proxy in proxies:
        print('Found proxy: %s' % proxy)

PROXY-List:

with open('http.txt', 'r') as file:
    proxies = file.readlines()
for proxy in proxies:
    print('Proxy:', proxy.strip())

Summary

ProxyBroker is a more comprehensive tool for finding, checking, and using proxies programmatically. It offers advanced features like proxy filtering, checking, and integration into applications. However, it requires more setup and coding knowledge.

PROXY-List, on the other hand, provides simple, ready-to-use proxy lists that are regularly updated. It's easier to use for basic proxy needs but lacks the advanced features and flexibility of ProxyBroker. The choice between the two depends on the specific requirements of your project and your technical expertise.

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

ProxyBroker

.. image:: https://img.shields.io/pypi/v/proxybroker.svg?style=flat-square :target: https://pypi.python.org/pypi/proxybroker/ .. image:: https://img.shields.io/travis/constverum/ProxyBroker.svg?style=flat-square :target: https://travis-ci.org/constverum/ProxyBroker .. image:: https://img.shields.io/pypi/wheel/proxybroker.svg?style=flat-square :target: https://pypi.python.org/pypi/proxybroker/ .. image:: https://img.shields.io/pypi/pyversions/proxybroker.svg?style=flat-square :target: https://pypi.python.org/pypi/proxybroker/ .. image:: https://img.shields.io/pypi/l/proxybroker.svg?style=flat-square :target: https://pypi.python.org/pypi/proxybroker/

ProxyBroker is an open source tool that asynchronously finds public proxies from multiple sources and concurrently checks them.

.. image:: https://raw.githubusercontent.com/constverum/ProxyBroker/master/docs/source/_static/index_find_example.gif

Features

Finds more than 7000 working proxies from ~50 sources.
Support protocols: HTTP(S), SOCKS4/5. Also CONNECT method to ports 80 and 23 (SMTP).
Proxies may be filtered by type, anonymity level, response time, country and status in DNSBL.
Work as a proxy server that distributes incoming requests to external proxies. With automatic proxy rotation.
All proxies are checked to support Cookies and Referer (and POST requests if required).
Automatically removes duplicate proxies.
Is asynchronous.

.. * Save found proxies to a file in custom format.

Requirements

Python 3.5 or higher
aiohttp <https://pypi.python.org/pypi/aiohttp>_
aiodns <https://pypi.python.org/pypi/aiodns>_
maxminddb <https://pypi.python.org/pypi/maxminddb>_

Installation

To install last stable release from pypi:

.. code-block:: bash

$ pip install proxybroker

The latest development version can be installed directly from GitHub:

.. code-block:: bash

$ pip install -U git+https://github.com/constverum/ProxyBroker.git

Usage

CLI Examples



Find
""""

Find and show 10 HTTP(S) proxies from United States with the high level of anonymity:

.. code-block:: bash

    $ proxybroker find --types HTTP HTTPS --lvl High --countries US --strict -l 10

.. image:: https://raw.githubusercontent.com/constverum/ProxyBroker/master/docs/source/_static/cli_find_example.gif


Grab
""""

Find and save to a file 10 US proxies (without a check):

.. code-block:: bash

    $ proxybroker grab --countries US --limit 10 --outfile ./proxies.txt

.. image:: https://raw.githubusercontent.com/constverum/ProxyBroker/master/docs/source/_static/cli_grab_example.gif


Serve
"""""

Run a local proxy server that distributes incoming requests to a pool
of found HTTP(S) proxies with the high level of anonymity:

.. code-block:: bash

    $ proxybroker serve --host 127.0.0.1 --port 8888 --types HTTP HTTPS --lvl High


.. image:: https://raw.githubusercontent.com/constverum/ProxyBroker/master/docs/source/_static/cli_serve_example.gif

| Run ``proxybroker --help`` for more information on the options available.
| Run ``proxybroker <command> --help`` for more information on a command.


Basic code example

Find and show 10 working HTTP(S) proxies:

.. code-block:: python

import asyncio
from proxybroker import Broker

async def show(proxies):
    while True:
        proxy = await proxies.get()
        if proxy is None: break
        print('Found proxy: %s' % proxy)

proxies = asyncio.Queue()
broker = Broker(proxies)
tasks = asyncio.gather(
    broker.find(types=['HTTP', 'HTTPS'], limit=10),
    show(proxies))

loop = asyncio.get_event_loop()
loop.run_until_complete(tasks)

More examples <https://proxybroker.readthedocs.io/en/latest/examples.html>_.

Documentation

https://proxybroker.readthedocs.io/

TODO

Check the ping, response time and speed of data transfer
Check site access (Google, Twitter, etc) and even your own custom URL's
Information about uptime
Checksum of data returned
Support for proxy authentication
Finding outgoing IP for cascading proxy
The ability to specify the address of the proxy without port (try to connect on defaulted ports)

Contributing

Fork it: https://github.com/constverum/ProxyBroker/fork
Create your feature branch: git checkout -b my-new-feature
Commit your changes: git commit -am 'Add some feature'
Push to the branch: git push origin my-new-feature
Submit a pull request!

License

Licensed under the Apache License, Version 2.0

This product includes GeoLite2 data created by MaxMind, available from http://www.maxmind.com <http://www.maxmind.com>_.

Top Related Projects

proxy-list

1,260

Lists of HTTP, SOCKS4, SOCKS5 proxies with geolocation info. Updated every hour.

PROXY-List

4,458

Get PROXY List that gets updated everyday

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot