fireprox
AWS API Gateway management tool for creating on the fly HTTP pass-through proxies for unique IP rotation
Top Related Projects
Incredibly fast crawler designed for OSINT.
A versatile and portable proxy for capturing, manipulating, and replaying HTTP/HTTPS traffic on the go.
The Social-Engineer Toolkit (SET) repository from TrustedSec - All new versions of SET will be deployed here.
Framework for Man-In-The-Middle attacks
An interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers.
Quick Overview
FireProx is a tool designed to create and manage AWS API Gateway proxies dynamically. It allows users to rotate IP addresses during web scraping, API testing, or other activities where IP rotation is beneficial. FireProx leverages AWS services to provide a scalable and cost-effective solution for proxy management.
Pros
- Easy IP rotation without managing proxy servers
- Leverages AWS infrastructure for scalability and reliability
- Cost-effective solution compared to traditional proxy services
- Provides detailed logging and management of proxy usage
Cons
- Requires an AWS account and associated costs
- Limited to AWS regions and their IP ranges
- May require additional setup for HTTPS support
- Learning curve for users unfamiliar with AWS services
Code Examples
# Create a new FireProx session
fp = FireProx(access_key='YOUR_AWS_ACCESS_KEY', secret_access_key='YOUR_AWS_SECRET_KEY')
# Create a new API proxy
api_id = fp.create_api('https://api.example.com')
print(f"API created with ID: {api_id}")
# List all existing API proxies
apis = fp.list_apis()
for api in apis:
print(f"API ID: {api['id']}, URL: {api['url']}")
# Delete an existing API proxy
fp.delete_api('API_ID')
print("API deleted successfully")
Getting Started
-
Install FireProx:
git clone https://github.com/ustayready/fireprox.git cd fireprox pip install -r requirements.txt
-
Set up AWS credentials:
- Create an IAM user with appropriate permissions
- Configure AWS CLI or set environment variables
-
Run FireProx:
python3 fire.py
-
Use FireProx commands to create, list, and manage API proxies:
python3 fire.py --command create --url https://api.example.com python3 fire.py --command list python3 fire.py --command delete --api-id YOUR_API_ID
Competitor Comparisons
Incredibly fast crawler designed for OSINT.
Pros of Photon
- Broader functionality: Photon is a multi-purpose OSINT tool for web crawling, information gathering, and reconnaissance
- More versatile: Can extract various types of data like emails, social media accounts, and metadata from websites
- Active development: Regular updates and improvements from the community
Cons of Photon
- More complex to use: Requires more setup and configuration compared to FireProx
- Potentially overwhelming: The extensive features might be unnecessary for simple proxy needs
- Higher resource usage: May consume more system resources due to its comprehensive functionality
Code Comparison
Photon:
import photon
photon.main(url, threads, depth, timeout)
FireProx:
from fireprox import FireProx
fp = FireProx(access_key, secret_key, region)
fp.create_api(url)
Summary
Photon is a feature-rich OSINT tool with broader capabilities, while FireProx focuses specifically on creating API gateways for IP rotation. Photon offers more versatility but may be more complex to use, whereas FireProx provides a simpler solution for proxy needs. The choice between the two depends on the specific requirements of the project and the user's familiarity with the tools.
A versatile and portable proxy for capturing, manipulating, and replaying HTTP/HTTPS traffic on the go.
Pros of Proxify
- More versatile: Supports HTTP, HTTPS, and WebSocket protocols
- Advanced features: Request/response modification, TLS inspection, and replay functionality
- Active development: Regular updates and improvements
Cons of Proxify
- More complex setup and configuration
- Steeper learning curve for users
- May be overkill for simple proxy needs
Code Comparison
Proxify (Go):
func main() {
options := &types.Options{}
flagSet := goflags.NewFlagSet()
flagSet.SetDescription(`Proxify is a Swiss Army knife Proxy tool.`)
flagSet.CreateGroup("input", "Input",
flagSet.StringVarP(&options.ListenAddr, "listen", "l", "127.0.0.1:8080", "listen address"),
)
// ... (additional configuration)
}
FireProx (Python):
def main(args):
if args.command == 'create':
create_api(args.url, args.region)
elif args.command == 'delete':
delete_api(args.api_id, args.region)
elif args.command == 'list':
list_apis(args.region)
# ... (additional command handling)
Both tools serve as proxies, but Proxify offers more advanced features and flexibility, while FireProx focuses on simplicity and AWS integration for creating API Gateway proxies. Proxify is written in Go and has a more complex setup, whereas FireProx is Python-based and easier to use for basic proxy needs.
The Social-Engineer Toolkit (SET) repository from TrustedSec - All new versions of SET will be deployed here.
Pros of Social-Engineer-Toolkit
- Comprehensive suite of social engineering tools
- Well-established project with a large user base
- Regularly updated and maintained
Cons of Social-Engineer-Toolkit
- More complex to use, requiring more technical knowledge
- Larger codebase, potentially harder to customize
- Primarily focused on social engineering attacks
Code Comparison
Social-Engineer-Toolkit:
# Example of a phishing attack setup
choice = raw_input(setprompt(["1"], "SET vector [phishing]"))
if choice == 'exit':
exit_set()
FireProx:
# Example of creating a new proxy
proxy = client.create_http_proxy(
Name=name,
TargetUrl=target,
RouteSelectionExpression="$request.header.x-my-header"
)
FireProx is a more focused tool for creating rotating proxies, while Social-Engineer-Toolkit offers a broader range of social engineering attack vectors. FireProx is simpler to use for its specific purpose, while Social-Engineer-Toolkit provides more comprehensive options for various attack scenarios. The code examples highlight the different focuses: FireProx on proxy creation and Social-Engineer-Toolkit on attack vector selection.
Framework for Man-In-The-Middle attacks
Pros of MITMf
- More comprehensive set of attack modules and plugins
- Supports a wider range of protocols and attack vectors
- Active community and regular updates
Cons of MITMf
- More complex setup and configuration
- Requires deeper networking knowledge to use effectively
- May be overkill for simple proxy needs
Code Comparison
MITMf example (config.py):
config = {
'sslstrip': True,
'hsts': False,
'dns_spoof': {
'google.com': '192.168.1.100'
}
}
FireProx example (fireprox.py):
@app.route('/', defaults={'path': ''})
@app.route('/<path:path>')
def proxy(path):
url = f"{target_url}/{path}"
resp = requests.request(
method=request.method,
url=url,
headers={key: value for (key, value) in request.headers if key != 'Host'},
data=request.get_data(),
cookies=request.cookies,
allow_redirects=False)
return (resp.content, resp.status_code, resp.headers.items())
MITMf offers more advanced features for complex man-in-the-middle attacks, while FireProx focuses on simple API proxying. MITMf requires more setup but provides greater flexibility, whereas FireProx is easier to use for basic proxying needs. The code examples show MITMf's configuration-based approach versus FireProx's straightforward request handling.
An interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers.
Pros of mitmproxy
- More comprehensive and feature-rich proxy tool
- Supports a wide range of protocols and use cases
- Active development with a large community
Cons of mitmproxy
- Steeper learning curve due to its complexity
- Requires more setup and configuration for specific tasks
Code Comparison
mitmproxy:
from mitmproxy import http
def request(flow: http.HTTPFlow) -> None:
if flow.request.host == "example.com":
flow.request.headers["Custom-Header"] = "Modified"
FireProx:
def lambda_handler(event, context):
host = event['headers'].get('Host', '')
if host == 'example.com':
event['headers']['Custom-Header'] = 'Modified'
return event
Summary
mitmproxy is a powerful, general-purpose proxy tool with extensive features for intercepting, modifying, and analyzing HTTP/HTTPS traffic. It offers a command-line interface, a web interface, and a Python API for scripting.
FireProx, on the other hand, is a specialized tool designed to create on-demand HTTP pass-through proxies using AWS API Gateway. It's more focused on providing dynamic IP rotation for web scraping and penetration testing.
While mitmproxy provides more flexibility and features, FireProx offers a simpler solution for specific use cases, particularly those involving IP rotation and AWS integration.
Convert
designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
FireProx
Overview
Being able to hide or continually rotate the source IP address when making web calls can be difficult or expensive. A number of tools have existed for some time but they were either limited with the number of IP addresses, were expensive, or required deployment of lots of VPS's. FireProx leverages the AWS API Gateway to create pass-through proxies that rotate the source IP address with every request! Use FireProx to create a proxy URL that points to a destination server and then make web requests to the proxy URL which returns the destination server response!
Brought to you by:
Maintainer
- Follow me on Twitter for more tips, tricks, and tools (or just to say hi)! (Mike Felch - @ustayready)
Benefits
-
Rotates IP address with every request
-
Configure separate regions
-
All HTTP methods supported
-
All parameters and URI's are passed through
-
Create, delete, list, or update proxies
-
Spoof X-Forwarded-For source IP header by requesting with an X-My-X-Forwarded-For header
Disclaimers
Source IP address is passed to the destination in the X-Forwarded-For header by AWS($100 to the first person to figure out how to strip it in the AWS config before it reaches the destination LOL!)- Thanks to
for the awesome X-Forwarded-For patch within 1 hour!
- I am not responsible if you don't abide by the robots.txt :)
- CloudFlare seems to sometimes detect X-Forwarded-For when blocking scrapers (NEED TO TEST W/ NEW PATCH)
- Use of this tool on systems other than those that you own are likely to violate the AWS Acceptable Use Policy and could potentially lead to termination or suspension of your AWS account. Further, even use of this tool on systems that you do own, or have explicit permission to perform penetration testing on, is subject to the AWS policy on penetration testing.
Credit
After releasing FireProx publicly, I learned two others were already using the AWS API Gateway technique. Researching the chain of events and having some great conversations, I came to the realization that the only reason I even knew about it was because of these people. I thought it would be cool to give them a few shout-outs and credit, follow these people -- they are awesome.
Credit goes to Ryan Hanson - @ryHanson who is the first known source of the API Gateway technique
Shout-out to Mike Hodges - @rmikehodges for making it public in at BlackHat Arsenal 2018
Major shout-out, once again, to my good friend Ralph May - @ralphte1 for introducing me to the technique awhile back.
Basic Usage
Requires AWS access key and secret access key or aws cli configured
usage: fire.py [-h] [--access_key ACCESS_KEY] [--secret_access_key SECRET_ACCESS_KEY] [--region REGION] [--command COMMAND] [--api_id API_ID] [--url URL]
FireProx API Gateway Manager
usage: fire.py [-h] [--profile_name PROFILE_NAME] [--access_key ACCESS_KEY] [--secret_access_key SECRET_ACCESS_KEY] [--session_token SESSION_TOKEN] [--region REGION] [--command COMMAND] [--api_id API_ID] [--url URL]
FireProx API Gateway Manager
optional arguments:
-h, --help show this help message and exit
--profile_name PROFILE_NAME
AWS Profile Name to store/retrieve credentials
--access_key ACCESS_KEY
AWS Access Key
--secret_access_key SECRET_ACCESS_KEY
AWS Secret Access Key
--session_token SESSION_TOKEN
AWS Session Token
--region REGION AWS Region
--command COMMAND Commands: list, create, delete, update
--api_id API_ID API ID
--url URL URL end-point
- Examples
- examples/google.py: Use a FireProx proxy to scrape Google search.
- examples/bing.py: Use a FireProx proxy to scrape Bing search.
Installation
You can install and run with the following command:
$ git clone https://github.com/ustayready/fireprox
$ cd fireprox
~/fireprox$ virtualenv -p python3 .
~/fireprox$ source bin/activate
(fireprox) ~/fireprox$ pip install -r requirements.txt
(fireprox) ~/fireprox$ python fire.py
Note that Python 3.6 is required.
Building a Docker image: (Currently does not work on Docker for Windows, possibly due to line endings in entrypoint.sh.)
$ git clone https://github.com/ustayready/fireprox
$ cd fireprox
$ docker build -t fireprox .
$ docker run --rm -it fireprox -h
Screenshots
Contributing
- Create an issue to discuss your idea
- Fork FireProx (https://github.com/ustayready/fireprox/fork)
- Create your feature branch (
git checkout -b my-new-feature
) - Commit your changes (
git commit -am 'Add some feature'
) - Push to the branch (
git push origin my-new-feature
) - Create a new Pull Request
Bug reports, feature requests and patches are welcome.
Top Related Projects
Incredibly fast crawler designed for OSINT.
A versatile and portable proxy for capturing, manipulating, and replaying HTTP/HTTPS traffic on the go.
The Social-Engineer Toolkit (SET) repository from TrustedSec - All new versions of SET will be deployed here.
Framework for Man-In-The-Middle attacks
An interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers.
Convert
designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot