Top Related Projects
Fast and customizable vulnerability scanner based on simple YAML based DSL.
Fetch known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, and Common Crawl.
Fast web fuzzer written in Go
The Swiss Army knife for automated Web Application Testing
Most advanced XSS scanner.
GF Paterns For (ssrf,RCE,Lfi,sqli,ssti,idor,url redirection,debug_logic, interesting Subs) parameters grep
Quick Overview
GF (short for "Grep Find") is a wrapper around grep designed to help find patterns in files or piped data. It's particularly useful for security researchers and bug bounty hunters to quickly search through large datasets for specific patterns or potential vulnerabilities.
Pros
- Comes with a set of pre-defined patterns for common security-related searches
- Easily extensible with custom patterns
- Lightweight and fast, built on top of grep
- Integrates well with other command-line tools
Cons
- Requires some familiarity with regular expressions for advanced usage
- Limited to text-based searches, not suitable for binary data
- May produce false positives, requiring manual verification
- Lacks advanced features found in more comprehensive security tools
Code Examples
- Basic usage to search for URLs in a file:
gf urls file.txt
- Piping output from another command to search for IP addresses:
cat logfile.txt | gf ip
- Using a custom pattern file:
gf -f custom-pattern.json interesting-stuff
Getting Started
- Install GF:
go install github.com/tomnomnom/gf@latest
- Add GF to your PATH:
echo 'export PATH=$PATH:$HOME/go/bin' >> ~/.bashrc
source ~/.bashrc
- Clone the repository to get the default patterns:
git clone https://github.com/tomnomnom/gf.git ~/.gf
- Start using GF with pre-defined patterns:
gf ssrf example.txt
Competitor Comparisons
Fast and customizable vulnerability scanner based on simple YAML based DSL.
Pros of Nuclei
- More comprehensive vulnerability scanning with customizable templates
- Supports a wide range of protocols and technologies
- Active development and community support
Cons of Nuclei
- Steeper learning curve due to complex template system
- Requires more system resources for large-scale scans
Code Comparison
Nuclei template example:
id: example-template
info:
name: Example Template
severity: low
requests:
- method: GET
path:
- "{{BaseURL}}/example"
matchers:
- type: word
words:
- "Example Response"
gf pattern example:
{
"flags": "-HnriE",
"pattern": "example.*pattern"
}
Key Differences
- Nuclei is a full-fledged vulnerability scanner, while gf is a wrapper for grep
- Nuclei uses YAML templates for defining scans, gf uses JSON for pattern matching
- Nuclei is more suitable for comprehensive security assessments, while gf excels at quick pattern searches in text data
Use Cases
- Nuclei: Automated vulnerability scanning, security assessments, and bug bounty hunting
- gf: Rapid pattern matching in log files, source code, or other text-based data
Community and Ecosystem
- Nuclei has a larger community and more extensive documentation
- gf is simpler and easier to integrate into existing workflows
Fetch known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, and Common Crawl.
Pros of gau
- Focuses on URL discovery from various sources
- Faster execution for large-scale URL gathering
- Supports multiple output formats (JSON, plain text)
Cons of gau
- Limited to URL discovery, lacks pattern matching capabilities
- Does not provide built-in filtering options
- Requires additional tools for advanced analysis of gathered URLs
Code Comparison
gf example:
cat subdomains.txt | gf xss
gau example:
gau example.com
Summary
gf and gau serve different purposes in the security toolchain. gf is a pattern matching tool for identifying potential vulnerabilities in text data, while gau focuses on discovering URLs from various sources. gf excels at filtering and analyzing existing data, whereas gau is better suited for initial URL discovery and enumeration. Both tools can be complementary in a security workflow, with gau providing input data for gf to analyze. The choice between them depends on the specific task at hand: URL discovery (gau) or pattern-based analysis (gf).
Fast web fuzzer written in Go
Pros of ffuf
- Faster and more efficient for web fuzzing and directory discovery
- Supports multiple HTTP methods and customizable output formats
- Offers advanced features like recursion and custom filters
Cons of ffuf
- Steeper learning curve due to more complex command-line options
- Primarily focused on web fuzzing, less versatile for general-purpose pattern matching
Code comparison
ffuf:
ffuf -w wordlist.txt -u https://example.com/FUZZ -mc 200,301
gf:
cat file.txt | gf xss
Key differences
ffuf is a web fuzzing tool designed for discovering hidden files, directories, and vulnerabilities in web applications. It excels in speed and customization for HTTP-based tasks.
gf, on the other hand, is a wrapper around grep, focusing on pattern matching and extraction from various file types. It's more versatile for general-purpose text searching but less specialized for web fuzzing.
While ffuf is optimized for web-specific tasks, gf shines in quickly identifying patterns across multiple files or data streams. The choice between the two depends on the specific use case and the user's familiarity with command-line tools.
The Swiss Army knife for automated Web Application Testing
Pros of Jaeles
- More comprehensive security scanning capabilities, including active vulnerability detection
- Supports custom scripts and plugins for extended functionality
- Offers a web interface for easier management and result visualization
Cons of Jaeles
- Steeper learning curve due to more complex features and configuration options
- Requires more system resources to run compared to the lightweight GF tool
- May produce false positives in certain scenarios, requiring manual verification
Code Comparison
Jaeles (YAML configuration):
id: example-scan
info:
name: Example Scan
risk: Medium
requests:
- method: GET
path: /api/users
detections:
- >-
StatusCode() == 200 && StringSearch("response", "admin")
GF (JSON pattern):
{
"flags": "-HnriE",
"pattern": "admin.*password"
}
While GF focuses on pattern matching in text files, Jaeles provides a more comprehensive framework for security scanning with customizable detection rules and request configurations.
Most advanced XSS scanner.
Pros of XSStrike
- Specialized for XSS detection and exploitation
- Advanced fuzzing and payload generation capabilities
- Includes a powerful crawling engine for thorough scanning
Cons of XSStrike
- More complex to use, requiring deeper understanding of XSS
- Focused solely on XSS, less versatile for general-purpose scanning
- May produce more false positives due to aggressive scanning
Code Comparison
XSStrike (Python):
def scan(url, params, headers, GET, delay, timeout):
global globalVariables
globalVariables = {}
paramData = url.split('?')[1]
paramData = paramData.split('&')
for param in paramData:
paramName = param.split('=')[0]
canInject = '1' * len(paramName)
globalVariables[paramName] = canInject
gf (Go):
func (p *Pattern) Match(line string) bool {
if p.Regex != nil {
return p.Regex.MatchString(line)
}
return strings.Contains(line, p.Pattern)
}
Summary
XSStrike is a specialized tool for XSS detection and exploitation, offering advanced features but with a steeper learning curve. gf, on the other hand, is a more versatile pattern-matching tool that can be used for various purposes beyond XSS detection. XSStrike's code focuses on parameter parsing and injection, while gf's code demonstrates simple pattern matching functionality.
GF Paterns For (ssrf,RCE,Lfi,sqli,ssti,idor,url redirection,debug_logic, interesting Subs) parameters grep
Pros of Gf-Patterns
- Larger collection of pre-defined patterns for various security vulnerabilities and common web application issues
- Includes patterns for specific technologies and frameworks, making it more comprehensive
- Regular updates and contributions from the community, expanding its pattern library
Cons of Gf-Patterns
- Less integrated with other tools compared to gf
- May require more manual filtering due to the larger number of patterns
- Potentially higher false-positive rate due to broader pattern matching
Code Comparison
gf example pattern:
{
"flags": "-HnriE",
"pattern": "aws_access_key_id|aws_secret_access_key"
}
Gf-Patterns example pattern:
{
"flags": "-iE",
"pattern": "(A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}"
}
Both repositories provide valuable pattern matching capabilities for security researchers and penetration testers. gf offers a more streamlined and integrated approach, while Gf-Patterns provides a wider range of pre-defined patterns for various scenarios. The choice between the two depends on the specific needs of the user and the level of customization required.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
gf
A wrapper around grep to avoid typing common patterns.
What? Why?
I use grep a lot. When auditing code bases, looking at the output of meg, or just generally dealing with large amounts of data. I often end up using fairly complex patterns like this one:
ⶠgrep -HnrE '(\$_(POST|GET|COOKIE|REQUEST|SERVER|FILES)|php://(input|stdin))' *
It's really easy to mess up when typing all of that, and it can be hard to know if you haven't got any results because there are non to find, or because you screwed up writing the pattern or chose the wrong flags.
I wrote gf
to give names to the pattern and flag combinations I use all the time. So the above command
becomes simply:
ⶠgf php-sources
Pattern Files
The pattern definitions are stored in ~/.gf
as little JSON files that can be kept under version control:
ⶠcat ~/.gf/php-sources.json
{
"flags": "-HnrE",
"pattern": "(\\$_(POST|GET|COOKIE|REQUEST|SERVER|FILES)|php://(input|stdin))"
}
To help reduce pattern length and complexity a little, you can specify a list of multiple patterns too:
ⶠcat ~/.gf/php-sources-multiple.json
{
"flags": "-HnrE",
"patterns": [
"\\$_(POST|GET|COOKIE|REQUEST|SERVER|FILES)",
"php://(input|stdin)"
]
}
There are some more example pattern files in the examples
directory.
You can use the -save
flag to create pattern files from the command line:
ⶠgf -save php-serialized -HnrE '(a:[0-9]+:{|O:[0-9]+:"|s:[0-9]+:")'
Auto Complete
There's an auto-complete script included, so you can hit 'tab' to show you what your options are:
ⶠgf <tab>
base64 debug-pages fw php-curl php-errors php-sinks php-sources sec takeovers urls
Bash
To get auto-complete working you need to source
the gf-completion.bash
file in your .bashrc
or similar:
source ~/path/to/gf-completion.bash
Zsh
To get auto-complete working you need to enable autocomplete (not needed if you have oh-my-zsh) using autoload -U compaudit && compinit
or by putting it into .zshrc
Then source
the gf-completion.zsh
file in your .zshrc
or similar:
source ~/path/to/gf-completion.zsh
Note: if you're using oh-my-zsh or similar you may find that gf
is an alias for git fetch
. You can either
alias the gf binary to something else, or unalias gf
to remove the git fetch
alias.
Using custom engines
There are some amazing code searching engines out there that can be a better replacement for grep.
A good example is the silver searcher.
It's faster (like way faster) and presents the results in a more visually digestible manner.
In order to utilize a different engine, add engine: <other tool>
to the relevant pattern file:
# Using the silver searcher instead of grep for the aws-keys pattern:
# 1. Adding "ag" engine
# 2. Removing the E flag which is irrelevant for ag
{
"engine": "ag",
"flags": "-Hanr",
"pattern": "([^A-Z0-9]|^)(AKIA|A3T|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{12,}"
}
- Note: Different engines use different flags, so in the example above, the flag
E
has to be removed from theaws-keys.json
file in order for ag to successfully run.
Install
If you've got Go installed and configured you can install gf
with:
ⶠgo get -u github.com/tomnomnom/gf
If you've installed using go get
, you can enable auto-completion to your .bashrc
like this:
ⶠecho 'source $GOPATH/src/github.com/tomnomnom/gf/gf-completion.bash' >> ~/.bashrc
Note that you'll have to restart your terminal, or run source ~/.bashrc
for the changes to
take effect.
To get started quickly, you can copy the example pattern files to ~/.gf
like this:
ⶠcp -r $GOPATH/src/github.com/tomnomnom/gf/examples ~/.gf
My personal patterns that I've included as examples might not be very useful to you, but hopefully they're still a reasonable point of reference.
Contributing
I'd actually be most interested in new pattern files! If you've got something you regularly grep for then feel free to issue a PR to add new pattern files to the examples directory.
Bug fixes are also welcome as always :)
Top Related Projects
Fast and customizable vulnerability scanner based on simple YAML based DSL.
Fetch known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, and Common Crawl.
Fast web fuzzer written in Go
The Swiss Army knife for automated Web Application Testing
Most advanced XSS scanner.
GF Paterns For (ssrf,RCE,Lfi,sqli,ssti,idor,url redirection,debug_logic, interesting Subs) parameters grep
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot