Convert Figma logo to code with AI

tomnomnom logogf

A wrapper around grep, to help you grep for things

1,768
312
1,768
55

Top Related Projects

19,837

Fast and customizable vulnerability scanner based on simple YAML based DSL.

3,859

Fetch known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, and Common Crawl.

12,220

Fast web fuzzer written in Go

2,138

The Swiss Army knife for automated Web Application Testing

13,141

Most advanced XSS scanner.

GF Paterns For (ssrf,RCE,Lfi,sqli,ssti,idor,url redirection,debug_logic, interesting Subs) parameters grep

Quick Overview

GF (short for "Grep Find") is a wrapper around grep designed to help find patterns in files or piped data. It's particularly useful for security researchers and bug bounty hunters to quickly search through large datasets for specific patterns or potential vulnerabilities.

Pros

  • Comes with a set of pre-defined patterns for common security-related searches
  • Easily extensible with custom patterns
  • Lightweight and fast, built on top of grep
  • Integrates well with other command-line tools

Cons

  • Requires some familiarity with regular expressions for advanced usage
  • Limited to text-based searches, not suitable for binary data
  • May produce false positives, requiring manual verification
  • Lacks advanced features found in more comprehensive security tools

Code Examples

  1. Basic usage to search for URLs in a file:
gf urls file.txt
  1. Piping output from another command to search for IP addresses:
cat logfile.txt | gf ip
  1. Using a custom pattern file:
gf -f custom-pattern.json interesting-stuff

Getting Started

  1. Install GF:
go install github.com/tomnomnom/gf@latest
  1. Add GF to your PATH:
echo 'export PATH=$PATH:$HOME/go/bin' >> ~/.bashrc
source ~/.bashrc
  1. Clone the repository to get the default patterns:
git clone https://github.com/tomnomnom/gf.git ~/.gf
  1. Start using GF with pre-defined patterns:
gf ssrf example.txt

Competitor Comparisons

19,837

Fast and customizable vulnerability scanner based on simple YAML based DSL.

Pros of Nuclei

  • More comprehensive vulnerability scanning with customizable templates
  • Supports a wide range of protocols and technologies
  • Active development and community support

Cons of Nuclei

  • Steeper learning curve due to complex template system
  • Requires more system resources for large-scale scans

Code Comparison

Nuclei template example:

id: example-template
info:
  name: Example Template
  severity: low
requests:
  - method: GET
    path:
      - "{{BaseURL}}/example"
    matchers:
      - type: word
        words:
          - "Example Response"

gf pattern example:

{
  "flags": "-HnriE",
  "pattern": "example.*pattern"
}

Key Differences

  • Nuclei is a full-fledged vulnerability scanner, while gf is a wrapper for grep
  • Nuclei uses YAML templates for defining scans, gf uses JSON for pattern matching
  • Nuclei is more suitable for comprehensive security assessments, while gf excels at quick pattern searches in text data

Use Cases

  • Nuclei: Automated vulnerability scanning, security assessments, and bug bounty hunting
  • gf: Rapid pattern matching in log files, source code, or other text-based data

Community and Ecosystem

  • Nuclei has a larger community and more extensive documentation
  • gf is simpler and easier to integrate into existing workflows
3,859

Fetch known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, and Common Crawl.

Pros of gau

  • Focuses on URL discovery from various sources
  • Faster execution for large-scale URL gathering
  • Supports multiple output formats (JSON, plain text)

Cons of gau

  • Limited to URL discovery, lacks pattern matching capabilities
  • Does not provide built-in filtering options
  • Requires additional tools for advanced analysis of gathered URLs

Code Comparison

gf example:

cat subdomains.txt | gf xss

gau example:

gau example.com

Summary

gf and gau serve different purposes in the security toolchain. gf is a pattern matching tool for identifying potential vulnerabilities in text data, while gau focuses on discovering URLs from various sources. gf excels at filtering and analyzing existing data, whereas gau is better suited for initial URL discovery and enumeration. Both tools can be complementary in a security workflow, with gau providing input data for gf to analyze. The choice between them depends on the specific task at hand: URL discovery (gau) or pattern-based analysis (gf).

12,220

Fast web fuzzer written in Go

Pros of ffuf

  • Faster and more efficient for web fuzzing and directory discovery
  • Supports multiple HTTP methods and customizable output formats
  • Offers advanced features like recursion and custom filters

Cons of ffuf

  • Steeper learning curve due to more complex command-line options
  • Primarily focused on web fuzzing, less versatile for general-purpose pattern matching

Code comparison

ffuf:

ffuf -w wordlist.txt -u https://example.com/FUZZ -mc 200,301

gf:

cat file.txt | gf xss

Key differences

ffuf is a web fuzzing tool designed for discovering hidden files, directories, and vulnerabilities in web applications. It excels in speed and customization for HTTP-based tasks.

gf, on the other hand, is a wrapper around grep, focusing on pattern matching and extraction from various file types. It's more versatile for general-purpose text searching but less specialized for web fuzzing.

While ffuf is optimized for web-specific tasks, gf shines in quickly identifying patterns across multiple files or data streams. The choice between the two depends on the specific use case and the user's familiarity with command-line tools.

2,138

The Swiss Army knife for automated Web Application Testing

Pros of Jaeles

  • More comprehensive security scanning capabilities, including active vulnerability detection
  • Supports custom scripts and plugins for extended functionality
  • Offers a web interface for easier management and result visualization

Cons of Jaeles

  • Steeper learning curve due to more complex features and configuration options
  • Requires more system resources to run compared to the lightweight GF tool
  • May produce false positives in certain scenarios, requiring manual verification

Code Comparison

Jaeles (YAML configuration):

id: example-scan
info:
  name: Example Scan
  risk: Medium
requests:
  - method: GET
    path: /api/users
    detections:
      - >-
        StatusCode() == 200 && StringSearch("response", "admin")

GF (JSON pattern):

{
  "flags": "-HnriE",
  "pattern": "admin.*password"
}

While GF focuses on pattern matching in text files, Jaeles provides a more comprehensive framework for security scanning with customizable detection rules and request configurations.

13,141

Most advanced XSS scanner.

Pros of XSStrike

  • Specialized for XSS detection and exploitation
  • Advanced fuzzing and payload generation capabilities
  • Includes a powerful crawling engine for thorough scanning

Cons of XSStrike

  • More complex to use, requiring deeper understanding of XSS
  • Focused solely on XSS, less versatile for general-purpose scanning
  • May produce more false positives due to aggressive scanning

Code Comparison

XSStrike (Python):

def scan(url, params, headers, GET, delay, timeout):
    global globalVariables
    globalVariables = {}
    paramData = url.split('?')[1]
    paramData = paramData.split('&')
    for param in paramData:
        paramName = param.split('=')[0]
        canInject = '1' * len(paramName)
        globalVariables[paramName] = canInject

gf (Go):

func (p *Pattern) Match(line string) bool {
	if p.Regex != nil {
		return p.Regex.MatchString(line)
	}
	return strings.Contains(line, p.Pattern)
}

Summary

XSStrike is a specialized tool for XSS detection and exploitation, offering advanced features but with a steeper learning curve. gf, on the other hand, is a more versatile pattern-matching tool that can be used for various purposes beyond XSS detection. XSStrike's code focuses on parameter parsing and injection, while gf's code demonstrates simple pattern matching functionality.

GF Paterns For (ssrf,RCE,Lfi,sqli,ssti,idor,url redirection,debug_logic, interesting Subs) parameters grep

Pros of Gf-Patterns

  • Larger collection of pre-defined patterns for various security vulnerabilities and common web application issues
  • Includes patterns for specific technologies and frameworks, making it more comprehensive
  • Regular updates and contributions from the community, expanding its pattern library

Cons of Gf-Patterns

  • Less integrated with other tools compared to gf
  • May require more manual filtering due to the larger number of patterns
  • Potentially higher false-positive rate due to broader pattern matching

Code Comparison

gf example pattern:

{
  "flags": "-HnriE",
  "pattern": "aws_access_key_id|aws_secret_access_key"
}

Gf-Patterns example pattern:

{
  "flags": "-iE",
  "pattern": "(A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}"
}

Both repositories provide valuable pattern matching capabilities for security researchers and penetration testers. gf offers a more streamlined and integrated approach, while Gf-Patterns provides a wider range of pre-defined patterns for various scenarios. The choice between the two depends on the specific needs of the user and the level of customization required.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

gf

A wrapper around grep to avoid typing common patterns.

What? Why?

I use grep a lot. When auditing code bases, looking at the output of meg, or just generally dealing with large amounts of data. I often end up using fairly complex patterns like this one:

▶ grep -HnrE '(\$_(POST|GET|COOKIE|REQUEST|SERVER|FILES)|php://(input|stdin))' *

It's really easy to mess up when typing all of that, and it can be hard to know if you haven't got any results because there are non to find, or because you screwed up writing the pattern or chose the wrong flags.

I wrote gf to give names to the pattern and flag combinations I use all the time. So the above command becomes simply:

▶ gf php-sources

Pattern Files

The pattern definitions are stored in ~/.gf as little JSON files that can be kept under version control:

▶ cat ~/.gf/php-sources.json
{
    "flags": "-HnrE",
    "pattern": "(\\$_(POST|GET|COOKIE|REQUEST|SERVER|FILES)|php://(input|stdin))"
}

To help reduce pattern length and complexity a little, you can specify a list of multiple patterns too:

▶ cat ~/.gf/php-sources-multiple.json
{
    "flags": "-HnrE",
    "patterns": [
        "\\$_(POST|GET|COOKIE|REQUEST|SERVER|FILES)",
        "php://(input|stdin)"
    ]
}

There are some more example pattern files in the examples directory.

You can use the -save flag to create pattern files from the command line:

▶ gf -save php-serialized -HnrE '(a:[0-9]+:{|O:[0-9]+:"|s:[0-9]+:")'

Auto Complete

There's an auto-complete script included, so you can hit 'tab' to show you what your options are:

▶ gf <tab>
base64       debug-pages  fw           php-curl     php-errors   php-sinks    php-sources  sec          takeovers    urls

Bash

To get auto-complete working you need to source the gf-completion.bash file in your .bashrc or similar:

source ~/path/to/gf-completion.bash

Zsh

To get auto-complete working you need to enable autocomplete (not needed if you have oh-my-zsh) using autoload -U compaudit && compinit or by putting it into .zshrc

Then source the gf-completion.zsh file in your .zshrc or similar:

source ~/path/to/gf-completion.zsh

Note: if you're using oh-my-zsh or similar you may find that gf is an alias for git fetch. You can either alias the gf binary to something else, or unalias gf to remove the git fetch alias.

Using custom engines

There are some amazing code searching engines out there that can be a better replacement for grep. A good example is the silver searcher. It's faster (like way faster) and presents the results in a more visually digestible manner. In order to utilize a different engine, add engine: <other tool> to the relevant pattern file:

# Using the silver searcher instead of grep for the aws-keys pattern:
# 1. Adding "ag" engine
# 2. Removing the E flag which is irrelevant for ag
{
  "engine": "ag",
  "flags": "-Hanr",
  "pattern": "([^A-Z0-9]|^)(AKIA|A3T|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{12,}"
}
  • Note: Different engines use different flags, so in the example above, the flag E has to be removed from the aws-keys.json file in order for ag to successfully run.

Install

If you've got Go installed and configured you can install gf with:

▶ go get -u github.com/tomnomnom/gf

If you've installed using go get, you can enable auto-completion to your .bashrc like this:

▶ echo 'source $GOPATH/src/github.com/tomnomnom/gf/gf-completion.bash' >> ~/.bashrc

Note that you'll have to restart your terminal, or run source ~/.bashrc for the changes to take effect.

To get started quickly, you can copy the example pattern files to ~/.gf like this:

▶ cp -r $GOPATH/src/github.com/tomnomnom/gf/examples ~/.gf

My personal patterns that I've included as examples might not be very useful to you, but hopefully they're still a reasonable point of reference.

Contributing

I'd actually be most interested in new pattern files! If you've got something you regularly grep for then feel free to issue a PR to add new pattern files to the examples directory.

Bug fixes are also welcome as always :)