Convert Figma logo to code with AI

redhuntlabs logoAwesome-Asset-Discovery

List of Awesome Asset Discovery Resources

1,949
324
1,949
11

Top Related Projects

Community curated list of templates for the nuclei engine to find security vulnerabilities.

Find domains and subdomains related to a given domain

11,780

In-depth attack surface mapping and asset discovery

10,895

Incredibly fast crawler designed for OSINT.

This script is intended to automate your reconnaissance process in an organized fashion

3,859

Fetch known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, and Common Crawl.

Quick Overview

Awesome-Asset-Discovery is a curated list of resources and tools for asset discovery in cybersecurity. It provides a comprehensive collection of tools, techniques, and methodologies for identifying and mapping digital assets within an organization's network or on the internet. This repository serves as a valuable reference for security professionals, penetration testers, and researchers.

Pros

  • Extensive collection of asset discovery tools and resources
  • Well-organized and categorized for easy navigation
  • Regularly updated with new tools and techniques
  • Community-driven with contributions from security experts

Cons

  • May overwhelm beginners due to the large number of tools listed
  • Some listed tools may become outdated or discontinued over time
  • Lacks detailed explanations or comparisons of the listed tools
  • Primarily focuses on listing tools rather than providing in-depth tutorials

Note: As this is not a code library but a curated list of resources, there are no code examples or getting started instructions to provide.

Competitor Comparisons

Community curated list of templates for the nuclei engine to find security vulnerabilities.

Pros of nuclei-templates

  • Extensive collection of ready-to-use templates for vulnerability scanning
  • Regularly updated with new templates for emerging vulnerabilities
  • Supports a wide range of protocols and technologies

Cons of nuclei-templates

  • Requires the Nuclei engine to run, adding complexity
  • May generate false positives if not properly configured
  • Focused solely on vulnerability detection, not broader asset discovery

Code Comparison

While a direct code comparison isn't applicable due to the different nature of these repositories, here's a brief example of how they differ in content:

Awesome-Asset-Discovery (README.md):

## DNS
- [Amass](https://github.com/OWASP/Amass)
- [subfinder](https://github.com/projectdiscovery/subfinder)

nuclei-templates (example template):

id: cve-2021-44228-log4j-rce
info:
  name: Apache Log4j RCE
  severity: critical
http:
  - method: GET
    path:
      - "{{BaseURL}}/?x=${jndi:ldap://{{interactsh-url}}}"

Awesome-Asset-Discovery is a curated list of tools, while nuclei-templates contains actual scanning templates. The former aids in tool discovery, while the latter provides ready-to-use content for vulnerability scanning.

Find domains and subdomains related to a given domain

Pros of assetfinder

  • Lightweight and focused tool specifically for subdomain enumeration
  • Fast execution and efficient resource usage
  • Easy to integrate into existing workflows and scripts

Cons of assetfinder

  • Limited to subdomain discovery, lacking broader asset discovery features
  • May miss some assets that require more advanced discovery techniques
  • Less comprehensive compared to a curated list of multiple tools

Code Comparison

assetfinder:

func main() {
    domains := make(chan string, 100)
    go func() {
        sc := bufio.NewScanner(os.Stdin)
        for sc.Scan() {
            domains <- sc.Text()
        }
        close(domains)
    }()
    // ... (processing logic)
}

Awesome-Asset-Discovery:

## DNS
- [Amass](https://github.com/OWASP/Amass)
- [DNSRecon](https://github.com/darkoperator/dnsrecon)
- [Subfinder](https://github.com/projectdiscovery/subfinder)

## Web
- [Gospider](https://github.com/jaeles-project/gospider)
- [Hakrawler](https://github.com/hakluke/hakrawler)

The code comparison highlights the difference in approach. assetfinder is a Go-based tool with specific implementation for subdomain discovery, while Awesome-Asset-Discovery is a curated list of various asset discovery tools and resources, providing a broader scope but requiring manual integration of individual tools.

11,780

In-depth attack surface mapping and asset discovery

Pros of Amass

  • Active tool for network mapping and external asset discovery
  • Performs DNS enumeration, web scraping, and uses various data sources
  • Offers both CLI and GUI interfaces for flexibility

Cons of Amass

  • Steeper learning curve due to its extensive features and options
  • May generate more noise or false positives compared to curated lists
  • Requires more system resources and time to run comprehensive scans

Code Comparison

Amass (example usage):

amass enum -d example.com -o output.txt

Awesome-Asset-Discovery (example entry):

- [Amass](https://github.com/OWASP/Amass) - In-depth Attack Surface Mapping and Asset Discovery

Summary

Amass is an active, feature-rich tool for comprehensive asset discovery, while Awesome-Asset-Discovery is a curated list of asset discovery resources. Amass offers hands-on scanning capabilities but requires more expertise and resources. Awesome-Asset-Discovery provides a quick reference to various tools and techniques, including Amass itself, making it more accessible for beginners or those seeking an overview of available options.

10,895

Incredibly fast crawler designed for OSINT.

Pros of Photon

  • Active tool for web reconnaissance and information gathering
  • Provides automated crawling and data extraction capabilities
  • Offers multiple output formats for easy integration with other tools

Cons of Photon

  • Limited to web-based asset discovery
  • May require more setup and configuration compared to a curated list
  • Potential for false positives or incomplete results depending on target website structure

Code Comparison

Photon:

def photon(url, level, threadCount, delay, timeout, headers, cookie):
    # ... (initialization code)
    for url in urls:
        # ... (crawling and extraction logic)
    # ... (output generation)

Awesome-Asset-Discovery:

## DNS
- [Amass](https://github.com/OWASP/Amass)
- [subfinder](https://github.com/projectdiscovery/subfinder)

## Web
- [Photon](https://github.com/s0md3v/Photon)
- [gospider](https://github.com/jaeles-project/gospider)

Photon is an active tool for web reconnaissance, while Awesome-Asset-Discovery is a curated list of various asset discovery tools. Photon offers automated crawling and data extraction, but is limited to web-based assets. Awesome-Asset-Discovery provides a comprehensive collection of tools for different asset types, but requires manual selection and setup of individual tools.

This script is intended to automate your reconnaissance process in an organized fashion

Pros of lazyrecon

  • Automated reconnaissance tool that combines multiple techniques
  • Provides a streamlined workflow for bug bounty hunters
  • Generates organized output reports for easy analysis

Cons of lazyrecon

  • Limited to specific reconnaissance tasks compared to Awesome-Asset-Discovery's comprehensive list
  • May require more setup and dependencies
  • Less frequently updated compared to Awesome-Asset-Discovery

Code Comparison

lazyrecon:

#!/bin/bash
domain=$1
if [ -z "$domain" ]; then
    echo "Usage: $0 <domain>"
    exit 1
fi

Awesome-Asset-Discovery:

## DNS
* [Amass](https://github.com/OWASP/Amass)
* [Subfinder](https://github.com/projectdiscovery/subfinder)
* [Knockpy](https://github.com/guelfoweb/knock)

The code comparison shows that lazyrecon is a bash script for automated reconnaissance, while Awesome-Asset-Discovery is a curated list of tools and resources in Markdown format. lazyrecon provides a ready-to-use script, whereas Awesome-Asset-Discovery offers a comprehensive collection of tools for users to choose from based on their specific needs.

3,859

Fetch known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, and Common Crawl.

Pros of gau

  • Focused tool for URL discovery from various sources
  • Fast and efficient, designed for high-performance scanning
  • Easy to use with simple command-line interface

Cons of gau

  • Limited to URL discovery, lacks comprehensive asset discovery features
  • May require additional tools for complete asset enumeration
  • Less extensive documentation compared to Awesome-Asset-Discovery

Code Comparison

gau:

echo example.com | gau
gau example.com
cat domains.txt | gau --threads 5

Awesome-Asset-Discovery:

# No specific code, as it's a curated list of tools and resources
# Example usage might involve running multiple tools listed in the repository

Key Differences

  • Awesome-Asset-Discovery is a curated list of various asset discovery tools and resources, while gau is a specific tool for URL discovery
  • gau focuses on speed and efficiency for URL enumeration, whereas Awesome-Asset-Discovery provides a comprehensive overview of multiple asset discovery techniques
  • Awesome-Asset-Discovery offers a broader range of tools and methodologies, while gau specializes in one specific aspect of asset discovery

Use Cases

  • Use gau when you need quick and efficient URL discovery for a target domain
  • Refer to Awesome-Asset-Discovery when you require a comprehensive approach to asset discovery or need to explore various tools and techniques for different scenarios

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

Awesome Asset Discovery

Asset Discovery is the initial phase of any security assessment engagement, be it offensive or defensive. With the evolution of information technology, the scope and definition of assets has also evolved.

Earlier the servers, workstations and websites were primary IT assets of an organization, but today this definition is very limiting and should include anything and everything an organization and its entities has their data on (knowingly or unknowingly). The scope of ownership could differ, but it does not limit the attack surface, for example if an organization puts out open source code on Github, they are not the owner of Github but of the data they put under their repositories. In a scenario where some organization secret has been put on this Github account, it could pose a threat equal or more than running a vulnerable service.

We have explored this aspect of assets in our blog post here.

Through this repository, we want to put out a list of curated resources which help during asset discovery phase of a security assessment engagement. We welcome suggestions and contributions from the community in terms of resources as well as categories.

To know more about our Attack Surface Management platform, check out NVADR.

Contents

↑Content Discovery

  • rustbuster: Files, directories and vhost buster written in Rust.

↑IP Address Discovery

  • Mxtoolbox: Bulk Domain/IP lookup tool
  • Domaintoipconverter: Bulk domain to IP converter
  • Massdns: A DNS resolver utility for bulk lookups
  • Googleapps Dig: Online Dig tool by Google
  • DataSploit (IP Address Modules): An OSINT Framework to perform various recon techniques
  • Domain Dossier: Investigate domains and IP addresses
  • Bgpview: Search ASN, IPv4/IPv6 or resource name
  • Hurricane Electric BGP Toolkit: Keyword to ASN lookup
  • Viewdns: Multiple domain/IP tools
  • Ultratools ipv6Info: Multiple information related to IPv6 address
  • Whois: Command line utility usually used to find information about registered users/assignees of an Internet resource.
  • ICANN Whois: Whois service by Internet Corporation for Assigned Names and Numbers (ICANN)
  • Nslookup Linux / Windows: Command line utility usually used for querying the DNS records
  • bgp : Internet Backbone and Colocation Provider ... Hurricane Electric IP Transit. Our Global Internet Backbone provides IP Transit with low latency, access to thousands of networks, and dual-stack

↑Domain / Subdomain Discovery

  • RedHunt Labs Attack Surface Recon API: RedHunt Labs' Recon API offers comprehensive domain intelligence and reconnaissance capabilities. With access to their extensive in-house database of over 6 billion records, including domains, subdomains, third-party SaaS, data leaks, and intelligent correlations, this API empowers you to enhance your Attack Surface Management and InfoSec workflows.
  • SubFinder: SubFinder is a subdomain discovery tool that discovers valid subdomains for websites. Designed as a passive framework to be useful for bug bounties and safe for penetration testing.
  • Amass: A subdomain enumeration utility
  • Sublist3r: Subdomains enumeration tool with multiple sources
  • Aiodnsbrute: Asynchronous DNS brute force utility
  • LDNS: A DNS library useful for DNS tool programming
  • Dns-nsec3-enum: Nmap NSE Script for NSEC3 walking
  • Nsec3map: A tool to NSEC and NSEC3 walking
  • Crt.sh: Domain certificate Search
  • Ct-exposer: A tool to discovers sub-domains by searching Certificate Transparency logs
  • Certgraph: A tool to crawl the graph of certificate Alternate Names
  • Appsecco - The art of subdomain enumeration: The supplement material for the book "The art of sub-domain enumeration"
  • SSLScrape: A scanning tool to scrape hostnames from SSL certificates
  • Wolframalpha: Computational knowledge engine
  • Project Sonar: Forward DNS Data
  • Project Sonar: Reverse DNS Data
  • GoBuster: Directory/File, DNS and VHost busting tool written in Go
  • Bluto: Recon, Subdomain Bruting, Zone Transfers

↑Email Discovery

  • Hunter: Email search for a domain
  • Skrapp: Browser addon to find emails on Linkedin
  • Email Extractor: Chrome extension to extract emails from web pages
  • Convertcsv: Online tool to extract email addresses in text, web pages, data files etc.
  • linkedin2username: OSINT Tool: Generate username lists for companies on LinkedIn
  • Office365UserEnum: Enumerate valid usernames from Office 365 using ActiveSync.

↑Network / Port Scanning

  • Zmap: A fast network scanner designed for Internet-wide network surveys
  • Masscan: An asynchronously TCP port scanner
  • ZMapv6: A modified version of Zmap with IPv6 support.
  • Nmap: A free and open source utility for network discovery. The most popular port scanner.

↑Business Communication Infrastructure Discovery

  • Mxtoolbox: Online tool to check mail exchanger (MX) records
  • MicroBurst: PowerShell based Azure security assessment scripts
  • Lyncsmash: Tools to enumerate and attack self-hosted Lync/Skype for Business
  • Enumeration-as-a-Service: Script for SaaS offering enumeration through DNS queries
  • ruler : A tool to abuse Exchange services

↑Source Code Aggregators / Search - Information Discovery

  • Github: Github Advanced Search
  • Bitbucket: Bitbucket Search using Google
  • Gitrob: Reconnaissance tool for GitHub organizations
  • Gitlab: Search Gitlab projects
  • Publicwww: Source Code Search Engine
  • builtwith : Web technology information profiler tool. Find out what a website is built with.

↑Cloud Infrastructure Discovery

↑Company Information and Associations

  • Crunchbase: Information about companies (funding, acquisition, merger etc.) and the people behind them
  • Companieshouse: United Kingdom's registrar of companies
  • OverSeas Registries: List of company registries located around the world
  • Opencorporates: Open database of companies in the world

↑Internet Survey Data

  • Project Resonance: RedHunt Labs’s Internet wide surveys to study and understand the security state of the Internet.
  • Project Sonar: Rapid7’s internet-wide surveys data across different services and protocols
  • Scans.io: Internet-Wide Scan Data Repository, hosted by the ZMap Team
  • Portradar: Free and open port scan data by packet.tel

↑Social Media / Employee Profiling

↑Data Leaks

  • Dumpmon: A twitter bot which monitors multiple paste sites for password dumps and other sensitive information
  • Pastebin_scraper: Automated tool to monitor pastebin for interesting information
  • Scavenger: Paste sites crawler (bot) looking for leaked credentials
  • Pwnbin: Python based Pastebin crawler for keywords.
  • PwnedOrNot: Tool to find passwords for compromised accounts

↑Internet Scan / Archived Information

  • Cachedviews: Cached view of pages on the Internet from multiple sources
  • Wayback Machine: Internet Archive
  • Shodan: Search engine for Internet-connected devices
  • Censys: Another search engine for internet-connected devices
  • Zoomeye: Cyberspace Search Engine

Contributing

In case you would like to add information to this repository or suggest some ideas, please use one of the following options:

Connect

To connect with us:

License

This work is licensed under CC0 1.0 Universal