Top Related Projects
Community curated list of templates for the nuclei engine to find security vulnerabilities.
Find domains and subdomains related to a given domain
In-depth attack surface mapping and asset discovery
Incredibly fast crawler designed for OSINT.
This script is intended to automate your reconnaissance process in an organized fashion
Fetch known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, and Common Crawl.
Quick Overview
Awesome-Asset-Discovery is a curated list of resources and tools for asset discovery in cybersecurity. It provides a comprehensive collection of tools, techniques, and methodologies for identifying and mapping digital assets within an organization's network or on the internet. This repository serves as a valuable reference for security professionals, penetration testers, and researchers.
Pros
- Extensive collection of asset discovery tools and resources
- Well-organized and categorized for easy navigation
- Regularly updated with new tools and techniques
- Community-driven with contributions from security experts
Cons
- May overwhelm beginners due to the large number of tools listed
- Some listed tools may become outdated or discontinued over time
- Lacks detailed explanations or comparisons of the listed tools
- Primarily focuses on listing tools rather than providing in-depth tutorials
Note: As this is not a code library but a curated list of resources, there are no code examples or getting started instructions to provide.
Competitor Comparisons
Community curated list of templates for the nuclei engine to find security vulnerabilities.
Pros of nuclei-templates
- Extensive collection of ready-to-use templates for vulnerability scanning
- Regularly updated with new templates for emerging vulnerabilities
- Supports a wide range of protocols and technologies
Cons of nuclei-templates
- Requires the Nuclei engine to run, adding complexity
- May generate false positives if not properly configured
- Focused solely on vulnerability detection, not broader asset discovery
Code Comparison
While a direct code comparison isn't applicable due to the different nature of these repositories, here's a brief example of how they differ in content:
Awesome-Asset-Discovery (README.md):
## DNS
- [Amass](https://github.com/OWASP/Amass)
- [subfinder](https://github.com/projectdiscovery/subfinder)
nuclei-templates (example template):
id: cve-2021-44228-log4j-rce
info:
name: Apache Log4j RCE
severity: critical
http:
- method: GET
path:
- "{{BaseURL}}/?x=${jndi:ldap://{{interactsh-url}}}"
Awesome-Asset-Discovery is a curated list of tools, while nuclei-templates contains actual scanning templates. The former aids in tool discovery, while the latter provides ready-to-use content for vulnerability scanning.
Find domains and subdomains related to a given domain
Pros of assetfinder
- Lightweight and focused tool specifically for subdomain enumeration
- Fast execution and efficient resource usage
- Easy to integrate into existing workflows and scripts
Cons of assetfinder
- Limited to subdomain discovery, lacking broader asset discovery features
- May miss some assets that require more advanced discovery techniques
- Less comprehensive compared to a curated list of multiple tools
Code Comparison
assetfinder:
func main() {
domains := make(chan string, 100)
go func() {
sc := bufio.NewScanner(os.Stdin)
for sc.Scan() {
domains <- sc.Text()
}
close(domains)
}()
// ... (processing logic)
}
Awesome-Asset-Discovery:
## DNS
- [Amass](https://github.com/OWASP/Amass)
- [DNSRecon](https://github.com/darkoperator/dnsrecon)
- [Subfinder](https://github.com/projectdiscovery/subfinder)
## Web
- [Gospider](https://github.com/jaeles-project/gospider)
- [Hakrawler](https://github.com/hakluke/hakrawler)
The code comparison highlights the difference in approach. assetfinder is a Go-based tool with specific implementation for subdomain discovery, while Awesome-Asset-Discovery is a curated list of various asset discovery tools and resources, providing a broader scope but requiring manual integration of individual tools.
In-depth attack surface mapping and asset discovery
Pros of Amass
- Active tool for network mapping and external asset discovery
- Performs DNS enumeration, web scraping, and uses various data sources
- Offers both CLI and GUI interfaces for flexibility
Cons of Amass
- Steeper learning curve due to its extensive features and options
- May generate more noise or false positives compared to curated lists
- Requires more system resources and time to run comprehensive scans
Code Comparison
Amass (example usage):
amass enum -d example.com -o output.txt
Awesome-Asset-Discovery (example entry):
- [Amass](https://github.com/OWASP/Amass) - In-depth Attack Surface Mapping and Asset Discovery
Summary
Amass is an active, feature-rich tool for comprehensive asset discovery, while Awesome-Asset-Discovery is a curated list of asset discovery resources. Amass offers hands-on scanning capabilities but requires more expertise and resources. Awesome-Asset-Discovery provides a quick reference to various tools and techniques, including Amass itself, making it more accessible for beginners or those seeking an overview of available options.
Incredibly fast crawler designed for OSINT.
Pros of Photon
- Active tool for web reconnaissance and information gathering
- Provides automated crawling and data extraction capabilities
- Offers multiple output formats for easy integration with other tools
Cons of Photon
- Limited to web-based asset discovery
- May require more setup and configuration compared to a curated list
- Potential for false positives or incomplete results depending on target website structure
Code Comparison
Photon:
def photon(url, level, threadCount, delay, timeout, headers, cookie):
# ... (initialization code)
for url in urls:
# ... (crawling and extraction logic)
# ... (output generation)
Awesome-Asset-Discovery:
## DNS
- [Amass](https://github.com/OWASP/Amass)
- [subfinder](https://github.com/projectdiscovery/subfinder)
## Web
- [Photon](https://github.com/s0md3v/Photon)
- [gospider](https://github.com/jaeles-project/gospider)
Photon is an active tool for web reconnaissance, while Awesome-Asset-Discovery is a curated list of various asset discovery tools. Photon offers automated crawling and data extraction, but is limited to web-based assets. Awesome-Asset-Discovery provides a comprehensive collection of tools for different asset types, but requires manual selection and setup of individual tools.
This script is intended to automate your reconnaissance process in an organized fashion
Pros of lazyrecon
- Automated reconnaissance tool that combines multiple techniques
- Provides a streamlined workflow for bug bounty hunters
- Generates organized output reports for easy analysis
Cons of lazyrecon
- Limited to specific reconnaissance tasks compared to Awesome-Asset-Discovery's comprehensive list
- May require more setup and dependencies
- Less frequently updated compared to Awesome-Asset-Discovery
Code Comparison
lazyrecon:
#!/bin/bash
domain=$1
if [ -z "$domain" ]; then
echo "Usage: $0 <domain>"
exit 1
fi
Awesome-Asset-Discovery:
## DNS
* [Amass](https://github.com/OWASP/Amass)
* [Subfinder](https://github.com/projectdiscovery/subfinder)
* [Knockpy](https://github.com/guelfoweb/knock)
The code comparison shows that lazyrecon is a bash script for automated reconnaissance, while Awesome-Asset-Discovery is a curated list of tools and resources in Markdown format. lazyrecon provides a ready-to-use script, whereas Awesome-Asset-Discovery offers a comprehensive collection of tools for users to choose from based on their specific needs.
Fetch known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, and Common Crawl.
Pros of gau
- Focused tool for URL discovery from various sources
- Fast and efficient, designed for high-performance scanning
- Easy to use with simple command-line interface
Cons of gau
- Limited to URL discovery, lacks comprehensive asset discovery features
- May require additional tools for complete asset enumeration
- Less extensive documentation compared to Awesome-Asset-Discovery
Code Comparison
gau:
echo example.com | gau
gau example.com
cat domains.txt | gau --threads 5
Awesome-Asset-Discovery:
# No specific code, as it's a curated list of tools and resources
# Example usage might involve running multiple tools listed in the repository
Key Differences
- Awesome-Asset-Discovery is a curated list of various asset discovery tools and resources, while gau is a specific tool for URL discovery
- gau focuses on speed and efficiency for URL enumeration, whereas Awesome-Asset-Discovery provides a comprehensive overview of multiple asset discovery techniques
- Awesome-Asset-Discovery offers a broader range of tools and methodologies, while gau specializes in one specific aspect of asset discovery
Use Cases
- Use gau when you need quick and efficient URL discovery for a target domain
- Refer to Awesome-Asset-Discovery when you require a comprehensive approach to asset discovery or need to explore various tools and techniques for different scenarios
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
Awesome Asset Discovery
Asset Discovery is the initial phase of any security assessment engagement, be it offensive or defensive. With the evolution of information technology, the scope and definition of assets has also evolved.
Earlier the servers, workstations and websites were primary IT assets of an organization, but today this definition is very limiting and should include anything and everything an organization and its entities has their data on (knowingly or unknowingly). The scope of ownership could differ, but it does not limit the attack surface, for example if an organization puts out open source code on Github, they are not the owner of Github but of the data they put under their repositories. In a scenario where some organization secret has been put on this Github account, it could pose a threat equal or more than running a vulnerable service.
We have explored this aspect of assets in our blog post here.
Through this repository, we want to put out a list of curated resources which help during asset discovery phase of a security assessment engagement. We welcome suggestions and contributions from the community in terms of resources as well as categories.
To know more about our Attack Surface Management platform, check out NVADR.
Contents
- Content Discovery
- IP Address Discovery
- Domain / Subdomain Discovery
- Email Discovery
- Network / Port Scanning
- Business Communication Infrastructure Discovery
- Source Code Aggregators / Search - Information Discovery
- Cloud Infrastructure Discovery
- Company Information and Associations
- Internet Survey Data
- Social Media / Employee Profiling
- Data Leaks
- Internet Scan / Archived Information
âContent Discovery
- rustbuster: Files, directories and vhost buster written in Rust.
âIP Address Discovery
- Mxtoolbox: Bulk Domain/IP lookup tool
- Domaintoipconverter: Bulk domain to IP converter
- Massdns: A DNS resolver utility for bulk lookups
- Googleapps Dig: Online Dig tool by Google
- DataSploit (IP Address Modules): An OSINT Framework to perform various recon techniques
- Domain Dossier: Investigate domains and IP addresses
- Bgpview: Search ASN, IPv4/IPv6 or resource name
- Hurricane Electric BGP Toolkit: Keyword to ASN lookup
- Viewdns: Multiple domain/IP tools
- Ultratools ipv6Info: Multiple information related to IPv6 address
- Whois: Command line utility usually used to find information about registered users/assignees of an Internet resource.
- ICANN Whois: Whois service by Internet Corporation for Assigned Names and Numbers (ICANN)
- Nslookup Linux / Windows: Command line utility usually used for querying the DNS records
- bgp : Internet Backbone and Colocation Provider ... Hurricane Electric IP Transit. Our Global Internet Backbone provides IP Transit with low latency, access to thousands of networks, and dual-stack
âDomain / Subdomain Discovery
- RedHunt Labs Attack Surface Recon API: RedHunt Labs' Recon API offers comprehensive domain intelligence and reconnaissance capabilities. With access to their extensive in-house database of over 6 billion records, including domains, subdomains, third-party SaaS, data leaks, and intelligent correlations, this API empowers you to enhance your Attack Surface Management and InfoSec workflows.
- SubFinder: SubFinder is a subdomain discovery tool that discovers valid subdomains for websites. Designed as a passive framework to be useful for bug bounties and safe for penetration testing.
- Amass: A subdomain enumeration utility
- Sublist3r: Subdomains enumeration tool with multiple sources
- Aiodnsbrute: Asynchronous DNS brute force utility
- LDNS: A DNS library useful for DNS tool programming
- Dns-nsec3-enum: Nmap NSE Script for NSEC3 walking
- Nsec3map: A tool to NSEC and NSEC3 walking
- Crt.sh: Domain certificate Search
- Ct-exposer: A tool to discovers sub-domains by searching Certificate Transparency logs
- Certgraph: A tool to crawl the graph of certificate Alternate Names
- Appsecco - The art of subdomain enumeration: The supplement material for the book "The art of sub-domain enumeration"
- SSLScrape: A scanning tool to scrape hostnames from SSL certificates
- Wolframalpha: Computational knowledge engine
- Project Sonar: Forward DNS Data
- Project Sonar: Reverse DNS Data
- GoBuster: Directory/File, DNS and VHost busting tool written in Go
- Bluto: Recon, Subdomain Bruting, Zone Transfers
âEmail Discovery
- Hunter: Email search for a domain
- Skrapp: Browser addon to find emails on Linkedin
- Email Extractor: Chrome extension to extract emails from web pages
- Convertcsv: Online tool to extract email addresses in text, web pages, data files etc.
- linkedin2username: OSINT Tool: Generate username lists for companies on LinkedIn
- Office365UserEnum: Enumerate valid usernames from Office 365 using ActiveSync.
âNetwork / Port Scanning
- Zmap: A fast network scanner designed for Internet-wide network surveys
- Masscan: An asynchronously TCP port scanner
- ZMapv6: A modified version of Zmap with IPv6 support.
- Nmap: A free and open source utility for network discovery. The most popular port scanner.
âBusiness Communication Infrastructure Discovery
- Mxtoolbox: Online tool to check mail exchanger (MX) records
- MicroBurst: PowerShell based Azure security assessment scripts
- Lyncsmash: Tools to enumerate and attack self-hosted Lync/Skype for Business
- Enumeration-as-a-Service: Script for SaaS offering enumeration through DNS queries
- ruler : A tool to abuse Exchange services
âSource Code Aggregators / Search - Information Discovery
- Github: Github Advanced Search
- Bitbucket: Bitbucket Search using Google
- Gitrob: Reconnaissance tool for GitHub organizations
- Gitlab: Search Gitlab projects
- Publicwww: Source Code Search Engine
- builtwith : Web technology information profiler tool. Find out what a website is built with.
âCloud Infrastructure Discovery
- CloudScraper: A tool to spider websites for cloud resources (S3 Buckets, Azure Blobs, DigitalOcean Storage Space)
- InSp3ctor: AWS S3 Bucket/Object finder
- Buckets Grayhatwarfare: Search for Open Amazon s3 Buckets and their contents
- Spaces-finder: A tool to hunt for publicly accessible DigitalOcean Spaces
- GCPBucketBrute: A Google Storage buckets enumeration script
- CloudStorageFinder: Tools to find public data in cloud storage systems
âCompany Information and Associations
- Crunchbase: Information about companies (funding, acquisition, merger etc.) and the people behind them
- Companieshouse: United Kingdom's registrar of companies
- OverSeas Registries: List of company registries located around the world
- Opencorporates: Open database of companies in the world
âInternet Survey Data
- Project Resonance: RedHunt Labsâs Internet wide surveys to study and understand the security state of the Internet.
- Project Sonar: Rapid7âs internet-wide surveys data across different services and protocols
- Scans.io: Internet-Wide Scan Data Repository, hosted by the ZMap Team
- Portradar: Free and open port scan data by packet.tel
âSocial Media / Employee Profiling
- LinkedInt: A LinkedIn scraper for reconnaissance
- Glassdoor: Company review and rating search
- SocialBlade: Track user statistics for different platforms including YouTube and Twitter
- Social-Searcher: Social Media Search Engine
- Checkuser: Social existence checker
âData Leaks
- Dumpmon: A twitter bot which monitors multiple paste sites for password dumps and other sensitive information
- Pastebin_scraper: Automated tool to monitor pastebin for interesting information
- Scavenger: Paste sites crawler (bot) looking for leaked credentials
- Pwnbin: Python based Pastebin crawler for keywords.
- PwnedOrNot: Tool to find passwords for compromised accounts
âInternet Scan / Archived Information
- Cachedviews: Cached view of pages on the Internet from multiple sources
- Wayback Machine: Internet Archive
- Shodan: Search engine for Internet-connected devices
- Censys: Another search engine for internet-connected devices
- Zoomeye: Cyberspace Search Engine
Contributing
In case you would like to add information to this repository or suggest some ideas, please use one of the following options:
- Create an Issue.
- Send us Pull Requests
- Drop an email to contact@redhuntlabs.com
Connect
To connect with us:
License
This work is licensed under CC0 1.0 Universal
Top Related Projects
Community curated list of templates for the nuclei engine to find security vulnerabilities.
Find domains and subdomains related to a given domain
In-depth attack surface mapping and asset discovery
Incredibly fast crawler designed for OSINT.
This script is intended to automate your reconnaissance process in an organized fashion
Fetch known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, and Common Crawl.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot