Top Related Projects
Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.
Interactive roadmaps, guides and other educational content to help developers grow in their careers.
The Patterns of Scalable, Reliable, and Performant Large-Scale Systems
A curated list of engineering blogs
Linux, Jenkins, AWS, SRE, Prometheus, Docker, Python, Ansible, Git, Kubernetes, Terraform, OpenStack, SQL, NoSQL, Azure, GCP, DNS, Elastic, Network, Virtualization. DevOps Interview Questions
A collection of inspiring lists, manuals, cheatsheets, blogs, hacks, one-liners, cli/web tools and more.
Quick Overview
The jhuangtw/xg2xg repository is a comprehensive guide that maps Google-specific technologies to open-source or commercially available alternatives. It serves as a valuable resource for ex-Googlers and others interested in understanding how Google's internal tools compare to external options. The project aims to bridge the gap between Google's proprietary ecosystem and the wider tech industry.
Pros
- Provides a detailed comparison of Google technologies with external alternatives
- Regularly updated with new information and contributions from the community
- Covers a wide range of technology categories, from infrastructure to productivity tools
- Useful for both ex-Googlers transitioning to new roles and non-Googlers seeking insights into Google's tech stack
Cons
- Some comparisons may be subjective or based on limited information
- The rapidly evolving tech landscape means some information may become outdated quickly
- Focuses primarily on Google technologies, which may limit its usefulness for those interested in other tech giants' internal tools
- May not provide in-depth explanations or tutorials for the listed alternatives
As this is not a code library, we'll skip the code examples and getting started instructions sections.
Competitor Comparisons
Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.
Pros of system-design-primer
- Comprehensive coverage of system design concepts and principles
- Includes interactive coding challenges and exercises
- Regularly updated with new content and community contributions
Cons of system-design-primer
- More theoretical and less focused on specific tools or technologies
- May be overwhelming for beginners due to its extensive content
- Lacks direct mappings to specific company technologies
Code comparison
While both repositories don't primarily focus on code examples, system-design-primer does include some code snippets for illustrative purposes:
# system-design-primer example: Simple cache implementation
class Cache:
def __init__(self):
self.cache = {}
self.max_cache_size = 100
def set(self, key, value):
if len(self.cache) >= self.max_cache_size:
self.cache.pop(next(iter(self.cache)))
self.cache[key] = value
xg2xg doesn't typically include code snippets, as it focuses on mapping Google technologies to open-source alternatives.
Summary
system-design-primer is a comprehensive resource for learning system design concepts, while xg2xg serves as a quick reference for mapping Google technologies to alternatives. system-design-primer offers more in-depth learning materials but may be overwhelming for beginners, whereas xg2xg provides concise, practical information for those familiar with Google's tech stack.
Interactive roadmaps, guides and other educational content to help developers grow in their careers.
Pros of developer-roadmap
- Provides comprehensive visual roadmaps for various tech roles and skills
- Regularly updated with new content and technologies
- Offers interactive versions of roadmaps on the project website
Cons of developer-roadmap
- Focuses on general career paths rather than specific tool alternatives
- May be overwhelming for beginners due to the breadth of information
- Lacks direct comparisons between similar tools or technologies
Code comparison
While both repositories primarily consist of markdown files and don't contain significant code, developer-roadmap includes some JavaScript for its interactive features:
// developer-roadmap
import { writable } from 'svelte/store';
export const activeRoadmap = writable('frontend');
export const sidebarOpened = writable(false);
export const marketplaceOpened = writable(false);
xg2xg, on the other hand, is purely markdown-based and doesn't include any code.
Summary
developer-roadmap offers comprehensive visual guides for various tech career paths, making it ideal for those planning their learning journey. xg2xg, in contrast, provides a more focused list of tool alternatives, particularly useful for those transitioning from Google's tech stack. While developer-roadmap is more regularly updated and interactive, xg2xg offers a more concise and specific resource for tool comparisons.
The Patterns of Scalable, Reliable, and Performant Large-Scale Systems
Pros of awesome-scalability
- More comprehensive coverage of scalability topics, including principles, patterns, and technologies
- Better organized with clear categories and subcategories
- Includes visual diagrams and explanations for complex concepts
Cons of awesome-scalability
- Less focused on specific tools and technologies used by large tech companies
- May be overwhelming for beginners due to the breadth of information
- Lacks direct comparisons between similar tools or technologies
Code comparison
Not applicable for these repositories, as they are primarily curated lists of resources and don't contain significant code samples.
Summary
xg2xg is a concise list of tools and technologies used by large tech companies, particularly Google, and their open-source alternatives. It's useful for developers looking to replicate specific tech stacks.
awesome-scalability is a comprehensive guide to scalability concepts, patterns, and technologies. It covers a broader range of topics and provides more in-depth explanations, making it suitable for both beginners and experienced developers seeking to understand and implement scalable systems.
Both repositories serve different purposes and can be complementary resources for developers interested in building scalable systems and understanding the tools used by major tech companies.
A curated list of engineering blogs
Pros of engineering-blogs
- Extensive collection of engineering blogs from various companies and individuals
- Regularly updated with new blog entries
- Provides a valuable resource for staying up-to-date with industry trends and best practices
Cons of engineering-blogs
- Limited to blog links, lacking other types of resources or tools
- Less structured organization compared to xg2xg's categorized approach
- May require more time to find specific information or topics
Code comparison
While both repositories primarily consist of markdown files, engineering-blogs has a simpler structure:
# Engineering Blogs
* Airbnb Engineering: https://medium.com/airbnb-engineering
* Atlassian Developers: https://developer.atlassian.com/blog/
* AWS Blog: https://aws.amazon.com/blogs/aws/
xg2xg uses a more structured approach with categories:
## Career
| Google Internal | Google External | Open Source / Real-World |
| --------------- | --------------- | ------------------------ |
| EDGE Program | | |
| APLUS | | |
| Engineering Ladder | | [levels.fyi](https://www.levels.fyi/) |
Both repositories serve different purposes, with engineering-blogs focusing on curating blog links and xg2xg providing a broader range of resources and tools across various categories.
Linux, Jenkins, AWS, SRE, Prometheus, Docker, Python, Ansible, Git, Kubernetes, Terraform, OpenStack, SQL, NoSQL, Azure, GCP, DNS, Elastic, Network, Virtualization. DevOps Interview Questions
Pros of devops-exercises
- More comprehensive coverage of DevOps topics, including networking, Linux, CI/CD, and more
- Includes practical exercises and questions for hands-on learning
- Regularly updated with new content and contributions from the community
Cons of devops-exercises
- Less focused on specific tools and services used in large tech companies
- May be overwhelming for beginners due to the breadth of topics covered
- Lacks direct mappings between different cloud providers' services
Code comparison
While both repositories primarily contain markdown files and don't have extensive code samples, devops-exercises includes some code snippets for exercises. For example:
devops-exercises:
#!/usr/bin/env python
def factorial(n):
if n == 0:
return 1
else:
return n * factorial(n-1)
xg2xg doesn't typically include code snippets, focusing instead on tool and service comparisons:
| Google Internal | Google External | Open Source / Real-World |
| ---------------- | --------------- | ---------------------------------------- |
| Dremel | BigQuery | Apache Drill, Presto, Trino, Clickhouse |
| Colossus | GFS | HDFS, Ceph |
Both repositories serve different purposes: devops-exercises is a learning resource for DevOps practices, while xg2xg is a reference for mapping Google-specific tools to external equivalents.
A collection of inspiring lists, manuals, cheatsheets, blogs, hacks, one-liners, cli/web tools and more.
Pros of the-book-of-secret-knowledge
- Broader scope, covering a wide range of IT topics and tools
- More comprehensive with over 500 resources and tools listed
- Regularly updated with new content and contributions
Cons of the-book-of-secret-knowledge
- Less focused on specific company equivalents
- May be overwhelming for users looking for quick alternatives
- Requires more time to navigate and find relevant information
Code comparison
While both repositories primarily consist of markdown files with lists and links, they don't contain significant code snippets. However, here's a brief comparison of their structure:
the-book-of-secret-knowledge:
## CLI Tools & Utilities
### CLI Tools - Networking
- [Nmap](https://nmap.org/) - free, open source tool for network discovery and security auditing.
xg2xg:
## Infrastructure
| Google Internal | Google External | Open Source / Real-World |
| --------------- | --------------- | --------------------------- |
| Borg | [Kubernetes](https://kubernetes.io/) | [Apache Mesos](https://mesos.apache.org/) |
Both repositories use markdown formatting, but xg2xg focuses on comparing Google tools to external alternatives, while the-book-of-secret-knowledge provides a more general list of resources across various IT domains.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
A handy lookup table of similar technology and services to help ex-googlers survive the real world :) pull-requests are very welcomed. Please do not list any confidential projects!
For a working example of (some) of these technologies integrated together, see: https://github.com/google/startup-os
See also: System Design Primer, The Hadoop Ecosystem Table, Map AWS services to GCP products, Database of Databases, Google Open Source Glossary
Technology
Core Technology
Google Internal | Google External | Open Source / Real-World |
---|---|---|
MapReduce | Apache Hadoop, Spark | |
Flume | DataFlow | Apache Beam |
Protocol Buffer | Protobuf, FlatBuffers | Cap'n Proto, Thrift, Avro, Amazon Ion, CBOR, kryo |
Stubby | gRPC | Thrift, Bolt |
Chubby | Apache Zookeeper, etcd, HashiCorp Consul | |
Goops / PubSub | Cloud Pub/Sub | Apache Kafka, Apache Pulsar, Facebook LogDevice |
//base | abseil |
Infrastructure
Google Internal | Google External | Open Source / Real-World |
---|---|---|
Borg | Kubernetes | Apache Mesos, Apache Aurora, HashiCorp Nomad |
GSLB | Cloud Load Balancing - Internal | AWS ELB, Istio, linkerd |
OnePlatform | API Gateway | Swagger |
GFE, Maglev, uberproxy | Cloud Load Balancing - HTTPS / External | envoy, AWS ALB, HAProxy, nginx, F5 |
uberproxy (sso proxy) | Identity-Aware Proxy | buzzfeed-sso, Pomerium |
Zanzibar | Zanzibar Research Paper | SpiceDB/authzed, Ory Keto, topaz, Opal, (iam)Keycloak, Warrant |
data center hardware | open compute | |
Jupiter, Starblaze | ||
B4, Stargate, TE | ||
USPS, Andromeda | ||
ESDN | Faucet | |
broccoli man |
Storage
Google Internal | Google External | Open Source / Real-World |
---|---|---|
GFS/Colossus | HDFS, Ceph, GlusterFS, MooseFS | |
BigTable | Cloud BigTable | PrestoDB, Cassandra, HBase, Accumulo, DynamoDB, ScyllaDB |
Spanner | Cloud Spanner | Vitess, CockroachDB, TiDB |
ColumnIO / Capacitor | Apache Parquet, ORC | |
sstable | levelDB | RocksDB, pebble |
zippy | Snappy | lz4 |
RecordIO | Riegeli, TFRecords, & in OR-Tools, szl | stuffed-record-stream |
Services
Google Internal | Google External | Open Source | SaaS |
---|---|---|---|
Dremel | BigQuery | Apache Drill, Presto, Spark(sort-of), | AWS Athena, Redshift Spectrum, Snowflake |
Dremel UI | Redash, Metabase, Apache Superset | ||
Search (Mustang, Alexandria) | Elasticsearch, OpenSearch, Solr, Lucene | algolia | |
pubsub | pubsub | NATS.io, RabbitMQ, PubNub | AWS SQS/SNS, AWS AppSync |
MillWheel | Cloud Dataflow | Apache Flink, beam | |
Colab | Colaboratory | Jupyter | Observable |
PLX | Google Data Studio | Mode | |
Monarch | paper | ||
Napa | paper | ||
MakerSuite | MakerSuite |
DevOps
Security
Google Internal | Google External | Open Source |
---|---|---|
prodaccess/LOAS | Keymaster | |
prod secrets/identity management | chamber, knox, SPIFFE |
IT / Operations / Misc
Google Internal | Real-World |
---|---|
software engineering at google 1, 2 | Software Engineering at Google: Lessons Learned from Programming Over Time |
SRE @ google | |
valentine | Vault, 1Password, Lastpass, pass, keeper |
OWNERS files in repo | github CODEOWNERS |
snippets | Khan/snippets |
SnipIt | recordit, CloudApp, dropbox screenshots, Snippyly |
stuff (SaaS IT management) | productiv, intello, zylo |
stuff (Device Management) | jamf |
device security monitoring | Red Canary |
beyondcorp | beyondcorp |
go/ links | golinks, go, Goat, trotto, go-shorten |
google3 philosophy | innersource, monorepo, YouTube talk |
doing code review | code review |
safely sharing 1-time secrets | sendsecure.ly, croc, onetimesecret, privatebin |
messaging | mattermost, Slack, gchat |
tech talks | TechTalks @ Google |
g3doc | writethedocs, docs-as-code |
GUTS | spoke, freshservice |
MOMA | glean |
PeopleOps / Culture
Google Internal | Real-World |
---|---|
OKR | CultureAmp OKR, Lattice Goals, Ally, workboard |
HRIS/ERP | Namely, BambooHR, Workday, Paylocity |
peer bonus | bonus.ly |
kudos | heytaco, slack ++ bot |
perks | fond |
talks | Talks @ Google |
dory | slido |
edge, people ops training | LifeLabs |
googlegeist | Culture Amp, humu, tinypulse, peakon |
Meng | Search Inside Yourself |
Lazlo | Work Rules |
Claire Stapleton | Tech Support - existential advice for the modern tech worker |
books about google | How Google Works, In The Plex, Software Engineering at Google |
also check out xoogler.co, which organizes events, slack channels etc
disclaimer: I'm not affiliated with any of the technologies/products mentioned above.
disclaimer: I left Google a number of years ago so some of the naming might be dated.
Top Related Projects
Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.
Interactive roadmaps, guides and other educational content to help developers grow in their careers.
The Patterns of Scalable, Reliable, and Performant Large-Scale Systems
A curated list of engineering blogs
Linux, Jenkins, AWS, SRE, Prometheus, Docker, Python, Ansible, Git, Kubernetes, Terraform, OpenStack, SQL, NoSQL, Azure, GCP, DNS, Elastic, Network, Virtualization. DevOps Interview Questions
A collection of inspiring lists, manuals, cheatsheets, blogs, hacks, one-liners, cli/web tools and more.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot