Top Related Projects
Like Prometheus, but for logs.
The Prometheus monitoring system and time series database.
CNCF Jaeger, a Distributed Tracing Platform
OpenTelemetry Collector
Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
Developer-first error tracking and performance monitoring
Quick Overview
Elastic APM Server is an open-source application performance monitoring (APM) system that collects and processes application performance data. It is part of the Elastic Stack ecosystem and works in conjunction with Elasticsearch and Kibana to provide detailed insights into application performance, errors, and transactions.
Pros
- Seamless integration with the Elastic Stack ecosystem
- Supports multiple programming languages and frameworks
- Provides detailed performance metrics and distributed tracing
- Highly scalable and designed for high-throughput environments
Cons
- Requires setup and maintenance of additional infrastructure
- Learning curve for users new to the Elastic Stack
- Can be resource-intensive for large-scale deployments
- Limited built-in alerting capabilities compared to some commercial APM solutions
Getting Started
To get started with Elastic APM Server:
-
Download and install the APM Server:
curl -L -O https://artifacts.elastic.co/downloads/apm-server/apm-server-7.13.0-linux-x86_64.tar.gz tar xzvf apm-server-7.13.0-linux-x86_64.tar.gz cd apm-server-7.13.0-linux-x86_64/
-
Configure the APM Server by editing
apm-server.yml
:output.elasticsearch: hosts: ["localhost:9200"]
-
Start the APM Server:
./apm-server -e
-
Install and configure APM agents in your application code. For example, in a Node.js application:
const apm = require('elastic-apm-node').start({ serviceName: 'my-service', serverUrl: 'http://localhost:8200', })
-
View APM data in Kibana by navigating to the APM app.
For more detailed instructions and configuration options, refer to the official Elastic APM documentation.
Competitor Comparisons
Like Prometheus, but for logs.
Pros of Loki
- Lightweight and efficient log aggregation system
- Designed for high-volume log ingestion and querying
- Integrates seamlessly with other Grafana ecosystem tools
Cons of Loki
- Limited to log data, not a full-featured APM solution
- Less mature ecosystem compared to Elastic APM
- May require additional tools for complete observability
Code Comparison
Loki (Go)
func (i *instance) Push(ctx context.Context, req *logproto.PushRequest) error {
// ... (code for pushing log entries)
}
APM Server (Go)
func (p *Processor) ProcessTransformationBatch(ctx context.Context, batch *model.Batch) error {
// ... (code for processing APM data)
}
Summary
Loki focuses on efficient log aggregation and querying, making it ideal for high-volume log management. It integrates well with Grafana but is limited to log data. APM Server, on the other hand, provides a comprehensive Application Performance Monitoring solution with broader observability features. While Loki excels in log management, APM Server offers a more mature ecosystem for full-stack monitoring. The code comparison highlights their different focuses: Loki on log pushing and APM Server on processing diverse APM data.
The Prometheus monitoring system and time series database.
Pros of Prometheus
- Open-source and highly customizable monitoring solution
- Powerful query language (PromQL) for data analysis
- Native support for multi-dimensional data and time series
Cons of Prometheus
- Requires manual configuration for service discovery
- Limited built-in visualization capabilities (often paired with Grafana)
- Lacks distributed tracing functionality out-of-the-box
Code Comparison
APM Server (Go):
func (p *processor) ProcessTransform(tctx *transform.Context) []transform.Transformable {
events := tctx.Events()
if len(events) == 0 {
return nil
}
// ... (processing logic)
}
Prometheus (Go):
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
mux := http.NewServeMux()
mux.HandleFunc("/metrics", h.metrics)
mux.HandleFunc("/federate", h.federation)
mux.HandleFunc("/", h.graph)
mux.ServeHTTP(w, r)
}
The code snippets show different approaches:
- APM Server focuses on event processing and transformation
- Prometheus emphasizes HTTP request handling for metrics collection and federation
Both projects use Go, but their core functionalities differ significantly. APM Server is tailored for application performance monitoring, while Prometheus is designed for general-purpose metrics collection and monitoring.
CNCF Jaeger, a Distributed Tracing Platform
Pros of Jaeger
- Open-source and vendor-neutral, part of CNCF
- Native support for OpenTelemetry data model
- Highly scalable architecture suitable for large-scale distributed systems
Cons of Jaeger
- Steeper learning curve for setup and configuration
- Limited built-in alerting and monitoring capabilities
- Requires additional components for long-term trace storage
Code Comparison
Jaeger (Go):
import (
"github.com/uber/jaeger-client-go"
"github.com/opentracing/opentracing-go"
)
func initJaeger(service string) (opentracing.Tracer, io.Closer) {
cfg := jaegercfg.Configuration{ServiceName: service}
tracer, closer, err := cfg.NewTracer(jaegercfg.Logger(jaeger.StdLogger))
return tracer, closer
}
APM Server (JavaScript):
import { init as initApm } from '@elastic/apm-rum'
const apm = initApm({
serviceName: 'my-service',
serverUrl: 'http://localhost:8200',
})
Both Jaeger and APM Server provide distributed tracing capabilities, but they differ in their ecosystem integration and ease of use. Jaeger offers a more flexible, open-source solution with strong community support, while APM Server provides a more integrated experience within the Elastic ecosystem, including built-in visualization and analysis tools through Kibana.
OpenTelemetry Collector
Pros of opentelemetry-collector
- Vendor-agnostic and supports multiple backends, offering greater flexibility
- Extensive community support and contributions from various organizations
- Modular architecture allows for easy customization and extension
Cons of opentelemetry-collector
- Steeper learning curve due to its more complex configuration options
- May require additional setup and configuration compared to apm-server
Code Comparison
apm-server configuration example:
apm-server:
host: "localhost:8200"
output.elasticsearch:
hosts: ["localhost:9200"]
opentelemetry-collector configuration example:
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
exporters:
logging:
loglevel: debug
service:
pipelines:
traces:
receivers: [otlp]
exporters: [logging]
The opentelemetry-collector configuration is more verbose but offers greater flexibility in defining data pipelines and processing steps. apm-server's configuration is simpler but more focused on Elastic Stack integration.
Both projects aim to collect and process telemetry data, but opentelemetry-collector provides a more versatile solution for multi-vendor environments, while apm-server is optimized for Elastic Stack ecosystems.
Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
Pros of Telegraf
- More versatile: Supports a wide range of input plugins for collecting data from various sources
- Lightweight and efficient: Written in Go, consumes fewer resources
- Flexible output options: Can send data to multiple destinations simultaneously
Cons of Telegraf
- Less specialized for APM: Requires additional configuration for application performance monitoring
- Steeper learning curve: More complex setup due to its extensive plugin ecosystem
Code Comparison
APM Server (Go):
func (p *processor) ProcessTransform(tctx *transform.Context) []transform.Transformable {
events := tctx.Config.Processor.TransformTransaction(p.transactionRoot)
return events
}
Telegraf (Go):
func (a *Accumulator) AddFields(
measurement string,
fields map[string]interface{},
tags map[string]string,
t ...time.Time,
) {
a.metric.AddFields(measurement, fields, tags, t...)
}
Summary
Telegraf is a more general-purpose metrics collection agent with broader input and output capabilities, while APM Server is specifically designed for application performance monitoring within the Elastic ecosystem. Telegraf offers greater flexibility but may require more setup for APM-specific use cases. APM Server provides a more streamlined experience for application monitoring but is less versatile for other data collection needs.
Developer-first error tracking and performance monitoring
Pros of Sentry
- More comprehensive error tracking and monitoring features
- Supports a wider range of programming languages and frameworks
- Offers a user-friendly web interface for error analysis and management
Cons of Sentry
- Can be more complex to set up and configure initially
- May have higher resource requirements for self-hosted installations
Code Comparison
APM Server (Go):
func (p *processor) ProcessTransformationBatch(ctx context.Context, batch *model.Batch) error {
events := make([]beat.Event, 0, len(batch.Transactions)+len(batch.Spans)+len(batch.Metricsets))
for _, tx := range batch.Transactions {
events = append(events, p.transformTransaction(tx)...)
}
// ... (additional processing)
}
Sentry (Python):
def process_event(event, project, key, release, environment):
with metrics.timer('sentry.events.process_event'):
with configure_scope() as scope:
scope.set_tag('project', project.id)
scope.set_tag('organization', project.organization_id)
return _do_process_event(event, project, key, release, environment)
Both repositories focus on application monitoring and error tracking, but they differ in their approach and feature set. APM Server is part of the Elastic Stack ecosystem, providing application performance monitoring with a focus on integration with Elasticsearch. Sentry offers a more standalone solution with broader language support and advanced error tracking capabilities. The code snippets demonstrate different implementation languages and approaches to event processing.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
APM Server
The APM Server receives data from Elastic APM agents and transforms it into Elasticsearch documents. Read more about Elastic APM at elastic.co/apm.
For questions and feature requests, visit the discussion forum.
Getting Started
To get started with APM, see our Quick start guide.
APM Server Development
Requirements
Install
- Fork the repo with the GitHub interface and clone it:
git clone git@github.com:[USER]/apm-server.git
Note that it should be cloned from the fork (replace [USER] with your GitHub user), not from origin.
- Add the upstream remote:
git remote add elastic git@github.com:elastic/apm-server.git
Build
To build the binary for APM Server run the command below. This will generate a binary in the same directory with the name apm-server.
make
If you make code changes, you may also need to update the project by running the additional command below:
make update
Run
To run APM Server with debugging output enabled, run:
./apm-server -c apm-server.yml -e -d "*"
APM Server expects index templates, ILM policies, and ingest pipelines to be set up externally. This should be done by installing the APM integration. When running APM Server directly, it is only necessary to install the integration and not to run an Elastic Agent.
Tilt
You can also run APM Server in a containerized environment using Tilt.
tilt up
See dev docs testing for additional information.
Testing
For Testing check out the testing guide
Cleanup
To clean up the build directory and generated artifacts, run:
make clean
Contributing
See contributing for details about reporting bugs, requesting features, or contributing to APM Server.
Releases
See releases for an APM Server release checklist.
Updating dependencies
APM Server uses Go Modules for dependency management, without any vendoring.
In general, you should use standard go get
commands to add and update modules. The one exception to this
is the dependency on libbeat
, for which there exists a special Make target: make update-beats
, described
below.
Updating libbeat
By running make update-beats
the github.com/elastic/beats/vN
module will be updated to the most recent
commit from the main branch, and a minimal set of files will be copied into the apm-server tree.
You can specify an alternative branch or commit by specifying the BEATS_VERSION
variable, such as:
make update-beats BEATS_VERSION=7.x
make update-beats BEATS_VERSION=f240148065af94d55c5149e444482b9635801f27
Updating go-elasticsearch
It is important to keep the go-elasticsearch client in sync with the according major version. We also recommend to use the latest available client for minor versions.
You can use go get -u -m github.com/elastic/go-elasticsearch/v7@7.x
to update to the latest commit on the
7.x branch.
Packaging
To build all apm-server packages from source, run:
make package
This will fetch and create all images required for the build process. The whole process can take several minutes.
When complete, packages can be found in build/distributions/
.
Building docker packages
To customize image configuration, see the docs.
To build docker images from source, run:
make package-docker
When complete, Docker images can be found at build/distributions/*.docker.tar.gz
,
and the local Docker image IDs are written at build/docker/*.txt
.
Building pre-release images can be done by running make package-docker-snapshot
instead.
Documentation
Documentation for the APM Server can be found in the Observability guide's APM section. Most documentation files live in the elastic/observability-docs repo's docs/en/observability/apm/
directory.
However, the following content lives in this repo:
- The changelog page listing all release notes is in
CHANGELOG.asciidoc
. - Each minor version's release notes are documented in individual files in the
changelogs/
directory. - A list of all breaking changes are documented in
changelogs/all-breaking-changes.asciidoc
. - Sample data sets that are injected into the docs are in the
docs/data/
directory. - Specifications that are injected into the docs are in the
docs/spec/
directory.
Top Related Projects
Like Prometheus, but for logs.
The Prometheus monitoring system and time series database.
CNCF Jaeger, a Distributed Tracing Platform
OpenTelemetry Collector
Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
Developer-first error tracking and performance monitoring
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot