Top Related Projects
Define and run multi-container applications with Docker
Bitnami container images
Dockerfile for Apache Kafka
Kafka (and Zookeeper) in Docker
Kafka Docker for development. Kafka, Zookeeper, Schema Registry, Kafka-Connect, Landoop Tools, 20+ connectors
Quick Overview
The confluentinc/cp-docker-images repository contains Docker images for deploying and managing Apache Kafka in containerized environments. It provides a comprehensive set of images for various Confluent Platform components, enabling users to easily set up and run Kafka-based streaming applications using Docker.
Pros
- Simplifies deployment of Kafka and related services in containerized environments
- Provides official, well-maintained images for Confluent Platform components
- Supports easy scaling and management of Kafka clusters
- Includes images for additional tools like Schema Registry and Kafka Connect
Cons
- May require additional configuration for production-grade deployments
- Docker images can be large, potentially increasing resource usage
- Learning curve for users new to both Kafka and Docker
- Limited customization options compared to manual installations
Getting Started
To get started with confluentinc/cp-docker-images, follow these steps:
- Install Docker and Docker Compose on your system.
- Create a
docker-compose.yml
file with the desired Confluent Platform components:
version: '3'
services:
zookeeper:
image: confluentinc/cp-zookeeper:latest
environment:
ZOOKEEPER_CLIENT_PORT: 2181
kafka:
image: confluentinc/cp-kafka:latest
depends_on:
- zookeeper
environment:
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
schema-registry:
image: confluentinc/cp-schema-registry:latest
depends_on:
- kafka
environment:
SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: kafka:9092
SCHEMA_REGISTRY_HOST_NAME: schema-registry
- Run the following command to start the services:
docker-compose up -d
This will start a basic Kafka cluster with ZooKeeper and Schema Registry. You can now interact with Kafka using the exposed ports on your local machine.
Competitor Comparisons
Define and run multi-container applications with Docker
Pros of Compose
- More general-purpose, supporting a wide range of applications and services
- Extensive documentation and large community support
- Simpler syntax for defining multi-container applications
Cons of Compose
- Less specialized for Confluent Platform components
- May require additional configuration for Kafka-specific features
- Doesn't provide pre-built images optimized for Confluent Platform
Code Comparison
cp-docker-images:
---
version: '2'
services:
zookeeper:
image: confluentinc/cp-zookeeper:latest
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
Compose:
version: '3'
services:
zookeeper:
image: zookeeper:latest
ports:
- "2181:2181"
environment:
ZOO_MY_ID: 1
cp-docker-images focuses on Confluent Platform components with pre-configured images, while Compose offers a more flexible approach for defining multi-container applications. cp-docker-images provides optimized configurations for Kafka ecosystem services, whereas Compose requires manual setup but allows for greater customization across various technologies. The code examples illustrate the difference in specifying a Zookeeper service, with cp-docker-images using Confluent's custom image and Compose using a generic Zookeeper image.
Bitnami container images
Pros of containers
- Broader scope: Covers a wide range of applications and services beyond just Confluent Platform
- More frequent updates: Generally receives more regular updates and maintenance
- Simplified configuration: Often provides easier setup and configuration options for various applications
Cons of containers
- Less specialized: May not offer as deep integration with Confluent-specific features
- Community support: Might have less focused community support for Confluent-related issues
- Potential compatibility issues: Could face challenges with certain Confluent Platform versions or features
Code comparison
cp-docker-images:
zookeeper:
image: confluentinc/cp-zookeeper:${CONFLUENT_VERSION}
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
containers:
zookeeper:
image: bitnami/zookeeper:${ZOOKEEPER_VERSION}
environment:
ALLOW_ANONYMOUS_LOGIN: "yes"
ZOO_PORT_NUMBER: 2181
The code snippets show differences in image naming conventions and environment variable configurations between the two repositories. cp-docker-images uses Confluent-specific images and configurations, while containers uses more generic Bitnami images with slightly different environment variable names.
Dockerfile for Apache Kafka
Pros of kafka-docker
- Lightweight and simple setup, ideal for development and testing
- Highly customizable through environment variables
- Active community support and frequent updates
Cons of kafka-docker
- Limited enterprise features compared to cp-docker-images
- Less comprehensive documentation
- May require additional configuration for production-ready setups
Code Comparison
kafka-docker:
version: '2'
services:
kafka:
image: wurstmeister/kafka:latest
ports:
- "9092:9092"
environment:
KAFKA_ADVERTISED_HOST_NAME: localhost
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
cp-docker-images:
version: '2'
services:
kafka:
image: confluentinc/cp-kafka:latest
ports:
- "9092:9092"
environment:
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
The code comparison shows that both repositories use similar Docker Compose configurations. However, cp-docker-images uses Confluent's official Kafka image, which includes additional enterprise features and optimizations. The kafka-docker repository uses a community-maintained image that focuses on simplicity and flexibility.
Kafka (and Zookeeper) in Docker
Pros of docker-kafka
- Simpler and more lightweight, focusing solely on Kafka
- Easier to understand and modify for basic Kafka setups
- Faster startup times due to fewer components
Cons of docker-kafka
- Limited to Kafka only, lacking additional Confluent Platform components
- Less frequent updates and maintenance compared to cp-docker-images
- May not include the latest Kafka features and improvements
Code Comparison
cp-docker-images:
version: '2'
services:
zookeeper:
image: confluentinc/cp-zookeeper:latest
kafka:
image: confluentinc/cp-kafka:latest
depends_on:
- zookeeper
docker-kafka:
version: '2'
services:
kafka:
image: spotify/kafka:latest
environment:
ADVERTISED_HOST: kafka
ADVERTISED_PORT: 9092
The cp-docker-images example shows a more comprehensive setup with separate Zookeeper and Kafka services, while the docker-kafka example provides a simpler, all-in-one Kafka container. cp-docker-images offers more configuration options and follows Confluent's best practices, whereas docker-kafka aims for simplicity and ease of use in development environments.
Kafka Docker for development. Kafka, Zookeeper, Schema Registry, Kafka-Connect, Landoop Tools, 20+ connectors
Pros of fast-data-dev
- All-in-one solution with pre-configured components for quick setup
- Includes a web UI for easier management and monitoring
- Supports multiple Kafka versions out of the box
Cons of fast-data-dev
- Less flexibility for customization compared to individual components
- May include unnecessary services for specific use cases
- Potentially larger image size due to bundled components
Code Comparison
fast-data-dev:
FROM alpine:3.8
RUN apk add --no-cache curl openjdk8-jre bash coreutils
ENV KAFKA_VERSION=2.0.0 SCALA_VERSION=2.11
cp-docker-images:
FROM centos:7
ARG KAFKA_VERSION=5.3.1
ARG SCALA_VERSION=2.12
ENV KAFKA_HOME=/opt/kafka
The fast-data-dev Dockerfile uses Alpine Linux as the base image, resulting in a smaller overall image size. It also includes additional tools like curl and bash. The cp-docker-images Dockerfile uses CentOS 7 as the base image and focuses on Kafka-specific components.
fast-data-dev provides a more comprehensive solution for quick development and testing, while cp-docker-images offers greater flexibility for production deployments and custom configurations.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
Deprecation Notice
This is used for building images for version 5.3.x or lower, and should not be used for adding new images.
For the 5.4.0 release and greater the images have been migrated to the following repositories:
- common-docker
- control-center-images
- kafkacat-images
- kafka-images
- kafka-mqtt-images
- kafka-replicator-images
- kafka-rest-images
- kafka-streams-examples
- ksql-images
- schema-registry-images
Docker Images for Confluent Plaform
Docker images for deploying and running the Confluent Platform. The images are currently available on DockerHub. They are currently only available for Confluent Platform 3.0.1 and after.
Full documentation for using the images can be found here.
Networking and Kafka on Docker
When running Kafka under Docker, you need to pay careful attention to your configuration of hosts and ports to enable components both internal and external to the docker network to communicate. You can see more details in this article.
Known issues on Mac/Windows
For more details on known issues when running these images on Mac/Windows, you can refer to the following links:
- Hostname Issue
- Host networking on Docker for Mac: link 1, link 2, link 3
Building
Use make
to perform various builds and tests
Docker Utils
See Docker Utils
Contribute
Start by reading our guidelines on contributing to this project found here.
- Source Code: https://github.com/confluentinc/cp-docker-images
- Issue Tracker: https://github.com/confluentinc/cp-docker-images/issues
License
The project is licensed under the Apache 2 license. For more information on the licenses for each of the individual Confluent Platform components packaged in the images, please refer to the respective Confluent Platform documentation for each component.
Top Related Projects
Define and run multi-container applications with Docker
Bitnami container images
Dockerfile for Apache Kafka
Kafka (and Zookeeper) in Docker
Kafka Docker for development. Kafka, Zookeeper, Schema Registry, Kafka-Connect, Landoop Tools, 20+ connectors
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot