K3D in A Docker Sample App and Helm Chart

A production-ready Wikipedia-like API service with FastAPI, PostgreSQL, Prometheus, and Grafana, packaged as a Helm chart for Kubernetes and containerized using k3d with Docker-outside-of-Docker (DooD) approach.

Features

• Create and retrieve users
• Create and retrieve posts
• Async database operations with SQLAlchemy
• PostgreSQL database (production-ready)
• Prometheus metrics for monitoring
• Grafana dashboards for visualization
• Kubernetes-ready with Helm charts
• Docker-outside-of-Docker (DooD) support with k3d
• Traefik ingress (k3d default) for routing

Architecture

The solution consists of:

• FastAPI: REST API service with business logic
• PostgreSQL: Production database (replaces SQLite)
• Prometheus: Metrics collection from FastAPI
• Grafana: Visualization dashboard for creation rates
• Ingress: Traefik ingress controller (k3d default) for routing

Prerequisites

• Ubuntu/Debian-based Linux system (for bootstrap script)
• Docker (for building and running containers)
• Docker must support privileged mode (for Docker-outside-of-Docker)
• At least 4GB RAM available
• Port 8080 available
• kubectl, k3d, and Helm (or use the bootstrap script to install them)

Installing Required Tools

If you don’t have Docker, kubectl, k3d, or Helm installed, you can use the provided bootstrap script:

# Run the bootstrap script (requires sudo)
./bootstrap.sh

The bootstrap script will:

• Install Docker CE (latest stable version)
• Install kubectl (latest stable version)
• Install k3d (latest version)
• Install Helm (latest version)
• Add your user to the docker group

Note: After Docker installation, you may need to log out and log back in (or run newgrp docker) for the docker group changes to take effect.

Quick Start

Option 1: Run Complete Cluster in Docker (Recommended)

This runs the entire Kubernetes cluster inside a Docker container using k3d:

# Build the complete cluster image
docker build -t wiki-cluster .

# Run the cluster (requires privileged mode and host network for port access)
# Mount docker.sock to use host's Docker daemon (Docker-outside-of-Docker)
# Use -d to run in detached mode, --name to give it a name
docker run --privileged --network host -v /var/run/docker.sock:/var/run/docker.sock -d --name wiki-cluster-container wiki-cluster

The setup script will:

• Build the wiki-service Docker image
• Create a k3d Kubernetes cluster
• Use Traefik ingress (k3d default, no installation needed)
• Deploy all services via Helm chart
• Wait for all pods to be ready
• Keep the container running

Note:

• The container will keep running after setup completes
• Access services at http://localhost:8080 or http://<your-machine-ip>:8080
• To view logs: docker logs wiki-cluster-container
• To stop: docker stop wiki-cluster-container

Option 2: Build and Deploy Manually

Step 1: Build the FastAPI Service Image

cd wiki-service
docker build -t wiki-service:latest .
cd ..

Step 2: Deploy Using Helm Chart

If you have a Kubernetes cluster available:

# Load the image into your cluster (if using k3d)
k3d image import wiki-service:latest

# Install the Helm chart
cd wiki-chart
helm install wiki-release . --wait
cd ..

Or customize the image name:

# Edit values.yaml to set fastapi.image_name
# Then install
helm install wiki-release . --wait

Accessing Services

Once the cluster is running, access services through the ingress at http://localhost:8080 or http://<your-machine-ip>:8080:

FastAPI Endpoints

• Create User: POST http://localhost:8080/users
• Get User: GET http://localhost:8080/user/{id}
• Create Post: POST http://localhost:8080/posts
• Get Post: GET http://localhost:8080/posts/{id}
• Metrics: GET http://localhost:8080/metrics

Grafana Dashboard

• Dashboard URL: http://localhost:8080/grafana/d/creation-dashboard-678/creation or http://<your-machine-ip>:8080/grafana/d/creation-dashboard-678/creation
• Grafana Home: http://localhost:8080/grafana/ or http://<your-machine-ip>:8080/grafana/
• Username: admin
• Password: admin

The dashboard displays:

• Users creation rate over time
• Posts creation rate over time

Dashboard Preview:

The dashboard shows real-time metrics for user and post creation rates, updated every 5 seconds.

API Endpoints

POST /users

Create a new user.

Request body:

{
  "name": "John Doe"
}

Response:

{
  "id": 1,
  "name": "John Doe",
  "created_time": "2025-11-04T10:30:00"
}

POST /posts

Create a new post under a given user.

Request body:

{
  "user_id": 1,
  "content": "This is my first post!"
}

Response:

{
  "post_id": 1,
  "content": "This is my first post!",
  "user_id": 1,
  "created_time": "2025-11-04T10:35:00"
}

GET /user/{id}

Fetch a user by ID.

Response:

{
  "id": 1,
  "name": "John Doe",
  "created_time": "2025-11-04T10:30:00"
}

GET /posts/{id}

Fetch a post by ID.

Response:

{
  "post_id": 1,
  "content": "This is my first post!",
  "user_id": 1,
  "created_time": "2025-11-04T10:35:00"
}

GET /metrics

Prometheus metrics endpoint for monitoring.

Exposed Metrics:

• users_created_total – Counter tracking the total number of users created
• posts_created_total – Counter tracking the total number of posts created

Response: Prometheus text-based exposition format

Example Usage with curl

Create a user:

curl -X POST "http://localhost:8080/users" \
  -H "Content-Type: application/json" \
  -d '{"name": "John Doe"}'

Create a post:

curl -X POST "http://localhost:8080/posts" \
  -H "Content-Type: application/json" \
  -d '{"user_id": 1, "content": "Hello, World!"}'

Get a user:

curl "http://localhost:8080/user/1"

Get a post:

curl "http://localhost:8080/posts/1"

Get Prometheus metrics:

curl "http://localhost:8080/metrics"

Local Development (Without Docker)

For local development without Docker/Kubernetes:

• Install dependencies:

pip install -r requirements.txt

• Start the server:

uvicorn app.main:app --reload

The API will be available at http://localhost:8000

• Access API documentation:

• Swagger UI: http://localhost:8000/docs
• ReDoc: http://localhost:8000/redoc

Note: Local development uses SQLite by default. For PostgreSQL, set the DATABASE_URL environment variable.

Configuration

Environment Variables

• DATABASE_URL: Database connection string (defaults to SQLite for local dev)PostgreSQL format: postgresql+asyncpg://user:password@host:port/databaseSQLite format: sqlite+aiosqlite:///./app.db

Helm Chart Configuration

Edit wiki-chart/values.yaml to customize:

• FastAPI image name and tag
• PostgreSQL credentials
• Resource limits
• Grafana admin credentials
• Data generator settings (Job or CronJob)

Example:

fastapi:
  image_name: wiki-service
  image_tag: latest
  replicas: 1

postgresql:
  database: wiki_db
  username: wiki_user
  password: wiki_password

grafana:
  admin_user: admin
  admin_password: admin

dataGenerator:
  enabled: true
  userCount: 10
  postsPerUser: 5
  cronjob:
    enabled: true              # Set to true for CronJob, false for one-time Job
    schedule: "*/5 * * * *"     # Cron schedule (every 5 minutes in this example)

Cron Schedule Examples:

• “*/5 * * * *” – Every 5 minutes
• “*/10 * * * *” – Every 10 minutes
• “0 * * * *” – Every hour
• “0 */2 * * *” – Every 2 hours
• “0 0 * * *” – Daily at midnight

Resource Constraints

The entire cluster is configured to use at most:

• CPU: 2 cores (500m per service × 4 services)
• RAM: 2GB (512Mi per service × 4 services)
• Disk: 5GB (using emptyDir volumes)

Project Structure

.
├── app/                    # FastAPI application source
│   ├── __init__.py
│   ├── main.py            # FastAPI application and endpoints
│   ├── models.py          # SQLAlchemy models
│   ├── schemas.py         # Pydantic schemas
│   ├── database.py        # Database configuration
│   └── metrics.py         # Prometheus metrics
├── wiki-service/          # FastAPI service Docker setup
│   ├── Dockerfile
│   ├── requirements.txt
│   ├── pyproject.toml
│   └── app/               # Application code (copied from root)
├── wiki-chart/            # Helm chart
│   ├── Chart.yaml
│   ├── values.yaml
│   └── templates/
│       ├── fastapi-deployment.yaml
│       ├── postgresql-deployment.yaml
│       ├── prometheus-deployment.yaml
│       ├── prometheus-configmap.yaml
│       ├── grafana-deployment.yaml
│       ├── grafana-configmap.yaml
│       ├── ingress.yaml
│       ├── data-generator-job.yaml
│       └── data-generator-cronjob.yaml
├── Dockerfile             # Top-level Dockerfile for k3d cluster
├── bootstrap.sh           # Script to install required tools (Docker, kubectl, k3d, Helm)
├── setup-cluster.sh       # Cluster setup script
└── README.md             # This file

Cleanup and Destroy

Option 1: Stop Docker Container (Recommended)

If you ran the cluster using the Docker container approach:

# Find the running container
docker ps | grep wiki-cluster-container

# Stop the container (cluster will still exist)
docker stop wiki-cluster-container

# Remove the container
docker rm wiki-cluster-container

Option 2: Clean Up k3d Cluster Manually

If you deployed manually using k3d:

# Delete the k3d cluster
k3d cluster delete wiki-cluster

# Remove the wiki-service image (optional)
docker rmi wiki-service:latest

Option 3: Uninstall Helm Chart

If you want to keep the cluster but remove the application:

# Uninstall the Helm chart (run inside container)
docker exec wiki-cluster-container sh -c 'export KUBECONFIG=$(k3d kubeconfig write wiki-cluster 2>/dev/null); helm uninstall wiki-release'

# Or if container is stopped, delete cluster and recreate
k3d cluster delete wiki-cluster

Complete Cleanup

To remove everything (cluster, images, and containers):

# Stop and remove Docker container
docker stop wiki-cluster-container 2>/dev/null || true
docker rm wiki-cluster-container 2>/dev/null || true

# Delete k3d cluster
k3d cluster delete wiki-cluster 2>/dev/null || true

# Remove Docker images
docker rmi wiki-cluster:latest 2>/dev/null || true
docker rmi wiki-service:latest 2>/dev/null || true

# Clean up any dangling containers/images
docker container prune -f
docker image prune -f

Viewing Logs and Debugging

# View container logs
docker logs wiki-cluster-container

# Follow logs in real-time
docker logs -f wiki-cluster-container

# Execute commands inside container
docker exec -it wiki-cluster-container sh

# Run kubectl commands inside container
docker exec wiki-cluster-container sh -c 'export KUBECONFIG=$(k3d kubeconfig write wiki-cluster 2>/dev/null); kubectl get pods -A'

Note: Traefik ingress is part of k3d and doesn’t need separate cleanup. It will be removed when you delete the k3d cluster.

Troubleshooting

Docker Connection Issues

If the cluster doesn’t start:

• Ensure Docker is running with –privileged flag
• Mount /var/run/docker.sock to use host Docker daemon (Docker-outside-of-Docker)Correct: docker run –privileged –network host -v /var/run/docker.sock:/var/run/docker.sock wiki-clusterUse –network host to avoid port conflicts
• Check Docker daemon is accessible: docker info
• The container uses the host’s Docker daemon, so ensure Docker is running on the host

Port 8080 Already in Use

If you see “port is already allocated” or “Bind for 0.0.0.0:8080 failed” error:

# Check what's using port 8080
sudo lsof -i :8080
# or
sudo netstat -tulpn | grep :8080

# Stop the service using port 8080, or use a different port
# To use a different port, modify the k3d cluster creation in setup-cluster.sh:
# Change --port "8080:80@loadbalancer" to --port "8081:80@loadbalancer"
# Then rebuild and run the container

Pods Not Starting

Check pod status (run kubectl commands inside the container):

# View all pods
docker exec wiki-cluster-container sh -c 'export KUBECONFIG=$(k3d kubeconfig write wiki-cluster 2>/dev/null); kubectl get pods -A'

# Check specific pod logs
docker exec wiki-cluster-container sh -c 'export KUBECONFIG=$(k3d kubeconfig write wiki-cluster 2>/dev/null); kubectl logs <pod-name>'

# Describe pod for troubleshooting
docker exec wiki-cluster-container sh -c 'export KUBECONFIG=$(k3d kubeconfig write wiki-cluster 2>/dev/null); kubectl describe pod <pod-name>'

Important: Always run kubectl commands inside the wiki-cluster-container to avoid conflicts with other Kubernetes clusters on your machine.

Ingress Not Working

Ensure Traefik is running (k3d default):

# Run kubectl commands inside the container
docker exec wiki-cluster-container sh -c 'export KUBECONFIG=$(k3d kubeconfig write wiki-cluster 2>/dev/null); kubectl get pods -n kube-system | grep traefik'

Check ingress resources:

docker exec wiki-cluster-container sh -c 'export KUBECONFIG=$(k3d kubeconfig write wiki-cluster 2>/dev/null); kubectl get ingress -A'

Database Connection Issues

Verify PostgreSQL is running and FastAPI has correct DATABASE_URL:

# Check FastAPI logs
docker exec wiki-cluster-container sh -c 'export KUBECONFIG=$(k3d kubeconfig write wiki-cluster 2>/dev/null); kubectl logs deployment/fastapi'

# Check PostgreSQL status
docker exec wiki-cluster-container sh -c 'export KUBECONFIG=$(k3d kubeconfig write wiki-cluster 2>/dev/null); kubectl get pods -l app=postgresql'

Prometheus Metrics

The service exposes Prometheus metrics at /metrics endpoint. These metrics can be scraped by a Prometheus server for monitoring and alerting.

Available Metrics:

• users_created_total – Total count of users created since the service started
• posts_created_total – Total count of posts created since the service started

Example Prometheus Configuration:

scrape_configs:
  - job_name: "user_post_api"
    static_configs:
      - targets: ["fastapi:8000"]
    metrics_path: "/metrics"

Database

The service uses PostgreSQL in production (via Helm chart) and SQLite for local development. The database connection is configured via the DATABASE_URL environment variable.

To switch databases:

• Update DATABASE_URL environment variable
• Install the appropriate database driver (e.g., asyncpg for PostgreSQL)
• Update requirements.txt accordingly

---

Originally published by Omer Sen on LinkedIn, 11 November 2025.