GitOps for Kubernetes: Implementation Guide (2025)

Introduction to GitOps: Git as Single Source of Truth

GitOps is a modern operational framework for Kubernetes where Git repositories serve as the single source of truth for declarative infrastructure and application configuration, with automated agents continuously synchronizing the actual cluster state to match the desired state defined in Git. Instead of manually running kubectl apply commands or using CI/CD pipelines that push changes directly to clusters, GitOps inverts the model: you commit YAML manifests to Git, and specialized operators running in your cluster (ArgoCD, Flux) automatically detect changes and apply them, creating a pull-based deployment model that provides numerous operational advantages including complete audit trails through Git history, easy rollback by reverting Git commits, declarative disaster recovery where rebuilding a cluster just means pointing a fresh cluster at your Git repository, and enhanced security since cluster credentials don't need to exist outside the cluster.

The GitOps methodology has gained massive adoption because it solves fundamental problems in Kubernetes deployment workflows. Traditional CI/CD pipelines where Jenkins or GitLab CI runs kubectl apply to push changes require storing cluster credentials in CI systems creating security risks, lack comprehensive audit trails since pipeline execution logs aren't as permanent or queryable as Git history, make rollbacks complex requiring re-running previous pipeline versions or manual kubectl commands, create configuration drift when emergency hotfixes are applied manually but not committed to Git, and struggle with multi-cluster deployments requiring complex logic to deploy the same application with environment-specific variations across development, staging, and production clusters.

This comprehensive guide teaches you everything about implementing GitOps for Kubernetes in production environments, covering: core GitOps principles and how they differ from traditional CI/CD push-based deployments, the two dominant GitOps tools ArgoCD and Flux with detailed comparison helping you choose, complete ArgoCD implementation from installation through application deployment with best practices, Flux implementation guide including bootstrap, source configuration, and kustomization, repository structure patterns organizing Kubernetes manifests for multiple environments and applications, handling secrets in GitOps using Sealed Secrets or External Secrets Operator since committing plain Secrets to Git violates security, implementing progressive delivery and canary deployments with Flagger, multi-cluster GitOps managing deployments across development, staging, and production clusters, and how Atmosly's Pipeline Builder integrates with GitOps workflows providing visual pipeline creation that generates ArgoCD or Flux configurations, deployment coordination across multiple services, automated rollback on health check failures, and environment cloning for testing changes before production.

By implementing GitOps following this guide, you'll achieve declarative, version-controlled, auditable, and easily rollback-able Kubernetes deployments while maintaining security and enabling team collaboration through Git's familiar pull request workflows.

Core GitOps Principles

The Four GitOps Principles

1. Declarative Configuration

Infrastructure and applications described declaratively using Kubernetes YAML manifests, not imperative scripts. Git stores the desired state (Deployment with 5 replicas), not commands to achieve it (kubectl scale --replicas=5).

2. Versioned and Immutable

All configuration stored in Git providing complete version history, ability to view any previous state, and rollback by reverting commits. Git commits are immutable—history cannot be rewritten protecting audit trail.

3. Pulled Automatically

Changes pulled from Git by agents running in cluster (ArgoCD, Flux), not pushed by external CI/CD. Cluster credentials stay in cluster, never exported to CI systems. Agents continuously reconcile actual state to match Git.

4. Continuously Reconciled

Agents continuously verify cluster state matches Git, automatically fixing drift. If someone manually runs kubectl delete to remove a deployment, GitOps operator detects missing resource and recreates it from Git within minutes.

GitOps vs Traditional CI/CD

GitOps Tools: ArgoCD vs Flux

ArgoCD: The Declarative GitOps CD Tool

ArgoCD, created by Intuit and now a CNCF graduated project, is the most popular GitOps tool for Kubernetes with rich UI, extensive features, and mature ecosystem.

Key Features:

• Web UI: Rich interface showing application health, sync status, resource tree, diff viewer
• Multi-Cluster Support: Manage deployments across hundreds of clusters from single ArgoCD instance
• Application CRD: Defines what to deploy (Git repo + path + cluster)
• Sync Strategies: Automatic (changes auto-deployed) or manual (require approval)
• Sync Waves: Deploy resources in specific order (databases before applications)
• Health Assessment: Knows if Deployment healthy (all replicas available)
• Kustomize and Helm: Native support for both templating tools
• RBAC: Fine-grained access control (who can deploy what where)
• SSO Integration: OIDC, SAML, GitHub, GitLab authentication

ArgoCD Architecture:

┌──────────────┐
│  Git Repo    │  ← Developers commit YAML
│  (GitHub)    │
└──────┬───────┘
       │
       │ Monitors for changes
       ↓
┌──────────────────┐
│  ArgoCD Server   │  ← Runs in cluster
│  (Controller)    │
└──────┬───────────┘
       │
       │ Applies changes
       ↓
┌──────────────────┐
│  Kubernetes      │  ← Target cluster(s)
│  Cluster         │
└──────────────────┘

Flux: The GitOps Toolkit

Flux v2, a CNCF graduated project, is a toolkit for building GitOps workflows with modular controllers.

Key Features:

• Multi-Tenancy: Different teams manage different Git repos/paths independently
• Modular Controllers: Source Controller (Git), Kustomize Controller (manifests), Helm Controller (charts), Notification Controller (alerts)
• GitRepository CRD: Defines Git source
• Kustomization CRD: Defines what to apply from Git source
• Flux Bootstrap: Self-managing (Flux manages its own updates via Git)
• Image Automation: Automatically updates image tags in Git when new images built
• Notification: Slack, MS Teams alerts on sync success/failure
• Progressive Delivery: Flagger integration for canary deployments

ArgoCD vs Flux Comparison

Recommendation: ArgoCD if you value UI and ease of use. Flux if you prefer automation and CLI workflows. Both are production-ready.

Implementing ArgoCD: Complete Guide

Step 1: Install ArgoCD

# Create namespace
kubectl create namespace argocd

# Install ArgoCD
kubectl apply -n argocd -f \\
  https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# Wait for pods to be ready
kubectl wait --for=condition=Ready pods --all -n argocd --timeout=300s

# Get initial admin password
kubectl -n argocd get secret argocd-initial-admin-secret \\
  -o jsonpath="{.data.password}" | base64 -d

# Access ArgoCD UI
kubectl port-forward svc/argocd-server -n argocd 8080:443
# Visit: https://localhost:8080
# Login: admin / (password from above)

Step 2: Configure Git Repository

Structure your Git repository:

my-k8s-manifests/
├── apps/
│   ├── frontend/
│   │   ├── base/
│   │   │   ├── deployment.yaml
│   │   │   ├── service.yaml
│   │   │   └── kustomization.yaml
│   │   ├── overlays/
│   │   │   ├── dev/
│   │   │   │   └── kustomization.yaml
│   │   │   ├── staging/
│   │   │   │   └── kustomization.yaml
│   │   │   └── production/
│   │   │       └── kustomization.yaml
│   ├── backend/
│   │   └── ... (same structure)
└── infrastructure/
    ├── ingress-nginx/
    ├── cert-manager/
    └── monitoring/

Step 3: Create ArgoCD Application

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: frontend-production
  namespace: argocd
spec:
  project: default
  
  source:
    repoURL: https://github.com/myorg/k8s-manifests
    targetRevision: main
    path: apps/frontend/overlays/production
  
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  
  syncPolicy:
    automated:
      prune: true      # Delete resources not in Git
      selfHeal: true   # Auto-fix manual changes
    syncOptions:
    - CreateNamespace=true

# Apply Application resource
kubectl apply -f frontend-app.yaml

# ArgoCD automatically:
# 1. Connects to GitHub repo
# 2. Reads manifests from apps/frontend/overlays/production
# 3. Applies to production namespace
# 4. Monitors for Git changes
# 5. Auto-syncs on new commits

Implementing Flux: Complete Guide

Step 1: Bootstrap Flux

# Install Flux CLI
brew install fluxcd/tap/flux

# Bootstrap Flux (creates components in cluster + commits to Git)
flux bootstrap github \\
  --owner=myorg \\
  --repository=k8s-manifests \\
  --branch=main \\
  --path=clusters/production \\
  --personal

# This:
# 1. Installs Flux controllers in flux-system namespace
# 2. Commits Flux manifests to your Git repo
# 3. Configures Flux to sync from repo
# 4. Flux now manages itself (self-updating)

Step 2: Define GitRepository Source

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: app-repo
  namespace: flux-system
spec:
  interval: 1m  # Check Git every minute
  url: https://github.com/myorg/k8s-manifests
  ref:
    branch: main
  secretRef:
    name: github-credentials  # If private repo

Step 3: Create Kustomization for Application

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: frontend
  namespace: flux-system
spec:
  interval: 5m
  path: ./apps/frontend/overlays/production
  prune: true  # Delete resources not in Git
  sourceRef:
    kind: GitRepository
    name: app-repo
  healthChecks:
  - apiVersion: apps/v1
    kind: Deployment
    name: frontend
    namespace: production

GitOps Best Practices for Production

1. Repository Structure: Monorepo vs Multi-Repo

Monorepo (Recommended for Most Teams):

• Single Git repository containing all applications and infrastructure
• Easier to manage, atomic changes across multiple apps
• Clear directory structure separating apps, environments, infrastructure

Multi-Repo (For Large Organizations):

• Separate repo per application or team
• Better access control (team A can't modify team B's repo)
• More complex (managing multiple repos)

2. Handling Secrets Securely

Never commit plaintext Secrets to Git!

Option 1: Sealed Secrets

# Install Sealed Secrets controller
kubectl apply -f https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.24.0/controller.yaml

# Encrypt secret
echo -n mypassword | kubectl create secret generic db-password \\
  --dry-run=client --from-file=password=/dev/stdin -o yaml | \\
  kubeseal -o yaml > sealed-db-password.yaml

# Commit sealed-db-password.yaml to Git (safe, encrypted)
git add sealed-db-password.yaml
git commit -m "Add database password (sealed)"

Option 2: External Secrets Operator

# Store secret in AWS Secrets Manager
aws secretsmanager create-secret \\
  --name prod/database/password \\
  --secret-string "mypassword"

# Commit ExternalSecret to Git (references AWS, not plaintext)
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-password
spec:
  secretStoreRef:
    name: aws-secretstore
  target:
    name: db-secret
  data:
  - secretKey: password
    remoteRef:
      key: prod/database/password

3. Environment-Specific Configuration with Kustomize

# Base configuration (shared)
# apps/frontend/base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
spec:
  replicas: 3  # Default
  template:
    spec:
      containers:
      - name: app
        image: frontend:v1.0.0
        resources:
          requests:
            cpu: 100m
            memory: 128Mi

# Development overlay
# apps/frontend/overlays/dev/kustomization.yaml
bases:
- ../../base
patchesStrategicMerge:
- patch.yaml

# apps/frontend/overlays/dev/patch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
spec:
  replicas: 1  # Only 1 replica in dev

# Production overlay
# apps/frontend/overlays/production/kustomization.yaml
bases:
- ../../base
patchesStrategicMerge:
- patch.yaml

# apps/frontend/overlays/production/patch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
spec:
  replicas: 10  # 10 replicas in production
  template:
    spec:
      containers:
      - name: app
        resources:
          requests:
            cpu: 500m      # More resources in prod
            memory: 512Mi

4. Implementing Progressive Delivery with Flagger

Flagger automates canary deployments with automatic rollback on errors:

# Install Flagger
kubectl apply -f https://raw.githubusercontent.com/fluxcd/flagger/main/artifacts/flagger/crd.yaml
helm upgrade -i flagger flagger/flagger \\
  --namespace=istio-system \\
  --set meshProvider=istio

# Define Canary
apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
  name: frontend
  namespace: production
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: frontend
  service:
    port: 80
  analysis:
    interval: 1m
    threshold: 5
    maxWeight: 50
    stepWeight: 10
    metrics:
    - name: request-success-rate
      thresholdRange:
        min: 99  # Rollback if success rate < 99%
    - name: request-duration
      thresholdRange:
        max: 500  # Rollback if p99 latency > 500ms

# Flagger gradually shifts traffic: 0% → 10% → 20% → 30% → 50%
# Monitors metrics at each step
# Auto-rollback if metrics degrade

5. Multi-Cluster GitOps

Pattern: App of Apps

# Root application defining all other applications
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: root-app
  namespace: argocd
spec:
  source:
    repoURL: https://github.com/myorg/k8s-manifests
    path: clusters/production
  destination:
    server: https://kubernetes.default.svc
  syncPolicy:
    automated: {}

# clusters/production/ contains Application manifests for:
# - frontend
# - backend
# - database
# - monitoring
# Each deployed automatically when root-app syncs

How Atmosly's Pipeline Builder Integrates with GitOps

While GitOps tools like ArgoCD and Flux handle the deployment automation, Atmosly's Pipeline Builder complements them by simplifying pipeline creation and providing deployment intelligence.

Visual Pipeline Creation (≤20% Atmosly Content)

Atmosly Pipeline Builder provides:

• Visual Editor: Create CI/CD pipelines through drag-and-drop interface generating ArgoCD Application manifests or Flux Kustomization automatically
• GitOps Configuration Generator: Answers questions about your setup (Git repo URL, target cluster, deployment path) and generates complete ArgoCD or Flux configuration files ready to commit to Git
• Deployment Coordination: For microservices architectures, coordinates deployments across multiple services ensuring dependencies deploy in correct order (databases before applications using them)
• Health Check Integration: Configures ArgoCD/Flux health checks, adds custom health assessments beyond default Kubernetes readiness (database connection tests, API endpoint validation)
• Automated Rollback: If deployment causes elevated error rates or latency degradation detected by Atmosly monitoring, triggers automatic Git revert and ArgoCD/Flux re-sync to previous working version
• Environment Cloning: Clone entire GitOps-managed environment (production → staging for testing) with automated manifest transformation (different namespaces, resource limits, replicas)

Atmosly enhances GitOps workflows but doesn't replace ArgoCD or Flux—it provides the layer above helping teams implement GitOps more easily.

Troubleshooting GitOps Deployments

Common Issues and Solutions

Issue 1: Application Shows "OutOfSync"

# Check sync status
kubectl get application frontend -n argocd

# View diff
argocd app diff frontend

# Manual sync if automatic sync disabled
argocd app sync frontend

Issue 2: Sync Failed with Errors

# Check sync operation status
argocd app get frontend

# View detailed error
kubectl logs -n argocd deployment/argocd-application-controller

# Common errors:
# - YAML syntax error (validate locally first)
# - Resource already exists (set sync policy to prune/replace)
# - RBAC permissions (ArgoCD service account lacks permission)

Issue 3: Manual Changes Keep Getting Reverted

This is GitOps working correctly! Manual kubectl changes are detected as drift and automatically reverted to match Git.

Solution: If change is needed, commit to Git first, then ArgoCD applies it.

Conclusion: GitOps for Reliable Kubernetes Deployments

GitOps transforms Kubernetes deployments from error-prone manual operations to declarative, version-controlled, auditable, automatically reconciled systems. Whether you choose ArgoCD for its UI and ease of use, or Flux for its automation and modularity, GitOps principles provide significant operational benefits.

Key Implementation Steps:

1. Structure Git repository with clear separation (apps, infrastructure, environments)
2. Install ArgoCD or Flux in your cluster
3. Define Applications or Kustomizations pointing to Git paths
4. Enable automatic sync for continuous deployment
5. Handle secrets with Sealed Secrets or External Secrets
6. Implement progressive delivery with Flagger for canary rollouts
7. Use Atmosly's Pipeline Builder to simplify setup and add deployment intelligence

GitOps Benefits:

• ✅ Single source of truth (Git)
• ✅ Complete audit trail via Git history
• ✅ Easy rollback (git revert)
• ✅ Declarative disaster recovery
• ✅ No cluster credentials in CI/CD
• ✅ Automatic drift correction
• ✅ Pull request workflows for changes

Ready to implement GitOps for your Kubernetes deployments? Start with Atmosly's Pipeline Builder to generate ArgoCD or Flux configurations visually, then enhance with automated rollback, health checks, and deployment coordination across your microservices.