DevOps Automation in Action: Building a CI/CD Pipeline That Deploys 10+ Times Daily

📌 This article was AI-assisted generated and human-reviewed | TechPassive — An AI-driven content testing site focused on real tool reviews

In modern software development, delivery speed determines competitive advantage. Traditional manual deployment practices can no longer meet the demands of rapid business iteration. DevOps automation pipelines have become essential infrastructure for engineering teams. This guide covers the complete process of building an enterprise-grade CI/CD pipeline that enables fully automated deployment from code commit to production, easily supporting 10+ daily releases.

What Is a DevOps Automation Pipeline

A DevOps automation pipeline is a toolchain connecting the entire lifecycle from code writing, building, testing, and deployment. The moment a developer pushes code to the repository, the pipeline automatically triggers a series of predefined tasks: code pulling, dependency installation, unit testing, integration testing, image building, security scanning, pre-production verification, and finally deploying verified artifacts to target environments. The entire process requires no human intervention, reducing average time from several hours to a dozen minutes.

Continuous Integration requires developers to frequently merge code to the main branch, with each merge automatically triggering builds and tests. Continuous Delivery automatically pushes built artifacts to various environments on top of CI. Continuous Deployment achieves fully automated production releases. Teams can choose the appropriate automation level based on their risk tolerance.

Core Technology Stack Selection

GitLab CI/CD: Open Source and Powerful Pipeline Engine

GitLab's built-in CI/CD functionality works out of the box. Pipeline stages and jobs are declared through . Runners as executors support multiple runtime environments including Docker, Shell, and Kubernetes, natively coupled with code repositories. GitLab CI's advantages include simple configuration, comprehensive permission systems, and the fact that the free tier already includes complete pipeline functionality.

stages:
  - build
  - test
  - deploy

build_job:
  stage: build
  image: maven:3.8-openjdk-17
  script:
    - mvn clean package -DskipTests=false
  artifacts:
    paths:
      - target/*.jar

Jenkins: Flexible and Extensible Automation Engine

Jenkins is renowned for its plugin ecosystem, with plugins available for virtually every mainstream development tool. The Blue Ocean interface provides a modern visual pipeline editing experience. Jenkinsfile supports both declarative and scripted approaches, accommodating complex business scenarios. Many traditional enterprises choose Jenkins for its high customizability and excellent compatibility with legacy systems.

Argo CD: Cloud-Native GitOps Deployment Tool

Argo CD focuses on continuous deployment for Kubernetes environments, following the GitOps principle—all environment configurations are stored in Git repositories, and Argo CD continuously monitors cluster state, comparing it with Git-declared configurations, automatically syncing any differences. Developers only need to modify configuration files and push; Argo CD handles the rest. This model naturally ensures environment consistency and provides audit traceability.

Building a Complete Automation Pipeline

Phase 1: Environment Preparation and Toolchain Installation

Infrastructure needs to be prepared before building the pipeline. Linux servers should have 4+ CPU cores and 8GB+ RAM. The operating system should be Ubuntu 22.04 LTS or CentOS Stream. The Docker engine is the underlying dependency for most CI tools and must be installed first.

# Install Docker
curl -fsSL https://get.docker.com | bash
systemctl enable docker
systemctl start docker

# Install GitLab Runner
curl -fsSL https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | bash
apt-get install gitlab-runner
gitlab-runner register

During Runner registration, you need to provide the GitLab instance URL and registration token, found in GitLab admin settings under Settings → CI/CD → Runners. It is recommended to configure separate Runner tags for different environments, such as , , and , facilitating on-demand scheduling in pipelines.

Phase 2: Writing Pipeline Configuration Files

Using a Java Spring Boot project as an example, a complete pipeline includes stages such as building, testing, security scanning, image building, and deployment. Each stage failure terminates the pipeline, preventing defective code from progressing to downstream stages.

variables:
  DOCKER_DRIVER: overlay2
  MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"

cache:
  paths:
    - .m2/repository
    - target/

stages:
  - build
  - test
  - security
  - dockerize
  - deploy-staging
  - deploy-production

build:
  stage: build
  image: maven:3.8-openjdk-17
  script:
    - mvn clean compile
  artifacts:
    paths:
      - target/classes/

test:
  stage: test
  image: maven:3.8-openjdk-17
  script:
    - mvn test
  coverage: '/Total:[^\d]*(\d+)/'

security-scan:
  stage: security
  image: aquasec/trivy:latest
  script:
    - trivy image --severity HIGH,CRITICAL $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
  allow_failure: true

docker-build:
  stage: dockerize
  image: docker:latest
  services:
    - docker:dind
  script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA .
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    - docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:latest
    - docker push $CI_REGISTRY_IMAGE:latest

Phase 3: Kubernetes Deployment Configuration

Containerized applications ultimately run in Kubernetes clusters. Deployment and Service definitions are standard configurations. HPA (Horizontal Pod Autoscaler) ensures service availability under high load, while PodDisruptionBudget prevents service disruptions caused by simultaneously taking down too many replicas.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-deployment
  labels:
    app: myapp
spec:
  replicas: 3
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
      - name: myapp
        image: registry.gitlab.com/username/myapp:latest
        ports:
        - containerPort: 8080
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /actuator/health
            port: 8080
          initialDelaySeconds: 60
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /actuator/health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: myapp-service
spec:
  selector:
    app: myapp
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080
  type: ClusterIP
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: myapp-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: myapp-deployment
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70

Phase 4: Production Deployment Strategy

Production deployments require extra caution. Blue-green deployment and canary release are the two mainstream strategies. Blue-green deployment maintains two identical environments, switching traffic instantly; issues can be rolled back immediately. Canary release gradually migrates traffic from old to new versions, judging new version performance through monitoring metrics.

# Argo CD Application configuration
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp-production
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://registry.gitlab.com/username/myapp.git
    targetRevision: HEAD
    path: k8s/production
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

Pipeline Security Hardening

Security scanning must be integrated into core pipeline stages, not added as an afterthought. Trivy or Snyk scans images immediately after building; high-risk vulnerabilities automatically block deployment. SAST (Static Application Security Testing) catches potential security flaws during code compilation. Dependency scanning ensures third-party libraries have no known CVEs.

Secret management requires never hardcoding passwords in configuration files. HashiCorp Vault or Kubernetes Secrets combined with RBAC permission control enables secure storage and on-demand access to sensitive information. Environment variables declared as secrets are also automatically masked in pipeline execution logs.

Monitoring and Alerting Systems

The ultimate value of a pipeline lies in continuous improvement of business metrics. Grafana displays key indicator trends such as build frequency, success rate, and average build duration. Prometheus collects timing data from each stage, with Grafana alerting rules notifying responsible parties immediately when pipeline anomalies occur.

name: pipeline-alerts

# Prometheus alerting rules
groups:
  rules:
  - alert: PipelineFailureRateHigh
    expr: rate(pipeline_jobs_failed_total[5m]) / rate(pipeline_jobs_total[5m]) > 0.1
    for: 5m
    labels:
      severity: critical
    annotations:
      summary: "Pipeline failure rate exceeds 10%"
      description: "Pipeline failure rate over the last 5 minutes is {{ $value | humanizePercentage }}"

Practical Results and Benefits

After implementing a complete DevOps automation pipeline, a team's delivery capability undergoes a qualitative transformation. Deployment frequency increases from once weekly to 10+ times daily. Change lead time shrinks from several days to minutes. MTTR (Mean Time To Recovery) decreases significantly, while defect escape rates drop markedly.

A real case from a mid-sized technology team shows that after adopting the automation pipeline, QA personnel were freed from the shackles of manual regression testing, focusing on exploratory testing and user experience optimization. The fastest time from developer code submission to production environment completion was just 8 minutes, truly achieving the "code as product" development philosophy.

Starting Your Automation Journey

Building a DevOps pipeline is not an overnight project. It is recommended to start with a small-scale pilot. Choose a non-core business line to validate the complete process, accumulate experience, then expand horizontally to the entire team. Continuous improvement is key—every bottleneck discovered is an opportunity for optimization. The pipeline will gradually perfect through iteration.

For tool selection, there is no need to pursue the newest or most comprehensive options. Solve actual problems first. Startup teams can quickly get results starting with GitLab CI. Medium-to-large enterprises can consider Jenkins plus Argo CD combinations to cover complex scenarios. Consistently investing in automation construction will yield compound efficiency returns in future development.