# Shell Scripting — Advanced Real-Time DevOps Automation

In the previous article, we learned about **Shell Scripting — Security & Best Practices**.

Now let's move to the next level: **using Shell scripting for real-world DevOps automation**.

Shell scripting is not only about executing Linux commands. In DevOps, we can use Shell scripts to automate **AWS operations, Docker, Kubernetes, CI/CD pipelines, monitoring, deployments, backups, health checks, and incident-response tasks**.

* * *

## 1\. What is Advanced DevOps Automation?

Advanced automation means combining Shell scripting with DevOps tools and cloud platforms to reduce manual work.

```text
Developer
    ↓
Git
    ↓
CI/CD Pipeline
    ↓
Shell Script
    ↓
Build / Test / Scan
    ↓
Docker
    ↓
Kubernetes / AWS
    ↓
Monitoring
```

The goal is:

> **Automate repetitive tasks, reduce human errors, and make deployments faster and more reliable.**

* * *

## 2\. Real-Time DevOps Use Cases

Shell scripting can automate:

*   Application deployment
    
*   Server provisioning
    
*   AWS resource operations
    
*   Docker image management
    
*   Kubernetes deployments
    
*   Log collection
    
*   Health checks
    
*   Backup and restore
    
*   Disk and memory monitoring
    
*   Service restart
    
*   CI/CD tasks
    
*   Cleanup operations
    
*   Incident response
    

* * *

## 3\. Advanced Shell Script Structure

A production-oriented script can follow this structure:

```bash
#!/bin/bash

set -euo pipefail

# Configuration
APP_NAME="myapp"
APP_DIR="/opt/myapp"

# Functions
check_dependencies() {
    echo "Checking dependencies..."
}

deploy_application() {
    echo "Deploying application..."
}

health_check() {
    echo "Running health check..."
}

main() {
    check_dependencies
    deploy_application
    health_check
}

main "$@"
```

This structure makes the script easier to understand and maintain.

* * *

## 4\. Check Dependencies

Before executing automation, check whether required commands exist.

```bash
check_dependencies() {
    local commands=("git" "docker" "curl")

    for cmd in "${commands[@]}"; do
        if ! command -v "$cmd" >/dev/null 2>&1; then
            echo "ERROR: $cmd is not installed"
            exit 1
        fi
    done
}
```

Run:

```bash
check_dependencies
```

This prevents the script from failing unexpectedly later.

* * *

## 5\. Using Functions for Automation

Instead of writing one huge script:

```text
Build
Deploy
Verify
Cleanup
```

create separate functions:

```bash
build_app() {
    echo "Building application..."
}

deploy_app() {
    echo "Deploying application..."
}

verify_app() {
    echo "Verifying application..."
}

cleanup() {
    echo "Cleaning temporary files..."
}
```

Then:

```bash
build_app
deploy_app
verify_app
cleanup
```

* * *

## 6\. Automating Git Operations

Example:

```bash
#!/bin/bash

set -euo pipefail

APP_DIR="/opt/myapp"

cd "$APP_DIR"

git fetch origin
git checkout main
git pull --ff-only origin main

echo "Latest code pulled successfully"
```

Using `--ff-only` helps avoid silently creating an unexpected merge commit during an automated deployment.

* * *

## 7\. Application Deployment Automation

A simple deployment workflow:

```bash
#!/bin/bash

set -euo pipefail

APP_DIR="/opt/myapp"

echo "Starting deployment..."

cd "$APP_DIR"

git pull --ff-only origin main

echo "Installing dependencies..."
# ./install.sh

echo "Building application..."
# ./build.sh

echo "Restarting service..."
sudo systemctl restart myapp

echo "Deployment completed"
```

* * *

## 8\. Add Health Checks

Deployment should not finish simply because the restart command succeeded.

Check whether the application is actually responding.

Example:

```bash
health_check() {
    local url="http://localhost:8080/health"

    if curl -fsS "$url" >/dev/null; then
        echo "Application is healthy"
    else
        echo "Application health check failed"
        exit 1
    fi
}
```

Then:

```bash
health_check
```

* * *

## 9\. Retry Logic

Network operations can fail temporarily.

Instead of immediately failing:

```bash
for i in {1..5}; do
    if curl -fsS "http://localhost:8080/health" >/dev/null; then
        echo "Health check successful"
        exit 0
    fi

    echo "Attempt $i failed. Retrying..."
    sleep 5
done

echo "Health check failed after retries"
exit 1
```

This is useful for applications that need a few seconds to start.

* * *

## 10\. Docker Automation

Shell scripts can automate Docker operations.

Example:

```bash
#!/bin/bash

set -euo pipefail

IMAGE="myapp:latest"

docker build -t "$IMAGE" .

docker stop myapp 2>/dev/null || true
docker rm myapp 2>/dev/null || true

docker run -d \
    --name myapp \
    -p 8080:8080 \
    "$IMAGE"

echo "Docker deployment completed"
```

* * *

## 11\. Docker Image Cleanup

Unused Docker resources can consume disk space.

Check:

```bash
docker system df
```

For cleanup, choose commands carefully according to your environment.

For example:

```bash
docker image prune
```

Always understand what will be removed before automating destructive cleanup commands.

* * *

## 12\. AWS CLI Automation

Shell scripts can automate AWS operations using the AWS CLI.

For example:

```bash
aws sts get-caller-identity
```

This verifies which AWS identity is being used.

List EC2 instances:

```bash
aws ec2 describe-instances
```

List S3 buckets:

```bash
aws s3 ls
```

* * *

## 13\. AWS EC2 Automation

Example:

```bash
INSTANCE_ID="i-xxxxxxxxxxxxxxxxx"

aws ec2 describe-instances \
    --instance-ids "$INSTANCE_ID" \
    --query 'Reservations[].Instances[].State.Name' \
    --output text
```

The script can then make decisions based on the returned state.

* * *

## 14\. AWS S3 Backup Automation

For example:

```bash
BACKUP_DIR="/backup"
S3_BUCKET="s3://my-company-backups"

aws s3 sync "$BACKUP_DIR" "$S3_BUCKET"
```

Workflow:

```text
Application
     ↓
Backup Script
     ↓
Local Backup
     ↓
AWS S3
     ↓
Remote Backup Storage
```

* * *

## 15\. Kubernetes Automation

Shell scripts can also interact with Kubernetes through `kubectl`.

Check nodes:

```bash
kubectl get nodes
```

Check pods:

```bash
kubectl get pods
```

Check deployments:

```bash
kubectl get deployments
```

Restart a deployment:

```bash
kubectl rollout restart deployment/myapp
```

* * *

## 16\. Kubernetes Deployment Automation

Example:

```bash
#!/bin/bash

set -euo pipefail

NAMESPACE="production"
DEPLOYMENT="myapp"

kubectl -n "$NAMESPACE" rollout restart deployment "$DEPLOYMENT"

kubectl -n "$NAMESPACE" rollout status \
    deployment "$DEPLOYMENT" \
    --timeout=180s

echo "Deployment completed successfully"
```

This is much better than restarting pods manually.

* * *

## 17\. Kubernetes Health Verification

After deployment:

```bash
kubectl -n production get pods
```

Check rollout:

```bash
kubectl -n production rollout status deployment/myapp
```

Check recent events:

```bash
kubectl -n production get events --sort-by=.lastTimestamp
```

* * *

## 18\. CI/CD Integration

Shell scripts are commonly executed from CI/CD tools.

Example Jenkins pipeline:

```text
Git
 ↓
Jenkins
 ↓
Shell Script
 ↓
Build
 ↓
Unit Tests
 ↓
Security Scan
 ↓
Docker Build
 ↓
Push Image
 ↓
Kubernetes Deployment
 ↓
Health Check
```

A Shell script can be one component of the pipeline rather than the entire deployment system.

* * *

## 19\. Jenkins Shell Example

A Jenkins step could execute:

```bash
#!/bin/bash

set -euo pipefail

echo "Building application..."

./build.sh

echo "Running tests..."

./test.sh

echo "Build and test completed successfully"
```

If the script exits with a non-zero status, Jenkins can mark the build as failed.

* * *

## 20\. Monitoring Automation

Shell scripting can monitor system resources.

### Disk:

```bash
df -h
```

### Memory:

```bash
free -h
```

### CPU:

```bash
uptime
```

### Processes:

```bash
ps aux
```

* * *

## 21\. Disk Monitoring Script

```bash
#!/bin/bash

THRESHOLD=80

USAGE=$(df / | awk 'NR==2 {gsub("%","",$5); print $5}')

if [ "$USAGE" -ge "$THRESHOLD" ]; then
    echo "WARNING: Disk usage is ${USAGE}%"
    exit 1
else
    echo "Disk usage is ${USAGE}%"
fi
```

This can be scheduled through Cron or integrated into monitoring workflows.

* * *

## 22\. Service Monitoring

Check whether a service is running:

```bash
if systemctl is-active --quiet nginx; then
    echo "Nginx is running"
else
    echo "Nginx is not running"
    exit 1
fi
```

For production, automatic restart should be used carefully and preferably alongside proper service management and alerting.

* * *

## 23\. Automated Log Analysis

Shell scripts can search logs for errors.

Example:

```bash
ERROR_COUNT=$(grep -ci "error" /var/log/myapp/application.log || true)

echo "Errors found: $ERROR_COUNT"
```

You could trigger an alert if the count exceeds a defined threshold.

* * *

## 24\. Log Monitoring Workflow

```text
Application Logs
       ↓
Shell Script
       ↓
Search Errors
       ↓
Count Errors
       ↓
Threshold Check
       ↓
Alert / Incident
```

* * *

## 25\. API Automation with `curl`

Shell scripts can call REST APIs.

Example:

```bash
curl -fsS \
    -H "Content-Type: application/json" \
    "https://example.com/api/health"
```

For authenticated APIs, use a secure token mechanism rather than hardcoding credentials.

* * *

## 26\. JSON Processing with `jq`

`jq` is useful for processing JSON output.

Example:

```bash
aws ec2 describe-instances \
    --output json |
jq '.Reservations[].Instances[].InstanceId'
```

This can be used in automation workflows where structured API output needs to be processed.

* * *

## 27\. Automated Server Health Check

A practical script could check:

```text
CPU
Memory
Disk
Services
Network
Application
```

Example:

```bash
#!/bin/bash

echo "===== Server Health Check ====="

echo "Hostname:"
hostname

echo "Uptime:"
uptime

echo "Disk:"
df -h /

echo "Memory:"
free -h

echo "Nginx:"
systemctl is-active nginx || true

echo "Network:"
ping -c 2 8.8.8.8
```

* * *

## 28\. Automated Incident Response

Suppose an application is unavailable.

A script could:

```text
Detect Failure
      ↓
Collect Logs
      ↓
Check Service
      ↓
Check Disk
      ↓
Check Memory
      ↓
Check Application
      ↓
Restart if Approved
      ↓
Run Health Check
      ↓
Generate Report
```

This can significantly reduce manual troubleshooting time.

* * *

## 29\. Command-Line Arguments

Make scripts reusable by accepting arguments.

Example:

```bash
#!/bin/bash

ENVIRONMENT="${1:-}"

if [ -z "$ENVIRONMENT" ]; then
    echo "Usage: $0 <environment>"
    exit 1
fi

echo "Deploying to: $ENVIRONMENT"
```

Run:

```bash
./deploy.sh staging
```

or:

```bash
./deploy.sh production
```

* * *

## 30\. Using `case` for Environments

```bash
case "$ENVIRONMENT" in
    dev)
        echo "Deploying to development"
        ;;
    staging)
        echo "Deploying to staging"
        ;;
    production)
        echo "Deploying to production"
        ;;
    *)
        echo "Invalid environment"
        exit 1
        ;;
esac
```

This prevents unexpected environment values from being accepted.

* * *

## 31\. Arrays for Multiple Servers

Bash arrays can store multiple targets:

```bash
SERVERS=(
    "server1"
    "server2"
    "server3"
)

for server in "${SERVERS[@]}"; do
    echo "Checking $server"
done
```

This can be useful for controlled automation across multiple systems.

* * *

## 32\. Parallel Automation

For independent tasks, background execution can sometimes reduce execution time.

Example:

```bash
check_server "server1" &
check_server "server2" &
check_server "server3" &

wait
```

`wait` ensures the script waits for the background jobs to finish.

Use parallelism carefully, especially when operations affect shared resources.

* * *

## 33\. Real-Time Deployment Script

A simplified deployment workflow:

```bash
#!/bin/bash

set -euo pipefail

ENVIRONMENT="${1:-}"

if [ -z "$ENVIRONMENT" ]; then
    echo "Usage: $0 <environment>"
    exit 1
fi

case "$ENVIRONMENT" in
    staging|production)
        ;;
    *)
        echo "Invalid environment"
        exit 1
        ;;
esac

echo "Starting deployment to $ENVIRONMENT"

echo "Pulling latest code..."
git pull --ff-only origin main

echo "Building application..."
./build.sh

echo "Running tests..."
./test.sh

echo "Building Docker image..."
docker build -t "myapp:$ENVIRONMENT" .

echo "Deployment completed"
```

* * *

## 34\. Add Rollback Thinking

A production deployment should have a rollback strategy.

```text
New Version
     ↓
Deploy
     ↓
Health Check
     ↓
 ┌───┴────┐
Pass     Fail
 ↓         ↓
Success   Rollback
```

For Kubernetes:

```bash
kubectl rollout undo deployment/myapp
```

Then verify:

```bash
kubectl rollout status deployment/myapp
```

* * *

## 35\. Automation with Notifications

A script can integrate with notification systems.

For example, after a deployment:

```text
Deployment Started
        ↓
Build
        ↓
Test
        ↓
Deploy
        ↓
Health Check
        ↓
Success / Failure
        ↓
Notification
```

The notification mechanism might be Slack, email, PagerDuty, or another incident-management platform.

* * *

## 36\. Advanced Automation Architecture

```text
                    Git
                     ↓
                  Jenkins
                     ↓
              Shell Automation
                     ↓
       ┌─────────────┼─────────────┐
       ↓             ↓             ↓
      AWS          Docker      Kubernetes
       ↓             ↓             ↓
    Resources      Images       Deployments
       └─────────────┼─────────────┘
                     ↓
              Health Checks
                     ↓
                Monitoring
                     ↓
                 Alerts
```

* * *

## 37\. Real-Time DevOps Project

A good real-time project is:

### **Automated Application Deployment & Monitoring**

The Shell script can:

```text
1. Validate environment
2. Pull latest code
3. Build application
4. Run tests
5. Build Docker image
6. Push image
7. Deploy to Kubernetes
8. Wait for rollout
9. Perform health check
10. Collect logs on failure
11. Roll back if required
12. Send notification
```

This demonstrates real DevOps automation skills.

* * *

## 38\. Example Project Workflow

```text
Developer Push
      ↓
     Git
      ↓
   Jenkins
      ↓
Shell Automation
      ↓
 Build + Test
      ↓
Docker Image
      ↓
Container Registry
      ↓
Kubernetes
      ↓
Rollout Status
      ↓
Health Check
   ↙       ↘
Pass       Fail
 ↓           ↓
Success    Rollback
 ↓           ↓
Notify     Notify
```

* * *

## 39\. Best Practices for Advanced Automation

### Security

*   Never hardcode credentials.
    
*   Use secret-management systems.
    
*   Validate input.
    
*   Use least privilege.
    
*   Avoid unsafe `eval`.
    

### Reliability

*   Use `set -euo pipefail`.
    
*   Validate dependencies.
    
*   Implement retries where appropriate.
    
*   Check exit codes.
    
*   Add health checks.
    
*   Have a rollback strategy.
    

### Maintainability

*   Use functions.
    
*   Keep scripts modular.
    
*   Use meaningful variables.
    
*   Add comments.
    
*   Store scripts in Git.
    
*   Run ShellCheck.
    

### Production

*   Test in development first.
    
*   Use staging before production.
    
*   Add logging.
    
*   Monitor automation.
    
*   Review destructive operations.
    
*   Make production changes controlled and auditable.
    

* * *

## 40\. Interview Questions

### Q1. How have you used Shell scripting in DevOps?

**Answer:**

> "I have used Shell scripting to automate repetitive DevOps activities such as application deployment, server health checks, backups, log analysis, Docker operations, AWS CLI commands, and Kubernetes deployment verification. I also integrate Shell scripts into CI/CD pipelines to reduce manual effort."

### Q2. How would you automate a Kubernetes deployment?

> "I would use `kubectl` commands from a controlled script, validate the environment, apply or update the deployment, wait for rollout completion, perform a health check, and trigger rollback if the deployment fails."

### Q3. How do you handle failures in automation?

> "I use `set -euo pipefail`, validate inputs and dependencies, check exit codes, add retries for transient failures, log important information, and implement rollback or recovery where appropriate."

### Q4. How do you secure Shell scripts?

> "I avoid hardcoded credentials, use secret-management systems, validate inputs, quote variables, follow least privilege, avoid unsafe `eval`, restrict file permissions, and run ShellCheck before deployment."

### Q5. How would you integrate a Shell script with Jenkins?

> "I store the script in Git, execute it as a pipeline step, inject required secrets through Jenkins Credentials, capture logs and exit codes, and make the pipeline fail when the script returns a non-zero status."

* * *

## 🎯 Key Takeaways

```text
Shell Scripting
      ↓
DevOps Automation
      ↓
AWS + Docker + Kubernetes
      ↓
CI/CD Integration
      ↓
Monitoring & Health Checks
      ↓
Error Handling
      ↓
Rollback
      ↓
Production Automation
```

You should now understand:

✅ Advanced Shell scripting  
✅ Real-time DevOps automation  
✅ Git automation  
✅ Deployment automation  
✅ Docker automation  
✅ AWS CLI automation  
✅ Kubernetes automation  
✅ Jenkins integration  
✅ Health checks  
✅ Monitoring  
✅ Log analysis  
✅ API automation  
✅ Retry logic  
✅ Rollback  
✅ Incident-response automation  
✅ Production best practices

> **Shell scripting becomes especially powerful in DevOps when it connects Linux commands with cloud, containers, Kubernetes, CI/CD, monitoring, and deployment workflows.**

![](https://cdn.hashnode.com/uploads/covers/6878a472fb2990c4c07d4019/e28f8d3a-b14f-4715-9033-37b512ca51b8.png align="center")