You've built scripts that can make decisions and run in loops. Now, we enter the big leagues. This is where your Bash skills transform from a personal convenience into a professional superpower.

In the world of DevOps, you are constantly working with log files, configuration files, and application outputs. Being able to programmatically find, process, and act on this information is what separates a good engineer from a great one. We are going to build real tools, starting with the foundational building blocks: functions and powerful text processing commands.

Writing Reusable Code with Functions

As your scripts grow, you’ll notice you repeat the same blocks of code. Repeating code is inefficient and a nightmare to maintain. If you need to fix a bug, you have to find and fix it everywhere. A function solves this problem. A function is a named, reusable block of code that you can write once and call whenever you need it. Think of it as creating your own custom command.

The syntax is straightforward. You can define a function in two common ways:

# Style 1
log_message() {
  echo "LOG: $1"
}

# Style 2
function print_hello {
  echo "Hello there!"
}

To run the code inside a function, you simply call it by its name. To pass information to your function, you use arguments, which are available inside the function as $1 (the first argument), $2 (the second), and so on, just like with scripts.

Let’s create a function that prints messages with a timestamp. This is incredibly useful for logging.

#!/bin/bash

log() {
  # The argument passed to the function is <span class="math-inline">1
echo "\[</span>(date +'%Y-%m-%d %H:%M:%S')] - $1"
}

# Now we can reuse our function
log "Starting the deployment process."
sleep 2
log "Deployment finished successfully."

This simple function keeps your code clean and your log output consistent. Functions are the key to writing organized, scalable, and maintainable scripts.

Finding What You Need: A Crash Course in grep

Imagine a thousand page book where you need to find every mention of the word "dragon." You wouldn't read it page by page. You'd want a magical search tool. In Bash, that tool is grep. The grep command is a master detective for text. It scans files or input and prints lines that match a specific pattern.

Its most basic use is grep "pattern" filename. For DevOps, this is your primary tool for digging through massive log files.

# Find all lines containing "ERROR" in the application log
grep "ERROR" /var/log/app.log

But grep has a utility belt of options that make it even more powerful.

  • Case Insensitive Search (-i): To find "error", "Error", and "ERROR".

    grep -i "error" /var/log/app.log

  • Invert Match (-v): To show all lines that do not match the pattern. This is great for filtering out noise.

    grep -v "DEBUG" /var/log/app.log

  • Count Lines (-c): To count how many times a pattern appears, instead of printing the lines themselves.

    grep -c "SUCCESS" /var/log/app.log

  • Recursive Search (-r): To search for a pattern in all files within a directory and its subdirectories. Perfect for finding a specific configuration across a whole project.

    grep -r "DATABASE_URL" /etc/my_app/

Mastering grep means you can find any needle in any haystack, a truly essential skill.

Editing on the Fly: An Introduction to sed

If grep is a detective, sed (Stream Editor) is a magician. It can modify text as it flows by, like a magical assembly line that changes items without you ever touching them. Its most common trick is finding and replacing text. This is perfect for changing values in configuration files programmatically.

The classic sed command for substitution looks like this: sed 's/find/replace/g' filename.

Let’s decode that spell:

  • s stands for substitute. It tells sed we want to perform a replacement.
  • find is the text we are looking for.
  • replace is the text we want to replace it with.
  • g stands for global. It means replace every occurrence on each line, not just the first one.

Imagine you have a config file settings.conf with the line ENVIRONMENT=staging. You need to promote it to production.

sed 's/ENVIRONMENT=staging/ENVIRONMENT=production/g' settings.conf

This will print the modified content to the terminal. To save the changes directly back to the file (be careful!), you can use the -i option.

sed -i 's/ENVIRONMENT=staging/ENVIRONMENT=production/g' settings.conf

With sed, you can automate configuration changes across dozens or hundreds of servers in seconds.

Practical Scripting: A Log Parser and a Service Health Checker

Now let's combine our knowledge to build two real world DevOps tools.

1. A Log File Parser

DevOps engineers live in log files. This script will read a log file, count the critical errors and warnings, and generate a quick report.

#!/bin/bash

# Check if a log file was provided as an argument
if [[ -z "$1" ]]; then
  echo "Usage: $0 <path_to_log_file>"
  exit 1
fi

LOG_FILE="$1"
echo "Analyzing log file: <span class="math-inline">LOG\_FILE"
\# Use grep's count option to find errors and warnings
error\_count\=</span>(grep -c -i "ERROR" "<span class="math-inline">LOG\_FILE"\)
warning\_count\=</span>(grep -c -i "WARNING" "$LOG_FILE")

# Print a summary report
echo "======================"
echo "Log Analysis Report"
echo "======================"
echo "Total warnings: $warning_count"
echo "Total errors:   $error_count"
echo "======================"

if [[ $error_count -gt 0 ]]; then
  echo "Action required: Errors found in log!"
  exit 1
else
  echo "Analysis complete. No critical errors found."
  exit 0
fi

2. A Service Health Checker

Is your website up? This script will check a list of web services to see if they are responding correctly. A service is "healthy" if it returns an HTTP 200 OK status code.

#!/bin/bash

# A list of services to check
SERVICES_TO_CHECK=("google.com" "github.com" "a_fake_website_that_fails.com")

# A function to check a single URL
check_service() {
  local service_url="<span class="math-inline">1"
\# Use curl to get only the HTTP status code
http\_code\=</span>(curl --output /dev/null --silent --write-out "%{http_code}" "$service_url")

  if [[ $http_code -eq 200 ]]; then
    echo "$service_url is UP (Status: $http_code)"
  else
    echo "$service_url is DOWN (Status: <span class="math-inline">http\_code\)"
fi
\}
echo "\-\-\- Starting Health Check \-\-\-"
for service in "</span>{SERVICES_TO_CHECK[@]}"; do
  check_service "$service"
done
echo "--- Health Check Complete ---"

These scripts are not just exercises; they are the foundation for robust automation pipelines. You can schedule them to run periodically, send alerts based on their output, and build entire monitoring systems. You are no longer just writing commands; you are engineering solutions ! 🎉