Every command line program in a Unix like environment (think Linux or macOS) automatically gets three standard communication channels when it runs:

  1. Standard Input (stdin or file descriptor 0): This is the default place a program looks for input. Usually, it's your keyboard. When a command is waiting for you to type something, it's reading from stdin.
  2. Standard Output (stdout or file descriptor 1): This is where a program sends its normal output. By default, this is your terminal screen. When a command like ls lists files, that list appears on your stdout.
  3. Standard Error (stderr or file descriptor 2): This is where a program sends its error messages or diagnostic information. By default, this also goes to your terminal screen. This is useful because it keeps error messages separate from normal output, so you can handle them differently if needed.

Think of it like a little workshop. Stdin is the delivery chute for raw materials (your typed input). Stdout is the main conveyor belt for finished products (normal output). Stderr is a separate chute for any faulty parts or urgent workshop alerts (error messages).

Redirecting I/O: Telling Your Data Where to Go ➡️⬅️

Now for the cool part! You can redirect these streams. Instead of everything just appearing on your screen or coming only from your keyboard, you can send output to files, or read input from files.

  • > (Redirect stdout to a file, overwrite): Sends the standard output of a command to a file. If the file exists, it gets overwritten. If it doesn't exist, it's created.

    • Example: ls > file_list.txt
      This will run the ls command, but instead of showing the file list on the screen, it will save it into file_list.txt. If file_list.txt already had content, that content is replaced.

  • >> (Redirect stdout to a file, append): Similar to >, but if the file exists, the new output is added to the end of the file instead of overwriting it.

    • Example: date >> activity_log.txt
      This will append the current date and time to activity_log.txt each time you run it.

  • < (Redirect stdin from a file): Makes a command take its input from a file instead of your keyboard.

    • Example: sort < names.txt
      If names.txt contains a list of names, this command will sort them and display the sorted list on your screen (stdout). The sort command reads from names.txt as if you typed the names directly.

  • 2> (Redirect stderr to a file): Specifically redirects error messages to a file. This is super handy for separating errors from normal output.

    • Example: find / -name "secretfile" > results.txt 2> errors.log
      This tries to find a file named "secretfile" starting from the root directory. Any successful findings go into results.txt, and any error messages (like "Permission denied" for certain directories) go into errors.log.

  • &> or >& (Redirect both stdout and stderr): Sends both standard output and standard error to the same place.

    • Example: my_script.sh &> all_output.log
      Runs my_script.sh and puts everything it prints, normal output and errors, into all_output.log.

  • << (Here Document): This is a way to feed multiple lines of input to a command directly within your script or command line, without using a separate file. You specify a delimiter word, and everything you type until that delimiter word appears again on its own line is treated as stdin.

    • Example:
    cat << EOF > greeting.txt
Hello there,
This is a multiline greeting.
EOF

    This creates a file named greeting.txt containing the two lines "Hello there," and "This is a multiline greeting.". The EOF (End Of File, but you can choose almost any delimiter) marks the end of the input.

Piping Commands: Connecting the Flow | 🔗

Piping, using the | symbol (often called a "pipe"), is one of the most powerful concepts. It lets you take the standard output (stdout) of one command and send it directly as standard input (stdin) to another command. It's like connecting a series of machines on an assembly line!

  • Example: ls -l | grep ".txt"

    1. ls -l lists all files in the current directory in long format. Its output (a list of files and details) would normally go to your screen.
    2. The | takes that output and "pipes" it as input to the grep command.
    3. grep ".txt" then searches that input for lines containing ".txt" and prints only those matching lines to your screen.
      So, this command lists only the files ending with .txt from the detailed ls -l output.

  • Another Example: cat story.txt | wc -w

    1. cat story.txt outputs the content of story.txt.
    2. This output is piped to wc -w, which counts the number of words in its input.
      The result is the word count of story.txt.

Using tee: Sometimes you want to send output to a file and see it on the screen at the same time. The tee command does exactly this! It reads from stdin and writes to stdout and to one or more files. Think of a T junction in a pipe.

  • Example: ls -lh | tee file_details.txt
    This will list files with human readable sizes (ls -lh), display that list on your screen, AND save the same list into file_details.txt.
    To append instead of overwrite, use tee -a: ls -lh | tee -a file_details.txt

Finding Files: Your Digital Bloodhound 🔎

Lost a file? Don't panic! The command line has excellent tools for sniffing them out.

  • find: This is the most powerful and flexible file finding tool. It searches for files in a directory hierarchy based on a wide range of criteria (name, type, size, modification time, permissions, etc.). It can be a bit complex at first, but it's incredibly useful.

    • Syntax idea: find <where_to_look> <criteria> <what_to_do_with_found_files>
    • Example (find by name): find /home/youruser/Documents -name "report*.pdf"
      This searches within your Documents folder (and its subfolders) for any PDF file whose name starts with "report".
    • Example (find by type, files only): find . -type f -name "*.log"
      Searches the current directory (.) for regular files (-type f) ending with .log.
    • Example (find and execute a command): find /tmp -name "*.tmp" -delete
      Finds all files ending in .tmp in the /tmp directory and deletes them (use with care!).

  • locate: This command is usually much faster than find because it searches a prebuilt database of files on your system. However, this database is typically updated only periodically (e.g., once a day), so locate might not find very recently created files or reflect recent deletions until the database is updated (you can often force an update with sudo updatedb).

    • Example: locate my_missing_file.txt
      Quickly searches its database for my_missing_file.txt.

  • which: This command is a bit different. It doesn't find general files; it specifically tells you the full path of an executable command. So, if you type a command like python, which python will show you where the python program is located on your system (e.g., /usr/bin/python). This is useful for figuring out which version of a program is being run if you have multiple installed.

    • Example: which ls
      Will probably show /bin/ls or /usr/bin/ls.

You're now equipped with some serious command line jujitsu for managing I/O, chaining commands, and finding anything on your system. Practice these concepts, play around with them, and you'll soon wonder how you ever lived without them ! 🎉