Scenario

A text file contains strings with unwanted leading and trailing characters that need to be cleaned. The characters vary across different strings in the dataset.

Task

Write a Python script at /home/interview/clean_data.py that reads /home/interview/raw_data.txt, removes all leading and trailing special characters (*, #, -, _, !, @) and whitespace from each line, and writes the cleaned strings to /home/interview/cleaned_data.txt.

Example

Before (raw_data.txt):

***Hello World***
###Python Programming
  Data Science  
___Machine Learning___

After (cleaned_data.txt):

Hello World
Python Programming
Data Science
Machine Learning

Step 1: Examine the raw data

head -20 /home/interview/raw_data.txt

Review the strings to identify the types of leading and trailing characters that need to be removed.

Step 2: Create the Python script

nano /home/interview/clean_data.py

Write a script that reads each line, strips unwanted characters, and writes to output:

# Define characters to strip (special chars + whitespace)
chars_to_strip = '*#-_!@ \t'

# Read, clean, and write
with open('/home/interview/raw_data.txt', 'r') as infile:
    with open('/home/interview/cleaned_data.txt', 'w') as outfile:
        for line in infile:
            # Remove newline, strip specified characters, write back
            cleaned = line.rstrip('\n').strip(chars_to_strip)
            outfile.write(cleaned + '\n')

print("Data cleaning completed")

The strip() method removes any combination of the specified characters from both ends of the string.

Step 3: Run the script

python3 /home/interview/clean_data.py

Step 4: Verify the output

head -20 /home/interview/cleaned_data.txt

Strings should have all leading and trailing special characters and whitespace removed.ss