Scenario

A CSV file contains e-commerce transaction data with both numeric and non-numeric columns. You need to generate a statistical summary report for all numeric columns.

Task

Write a Python script at /home/interview/calculate_stats.py using pandas that reads /home/interview/sales_data.csv, calculates descriptive statistics (mean, median, std, min, max, and quartiles: 25%, 50%, 75%) for all numeric columns only, and saves the results to /home/interview/statistics_report.csv. Round all values to 2 decimal places.

Example

Expected output format in statistics_report.csv:

statistic,price,quantity,discount_percent,shipping_cost,tax_amount
mean,45.23,3.42,12.45,8.90,4.12
median,42.00,3.00,10.00,7.50,3.80
std,15.67,2.31,8.45,3.20,2.10
min,10.00,1.00,0.00,2.50,0.50
25%,32.50,2.00,5.00,6.00,2.50
50%,42.00,3.00,10.00,7.50,3.80
75%,55.00,5.00,18.00,11.00,5.50
max,99.99,10.00,50.00,20.00,15.00

Step 1: Examine the input data

head /home/interview/sales_data.csv

Review the columns to identify numeric vs non-numeric data types and check for missing values.

Step 2: Create the Python script

nano /home/interview/calculate_stats.py

Write a script using pandas to calculate statistics for numeric columns only:

import pandas as pd

# Read the CSV file
df = pd.read_csv('/home/interview/sales_data.csv')

# Select only numeric columns
numeric_df = df.select_dtypes(include=['number'])

# Calculate descriptive statistics
stats_dict = {
    'statistic': ['mean', 'median', 'std', 'min', '25%', '50%', '75%', 'max']
}

for col in numeric_df.columns:
    stats_dict[col] = [
        round(numeric_df[col].mean(), 2),
        round(numeric_df[col].median(), 2),
        round(numeric_df[col].std(), 2),
        round(numeric_df[col].min(), 2),
        round(numeric_df[col].quantile(0.25), 2),
        round(numeric_df[col].quantile(0.50), 2),
        round(numeric_df[col].quantile(0.75), 2),
        round(numeric_df[col].max(), 2)
    ]

# Create DataFrame and save
stats_report = pd.DataFrame(stats_dict)
stats_report.to_csv('/home/interview/statistics_report.csv', index=False)

print(f"Generated statistics for {len(numeric_df.columns)} numeric columns")

The select_dtypes(include=['number']) method filters to numeric columns only. Pandas automatically handles NULL values by excluding them from calculations. Round to 2 decimal places for readability.

Step 3: Run the script

python3 /home/interview/calculate_stats.py

Step 4: Verify the output

cat /home/interview/statistics_report.csv

Should show statistics for all numeric columns with proper formatting.