Universal Data Science Python Template for Machine Learning

Universal Data Science Python Template

This template provides a streamlined workflow for common data science tasks, including data preprocessing, modeling, and visualization.

Step 1: Import Libraries

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression, LinearRegression
from sklearn.cluster import KMeans, DBSCAN
from sklearn.metrics import accuracy_score, r2_score, silhouette_score

Step 2: Load Dataset

df = pd.read_csv('your_dataset.csv') # Replace with your dataset name
print(df.head())

Step 3: Data Cleaning

df = df.dropna()

Step 4: Feature Selection

X = df.drop('target_column', axis=1)
y = df['target_column']

Step 5: Data Scaling

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

Step 6: Data Splitting

X_train, X_test, y_train, y_test = train_test_split(X_scaled, y, test_size=0.2, random_state=0)

Step 7: Model Selection

Example 1: Logistic Regression (Classification)

model = LogisticRegression()
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
print("Accuracy:", accuracy_score(y_test, y_pred))

Example 2: Linear Regression

# model = LinearRegression()
# model.fit(X_train, y_train)
# y_pred = model.predict(X_test)
# print("R2 Score:", r2_score(y_test, y_pred))

Example 3: K-Means Clustering

# model = KMeans(n_clusters=3, random_state=0)
# y_pred = model.fit_predict(X_scaled)
# print("Silhouette Score:", silhouette_score(X_scaled, y_pred))

Example 4: DBSCAN (Outlier Detection)

# model = DBSCAN(eps=0.5, min_samples=5)
# y_pred = model.fit_predict(X_scaled)
# print("Outliers:", list(y_pred).count(-1))

Step 8: Visualization

plt.figure(figsize=(6,4))
sns.scatterplot(x=X_scaled[:,0], y=X_scaled[:,1], hue=y_pred, palette='coolwarm')
plt.title("Model Output Visualization")
plt.show()

Exploratory Data Analysis (EDA)

Use these snippets for quick data inspection and outlier detection:

• Single Boxplot: sns.boxplot(x=df['column_name'])
• Multiple Boxplots: sns.boxplot(data=df)

Outlier Detection Logic:

Q1 = df['column_name'].quantile(0.25)
Q3 = df['column_name'].quantile(0.75)
IQR = Q3 - Q1
outliers = df[(df['column_name'] < (Q1 - 1.5 * IQR)) | (df['column_name'] > (Q3 + 1.5 * IQR))]
print("Number of outliers:", len(outliers))