Chapter 9 का पूरा Roadmap – 6 Topic

		क्या सीखेंगे	Project में Use
9.1	ML Workflow	Train-Test Split, Fit, Predict	हर model की base
9.2	Unsupervised: Clustering	KMeans, Hierarchical, DBSCAN	Student Clustering
9.3	Supervised: Regression	Linear, Ridge, Lasso	Marks Predict करना
9.4	Supervised: Classification	Logistic, KNN, Decision Tree	Pass/Fail Predict
9.5	Model Evaluation	Accuracy, MSE, Silhouette Score	कौनसा model best है
9.6	Feature Scaling + Pipeline	StandardScaler, Pipeline	Data को model ready करना

Install: pip install scikit-learn

9.1 सबसे Important Code –

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
import pandas as pd

#1. Data लो – Chapter 7 वाला df

df = pd.read_csv(‘students.csv’)
X = df[[‘Math’,’Science’,’English’,’History’]]

#2. Scale करो – जरूरी है

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

#3. KMeans Clustering – 3 Cluster

kmeans = KMeans(n_clusters=3, random_state=42)
df[‘Cluster’] = kmeans.fit_predict(X_scaled)

#4. Model कितना अच्छा है? Silhouette Score

score = silhouette_score(X_scaled, df[‘Cluster’])
print(“Silhouette Score:”, score) # 0.5+ अच्छा माना जाता है

#5. Cluster Center देखो

print(pd.DataFrame(kmeans.cluster_centers_, columns=X.columns))

बस। यही Students Clustering का final code है।

9.2 KMeans vs Hierarchical – कब क्या?

	KMeans	Hierarchical
Speed	Fast, बड़े data पर	Slow
Cluster Count	पहले बताना पड़ता है n_clusters=3	Dendrogram से decide
Output	Center + Label	Dendrogram + Label
Use	10000+ students	100-500 students

9.3 Bonus: Regression –

from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error

#मान लो History से बाकी predict करना है

X = df[[‘Math’,’Science’,’English’]]
y = df[‘History’]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

model = LinearRegression()
model.fit(X_train, y_train)
y_pred = model.predict(X_test)

print(“Error:”, mean_squared_error(y_test, y_pred))

9.1 EDA – 5 Step में Data

हम उसी students.csv वाला data use करेंगे

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

df = pd.read_csv(‘students.csv’) # मान लो ये file है

मान लो data ऐसा है:

Name Math Science English Class
Aman 88 85 90 A
Riya 92 95 88 A
Karan 75 70 65 B
Sneha 95 92 94 B
Rahul 60 55 58 A

STEP 1: Data को देखो –

df.head() # पहले 5 row
df.tail() # आखिरी 5 row
df.shape # (rows, columns) → (100, 5)
df.info() # कौनसे column में कितने null, datatype क्या है
df.describe() # mean, std, min, max, 25%, 50%, 75%

GATE Trick: describe() से outlier तुरंत पकड़ आते हैं। अगर Max 1000 है तो गड़बड़ है।

STEP 2: Missing Value + Duplicate Check

df.isnull().sum() # हर column में कितने null
df.duplicated().sum() # duplicate row कितने
df[‘Class’].value_counts() # Class A में कितने, B में कितने

STEP 3: 5 जरूरी Graph – 1 ही बार में पूरी कहानी

1.Distribution – Histogram → Marks कैसे बटे हैं

sns.histplot(df[‘Math’], bins=10, kde=True)
plt.title(‘Math Marks Distribution’)
plt.show()

देखोगे: ज्यादातर 70-90 के बीच हैं या fail ज्यादा हैं

2.Category Compare – Barplot → Class A vs B कौन आगे

sns.barplot(data=df, x=’Class’, y=’Math’, estimator=np.mean)
plt.title(‘Avg Math Marks by Class’)
plt.show()

3.Relation – Scatter Plot → Math अच्छे तो Science भी?

sns.scatterplot(data=df, x=’Math’, y=’Science’, hue=’Class’)
plt.title(‘Math vs Science’)
plt.show()

लाइन बन रही है = Correlation है

4.Correlation – Heatmap → सब subjects आपस में जुड़े हैं?

plt.figure(figsize=(6,4))
sns.heatmap(df[[‘Math’,’Science’,’English’]].corr(), annot=True, cmap=’coolwarm’)
plt.title(‘Correlation Heatmap’)
plt.show()

0.8+ = बहुत strong relation

5.Outlier Check – Boxplot

sns.boxplot(data=df[[‘Math’,’Science’,’English’]])
plt.title(‘Outlier Check’)
plt.show()

बाहर dot = Outlier। उसे handle करना पड़ेगा Step 9.3 में

STEP 4: Groupby से Insight निकालो

#हर Class का average

df.groupby(‘Class’)[[‘Math’,’Science’]].mean()

#Top 3 Students

df.nlargest(3, ‘Math’)

#Math में 90+ कितने हैं Class-wise

df[df[‘Math’]>90].groupby(‘Class’).size()

STEP 5: Pairplot – 1 Graph = पूरी EDA

sns.pairplot(df, hue=’Class’, diag_kind=’kde’)
plt.show()

ये 1 image बना देगा: Math vs Science, Math vs English, Distribution सब।

EDA के बाद हमें क्या पता चलना चाहिए?

1.Data कैसा है: 100 students, 3 subjects,2 class

2 .Null है क्या: नहीं है

3.Outlier है क्या: 1-2 बच्चे 40 marks वाले

4.Relation: Math और Science में 0.85 correlation

5.Insight: Class B का average Class A से 5 marks ज्यादा

अगला Step 9.2 Data Cleaning करें?
वहीं हम null और outlier को ठीक करेंगे।

9.2 Data Cleaning –

गंदगी	Problem	Solution
1.Missing Values	NaN आ गया	fillna() या dropna()
2.Duplicates	एक ही student 2 बार	drop_duplicates()
3.Outliers	1 बच्चा 1000 marks	IQR या Z-score से हटाओ
4.Wrong Datatype	Marks str में है	astype() से int करो

हम वही df use करेंगे। मान लो इसमें गंदगी डाल दी:

import pandas as pd
import numpy as np

df = pd.DataFrame({
‘Name’:[‘Aman’,’Riya’,’Karan’,’Sneha’,’Rahul’,’Aman’],
‘Math’:[88,92,np.nan,95,60,88],
‘Science’:[85,95,70,200,55,85],
‘English’:[90,88,65,np.nan,58,90],
‘Class’:[‘A’,’A’,’B’,’B’,’A’,’A’]
})

#Problem: NaN, Duplicate Aman, Science=200 outlier, wrong type

STEP 1: Missing Values Handle करो

पहले चेक करो कहाँ null है:

df.isnull().sum()

Output:

Math 1
Science 0
English 1

3 तरीके हैं भरने के:

#1. Average से भरो – सबसे common

df[‘Math’].fillna(df[‘Math’].mean(), inplace=True)

#2. 0 से भरो – अगर अनुपस्थित है

df[‘English’].fillna(0, inplace=True)

#3. पूरी row हटा दो – अगर बहुत null हैं

df.dropna(inplace=True) # सावधान: data कम हो जाएगा

STEP 2: Duplicates हटाओ

df.duplicated().sum() # पहले गिनो
df.drop_duplicates(inplace=True) # Aman वाली duplicate row गई

STEP 3: Outliers पकड़ो और हटाओ – सबसे Important

Science में 200 आ गया। ये गलत है। IQR Method से पकड़ते हैं।

Q1 = df[‘Science’].quantile(0.25)
Q3 = df[‘Science’].quantile(0.75)
IQR = Q3 – Q1

lower_bound = Q1 – 1.5 * IQR
upper_bound = Q3 + 1.5 * IQR

print(“Outlier range:”, lower_bound, “to”, upper_bound)

#Outlier वाली row हटा दो

df = df[(df[‘Science’] >= lower_bound) & (df[‘Science’] <= upper_bound)]

200 बाहर चला जाएगा।

दूसरा तरीका: Cap कर दो

df[‘Science’] = np.where(df[‘Science’] > upper_bound, upper_bound, df[‘Science’])

STEP 4: Datatype ठीक करो

df.dtypes # देखो कौन str में है
df[‘Math’] = df[‘Math’].astype(‘int’) # float को int करो

पूरी Cleaning Pipeline – 10 Line में

def clean_data(df):
# 1. Duplicate
df.drop_duplicates(inplace=True)

# 2. Missing Value
df.fillna(df.mean(numeric_only=True), inplace=True)

# 3. Outlier - IQR
for col in ['Math','Science','English']:
    Q1 = df[col].quantile(0.25)
    Q3 = df[col].quantile(0.75)
    IQR = Q3 - Q1
    df = df[(df[col] >= Q1 - 1.5*IQR) & (df[col] <= Q3 + 1.5*IQR)]

# 4. Datatype
df[['Math','Science','English']] = df[['Math','Science','English']].astype(int)

return df

df_clean = clean_data(df)
print(“Cleaning done. New shape:”, df_clean.shape)

GATE + Interview के 5 सवाल

Q1. fillna(mean) और fillna(median) में क्या फर्क?
Ans: Outlier है तो median use करो, वरना mean

Q2. कब row delete करोगे?
Ans: जब 1 row में 50%+ column null हो

Q3. IQR का full form?
Ans: Inter Quartile Range = Q3 – Q1

Q4. inplace=True क्या करता है?
Ans: Original df में ही change कर देता है, नया df नहीं बनाता

Q5. Cleaning के बाद क्या check करोगे?
Ans: df.isnull().sum() = 0 और df.describe() से range देखो

9.3 Feature Engineering – “Naye Column Banane ka Khel”

Raw data se insight nikalne ke liye naye features banate hain.

import pandas as pd
import numpy as np

#Clean wala df lete hain

df[‘Total’] = df[[‘Math’,’Science’,’English’]].sum(axis=1)
df[‘Average’] = df[[‘Math’,’Science’,’English’]].mean(axis=1).round(2)

#Grade column – condition se

def get_grade(avg):
if avg >= 90: return ‘A’
elif avg >= 75: return ‘B’
elif avg >= 60: return ‘C’
else: return ‘F’

df[‘Grade’] = df[‘Average’].apply(get_grade)

#Pass/Fail – Classification ke liye

df[‘Result’] = np.where(df[‘Average’] >= 60, ‘Pass’, ‘Fail’)

print(df.head())

Interview Q: apply() vs np.where() kab use karte?
Ans: Complex logic → apply, Simple if-else → np.where fast hota

9.4 Feature Scaling + Train-Test Split

ML model ko sab column same scale pe chahiye warna Math 100 wala Science 10 wale ko daba dega

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

#X = features, y = target. Clustering me y nahi hota

X = df[[‘Math’,’Science’,’English’,’Total’,’Average’]]

#1. Scale karo

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

#2. Split karo – Regression/Classification ke liye

#Clustering me split nahi karte, pura data ek saath

X_train, X_test = train_test_split(X_scaled, test_size=0.2, random_state=42)

print(“Scaled data ready:”, X_scaled.shape)

GATE Trick: StandardScaler = mean 0, std 1 kar deta hai

9.5 Clustering – 2 Sabse Important Algorithm

Algorithm	Kaam	Kab use kare
1.KMeans	K group banata hai	Jab pata ho 3 group chahiye
2.Hierarchical	Tree bana deta hai	Jab nahi pata kitne group chahiye

STEP 1: KMeans – Students ko 3 Group me baanto

from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt
import seaborn as sns

9.4 wala scaled data use karenge

X = df[[‘Math’,’Science’,’English’,’Average’]]
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

K=3 cluster

kmeans = KMeans(n_clusters=3, random_state=42, n_init=10)
df[‘Cluster_KMeans’] = kmeans.fit_predict(X_scaled)

print(df[[‘Name’,’Average’,’Cluster_KMeans’]].head(10))
print(“\nCluster Centers:”)
print(pd.DataFrame(scaler.inverse_transform(kmeans.cluster_centers_), columns=X.columns))

Output ka matlab:

Cluster 0 = Topper Group – Avg 90+

Cluster 1 = Average Group – Avg 70-89

Cluster 2 = Weak Group – Avg <70

STEP 2: Best K kitna hai? Elbow Method

Andha K=3 mat lo. Graph se decide karo.

wcss = []
for i in range(1, 11):
kmeans = KMeans(n_clusters=i, random_state=42, n_init=10)
kmeans.fit(X_scaled)
wcss.append(kmeans.inertia_) # distance ka sum

plt.plot(range(1, 11), wcss, marker=’o’)
plt.title(‘Elbow Method’)
plt.xlabel(‘Number of Clusters’)
plt.ylabel(‘WCSS’)
plt.show()

jahan line “elbow” banaye wahi best K hai. Yahan 3 aa raha.

STEP 3: Hierarchical Clustering + Dendrogram

from scipy.cluster.hierarchy import dendrogram, linkage

linked = linkage(X_scaled, method=’ward’)
plt.figure(figsize=(10, 5))
dendrogram(linked, labels=df[‘Name’].tolist(), orientation=’top’)
plt.title(‘Student Dendrogram’)
plt.show()

Is tree ko upar se kaato to jitne tukde utne cluster.

STEP 4: Cluster ko Visualize karo

plt.figure(figsize=(8,6))
sns.scatterplot(data=df, x=’Math’, y=’Science’, hue=’Cluster_KMeans’, palette=’viridis’, s=100)
plt.title(‘Student Clusters: Math vs Science’)
plt.show()

3 alag color = 3 alag group ban gaye

Interview + GATE ke 4 Sawal

Q1. KMeans vs Hierarchical ka difference?
KMeans fast, K pehle batana padta. Hierarchical slow, tree deta hai.

Q2. Scaling kyu zaroori hai?
Nahi karoge to jiska value bada hai wahi cluster decide karega.

Q3. Silhouette Score kya hai?
-1 se 1. 0.5+ = achhe cluster bane hain. Jitna high utna badhiya.

Q4. Real life use?
Customer segmentation, Student performance group, Anomaly detection

9.6 Model Evaluation – “Cluster kitna badhiya hai”

Clustering me accuracy nahi hoti kyunki true label nahi hota. Isliye 3 metric use karte hain

STEP 1: Silhouette Score – Sabse important-1 se +1 ke beech hota hai
+0.7 = mast cluster bane hain
0 = border pe latak rahe hain
-0.5 = galat cluster

from sklearn.metrics import silhouette_score, davies_bouldin_score

X = df[[‘Math’,’Science’,’English’,’Average’]]
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

score = silhouette_score(X_scaled, df[‘Cluster_KMeans’])
print(“Silhouette Score:”, round(score, 3))

Target: 0.5 se upar laana hai

STEP 2: Davies-Bouldin Index – Kam = Badhiya

Ye batata hai cluster ek dusre se kitne alag hain

db_score = davies_bouldin_score(X_scaled, df[‘Cluster_KMeans’])
print(“Davies-Bouldin Score:”, round(db_score, 3))

Target: 1 se kam hona chahiye

STEP 3: Cluster Profile – Business Insight

Model ke baad story batani padti hai

cluster_profile = df.groupby(‘Cluster_KMeans’)[[‘Math’,’Science’,’English’,’Average’]].mean().round(2)
print(cluster_profile)

Output Example:

Cluster_KMeans
0 92.5 89.3 91.0 90.93 <- Topper
1 75.2 72.8 74.5 74.17 <- Average
2 45.3 48.1 46.9 46.77 <- Needs Help

Ab school wale bolenge “Cluster 2 walo pe dhyan do”

STEP 4: KMeans vs Hierarchical – Kaun jeeta?

from sklearn.cluster import AgglomerativeClustering

hier = AgglomerativeClustering(n_clusters=3)
df[‘Cluster_Hier’] = hier.fit_predict(X_scaled)

print(“KMeans Silhouette:”, silhouette_score(X_scaled, df[‘Cluster_KMeans’]))
print(“Hier Silhouette:”, silhouette_score(X_scaled, df[‘Cluster_Hier’]))

jiska score zyada wahi winner

Interview ke 3 Fatake Sawal

Q1. Clustering me train-test split kyu nahi karte?
Ans: Label nahi hota, pura data ek saath pattern dhoondta hai

Q2. Elbow me 3 aur Silhouette me 4 aa raha to kya kare?
Ans: Business ko poochho. 3 group samjhana easy hai.

Q3. Real life me kaise use hoga?
Ans: Customer segmentation, Fraud detection, Student grouping

Chapter 9::Machine Learning Basics with Scikit-learn