Practical Machine Learning Labs in TensorFlow and PyTorch
English with a size of 14.76 KBLab 1: Basic TensorFlow Computation Graph
This example demonstrates how to define and execute a simple computation graph in TensorFlow using the @tf.function decorator, which converts a Python function into a high-performance TensorFlow graph.
import tensorflow as tf
# Define a simple computation graph
@tf.function # Converts Python function into a TensorFlow graph
def my_graph(x, y):
return x * y + 5 # Simple equation: (x * y) + 5
# Create TensorFlow constants (nodes)
x = tf.constant(3.0)
y = tf.constant(4.0)
# Run the computation graph
result = my_graph(x, y)
# Print the result
# Convert tensor to a NumPy array to see the value
print(f"Result of (x * y) + 5: {result.numpy()}")Lab 2: Simple Linear Regression with Keras
Here, we build, train, and evaluate a simple linear regression model using TensorFlow Keras. The example includes data generation, model compilation with Stochastic Gradient Descent (SGD), training, and plotting the loss curve. It also shows the modern way to save and load a model using the .keras format.
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
# Generate data (y = 3x + 7 with noise)
X = np.linspace(-1, 1, 100).reshape(-1, 1)
y = 3 * X + 7 + np.random.randn(100, 1) * 0.2
# Build a simple model
model = tf.keras.Sequential([tf.keras.layers.Dense(1)])
# Compile the model with Gradient Descent
model.compile(optimizer='sgd', loss='mse')
# Train the model
history = model.fit(X, y, epochs=100, verbose=0)
# Save & Load the model (NEW FORMAT)
model.save("model.keras") # ✅ Use .keras format
loaded_model = tf.keras.models.load_model("model.keras") # ✅ Works without errors
# Plot training loss
plt.plot(history.history['loss'])
plt.xlabel("Epochs")
plt.ylabel("Loss")
plt.title("Training Loss Curve")
plt.show()Lab 3: MLP for Binary Classification
This lab covers building a Multi-Layer Perceptron (MLP) for a binary classification task. The model uses ReLU activation functions in its hidden layers and a Sigmoid function in the output layer, compiled with the Adam optimizer and binary cross-entropy loss.
import tensorflow as tf
from tensorflow import keras
import numpy as np
# Generate dummy data
X_train = np.random.rand(100, 10)
y_train = (np.sum(X_train, axis=1) > 5).astype(int)
X_test = np.random.rand(20, 10)
y_test = (np.sum(X_test, axis=1) > 5).astype(int)
# Build a simple MLP model
def build_model():
model = keras.Sequential([
keras.layers.Dense(16, activation='relu', input_shape=(10,)),
keras.layers.Dense(8, activation='relu'),
keras.layers.Dense(1, activation='sigmoid')
])
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
return model
# Train the model
model = build_model()
model.fit(X_train, y_train, epochs=15, batch_size=5, validation_data=(X_test, y_test))Lab 4: MNIST Classification with Regularization
This example tackles the classic MNIST handwritten digit classification problem. It demonstrates how to build a neural network with regularization techniques like L2 regularization and Dropout to prevent overfitting.
import tensorflow as tf
from tensorflow import keras
# Load MNIST dataset
(X_train, y_train), (X_test, y_test) = keras.datasets.mnist.load_data()
# Normalize the data (0 to 1)
X_train, X_test = X_train / 255.0, X_test / 255.0
# Flatten images (28x28 → 784)
X_train = X_train.reshape(-1, 784)
X_test = X_test.reshape(-1, 784)
# Build a simple neural network with regularization
model = keras.Sequential([
# L2 Regularization
keras.layers.Dense(128, activation='relu', kernel_regularizer=keras.regularizers.l2(0.01)),
# Dropout Regularization
keras.layers.Dropout(0.5),
# Output layer (10 classes)
keras.layers.Dense(10, activation='softmax')
])
# Compile model
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
# Train model
model.fit(X_train, y_train, epochs=5, validation_data=(X_test, y_test))Lab 5: Convolutional Neural Network (CNN) for MNIST
This lab introduces a Convolutional Neural Network (CNN) for image classification on the MNIST dataset. The model consists of a convolutional layer (Conv2D), a pooling layer (MaxPooling2D), and dense layers for classification.
import tensorflow as tf
from tensorflow.keras import layers, models
# Load MNIST dataset
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
# Normalize and reshape data
x_train, x_test = x_train / 255.0, x_test / 255.0
x_train = x_train[..., None]
x_test = x_test[..., None]
# Define a simple CNN model
model = models.Sequential([
layers.Conv2D(16, (3, 3), activation='relu', input_shape=(28, 28, 1)),
layers.MaxPooling2D((2, 2)),
layers.Flatten(),
layers.Dense(10, activation='softmax')
])
# Compile and train model
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit(x_train, y_train, epochs=3, validation_data=(x_test, y_test))
# Evaluate model
test_loss, test_acc = model.evaluate(x_test, y_test)
print(f"Test accuracy: {test_acc:.4f}")Lab 6: RNNs (LSTM & GRU) for Text Processing
This example demonstrates text processing using Recurrent Neural Networks (RNNs). It covers tokenization, sequence padding, and building both LSTM (Long Short-Term Memory) and GRU (Gated Recurrent Unit) models for a simple prediction task.
import tensorflow as tf
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences
import numpy as np
# Small dataset
text = ["I love NLP", "LSTM is great", "GRU is simple"]
# Tokenization
tokenizer = Tokenizer()
tokenizer.fit_on_texts(text)
word_count = len(tokenizer.word_index) + 1
# Convert text to sequences
seqs = tokenizer.texts_to_sequences(text)
X = pad_sequences([s[:-1] for s in seqs], padding='pre')
y = tf.keras.utils.to_categorical([s[-1] for s in seqs], num_classes=word_count)
# Simple LSTM model
lstm = tf.keras.Sequential([
tf.keras.layers.Embedding(word_count, 10, input_length=X.shape[1]),
tf.keras.layers.LSTM(20),
tf.keras.layers.Dense(word_count, activation='softmax')
])
lstm.compile(loss='categorical_crossentropy', optimizer='adam')
lstm.fit(X, y, epochs=50, verbose=0)
# Simple GRU model
gru = tf.keras.Sequential([
tf.keras.layers.Embedding(word_count, 10, input_length=X.shape[1]),
tf.keras.layers.GRU(20),
tf.keras.layers.Dense(word_count, activation='softmax')
])
gru.compile(loss='categorical_crossentropy', optimizer='adam')
gru.fit(X, y, epochs=50, verbose=0)
# Safe prediction function (avoiding retracing)
def predict(model, text):
seq = tokenizer.texts_to_sequences([text])
if not seq[0]: # If the input text contains unseen words
return "Unknown word"
seq = pad_sequences(seq, maxlen=X.shape[1], padding='pre')
pred_index = np.argmax(model.predict_on_batch(np.array(seq)))
return next((word for word, index in tokenizer.word_index.items() if index == pred_index), "?")
# Test prediction
print("LSTM Prediction:", predict(lstm, "LSTM is"))
print("GRU Prediction:", predict(gru, "GRU is"))Lab 7: Autoencoder with PyTorch and MNIST
This lab switches to the PyTorch framework to build a simple autoencoder. The model is trained on the MNIST dataset to learn a compressed (latent) representation of the images and then reconstruct them.
import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms
# Define Autoencoder
class Autoencoder(nn.Module):
def __init__(self, input_dim, latent_dim):
super(Autoencoder, self).__init__()
self.encoder = nn.Linear(input_dim, latent_dim)
self.decoder = nn.Linear(latent_dim, input_dim)
def forward(self, x):
return self.decoder(self.encoder(x))
# Load dataset
transform = transforms.Compose([transforms.ToTensor(), transforms.Lambda(lambda x: x.view(-1))])
dataloader = torch.utils.data.DataLoader(datasets.MNIST("./data", train=True, download=True, transform=transform), batch_size=64, shuffle=True)
# Initialize model, loss, optimizer
model = Autoencoder(28 * 28, 10)
criterion = nn.MSELoss()
optimizer = optim.Adam(model.parameters(), lr=0.01)
# Train model
for epoch in range(5):
for images, _ in dataloader:
optimizer.zero_grad()
loss = criterion(model(images), images)
loss.backward()
optimizer.step()
print(f"Epoch {epoch + 1}, Loss: {loss.item():.4f}")
print("Training completed.")Lab 8: Advanced Autoencoder Architectures
This lab explores several advanced autoencoder variants using TensorFlow and Keras on the MNIST dataset. Each type is designed for a specific purpose, such as learning hierarchical features or robustness to noise.
Stacked Autoencoder
A deep autoencoder with multiple hidden layers for learning more complex data representations.
import numpy as np
import tensorflow as tf
from tensorflow.keras.models import Model, Sequential
from tensorflow.keras.layers import Input, Dense, Lambda
from tensorflow.keras import regularizers
from tensorflow.keras.datasets import mnist
# Load and preprocess MNIST
(x_train, _), _ = mnist.load_data()
x_train = x_train.astype("float32") / 255.
x_train = x_train.reshape((len(x_train), 784))
# Function to train any autoencoder
def train_autoencoder(model, input_data, target_data, name):
print(f"\nTraining {name}...")
model.compile(optimizer='adam', loss='mse')
model.fit(input_data, target_data, epochs=3, batch_size=256, verbose=1)
print(f"{name} training complete.")
# 1. Stacked Autoencoder
stacked_ae = Sequential([
Dense(128, activation='relu', input_shape=(784,)),
Dense(64, activation='relu'),
Dense(128, activation='relu'),
Dense(784, activation='sigmoid')
])
train_autoencoder(stacked_ae, x_train, x_train, "Stacked Autoencoder")Denoising Autoencoder
Trained to reconstruct a clean image from a noisy input, making the model robust.
# 2. Denoising Autoencoder
noise_factor = 0.5
x_train_noisy = x_train + noise_factor * np.random.normal(loc=0.0, scale=1.0, size=x_train.shape)
x_train_noisy = np.clip(x_train_noisy, 0., 1.)
denoising_ae = Sequential([
Dense(128, activation='relu', input_shape=(784,)),
Dense(64, activation='relu'),
Dense(128, activation='relu'),
Dense(784, activation='sigmoid')
])
train_autoencoder(denoising_ae, x_train_noisy, x_train, "Denoising Autoencoder")Sparse Autoencoder
Encourages sparsity in the latent representation by adding an activity regularizer, forcing many neurons to be inactive.
# 3. Sparse Autoencoder
sparse_ae = Sequential([
Dense(128, activation='relu', activity_regularizer=regularizers.l1(1e-5), input_shape=(784,)),
Dense(64, activation='relu'),
Dense(128, activation='relu'),
Dense(784, activation='sigmoid')
])
train_autoencoder(sparse_ae, x_train, x_train, "Sparse Autoencoder")Variational Autoencoder (VAE)
A generative model that learns a probabilistic mapping from inputs to a latent space, allowing for the generation of new data.
# 4. Variational Autoencoder (VAE)
class VAE(Model):
def __init__(self, encoder, decoder, **kwargs):
super(VAE, self).__init__(**kwargs)
self.encoder = encoder
self.decoder = decoder
def train_step(self, data):
if isinstance(data, tuple):
data = data[0]
with tf.GradientTape() as tape:
z_mean, z_log_var, z = self.encoder(data)
reconstruction = self.decoder(z)
reconstruction_loss = tf.reduce_mean(
tf.keras.losses.binary_crossentropy(data, reconstruction)
)
kl_loss = -0.5 * tf.reduce_mean(
1 + z_log_var - tf.square(z_mean) - tf.exp(z_log_var)
)
total_loss = reconstruction_loss + kl_loss
grads = tape.gradient(total_loss, self.trainable_weights)
self.optimizer.apply_gradients(zip(grads, self.trainable_weights))
return {"loss": total_loss}
# VAE Encoder
inputs = Input(shape=(784,))
x = Dense(128, activation='relu')(inputs)
z_mean = Dense(2)(x)
z_log_var = Dense(2)(x)
def sample_z(args):
z_mean, z_log_var = args
epsilon = tf.random.normal(shape=(tf.shape(z_mean)[0], 2))
return z_mean + tf.exp(0.5 * z_log_var) * epsilon
z = Lambda(sample_z)([z_mean, z_log_var])
encoder = Model(inputs, [z_mean, z_log_var, z])
# VAE Decoder
decoder_input = Input(shape=(2,))
x = Dense(128, activation='relu')(decoder_input)
outputs = Dense(784, activation='sigmoid')(x)
decoder = Model(decoder_input, outputs)
# Build and train VAE
vae = VAE(encoder, decoder)
vae.compile(optimizer='adam')
print("\nTraining Variational Autoencoder...")
vae.fit(x_train, x_train, epochs=3, batch_size=256)
print("Variational Autoencoder training complete.")Lab 9: Reinforcement Learning with Gymnasium
This final lab provides an introduction to Reinforcement Learning (RL) using the Gymnasium library (the successor to OpenAI Gym). It sets up the classic CartPole environment and defines a simple policy network with TensorFlow to take actions, demonstrating a basic RL loop with an untrained agent.
import numpy as np
# Compatibility fix for older numpy versions in some environments
if not hasattr(np, 'bool8'):
np.bool8 = np.bool_
import gymnasium as gym
import tensorflow as tf
# Create the environment
env = gym.make("CartPole-v1")
# Define a simple policy network
model = tf.keras.Sequential([
tf.keras.Input(shape=(4,)), # 4 = state size
tf.keras.layers.Dense(8, activation='relu'),
tf.keras.layers.Dense(2, activation='softmax') # 2 actions: left or right
])
# Function to choose an action based on policy
def choose_action(state):
state = np.expand_dims(state, axis=0)
probs = model(state).numpy()[0]
return np.random.choice(2, p=probs)
# Reset the environment
reset_result = env.reset()
state = reset_result[0] if isinstance(reset_result, tuple) else reset_result
total_reward = 0
done = False
# Run one episode
while not done:
action = choose_action(state)
# Handle different return formats from env.step()
step_result = env.step(action)
if len(step_result) == 5:
next_state, reward, terminated, truncated, _ = step_result
done = terminated or truncated
else:
next_state, reward, done, _ = step_result
state = next_state
total_reward += reward
print(f"Total reward collected by untrained model: {total_reward}")
env.close()