Neural Networks Fundamentals – Architecture & Training

Neural Networks Basics

A neural network consists of layers of interconnected neurons that apply linear transformations followed by non‑linear activations.

Core components

• Neuron: output = activation(weight·input + bias)
• Layer: collection of neurons.
• Activation functions: relu, sigmoid, tanh, softmax.
• Loss functions: mse (regression), binary_crossentropy (binary classification), categorical_crossentropy (multiclass).

Building a multi‑class classifier

import tensorflow as tf
from tensorflow.keras import layers, models

num_classes = 3
model = models.Sequential([
    layers.Dense(128, activation='relu', input_shape=(20,)),
    layers.Dropout(0.3),
    layers.Dense(64, activation='relu'),
    layers.Dense(num_classes, activation='softmax')
])

model.compile(optimizer='adam',
              loss='categorical_crossentropy',
              metrics=['accuracy'])
model.summary()

Training with callbacks

early_stop = tf.keras.callbacks.EarlyStopping(patience=5, restore_best_weights=True)
reduce_lr = tf.keras.callbacks.ReduceLROnPlateau(factor=0.5, patience=3)

history = model.fit(X_train, y_train,
                    epochs=100,
                    batch_size=32,
                    validation_data=(X_val, y_val),
                    callbacks=[early_stop, reduce_lr])

Evaluating the model

loss, acc = model.evaluate(X_test, y_test)
print('Test accuracy:', acc)

Overfitting mitigation

• Dropout layers.
• L2 regularization (kernel_regularizer=tf.keras.regularizers.l2(1e-4)).
• Data augmentation (especially for images).

Exporting to TensorFlow Lite (for mobile)

converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
with open('model.tflite', 'wb') as f:
    f.write(tflite_model)