7. Optimization of Training in Deep Learning (Design Robust Bi-Tempered Logistic Loss)


 AIM: Design a Deep learning Network for Robust Bi-Tempered Logistic Loss

Theory:

The Robust Bi-Tempered Logistic Loss is a loss function designed for training deep neural networks, particularly in the context of classification tasks. It addresses some of the limitations of traditional loss functions like the standard cross-entropy loss, which can suffer from issues like vanishing gradients and sensitivity to outliers. The Robust Bi-Tempered Logistic Loss aims to provide better convergence properties and improved robustness to noisy or mislabeled data.

Program: 

import tensorflow as tf
import numpy as np


# Define the Robust Bi-Tempered Logistic Loss
def robust_bi_tempered_logistic_loss(y_true, y_pred, t1=0.8, t2=1.2, label_smoothing=0.1):
    y_true = tf.cast(y_true, dtype=tf.float32)
    y_pred = tf.math.softmax(y_pred, axis=-1)


    temp1 = (1 - y_true) * tf.math.maximum(y_pred - t1, 0)
    temp2 = (1 - y_true) * tf.math.maximum(y_pred - t2, 0) + tf.math.log(1 + tf.exp(-tf.abs(y_pred - t2)))


    loss_value = - (1 - label_smoothing) * tf.reduce_sum(temp1 + temp2) - label_smoothing * tf.reduce_sum(y_true * tf.math.log(y_pred + 1e-10))


    return loss_value


# Create a simple noisy dataset
X_train = np.random.rand(100, 10)
y_train = np.random.randint(0, 10, size=(100,))


# Define a simple model
model = tf.keras.Sequential([
    tf.keras.layers.Dense(64, activation='relu', input_shape=(10,)),
    tf.keras.layers.Dense(10)
])


# Compile the model with the custom loss function
model.compile(optimizer='adam', loss=robust_bi_tempered_logistic_loss)


# Train the model
model.fit(X_train, y_train, epochs=10)

Output:

Output:
Epoch 1/10
4/4 [==============================] - 1s 4ms/step - loss: 252.8754
Epoch 2/10
4/4 [==============================] - 0s 5ms/step - loss: 245.7204
Epoch 3/10
4/4 [==============================] - 0s 3ms/step - loss: 253.1844
Epoch 4/10
4/4 [==============================] - 0s 3ms/step - loss: 255.6411
Epoch 5/10
4/4 [==============================] - 0s 3ms/step - loss: 240.7367
Epoch 6/10
4/4 [==============================] - 0s 3ms/step - loss: 247.8149
Epoch 7/10
4/4 [==============================] - 0s 7ms/step - loss: 245.1912
Epoch 8/10
4/4 [==============================] - 0s 6ms/step - loss: 248.3541
Epoch 9/10
4/4 [==============================] - 0s 5ms/step - loss: 246.3749
Epoch 10/10
4/4 [==============================] - 0s 3ms/step - loss: 247.6301

<keras.callbacks.History at 0x79425fd40ac0>

Comments

Post a Comment

Popular posts from this blog

11. List of Capstone Projects for SOC - Deep Learning

 List of Capstone Projects for SOC - Deep Learning  1. Build your Own Neural Net from Scratch. 2. Image Classification with CIFAR-10 Dataset.(Multi Class Classification) CNN 3. Human Face Detection.( detecting the bounding boxes of the human faces from a given Image)  YOLO  4. Traffic Sign Classification.(Collect Various traffic Signal Images and train Neural Network Model) CNN 5. Text Summarizer. RNN 6. Chatbot Using Deep Learning ( Training/Generative) 7. Colouring old B&W photos (Deep Neural Networks OpenCV) 8. Animal detection by training from multiple classes (CNN) 9. Face detection
Read more

8. Advanced CNN (Build AlexNet using Advanced CNN)

  Building and training an AlexNet-like model using the CIFAR-10 dataset in TensorFlow/Keras: Program to build the model: import tensorflow as tf from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout from tensorflow.keras.datasets import cifar10 from tensorflow.keras.utils import to_categorical from sklearn.model_selection import train_test_split # Load and preprocess CIFAR-10 dataset (x_train, y_train), (x_test, y_test) = cifar10.load_data() x_train = x_train.astype( 'float32' ) / 255.0 x_test = x_test.astype( 'float32' ) / 255.0 y_train = to_categorical(y_train, num_classes= 10 ) y_test = to_categorical(y_test, num_classes= 10 ) # Split into training and validation sets x_train, x_val, y_train, y_val = train_test_split(x_train, y_train, test_size= 0.1 , random_state= 42 ) # Define the model architecture '''model = tf.keras.Sequential([     Conv2D(96, (11, 11), strides=(4, 4), activation='relu', input_shape=(32...
Read more