Introduction

After data collection and data preprocessing, we have to build the model, train it, validate it and test it. In this article, we will delve into Model creation, its architecture, training, validation and testing.

 The input

all_features = layers.concatenate(
   [
        sex_encoded,
        cp_encoded,
        fbs_encoded,
        restecg_encoded,
        exang_encoded,
        slope_encoded,
        ca_encoded,
        thal_encoded,
        age_encoded,
        trestbps_encoded,
        chol_encoded,
        thalach_encoded,
        oldpeak_encoded,
    ]
)

 Keras is used to create a tensor named all_features. It concatenates multiple encoded categorical and numerical features, including sex, chest pain type, fasting blood sugar, resting electrocardiographic results, exercise-induced angina, slope, number of major vessels coloured by fluoroscopy, thalassemia, age, resting blood pressure, serum cholesterol, maximum heart rate achieved, and ST depression induced by exercise relative to rest. This combined tensor can be used as input for a neural network in machine learning tasks.

Model creation

x = layers.Dense(32, activation="relu")(all_features)
x = layers.Dropout(0.5)(x)
output = layers.Dense(1, activation="sigmoid")(x)
model = keras.Model(all_inputs, output)
model.compile("adam", "binary_crossentropy", metrics=["accuracy"])

 We created a dense (fully connected) layer with 32 neurons and applies the ReLU activation function to each neuron's output and then added  a dropout layer with a dropout rate of 0.5 (50% of neurons' outputs are randomly set to zero during training) to the previously defined x layer.

• Dropout is used as a regularization technique to prevent overfitting and improve generalisation of the model.

Finally, the output layer of the neural network is created. It is a dense layer with a single neuron, which is often used for binary classification problems like ours.

• The `x` variable represents the input to this output layer, which is the result after passing through the dropout layer.

 Then, creation and compilation of model is done using the functional API. The optimizer, loss function and the metric used here are adam, binary crossentropy and accuracy respectively.

 The architecture of our model will be like:

keras.utils.plot_model(model, show_shapes=True, rankdir="LR")

Training

model.fit(train_ds, epochs=50, validation_data=val_ds)
 
Epoch 1/50
8/8 [==============================] - 1s 42ms/step - loss: 0.9941 - accuracy: 0.3264 - val_loss: 0.9536 - val_accuracy: 0.2295
Epoch 2/50
8/8 [==============================] - 0s 6ms/step - loss: 0.9061 - accuracy: 0.3471 - val_loss: 0.8579 - val_accuracy: 0.2951
Epoch 49/50
8/8 [==============================] - 0s 4ms/step - loss: 0.2995 - accuracy: 0.8967 - val_loss: 0.4114 - val_accuracy: 0.8361
Epoch 50/50
8/8 [==============================] - 0s 4ms/step - loss: 0.2611 - accuracy: 0.8926 - val_loss: 0.4130 - val_accuracy: 0.8361

Now, we have performed training on our model with 50 epochs.  During training, it also evaluates the model performance on a dataset with validation.  The model is updated iteratively based on the optimization algorithm and loss function previously defined.

Testing

The final part will be testing, our model is tested to make predictions on a single input sample and the model is able to predict the result and prints the probability of having a heart disease as a percentage.

sample = {
    "age": 60,
    "sex": 1,
    "cp": 1,
    "trestbps": 145,
    "chol": 233,
    "fbs": 1,
    "restecg": 2,
    "thalach": 150,
    "exang": 0,
    "oldpeak": 2.3,
    "slope": 3,
    "ca": 0,
    "thal": "fixed",
}
 
input_dict = {name: tf.convert_to_tensor([value]) for name, value in sample.items()}
predictions = model.predict(input_dict)
 
print(
    "This particular patient had a %.1f percent probability "
    "of having a heart disease, as evaluated by our model." % (100 * predictions[0][0],)
)
 
1/1 [==============================] - 0s 301ms/step
This particular patient had a 26.4 percent probability of having a heart disease, as evaluated by our model.

Conclusion

We went through the creation, compilation, and training of a Keras neural network model for binary classification, specifically for predicting heart disease. The model is built using the functional API with a hidden layer of 32 neurons, dropout regularization, and a sigmoid output. It uses the Adam optimizer and binary crossentropy loss, evaluating accuracy during training. Additionally, there's a code snippet to showcase how to use the trained model to predict on a single input sample and obtain the probability of the patient having heart disease.

In the next article I walk talk about tracking ML model training using MLFlow.

References

[1] https://www.ibm.com/topics/neural-networks

[2]https://www.analyticsvidhya.com/blog/2021/11/neural-network-for-classification-with-tensorflow/

[3]https://www.activestate.com/resources/quick-reads/how-to-create-a-neural-network-in-python-with-and-without-keras/

[4]https://developers.google.com/machine-learning/crash-course/training-neural-networks/video-lecture#:~:text=Backpropagation%20is%20the%20most%20common,deep%20understanding%20of%20the%20algorithm.

[5] https://www.neuraldesigner.com/learning/tutorials/testing-analysis

By Maddula Syam Pavan