Neural Networks and KerasTuner

Source: https://towardsdatascience.com/a-soft-introduction-to-neural-networks-6986b5e3a127

What are Neural Networks

Artificial neural networks are a subset of machine learning and are at the heart of deep learning algorithms. Artificial neural networks (ANNs) are comprised of a node layers as shown in the diagram below, containing an input layer, one or more hidden layers, and an output layer. Each node, or artificial neuron, connects to another and has an associated weight and threshold. If the output of any individual node is above the specified threshold value, that node is activated, sending data to the next layer of the network. Otherwise, no data is passed along to the next layer of the network.

Source: https://towardsdatascience.com/applied-deep-learning-part-1-artificial-neural-networks-d7834f67a4f6

Neural networks models rely on training data to learn and improve their accuracy over time. However, once these learning algorithms are fine-tuned for accuracy, they are powerful tools in computer science and artificial intelligence, allowing us to classify and cluster data at a high velocity.

KerasTuner

KerasTuner is an easy-to-use, scalable hyperparameter optimization framework that solves the pain points of hyperparameter search. Easily configure your search space with a define-by-run syntax, then leverage one of the available search algorithms to find the best hyperparameter values for your models. KerasTuner comes with Bayesian Optimization, Hyperband, and Random Search algorithms built-in, and is also designed to be easy for researchers to extend in order to experiment with new search algorithms.

Finding an Optimal Neural Network Model using Keras-Tuner

Preprocessing Data: First all the categorial data from the data set was transformed to numerical data using the pandas get dummies function. Then the data was split into training and testing set. Then using the standard scaler data training and testing datasets were scaled.

Creating a new Sequential Model: Next a function was defined to create a new sequential model. This function was intended to return a compiled model with the optimizer function Adam and used the hyperparameters we defined inorder to hypertune the model. Next we instantiated the the tuner to perform hyper tuning using the Hyperband tuner with the objective to optimize the model. After that we ran the tuner to search for the best hyper parameters for our model.

Building and Training the Optimal Model: Using the best hyperparameters obtained we created a neural network model. Trained it on our training dataset and then evaluated the model against the test data set.

Results

The best model that KerasTuner picked for the training and test data set was a sequentail model with four hidden layers with three nodes in the first and third hidden layers and nine nodes in the second and fourth hidden layers with the activation function as 'sigmoid'. When we fit and trained the data on this optimal model using 50 epochs and plotted the results. The accuracy achieved when the model was evaluated against the test data was 0.838 and the model loss recorded was 0.649.

Figure 1: Accuracy recorded on the optimized model

Figure 2: Loss recorded on the optimized model

The graphs above were plotted to have an idea of the accuracy and loss achieved over each epoch to determine if the optimal model as suggested by KerasTuner was under or over fit. As per the graphs suggest we can see the model was trained well, and the test data results were inline with how the model was trained.