Visual Basic AI: Creating a Genetic Algorithm

It is all in the genes. In Part 1 of the AI and Visual basic article series, “Setting Up a Neural Network Using Visual Basic and AI,” I have shown you how to create a basic Neuron Network. This article will delve into Genetic Algorithms and create a way to train our already created Neuron Network to distinguish right from wrong.

Let’s start with biology first, again

Genes

A gene is a region (or locus) of DNA which is made up of nucleotides and is the molecular unit of heredity. The transmission of genes to an organism’s offspring is the basis of the inheritance of phenotypic traits. Genotypes, along with environmental and developmental factors, determine what the phenotypes will be.

Genotypes

A genotype is the DNA sequence of the genetic makeup of a cell of an organism or individual. It determines specific phenotypes (characteristics) of that cell or organism or individual.

Genetic Recombination

Genetic recombination is the production of offspring with combinations of traits that may differ from those found in either parent.

Mutation

A mutation is the permanent alteration of the nucleotide sequence of the genome of an organism or virus. Mutations result from errors during DNA replication or other types of damage to DNA.

Chromosomes

A chromosome is a DNA molecule with part or all of the genetic material of an organism.

You might say: Enough biology! Fact is: You need to understand how cells and neurons are put together to create a decent Neural network.

Genetic Algorithm

A Genetic Algorithm is inspired by the process of natural selection that belongs to the larger class of evolutionary algorithms. Genetic algorithms are commonly used to generate solutions to optimization and search problems by relying on operators such as mutation, crossover and selection.

Crossover

A Crossover is a genetic operator used to vary the programming of a chromosome or chromosomes from one generation to the next.

Selection

Selection is the stage of a genetic algorithm in which individual genomes are chosen from a population for later breeding.

Roulette Wheel Selection

Roulette Wheel Selection (fitness proportionate selection), is a genetic operator used in genetic algorithms for selecting potentially useful solutions for recombination. In Roulette Wheel Selection, the fitness function assigns a fitness to possible solutions or chromosomes. This fitness level is used to associate a probability of selection with each individual chromosome. This selection is similar to a Roulette wheel in a casino where a proportion of the wheel is assigned to each of the possible selections based on their fitness value.

Rank Selection

Rank Selection is similar to Roulette Wheel Selection except that selection probability is proportional to relative fitness rather than absolute fitness. This means that it doesn’t make any difference if the fittest candidate is hundred times fitter than the next fittest or 0.1% fitter.

Elitism

Elitism involves copying a small proportion of the fittest candidates, unchanged, into the next generation.

Our Project

You will pick up today where you left off in Part 1, “Setting Up a Neural Network Using Visual Basic and AI.” If you haven’t completed Part 1 yet, I suggest you have a read through it and download the project so that you can follow along.

Open the Network class and add the Sigmoid Function:

   Private Function Sigmoid(dblSignal As Double) As Double

      Return 1 / (1 + Math.Exp(-dblSignal))

   End Function

The Sigmoid function above works out value to set for the current Neuron.

Still in the Network class, add the Configure Function:

   Public Function Configure(lstInput As List(Of Double), _
         lstOutput As List(Of Double)) As Boolean

      If (lstInput.Count <> Me.Layers(0).Neurons.Count) _
            OrElse (lstOutput.Count <> _
            Me.Layers(Me.Layers.Count - 1).Neurons.Count) Then

         Return False

      End If

      Fire(lstInput)

      For i As Integer = 0 To _
            Me.Layers(Me.Layers.Count - 1).Neurons.Count - 1

         Dim neur As Neuron = _
            Me.Layers(Me.Layers.Count - 1).Neurons(i)

         neur.Delta = neur.Value * (1 - neur.Value) * _
            (lstOutput(i) - neur.Value)

         For j As Integer = Me.Layers.Count - 2 To 3 Step -1

            For k As Integer = 0 To Me.Layers(j).Neurons.Count - 1

               Dim neu As Neuron = Me.Layers(j).Neurons(k)

               neu.Delta = neu.Value * (1 - neu.Value) * _
                  Me.Layers(j + 1).Neurons(i).Dendrites(k).Weight * _
                  Me.Layers(j + 1).Neurons(i).Delta

            Next

         Next

      Next

      For i As Integer = Me.Layers.Count - 1 To 2 Step -1

         For j As Integer = 0 To Me.Layers(i).Neurons.Count - 1

            Dim n As Neuron = Me.Layers(i).Neurons(j)

            n.Bias = n.Bias + (Me.LearningRate * n.Delta)

            For k As Integer = 0 To n.Dendrites.Count - 1

               n.Dendrites(k).Weight = n.Dendrites(k).Weight + _
                  (Me.LearningRate * _
                  Me.Layers(i - 1).Neurons(k).Value * n.Delta)

            Next

         Next

      Next

      Return True
   End Function

First, we check how many layers there are and how many Neurons objects. We loop through the layers and Neurons inside the Layers and set the Delta values of the Neurons. Lastly, we loop through the Neurons again and set the Dendrite values.

Add the next Function:

   Public Function Fire(lstInput As List(Of Double)) As Double()

      If lstInput.Count <> Me.Layers(0).CountNeurons Then

         Return Nothing

      End If

      For lay As Integer = 0 To Layers.Count - 1

         Dim lLayer As Layer = Layers(lay)

         For neu As Integer = 0 To lLayer.Neurons.Count - 1

            Dim nNeuro As Neuron = lLayer.Neurons(neu)

            If lay = 0 Then

               nNeuro.Value = lstInput(neu)

            Else

               nNeuro.Value = 0

               For np As Integer = 0 To _
                  Me.Layers(lay - 1).Neurons.Count - 1

                  nNeuro.Value = nNeuro.Value + _
                     Me.Layers(lay - 1).Neurons(np).Value * _
                     nNeuro.Dendrites(np).Weight

               Next

               nNeuro.Value = Sigmoid(nNeuro.Value + nNeuro.Bias)

            End If

         Next

      Next

      Dim lLastLayer As Layer = Me.Layers(Me.Layers.Count - 1)

      Dim intLayerCount As Integer = lLastLayer.Neurons.Count

      Dim dblOutput As Double() = New Double(intLayerCount - 1) {}

      For i As Integer = 0 To lLastLayer.Neurons.Count - 1

         dblOutput(i) = lLastLayer.Neurons(i).Value

      Next

      Return dblOutput

   End Function

This Function produces the output based on our input.

Design your Form to resemble Figure 1:

Our Design
Figure 1: Our Design

Add the following code to train our Neurons:

   Private Sub Button2_Click(sender As Object, _
         e As EventArgs) Handles Button2.Click

      Dim lstInputs As New List(Of Double)()

      lstInputs.Add(0)
      lstInputs.Add(1)

      Dim lstOutputs As New List(Of Double)()
      lstOutputs.Add(0)
      lstOutputs.Add(0)

      For i As Integer = 0 To 999999

         nNeuralNet.Configure(lstInputs, lstOutputs)

      Next

      CreateTree(nNeuralNet, TreeView1)

   End Sub

We supply numbers via the input received from the Textboxes, and then output the results into a Treeview.

Add the following code to execute its own output, which it has calculated itself:

   Private Sub Button1_Click(sender As Object, e As EventArgs) _
         Handles Button1.Click

      Dim lstInputs As New List(Of Double)()
      lstInputs.Add(0)
      lstInputs.Add(1)

      nNeuralNet.Fire(lstInputs)

      CreateTree(nNeuralNet, TreeView1)

   End Sub

Add the CreateTree method:

   Private nNeuralNet As New Network(New Integer() {2, 4, 2}, 21.5)

   Private Sub CreateTree(nNet As Network, t As TreeView)

      t.Nodes.Clear()

      Dim root As New TreeNode("Artificial Neural Network")

      nNet.Layers.ForEach(Function(NetworkLayer)

         Dim nLayer As New TreeNode("Network Layer")

         NetworkLayer.Neurons.ForEach(Function(Neuron)

            Dim nNeuron As New TreeNode("Neuron")

            nNeuron.Nodes.Add("Bias: " + Neuron.Bias.ToString())
            nNeuron.Nodes.Add("Delta: " + Neuron.Delta.ToString())
            nNeuron.Nodes.Add("Value: " + Neuron.Value.ToString())

               Neuron.Dendrites.ForEach(Function(Dendrite)

                  Dim nDendrite As New TreeNode("Dendrite")

                  nDendrite.Nodes.Add("Weight: " + Dendrite.Weight.ToString())

                  nNeuron.Nodes.Add(nDendrite)

               End Function)

               nLayer.Nodes.Add(nNeuron)

            End Function)

            root.Nodes.Add(nLayer)

         End Function)

      root.ExpandAll()

      t.Nodes.Add(root)

   End Sub

This outputs a Treeview based on the results, shown in Figure 2 underneath

Running
Figure 2: Running

Conclusion

Neural Networks and genetic Algorithms are only the tip of the iceberg in the world of A.I. I hope I have introduced AI to you in this series, so that you can take it further.

Hannes DuPreez
Hannes DuPreez
Ockert J. du Preez is a passionate coder and always willing to learn. He has written hundreds of developer articles over the years detailing his programming quests and adventures. He has written the following books: Visual Studio 2019 In-Depth (BpB Publications) JavaScript for Gurus (BpB Publications) He was the Technical Editor for Professional C++, 5th Edition (Wiley) He was a Microsoft Most Valuable Professional for .NET (2008–2017).

More by Author

Get the Free Newsletter!

Subscribe to Developer Insider for top news, trends & analysis

Must Read