Application Security Testing: An Integral Part of DevOps
I have epilepsy, so you can imagine how fascinated I am about the inner workings of our brains. I have also, for a long time, been wanting to do an article about Artificial Intelligence! The first step in creating a decent A. I. (artificially intelligent) app is to set up a neural network. This enables us to teach it teach itself. Let's explore this topic more.
A neuron, also called a nerve cell, is the basic unit of the nervous system. Neurons are the unit which the brain uses to process information. Each neuron is made of a soma (cell body), axon, and dendrites. Dendrites and axons are nerve fibres.
Neurons do not touch; instead, they form tiny little gaps called synapses. These gaps can be electrical synapses or chemical synapses, but ultimately these gaps pass the signal from one neuron to the next. You get excitatory synapses and inhibitory synapses. Signals arriving at an excitatory synapse cause the receiving neuron to fire. Signals arriving at an inhibitory synapse inhibit the receiving neuron from firing. Here is more information on the subject: Spatial and Temporal Summation.
An axon typically conducts electrical impulses away from the neuron's cell body (soma). Axons, or nerve fibres, transmit information to different neurons, muscles, and glands. In pseudounipolar neurons (sensory neurons), such as those for warmth and touch, the electrical impulse travels along an axon from the periphery to the cell body, and from the cell body to the spinal cord along another branch of the same axon. Nerve fibres are classed into three types: A delta fibres, B fibres, and C fibres.
Dendrites are the branched projections of a neuron that act to propagate the electrochemical stimulation received from other neural cells to the cell body, or soma, of the neuron from which the dendrites project. Electrical stimulation is transmitted onto dendrites by upstream neurons (usually their axons) via synapses which are located at various points throughout the dendritic tree.
Artificial Neural Network
An artificial neural network (ANN) is an interconnected group of nodes, similar to the vast network of neurons in a human brain. Neural networks consist of multiple layers and the signal path traverses from the first (input), to the last (output) layer of neural units.
The purpose of a neural network is to solve problems similar to the way a human brain would. Neural networks are based on real numbers, with the value of the core and of the axon typically being a representation between 0.0 and 1.
A Perceptron is an algorithm for supervised learning of binary classifiers which are functions that can decide whether or not input, represented by a vector of numbers, belongs to some specific class.
Layers are made up of a number of interconnected nodes which contain a sigmoid (activation function). Information is presented to the neural network via an input layer that communicates to one or more hidden layers. The actual processing in hidden layers is done with the use of weighted connections. The hidden layers then link to an output layer.
Single-layer Neural Networks
A Single-layer neural network is a network in which the output unit is independent of the other layers—each weight effects only one output.
Multi-layer Neural Networks
A multi-layer network is a feedforward artificial neural network model that maps sets of input data onto a set of appropriate outputs.
In a feedforward neural network the information moves in only one direction, forward—obviously, from the input nodes, through the hidden nodes (if any), and to the output nodes.
Back propagation is a training algorithm consisting of feeding forward values and calculating errors and propagating it back to the earlier layers.
Reinforcement Learning vs. Supervised Learning
Reinforcement learning differs from supervised learning in that correct input and output pairs are never presented, and sub-optimal actions are not explicitly corrected. There is a focus on on-line performance which involves finding a balance between exploration of uncharted territory and exploitation of current knowledge.
Creating a Simple Artificial Neural Network
Let's create a neural network, based on what we have discussed above.
Open Visual Studio and create a new Visual Basic Windows Forms project. There is no need to add controls to your form yet; we are only creating the neural network. In Part 2 of this article series, you will see how to implement the network you'll be creating today.
Create a new class, name it Neuron, and add the following code:
Private lstDendrites As List(Of Dendrite) Private intDendrites As Integer Private dblBias As Double Private dblDelta As Double Private dblValue As Double
These members will represent their associated Properties that will be exposed to other classes. Add the Properties now:
Public Property DendriteCount() As Integer Get Return intDendrites End Get Set(value As Integer) intDendrites = value End Set End Property Public Property Dendrites() As List(Of Dendrite) Get Return lstDendrites End Get Set(value As List(Of Dendrite)) lstDendrites = value End Set End Property Public Property Value() As Double Get Return dblValue End Get Set(value As Double) dblValue = value End Set End Property Public Property Bias() As Double Get Return dblBias End Get Set(value As Double) dblBias = value End Set End Property Public Property Delta() As Double Get Return dblDelta End Get Set(value As Double) dblDelta = value End Set End Property
The DendriteCount property establishes the number of Dendrites present for this Neuron. Other properties include the Bias, Value, and Delta, which aid in identifying the correct output for our supplied input. In Part 2, you will train these values to produce the correct output. Now, add the Constructor for the Neuron class:
Public Sub New() Dim n As New Random(Environment.TickCount) Me.Bias = n.NextDouble() Me.Dendrites = New List(Of Dendrite)() End Sub
The Bias gets set to a random value. Add the Dendrite class now:
Public Class Dendrite Private dblWeight As Double Public Property Weight() As Double Get Return dblWeight End Get Set(value As Double) dblWeight = value End Set End Property Public Sub New() Dim rndRandom As New Random Dim dblNumber As Double = (rndRandom.NextDouble() * _ (1.0 - 0.00000001)) + 0.00000001 Me.Weight = dblNumber End Sub End Class
The Dendrite class contains only a Weight property that gets set to a randomly generated number. Based on this value, we will be able to train our Neurons to distinguish between what is acceptable and what is not.
Add the Layers class:
Public Class Layer Private lstNeurons As List(Of Neuron) Private intNeurons As Integer Public Property CountNeurons() As Integer Get Return intNeurons End Get Set(value As Integer) intNeurons = value End Set End Property Public Property Neurons() As List(Of Neuron) Get Return lstNeurons End Get Set(value As List(Of Neuron)) lstNeurons = value End Set End Property Public Sub New(numNeurons As Integer) Neurons = New List(Of Neuron)(numNeurons) End Sub End Class
The Layers class contains a counter to keep track of the number of Neurons as well as a Neurons list that will contain the list of all the Neurons.
Add the Network class:
Public Class Network Private lstLayers As List(Of Layer) Private intLayers As Integer Private dblLearningRate As Double Public Property CountLayers() As Integer Get Return intLayers End Get Set(value As Integer) intLayers = value End Set End Property Public Property Layers() As List(Of Layer) Get Return lstLayers End Get Set(value As List(Of Layer)) lstLayers = value End Set End Property Public Property LearningRate() As Double Get Return dblLearningRate End Get Set(value As Double) dblLearningRate = value End Set End Property Public Sub New(intLayers As Integer(), _ dblRate As Double) If intLayers.Length >= 2 Then Me.LearningRate = dblRate Me.Layers = New List(Of Layer)() For l As Integer = 0 To intLayers.Length - 1 Dim iLayer As New Layer(intLayers(l)) Me.Layers.Add(iLayer) For n As Integer = 0 To _ intLayers(l) - 1 iLayer.Neurons.Add(New Neuron()) Next iLayer.Neurons.ForEach(Function(neu) If l = 0 Then neu.Bias = 0 Else For d As Integer = 0 To intLayers(l - 1) - 1 neu.Dendrites.Add(New Dendrite()) Next End If End Function) Next Else Return End If End Sub End Class
This class contains a list of Layers as well as a counter that counts the number of Layers present in the network. The LearningRate property determines how long the Network should take to learn the correct output from the correct input.
The Constructor adds the Dendrites to the Neurons and the Neurons to the Layers, with all their respective properties. In Part 2, you will explore and extend the Network class further.
A Neural Network's main purpose is to replicate a human's brain as closely as possible. In Part 2, you will learn more about Genetic Algorithms. See you then.