Deep Neural Networks and .NET

Artificial Intelligence Tutorials

Artificial Intelligence is quickly becoming part of everyday life. Whether you are using your PC or your mobile phone, you are making use of some variation of AI. The Web has given Artificial Intelligence wings and it just keeps on improving.

I have written about AI and .NET before. To understand the underlying concepts of Deep Neural Networks, I recommend that you read the following two articles first. These articles explain how a neural network is put together, as well as setting up a genetic algorithm.

What is Deep Learning?

Deep Learning (or Hierarchical Learning or Deep Structured Learning) is a type of machine learning method that is based on learning data representations instead of task-specific algorithms. Deep Learning can be unsupervised, semi-supervised, or supervised.

Some Deep Learning architectures, such as deep neural networks, deep belief networks, and recurrent neural networks have been applied to the following fields:

  • Computer Vision
  • Speech Recognition
  • Natural Language Processing
  • Audio Recognition
  • Social Network Filtering
  • Machine Translation
  • Bioinformatics

What Are Deep Neural Networks?

A Deep Neural Network (DNN) is an artificial neural network that has multiple hidden layers between the input and output layers. Deep Neural Networks models complex non-linear relationships. Deep Neural Networks are usually feedforward networks in which data flows from the input layer to the output layer without looping back.

Read: Machine Learning versus Deep Learning

How to Implement Deep Learning Logic in .NET

The small program that you will build today is a small example of how to implement Deep Learning logic into your application. You can use either C# or VB.NET. Let’s jump right in, shall we?

Create a new Console App and add a class to hold all the Deep Learning Logic. Name this class DeepNeuralNetwork or DNN.

Add the following fields into the class you have just created with the following C# code:

      private int intInput;
      private int intHiddenA;
      private int intHiddenB;
      private int intOutput;

      private double[] dblInputs;

      private double[][] dbliaWeights;
      private double[][] dblabWeights;
      private double[][] dblboWeights;

      private double[] dblaBiases;
      private double[] dblbBiases;
      private double[] dbloBiases;

      private double[] dblaOutputs;
      private double[] dblbOutputs;
      private double[] dbloutputs;
      private static Random rndRand;

You can achieve the same thing using VB.NET:

   Private intInput As Integer
   Private intHiddenA As Integer
   Private intHiddenB As Integer
   Private intOutput As Integer

   Private dblInputs As Double()

   Private dbliaWeights As Double()()
   Private dblabWeights As Double()()
   Private dblboWeights As Double()()

   Private dblaBiases As Double()
   Private dblbBiases As Double()
   Private dbloBiases As Double()
   Private dblaOutputs As Double()
   Private dblbOutputs As Double()
   Private dbloutputs As Double()

   Private Shared rndRand As Random

Adding Constructors in C#

These fields will hold the input Weights and their corresponding biases. Add the Constructor in C#:

      public DNN(int iInput, int iHiddenA, int iHiddenB,
         int iOutput)
      {

         intInput = iInput;
         intHiddenA = iHiddenA;
         intHiddenB = iHiddenB;
         intOutput = iOutput;

         dblInputs = new double[iInput];

         dbliaWeights = CreateMatrix(iInput, iHiddenA);
         dblabWeights = CreateMatrix(iHiddenA, iHiddenB);
         dblboWeights = CreateMatrix(iHiddenB, iOutput);

         dblaBiases = new double[iHiddenA];
         dblbBiases = new double[iHiddenB];
         dbloBiases = new double[iOutput];

         dblaOutputs = new double[iHiddenA];
         dblbOutputs = new double[iHiddenB];
         dbloutputs = new double[iOutput];

         rndRand = new Random(0);

         InitWeight();

      }

Adding the Constructor in VB.NET

   Public Sub New(ByVal iInput As Integer, ByVal iHiddenA _
         As Integer, ByVal iHiddenB As Integer, _
         ByVal iOutput As Integer)

      intInput = iInput
      intHiddenA = iHiddenA
      intHiddenB = iHiddenB
      intOutput = iOutput

      dblInputs = New Double(iInput - 1) {}

      dbliaWeights = CreateMatrix(iInput, iHiddenA)
      dblabWeights = CreateMatrix(iHiddenA, iHiddenB)
      dblboWeights = CreateMatrix(iHiddenB, iOutput)

      dblaBiases = New Double(iHiddenA - 1) {}
      dblbBiases = New Double(iHiddenB - 1) {}
      dbloBiases = New Double(iOutput - 1) {}
      dblaOutputs = New Double(iHiddenA - 1) {}
      dblbOutputs = New Double(iHiddenB - 1) {}
      dbloutputs = New Double(iOutput - 1) {}

      rndRand = New Random(0)

      InitWeight()

   End Sub

Read: Machine Learning in .NET

Add the CreateMatrix Function in C#

      private static double[][] CreateMatrix(int iRows, int iCols)
      {

         double[][] dblRes = new double[iRows][];

         for (int r = 0; r < dblRes.Length; ++r)
         {

            dblRes[r] = new double[iCols];

         }
         return dblRes;

      }

Add the CreateMatrix Function in VB.NET

   Private Shared Function CreateMatrix(ByVal iRows As Integer, _
         ByVal iCols As Integer) As Double()()

      Dim dblRes As Double()() = New Double(iRows - 1)() {}
      Dim r As Integer

      While r < dblRes.Length

         dblRes(r) = New Double(iCols - 1) {}

         Threading.Interlocked.Increment(r)

      End While

      Return dblRes

   End Function

This creates a matrix for each of our weights. Add the Weight functions to set random values for each weight in C#:

      private void InitWeight()
      {

         int intWeights = (intInput * intHiddenA) + intHiddenA +
            (intHiddenA * intHiddenB) + intHiddenB + (intHiddenB *
            intOutput) + intOutput;

         double[] dblWeights = new double[intWeights];

         double dblLO = -0.01;
         double dblHI = 0.01;

         for (int i = 0; i < dblWeights.Length; ++i)
         {

            dblWeights[i] = (dblHI - dblLO) * rndRand.NextDouble()
               + dblLO;

         }

         SetWeight(dblWeights);

      }

      public void SetWeight(double[] dblWeights)
      {
         int numWeights = (intInput * intHiddenA) + intHiddenA +
            (intHiddenA * intHiddenB) + intHiddenB + (intHiddenB *
            intOutput) + intOutput;

         if (dblWeights.Length != numWeights)
         {

            throw new Exception("Weight Length Error");

         }

         int j = 0;

         for (int i = 0; i < intInput; ++i)
         {

            for (int j = 0; j < intHiddenA; ++j)
            {

               dbliaWeights[i][j] = dblWeights[j++];

            }

         }

         for (int i = 0; i < intHiddenA; ++i)
         {

            dblaBiases[i] = dblWeights[j++];

         }

         for (int i = 0; i < intHiddenA; ++i)
         {

            for (int j = 0; j < intHiddenB; ++j)
            {

               dblabWeights[i][j] = dblWeights[j++];

            }

         }


         for (int i = 0; i < intHiddenB; ++i)
         {

            dblbBiases[i] = dblWeights[j++];

         }

         for (int i = 0; i < intHiddenB; ++i)
         {

            for (int j = 0; j < intOutput; ++j)
            {

               dblboWeights[i][j] = dblWeights[j++];

            }

         }

         for (int i = 0; i < intOutput; ++i)
         {

            dbloBiases[i] = dblWeights[j++];

         }

      }

VB.NET

   Private Sub InitWeight()

      Dim intWeights As Integer = (intInput * intHiddenA) + _
         intHiddenA + (intHiddenA * intHiddenB) + intHiddenB + _
         (intHiddenB * intOutput) + intOutput

      Dim dblWeights As Double() = New Double(intWeights - 1) {}

      Dim dblLO As Double = -0.01
      Dim dblHI As Double = 0.01

      Dim i As Integer

      While i < dblWeights.Length

         dblWeights(i) = (dblHI - dblLO) * rndRand.NextDouble() + _
            dblLO
         Threading.Interlocked.Increment(i)

      End While

      SetWeight(dblWeights)

   End Sub

   Public Sub SetWeight(ByVal dblWeights As Double())

      Dim numWeights As Integer = (intInput * intHiddenA) + _
         intHiddenA + (intHiddenA * intHiddenB) + intHiddenB + _
         (intHiddenB * intOutput) + intOutput

      If dblWeights.Length <> numWeights

         Throw New Exception("Weight Length Error")

      End If

      Dim j As Integer = 0
      Dim i As Integer = 0

      While i < intInput

         While j < intHiddenA

            dbliaWeights(i)(j) = dblWeights(Math.Min(Threading. _
               Interlocked.Increment(j), j - 1))
            Threading.Interlocked.Increment(j)

         End While

         Threading.Interlocked.Increment(i)

      End While

      While i < intHiddenA

         dblaBiases(i) = dblWeights(Math.Min(Threading. _
            Interlocked.Increment(j), j - 1))
         Threading.Interlocked.Increment(i)

      End While

      While i < intHiddenA

         While j < intHiddenB

            dblabWeights(i)(j) = dblWeights(Math.Min(Threading. _
               Interlocked.Increment(j), j - 1))
            Threading.Interlocked.Increment(j)

         End While

         Threading.Interlocked.Increment(i)

      End While

      While i < intHiddenB

         dblbBiases(i) = dblWeights(Math.Min(Threading. _
            Interlocked.Increment(j), j - 1))

         Threading.Interlocked.Increment(i)

      End While

      While i < intHiddenB

         While j < intOutput

            dblboWeights(i)(j) = dblWeights(Math.Min(Threading. _
               Interlocked.Increment(j), j - 1))

            Threading.Interlocked.Increment(j)

         End While

         Threading.Interlocked.Increment(i)

      End While

      While i < intOutput

         dbloBiases(i) = dblWeights(Math.Min(Threading. _
            Interlocked.Increment(j), j - 1))

         Threading.Interlocked.Increment(i)

      End While

   End Sub

Read: Azure Machine Learning Tutorial

Complete the Class in C#

Add the following C# code to complete the class:

      public double[] Compute(double[] dblValues)
      {

         double[] dblaSums = new double[intHiddenA];
         double[] dblbSums = new double[intHiddenB];
         double[] dbloSums = new double[intOutput];

         for (int i = 0; i < dblValues.Length; ++i)
         {

            dblInputs[i] = dblValues[i];

         }

         for (int j = 0; j < intHiddenA; ++j) {

            for (int i = 0; i < intInput; ++i)
            {

               dblaSums[j] += dblInputs[i] * dbliaWeights[i][j];

            }

         }

         for (int i = 0; i < intHiddenA; ++i)
         {

            dblaSums[i] += dblaBiases[i];

         }

         Console.WriteLine("nInternal aSums:");

         Program.ShowGrid(dblaSums, dblaSums.Length, 4, true);

         for (int i = 0; i < intHiddenA; ++i)
         {

            dblaOutputs[i] = HyperbolicTangent(dblaSums[i]);

         }

         Console.WriteLine("nInternal aOutputs:");

         Program.ShowGrid(dblaOutputs, dblaOutputs.Length, 4,
            true);

         for (int j = 0; j < intHiddenB; ++j)
         {

            for (int i = 0; i < intHiddenA; ++i)
            {

               dblbSums[j] += dblaOutputs[i] * .dblabWeights[i][j];

            }

         }

         for (int i = 0; i < intHiddenB; ++i)
         {

            dblbSums[i] += dblbBiases[i];

         }

         Console.WriteLine("nInternal bSums:");

         Program.ShowGrid(dblbSums, dblbSums.Length, 4, true);

         for (int i = 0; i < intHiddenB; ++i)
         {

            dblbOutputs[i] = HyperbolicTangent(dblbSums[i]);
         }

         Console.WriteLine("nInternal bOutputs:");

         Program.ShowGrid(dblbOutputs, dblbOutputs.Length, 4,
            true);

         for (int j = 0; j < intOutput; ++j)
         {
            for (int i = 0; i < intHiddenB; ++i)
            {

               dbloSums[j] += dblbOutputs[i] * dblboWeights[i][j];

            }

         }

         for (int i = 0; i < intOutput; ++i)
         {

            dbloSums[i] += dbloBiases[i];

         }

         Console.WriteLine("nInternal oSums:");

         Program.ShowGrid(dbloSums, dbloSums.Length, 4, true);

         double[] dblAllOut = MaxOutput(dbloSums);
         Array.Copy(dblAllOut, dbloutputs, dblAllOut.Length);

         double[] dblResult = new double[intOutput];
         Array.Copy(this.dbloutputs, dblResult, dblResult.Length);

         return dblResult;

      }

      private static double HyperbolicTangent(double val)
      {

         if (val < -20.0)
         {

            return -1.0;

         }

         else if (val > 20.0)
         {

            return 1.0;

         }

         else
         {

            return Math.Tanh(val);

         }

      }

      private static double[] MaxOutput(double[] oSums)
      {
         double dblMax = oSums[0];

         for (int i = 0; i < oSums.Length; ++i)
         {

            if (oSums[i] > dblMax)
            {

               dblMax = oSums[i];

            }

         }

         double dblScale = 0.0;

         for (int i = 0; i < oSums.Length; ++i)
         {

            dblScale += Math.Exp(oSums[i] - dblMax);

         }

         double[] dblRes = new double[oSums.Length];

         for (int i = 0; i < oSums.Length; ++i)
         {

            dblRes[i] = Math.Exp(oSums[i] - dblMax) / dblScale;

         }

         return dblRes;
      }

Complete the Class in VB.NET

Add the following code to complete the class in VB.NET:

   Public Function Compute(ByVal dblValues As Double()) As Double()

      Dim dblaSums As Double() = New Double(intHiddenA - 1) {}
      Dim dblbSums As Double() = New Double(intHiddenB - 1) {}
      Dim dbloSums As Double() = New Double(intOutput - 1) {}

      Dim i As Integer
      Dim j As Integer

      While i < dblValues.Length

         dblInputs(i) = dblValues(i)
         Threading.Interlocked.Increment(i)

      End While

      While j < intHiddenA

         While i < intInput

            dblaSums(j) += dblInputs(i) * dbliaWeights(i)(j)
            Threading.Interlocked.Increment(i)

         End While

         Threading.Interlocked.Increment(j)

      End While

      While i < intHiddenA

         dblaSums(i) += dblaBiases(i)
         Threading.Interlocked.Increment(i)

      End While

      Console.WriteLine(vbLf & "Internal aSums:")
      ShowGrid(dblaSums, dblaSums.Length, 4, True)

      While i < intHiddenA

         dblaOutputs(i) = HyperbolicTangent(dblaSums(i))
         Threading.Interlocked.Increment(i)

      End While

      Console.WriteLine(vbLf & "Internal aOutputs:")

      ShowGrid(dblaOutputs, dblaOutputs.Length, 4, True)

      While j < intHiddenB

         While i < intHiddenA

            dblbSums(j) += dblaOutputs(i) * dblabWeights(i)(j)

            Threading.Interlocked.Increment(i)

         End While

         Threading.Interlocked.Increment(j)

      End While

      While i < intHiddenB

         dblbSums(i) += dblbBiases(i)

         Threading.Interlocked.Increment(i)

      End While

      Console.WriteLine(vbLf & "Internal bSums:")

      ShowGrid(dblbSums, dblbSums.Length, 4, True)

      While i < intHiddenB

         dblbOutputs(i) = HyperbolicTangent(dblbSums(i))

         Threading.Interlocked.Increment(i)

      End While

      Console.WriteLine(vbLf & "Internal bOutputs:")

      ShowGrid(dblbOutputs, dblbOutputs.Length, 4, True)

      While j < intOutput

         While i < intHiddenB

            dbloSums(j) += dblbOutputs(i) * dblboWeights(i)(j)
            Threading.Interlocked.Increment(i)

         End While

         Threading.Interlocked.Increment(j)

      End While

      While i < intOutput

         dbloSums(i) += dbloBiases(i)
         Threading.Interlocked.Increment(i)

      End While

      Console.WriteLine(vbLf & "Internal oSums:")

      ShowGrid(dbloSums, dbloSums.Length, 4, True)

      Dim dblAllOut As Double() = MaxOutput(dbloSums

      Array.Copy(dblAllOut, dbloutputs, dblAllOut.Length)

      Dim dblResult As Double() = New Double(intOutput - 1) {}
      Array.Copy(Me.dbloutputs, dblResult, dblResult.Length)

      Return dblResult

   End Function

   Private Shared Function HyperbolicTangent(ByVal val As Double) _
         As Double

      If val < -20 Then

         Return -1

      ElseIf val > 20 Then

         Return 1

      Else

         Return Math.Tanh(val)

      End If

   End Function

   Private Shared Function MaxOutput(ByVal oSums As Double()) _
         As Double()

      Dim dblMax As Double = oSums(0)
      Dim i As Integer

      While i < oSums.Length

         If oSums(i) > dblMax Then

            dblMax = oSums(i)

         End If

         Threading.Interlocked.Increment(i)

      End While

      Dim dblScale As Double = 0

      While i < oSums.Length

         dblScale += Math.Exp(oSums(i) - dblMax)

         Threading.Interlocked.Increment(i)

      End While

      Dim dblRes As Double() = New Double(oSums.Length - 1) {}

      While i < oSums.Length

         dblRes(i) = Math.Exp(oSums(i) - dblMax) / dblScale

         Threading.Interlocked.Increment(i)

      End While

      Return dblRes

   End Function

The calculated inputs, along with the biases, get computed and outputted. Add the code for the Program class or the Module to make use of the DeepNeuralNetwork Class.

C#

      static void Main(string[] args)
      {
         Console.WriteLine("nStart Creating a 3-4-5-2 neural
            network");

         int intInput = 3;
         int intHiddenA = 4;
         int intHiddenB = 5;
         int intOutput = 2;

         DNN dnn = new DNN(intInput, intHiddenA, intHiddenB,
            intOutput);

         double[] dblWeights = new double[] {

            0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09,
               0.10,
            0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19,
               0.20,
            0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29,
               0.30,
            0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39,
               0.40,
            0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49,
              0.50,
         };

         dnn.SetWeight(dblWeights);

         double[] dblX = new double[] { 1.0, 2.0, 3.0 };

         Console.WriteLine("nDummy Weights + Bias Values");

         ShowGrid(dblWeights, 10, 2, true);

         Console.WriteLine("nDummy Inputs");

         ShowGrid(dblX, 3, 1, true);

         double[] dblY = dnn.Compute(dblX);

         Console.WriteLine("nComputed Outputs");

         ShowGrid(dblY, 2, 4, true);


         Console.WriteLine("nEnd");

         Console.ReadLine();
      }

      static public void ShowGrid(double[] dblVector, int intCols,
         int intDec, bool blnNewLine)
      {
         for (int i = 0; i < dblVector.Length; ++i)
         {

            if (i % intCols == 0)
            {

               Console.WriteLine("");

            }

            Console.Write(dblVector[i].ToString("F" +
               intDec).PadLeft(intDec + 4) + " ");
         }

         if (blnNewLine == true)
         {
            Console.WriteLine("");

         }

      }

VB.NET

   Sub Main(ByVal args As String())

      Console.WriteLine(vbLf & "Start Creating a 3-4-5-2 _
         neural network")

      Dim intInput As Integer = 3
      Dim intHiddenA As Integer = 4
      Dim intHiddenB As Integer = 5
      Dim intOutput As Integer = 2

      Dim dnn As DNN = New DNN(intInput, intHiddenA, intHiddenB, _
         intOutput)

      Dim dblWeights As Double() = New Double() {

         0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1,
         0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2,
         0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3,
         0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4,
         0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49, 0.5
      }
      dnn.SetWeight(dblWeights)

      Dim dblX As Double() = New Double() {1, 2, 3}

      Console.WriteLine(vbLf & "Dummy Weights + Bias Values")

      ShowGrid(dblWeights, 10, 2, True)

      Console.WriteLine(vbLf & "Dummy Inputs")

      ShowGrid(dblX, 3, 1, True)

      Dim dblY As Double() = dnn.Compute(dblX)

      Console.WriteLine(vbLf & "Computed Outputs")

      ShowGrid(dblY, 2, 4, True)

      Console.WriteLine(vbLf & "End")

      Console.ReadLine()

   End Sub

   Public Sub ShowGrid(ByVal dblVector As Double(), _
         ByVal intCols As Integer, ByVal intDec As Integer, _
         ByVal blnNewLine As Boolean)

      Dim i As Integer

      While i < dblVector.Length

         If i Mod intCols = 0 Then

            Console.WriteLine("")

         End If

         Console.Write(dblVector(i).ToString("F" & intDec) _
            .PadLeft(intDec + 4) & " ")
         Threading.Interlocked.Increment(i)

      End While

      If blnNewLine = True Then

         Console.WriteLine("")

      End If

   End Sub

Here you supply values for the weights. The ShowGrid procedure displays the results.

Conclusion to Deep Neural Networks Programming

Deep Neural Networks are not too complicated. They are just the beginning. Hopefully, in a future article, I can delve a bit deeper. Until then, goodbye.

Hannes DuPreez
Hannes DuPreez
Ockert J. du Preez is a passionate coder and always willing to learn. He has written hundreds of developer articles over the years detailing his programming quests and adventures. He has written the following books: Visual Studio 2019 In-Depth (BpB Publications) JavaScript for Gurus (BpB Publications) He was the Technical Editor for Professional C++, 5th Edition (Wiley) He was a Microsoft Most Valuable Professional for .NET (2008–2017).

More by Author

Get the Free Newsletter!

Subscribe to Developer Insider for top news, trends & analysis

Must Read