Neural Net XOR Genetic Algorithms

Question

New to Neural Networks and before I move on to gradient descent I would like to make sure I have got basic idea right. (Sorry that the class is called perceptron I know that this isnt technically right, I adapted this code from and AND gate NN). This NN is made to learn the XOR function.

#include <iostream>
#include <ctime>
#include <stdlib.h>
#include <valarray>
#define POPSIZE 100
using namespace std;
float randfloat();
float Sigmoid(float A);
class Perceptron{
public:
 Perceptron()
 {
 weights[0] = randfloat();
 weights[1] = randfloat();
 weights[2] = randfloat();
 weights[3] = randfloat();
 weights[4] = randfloat();
 weights[5] = randfloat();
 weights[6] = randfloat();
 weights[7] = randfloat();
 weights[8] = randfloat();
 CalculateError();
 }
 float FeedForward(int A, int B)
 {
 float Sum1 = A*weights[0] + B*weights[1] + bias*weights[4];
 float Sum2 = A*weights[2] + B*weights[3] + bias*weights[5];
 float Sum3 = Sigmoid(Sum1)*weights[6]
 + Sigmoid(Sum2)*weights[7]
 + bias*weights[8];
 return Sigmoid(Sum3);
 }
 void PrintStats()
 {
 cout << "weights =" << weights[0] << " " << weights[1] << " " << weights[2];
 cout << "\nerror= " << normalisedError;
 }
 void CalculateError()
 {
 error = abs(0-FeedForward(1,1)) +
 abs(1-FeedForward(1,0)) +
 abs(1-FeedForward(0,1)) +
 abs(0-FeedForward(0,0));
 normalisedError = error*10;
 }
 float GetNormalisedError()
 {
 return normalisedError;
 }
 void SetFitnessScore(int i)
 {
 fitnessScore = i;
 }
 int GetFitnessScore()
 {
 return fitnessScore;
 }
friend void CrossOver(Perceptron A, Perceptron B, Perceptron &C);
friend void TransferGenes(Perceptron A, Perceptron &B);
friend int ChooseFittestSpecies(Perceptron pop[POPSIZE]);
private:
 float weights[9];
 int bias = 1;
 float error =0;
 int normalisedError;
 int fitnessScore;
};
int ChooseParent(Perceptron pop[POPSIZE]);
int ChooseFittestSpecies(Perceptron pop[POPSIZE]);
int ChooseWorstSpecies(Perceptron pop[POPSIZE]);
void Reproduce(Perceptron pop[POPSIZE], Perceptron C[POPSIZE]);
int main()
{
 srand(time(0));
 Perceptron pop[POPSIZE];
 Perceptron Children[POPSIZE];
for(int x =0 ; x < 100 ; x++){
 Reproduce(pop, Children);
 for(int i = 0 ; i < POPSIZE ; i ++)
 {
 TransferGenes(Children[i], pop[i]);
 pop[i].CalculateError();
 }
}
int alphaSpecies = ChooseFittestSpecies(pop);
cout << "Most fit species is: " << alphaSpecies << endl;
 cout << "inputs:" << endl;
 cout << "1 1 : " << pop[alphaSpecies].FeedForward(1,1) << endl;
 cout << "0 1 : " << pop[alphaSpecies].FeedForward(0,1) << endl;
 cout << "1 0 : " << pop[alphaSpecies].FeedForward(1,0) << endl;
 cout << "0 0 : " << pop[alphaSpecies].FeedForward(0,0) << endl;
 return 0;
}
void TransferGenes(Perceptron A, Perceptron &B)
{
 for(int i = 0 ; i < 9 ; i++)
 {
 B.weights[i] = A.weights[i];
 }
}
void Reproduce(Perceptron pop[POPSIZE], Perceptron C[POPSIZE])
{
 int ParentA, ParentB;
 for(int i = 0; i < POPSIZE ; i++)
 {
 ParentA = ChooseParent(pop);
 ParentB = ChooseParent(pop);
 CrossOver(pop[ParentA], pop[ParentB], C[i]);
 }
}
void CrossOver(Perceptron A, Perceptron B, Perceptron &C)
{
 int random;
 for(int i = 0 ; i < 9 ; i ++)
 {
 random = rand()%2;
 if(random)
 {
 C.weights[i] = A.weights[i];
 }else{
 C.weights[i] = B.weights[i];
 }
 }
}
int ChooseParent(Perceptron pop[POPSIZE])
{
 int GreatestError = pop[0].GetNormalisedError();
 for(int i = 1; i < POPSIZE ; i++)
 {
 if(pop[i].GetNormalisedError() > GreatestError)
 {
 GreatestError = pop[i].GetNormalisedError();
 }
 }
 int totalFitness=0;
 for(int i =0 ; i < POPSIZE ; i++)
 {
 pop[i].SetFitnessScore(GreatestError + 1 - pop[i].GetNormalisedError());
 totalFitness = totalFitness + pop[i].GetFitnessScore();
 }
 int random = rand()%totalFitness +1;
 int ParentSelector = 0;
 int ParentIndex;
 for(int i=0;ParentSelector<random;i++)
 {
 ParentSelector = ParentSelector + pop[i].GetFitnessScore();
 ParentIndex = i;
 }
 return ParentIndex;
}
float randfloat()
{
 float f = static_cast <float> (rand()) / (static_cast <float> (RAND_MAX/10));
 int Pos_or_Neg = rand()%2;
 if(Pos_or_Neg)
 {
 return f;
 }else{
 return -f;
 }
}
float Sigmoid(float A)
{
 return 1/(1+exp(-2*A));
}
int ChooseFittestSpecies(Perceptron pop[POPSIZE])
{
 int MostFitParent = 0;
 for(int i = 0; i < POPSIZE ; i++)
 {
 if(pop[i].error < pop[MostFitParent].error)
 {
 MostFitParent = i;
 }
 }
 return MostFitParent;
} 

Answer 1

There's a lot that I could say, but I'm going to limit to what I consider the most major issues:

Questionable Coding Practices

The big two are using namespace std is bad practice and rand() is considered harmful. The former is straightforward: just drop it and prefix all the std references with std::. It's only five characters and adds a great deal to readability. There's really no reason to avoid it.

For the latter, rand() just is not the best way to generate random numbers. Additionally, it is very unclear to me what is actually going in in randfloat(). It'd be much better to use the C++11 <random> library for this, specifically std::uniform_real_distribution:

float randfloat() {
 static std::random_device rd;
 static std::mt19937 gen(rd());
 static std::uniform_real_distribution<float> dis(YOUR_MIN, YOUR_MAX);
 return dis(gen);
}

I cannot tell what your actual min and max are - so fill those spaces in accordingly. Once you get over the mt19937 boilerplate, this is a much more straightforward way to do this.

Additionally, avoid macros:

#define POPSIZE 100

should be:

static constexpr int POPSIZE = 100;

And please use consistent indentation!

Weights

First, you could generate them all using std::generate instead of repeating the same line of code:

std::generate(std::begin(weights), std::end(weights), randfloat);

Secondly, prefer std::array to a raw array. This will let you do:

void TransferGenes(Perceptron A, Perceptron &B)
{
 B.weights = A.weights;
}

No loop necessary. Although here, the first argument should definitely be a reference to const to avoid the copy. And A and B are rather unhelpful names, prefer something like:

void TransferGenes(Perceptron const& src, Perceptron& dst)
{
 dst.weights = src.weights;
}

Typically the dst argument goes first (e.g. memcpy), but YMMV.

You may also want to strongly consider using double instead of float.

Passing Arrays

Writing something like:

int ChooseFittestSpecies(Perceptron pop[POPSIZE]);

is meaningless since the POPSIZE value doesn't matter. This is a function that takes a Perceptron* as an argument. So you don't actually know what the size of the array is, despite the code giving the appearance that you do. Here, I would encourage you strongly to use std::vector:

int ChooseFittestSpecies(std::vector<Perceptron>& pop);

Vectors are much easier to deal with. Plus, if somebody passes you a vector whose size is different from POPSIZE, your code will do the right thing still.

Furthermore, this particular function could take advantage of std::min_element, which clarifies the intent better:

int ChooseFittestSpecies(const std::vector<Perceptron>& pop)
{
 auto it = std::min_element(pop.begin(), pop.end(), 
 [](const Perceptron& lhs, const Perceptron& rhs){
 return lhs.error < rhs.error;
 });
 return std::distance(pop.begin(), it);
}