Problem with simple artificial neural network -- adding

[declan]

Hi everyone! I am trying to make a simple artificial neural network work with the backpropagation algorithm. I have created an ANN and I believe I have implemented the BP algorithm correctly, but I may of course be wrong.

Right now, I am trying to train the network by giving it two random numbers (a, b) between 0 and 1, and having it add them. Then, of course, each time the output the network gives is compared to the theoretical answer of a + b.

Strangely, the output always converges to a number between 0 and 1 (as it must, because of the sigmoid function), but the random numbers I'm putting in seem to have no effect on it.

Edit: Sorry, it appears it doesn't converge when I am trying to get it to add. But it still converges like I mention below**

The weights are randomly distributed between -1 and 1, but I have also tried between 0 and 1.

**I also tried giving it two constant numbers (0.35,0.9) and trying to train it to spit out 0.5. This works and converges very fast to 0.5. I have also trained it to spit out 0.5 if I give it any two random numbers between 0 and 1, and this also works.

I have tried a couple different networks, since I made it very easy to add layers to my network. The standard one I am using is one with two inputs, one layer of 2 neurons, and a second layer of only one neuron (the output neuron). However, I have also tried adding a few layers, and adding neurons to them. It doesn't seem to change anything. My learning rate is equal to 1.0, though I tried it equal to 0.5 and it wasn't much different.

Does anyone have any idea of anything I could try?

Is this even something an ANN is capable of? I can't imagine it wouldn't be, since they can be trained to do such complicated things.

Any advice? Thanks!

Here is where I train it:

//Initialize it. This will be one with 2 layers, the first having 2 Neurons and the second (output layer) having 1.
vector<int> networkSize;
networkSize.push_back(2);
networkSize.push_back(1);
NeuralNetwork myNet(networkSize,2);

for(int i = 0; i<5000; i++){
	double a = randSmallNum();
	double b = randSmallNum();
	cout << "\n\n\nInputs: " << a << ", " << b << " with expected target: " << a + b;

	vector<double> myInput;
	myInput.push_back(a);	
	myInput.push_back(b);	

	vector<double> target;
	target.push_back(a + b);

	cout << endl << "Iteration " << i;
	vector<double> output = myNet.backPropagate(myInput,target);
	cout << "Output gotten: " << output[0];
	resultPlot << i << "\t" << abs(output[0] - target[0]) << endl;
}