Correct output(green line) vs network output(blue line), after about 15,000 iterations

Can anyone offer any feedback on my implementation, and whether the backprop algorithm is 'correct'?

EDIT: I tried leaving the network to learn for a few hours, over probably several million or more iterations, and the two lines on the grah did seem to come together more, however, the cost refused to budge below 0.41-ish.

Correct output(green line) vs network output(blue line), after about 15,000 iterations

Can anyone offer any feedback on my implementation, and whether the backprop algorithm is 'correct'?

EDIT:

Below is the testing code that I am using (just put after the neural network code in the same file). I realise that it is extremely messy, and contains many examples of bad practice, but it seems to work (however inelegantly). I have tested extensively with different hyperparameters, not just the ones used in the code below, and I have not been able to get the cost down below 0.41-ish, which does seem pretty poor. If anyone can find hyperparameters which would decrease the minimum bound on the cost significantly, that would also basically answer my question.

np.seterr(all='raise')
def plot():
 plt.clf()
 plt.plot([i/100 for i in range(100)],[net.predict(np.array([[1,f/100]]))[0][0] for f in range(100)])
 plt.plot([i/100 for i in range(100)],[((f/100)**2) for f in range(100)])
 plt.pause(0.05)
import random
net=neural_network([2,400,1],lambda x:1/(1+np.exp(-x)),lambda x:(1/(1+np.exp(-x)))*(1-(1/(1+np.exp(-x)))))
count=0
import matplotlib.pyplot as plt
plt.ion()
r=[i/1000 for i in range(1,1000)]
inp=np.array([[1,f] for f in r])
outp=np.array([[f**2] for f in r])
net.predict(inp)
while True:
 count+=1
 
 net.train(inp,outp,1,0.001)
 if count%100==0:
 
 print(net.cost(0.001))
 plot()