Neural Networks in 3 Minutes

By ai-depot | June 30, 2002

Things to Keep in Mind

Facing Facts

Despite what people say, and how simple they appear here, Neural Networks aren’t a miracle solution!!

  • Quality of the results is usually that of a good approximation, but rarely optimal. Training and learning is a hard problem, and that’s what it’s all about!
  • NN are quite fussy. Chosing the right structure and size can have a huge impact on the solution. Hence the need for NN design, or automated solutions.
  • The results provided are approximate, and can diverge a lot in some cases.
  • You can’t model a human brain with current NN! The technology does not scale up to handle billions of neurons… yet.

That said, it’s not only about the neural networks alone! There are lots of clever things that can be done to remedy these problems, without which a purely NN solution would be poor… to tackle complex problems you really need to understand what’s going on.

Future Improvements

We’ve learnt a bit about real brains from ANN, but not that much. Models are still not especially realistic. Many new factors are being discovered by neuro-scientists and included into models:

  • Gas - Some neurons emit gas as well as electricity to communicate.
  • Spikes - It’s possible that the frequency of the neuron’s electrical impulses are also very important.

That aside, there’s also plenty to do for pratical applications.

  • Theory - Mathematical understanding of original models is good, but we need really need to know more about how the two factors above affect the learning.
  • Plasticity - Most NN models are time independant, so their results are the same for each input. This is not convenient for problems where time is important. More plasticity would help the network keep track of what it’s doing, and allow more complex behaviours.
  • Scaling Up - I’ve heard of a very simple NN with over 1 million neurons, but the model didn’t learn. On average, neuron counts rarely exceed the hundreds. We really need to look into efficient solutions for getting more neurons to learn more complex tasks, and not just the ones listed in the previous page.

And much more! If you this has got you interested in neural networks, I recommend getting a good book on the subject. See the books section for good recommendations.

Written by Alex J. Champandard.

Pages: 1 2 3

Tags: none
Category: tutorial |

Comments