Saturday, 23 March 2013

Artificial Neural Networks Part 1: Why are they important?

Some products have algorithms that the program seems to learn over time, improving at its task. For example, Furbies.

As always in theoretical computer science, Furbies are by far the most demonstrative example.

How does that happen? If you have any algorithms experience, you’ll know that computers don’t ‘learn’ like a people do. How could you get bad output the first time you do something, yet good output a couple of times later? How could the Furby slowly learn to say the things you say to it? Is it, perhaps, a demon posessing powers far beyond those of modern machinery? Or is it using a neural network?


Hopefully, the latter. An artificial neural network (ANN) is a model for solving problems in the same way that a brain might. It uses many tiny components that mimic the work of neurons; they each read a small piece of input, process it, and pass it out to other neurons in the network. Each piece of input is brought closer to the correct output when it is processed, so the system eventually migrates to a solution.

ANNs have a large number of these neuron-like components, each doing a very small amount of work, which has advantages over the typical computer model:

  • ANNs can be incredibly fast, because they share the workload over so many components.
  • ANNs can be incredibly robust. Since no single component is doing all of the work, the algorithm can still produce the correct output with a few components missing or working incorrectly.
  • ANNs can be comparatively simple to implement because you don’t have to encode information about the specific problem you’re solving; you just make your model and take it through a ‘teaching’ process where it learns on its own to solve that problem. This means you can solve problems with ANNs that you don’t even understand the solution to.
A diagram of this process, from this great French theoretical computer science blog.

This may all seem magical and/or impossible, but it’s so not. In future posts I’ll explain myself by going through ANN implementation details, but I think it’s more to start off knowing how awesome they are before careening into the math. Because they are very, very awesome.

Some neurons, a.k.a. furby guts. Look at all that science.

No comments:

Post a Comment