To me, one of the most exciting, intriguing things about "connectionist machines" is that they can learn!

• A so called neural network can be thought of as a collection of highly interconnected elements in which the strength of individual links is adjusted as a result of the success of its functioning. The links are both feedforward, from place A to places B, C and D (for example) as well as feedback based on their activation from, say places B and C onto A. The important point is that a neural network continuously changes its internal workings as a function of the way its outputs correlate with its inputs. Further, it does so in an automatic fashion without the need of continued intervention from a programmer.
• Typically, neural networks are not built up in real life using analog, electrical elements like resistors and integrated circuit chips. Rather they are usually implimented by a complex, self-modifing computer program.

An very simplified example.

• The "Alphabet Machine" has the capacity to input any written capital letter (A, B, C, . . . , Z) via a scanner and break it up into its component lines, angles, curves, etc. It also has the ability to say (in synthesized speech played through a speaker) the name of each of the letters.
• The artificial neurons that comprise the machine are very interconnected and the strength of each connection can vary between -1 (fully inhibitory) and +1 (fully excitatory).
• Initially all connection strengths are set at zero. A letter, say R, is provided as an input and the output of the machine is noted. If the machine "says" R, the operator tells the machine it has succeeded, (and nothing else) and nothing is changed and we go to the next letter to test.
• It's more likely that the machine will say something other than R. In that case the operator tells the machine it failed (and nothing else) and the machine changes some of its connection weights, initially randomly and tries again. This cycle is repeated over and over until the machine finally says R.
• Then, a different letter is input, say G. The output of the machine is communicated and if it is not G the machine again goes through a cycle of changing the weights of its internal connections. However, now when it does so it tries not to modify the non-zero weights that it correlated with the letter R, if at all possible.
• This is repeated over and over for all the letters, including repeats to check that "good" connections already made are not degraded too much, until the machine can correctly say the name of every letter in the alphabet.
• Realize what has occurred: the machine has learned to recognize written capital letters. There was nothing in its circuitry (or programming) that was specific to recognizing any letter. There was nothing in its weight-changing process that was preprogrammed in favor of any particular letter. It just used trial and error, with feedback based on success, to arrive at a set of interconnections between its artificial neurons that are adequate to the task.

Now, if you are skeptical I suggest you try this for yourself. There is a game called SUSAN (a relative of Chinese Checkers, or perhaps GO) available on the Web here. (You download it to a Macintosh computer and then play offline.) Susan is a game you play against the computer as your opponent. It uses a computer-realized neural network to learn the strategy completely based on its success in playing YOU; thus, it quickly arrives at and then plays at your skill level. The game involves levels of strategy, and Susan seems to learn them very well. When I played it seemed like she grew to "know" just what to do. (I won't tell you how many hours I spent.)