Introduction

Oliver Selfridge’s Pandemonium model was a very early (1957) attempt to model a complex cognitive process like letter recognition using a computer simulation. The model became famous for two reasons. First, as scientists learned more about the primate brain, it became clear that Pandemonium’s structure mimics the brain’s structure in some important ways. Second, Selfridge described the model in a highly entertaining way.

At left you see a simplified version of Selfridge’s Pandemonium model of letter recognition.

Instructions

Click on one of the patterns shown at the very bottom of the screen to present the pattern to the model and see what all the demons do. The letter being “shouted” by the Decision Demon will be the model’s guess about what letter the pattern represents.

Click on the Cognitive Demons to see which Feature Demons they are listening to.

The links provide detailed descriptions of the model, how it works, and how it relates to the brain. After you have experimented with the model a bit, start by reading about the Feature Demons.

Feature Demons

Selfridge called the basic units in the model “demons.” Demons are roughly analogous to neurons in the brain, and just as neurons can have varying degrees of activity (as reflected in each neuron’s firing rate), the demons in the Pandemonium model exist in varying stages of excitement. In our demonstration, at different times, each demon may be asleep, awake but bored, moderately interested, or jumping-up-and-down excited.

In our version of the model, there are three levels of demons: eight Feature Demons, five Cognitive Demons, and a single Decision Demon sitting at the top of the model. Demons on each level have different jobs. Each Feature Demon is awakened when its particular feature shows up in the input to the model, shown in the white square near the bottom of the screen. Right now, there is no pattern present (the input square is blank), so all the Feature Demons are asleep.

If you click on one of the possible inputs, you will see the appropriate Feature Demons respond to the presence of a feature each one is looking for.

Cognitive Demons

Now you should see a single vertical line in the white input square near the bottom of the screen. The Feature Demon on the far left adores vertical lines, so a second after the pattern appeared, you saw him start jumping up and down and screaming. None of the other Feature Demons care about vertical lines, so they stayed asleep.

Now consider the Cognitive Demons on the level above the Feature Demons. These demons don’t know anything about the input pattern directly, but they listen to the activity of the Feature Demons. Take the H demon on the far right, for example. It gets excited when it hears the vertical and horizontal Feature Demons screaming, while all the other Feature Demons just tend to put it to sleep if they yell. You can click on any of the Cognitive Demons at any time to see what they want to hear: a green line over a Feature Demon means this Cognitive Demon likes to hear from him, a red line indicates a preference for silence, and no line means the Cognitive Demon just doesn’t care about that Feature Demon one way or the other.

If a Cognitive Demon hears screams from the Feature Demons he likes and doesn’t hear anything from his non-preferred Feature Demons, then he will in turn get excited and start screaming after a couple of seconds. For this simple pattern, the A, O, and X demons didn’t even wake up, the R demon began vacillating between sleep and wakefulness, and the H demon became mildly interested.

The Decision Demon

At the top of the model is the purple Decision Demon. Her job is to decide what letter is being presented down at the bottom of the model in the input space. But like the Cognitive Demons, the Decision Demon doesn’t know anything about the input directly. In fact, the Decision Demon can’t even see or hear the Feature Demons. All the Decision Demon can do is listen to the Cognitive Demons. She makes her decision very simply: Once the Cognitive Demons start shouting, the one that makes the most noise wins her over.

With just a vertical line in the input space, the H demon is generally the loudest but there’s a certain amount of random variation in the demons’ behavior (this is also true for neurons), so the Decision Demon vacillates a bit between H and R. When you see an exclamation point next to the Decision Demon’s letter pronouncement, she is fairly sure of herself. When you see a question mark, she is not quite as sure because two or more Cognitive Demons are vying for her attention. You will also see her get more or less excited, depending on the excitement level of the Cognitive Demons beneath her.

Of course, the pattern that the model is looking at now doesn’t really look like either an R or an H, so it makes sense that the Decision Demon isn’t convinced about what it is. Click in the strip at the very bottom of the screen to show different patterns to the model and see how the demons respond. The first time through, it is probably best to go in order from left to right.

Modularity

As you saw while you worked through the various patterns, Pandemonium is far from a perfect model of letter recognition. However, as noted in the introduction, the model is still discussed today because it illustrates, in its whimsical way, some important principles about how the brain works.

One such principle is the idea that each area of the brain, represented in the Pandemonium model by the different levels of demons, is a “mini-computer,” receiving information from another part of the brain, processing that information in some way, and sending the results off to another part of the brain for further processing. (For illustration purposes, we delayed each layer’s responses by a couple of seconds in our model but real neurons transmit information much more quickly.)

It is dangerous to draw precise analogies, but most vision scientists believe that simple and complex cells in striate cortex process raw input from the eye and then send the results on to extrastriate areas, in roughly the same way that the Feature Demons in Pandemonium examine the input and then communicate with the Cognitive Demons.

Parallel Processing

Another way that the Pandemonium model mimics the brain is by processing information in parallel, rather than operating serially like most digital computers. That is, instead of checking first for vertical lines, then for horizontal lines, then for left oblique lines, etc., Pandemonium employs a group of demons that each check for their own preferred feature all at the same time. Similarly, the Cognitive Demons are all looking for their letters at the same time and the Decision Demon is always ready to receive input from any Cognitive Demons that become active.

If you think about it, this is a very efficient way of doing pattern recognition. Imagine expanding the model so that it could recognize all the letters, in both upper and lower case, as well as all the punctuation marks, digits, and other symbols used in English writing. If the model had to go through each pattern in turn (“Is it an ‘A’? Is it a ‘p’? Is it a ‘#’? ...”), it would quickly get bogged down as we added more and more to-be-recognized patterns. But with parallel processing, we just add some more demons and the process doesn’t take any longer.

Serial processing is good for some things, such as number crunching, but brains turn out to be massively parallel devices. This is one reason why computers are much better than humans at long division, but no one has yet developed a computer system that comes even close to the object recognition abilities of a 6-year-old human brain.

Perception by Committee

A final concept that the Pandemonium model can help illustrate is the idea of perception by committee introduced in your textbook. Each of the Cognitive Demons is looking for a certain pattern, but the A demon, for example, doesn’t require a perfectly formed “A” to respond. This is adaptive, since not all “A”s are alike (this is why we can’t simply use a naïve template system for object recognition, as noted in your textbook). However, it also means that an “X” pattern looks pretty good to the A demon (since it has two of the three features that are required for a pattern to be an A). So why doesn’t the model call “X”s “A”s?

The answer is that the model doesn’t just have one Cognitive Demon, but instead employs a committee of demons. In addition, the model has a protocol for getting the committee to come to a final conclusion: the demon that shouts the loudest wins. (This might not be the best way for a committee to decide on a Middle East peace agreement, but it works just fine for the Pandemonium model.) Thus the A demon is fairly enthusiastic about the “X” pattern, but the X demon shouts even louder, so the right decision ends up getting made in the end.

Other perceptual decisions get made in similar ways. Take the Gestalt Grouping Principles, for example. One could imagine a Gestaltimonium model, with Principle Demons each shouting with a volume proportional to their belief that the elements in an image should be organized in one way or another. Given the pattern at right, the Proximity Demon would be yelling to organize the elements into columns, but the Similarity Demon might shout even louder that the elements should form rows.

This pattern gets the horizontal, left oblique, and right oblique Feature Demons shouting at their loudest. The vertical demon is also mildly interested, since the two sides of the pattern are fairly close to vertical. If you click on the A Cognitive Demon, you’ll see that this is exactly the pattern he likes to see—all of the Feature Demons that excite him are active, and none of the Feature Demons that inhibit him (the demons looking for curves) are active. The X, R, and H demons are also fairly interested in this pattern, which sometimes gives the Decision Demon pause (note that you sometimes see a question mark next to her announcement), but she’s pretty confident that this pattern is an “A.”

This pattern is just a smaller version of the one directly to its left, and as you can see, the model responds exactly the same way to the two patterns. This is an important behavior, called size invariance, which is also shown by humans. Note that a naive template theory would not show size invariance—if you had a template for the larger “A” stored in memory, this template would not match the smaller A pattern, so the template model would have much more difficulty recognizing the two patterns as the same letter.

You should also be able to see that Pandemonium, but not naive template models, would also show translation invariance (it wouldn’t care where in the white square the letter was located).

This pattern is unambiguously interpreted by the model as an H (note that the exclamation point stays on almost continuously). The A and R demons sporadically wake up for a look, since they are mildly excited by the horizontal and vertical lines, respectively, but only the H demon is looking for both a horizontal and a vertical line.

Remember that the variations in the model’s responses over time are caused by random fluctuations in all the demons’ activity levels. This mimics the responses of brain neurons, whose firing rates also fluctuate randomly to some extent.

This pattern demonstrates a major problem with the Pandemonium model. Clearly, this is a different letter than the pattern to the left. However, since “T” and “H” have the same two features (horizontal and vertical lines), the model cannot tell the difference between them. Note that adding a T demon would not solve the problem—it would be looking for the same thing as the H demon, so the Decision Demon would simply vacillate back and forth between a “T” and “H” interpretation. What is needed is some way to register not just what features are present, but also how the features relate to each other: In an “H” the horizontal line is connected to the middle of the verticals, while in a “T” the horizontal line is on top of the vertical. (Understanding how many of each feature are present would also be helpful.)

Coding for relationships as well as features, an issue known as the “binding problem,” has turned out to be one of the most difficult nuts for structural description theorists to crack.

This pattern is easily recognized by the model as an “X,” although the A demon competes a little with the X demon for the Decision Demon’s attention (just as the X demon was fairly excited by the “A” patterns).

This pattern is interesting because it is not a well-formed example of any letter. If forced to make a decision, you would probably say it is an “A,” which is also what the model thinks most of the time. But the Decision Demon is often not too sure of herself, and sometimes interprets the pattern as an “H” or an “X.” Note that as with other ambiguous figures such as the Necker cube, you can interpret this pattern as only one letter at a time—you cannot simultaneously see it as both an “A” and an “X.” Interestingly, the model shows the same behavior: The Decision Demon switches her interpretation periodically, but she does not entertain two interpretations at the same time.

This is the least ambiguous of any of the patterns: Only the O demon is looking for all four of the curved Feature Demons to be active, and the R demon is the only other one that shows any interest in the pattern at all.

Our model has no chance of interpreting this pattern “correctly,” because we haven’t included a G Cognitive Demon (but we could—which Feature Demons would we want to excite and inhibit such a demon?). Given the demons we do have, the pattern looks most like an O, but note that the Decision Demon isn’t too excited about her decision. The small horizontal and vertical lines are at times enough to lead the model to an “H” decision.

This pattern is clearly interpreted as an “R,” although the A, X, and H demons are mildly interested in the vertical and right oblique lines in the pattern.

This pattern provides another example of the crippling effect of not coding for relations in the model. Here we have something that is clearly (to our visual systems) not an “R.” However, since it contains the same features as an “R” (a vertical line, a right oblique, and a right curve), the model interprets this pattern in exactly the same way as it did the real “R.”

Here we see a pattern that looks much more like an “R” to us than the scrambled “R” to the left (in fact, it is a mirror-reversed “R”). However, the R demon isn’t interested in this pattern at all and the Decision Demon vacillates unenthusiastically between A, H, and X interpretations.

Humans are quite good at recognizing mirror reversals of known patterns, but this example shows that the Pandemonium model is not.