Classical and Connectionist Cognitive Models
By ai-depot | September 2, 2002
The Debate
The Debate
In 1988 Fodor and Pylyshyn published their critique of the connectionist approach to cognitive architecture and sparked a furious debate that has raged for well over a decade. The critique runs to 60 pages, so as with the rest of this article this is a summary of their argument only. The basis of their paper rests on the belief that syntactic complexity is essential for cognitive processes. This belief is backed up by a two-fold argument: Productivity and Systematicity.
Productivity is the harder of the two concepts to grasp. Essentially, this is the idea that the representational capacity of the brain is unbounded given finite resources. What this means is that an unbounded number of propositions can in theory be expressed, but is restricted by the finite capacity of the thinking agent. To draw a parallel with language, the rules of grammar allow an infinite number of sentences to be constructed, in much the same way that knowledge of numbers allows an infinite number of sums to be formed. In practice humans are unable to express more than a certain number due to the finite capacity of the brain, but this does not preclude the possibility of constructing any and all sentences. Experiments have been used to show that the capacity of a human can be increased with certain motivating techniques, suggesting that the only limitation on the number of sentences constructed is the computational ability of the human. The separation of rules and content (symbols are related to each other in some causal way, rather than explicitly stating each and every proposition) in classical models allows productivity. To achieve unbounded productivity in this way with connectionist networks would require an infinite memory capacity due to the necessity of representing every single expression with some grouping of nodes. This ignores related concepts because one assumes that there are an infinite number of unrelated concepts in the universe.
The second of their arguments for syntactic complexity, systematicity of cognitive representation, is the stronger of the two. Systematicity refers to the phenomenon that the ability to understand propositions is intrinsically connected to the ability to understand others of related structure. Again, understanding this is easier when looking at natural language. There is no-one who can construct and understand the sentence “John loves Mary” who cannot also construct and understand the sentence “Mary loves John” or “The book loves Jill”. The fact that these sentences may be true or false is irrelevant, the assertion is that they are syntactically and semantically correct. Inferential capacities are also systematically related; we do not find people who can infer P from P&Q but who cannot also infer (A v B) from (A v B) & (C v D). Syntactic complexity can account for systematicity because two systematically related attitudes share some of the same parts (Lormand, 1991). Fodor and Pylyshyn contend that connectionist frameworks are incapable of accounting for systematicity without implementing a syntactically complex representational structure.
Conclusion
Discussing the pros and cons of the myriad of different systems that have been proposed as viable systematic systems using connectionist networks is beyond the remit of this article. However, the debate does leave the AI practitioner with a significant question to answer. Since neither approach solves all aspects of cognition (even in combination they fail to deal with all aspects of human intelligence), the question is what approach should be adopted in a new system? The trendier practitioners will immediately say that neural networks are the obvious choice, but there are many attractions to the use of a symbolic architecture. Whatever the state of systematicity in connectionist models, it is still easier to implement and understand it in symbolic architectures. The computational and inferential power offered by such architectures is also attractive. For instance, at one extreme, theorem proving would almost certainly require the use of a symbolic architecture. Symbolic models for Natural Language Parsing (NLP) have also been more successful than connectionist ones, though how long that will continue to be the case is debatable.
On the other hand, neural networks have obvious benefits for certain types of cognition, in particular pattern recognition. Though still not as proficient as human beings, neural networks are vastly superior to previous symbolic efforts at pattern recognition in domains such as image analysis. These recognition tasks tend to be of a certain kind – they are the result of statistical weightings brought about by the initial training. Whether there is really any kind of deductive, intelligent process occurring within these networks is open to argument. Symbolic processes suffer from similar criticisms, but they do offer an account of their deductive process. Can neural networks interpret and produce a set of propositions such as “John loves Mary. John is in love” or “If John loves Mary and Jane, then John loves two people”? As these propositions and inferences become more complex, neural networks struggle to explain how they might make such deductions.
In many cases the difference between the two approaches may come down to resources. For example, neural networks require training data in order to set the weights and thresholds appropriately. If this data is unavailable, as is often the case in expert systems, then there are practical limitations to using a neural network.
This all supports the conclusion that both approaches have their advantages and disadvantages, and it is left to the system designer to decide which properties are most attractive for the given task. Deciding this correctly could be the difference between a successful system and a dead-end.
Written by Linden Hutchinson.
Tags: none
Category: essay |
