Class Notes: Chapter 9

Competence of Connectionist Models

An essential feature of connectionist models: There is no difference between what is learned and the rules for using that knowledge
Everything is a matter of connections among the nodes
Can such a simplified view of the mind account for all its complexities?

Models of Language Understanding

Language has been an important test
It provides a framework for comparing physical symbol systems and connectionist networks
How do we learn to use and understand a language?
What does understanding a language involve?

Chapter 9: Section 9.1

Explain the default hypothesis about what it is to understand a language.

On its face, rules appear to be central to understanding a language
Rules for combining words grammatically
Rules for understanding deep structure
The hypothesis: Rules are represented explicitly
I.e., a rule-based system must be necessary
Are connectionist theories inadequate(?)

Understanding Language

What does it mean to “comprehend” a sentence?
Example: “UNC was favored to win the ACC tournament”
How do you know that you know what that means?

Fodor: Comprehension requires a general “truth condition”, plus rules of individual reference
What events or objects do “UNC”, “favor”, “ACC”, “tournament” refer to?
What would make the statement true?

Explain Fodor’s argument from language learning for the language of thought hypothesis.

Language learning occurs by testing hypotheses
The hypotheses must be expressed in the “language of thought” (we don’t have any other language yet)
Thus the LOT has to exist before you can learn your first language

Chapter 9: Section 9.2

Learning a Language

What can we discover by studying children as they acquire a language?
Regardless of the language, there are well defined stages that children pass through.
“The language instinct”: There is a language module that acquires the components of language very rapidly
Pinker: Connectionist models cannot in principle account for the way language develops

Describe the three stages in learning the tenses of verbs.

Consider the learning of the past tense:
Stage 1: Learn common verbs, mostly irregular: “I give” ? “I gave”
Stage 2: Learn the regular rule (add ‘-ed’)
Thus, errors of over-generalization occur: “I give” ? “I gived”
Stage 3: Master the irregular exceptions

How do Pinker and Prince explain the trajectory of tense learning?

A dual route process:
Rule learning for the regular verbs
Associational learning for the irregular verbs

Describe the structure and operation of Rumelhart and McClelland’s network for past tense acquisition.

Three inter-connected single level networks:
1. Phonemes into root form: Wicklephones (Wicklefeatures)
2. Patterns of past tense associations
3. Past tense into phonemes
This model also mimics the three stages of learning

What is Pinker and Prince’s criticism of the way that Rumelhart and McClelland model the over-regularization phenomenon?

An extended critique of the R & M model
Argue that its successes are artifacts of the way in which the network was trained.
Closer inspection of the details shows many failings

it cannot handle some words and rules
it cannot explain certain regularities in irregular forms (e.g., “send” - “sent”, “bend” – “bent”, "wend" - "went")
it cannot explain the when forms are regular or irregular
it gives an incorrect explanation for some developmental phenomena, and it gives accounts of others that are indistinguishable from rule-based theories.

How did Plunkett and Marchman improve on Rumelhart and McClelland’s network?

Three layer network permits more complex learning
Input phonemes --> Hidden units --> Output phonemes
But is this modification enough?

Rules and Associations

Kim, Pinker, Prince & Prasada “Why no mere mortal has ever flown out to center field”
Compare:
“Sam flew out to see his uncle in California”
“Sam flied out to see his uncle in California”
“Ted flew out to center field to end the inning”
“Ted flied out to center field to end the inning”
When is the regular tense better, when is the irregular tense better?

When is the Irregular iregular?

Another example of regular and irregular verbs:
1. When guests come, I hide the empty dishes in the sink.
2a. Bob was early, so I quickly sinked the plates.
2b. Bob was early, so I quickly sank the plates.
Why does the regular past tense (a) sound better than the irregular past tense (b) ?

The Plural of Nouns

Some plurals are regular, some are irregular:
1a. He is a tribal chief. 1b. On our trip we met many tribal chiefs.
2a. This is a maple tree with a maple leaf. 2b. On the tree are many maple leaves.
3a. A hockey player for Toronto is a Maple Leaf. 3b. The team is the Toronto Maple Leafs.
So, what is the plural of leaf?

When is a Verb not a Verb?

If we have never learned an irregular tense or plural, we use the regular rule.
“Fly” has a known irregular form: “flew”
But “fly” as in “fly out” is derived from a noun – “fly ball”
The irregular does not apply in this case
“Sink” works the same way. I “sank” a put (known verb), but “sinked” the dishes (a verb coined from a noun)

And Those Toronto Maple Leafs

For the tree we have a known irregular plural – “leaf”, “leaves”.
But the hockey player is not a tree!
One player is a“maple-leaf” – a coined word.
The association with “leaves” is not there, so we revert to the regular rule – “maple-leafs”
So, a question for you to worry about: Why are the Minnesota ballplayers called “timberwolves”?

Rules and Associations

Pinker: To explain results like these, we need both associations and rules.
Irregular verbs and nouns involve associations - Easily handled by connectionist theories
Regular verbs and nouns: Connectionist theories may be able to mimic the rules
But other rules tell us when to use which. This cannot be handled by connectionist theories
There is no way to learn association weights that can provide the “meta-rules”

Is there any way to reconcile the neural network models of tense learning and the rule-based models?

Should we trust intuitive (subjective) “acceptability” judgments?
Can connectionist networks give rise to an intuitive sense of rules?
Might rules be an epi-phenomenon?

Chapter 9: Section 9.3

Describe the dishabituation paradigm. What did it demonstrate?

Research by Baillargeon: Infants look longer at “surprising” events
Example: mental addition and subtraction
Do infants know the rules of addition?

What is folk physics?

An understanding of how physical objects behave and interact
These principles appear very early in life (probably because of genetic programming)
They can, of course, develop and become refined as the child grows older
Research suggests that such concepts as object permanence appear much earlier than Piaget and others thought

Possible and impossible motion of objects
Do infants know the laws of physics?

Describe Spelke’s four principles of folk physics.

Cohesion: Surfaces belong to the same object if they are in contact
Contact: Only surfaces in contact can move together
Continuity: Objects move on a single track through space and time
Solidity: Objects cannot occupy same point in space-time

Is the dishabituation paradigm a useful and valid methodology?

The only observable behavior is the infant’s looking
Are there other explanations for the looking behavior?

Chapter 9: Section 9.4

Describe how Munakata uses a recurrent network to model the phenomenon of object permanence.

A new kind of network: Recurrent networks use feedback loops from hidden units
These enable the system to generate predictions of what input occurs
Learning is driven by differences between input and the predictions
A kind of hypothesis testing process

Describe the four stages children that go through with respect to the balance beam problem.

Number of weights
Distance if number of weights is equal
Correct rule, provided there is only one difference (will not work with above)
General competence

Describe how McClelland and Jenkins model the stages of reasoning about the balance beam problem, and how they train their network.

Input units are weights and distances
Hidden units represent “understanding”
Output is the judgment
Learning by backpropagation
Demonstrates four stages, but is this an artifact of the way the model is trained?

Outline the different ways in which physical symbol systems and neural networks supporters would view the information processing underlying infants’ physical reasoning.

PSS: “Understanding” consists of learning the rules
Networks: No need to postulate rules – they are merely a way of describing the behavior
But is there an empirical way to choose between them?

Do the models described in Section 9.4 provide a complete explanation for the phenomena that they target?

There is a sense in each case that they are ad hoc explanations
Are they useful? Do they add anything to PSS theories?

Chapter 9: Section 9.5

Describe the question of levels and explain why it is important.

Marr/Stanovich: Computational/Intentional, Algorithmic, Implementation
Physical Symbol Systems: Cleary pitched at algorithmic level
Networks: Implementation level?

Explain Fodor and Pylyshyn’s argument concerning artificial neural networks.

We cannot think of network models at the algorithmic level – there is no structure, no rules
Do networks contain representations?
If so, they are merely implementations of PSS
If not, they cannot be described as algorithmic
Ergo, they are not competitors to PSS

Implementational Use of Connectionist Models

The models can be useful for providing quantitative, testable versions of theories
Example: Working memory
Time-Based Resource-Sharing: Popular theory developed by Barroulliet and others
Oberauer & Lewandowsky (2011): A network implementation of the theory
Their conclusion: “All models are wrong, but some models are useful” (George Box)

Is Fodor and Pylyshyn’s argument convincing.

Conclusion: Rather then either/or, we should probably think of both serving different purposes