Computational Models of Natural Language Acquisition Transcript of a discussion between Karl Kornacker and Steve Pinker, held September 12-16, 1988. *FOR INDIVIDUAL USE ONLY; NOT FOR PUBLICATION OR BROADCAST OVER ANY NETWORK WITHOUT PERMISSION OF THE AUTHORS* Date: Mon, 12 Sep 88 To: steve@psyche.mit.edu (Steve Pinker) From: kornacker-k@osu-20.ircc.ohio-state.edu Subject: computational models of natural language acquisition Dear Steve, When George Stelmach, series editor for Advances in Psychology, unexpectedly invited me to edit a volume for the series, I accepted and proposed the topic "Computational Models of Natural Language Acquisition" (CMNLA). My stated goal at the time was to foster serious consideration of new ways to integrate connectionist and production-system models of learning in order to build more realistic models of language acquisition in children. Your recent e-mail discussions with Harnad, McClelland, Marchman and Bates have helped clarify the need for developing new integrated approaches to CMNLA. Four questions: Do you think it is feasible to produce such a book for publication in 1990? Would you consider being co-editor for this book? Do you agree with Laura Petitto that the communicative behavior of children between the ages of 18 and 22 months reveals a developmental transition from predominantly nonlinguistic to predominantly linguistic means, and that apes communicate by predominantly nonlinguistic means regardless of their age or training? Do you think it is feasible to determine the extent to which a particular CMNLA can be used to model: a) the improved nonlinguistic communication that occurs in apes; b) the improved linguistic communication that normally occurs in humans older than 22 months; c) the developmental transition that normally occurs in humans between the ages of 18 and 22 months? Thanks in advance for your reply. Best Regards, Karl Kornacker Division of Sensory Biophysics The Ohio State University ---------------- Date: Mon, 12 Sep 88 15:05:30 edt From: Steve Pinker Site: MIT Center for Cognitive Science To: kornacker-k@osu-20.ircc.ohio-state.edu Subject: CMNLA's Dear Prof. Kornacker, Thanks for your inquiry. Let me try to answer the questions you posed. 1. I'm not sure whether it would be feasible to have a book on "Computational Models of Natural Language Acquisition" ready for publication by 1990. It would depend on how many new projects are far enough along right now that their authors would be able to write them up within a year or less, which is probably what you'd require for an 1990 publication date. I doubt it, but I don't know everything that's out there. More generally, there is the question of quality. I think that computer models of certain aspects of language acquisition can be extremely valuable. For example, in my 1979 review (S. Pinker, Cognition, Vol. 7, "Formal models of language learning") I argued that they could make certain theoretical issues in language acquisition more precise. Prince and I also obviously found a lot to discuss about the Rumelhart-McClelland model. But unfortunately, the majority of CMNLA's that I have read about are a complete waste of time. There is by now a vast scientific literature on virtually every aspect of language and its development in children. But somehow the culture of computer simulation very often discourages the kind of serious scholarship that would review these discoveries and find out exactly where simulation would solve outstanding problems. Instead the designer typically sits down and immediately writes a kludge-filled toy program acquiring a random fragment of language, not very well, and usually without the faintest resemblance to children. The author then proceeds to offer his opinions, usually at length and with vehemence, about Chomsky, Piaget, innateness, learning, etc. Not everyone works this way (see, e.g. J. Anderson, Berwick, Rum-McC) but, sad to say, most people do. A volume that contained a representative sample of current CMNLA's would simply be of low scientific quality. (Drew McDermott's article "Artificial Intelligence meets Natural Stupidity", though not addressed to CMNLA's per se, summarizes the kinds of problems I have in mind fairly well. Dresher & Hornstein's article in Cognition in 1978 makes some points that also apply to CMNLA's, though they also make arguments that I disagree with). In general, I think that computer simulations should function as a tool that would combine with linguistic and psycholinguistic research to help understand circumscribed subtopics in the study of acquisition. We know too much about how complex language is to profit from unmotivated simulations of carried out in isolation as an end in themselves. The quality of your volume would, I think, depend on how many scholarly, integrated projects you could find at present. 2. Thank you for the invitation to be Co-Editor. Unfortunately I am afraid I couldn't commit myself to the project, simply because of my other writing and editing responsibilities through 1990. I would be happy to consult with you about it and to help in any other way I can. 3. I find Laura Petitto's evidence to be very interesting and convincing, though of course the issue is an empirical one and future data could change the picture. 4. I am not sure whether computer simulation could shed light on the dissociation between linguistic and nonlinguistic communication in apes vs. children, or in pre-language vs. post-language children. We may be looking at two separate systems, one of them put into place as brain development in humans (but not apes) reaches some critical point during the second year. If so, nothing less than a epigenetic-neural model would should light on the transition. At the level of software-modeling of the mental processes, you would need an entire model of nonlinguistic communication (possibly equivalent to a model of all of cognition), and models of language abilities at several maturational stages, and a supermodule (for humans) that would activate the language modules at predefined clock ticks. My guess is that smaller-scale models of the acquisition of well-defined subdomains of language, integrated with linguistic and developmental discoveries, would be the most fruitful approach. I wish you the best of luck with your project. Please let me know if I can be of further assistance. Steve Pinker E10-018 Dept. of Brain and Cognitive Sciences MIT ======================= Date: Tue, 13 Sep 88 To: steve@psyche.mit.edu (Steve Pinker) From: kornacker-k@osu-20.ircc.ohio-state.edu Subject: CMNLA Dear Prof. Pinker, Thank you for offering to consult with me on the CMNLA book. I will keep you informed of any significant developments; hopefully, I will be able to gather enough work of high scientific merit to warrant the production of a book. Here is a more pointed version of the last question I posed to you: If apes are incapable of true linguistic communication, then in what sense are apes capable of "language acquisition"? (It seems to me that while an ape may acquire a repertoire of communicative gestures, its repertoire remains devoid of any lexicon or grammar, even though the ape may become capable of simulating correct linguistic expressions in ASL.) Similarly, when a CMNLA is alleged to acquire language within a circumscribed domain, how important is it to show that the CMNLA actually acquires a capability for true linguistic communication within the corresponding circumscribed domain? I greatly appreciate your constructive responses to these questions. Best Regards, Karl Kornacker 169 Westwood Rd. Columbus OH 43214 (614) 263-7493 ----------- Date: Tue, 13 Sep 88 10:32:50 edt From: Steve Pinker Site: MIT Center for Cognitive Science To: KORNACKER-K@OSU-20.IRCC.OHIO-STATE.EDU, Subject: Re: CMNLA Dear Prof. Kornacker, To answer your questions briefly: 1. If apes are incapable of true linguistic communication, then in what sense are apes capable of "language acquisition"? Probably they are not capable of "language acquisition", if we are talking about the kind of thing human children do. By the way, the apes do *not* become capable of simulating correct linguistic expressions in ASL. ASL has a fairly complex grammar and thousands of signs; all apes do is approximate 25-100 signs, depending on how you count. 2. Similarly, when a CMNLA is alleged to acquire language within a circumscribed domain, how important is it to show that the CMNLA actually acquires a capability for true linguistic communication within the corresponding circumscribed domain? Communication per se may not be important; you need many modules of grammar, plus quite a lot else (cognition, social awareness, etc.) to communicate, whereas it might be reasonable to simulate the acquisition of one mini-module of grammar which by itself cannot communicate a thing. But I assume your question is more general; i.e. how important is it to show that a simulation really grasps the organization of grammar, as opposed to being able to mimic some utterances. If that is the question, then the answer is "nothing is more important, for a scientist interested in the process of language acquisition". I hope I have understood your questions properly. Regards, Steve Pinker ================== Date: Wed, 14 Sep 88 To: steve@psyche.mit.edu (Steve Pinker) From: kornacker-k@osu-20.ircc.ohio-state.edu Subject: last questions (for a while) Dear Prof. Pinker, You do indeed understand my questions properly. I would state your position regarding CMNLA as follows: A simulation may be regarded as an explanatory model for the way in which a child learns the structure of a mini-module of grammar only in so far as the simulation can account for the observed statistical patterns of generalization errors that the child actually makes. Is the above statement accurate? My understanding of the indexical error that some two-year old children make, confusing "I" with "YOU", is that the error is distinctly: a) linguistic; b) human; c) elementary; d) an error of generalization; e) an error of communication. Is the above characterization substantially correct, as far as the evidence goes? Do you think that it is important to consider how a CMNLA could learn a mini-module of linguistic communication and account for a limited family of elementary linguistic communication errors that are related to the indexical error mentioned above? I intend not to ask you any more questions for a while. Thanks again for your generous help. Best Regards, Karl -------------- Date: Wed, 14 Sep 88 11:38:35 edt From: Steve Pinker Site: MIT Center for Cognitive Science To: KORNACKER-K@OSU-20.IRCC.OHIO-STATE.EDU Subject: questions Dear Karl, You asked whether I agree with the following: A simulation may be regarded as an explanatory model for the way in which a child learns the structure of a mini-module of grammar only in so far as the simulation can account for the observed statistical patterns of generalization errors that the child actually makes. I would make it a bit broader -- not only children's statistical patterns of generalization, but also order of acquisition of different structures, qualitative kinds of errors that children and the model do or don't make, and successful attainment of adult abilities. In other words I think that cmnla's should be treated like any scientific theory, and should be tested against as wide an empirical database as possible. Of course no model will account for all such data, but the more data accounted for, the better the model is as a theory. (I exclude "pure AI" models, i.e. programs designed to acquire language for practical purposes, with no attempt to model the human child). I think the characterization of the I/you errors you suggest seems pretty accurate, though I don't think it is too fruitful for anyone to debate whether they are truly "linguistic", "human", etc. (these labels are a bit vague and it's more important to understand as fully as possible what causes the errors, rather than what label to apply to them). As for whether computer simulation could shed light on them -- I guess I'm open. I don't really see myself how one would do a simulation that would yield surprising results or explanations for the indexical errors, but I haven't really thought about it much. If someone could do that, it would be great. I should mention that another possible benefit of a computer simulation of the acquisition of a language module might be to raise important empirical questions, currently unanswered, that no one would have thought to ask otherwise. Probably a cmnla addressed to indexical errors should aim at some well- defined subsystem within which the errors occur, such as pronouns, or perhaps just deictic pronouns, as opposed to being directed exactly at the errors themselves. --Steve ============== Date: Thu, 15 Sep 88 To: steve@psyche.mit.edu (Steve Pinker) From: kornacker-k@osu-20.ircc.ohio-state.edu Subject: working hypothesis on connectionist cmnla's Dear Steve, I agree that the labels "human" and "linguistic" might trigger pointless debate, and I am open to suggestions for more precise alternatives. As applied to the I/you indexical error, I intend "human" to mean "unique to the human species" and "linguistic" to mean "unique to linguistic communication". My motivation for using these lables (or appropriate alternatives) is linked to my current working hypothesis -- Connectionist cmnla's probably cannot explain the temporary occurrence of the I/you indexical error in two-year old children because: a) the error is distinctly human (does not occur while apes are learning to communicate by means of gestures taken from ASL), and therefore is probably unexplainable by a *universal* connectionist learning mechanism that makes no distinction between apes and humans; b) the error is distinctly linguistic (does not occur in deaf children until they make the normal developmental transition from nonlinguistic to linguistic communication), and therefore is probably unexplainable by a *universal* connectionist learning mechanism that makes no distinction between linguistic and nonlinguistic learning; c) the error is abstract (does not depend on any specific properties of the you-person other than "being a person who is not one's self"), and therefore the child's process of learning the correct use of I/you is probably unexplainable by a connectionist cmnla that is limited to content-addressable memory and similarity-based generalization. Argument c) seems closely related to the points you made in the recent Great Debate regarding the learning of *regular* verb forms, since the property of being a regular verb is abstract (does not depend on any specific properties of the regular verb other than "being a verb that is not irregular".) I hope to find sound productive ways of using arguments a) and b) to help guide the development of more realistic cmnla's. As always, your comments are most welcome. Best Regards, -Karl ------------- Date: Thu, 15 Sep 88 09:32:03 edt From: Steve Pinker Site: MIT Center for Cognitive Science To: KORNACKER-K@OSU-20.IRCC.OHIO-STATE.EDU, osu-20.ohio-state:steve@edu.mit.edu Subject: Re: working hypothesis on connectionist cmnla's That clarification was very helpful. I see now how you are applying the argument. The line of reasoning is very interesting and cogent, and I think it is well worth developing along the lines you sugguest. Please keep in touch as you work on these issues. Best, Steve ============== Date: Fri, 15 Sep 88 To: steve@psyche.mit.edu (Steve Pinker) From: kornacker-k@osu-20.ircc.ohio-state.edu Subject: request to archive our discussion Steve, I would like to put a complete transcript of our recent discussion into the connectionists archive. I think the material would probably be interesting and useful to several researchers in the network, and it might also help me to contact appropriate authors for my cmnla book. How do you feel about this idea? -Karl --------------- Date: Fri, 16 Sep 88 10:47:48 edt From: Steve Pinker Site: MIT Center for Cognitive Science To: KORNACKER-K@OSU-20.IRCC.OHIO-STATE.EDU Subject: archives Dear Karl, I have no objection to your archiving the exchange. That is on the assumption that the exchange is not actually broadcast on the connectionists-network; at this point I don't have time to respond to the inevitable commentaries that it would provoke. Also, could you add a header asking people not to cite or quote from it without permisssion? I wrote my half as an informal, private set of letters, with no attempt to state everything perfectly precisely. Thanks very much. --Steve ============== Reference added by KK for archive: Petitto, L., (1988). "Language" in the Prelinguistic Child. In -- The Development of Language and Language Researchers. Essays in Honor of Roger Brown. F. S. Kessel (ed.). Lawrence Erlbaum Associates (pub.). -------