As Craig Silverstein was talking yesterday, I thought of Marvin Minsky the father and grand promoter of Artificial Intelligence at MIT. Not because Google uses very smart AI, but because they don’t do much of it at all.
Folks on the East Coast (overgeneralizing for effect) grabbed the Minsky challenge to create a machine intelligence that mimics and surpasses human intelligence. Afterall, machines can operate, doing certain functions, much more quickly and accurately than humans. Minsky-ites were also influenced by MIT linguists like Chomsky who pushed that concepts of deep structure that could be handled, if properly understood, by machines.
Salton‘s great work and insights at Cornell while not tied to AI seemed to promise that machine understanding of keyword might make for successful searches. But the real magic in SMART (Salton’s Magical Retriever of Text) was and is relevance feedback in which human evaluation of results, whether explicit or implicit, increased both the precision and recall of a search.
The more intelligently that the system could learn from the results noted as relevant, the better the subsequent result sets. Aggregating group results gave even better “Good” results and seemed magic (at the time) compared to simple keyword searches.
Larry Page’s insight in constructing PageRank was to see that goodness could also be indicated by the number of links related to a certain document. Again capturing the human intelligence embedded in the act of linking. Not just human intelligence but social intelligence. What we call knowledge capital or in a larger sense social capital.
Successful searches moved from less modeling of the brain, less artificial intelligence, and more and more social intelligence and what we now see in Web 2.0 as social networking. Not that computing and engineering didn’t matter, but that the machine component is most effective as a magnifier and enhancer and connector of the social intelligence supplied by humans.
Will AI ever be successful? Silverstein says “My guess is (it will be) about 300 years until computers are as good as, say, your local reference library in doing search.”
NOTE: I have not yet read John Battelle‘s The Search