First Experience with WolframAlpha

Yesterday I finally got early access to WolframAlpha. I understand it will be coming out publicly in the coming week(s). It is being positioned as '... Making the world's knowledge computable ... '. I have also seen two demonstrations by Steve Wolfram and listened in on question sessions.

The interface is very simple. You type in English (only for now) questions. There are also a large selection of good examples you can experience. They are also careful to point out that ' ... Wolfram|Alpha is an ambitious long-term project to make all systematic knowledge immediately computable to anyone ... '.

Have seen much gushing over the emergence of this package. I applaud the effort, it is a great idea. I also point out that the package is in a very early phase of what they correctly say is a long term effort.

I have tested it so far in the domains of food, math, physics, chemistry and economics. As might be expected, it does best in math.

First of all, in all of my experiments I have yet to see it do any real natural language. Language is hard, often ambiguous, so I understand that they have not solved the natural language problem. Yet even relatively simple fairly unambiguous statements like 'Plot US Deficit by Year from 1990 to 2000' cannot be interpreted. Lots of similar examples.

It understands something like: Atomic weight of O to be a physics question and answers it. It does not understand: Atomic weight of O plus the atomic weight of H, which is a (very) simple language/arithmetic addition to the physics. Adding more language does not help. Forget subtlety. When confronted with this kind of behavior you tend to simplify the language, which also does not help. It does not seem to 'understand' the language at all.

Note the comparison to search. Google understands 'How many quarts in a peck' (as does WA) but not because it has solved the deeper language problem, but because it does simple pattern recognition. We have become accustomed to the inaccuracy of search results and adjust our expectations accordingly. WA is different because the interpretation of the language and the search will be used to drive numerical computation. We expect a search to give at least some reasonable results, and we can sort out any ambiguities. WA may often provide nothing at all.

Also, haveing been involved with this kinds of effort before, this is not something you can simply patch. You either have some reasonable underlying language understanding model, or you don't.

I repeat, I applaud this effort .. it is a great idea. The examples shown are intriguing, excellent and well thought out. They show the ultimate scope of knowledge being addressed. I am afraid though that what is being being delivered as a 'long term effort' is very premature. I understand the pressure to deliver this kind of thing once revealed. Worried about the reaction by the general public to discount it as a competitor to search engines. I continue to experiment with it.

Update: The NYT on WolframAlpha.
-
-