Seeing a computer play two humans at Jeopardy! is a lot more entertaining than I thought it’d be. I’d been ignoring most of the hoopla around Watson figuring it for a big, effective PR campaign on IBM’s part. It is that for certain, and good on them for doing it. I’ve been more interested in what practical and “work-place” applications the technology behind Watson has, and I got a little bit of that along with some other interesting tidbits at a Watson event this week at IBM’s Austin campus.
The Technology Used
The Watson system uses UIMA as its principal infrastructure for component interoperability and makes extensive use of the UIMA-AS scale-out capabilities that can exploit modern, highly parallel hardware architectures. UIMA manages all work flow and communication between processes, which are spread across the cluster. Apache Hadoop manages the task of preprocessing Watson’s enormous information sources by deploying UIMA pipelines as Hadoop mappers, running UIMA analytics.
The ASF press release is actually jammed with a lot of “how it works” info.
Additionally, Watson is run on POWER7 machines with Linux, one of IBM’s exotic (but revenue pulling – $1.35B last quarter by TPM’s estimates) platforms. I was wondering why the team chose POWER, and though I didn’t get a chance to ask, once of the IBM’ers I was sitting next to said that the cooling ability of POWER machines meant they could pack more of them into the Watson cluster(s).
Here’s a brief hardware description from an overview whitepaper:
Early implementations of Watson ran on a single processor, which required two hours to answer a single question. The DeepQA computation is embarrassing parallel, however, and so it can be divided into a number of independent parts, each of which can be executed by a separate processor. UIMA-AS, part of Apache UIMA, enables the scale-out of UIMA applications using asynchronous messaging. Watson uses UIMA-AS to scale out across 2,880 POWER7 cores in a cluster of 90 IBM Power® 750 servers. UIMA_AS manages all of the inter-process communication using the open JMS standard. The UIMA-AS deploy- ment on POWER7 enabled Watson to deliver answers in one to six seconds.
Watson harnesses the massive parallel processing performance of its POWER7 processors to execute its thousands of DeepQA tasks simultaneously on individual processor cores. Each of Watson’s 90 clustered IBM Power 750 servers features 32 POWER7 cores running at 3.55 GHz. Running the Linux® operating system, the servers are housed in 10 racks along with associated I/O nodes and communications hubs. The system has a combined total of 16 Terabytes of memory and can operate at over 80 Teraflops (trillions of operations per second).
During Q&A an audience member asked if Watson could do better if it took longer to answer questions. In the game, of course, Watson is trying to answer questions as quickly as possible. The answer was, yes. And, in fact, Watson already does this: it’s actually running two processes to answer a question:
- The first is a quick process that favors speed instead of accuracy. This fast process is used by Watson to see if it should buzz in at all.
- The second is a longer process that favors accuracy and is the process used to actually answer questions.
So, you’d think, at the start of each question, Watson spins up these two processes, handing the real answer off to the one that gets a few more seconds.
There’s a 6 page whitepaper on the POWER7 (and software) angles of Watson over at IBM, tragically, you have to lead-gen your way into it, but it’s worth the typing if you’re interested.
What I find interesting here is the big reliance on open source software for this impressive Big Data application. The innovations are interesting on their own, but from a “how do I apply this to my situation?” perspective, in theory, the fact that it’s open source opens the possibilities of using the underlying technology to a wider set of people, if only because it’s cheaper than proprietary options.
For the IBM Systems & Technology Group (STG, who produces and sells all the hardware IBM has), it’d be gravy: why spend all that money of software when you can spend it on hardware? (To be fair, for sometime now and especially with Software Group [SWG] head-honcho Steve Mills running both STG and SWG, IBM would prefer to collect on both types of -ware.)
It’s part of what John Willis would call “The Cambrian Cloud Computing Explosion.” In my words: there’s an excess of technological innovation at affordable prices (the big difference) out there just waiting for business demand.