Over on Debbie Wilson’s blog, there’s a good entry that raises the question of the role of product demonstrations in evaluating technology solutions. Even though Debbie tackles it from the role of analyst, I think there are some lessons and broader discussion points to come out of it. For one, consider the role of references in decisions as much as a demonstration and RFP response. However, the best references are not necessarily the ones provided by a vendor — they often come through your own network. If a vendor provides them, make sure you ask as many open-ended questions as possible to gauge the level of enthusiasm in the response. “Damning with faint praise” should damn a vendor off your shortlist. I often request references from vendors but rely on industry (or channel) connections when possible to do my real homework.
But back to the topic at hand — the value of demonstrations. All too often, unless the analyst or prospect orders up the demonstration himself or herself, requesting their own demo scripts and scenarios, vendors will attempt to pull the shades over the shortcomings in their solutions (or at least provide basic whitewash to areas where functionally, they can’t quite compare to others). In other words, be prescriptive about what you want to see. ..