The scope, scale and variety of enterprise software review sites continues to accelerate, each promising a true glimpse into a buyer’s experience. These sites include Credii, G2Crowd, Google Apps Marketplace, Microsoft’s Pinpoint, LinkedIn, Trustradius, Salesforce AppExchange, and many others. Each of these sites claims to have safeguards in place to ensure authenticity of reviews. A few including G2Crowd are positioning themselves as Yelp for Software.
That’s a great vision for bringing greater clarity and simplicity to the process of evaluating and buying enterprise software. The trouble is buying enterprise applications is inherently more challenging and multifaceted. One often has to anticipate and plan for future changes in business models now. That’s far beyond the simplicity of Yelp and other consumer-based reviews.
The hard reality is that enterprise software selection decisions have a direct impact on future financial performance. Choosing the right enterprise software can strengthen customer relationships and streamline production. From having been involved in many software selection projects over the last twenty years, one resoundingly clear lesson I’ve learned is: when you marry the application you marry the vendor. That’s why putting blind trust into these review sites is a bad idea. These sites need greater transparency and a more proven track record to earn and keep trust.
Fake It Till You Make It And The Power of Fiverr
Professors Michael Luca of the Harvard Business School and Georgios Zervasy of Boston University published the study Fake It Till You Make It: Reputation, Competition, and Yelp Review Fraud, reporting how pervasive fake reviews are on Yelp, and what motivates businesses to pay for them. The study’s authors found businesses with fewer reviews are more likely to engage in positive review fraud. Enterprise software companies lacking real customer reviews have no doubt considered the same.
Any business can purchase a positive review for their Yelp profile for just $5 and 1 day turn-around on Fiverr.com. Any business can also pay to get positive Google and Yelp reviews on Freelance.com as well.
Stories of companies paying for positive reviews are now commonplace and too numerous to mention in this post. Here is a sample of them: Samsung Fake Reviews About HTC Smartphones ‘Written By Paid Students and Fake online reviews crackdown in New York sees 19 companies fined.
It’s just a matter of time until a start-up creates Fiverr Enterprise Edition.
Time For A Reality Check
Given that SAP is the global leader in ERP, one would think a decent review site should have a robust set of critiques of its software. So let’s see how G2Crowd did on the task of determining if an SAP ERP system is good for your business or not.
- G2Crowd has just ten reviews for SAP ERP, and just 3 are verified. No way of knowing if the other reviewers are SAP employees or competitors.
- Reading each of these entries sounds like a years-old industry analyst report with the usual and universally known weaknesses of SAP: poor user interface, complex approach to licensing, expensive costs and shortage of SAP expertise. SAP Reviews are here on G2Crowd.
- G2Crowd claims that its approach is disruptive, but I’m not sure it is disrupting anything, just underscoring how critical it is to make an effort to speak with real customers. If you want the truth, speak to actual customers even if your vendor doesn’t want you to. That’s where independent analyst firms can help, they can get you to actual customers running the apps you are considering.
More Oversight And Transparency Needed
Trust is the currency these sites trade with and they need to figure out how to make this easy to do yet powerful to serve buyers. Enterprise review sites need to do the following to make sure they don’t get “Yelped” by fake reviews:
- Require proof that the enterprise application was indeed purchased and is running in the company the contributors are from, and they are registered users on the system.
- Create and continually improve algorithms that will catch fake or highly suspect reviews.
- As many of these sites, including G2Crowd, use a contests to motivate people to provide more reviews, these sites should publish what they gave away to top contributors and also provide a ranking of content authenticity using their algorithms. This would ensure greater oversight and transparency.
- Members of the senior management for these review sites need to stay impartial and not involved with software vendors to alleviate potential conflict of interest. This is where G2Crowd becomes a fascinating company to watch. Godard Abel, an acquaintance I’ve known and respected for over a decade, turned Big Machines into an exceptional company and eventually sold it to Oracle last year. He is the co-founder of G2Crowd and last week invested $5 million in Steelbrick and also became its CEO. This is like the head of Consumer Reports becoming the CEO of a vacuum cleaner company regularly reviewed by Consumer Reports. Steelbrick and the entire CPQ market should be excluded from G2Crowd to preserve impartiality. This also opens up a dilemma for Steelbrick’s future analyst relations and influence strategy. Gartner is one of the most relied-on industry analyst firms guiding enterprises’ CPQ selection decisions today. How Mr. Abel and Steelbrick explain G2Crowd’s critical words for Gartner while seeking their recommendation into deals will be awkward.
Bottom line: Instead of completely trusting enterprise review sites, go visit the companies using the enterprise applications you are considering. If you’re looking at an ERP system, go walk the plant floor with the operations, IT and senior management teams to see what’s working or not. You’re not marrying the app, you’re marrying the vendor, so get some face-time in with real customers too.
[…] Related: Why Completely Trusting Enterprise Software Review Sites Is A Bad Idea […]
Louis – since this is a cross-post from your article in Forbes, I thought it was appropriate to re-post my comment.
I’m CEO of TrustRadius – one of the review sites you mention. I am glad that you’re bringing attention to this important issue. I agree with you that trust is a critical issue. It’s one of our core values at TrustRadius, and a principal reason why we chose our brand name. I’d like to describe some of the approaches we use to maximize trustworthiness of our community:
1. Beyond LinkedIn Authentication, we stage and check every review
We are the only site that I’m aware of that stages every review before its published and has a human being review its legitimacy. Beyond LinkedIn authentication, we verify that the review is not written by a competitor or the vendor itself. The human check also assesses whether the review seems balanced, legitimate and from an actual user. We regularly reject 3-5% of inbound reviews, either because we suspect fraud, or because the review contains insufficient details to provide value to a reader.
2. We qualify reviewers
We reject reviews from people who say that they have only seen a demo etc, or have not used the product recently. Our cut off is typically 6 months.
3. We declare important affiliations
We add a summary at the top of every review which includes key factors, like if the reviewer is consultant who generates fees from implementing the product.
4. We are the most vendor-independent site
87% of our reviews are based upon our own sourcing efforts – where we’ve identified people with specific product expertise, or they’ve profiled themselves previously in our community when using it.
5. Our review totals are just that – real reviews, not just ratings
Unlike G2Crowd, we do not allow people to rate products without providing a full review as that’s too easy to fake/game.
6. We invite vendor comments
We encourage vendors to comment on their own reviews so long as they do so appropriately, being respectful to the reviewer etc. They are also an extra safety check for legitimacy.
7. We have no vendor affiliations
I concur with you that it’s problematic for any review site to have an affiliation with a software company. As I raised capital for TrustRadius last year, I had the opportunity to take capital from some prominent software company CEOs and I did not do so.
8. We are focused on collecting in-depth, high quality insights
Our average review is 500 words, and the average reviewer spends 18 minutes writing their review.
9. Our reviewers can update their review at any time
Sentiment often changes. Our reviewers can update or augment their review at any time. We display the previous ratings and comments, so our audience can understand how opinion has changed.
Since our launch in May, we’ve sourced ~3000 in-depth reviews. In that time, only two reviews have been published that were later found to be illegitimate.
Here is an article that I previously wrote on this topic:
http://www.shoretelsky.com/modern-company/blog/what-can-b2b-learn-from-tripadvisor/
How Reviews Fit Into the Buyer’s Journey
Reviews are a wonderful additional source of information, but not a substitute for vendor engagement. They help buyers identify good choices, narrow to a short list, and assess their candidates. It helps the buyer ask smarter questions of the vendor, and make a decision faster and with more confidence. We also encourage our users to connect with reviewers to talk live to them. I will however say that an increasing number of buyers are looking to do independent online or back-channel diligence before they engage vendors. We’ve found that 90% of the people are on our site are in an active purchase cycle and 42% of them have not yet engaged a vendor.
We have not covered the ERP space, but for a representative set of coverage for a large enterprise oriented solution, please take a look at our coverage of Workday or Eloqua:
http://www.trustradius.com/products/workday
http://www.trustradius.com/products/eloqua
Vinay Bhagat
CEO, TrustRadius
@vinaybhagat
Louis, thanks for the post and for stirring up a good discussion around enterprise review sites. I would like to respond to points you raised in your post:
1) At G2 Crowd, we agree that preventing fraud and providing transparency are key. Thus from day 1 we have required users to sign up with their LinkedIn profile which allows us to validate identities and filter obvious fraud such as reviewing one’s own or a competitors products. Furthermore software buyers can drill down into reviews and see the profiles of the reviewers and to filter reviews by their LinkedIn connections. Our users have shared with us that they will most likely trust the opinions of people they already know, and so we have made it as easy as possible for them to find and leverage their social connections. In addition reviewers can validate their reviews by submitting screen shots of them using the application and submitting for additional screening by our team. The whole model is transparent to users of our site, who can for themselves decide which content to trust. Furthermore our user crowd helps to identify the best content by grading other reviews.
2) You rightly point out that some categories on our site such as ERP are still lacking in review content which is fair. We launched our G2 Crowd Beta 12 months ago and have initially focused on CRM and marketing software categories where we have also published our first Grids. Many software buyers have already made successful software selections based on the content in the CRM (http://www.g2crowd.com/categories/crm) and marketing categories (http://www.g2crowd.com/categories/marketing-automation) featured on our home page for which we now have a critical mass of content.
3) Unlike the traditional analyst model, all rankings on our site are driven algorithmically by applying our ratings engine to data we gather from user reviews, social sources (e.g. LinkedIn, Twitter, Klout) and online sources (e.g. Google, Alexa). Thus they are not influenced by my or our own staff’s opinions. All the data is transparent for any product rated on our site and SteelBrick’s positive reviews from real users with their LinkedIn profiles are a big part of what gave me the confidence to make a significant investment.
Look forward to continuing the dialogue as this enterprise review market evolves.