Wednesday, March 4, 2009

Evaluating enterprise software

Two things have happened at pretty much the same time. The first is an evaluation of some enterprise class software in conjunction with and on behalf of a client. The second, the appearance of this article in my email - from Alex Rosen. http://www.pentaho.com/docs/a_new_business_model_for_bi.pdf

While the problem to be solved for the client is not BI, there are a lot of behaviors in common between the paper and the experience with the vendor at this client.

We had gone through many of the due diligence stages, spoken to various vendors, created shortlists, evaluated Proofs of Concept until we were down to a vendor that we wished to trial more deeply.

There were some key things we wanted to find out, so we devised tests, questionnaires, scheduled calls, read as much documentation as the vendor would let us see, but we really felt that the vendor was being very stingy with information.

Often direct questions would be answered with, "It doesn't work that way" or "no we can't do that". In the vendor's mind this might have meant, "This is not in scope for the pilot." That isn't how it came across.

It took a huge amount of probing before the vendor finally started to be more forthcoming with information - at which point things went a lot more smoothly.

So, since this is a pattern oft repeated, why do many vendors behave this way?

I can't answer for this specific case, but some ideas include:
  • The process is still sales controlled and the sales team wants to control everything.
  • The process is still sales controlled, so it is important to marginalize the people who can't say yes (but who can say no).
  • The process is still sales controlled and the teams aren't used to the more implementation oriented lines of questioning
  • There are opportunities for consulting dollars, so by keeping the prospect in the dark there is more revenue.
  • The vendor believes that the solution takes some getting used to, so is rationing the information to prevent the customer from being overwhelmed.
  • The vendor doesn't have adequate documentation and doesn't wish to expose that fact just yet.
  • The vendor wants to see the possibilities of hard questions coming so that they can prepare for answers.
There may be other reasons that I am not aware of - and the ones I mention are pure speculation. On the current project, I can't say that any of these reasons are correct.

The key takeaway for me is to learn as much as possible about the solution before pilots. Get any FAQ documentation, get the configuration documentation so you can what's possible from the configuration. Ask the hard questions, don't take the answer, "You'll see it in the Pilot."

2 comments:

Richard Veryard said...

Your reasons all sound much too rational and self-interested, but I often suspect something irrational going on. For example, an uneasy sense that the customer's questions are not straightforward, that there is some trick hidden in the question, or that the customer is just tyre-kicking, which results in an irrational and self-damaging lack of engagement with the customer.

I note that there was a point where the vendor became much more forthcoming. To me this suggests that the vendor had started to believe in the possibility that the customer really wanted to buy, which meant that the vendor started to act (for the first time) as if he really wanted to sell.

Chris Bird said...

That's a good set of observations, Richard. Since the buggers were getting paid to do a pilot, it would have been nice for them to have been more forthcoming. It is just terminally frustrating - but it is probably just as hard for the vendor to figure out the client's intentions as it is for us to understand vendor motivation.