Criteria (weighted)
- Functional fit — does it do the job for your use case
- Total cost — including migration and internal ops
- Time to value — realistic weeks to production
- Risk — vendor lock-in, security posture, roadmap stability
- Operational cost — who runs it and at what load
Score each vendor
1-5 per criterion, weighted. Force separate scoring by 2-3 people before comparing — otherwise the loudest opinion wins.
Build-vs-buy line
Include 'build in-house' as a row. Compare on the same criteria. Most buys look worse against a real build estimate than the sales deck suggests.
Reference checks
Talk to two customers off the vendor's list, and one you find yourself. The self-found one is the most honest.
Common pitfalls
- — Weighting features and ignoring operational cost. Where budgets die.
- — Skipping build-vs-buy for tools that feel small. Small tools compound.
- — Only talking to vendor-provided references. Curated.