The hidden danger that kills search products

Having seen search solutions birthed (and die) since 2012, I have to warn you: there be dragons! What was a nice niche (good site search) with a meaty, but defined market (big e-commerce, job search, etc companies), now has exploded with AI and RAG.

I’m often asked how to think about the “vector DB” and adjacent space by investors, product companies, and customers. There’s a range of companies: some provide components targeting developers - just the search engine / vector DB. Others sell the full solution of “good search” for your RAG, etc app. Just give us your data and set a few things up, you get great, relevant search results for your product.

Or so it’s promised.

I want to focus on this latter area - the temptation to “solve search” with your product, only to end up humbled by the amount of custom, per-customer work you’ll need to take on.

Solution providers face a major challenge - the complete invisibility and subjectivity of “relevant search”.

Customers don’t see the algorithmic improvements. There’s no button you added to the users screen. No obvious difference unless you know what to look for. Now RAG buries search even deeper under a chat interface, multiplying the challenge.

In my first search project, my company built a beautiful UI, only to put the app in front of my customer to realize that the accuracy of the results was extremely subpar.

Through many rounds of heroics, we made some progress. The customer assumed it was “obvious” what search results should come back. They assumed it should be easy. Yet their “obvious” was actually very specific to their use case. After unexpected delays and costs - we had to nail down how search worked very concretly and prove to the customer search responded with the expected behavior. We left the experience rather humbled.

Search algorithm expectations happen unconsciously. Customers know it when they see it. They’ll be surprised when their good search expectations are not so simple after all.

Therein lies huge risk for large search solution companies trying to scale a promise of “good search made simple”.

If you sell customers a full solution, they will expect their subjective, unconscious, and domain specific definition of success. Customers assume you read their mind, projecting their definition of “good search” onto your product.

Multiply that by every stakeholder in that company. Multiply that by every domain—medical search is completely dissimilar from e-commerce. Even in e-commerce- book search differs from fashion search. Passive discovery for consumers differs from B2B. Book e-commerce for librarians differs from search for sci-fi nerds. And on and on.

Founders assume that after solving search 1-2 times at big companies, they have found “the solution” to search and bundle it in their product. Yet they take a promise of “easy search made simple” to actual customers and get embarrassed (like I once was) by that “I know it when I see it” subjective level of success of their stakeholder.

Indeed in my early consulting gig - objectively we could prove we delivered the UI. All the buttons and widgets were there. Proving that search was “good” is a whole other matter.

So what happens is to get the big customers and fulfill the promise of great search, companies end up building out large consulting arms to tailor their product. They rush to find a way to build in these costs. They can - in the end - look more like a consulting company than a product company.

That’s not to dissuade anyone. I’m a big believe in “not because it’s easy, but because we thought it would be easy”. Hubris drives progress after all!

As a product company, you need to seed these expectations very early with your customers. You need to build in search relevance measurement of some sort to what success looks like. You need to build in this level of ongoing consulting and sales engineering to your costs.

When selling a solution, you don’t just sell the product - but a productized process to understand the users definition of success, and customize your product to deliver on it. You need to be a trusted partner, advisor, and consultant. You don’t just load your data and leave.

It helps to know the niches you’re targeting and build/market to that. Being very careful not to oversell. There’s long been an appreciation of e-commerce search as a domain for the solution companies. Lately product companies focus on RAG and target solutions there. Yet we all need to appreciate that search, RAG, etc solution require as much customization as buulding traditional apps. There is no silver bullet, only hard work.

Frustratingly, you’re up against folks who will promise magic. They may close more sales but not survive the onslaught of subjective customer expectations at implemention time. Ensure your company’s definition of success involves ongoing, successful relationships with customers - involving constantly working with stakeholders evolving definition of success.

If you’re NOT a solution provider, and you think, “let’s build a solution so we can sell to enterprise customers” be very very hesitant. You will be surprised by all the hidden costs and dangers I mention above. Implementing a product is at least a 3x the cost and that doesn’t count all the consulting, sales, procurement, legal you’ll experience!

Also if you’re not a solution provider - you can end up in an uncanny valley between fully comitting to development lego-pieces and selling a full solution. These are two completely different companies in my experience. The lego-pieces company wants the best lego piece at the lowest cost. The winners don’t get ahead of their skis and focus on the very hard task of simple-made-easy for this extremely narrow lego piece. Think of the early days of Stripe: a boring function made very easy for developers. The solution providers, as I’ve described, need to think about a very deep relationship and complex sales / consulting cycle. Doing BOTH in my opinion is near impossible - now you need both these competencies in one company with limited resources. Fully commit to one direction or another, or else you’ll waste resources trying to be all things to all people.

As an investor in one of these companies: it’s not enough to know sales are closed, but more than other domains, when the rubber hits the road with customers - are they getting their definition of “success” delivered? Is the company walking customers through a careful process of defining success, measuring it, and keeping happy customers?

Anyway, I hope this lays out the landscape for you. It’s an exciting time to be in the space, and I’m excited to learn about your product.