Failing at an Elasticsearch 'full' phrase match

Elasticsearch doesn't have a straight-forward way to match the 'full' field (all the tokens as a phrase).

RRF is Not Enough

Reciprocal Rank Fusion, while a useful tool, doesnt magically make hybrid search relevant

Real life NDCG notebook

A notebook showing the real decisions computing search evaluation stats

The hidden danger that kills search products

The lack of objective definition of good search creates huge hazards when creating search, RAG, AI solutions

Stop avoiding conflict on your teams

Avoiding conflict is the death knell of organizations that leads to a lack of progress and careers that implode.

Staff engineers exist in a system of patronage

In reality, staff engineers aren't about 'company wide' impact but a system of patronage where managers reward behaviors they value

Generative AI Augmented Retrieval - GAR presi

My GAR slides from Systematically Improving RAG Applications Sept 2024 course

Your company needs Junior devs

Junior engineers are foundational to whether a team can collaborate and innovate

Search query analysis minus the noise

Finding search queries to improve is harder than you think. Here's one statistical procedure for deciding whether a query really has a problem -- or if its just noise.

I made a worse search engine than Elasticsearch

Integrating my BM25 pandas search library, SearchArray, into BEIR, in order to embarass myself in public.

Will AI Chickens come home to roost?

With normal/boring stock returns - have GOOG, MSFT, etc,run out of AI+layoff cards to play?

What AI Engineers Should Know about Search

All that lexical search context you need to build that RAG app

Planning of E-Commerce Relevance Work - MICES 2024

MICES (Mix Camp E-Commerce) talk about planning e-commerce search relevance work with fast prototypes

Reddit Learning to Rank - project retro

Berlin Buzzwords + Haystack talks about Reddit's Learning to Rank journey

Flavors of NDCG - normalized to what!?

Every team chooses different types of NDCG, choosing your ideal is perhaps the most consequential decision

Just code dumb shit to impress your friends

Groundbreaking and courageous software ideas start by first impressing 3 good friends

Dont have F-You money? Build an F-You Network.

Be of service to others and true to your craft to build a great network

100x faster sorted array intersections

Implementing an exponential search in Cython to speed up position intersections in SearchArray phrase search

SearchArray Results Gathering Performance

The actual bottlenecks are the search results we gather along the way

Adding position-aware search to SearchArray

As Yoda would Say - A joke it is not to add slop to a search system.

The other hard retrieval problems

We need more than dense embeddings in our 'vector' search

Are we at peak vector database?

Seriously - why do we need all these vector databases? Do we need dozens of them?

A Roaringish phrase search algorithm

How phrase search works in search array by intersecting roaring-like numpy arrays.

SearchArray: Making search relevance not so special

Make traditional text search a core part of the Python data stack

View VisualVM Java profiler output as a flamegraph

Convert VisualVM's profiler output to a format suitable for a flamegraph

Fighting undead documentation

Software documentation that doesn't suck needs to exist with the living

NDCG is Overrated (talk at Berlin Search Technology Meetup)

Slides from Berlin Search Tech. Meetup describing an alternative way beyond Judgments and NDCG to think about search offline evaluation.

Take calls. Help people :).

Helping random people, for free, can be one of the best things you can do for your career.

It belongs in a foundation

Why do we rely on such a fractured, vendor-dominated database layer in our supply chain? Why aren't we more worried?

Vector Search The Hard Way (talk at Chicago Search Meetup)

Slides from Chicago Search Meetup, discussing real-world tradeoffs in vector search beyond just the benchmarks

LSH in Numpy C for some fun but no profit

Implementing random projections based LSH as a C Numpy function

A pure python LSH nearest neighbors implementation

Badly implementing locality-sensitive hashing as a vector search solution... for science, edification, 💩, and giggles.

The wrong feedback loop

Software engineering is about designing the right feedback loop(s) with limitted resources.

One big reason search teams fail

Search orgs fail because teams get stuck in functional silos rather than empowering their peers

Stay a Beginner

Visibly failing, learning, and discovering first principles is how you have real influence on a field

Hyperfocus vs perspective as a staff engineer

Navigating between hyperfocus/executing vs perspective/information-gathering mentals states is f*cking hard.

Search relevance for understaffed teams

Where to get started, next steps to take, how to evolve search from 0 to 1.

When feedback is not a gift

Feedback is the lifeblood of getting better, but be careful who you accept feedback from.

Switch repos with "git cd"

Idiot proof "git cd" command to cd to repos in your project dir with fuzzy matching and tab completion.

NDCG is overrated

You can get started improving search relevance without labels and judgments. Which is an imperfect model anyway.

Recover relevance ranking from weak labels

In this post, I have very weak, uncertain labels of relevance / not. However, in aggregate, they may be able to help us make strong determination on the importance of ranking signals.

Reconstructing a cosine similarity

If you know u and v's dot products to A1...An can you reconstruct u.v?

Estimate dot product through two shared references

Taking things up exactly one notch, from one shared reference, to two to estimate a dot product.

Estimate dot product through a shared reference

Given a reference vector `A`, where we know `u.A` and `v.A` what can we say about `u.v`?

How likely is a given dot product?

Finding the probability of a dot product between two vectors lets us quantify how much information is in cosine similarity.

Vector Search for the Uninitiated

What is vector search and why all the sudden are we talking about it?

Orthogonality expected at higher dimensions

Ninety degrees isn't particularly special in 2D, but 3D and beyond, it's the expected angle between two unit vectors.

What ChatGPT Says About The Web

ChatGPT unlocks information from the Web, and away from sites that abuse their users attention with spam and writing that targets Google, not humans.

Make a search engine in ChatGPT

Index some documents, provide some queries, ChatGPT will tell you the most relevant documents for those queries

Meet Fred, a person living inside ChatGPT

Fred works in marketing in New York City and enjoys running. He lost his job at the beginning of the pandemic.

The Importance of Naive Solutions

With algorithm development, naive solutions provide a crucial reference implementation for your testing.

Idiot proof git

Aliases, etc that have made rebase-based workflows in Git much less advanced feeling.

We always work with a broken definition of the problem

Experiments are to search relevance correctness as unit tests are to code correctness. By definition they're a broken but nescesarry defition of the problem we need to get started.

Using Elasticsearch from Google Colab with Bonsai

No need for local setup to play with Elasticsearch from a Jupyter notebook - just use Bonsai + Colab!

What is Presentation Bias in search?

Let's explore this key bias in search systems towards the old algorithm and how to overcome it!

Reconstructing relevance judgments - two scenarios

Analyzing the plausibility of guessing relevance judgments from runs in the VMWare Zero Shot Kaggle Competition

Deriving Search Relevance Judgments from an A/B Test

Can we simulate the likely search relevance labels just from knowing which results shifted and the outcome of an A/B test?

Start with Who, not Why

Work with amazing people you love collaborating with, the rest (mission, purpose, etc) falls out from that.

LambdaMART in Depth

Reimplementing LambdaMART in Python for endless tinkering and learning

How LambdaMART works - optimizing product ranking goals

LambdaMART directly optimizes whatever search relevance ranking metric matters to your business. This article details how this neat machine learning trick works to target what matters most to your product

Ruby vs Python comes down to the for loop

Contrasting how each language handles iteration helps understand how to work effectively in either.

Finding the relevance cutoff: when to stop showing search results

In this article: we assume users review every search result. So we need to find that sweet spot when we get to look reaaalllly fricken smart and declare, with confidence, "we have nothing else that matches your query".

Compute Mean Reciprocal Rank (MRR) using Pandas

Using Pandas to compute Mean Reciprical Rank using the MSMarco Dataset

What Is a Judgment List?

Judgment lists prevent search whack-a-mole. They provide a safety net for search, allowing you to innovate quickly on relevance with a high degree of confidence.

Hack your Career With Consulting

High end technical consulting is a fantastic thing for you to do mid career. It helps you build a personal brand, deepen soft skills, and focus on challenging technical problems. Why and why not you might want to take this step.

Political Twitter is The Opposite of Activism

Twitter gives you an illusion of influence over political events. In reality, it meaninglessly fiddles our energy away. Doing our duty requires real work in the real world.

Avoiding Grubhub: Ethical Online Delivery Options in Charlottesville

What I know so far on getting cheaper delivery in Charlottesville that avoids Grubhub's shenanigans and supports local restaurants

Kill Your Twitter Addiction With This One Weird Trick

Add friction to your twitter login to keep yourself sane.

Write for yourself, not the audience

Write to grow closer to the truth. Not because you have all the answers, not to get page views or win internet points. Instead write to broach a point of view and test it against your audience's norms and points of view.