OpenAI lost the plot on 'boring' LLM use-cases

I recently began updating my notebooks for Cheat at Search with LLMs.

It’s a training course using LLMs in what’s become their “boring” applications - NLP. I care about taking search queries and classifying them into high-level categories (furniture or electronics). Or extracting entities like colors or brands. Or correcting spelling mistakes. These used to be hard NLP problems that took months, now LLMs help practitioners like me quickly get to an 80% solution.

Back in my day (ie 2020), one the “cool” applications of LLMs - at the time we just said transformer models - was this zero/few shot classification. Now I realize the cool/sexy topic has become agents.

With GPT-5 OpenAI seems to have forgotten NLP use-cases in favor of the latter. Agents benefit from reasoning models. Boring NLP tasks want speed, low cost, and don’t benefit from time consuming reasoning.

Form open Source → Nano + Mini models

With the NLP use-cases I care about, I was at one time enthusiastic about smaller open source models. They solved these problems well enough. They let me host models wherever I wanted.

Then the major providers, like OpenAI, released mini and nano models, and I realized it wasn’t “open source” I cared about, but just reasonably small and cost-effective. I care about structured outputs - specifically the ability to constrain to a small set of labels.

It turned out, I’m happy to pay a nominal fee for someone to host a tiny LLM for me to batch process thousands of search queries or documents at reasonable cost. With this setup, I felt I had an ability to carefully dial my token usage, latency, and accuracy for classification problems I would never have dreamed of.

More importantly, I wanted to support OpenAI, who started this revolution. I want to be a fanboy.

Let them eat reasoning

Imagine my surprise when I started migrating my classification code to GPT-5.

They force reasoning into my carefully tuned classification pipeline. The best I can do is set reasoning=minimal. I can’t turn it off. And OpenAI doesn’t seem to have a roadmap for a model without “reasoning” (gpt-5-chat doesn’t have reasoning, but its targeting conversation).

Now I am an avid user of agents like Claude Code. And I realize reasoning carefully about evidence is important for these use cases. I realize OpenAI cares about keeping up with Anthropic and Google in coding and search.

However, for my use cases, there is very little benefit when asking an LLM to pull the color, brand, etc out of a bit of text like a search query. It’s expensive overkill, and not something I’m enthusiastic to pay for.

As many OpenAI platform users report, for meat-and-potato use cases, reasoning adds 10x the latency with little benefit. Data from user _j :

Analysis with longer reasoning to show reasoning = latency here

More tokens, more complexity, slower latency, at worse performance. What’s not to love about GPT-5 for the NLP practitioner?

NLP use cases matter too

The history of AI is one of mini-bubble after mini-bubble, sometimes losing the plot on what’s important. We had the vector database bubble, which was laser-focused on RAG, and not the other non-AI use-cases of RAG like just traditional search, personalization, and recommendations.

Now the agentic world is taking off, which is great, but we may look around and realize “oh wait, remember when LLMs could be used for other things too!”

It all strikes me as lack of vision, and suddenly stumbling into a new thing after new thing. Claude Code was a game changer, almost accidentally discovered. Same with ChatGPT. We stumble into an unpredictable success, almost accidentally, via some side project away from the current hype. Then suddenly everyone laser focuses only on that new thing.

How are we in a place of such incredible innovation coupled with unprecedented lack of imagination? And at the same time somehow forgetting the real use cases?

Ah well, time to look into other providers.