OpenAI has made its Decisions API model widely available to users for the first time, entering the market for fast, cheap intelligence that recently turned TypeSafe, a startup founded by a former Frontier Lab researcher, into a $7.6 billion company.
TypeSafe’s Jev model created a category for deep learning models that act as super classifiers: given a set of options, they can quickly make choices and offer a hint of confidence in their decision. Software developers quickly discovered that these decision models or System One could replace LLMs in a variety of workflows, operating faster and at significantly lower cost.
Nikunj Handa, product manager of OpenAI’s APIs, told TechCrunch that his colleagues saw Jev and said “wow, we have about a million use cases for this.” Handa’s team rushed to come up with their own version: rather than building a specifically calibrated decision model, the team took ChatGPT Luna and reconfigured it to speed up inference and reduce cost.
“The big positive of the Decisions API has been speed, reducing things from one to three seconds to 100, 120, 180 milliseconds,” Handa said. “In the future, we might somehow benefit from some specific training to make our models even better in this way. But for now, it’s just plain old Luna that gets the job done.”
Announced during the company’s developer day on September 29, the Decisions API became available to all customers on Thursday. OpenAI added new voice and visual modalities to the public release and worked to ensure lower latency in global regions by geographically targeting its inference resources.
In early trials with customers like Box, Datadog, Ramp, and Speak, engineers used the API to replace OpenAI’s LLMs with higher throughput. Andrew Hsu, CTO of Speak, which creates AI language tutors, said the Decisions API allowed the company to reduce the time between actions from 1.4 seconds to 0.13 seconds. OpenAI also used the product to improve the responsiveness of its Dots voice assistant.
Yet it is unclear what constitutes a good decision model, other than “it works for you”: optimization in terms of intelligence, cost, benchmarking, and speed appears to differ depending on the workload. TypeSafe CEO Diogo Almeida is wary of benchmarks and urges people to test the models in their own work, but even TypeSafe released its own rankings last week alongside the release of its latest Jev model, which includes new features including support for images and smart statements.
Public benchmarks such as the AIM System One Index and the Artificial Analysis Decision Index tend to favor D1 Ultra, a new decision model released by Liquid last week, and Base Lab’s Elephant-1. The latest Jev model remains competitive on both indices, in seventh and third place, with OpenAI’s decision model falling to eighth and fourteenth respectively. However, TypeSafe offers are, for the moment, the most requested on OpenRouter.
However, the most fun Jev benchmark is JevMan, a benchmark in which decision models play PacMan; the latest Jev model tops this ranking. Felix Wunderlich, chief growth officer at Opper AI, which built the benchmark, said the classic game “is an interesting test because of its uninterrupted flow of rapid, high-stakes decisions. At each intersection, the model must decide where to go in a few hundred milliseconds.”
Where OpenAI’s arguably top-performing model may have an advantage is in multimodality: For image classification, “Luna is extremely powerful and showed better calibrations,” Handa told TechCrunch, adding that OpenAI’s customers want the Decisions API to also be more flexible in using images as outputs.
Handa also saw further increases in speed enabling new use cases: “Can we get this to something like you know 60 milliseconds, and then you could imagine like an API like a call like for every frame like a video?
As these offerings become faster and cheaper, we can expect them to live up to Jev’s namesake, an economist who explained how using a resource more efficiently makes it more ubiquitous. Or, as Handa sees it, there will be a growing “crazy set of use cases” that have been hiding in plain sight until decision models reveal them.
When you purchase through links in our articles, we may earn a small commission. This does not affect our editorial independence.
Gn tech
