OpenAI has made its Decisions API widely available to all customers for the first time, opening a direct challenge to TypeSafe's Jev model in the emerging market for fast, cheap artificial intelligence. The move, reported by TechCrunch, follows the product's initial announcement at OpenAI's developer day on September 29.
The Decisions API belongs to a category of models known as decision or System One models. They act as super classifiers: given a set of options, they make quick choices and offer a confidence rating. Developers found they could replace large language models in many workflows at far higher speed and dramatically lower cost, a shift that helped TypeSafe reach a $7.6 billion valuation.
How OpenAI's Decisions API works
Rather than building a dedicated decision model, OpenAI reconfigured ChatGPT Luna to speed up inference and cut costs. Nikunj Handa, product lead for OpenAI's APIs, told TechCrunch the team moved fast after seeing Jev's potential. "The big unlock with Decisions API has been the speed, shrinking things from one to three seconds to 100, 120, 180 milliseconds," he said.
The public version adds new voice and visual modalities, and OpenAI targeted its inference resources geographically to lower latency worldwide. In early trials, customers including Box, Datadog, Ramp and Speak used the API to replace OpenAI's LLMs with higher throughput.
Decisions API speed and early customer results
Andrew Hsu, CTO of the AI language tutor Speak, said the Decisions API cut the time between actions from 1.4 seconds to 0.13 seconds. OpenAI also used the product to improve the responsiveness of its Dots voice assistant.
Handa said multimodal work is where OpenAI may hold an advantage, noting that Luna is strong at image classification with better calibration. He also raised the prospect of pushing speed toward 60 milliseconds, which could enable an API call for every frame in a video.
