FIG. 54GAP, UNBANKED

Budgeting for deflation

In March a business spent an average of $1.15 per million AI tokens. By September that cost was 68 cents, down 41% over six months, according to Ramp’s spending data reported by Fortune. Since 1 August, OpenAI’s effective rate has fallen 38%, to 48 cents, while Anthropic’s has declined 22%, to 90 cents.

Nobody showed finance how to budget for that. Every budgeting exercise I have led assumed inputs would cost more each year. Pay goes up, rent goes up, and software suppliers raise licence fees with a courteous letter at each renewal. We start with the previous year’s run rate, allow for inflation, include volume and debate the result. AI is the first substantial operating budget item with a unit cost that drops faster than most teams can update their forecasts.

The method breaks in two places

Guido Appenzeller of Andreessen Horowitz called the pattern LLMflation and measured it. A model matching GPT-3 quality cost $60 per million tokens in November 2021, falling to $0.06 by late 2024. That works out at roughly a tenfold annual reduction. Ramp’s figures indicate a more gradual decline, but few budget items drop 41% over six months.

A conventional budgeting process goes wrong twice along that curve.

The first mistake involves price. A budget prepared in March, when rates peaked, took the prevailing token price and added a buffer. Six months on, expenditure falls below plan, and the underspend suggests good management. It merely tracks market movements.

The second mistake involves volume and mix. Customers are moving towards less expensive models. Frontier models represented 53% of usage in early August and 45% in September, Ramp reports. Lower prices also encourage fresh uses that nobody included in their projections, and budgets must allow for these before they appear in the data. The changing mix reduces expenditure per person while additional uses increase it, and one yearly total cannot reveal which effect dominates.

The two mistakes do not offset each other. They conceal each other.

The underspend trap

Banking the difference is the costliest mistake. A CFO finds AI spending 30% below budget, books a saving and reduces the following year’s allocation accordingly. Then a team shifts an entire process onto a model, usage triples, and spending exceeds a budget based on a price nobody pays any more and usage nobody anticipated.

A CFO should treat that underspend as a reserve. Part reflects lower prices, part covers volume still to come, and none counts as efficiency until someone demonstrates which process now costs less and explains why.

Who needs the price to stop falling

Buyers benefit from deflation, but whoever funded the capacity has a problem. Morgan Stanley has identified up to $300bn in bonds financing neocloud data centres as exposed if token prices do not hold up. Ramp’s chief economist, Ara Kharazian, said the trend was not a disaster or the bubble bursting, but a “crack in the AI thesis” (Fortune).

The labs understand this. OpenAI’s CFO, Sarah Friar, told a Goldman Sachs conference this month: “I would love to get us away from token-counting.” (Fortune) Suppliers will reshape their offers. They will pitch seats, outcomes and multi-year spending commitments at a discount that seems attractive today. Each arrangement obscures the price. When a buyer signs a committed-spend agreement at this quarter’s rate, it fixes a price that was falling and surrenders the deflation it was about to benefit from.

A budget built for a falling input

Five adjustments cover most of the work.

  • Volume and price, budgeted apart: Plan the units first (tasks, documents, calls) and price them second. A single spend figure cannot tell you which assumption failed.
  • A price path, not a price: Set the budget on a declining curve with a floor, drawn from the observed rate of decline, and re-price it every quarter.
  • Short or indexed commitments: Keep committed spend to a year, or tie it to the vendor’s list price with a price-down clause. Multi-year volume at today’s rate is a bet against the trend.
  • The underspend held as a reserve: Report it on its own line until volume settles. Only call it a saving when a named process costs less.
  • The review cost kept in view: The token price keeps falling. The expert who checks the output does not. I set out that arithmetic in The verification premium, and it matters more as the token line shrinks.

If prices continue dropping at anything close to this pace, token expenditure will contract, and time devoted to reviewing output will account for most of the cost. Finance teams budgeting for declining input prices today will anticipate that change. Everyone else will continue to report savings against the wrong line.