As AI revolutionizes all industries, businesses have had to grapple with a new term and concept, tokens. Tokens are now a foundational expense for businesses as well as a critical item in return on investment and employee performance assessment. Terms like tokenmaxxing, the business behavior of maximizing token assets, are spreading beyond Silicon Valley. Companies are publicly assessing employees by token use, with both too high and too low values generating concern. What are tokens, and what should your business leaders know about them?
What Is A Token?
A token is the unit of an AI’s processing. Originating from the natural language processing technology at the heart of ChatGPT and other large language models, a token is either a group of words, a word, or part of a word. Each AI is capable of processing some number of tokens at a time (called the content window). The AI services are billed by tokens, and depending on the level of service purchased, may have fixed or variable token prices.
A Practical Example
Say that an employee is using an LLM to revise an email. The first message, including the draft email test and instructions, will contain a number of tokens. The response will as well. Any follow-on messages will be added to the context window and sent back as additional tokens. Studies have shown that a simple task like this consumes around 100 tokens per email on common LLMs like ChatGPT, Claude, or Gemini. At the time of this writing, common token prices would suggest this task costs around 0.25 US cents.
Relationship To Business Success
There are several reasons why tokens have become critical to business accounting:
Productivity: Companies are pushing employees to increase AI use, expecting that this will lead to productivity gains. One way to assess AI use is tokens per employee per unit of time. This metric captures all AI use, whether it be for emails, content creation, software development, or other activities.
Cost: While tokens are “cheap” and getting cheaper, as usage of AI (both in the variety of tasks and the complexity of each task) explodes, token usage is growing. Companies that initially offered unlimited tokens to their employees to encourage use are now anticipating token restrictions. This balance will inevitably lead to a concept of token efficiency, ensuring that employees don’t just consume tokens; the use is linked to productivity gains to confirm that the usage is optimal.
Pricing Models; Token prices can vary, both across time and across AI models. As use expands and token prices become a notable entry on a company’s bottom line, it will become necessary to understand if the right (or appropriate cost) models are being used for each task.
Key Tradeoffs And Metrics
The space of token assessment is still evolving, but several trends have emerged
Token budget per employee. Companies are starting to see an employee’s token budget as part of their cost, in addition to salary and benefits. This consideration is two-fold. First, it acknowledges that token expense is desired to be a significant part of a working employee’s cost, as are other costs like office space, supplies, travel budgets, etc. It also implies that the employee is expected to spend this cost to maintain maximal productivity. An analogy may be a sales associate who is assigned a travel budget but does not use it because they do not book any customer visits. While the immediate action may cost the company less, the overall message is one of lower productivity and business gain.
Token usage assessments. A trend, particularly among software companies where AI coding consumes significant tokens, is to encourage or mandate that increasing fractions of software development leverage AI. One measure of assessment is to track fine-grained token usage, not just overall but also within specific tools. This is the first step towards linking tokens with desired outcomes. Returning to our sales example, it is technically possible for a sales associate to book trips all around the country to soak up their travel budget. Whether these trips are tied to sales outcomes, and whether they are reasonable costs for each unit of outcome, also needs to be assessed. This trend is also leading to Token maxxing, an emerging controversial trend where teams or employees take unnecessary and possibly damaging steps to maximize token use.
Token budgeting. As tokens become prized, and trends like tokenmaxxing push consumption higher, corporations are recognizing that tokens are an asset whose purchase (and reservation if needed) should be a planned event. Back to our sales example, this would be the equivalent of companies negotiating airline rates for bulk purchase to ensure all sales associates can travel at a lower cost.
Operational Elements and Challenges
While the above is intuitive, the idea of tokens as a foundational expense brings several challenges that companies are grappling with.
Are all tokens created equal? The answer is, they are not. If tokens become sufficiently cheap, it may be sufficient to consider all tokens equal, greatly simplifying the accounting process. However, it is still likely that different teams will leverage tokens differently, and that the link between tokens and productivity will vary with each business function. For example, travel for a sales associate may be directly linked to a specific customer and sale, while travel for an engineer for an internal meeting, while also being necessary, has a different connection to business ROI.
How to track tokens? This may be the easiest challenge to overcome. LLMs today offer enterprise features for this purpose. As token use becomes more foundational, one can expect independent vendors to offer token tracking and token management tooling, much like products exist today to track and optimize cloud costs.
Are token prices standard? No, they are not. What each vendor refers to as a token and what each token costs is not standard across or even within a vendor. Much like airline prices, it is likely that a business will select several vendors, but their prices will fluctuate with both time and external conditions. While the concept of a token (as a unit or AI design) is common, even technological developments will change the connection between tokens and efficacy. For example, if a new AI technology is developed that consumes fewer tokens, the connection between a token and a business outcome will change.
Cautionary Observations
Token Quality vs. Token Quantity. Tokens are trivial to abuse. If an employee is told to consume X tokens per month, they can do so by, for example, generating music, writing books, or just chatting with an AI. It is important to infuse the AI force multiplier. The best use of a token is to multiply it by the human’s domain expertise to maximize the business value of the token.
Token Understanding. Employees need to understand tokens. Otherwise, they will be, intentionally or accidentally, misused. Think of a sales associate who does not know how to book a flight or select the best route to a customer meeting, balancing schedules, their own time, and price. Understanding tokens, what they are and how they relate to business outcomes, is a key element of AI Fluency. Managers need to understand the link between tokens and productivity. The engineer who needs to travel to an internal meeting will get manager approval if the manager believes the meeting will lead to business ROI down the line. The manager also likely has a budget that determines how many such trips can be approved.
Tokens Over Time. Budgets may change. Much like travel budgets, which go up and down with business outcomes, token budgets may also change. It is important to ensure that employees’ use of token budgets is appropriately distributed to avoid using up the budget too early.
Setting Appropriate Metrics. The tokenmaxxing trend shows what will happen when simply using tokens is set as the goal. It is human nature to optimize for metrics. It is better to understand the relationship between tokens and business functions, and optimize for the business ROI.
What You Should Do Now
There are a few tasks that a business or team leader can take immediately to set the company on the right path.
- Ensure your teams understand the role, benefit, and cost of tokens, both immediate and long-term.
- Set token budgets and provide guidance on use.
- Invest in tools and practices to track token use and its relationship to business ROI.
- Be cautionary. Metrics are meant to be optimized. If token use is advertised as important, it is human nature to try to increase it. Think about the value to the business and proceed accordingly.
Tokens are likely here to stay, and are a reality to deal with. By taking these steps, a business can start along the path of integrating token use practically and effectively with business success as the targeted outcome.
