When you’re trying to run and manage AI projects, there’s a lot that needs to be considered in terms of determining project budgets and cost. All technology projects have three major aspects of cost: software, hardware, and services. But when it comes to AI projects an additional aspect needs to come into play: data.
If you’re trying to generate a return on investment, you need to know what the investment part is. You have to know what the return is, too. But of the two things, you’re guaranteed to spend at least what the investment part is, if not more, but you’re not guaranteed anything about the return. People underestimate the complexity and cost of dealing with data. So, it’s crucial to think about how big your AI project is. Because when it comes to AI projects, size really does matter.
What is the actual cost of AI?
On a recent AI Today podcast detailing AI project costs, the topic of how the different aspects of people, process, and technology contributed to AI project costs. The challenge is going beyond software, hardware, and services costs because data is the heart of AI. Specifically, the costs of gathering, preparing, and cleaning data can really add up. Data is continuing to grow, and unfortunately not all data is clean and ready to use in its raw form. Compounding these data quality and quantity issues are challenges around getting access to the data you need.
AI project managers often underestimate the full cost of AI systems, which can have major impacts on your project. There’s many different things that go into thinking about the total cost of an AI project. One of which has to do with building versus buying your AI models. Are you using someone else’s model? Are you going to focus on prompt engineering or fine tuning? Are you building a retrieval augmented generation (RAG) solution? Or, are you going to be building your own model?
You also need to consider where the model is going to be tested and used in the real world. Are you going to be using it in the cloud? Is it local? How are you training your model? And of course, you need to consider all the aspects of data engineering.
How to reduce the cost of an AI project
Thinking big, starting small, and iterating often is really the motto for AI project success. When starting small, it matters how small you go if you want to control project costs. For every project and every organization how small to go is going to be different. Because of this, you really need to understand your project scope.
The scope of your AI project iteration will have a direct impact on cost. Your project iteration should be short. Each iteration should take about two weeks – not months of years. So that means controlling the project scope so you can have that short iteration.
One way to control scope is by using and building off of someone else’s already-built model. It’s going to have the lowest cost and the fastest iteration time. If it’s already available, then go ahead and use it. That’s one of the most effective ways you can start small.
That’s the reason why LLMs and foundation models are so hot right now. The cost is low, the iteration time is fast, and the potential time to return is very quick. So if cost is something that you are paying attention to, and you might not have large budgets, then a really great option is to use someone else’s model.
Keeping AI Costs Low
There’s a few easy, and low cost ways of using someone else’s model. If it has an API, use that. If it has a chat interface, use that. You might be able to get it for free, or maybe you get it for a low subscription cost. You could also build an incremental solution on top of the API, such as a Retrieval-Augmented Generation (RAG) solution or something else that doesn’t need to train a new model. If you can’t just use the model directly, you can also tweak it by fine tuning the model or augmenting the model with additional sources.
It’s important to understand these options aren’t all free, but they will help keep prices in check. Some AI systems may have a monthly subscription fee or an API cost for using someone else’s model. Additionally, there may be some time that’s needed from the analyst, developer, user, and/or the citizen developer, and remember that people’s time is not free. If you want to generate prompts or inputs for the model there will be some time and costs associated with this. If you’re going to build a RAG or some other solution on top of it, there will be additional development time associated.
Additionally there may be some data preparation required, either for the data you’re putting into someone else’s model or the results coming out of that model. If you do need to do fine tuning and model tweaking, there’s going to be some costs involved there.
Overall, using someone else’s model can dramatically reduce the cost and time for AI project deployment. But like with any AI application, you always need to check the results. This is especially the case with LLMs because they can come back with some pretty bad results. Sanity checking is incredibly important.
Should I Spend the Time and Effort to Build my Own AI Models?
There may be situations where you can’t just use or sit on top of somebody else’s model. In this case, building your own model may make the most sense. There’s many different reasons and use cases and justifications to take this approach.
When thinking about project costs, building your own models isn’t usually the cheapest solution. So when taking this “think big, start small, and iterate often” mindset and approach to building models, starting small will really reduce costs considerably because we’re keeping the scope in check.
When you are trying to figure out and determine your AI project costs, you also need to think about AI service costs. What sort of services are needed for this AI project iteration? Who is on the AI team, both now and in the future of this project? This matters when determining the costs of your AI project because we have to pay people, and it also is going to potentially impact project scope.
When we’re thinking about the AI team specifically the makeup will be different if you use someone else’s models versus building your own models. If you’re using someone else’s model, then you’re most likely looking at having “ citizen developers” which may be easier to find and more affordable. This recent rush to generative AI is because people have seen they can get results very quickly without having fully trained and possibly hard to find data scientists and data engineers on their team.
However, even if not building your own models, you still need to identify who on the team will be managing the data and monitoring the data quality. Garbage in is garbage out, so you want good quality data that’s coming in. You also need to identify who’s going to be monitoring the use of the model and the quality of the inputs and outputs. You can’t just set it and forget it when it comes to AI, and that includes generative AI. This all impacts the overall AI project cost.
On the other hand, if you’re building your own model, you need to have your data engineering and preparation teams. When you’re operationalizing the model, you need to have the right teams to put the model up, and constantly monitor the model for data drift and model performance drift. These costs are more likely to be fixed and you know exactly how much the costs will be. So that even if the demand for your model really inflates, your costs don’t fluctuate.
When determining the real costs to your AI project, think about all the factors outlined above. Understand your team. Understand your budget. Understand your scope, and follow a best practices methodology like CPMAI for a successful AI strategy. (disclosure: I am a managing partner and co-host of the AI Today podcast)
