AI cost control: Thinking of early cloud adoption days when we blew our entire annual cloud budget in one month due to poor cost controls... What are you doing to be mindful of AI cost control in these early days? Anyone taking any lessons learned from their cloud pricing and cost control experiences into their AI projects and making sure contracts with third-party vendors and your engineering team developments don't get you into cost trouble?
Sort By:
Oldest
Director of IT in IT Services4 months ago
Yes. we are ensuring clear contracts with vendors and close collaboration with our teams to prevent cost overruns.Director of IT in Travel and Hospitality4 months ago
Close collaboration is the key. The unit costs for AI tend to be on the low side and manageable for everything but the highest volume use cases. Areas to watch for are the cost differences between GPT3.5 and GPT4 models, as the costs increases by orders of magnitude. To a lesser extent token size (4k to 16k) needs monitoring too.Cost modelling using a set of test scenarios is important. Quality should not be forsaken either. We have found reponse quality significantly varies both between older and newer versions of the same model, where newer is not necessarily an improvement for the same prompt.
On the whole however the risks of spiralling AI costs do not present the same challenge as the early days of cloud adoption.
VP of IT in Manufacturing3 months ago
Based on lessons learned from early cloud adoption, effective AI cost control involves setting and monitoring strict budgets, optimizing resource usage, and negotiating clear vendor contracts with cost ceilings and transparent pricing. Identifying if multiple vendors are being leveraged for the same use case is crucial and adjusting accordingly is crucial. Cost control strategies should also vary depending on whether to build custom AI solutions or leverage SaaS AI solutions. Additionally, conducting small-scale pilot tests to understand cost implications before full-scale deployment and ensuring cross-functional collaboration between finance, procurement, and engineering teams are essential to maintain cost awareness and control. These measures help prevent uncontrolled spending and ensure sustainable AI project deployment.