If you have internal data scientists, which is better for risk reduction: building your own proprietary LLM, or using one built by a major third-party provider?
Proprietary LLM36%
Third-party LLM58%
I don't have an opinion6%
184 PARTICIPANTS
Sort By:
Oldest
CISO in Software2 months ago
How many companies have the skilled and experienced personnel to build their own LLMs? Hmmm....Founder in Finance (non-banking)2 months ago
I think building a foundational LLM from scratch is highly impractical / out of reach for most corporates. It requires gathering and pre-processing petabytes of data (without violating T&Cs, copyrights, privacy and confidentiality, etc..), millions of $ in infra to training and fine-tune... and that is not just a one-off - you have to upgrade, update and maintain their after. But perhaps the main reason is the pace of development across both commercial and open-source third-party models makes it like jumping from stand-still onto a moving train. All in all, even if you could create a foundational model, it has every probability of (i) losing you money, (ii) falling short in performance relative to third-party models very quickly, and (iii) becoming yet another enterprise IT legacy system.I would recommend breaking down the alternative in your question of leveraging third party models further and strategizing from there - multiple options within that branch:
1) Open Source vs Commercial: The most obvious one is using a commercially available LLM via a paid API (e.g. OpenAI GPTs, Anthropic Claudes...), but you can also consider highly capable open source models (e.g. LlaMA, Mistral...). As rules of thumb, I'd pick open-source if I'm prioritizing privacy and intend to fine-tune the model at a parameter level, and commercial if the company's skillset in this field is limited and LEGACY systems are your biggest IT-related nightmare!
2) Open-source LLMs served and hosted by a third party (vs built from source onPrem / own cloud): If you go down the open source route you have a few options. Companies like Ollama, Replicate and even Hugging Face can host and serve various open-source, pre-trained LLMs for you. This is a middle ground in privacy - you are interacting with a commercial cloud provider and I'm not sure to what extent they have the same privacy guarantees as popular multi-purpose public cloud service providers. Also, there is some lack of transparency as to how the original open source model is being implemented and deployed (though I believe at least with OLlama you can verify the binaries to check for any variations). This is a very cost-friendly option (cheaper per 1M tokens than OpenAI for similar performance to GPT-3.5-turbo / GPT-4), offers more control and potentially more transparency.
Building from source is the ultimately solution for transparency, control and, assuming the right cybersecurity controls are in place which is down to the corporate implementing it, privacy. The corporate takes and open source model, complies and deploys the LLM using its own IT infrastructure (e.g. onPrem servers). Note however this also brings major compute requirements not just to build but for ongoing inferencing and so far the numbers I have seen result in orders of magnitude higher cost per 1M tokens than commercial models.