Post by @stani • Hey

It costs over 100M to train OpenAI's LLM and 250M to run ChatGPT annually. Virtually startups can't compete that unless there are cheaper ways to train a m

Stats

Comments

from what I understand once a model is trained it is effectively just a file that represents the trained model. is 250M annual cost for massively parallel usage, server costs and security / firewalling? I get that as a business model the 100M training investment needs to be protected and profited on. But the future is opensource 'brain files' correct? Am I wrong about the trained model being a completed piece of software?
I have used Microsoft Azure OpenAI service, if you use their base model as is without any retraining, it costs nothing, just the cost of requests/responses which is less than few cents per 1000 tokens. If you want to use your data with it, you can use RAG pattern - retrieval augmented generation where you upload your data to a storage service, so you pay for storage + indexing which is cheap. Retraining is expensive and generally not required in all cases, there are workarounds.
The barrier to entry is high.
Groq and similar LPUs in the future maybe the solution for new startups to enter this space. Costs decreasing will prevent monopoly and open sourced LLMs will be more widely used.
Are there any good projects that incentivise sharing your resources for training LLMs?
Open source Decentralised AI like bittensor are some of the only options we have and it's kinda imperative we do imo
Means there is big entry barrier
That's a lot indeed
That's too much though
Sharing the resources and anomizations of the data sets on crypto mining principle could be a solution? Does any one knows such a project currently ?
:O
Word
Depin AI POW