The Challenges of Using Large Language Models in Production

A lot of people highlighted that they are not very excited to be dependent on just one company's API. Many respondents said that they prefer if there was some kind of public model with the same performance and weights that were public. Some respondents asked should we pay for a third party provider to train our own model so that we do not have to be dependent upon an external API. The high cost of production is something that people could not stop talking about whether it's inference cost, infrastructure cost or human cost of people that actually know how to do stuff with this.