Is this correct? I was under the impresion that the most expensive part of an llm is the training, and once that’s done the cost of running a prompt is negligible.

I get your point that this last part doesn’t scale well, but the far larger cost of training must get very diluted if they distribute it across a large user base.