they are building a product and said the unit economics must make sense, local models have slower latency, unless you run a gpu on for hours which gets expensive fast

Local models will make a lot more sense once we have the scale for it, but when your user count is still small paying cents per image is a much better deal than paying for a GPU either in a data center or physically.

Local models are definitely something I want to dive into more, if only out of personal interest.