Farms already produce more than enough food to feed everyone (and, indeed, the excess is a feature because food security is really important). The reason not everyone is fed is not due to needing to divide water resources between farms and other uses.
Going only by the effective need of humans is a bad argument. A lot of farmers wouldn't survive without subsidies and are not vital to our food supply.
Stop eating beef. With the water saved we can grow enough food for any realistic human population. Ok we solved this one. Or do humans NEED burgers as well? We can already feed all people, any starvation is strictly a political problem not a food existing on the planet problem
We produce enough food for everyone already, and then waste a huge amount of it. Our food problem isn't about producing more, it's about distributing what we have.
Humans NEED food, the output of agriculture. Humans do not NEED any of LLMs' outputs.
Once everyone is fed, then we can talk about water usage for LLMs.