No, low latency is not needed for all non-realtime things. Even then, you are looking for latency compensation. Just add one VST plugin to an audio track and you will instantly add latency.
The crux is you want everything to play in sync when doing recording and overdubbing, e.g. "hit record and what I hear live is in sync with what I have recorded already". Almost all DAWs solve this by just starting things a bit later (latency compensation). Some audio cards solve this by allowing direct hardware monitoring. But even then you will have some samples of latency.
All plugins add latency. They just report that in their API and your DAW compensates. There is no way eliminate CPU cycles being spent on whatever the plugin does.
I wrote a DAW (https://ossia.io) and a few dozen plugins (actually 150+ if I count the various small tests and examples in https://github.com/celtera/avendish) and I can assure you that most plug-ins don't add latency.
> There is no way eliminate CPU cycles being spent on whatever the plugin does.
that's not how DAW works, they don't output audio immediately anyways, everything is buffered at the driver level or just above so that there's always 1 or 2 buffers of delay between the input and the output.
There's no difference in terms of latency between putting one or 50 bitcrushers in series for instance because in the end the audio main loop looks like (in a very simplified way):
void process_soundcard_buffers(float** in, float** out, int samples) {
for(int i = 0; i < samples; i++) // samples would usually be the buffer size you set in your audio config
out[0][0] = f(in[0][0]);
}
where f can be one distortion, or 50 distortions - what matters is that their processing time is < than the time you have to write the output and if so you'll always get the same fixed latency. (And if not so, you don't get delayed output, you get crackles because the soundcard will be reading its buffer whether you wrote something meaningful in it or not)
That's for plugins with an explicit fixed latency to use PDC. If a plugin doesn't request latency, which most do not, they have the duration of the DAW audio buffer to complete their processing. If the combined processing of all active plugins on a single thread exceeds the audio buffer duration you get audio drop-outs, or portions of the buffer which are just zeroes because they were unable to be processed in time.
The crux is you want everything to play in sync when doing recording and overdubbing, e.g. "hit record and what I hear live is in sync with what I have recorded already". Almost all DAWs solve this by just starting things a bit later (latency compensation). Some audio cards solve this by allowing direct hardware monitoring. But even then you will have some samples of latency.