That got me thinking that it would be nice to have something like ComfyUi to work with diffusion based LLMs. Apply LORAs, use multiple inputs, have multiple outputs.
Something akin to ComfyUi but for LLMs would open up a world of possibilities.
ComfyUI already has nodes (mostly in extensions available through the built in manager) for working with LLMs, both remote LLMs accessed through APIs and local ones running under Comfy itself, the same as it runs other models.
Maybe not even 'akin' but literally ComfyUI. Comfy already has a bunch of image-to-text nodes. I haven't seen txt2txt or Loras and such for them though. But I also haven't looked.
It's complicated by the ComfyUI data model, which treats strings as immediate values/constants and not variables in their own right. This could ostensibly be fixed/worked around, but I imagine that it would come at a cost to backwards compatibility.
Something akin to ComfyUi but for LLMs would open up a world of possibilities.