Looks like I might be misusing it then. I've been migrating some old spaghettied servers to single-use VMs recently, each server has had almost half a dozen admins all with their own weird ideas. The idea of saying 'hey, all the custom stuff is in this directory' for future admins is seductive. Basically I'm replacing half a dozen admins' peculiarities with one admin's peculiarity...
Its for serving using various services, so you would have /srv/http /srv/ftp and so on, you dont put packages/programs into /srv