Another (similar) very irritating thing I've found with Drive is that while there is an option to remove a shared file from your drive (eg. if it's only listed because of link sharing), there are some that (seemingly at random) can't be un-shared.