One way or another, an employee has to be part of the workplace - it's their job. An non-employee doesn't have to be there, so access to the workplace is a privilege (and one that not all employers permit). When a non-employee misuses that privilege (e.g. by getting the the way of employees doing their work), it can and should be revoked.

Moreover, making mistakes in the context of misusing a privilege can and should carry greater stigma than making those same mistakes in some other contexts.