I was guilty of this once and would add some more specific advice; if your dataset consists of multiple possible labels for the same samples, do not just assume that the average of these labels will describe the best label. And don't assume that training with the unclean data will produce a net that magically learns to do the aggregation for you.