They might specifically mean blacks when they say "people of color", not that asians aren't a minority, but they person saying that might only be referring to their own race. I am not making a judgement call on what should be a "person of color", I'm just pointing out what they might mean.
In usage, "people of color" tends to be malleable. Sometimes it means "black". Other times it means "not white" or "black or hispanic" or something else entirely.
Sometimes this is just ambiguous. Other times, it becomes a source of liguistic slight-of-hand when people use one meaning (without specifying which) but seek to equivocate with a very different usage with a very different meaning.