It matters quite a bit. The view that "it only takes one unhinged person" tells you to treat homeless people the same way you treat everyone else. The view that homeless people are much more likely to assault you than normal people tells you to be more cautious around them than you are around normal people.