There are only a handful of western countries that use this approach. As counterexamples, Switzerland, the UK, Spain, and many other countries tend to have local municipal police departments, and Germany has state/Bundesland-level police departments as well as the random auxiliary departments that a sibling comment mentioned.
France and Sweden are both far more centralized and less federalized. There's a reason more federal countries such as Switzerland or Germany have more decentralized policing. I think it would be beyond unacceptable to most Americans, myself included, for the US to have a single police department with its bureaucracy in Washington. If anything, I think far too power in the US is already centralized in DC which contributes to bad governance due to the vastly increased distance (in many senses) between politicians and their constituents.
I would be fine with 50+6 state/district/territorial police departments, which is sort of the Germany or Canada (with Ontario and Quebec) approach.
None of these have a police per government agency, that's what we are mainly talking about. That there are local police departments or state police departments is not the main issue I think.
Also, counties often have their own sheriffs with a separate set of powers and responsibilities. Sometimes court systems have their own enforcement arms, separate from other agencies.
The US often seems to have more layers of administration and law-making than some other places. Making individual officers work through which set of powers and laws they're dealing with in a given moment seems like it might at times be a daunting prospect. You could in theory organize them into a single bureau, but I suspect you'd inevitably wind up with specialized sections to deal with the particular laws in given cities.
