Any idea why Craigslist is so vehemently opposed to either providing an API, or allowing others to scrape their site for data?

It doesn't seem to align with many of their other principles of openness.

It goes against what their intentions are. As Jeff Atwell found out http://www.codinghorror.com/craigslist/

If they released an API that allowed searches in multiple cities than it becomes less craigslist and more ebay. Where buyers are looking for deals all over the place instead of the deals in their town.

It appears that they've disabled RSS feeds for custom searches (pretty much any search that sets url query keywords), but they used to accidentally have a pretty sweet REST api.