No. If you take the attitude that you're only covered if you use the concurrency primitives correctly, then you don't need atomic reference counting for everything. Basically, the programmer can use CSP to ensure that only one thread is messing around with any given section of the heap at a time, and the language implementers could say you're SOL if you do otherwise. (That probably isn't the Apple way, though.)
Swift uses the Obj-C runtime and interoperates with Obj-C code. Those languages assume a shared heap. If Swift modules didn't perform atomic reference counts, that would quite likely break Obj-C code operating on the same heap.
It should still be possible to have the compiler interpose mechanisms between everything else and the Swift code, such that your Swift code has a section of heap all to itself. By the time you're done with that, you're halfway to having implemented your own Erlang on the Obj-C runtime. That might be worth doing, though.