Even though I have a couple of patents to my name I am generally against software patents, sometimes I think that for genuinely novel algorithms there should be an exception (e.g. RSA encryption). However, as there is clearly no way of objectively defining what a "novel" algorithm is (i.e. there is no algorithm for it) we can't really have this exception.
So yes, I would agree that algorithms should not be patentable.
Even if RSA really is sufficiently novel (note that it had already been discovered by a secret service four years before publication), is that really worth a 20-year monopoly?
More info: http://en.wikipedia.org/wiki/Karatsuba_algorithm