Personally, I use R. Importing and exporting CSV files in R is trivially easy (it's what it's made for, after all). Doing transformations on an entire column or row at a time is also very simple.
Import a csv, then take the logarithm of one column and plot that against some transformation of another column? No problem - two lines in R, and still highly readable.
I do all the things one might use a spreadsheet for in awk :-).
One might say it's slower then a spreadsheet. I don't know. I can type an one liner awk summation/average, faster then it takes OpenOffice to load. This is to me the most common spreadsheet operation.
For more complex task, I feel that I'm working faster in awk (it's a full blown high level programming language, after all). I'm not sure it it's actually faster or if it's only my opinion though.
Financial consultant here. I use Haskell for this kind of thing.
In the event that I need a visual prototype to work with, together with a client or business partner, I don't think it's all that horrible to prototype something in Excel, and later port it to Haskell if necessary. But that's of course because I already, at some point, reinvented the wheel to make this possible. So now I'm mostly just reusing code which I already have available.
Awk makes it trivially easy to parse text. How does Haskell handle this parsing task? I am curious because I've always wanted to learn Haskell, mainly because of its lazy evaluation, but hadn't had the time yet. Hope to do it soon.
Out of the box, it might not be the most intuitive text parsing tool indeed. It's trivial, but not immediately intuitive.
That said: strings are lists of chars, and Haskell is strong at pattern matching (esp within lists) in a bunch of different ways, so trivial text parsing tasks are trivial and intuitive in Haskell. For anything slightly more complex, there are regular expression and enumerating libraries which can do the job alright (and efficiently if I may say so).
EDIT: I'm sorry, can you give me an example of a problem so I can give you an example of a solution? :-)
I end up using Python. In the end it's much more powerful and less error-prone (the oops factor of spreadsheet apps is ginormous). Quality plot, graph and stat modules abound.
Plus it has the advantage of being able to trivially scale in any direction, like refactoring or working on a real database (sqlite) when the dataset grows instead of shoehorning a database into a sheet, or making a module out of it, or integrating it with other random tools.
Otherwise, terminal + browser gets me through 90% of the day.