I have to agree letters of reference are the most important factor. Of course you need good GRE scores as well. Unless you do some academic research and publish a paper or two before you apply (senior thesis at least), there is little chance you will get into MIT or Stanford. A friend of mine prolonged his undergrad for a year or two in order to work on some undergrad research for a while and got into MIT that way. Of course, if you are willing to settle for something other than a #1 school then you could get by without any publications as long as you have an interesting class project or two that you can talk about and you do well on your GREs (remember to study your vocabulary). Programming experience will not help you get in. If you can find some kind of research position, that would help, but it could be hard to find a position like that at IBM or HP with your experience. A better bet would be to look for a prof somewhere that does something you are interested in and go work for him for a while (paid or not). Starting with a Masters is another route that can work.

Also, I have to put a disclaimer here that getting a PhD will probably not be worth it from a financial standpoint, and most likely you will not find a tenure track position afterwards.