Some percentage of the population is comprised of violent sociopaths. Those people need to be locked up for the safety of the rest of us. People who have committed victimless crimes, like minor drug use, should almost never be incarcerated. People who have defrauded other people in some way, including petty larceny, should be punished by some means other than incarceration.
Pretty much what I'm saying is that jail time should be reserved solely for the violent.
A very, very, very small percentage of the population is "violent sociopaths", though, and optimizing your society for solving that problem incurs huge amounts of waste.
Which isn't to say there should be _no_ answer to that question, but more that the answer to that question shouldn't also be the answer to all the other crime encountered. Not even all the violent crime, which is frequently a product not of the individual, but of the situation that individual is placed in.
Pretty much what I'm saying is that jail time should be reserved solely for the violent.