If you have a crawl space or basement, you can install electrical under-floor heating at fairly affordable costs. Basically the heating element comes in a roll which you then tack the the underside of your floor.
I have lived with such a system before for a year and it was both easy to use and did a good job of heating the space. I don't remember the electricity costs so I can't comment on the economic efficiency, but AFAIK it's moderately good. I wish I had it in this house instead of the noisy, draughty, and inefficient gas-powered duct heating which I avoid switching on whenever possible.
Heat pump! You can get wall mounted units that work just like an A/C (but in reverse of course.) Efficiencies have gotten pretty good in the last decade. Check out units made by Daiken, Hitachi, Mitsubishi for example. I know they make multi-room versions. You'll need to run the refrigerant lines, but that's usually a much smaller task than adding ducting.
Any electric heater will be approximately 100% efficient, but electricity is generally much more expensive per unit energy than than natural gas or oil. If you have decided that electricity is your best option, you should compare convenience, installation cost, and whether you want to heat specific areas or the entire house.
However, there's a big difference between electric resistive heating (generating heat) versus electric heat pumps (moving heat around.) The latter can be 3x or more efficient than the former.
No access to natural gas? I'm assuming you live too far from the gas lines to make a new gas installation possible?
That said, I had a instructor who's friend bought a house out at the beach 30 years ago. He didn't know what to do about heating.
He decided to put a large diesel tank on his property, and use that fuel to run a boiler. He installed baseboard hydronic tubing. He was very happy with the heating set up.
His previous system was electric, and his monthly bill was
outragious.(electrical rates really do vary from area to area.)
He also had an old Mercedes diesel automobile. One day he used the heating grade diesel for fuel for his Mercedees. His car ran fine. He was paying someting like 35 cents a gallon for gas. He used that car to commute for years. There are no taxes on the heating fuel, or at that time.
(That 35 cent quote is from a story told to me 20 years ago. I don't know if heating diesel was ever that cheap? I was told this story from a instructor who I don't completely trust. He tended to embellish.)
That said I always thought if I lived in an area where I couldn't use solar, or natural gas; I would definetly use a boiler, with propane, or diesel fuel? They make condensing boilers that are very efficient. I installed one and it wasen't that hard. I don't know if a condensing boiler can use diesel fuel, but I know you can use them with propane.
If you decide to do the work yourself; buy the third edition of Modern Hydronics. It's expensive, but go for it. Hole yourself up in a room for a weekend, and read that book. It's geared for large hydronic installations, but will handle residential installations fine. Hydronic heating is really under utilized, especially on the west coast. It's a great, energy efficient way to heat a home. I bought a fancy condensing boiler and happy with it, but I have a feeling I could heat a house with a water heater if your have enough baseboard, or tubing? And you live in a temperate climate?
Of course, hire a professional if funds are tight. Make sure the professional knows about boilers, and hydronic heating. A good first question for the contractor is what brand of 02 sensor he/she uses to calibrate the boiler. If he doesn't immediately come up with a answer, or looks confused--run. The second question should be "could you write down the what a primary/secondary hydronic piping configuration would look like?" Verify he knows what a primary/secondary system looks like, and why it's important.
Hydronic heating is vastly underutilized in the U.S., at least in California? I don't know why it isn't used in more situations? I guess engineers feel that if you are going to need air conditioning, why bother with it? I have heard incorporating air conditioning with hydronic heating is not that difficult? As to keeping zones at different temperatures, I did not find that to be a problem if you run your zones correctly.
(One other painful lesson I learned by relying on wood burning too long. Don't buy a cord of wood off Craigslist. I guarantee a guy will show up with less than a cord. If you are forced to use the fireplace; put a insert in.)
Diesel in the US has a red dye added if it's for agricultural use. "Detection of red-dyed fuel in the fuel system of an on-road vehicle will incur substantial penalties."
Presumably you have to piss off a cop in your diesel-powered vehicle enough for them to test your tank.
Hydronic heating is great, but I don't think it's well suited for retrofits.
I'm not very far from the gas lines, just on the wrong side of the street, and digging up the street is expensive.
Oil burning is more expensive than using a modern-day electric water heater, by the way. Oil has gone up in price and electric heaters have gotten much more efficient.
Let's not be cutting down the carbon dioxide remover when we're trying to go eco-friendly. :)