I've always wondered about this, especially in embedded devices. Amongst the many mistakes Microsoft made with Vista, first and foremost was allowing the bloat to get so bad that it wouldn't run at all on machines that ran XP quite happily. Later Windows 7 could actually run on those old XP machines with many, if not most, of the features that Vista bragged about.
So here we have Google, arguably a company which can hire all of the worlds best engineers, why are they succumbing to the code first and ask questions later philosophy that gave Microsoft such heartburn?
Andy, I know you read HN, tell the team that their bonus depends on Android 4.0 being runnable on a Nexus 1. It isn't about supporting the platform with the least market share of any Android phone, its about instilling in the team a philosphy that taking the time to run great on hardware of that capability pays huge dividends in the phone world.
Many engineers today were brought up in a world where the next PC will be 1.5 - 2x the 'size' of the current PC so bloat is fine, schedule is king. This is a bad philosophy in the embedded space. That 1.5 - 2x the 'size' phone means you can't cut your phone costs, you can't grow your market, you are stuck waiting only the latest / greatest chips which are always in short supply to ship volume. Help those engineers break the habit, I know they can they are smart folks. Challenge them, as Ben Horowitz would say use your lead bullets on this one.
This isn't a hardware capacity issue. The Snapdragon has a 1 GHz CPU very similar to the Cortex A8, an Adreno 200 GPU broadly comparable to an SGX 535, and the same 512MB of DRAM that all the other phones of its generation have.
Its UMTS radio wouldn't make the cut on today's store shelves, and its camera sucks. But fundamentally this is about planned obsolescence. Google doesn't want to have to pay to maintain the driver stack and QA infrastructure for an old phone.
The Nexus One has 512MB of internal storage. The Nexus Galaxy has 32GB of internal storage. If ICS weighs in at much more than 400MB and has the stellar (read as: "perfunctory") MicroSD support we've come to expect from Android, it'd be about as enjoyable as a fart in a spacesuit on a Nexus One.
That's a good point. They may not be able to fit the standard image. But if that's the only issue, then expect Cyanogenmod et. al. to have this fixed within weeks. Running real filesystems on the sdcard has been routine in the AOSP community since forever.
Just to clarify what you're saying here:
Are you saying that simply because a file is encrypted, that it'd be equivalent to the uncompressed size?
There are a fair few modes of encryption (such as CFB, OFB and CTS) out there which ensure that the encrypted data is the same size as the input data. Even if Apple uses a mode which requires padding, the padding should not be so large as to increase the size of the iOS image to the size that you are suggesting.
Encrypted data is not very compressible. Compression algorithms exploit statistical regularities in the bitstream, whereas the whole point of encrypting something is to get a result with no statistical regularity whatsoever.
If you compress first and then encrypt, you can then get smaller file sizes than the input, because the compression algorithm can work on the plaintext.
Let me get this conversation straight.. Someone says that it's strange that android is so large. Someone else says iOS is big too. Then someone comes in to say that they are not large, they just suck at getting the compress/encrypt issue right. No one, other than obtino, thinks that it was simply a mistake that Xuzz put 'then' in his/her post?
It's compressed after encryption, so the effects of the compression is minimal. I believe that my post was correct: due to the effects of the encryption, the compression is almost useless, rendering the large iOS file size essentially equivalent to the size when installed to disk. That's all.
This is correct, and yet at the same time I believe it is incorrect. I am totally willing to believe I'm wrong here, though (as it has been two years since I actually did this process manually). I will explain. ;P
So, it is my understanding that an IPSW file is a ZIP archive containing a number of files, the largest one being where the main filesystem is stored. This file is encrypted, and does not compress very well at all.
However, that file is itself a dmg (Apple disk image) file, which is a compressed file format: a dmg is a compressed HFS+ image. Therefore, the encryption is happening after the compression.
Therefore, I do not believe it is accurate to claim that this is key to the problem. While it is humorous that the files are being compressed, encrypted, and then compressed again, that is not what is causing them to fail to compress: the first compression should work.
Instead, if we go one level deeper, we can ask the question "what is Apple even storing on this filesystem", and the answer is "maybe one or two hundred megabytes of executable code, and a few hundred megabytes of graphics".
The images are stored as PNG and JPEG: file formats that are already compressed. We therefore would not expect the version in the final output file to be much smaller than that on the filesystem. These files are, in essence, being compressed, compressed, encrypted, and compressed. ;P
The executable code, meanwhile, really doesn't compress well with algorithms like deflate: while it has reasonably low entropy, its encoding looks irritatingly random to algorithms that are looking for sequences of bytes (or bits) that are actually identical, especially over small window sizes.
The problem is that you may see "add one, compare, branch if equal" all over the place, but it is "add one (to X), compare (with Y), branch if equal (to Z)", which breaks up the nice sequence. Even just reorganizing the data bits based on the instruction encoder then helps /tremendously/.
However, it is also often the case that X is one of just a few numbers, Z is one of a small range (loops aren't usually that large), etc.: however, normal algorithms look for "exactly this", not "something similar to this with an offset" (or even switching to a general integer encoder); again, minor details, but it breaks deflate.
...and, indeed, there are better compression algorithms out there already that are designed to handle code well. I swear Google even had some cool stuff for this, but I'm not finding it right now :(. Regardless, a quick (silly) citation for validity:
"""While we have not addressed the compression of machine code, others have shown that it is possible to compress machine code by a factor of 3 using a specially tuned version of a conventional compressor [Yu96] and by as much as a factor of 5 using a compressor that understands the instruction set [EEF +97]."""
So, yeah: I think the key problem is that Apple is not wasting disk space on the device. And, when you put it that way, it is obvious: why would Apple waste 700MB of flash on a 32GB device, space the user would probably really love to be storing music in, when they only have 100MB of entropy?
The answer is: "they wouldn't", and so (modulo the further compressibility of binaries, an interesting and partially open academic problem) the result is that most of the data on the filesystem is already compressed images/audio, and therefore compressing, encrypting, and even compressing again, doesn't matter to the result.
These results pretty much speak for themselves. Just think of it this way, compression works by finding patterns (like every byte is zero), and only storing the patterns. If the encrypted data has patterns, then the plain text could more easily be found.
It depends what you want to show, in this case I was showing the size difference between a file unencrypted and encrypted. An all-zeros file has nearly no randomness, so it compressed very well. Then I show that somehow the encryption process takes away this lack of randomness, and leaves a file almost incompressible. I'm really just increasing the scale to make it more 'dramatic'.
If I happened to be showing how compressed codecs like jpeg, mp3 or h264 weren't compressible, I would definitely pick something more like an actual text file.
A fair jump, but not a huge one. The Adreno 200 is an ES2 part with acceptable performance. I'd be very surprised if there were serious performance problems under ICS, but obviously without having seen the software we can't say.
taking the time to run great on hardware of that capability pays huge dividends in the phone world.
It really won't though, that's the problem.
The Nexus One is nearly two years old. It's somewhat of an anomaly because it'll have a higher than average number of off-contract purchases, but in general once a person's contact has run for two years they get a new phone.
Two year old phones just aren't that important. Speaking as an N1 owner, I'd love for them to be. But they aren't.
in general once a person's contact has run for two years they get a new phone
That kind of market calculus made sense before the recession. Now the value proposition has changed, and while people may still buy newer hardware (such as tablets) they expect better value from their tech purchasing because increasing wealth is no longer a given. Also, people are more suspicious of corporations in general, especially ones that are sitting on a large cash pile. Public sentiment is fickle and can quickly bite a company in the ass - look at how far the stars of Groupon and Netflix have fallen in just two quarters.
The idea that everyone should replace their smartphone every two years says either that your firm is not interested in supplying the less well-off, or else that it doesn't much care whether its products end up in landfill or not. Sure, Apple locks down its hardware platform so tightly that maintaining backwards compatibility is far easier than it is for Google, but note how Apple products maintain a high resale value and that allows them to sell new ones at a premium price point without hurting demand. Apple may be hard to deal with for developers, but Apple consumers feel the firm treats them like Kings or Queens. Google overlooks this market positioning strategy at its peril; loyalty is a two-way street, and the firm should know this very well given its historical disruption of the Yahoo/Altavista/Lycos triumvirate that dominated search back in ~1998.
I use an IPod and I don't feel treated as a King, more like a pigeon: It is my gadget, with my music in it, and I am being forced to use a software I dislike (iTunes) which don't even work on my OS of choice.
You have a weird view on how Kings are treated, or on Apple itself.
That kind of market calculus made sense before the recession. Now the value proposition has changed, and while people may still buy newer hardware (such as tablets) they expect better value from their tech purchasing because increasing wealth is no longer a given.
I'm not so sure about that. You can (probably- I'll be honest, I haven't checked for sure) get a more powerful Android handset than a Nexus One for free if you sign a new two-year agreement right now. There's very little to lose there- except having to take on a two year plan. Except...
Also, people are more suspicious of corporations in general, especially ones that are sitting on a large cash pile. Public sentiment is fickle and can quickly bite a company in the ass - look at how far the stars of Groupon and Netflix have fallen in just two quarters.
I'm extremely skeptical of the former claim- people's anger is directed towards banks, not large corporations with hordes of cash (by and large). If they were, people would hate Apple.
Groupon and Netflix are both things that you can quite conceivably do without. A cellphone these days is seen as an essential, so signing up for a new two year term isn't such a friction point. You're going to need cellphone service from somewhere, after all.
A 2-year contract costs $20/month more than a comparable month-to-month plan. (T-mobile's Even More /Plus plans make this explicit.) That's where the phone subsidy comes from.
There's very little to lose there- except having to take on a two year plan.
Creditworthiness is a major barrier for many people. Many more resent being tied to a single vendor. Hence the increasing market share of non-contract vendors, although they've been taking a beating too because cell service demand at the bottom end is highly elastic compared to food and housing.
If you're an existing customer of a cell phone provider without a contract, I kind of doubt they'll refuse to sign you up to a new two-year contract because of credit quality, as long as you've been paying your bils. It doesn't make sense to drive you to jump to another carrier.
I bought an Apple 3G new in August 2009, after the 3GS came out and it felt like obsolete junk with the next major iOS upgrade, long before the contract expired. I certainly did not feel like I was treated like a king.
Nobody is denying the way 3G users got screwed. I should know, I'm still paying out a contract for my 3G which was boxed up long ago and replaced with a 4. However this situation didn't repeat itself with the 3GS & iOS 5. They learned from that and corrected it, especially now that they are still selling it. iOS 5 runs beautifully on the 3GS.
Point being, all of the Android phones released around the original iPhone and iPhone 3G have most definitely been cut off. However, the Nexus One is newer than the 3GS, not to mention the benchmark model for that generation of Android device, and it has already been cut off.
This is separate from the 3G issue which was bad, but is in the past at this point. It's also arguable that even slow 3G users are better off than users of the same era stuck with Android 1.5/1.6. Hell, just throw ultrasn0w on your 3G and it becomes perfectly usable again.
No, it's correct that iOS 4.0 was a huge performance hit on the 3G: it significantly increased iOS memory requirements, on a phone which was already memory constrained It was unusably slow.
iOS 4.2 improved some things and made it usable again, but the phone was still nowhere near as responsive as under iOS 3.1
On the other hand, rsheridan6 had bought a year-old design.
>On the other hand, rsheridan6 had bought a year-old design.
Well, yes, my previous 3G had been stolen, I needed a replacement right away, and I couldn't afford the premium for the 3GS right then. But if the 3G was intended to be obsolete less than a year after I bought it, it was a dick move to sell it with a 2 year contract. I don't think it's too much to ask that a phone not be obsolete before the contract runs out.
But I have learned my lesson. I'm only buying top-of-the-line from now on. I need to switch to Verizon because I moved to an area where they have the only decent network, and I'm holding out for the Rezound or Nexus Prime (Nov 10th).
But this also favors Android over the iPhone. New iPhones only come out every so often. Suppose your iPhone broke/got lost/got stolen at a time when the current iPhone is 8 months into its lifecycle. Your choices are to sign a 2 year contract for a phone that will be likely be obsolete before the contract runs out, pay an exorbitant cash price, go without a phone until the next release, or buy a cheap phone with no contract and wait for the next release. Or you could buy Android. There's always a latest and greatest Android phone that's not more than a month or so old.
Getting a new phone every two years is unsustainable, not to mention the problem with securing the necessary rare earth metals to continue building them.
I appreciate you acknowledging the products ending up in a landfill.
We really need to change the way we do things not just because of the recession but because it is entirely unsustainable.
I had hope google would recognize this problem but it seems they are more interested in the bottom line.
Those downvotes need an explanation. If you think it's Apple fanboyism, I'd like to point out that I own 3 Android devices, beta-tested the CR-48, and have never owned anything from Cupertino.
All the world is not the US, in most places the incentives aren't set up for upgrading phones "just because". But even ignoring that, the benefits to the platform as a whole for getting to upgrade as many users as possible to latest version should be obvious. And if the top range of 2 years ago isn't good enough for an upgrade, chances are that a lot of the midrange of today isn't either.
(That said, if there really is a performance bottleneck preventing the port, the most likely candidate would be the GPU. And once you've made the decision to finally embrace the GPU, there's not anything you can do to fix the problem of going with a lame GPU when speccing out the hardware 3 years ago).
Supporting a two year old phone might be important to say X% of users, but I'd argue that Android phones include features that are important to Y% << X% of users.
My girlfriend cares way more that her 3GS runs iOS5 than she does that it doesn't have a 4.5" screen.
What does iOS 5 do that she actually needs? My nontechnical friend anecdote is the opposite: she likes the big screen and Swype on the Evo, and doesn't care what OS point version it's running.
This shouldn't be a performance issue. It's a storage issue. The phones from early 2010 made by HTC had very low storage, like 512 MB, where 300-350 was occupied by the OS itself, and the rest was free for the user.
HTC Desire which basically is a Nexus One clone, couldn't even upgrade to Gingerbread+Sense, because there wouldn't be enough free storage left, so they had the choice of either using only stock Gingerbread to upgrade it, or leave it to Froyo+Sense.
Now I assume ICS is even bigger than Gingerbread, so I'm pretty sure it wouldn't fit in there. Now, will we see CM9 with ICS on Nexus One? It's possible, but maybe the CM team will be able to cut what the Android team can't cut from it, and offer a more stripped down version of it.
Android is not a real-time OS and is not designed for embedded devices. the kernel is not real time and java has severe latency problems caused by asynchronous garbage collection. A workaround is to allow the garbage collector to run on a separate core (one of ICS's main feature), but Nexus One's cpu is single core. I guess Google did not have the foresight to design Android to disable garbage collection. as I mentioned before, automatic garbage collection is disabled in iOS/Obj C primarily because of performance issues/battery life.
The Nexus S also has only a single core, but is already scheduled to get ICS, and I actually already have an SDK port running on it just fine. I think the more plausible excuse is that the Nexus One has a pitiful amount of onboard ROM, which already has issues with limited space for both the OS and apps in 2.3. Granted there are SDK ports to the N1 already as well, but I wonder just how much space is available on the device for applications. Even though installing apps to SD card is supported, it's not optimal, and certain apps just won't work that way.
I was explaining why Android is not designed for embedded devices. Sure ICS can run on phones with one core as long as their fast enough but theere will always be lag and delays due to the garbage collector constantly running in the background.
Really? Says who? The imaginary Apple-fanboy strawman?
> also made several hardware updates that left older Macs behind. So who is right?
What's "right" in this context? It's an engineering tradeoff. Considerations:
1) the cost of adding support for newer features for soon to be obsolete hardware
2) the benefits of using the added capabilities of newer hardware with no backward-compatibility restrictions (new sensors, etc).
3) the engineering effort required to make something performant in older hardware with less memory / cpu power.
From a customer standpoint, it's only wrong if the above don't hold, and the sole company's incentive is to artificially obsolete perfectly fine older devices.
Most of abandoned devices out there are by third party Android cellphone makers, who have a lot of incentive of making users just buy the new version. After all, it's their only revenue stream.
Apple, on the other hand, besides being traditionally nicer to its customers (as shown by them toping the user and support satisfaction surveys every damn year), also have the iTunes and iPhone App Store to get money off of customers with older iPhones, so they can afford to give them a new iOS version. And they also use the iPhone as a lure to their other device offerings (iPad, Macs, etc), so the benefit from users of an older iPhone not feeling left behind iOS-wise.
Perhaps the biggest reason is this: Apple still sells older models (such as the 3GS, 4G etc), especially to other parts of the world. So it's an added incentive that those are upgradable to the latest iOS.
Now, since you asked, perhaps Google doesn't want to add to their Android dev expenses the potentially huge expense of supporting older devices.
So here we have Google, arguably a company which can hire all of the worlds best engineers, why are they succumbing to the code first and ask questions later philosophy that gave Microsoft such heartburn?
Andy, I know you read HN, tell the team that their bonus depends on Android 4.0 being runnable on a Nexus 1. It isn't about supporting the platform with the least market share of any Android phone, its about instilling in the team a philosphy that taking the time to run great on hardware of that capability pays huge dividends in the phone world.
Many engineers today were brought up in a world where the next PC will be 1.5 - 2x the 'size' of the current PC so bloat is fine, schedule is king. This is a bad philosophy in the embedded space. That 1.5 - 2x the 'size' phone means you can't cut your phone costs, you can't grow your market, you are stuck waiting only the latest / greatest chips which are always in short supply to ship volume. Help those engineers break the habit, I know they can they are smart folks. Challenge them, as Ben Horowitz would say use your lead bullets on this one.