The tech industry, where just about everyone is capable of cleaving off aspects of a problem and reasoning whether those aspects apply to the general case do not just fail to apply serious scrutiny to certification and licensing schemes in other fields but tend to actively go out on limbs to defend them.
I don't know what that says but it sure says something.
The farm bill makes 'hemp' anything with below 0.3% THC legal. For this reason, we have a LOT of testing on the THC content of cannabis, since it is required to sell and manufacture. As it turns out, naturally cannabis quite commonly has >0.3% THC even before heating or activation of THCa.
Any human-like animal with our receptors eating a large amount would get high as fuck, cooked or not. A ruminant eating pounds of the stuff raw, would not be that different from a human consuming an ounce of baked pot.
And yet someone IS pushing code to these devices. Every single one.
So the question really becomes - Are these people working on their own pumps with open source more or less invested than the random programmers hired by a company that pretty clearly can't get details right around licensing, and is operating with a profit motive?
More reckless as well? Perhaps. But at least motivated by the correct incentives.
You, an engineer at a major aircraft manufacturer that isn't Boeing, have been working after hours with some of your colleagues on a hobby project to add some modern safety features to an older model of small private plane, because you regard it as unsafe even though it still has a government certification and you got into this field because you want to save lives.
Your "prototype" is a plane from the original manufacturer with no physical modifications but a software patch to use data from sensors the plane already had to prevent the computer from getting confused under high wind conditions in a way that has already caused two fatal crashes.
Now you have to fly somewhere and your options for a plane are the one with the history of fatal crashes or the same one with your modifications, and it's windy today. Which plane are you getting on?
Are you kidding me? How many times have you unwillingly introduced bugs into a code base you didn’t fully understand? That’s basically table stakes for software engineering.
> Definitely not the untested code I wrote myself!
Nobody said it was untested.
> How many times have you unwillingly introduced bugs into a code base you didn’t fully understand? That’s basically table stakes for software engineering.
Which applies just the same to the people the company hired to do it, and now we're back to "the people with a stronger incentive to get it right are the people who die if it goes wrong".
I can’t tell if you seriously think a random person writing code in their basement is equivalent to a company that has access to API docs, design specs, actual test hardware, the expertise of a ton of engineers that have worked on the project and understand how it can go wrong, not to mention all the regulations and verifications they’re subject to.
But if you do then wow. That really puts in perspective the kind of people that use hacker news. I’m gonna be more selective about who I bother replying to going forward.
> I can’t tell if you seriously think a random person writing code in their basement is equivalent to a company that has access to API docs, design specs
Are you saying not having those things is dangerous? They should be required to publish all of that for safety-critical devices then.
> actual test hardware
Why would arbitrary people be unable to buy test hardware? Again something to be addressed if true rather than used as an excuse.
> the expertise of a ton of engineers that have worked on the project and understand how it can go wrong
Do they not have internet access? If they don't even work for the company anymore then that could be the only way to access that information.
Literally something which is happening on the linked Reddit page.
> not to mention all the regulations and verifications they’re subject to.
Regulations are for preventing someone else from harming you. You don't need a government incentive to protect you from yourself, you already come with that incentive.
Tested how? With 100% "unit test" coverage? I can certainly see how a random person on the internet might be highly motivated and actually talented enough to contribute to these sorts of projects. But they don't have the budget and resources that commercial entities have. They don't have the same due diligence requirements. They don't have the same liability. If I use a commercial device unaltered, it's the company's fault if the device fucks up or is defective and causes harm. If I install random internet software on my medical device and it fucks up and causes harm, it's my fault.
I say this as someone who might modify my own medical devices because I'm so fucking jaded over the capitalist march towards enshitification and maximizing profit over human lives. There is simply no way random folks on the internet can test these types of systems to any reliable degree. It requires rigorous testing across hundreds to thousands of test cases. They at best can give you the recipe that works well for them and the few people that have voluntarily tried their version. That doesn't scale and certainly isn't any safer than corporate solutions.
Why do people think constantly something made by some random company is automatically better than something made "DIY".
I totally understand, that because of liability and some more availability of resources, you would expect a company product to be "safe". BUT: if it is your butt that is going to be in the line, then I bet you: you will be much more careful that a random engineer in some random company. About the resources available in a big company, they are usually more directed to marketing, legal (including lobbing to avoid right to repair) and oder areas to maximize revenue, and not exactly in quality.
I worked in 2 different big companies which worked in "mission critical systems" and boy! I can tell you some stories about how unsafe is what they do, and how much money is invested in "cover your ass" instead of making products better/safer.
I thought I explained it, but I'll break it down into smaller words. Medical software doesn't just have to solve one particular users's problems. It has to be generalized to the majority of folk seeking treatment for a particular problem. If one particular CPAP user is able to tweak their settings to work better for their particular lifestyle, it is not generalizeable to every CPAP user. A corporation offering a general solution is put under *far* more scrutiny than a random github repo is. A corporation can be sued for releasing a product that kills people, but good luck convincing a court that your family deserves restitution for you installing a random script you found on the internet into your insulin pump.
This has fuck all to do with how much corporations care about people. It has everything to do with liability laws and how victims can get restitution. It has everything to do with the actual risks of installing random internet scripts versus the corporations who have to jump through regulatory hoops. And it's not to say corporations get everything right. They fuck things up constantly. But they fuck things up constantly with oversight and regulation and you want me to believe random internet users will make a better product without it. It's nonsense.
I have explained it already in other comments, but let me break it down for you again:
The “liability”, “scrutiny”, “regulation” only generate “cover your ass” measures, bureaucracy, red tape, costs, and hardly any real measure to increase quality or safety. My work is in such a critical mission systems company, and they don’t give a shit about safety, just are interested in coming out clean or not waste too much money in settlement with dead people relatives.
> but good luck convincing a court that your family deserves restitution for you installing a random script you found on the internet into your insulin pump.
And good luck fighting a Pharma corporation for whatever did wrong. BTW, you bring the CPAP topic. Maybe you can read this at leisure [1] in this case, because it was a huge scandal, they pay. But 90% of the time, they don’t. And even if this case, with legal cost deducted, and divided by all people, is not a real compensation (spoiler alert: it never ever is!).
Please note in this case they DID KNOW about the issue, and did nothing. So much for liability and scrutiny.
This is fucking retarded. Liability isn't just CYA. It's real fucking consequences when someone dies. From your own fucking source:
> Philips Respironics agreed to a $1.1 billion settlement on April 29, 2024, to compensate people for financial damages related to the recall.
Which open source individual contributor will agree to a $1.1 billion dollar settlement because of wrongdoing? Not a single fucking one because those numbers don't make sense when random internet users are promising salvation if you just download their firmware. What a complete crock of shit you're suggesting here and you're just reinforcing my point. Did you even do the barest amount of critical thinking here?
> But they don't have the budget and resources that commercial entities have.
Everyone is standing on the shoulders of giants. You're not going from stone tools to jet engines in a month, but you could fix a bug in one in that time.
> They don't have the same due diligence requirements. They don't have the same liability.
Things that exist to try to mitigate the misalignment of incentives that comes from paying someone else to create something you depend on. Better for the incentives to align to begin with.
Notice also that these things are floors, not ceilings. The company is only required to do the minimum. You can exceed it by as much as you like.
> If I use a commercial device unaltered, it's the company's fault if the device fucks up or is defective and causes harm. If I install random internet software on my medical device and it fucks up and causes harm, it's my fault.
And then if the community version fixes a bug that would have killed you and you stick with the commercial version you can sue them for killing you. Except that you're dead.
> There is simply no way random folks on the internet can test these types of systems to any reliable degree.
Basically the entire population is on the internet, so the set of them includes all the people doing it for a corporation. Are they going to forget how to do their jobs when they go home, or when they or a member of their family gets issued another company's device and they want it to be right?
Flying in a plane you built yourself is likely safer than flying in the same model of plane built by a company that assembled it for you using lowest-bid labor while making you sign a twenty page lawyer barf disclaiming liability.
See my other follow up comment ("same model"). Medical device software development feels much closer to homegrown (or worse) than aeronautical engineering.
Why do you think a random person, who is VERY passionate about something, as to invest all the free hours in life to do something, is less skilled that one who just does it because is needed to survive?
Sorry. I would be much more inclined to have something made by somebody passionate about it, as done by some guy that received hopefully some kind of instruction on how to do things and was then left alone.
In this context (GA) we are not comparing Airbus/Boeing with a garage build. We are comparing some small company making 2 seaters with your hangar and maybe 10 certified aircraft mechanics that will help you a lot on the process.
And why do you think pathos arguments are logical? Granted, they didn't cite them, but assuming it is true, empirical studies showing the accident rates are the logical point from which to draw conclusions. What you would like, how you and others feel about it, and what you would expect are meaningless.
You're also equivocating. They made it extremely clear they are referring to hobbyist and other such groups with vague or unknown qualifications; whereas, you go in and make stipulated claims about small businesses with certified mechanics, etc. These two are clearly not the same category, making your argument non-responsive. It's also contradictory in terms of discussed liabilities and such, as the small company, and its mechanics, that whoever worked with, would have liability as well, as opposed to the "random git repo".
You write that as if you have ample experience with codebases of medical devices and I'm going to take a stab at this and say that you don't. Prove me wrong.
My comment rests on the fact that the types of planes you can build yourself are completely different models than the fully assembled models from the likes of Boeing etc. I do agree that a kit 737, if such a thing existed, would be less safe than one off the line.
I think the Beechcraft Bonanza deserves special mention here. I'm sure all the people that worked on it were experts too!
The big problem with this analogy is that it conflates three very important things:
- GA is more dangerous, period. Doesn't matter whether you build the plane yourself or if you bought it ready made (hopefully new, hopefully very well maintained if second hand)
- GA craft tend to have less experienced pilots than airliners, but even airliner pilots tend to do worse as GA pilots than when they're at work. The reason for that is simple: the processes are what keeps commercial aviation (mostly) safe.
- GA craft tend to kill the pilots, because they are more often than not the only person on the plane.
- GA craft have malfunctions like larger aircraft, there is nothing special about them in that sense. But there is something that they don't have that larger aircraft do have: redundancy. In electronics systems, in the design of the mechanical bits, and finally in the people.
- GA craft that are designed and built by their operators are experimental class for a reason: they are untested and so more likely to fail than the ones that are certified. The design processes for commercial aircraft are nothing compared to the design processes employed by what we'll call hobbyists to distinguish them.
- And finally, even though it is a fun analogy I only meant it from a skin-in-the-game point of view, a GA hobbyist is still going to do his level best to make sure that he's not going to get killed. Boeing executives only care about the bottom line, safety is a distant second. And based on my experience with the difference between the guts of various bits and pieces of avionics and the software that they run on compared to my experience looking at medical devices, their guts and the software that they run on I would be more than happy to bet that the loop hackers know as much more more about the failure modes of these devices as the manufacturers do.
Cleanroom manufacturing under sterile conditions is the main differentiator here, and that just applies to the hardware, and it is an art that the medical industry understands very well. Electronics is already at a lower level of competence and their software knowledge tends to be terrible, not to mention the QA processes on said software.
Programmers working for corporations don't necessarily suddenly grow an extra quality brain when they do their work.
Now look at something like the Bede BD-5 and see how many of it's amatuer builders IT killed. Death rate on the first flight alone was something like 10%.
PS: AIrcraft aren't assembled in cleanrooms.
Frankly, you don't have a damn clue on and are getting basically everything wrong in the process
You can believe it and simultaneously function in society.
We aren't all building our own planes because it's worse, but because it's time consuming. I don't have 20,000 hours to burn learning about how planes work to make my own.
If we magically beamed the knowledge straight into people's heads and also had a matter fabricator, I'd imagine yes - everyone would build their own plane. And it might be safer, I don't know.
Point is, the ideas are not mutually exclusive. You can believe both and still resolve it internally and with the world
Not the original poster, but that was snark and not meant literally.
Also, building your own plane is absolutely worse, even if you do have expert-level knowledge. That's true for any complex design. Aircraft design, material sourcing, fabrication, assembly and quality control are all very different skill sets, but the real kicker is experience.
The reason why commercial aircraft are so safe is a lot of work goes into investigating and understanding the root causes of accidents, and even more work goes into implementing design fixes and crew training.
The problem is that the system incentivizes incompetence. The mechanics who are paid a skilled wage, take their time, and double check to make sure they are not missing anything show up as big red problems on the beancounters' spreadsheets and get optimized away.
The system can make up for this in other ways like repeatability of processes, redundancy, etc. Which is why commercial aviation is safer than general aviation, and also why I specifically worded my comment as being about the same model of plane - ie if instead of building your own experimental-class kit plane, you hired it out to a liability-limiting company hiring minimum-wage workers to follow the directions. I'm guessing such a thing is illegal per FAA regs, but that kind of proves my point.
For another example, have you experienced the medical system lately? Doctors are generally smart people, but that intelligence is squandered by having their attention smashed into 10 minute chunks, with the entire rest of the system revolving around blame passing - the end result is a lot of smart and well-meaning people ending up grossly incompetent through emergent effects. I would much rather be able to go to a doctor and trust whatever answers they gave me rather than having to do my own independent research and advocacy to drive the process. But that is not how the system we have works.
I don’t even disagree with you about the system incentives. I hate capitalism just as much as you!
But I still trust the institutions around me to keep me safe. Obviously that depends on where you live, I wouldn’t feel the same way if I still lived in Brazil.
Last time I went to a doctor was about 3 years ago. They diagnosed me in 5 minutes, and took another 10 to treat me and write me a prescription. It was great, I loved it.
Sounds like you have this trust issue with lots of different areas of your life, it might be worth reexamining your own perspective. Or maybe you just have to move to somewhere that you do trust.
I'm glad for you that you've had good experiences so far! "Diagnosed me in 5 minutes" doesn't sound like anywhere near a complex medical issue though.
I certainly keep trying to obtain good results from the system, ie extend trust, but situations routinely run aground. Can you really say it's a "trust issue" when the problem is that I dig into details of situations and repeatedly discover how so-called professionals abjectly drop key issues on the floor?
Latest example: I need a new dishwasher. I should be able to read some reviews, spend $1k, and get the problem solved, right? Guess again - first delivery, a dent (crease) in the tub from the thing being slammed so hard that its plastic frame deformed and pushed up into the metal tub. Second delivery - loud noise from wash motor. I try to engage with warranty service figuring I'd be fine with them swapping the whole pump assembly. Nope, the guy that comes can't even be assed to do his job either! "Oh that's normal so there is nothing to fix, this is a good model, you should keep it". Third try, wash motor sounds a little better but still has a problem. The third set of delivery guys didn't even take away unit #2 for the exchange (even though I even pushed back when they said someone else was going to come later). I had wanted to simply pay money to solve the problem, but instead I'm left with two noisy dishwashers and a big ole project in my court. (do I keep pushing this exchange button? do I just order a new pump assembly and fix it myself, considering the bonus dishwasher compensation for that? do I say fuck it to the whole brand and rethink the purchase decision?)
Sure, I could drop my standards here, check out, and stop caring about the details. The dented tub probably wouldn't leak a decade down the line, the loud motor isn't really that big of a deal if I only run it overnight, and if the motor needs replacing in a few years it's only a $200 repair. But should not giving in to this "best effort" service (after paying $1k) really be considered a "me" problem? It seems more like an economy problem, with me only being exceptional for noticing, having some expertise on how these things should function, and having the willingness to push back.
(although I am thankful that the thing in the front of my mind that I'm frustrated with is an appliance rather than dealing with the medical system again)
I don't actually find it a particularly strange interpretation.
Here's another lens:
I install cabinets in your kitchen. Your loved one trips, hits the cabinets, breaks their neck and dies.
Should I be liable in this case as well? I did a thing that was involved in harming your loved one... if the cabinet hadn't been there, they might not have died.
---
In both cases, it's pretty clear that there's no intent to harm your loved one. At best you're arguing that it was "foreseeable" that hitting a baseball might harm someone, and that it wasn't "foreseeable" that installing cabinets would harm someone.
But clearly that's ALSO wrong, because we know people have been hurt hitting cabinets before.
So clarify how you'd assign blame in this case, and why it's different from the baseball case?
Basically - your stance is that risk is always a decision someone has made, but I find disagrees with my intuition. Risk is an inherent part of life.
And when we're talking about things in this realm, the general saying is "The dose makes the poison"... Water will kill you if you drink enough of it.
And we do have all sorts of studies showing that harm from these substances isn't immediately apparent (they all have safety sheets, and maximum safe exposure levels) . What we're missing, mainly because it's just incredibly hard to ethically source, is long term studies.
So the question you're really asking is "what's your tolerance to risk?". I think it's fine to have different governing bodies take different stances on that scale. What's less fine is failure to act on information because of profit motives.
Long story short - this isn't so simple. You bathe in chemicals all day every day.
I daresay that the issue is less about "chemicals" and more about "new chemicals". If a substance already exists in nature and has been in use for a long time, then it's reasonable to take the position that it is probably within harm limits. If it's a newly synthesised/extracted substance, then it should be subject to reasonable testing.
Also, if a chemical is known to be toxic, then rigorous testing should be performed before allowing it to be widely distributed and used.
> If a substance already exists in nature and has been in use for a long time, then it's reasonable to take the position that it is probably within harm limits.
Reasonable, but wrong.
Simple case: Did you know that occupational sawdust exposure is strongly associated with cancer in the paranasal sinuses and nasal cavity?
There's also some pretty compelling evidence that coronavirus's (so common cold & flu) are associated with dementia/Alzheimer's.
Alcohol increases cancer rate more than some of the "chemicals" people will complain about. So does Bacon. So does sunlight.
All of which have been floating around in Human contact for a LONG time.
Again - we do a pretty good job at filtering out the stuff that's fast acting and harmful. It's just really difficult to tease out information that requires long term monitoring and involves small/moderate increases in risk.
Think about how long it took us to figure out that lead exposure is really nasty. We used lead for thousands of years prior, and it's literally a base element.
---
As for
> Also, if a chemical is known to be toxic, then rigorous testing should be performed before allowing it to be widely distributed and used.
No one is arguing otherwise, and normally large and expensive studies are done on short term harm (extensive animal testing). But you tell me how we can reasonably and ethically do longitudinal studies on large groups of humans to determine if a new substance is going to cause small/moderate cancer rate bumps over 50+ years?
This is just genuinely a difficult problem to address, and it's not simply like we can go "wait 50 years and see"! Because usually we're trying to use these things to address existing problems. Ex - pesticides and fertilizers might still be net positives even with the cancer risk - do we avoid them and let people starve today? Or feed everyone now and have a 10% bump in cancer rates 50 years later? There's no golden ticket here.
Personally, I have a Roomba I bought in Jan 2019 that's still doing just fine (So 7 years now).
Most of the parts are pretty easily replaced (genuinely pleasant surprise, as an aside) and the company stocked most replacement parts for a long time - I just checked again and I can still get parts for my model (I-series) incl batteries, wheels, brushes, filters, etc. Which is less than it used to be, but still enough to keep mine rolling around for another 3ish years without any likely problems.
And that's outside of the whole "unofficial" replacement parts ecosystem that popped up online.
3 years doesn't track with my experience on this one. I'd bet it's 5 to 10.
---
For context, Amazon tried to buy them for exactly the same purpose ~2 years back (home/house data) and failed to get EU regulatory approval, so scrapped the deal.
I'm not thrilled to have ownership transferred to another company (I was also very unhappy to hear the Amazon rumors back then) and I think this is a pretty clear risk.
Even if a user is no longer using the device, Roomba still likely has plenty of data about their home floating around.
> Most of the parts are pretty easily replaced (genuinely pleasant surprise, as an aside) and the company stocked most replacement parts for a long time - I just checked again and I can still get parts for my model (I-series) incl batteries, wheels, brushes, filters, etc. Which is less than it used to be, but still enough to keep mine rolling around for another 3ish years without any likely problems.
I've never owned or really used a different brand than roomba (I've joked that I've owned 4 roombas, but never purchased a single one...) but I fully agree that the modular nature of their parts replacement is a super welcome thing. The fact that the electrical contacts are all just sprung into each other, and each component is basically designed for near-minimal replacement overlap (not replacing things that are not broken) is something that I would LOVE to be implemented in more things. I always assumed that it was this 'forward thinking' design that a) Likely added a bit to the cost of the brand b) Likely didn't assist with future sales from breakages, etc.
Out of the 4 I've acquired over the years, one has been stripped of parts and discarded. One is relatively in that process, and the other 2 are happily (?) doing the different areas of my house. A few amazon batteries later (Which I originally only charge when I am home and able to check on them, then place faith in 'not burning down the house') and everything is hunky doory.
Also, they have been around so long, there are a boatload of 3d printed replacement parts floating around that can be quite useful if one has a 3d printer.
I've always held them in pretty high regard for repairable tech.
In my first job out of school, I did security work adjacent to fortune 50 banks and the (now defunct) startup I worked at partnered some folks working on Pindrop (https://www.pindrop.com/).
Their whole thing at the time was detecting when it was likely that a support call was coming from a region other than the one the customer was supposed to be in (read: fraudulent) by observing latency and noise on the line (the name is a play on "We're listening closely enough to hear a pin drop".)
Long story short, it's a lot more than just the latency that can clue someone in on the actual source location, and even if you introduce enough false signal to make it hard to identify where you actually are, it's easy to spot that and flag you as fake, even if it's hard to say exactly what the real source is.
The "fancy" hardware is going to get dirt cheap, and in a game where you're asking your customer to trust you with their lives, reliability is going to win. Combine that with time to market, and Tesla feels like a pretty clear "risky bet" at best... Maybe they make it work, but they have to do it before the other companies make lidar cheap, and prices have fallen dramatically over the past 10 years, for much better hardware.
Yeah, that is all reasonable. I think the jury is still out on if sensor fusion can really get far enough up the march of nines (will it work in 99.99 percent of scenarios)? Karpathy has given some good interviews about why Tesla ditched the sensor fusion approach and switched to vision-only.
Same can be said for vision-only, of course. Maybe it won't every quite get to 99.99.
They might still be able to scam folks into taking the test, but the test itself has essentially no meaningful value in industry.
Personally - I see "Agile certifications" as the same thing but from the last decade.
reply