Are you asserting that we gain better insight into the reliability of a system by thinking about it deeply rather than by observing it perform its function? Because I don't believe that for a minute.
I'm not saying you can get by without thinking, but it's difficult for humans to estimate the reliability of a complicated system. Reality, though, has no problem doing it.
Plus, I think your analogy is flawed. NASA surely has a more hands-off approach on the CCP than on the Shuttle, but to say it's hands-off is misleading. They do have a lot of access.
No, I'm not saying by "thinking about it" (although that has its place). Everything I listed is a form of testing. But there's a distinction between iterative testing at a lower level, and end-to-end testing. Again, both have their place.
Take the example of the F9 strut failure. They could have tested the material outside of the final test configuration and saved themselves a lot of trouble. They chose to forego that testing, and instead 'tested ' it as part of part of their launch configuration. (I put it in quotes because it's not clear to me that this was a conscious testing decision).
There’s also a difference between “we’re not completely sure of the fundamental principles, but our testing indicates it works” and “our testing indicates it works and we have a solid understanding why”. The latter allows you to know the limits of your application much more readily. The risk in the former is that you don’t know what you don’t know, so you can never be wholly sure if you’re good or lucky. And luck can be fickle. And this is also where rapid iteration can lead to issues: the more you change, the less sure you can be about whether your results are attributable to luck.
>Plus, I think your analogy is flawed. NASA surely has a more hands-off approach on the CCP than on the Shuttle, but to say it's hands-off is misleading. They do have a lot of access.
They have many engineers who want more access and are effectively told to back off because it's not their place in this type of contract. So I'm not saying they have no access, I'm saying they have very limited access by comparison. It would have been better if I made the analogy that they get the results to a small select number of tests, but not all.
I'm not saying you can get by without thinking, but it's difficult for humans to estimate the reliability of a complicated system. Reality, though, has no problem doing it.
Plus, I think your analogy is flawed. NASA surely has a more hands-off approach on the CCP than on the Shuttle, but to say it's hands-off is misleading. They do have a lot of access.