I've been working on my own version of a literate programming system (https://github.com/adam-ard/organic-markdown). It's kind of a mix of emacs org-mode, jupyter, and Zettelkasten. But, because it's based on standard pandoc-style markdown, you can use it with a much wider range of tools. Any markdown editor will do.
Even though I made it as a toy/proof of concept, it's turned out to be pretty useful for small to medium size projects. As I've used it, I've found all kinds of interesting benefits and helpful usage patterns. I've tried to document some; I hope to do more soon.
The project is at a very early stage, but is finally stable enough that I thought it'd be fun to throw out here and see what people think. It's definitely my own unique spin on literate programming and it's been a lot of fun. See what you think!
Senior system engineer with 25+ years of experience in robotics, embedded devices, and 3D applications. I've worked on everything from humanoid robots and smart energy systems to AAA console games and graphical simulation tools. I’m passionate about developer experience and recently created Organic Markdown (https://github.com/adam-ard/organic-markdown), a literate programming framework. I write about software design and developer culture at https://rethinkingsoftware.substack.com. Looking for deeply technical roles where software meets hardware and math meets the real world.
I very much support this sentiment! If we want a decentralized internet, we need to stop relying on large companies to manage everything for us. Git was designed to be a p2p system, but we very quickly centralized it with forges like Github. It is very discouraging. Most of the internet is like this now--managed by a handful of very powerful organizations. There is no end to the problems this will cause.
There are so many private hosting options for forges at this point that I don't buy the Github excuse anymore. Anyone who actually cares about decentralization can set up a Gitea/Gitlab server (or one of their forks) or use one of the many public alternative code forges. It's also quite common for projects to have read-only copies of their code on Github, where people can easily find them, with links to their real upstream in the README.
These days, Git forges left and right are even working on decentralizing things like issues and comments, something Git doesn't track or care about.
People flock to Github because it's free and easy. Very few people care about the peer-to-peer internet and decentralization that Git was built for.
One thing which is particularly excellent about Gitea (and its fork, forgejo) is that it's quite lightweight for what it does. It's a single-process Go application with low memory requirements, and it can use a SQLite database (so you don't need a separate database server).
> Sometimes the exciting things are exciting for very valid reasons
Very true! I would also add: Sometimes boring things are boring for reasons that actually make them poor solutions.
For example, some systems require tedious, mind-numbing configuration. This is boring, but also a good reason to NOT use something. If it takes hours and hours of manual tuning, by someone with special training, then the solution is incomplete and at least needs some better defaults and/or documentation. It might be a poor option.
Another example is a system that does not lend itself to automation. Requiring manual interaction is certainly boring, but also does not scale well. It is a valid reason to disqualify a solution.
Boring can often be a smell--an intuition that a solution is not fully solving the problem.
I see your point here, but I want represent a counterpoint to this line of reasoning, from my personal experience. I've been in a lot of situations where someone simply wants their way--in this case they want the organization to choose their personal software preference--and so they call it the "boring" choice.
By calling it boring, they characterize their _preference_ as the majority accepted, mature, obvious decision and anything else is merely software engineers chasing shiny objects. The truth is almost always more nuanced. Both solutions have pros and cons. They trade-off different values that resonate more or less with different people.
So, please be careful with the "it's boring and therefore obviously better" argument. Don't let it be a way to summarily dismiss someone else's preferences in favor of your own--without a deeper discussion of trade-offs. Otherwise it's no better than any other condescending attempt (ex. I'm in charge, so we are doing it this way. No one ever got fired choosing IBM/Microsoft/..) to win an argument without having to present real arguments.
> be careful with the "it's boring and therefore obviously better" argument
The same way I would say be wary an argument which can be boiled down to "it's newer so it's obviously better". I used to made such argument myself in the beginning of my career but now I see that proffering everything new is a bad strategy.
Comparing software objectively is hard (if not impossible), and there is a place for personal presences too. But if after a discussion of pros and cons it's not obvious which of two options is significantly better I would be inclined to choose an older and more established technology.
Demagogy is MUCH better than real arguments. Most people you work with will have trouble even understanding the latter. They don't have time, they're overloaded, their kids are waiting at home. Give them the warm feeling that they're doing the right thing. Make them feel smart, experienced,
elite. That's how you get support, not by appealing to (ugh) reason.
I think people are just tired of arguing with people who think you need, say, an entire cluster orchestration system, and a renderer inside your renderer just to serve up some web pages. It's a lot easier to just tell people to use "boring technology". The ones who get it, get it.
Tim Bryce’s work, such as his “Is PRIDE too rigid?” article, uses analogies with traditional engineering (like jet engine design) to argue that skipping critical design steps can lead to costly rework later, but it stops short of offering rigorous empirical proof or extending the idea to the realm of software.
"Big Design Up Front" in software is the whole reason the Agile movement came about. To put BDUF into Agile is to miss the point completely--they are fundamentally incompatible. Any company that insists on "lots of analysis and design cycles before a single line of code is written" will be left in dust by competitors if all their programmers don't quit first.
Oh, and the Agile "movement" of the early 2000s was really nothing more than angry programmers looking to return to the good old days when management was so mystified by the complexity of the machine or the software it ran that programmers could do what they wanted, absent a reasonable semblance of discipline, organization, or accountability.
Tim Bryce also told the story of how he took a COBOL course in the 70s. While the programmers went straight to the keypunch and encountered numerous compiler errors, requiring recompiles, Tim pulled out his "trusty template", actually flowcharted the solution first, and then wrote the code, and it compiled and ran first time, every time. He said "That course taught me more about programmers than it did about programming."
Tim's father Milt explains the industrial approach behind PRIDE here:
Industrial products go through phases of specification, analysis, design, development, implementation, and review, in that order. Can you give me a reason why information systems (which, if you'd been paying attention, are much more than just software) shouldn't be the same? "Agile" sounds like skipping to implementation before the analysis or design have been completed -- going straight to the keypunch before you've documented and flowcharted exactly what you want to achieve. Of course what you're going to get is a buggy incomplete mess. But Agile says "ship the buggy incomplete mess and use your users' feedback to iterate on it!" That just annoys the users, as they just want something that works, but they have to go through many cycles of not-working software to get there.
With analysts (who are emphatically NOT programmers) in the loop, employing a structured, disciplined approach to systems design and implementation, the requirements can be clarified before a single line of code is written, leading to a better result the first time, saving the organization time, money, and frustration. It's just like building a car, or a skyscraper. You need to start with a clear design so the builders know what to build. Anything else is wasting time, money, and effort.
Another way of putting it is: Agile in its original form serves programmer needs while PRIDE serves management needs. Who actually runs the organization? Who has a big-picture view of what the organization's information needs are? Hint: It's not the programmers.
Relevant Bryce's Laws:
"Remember, it's ready, aim, fire; any other sequence is counterproductive."
"A system is a product that can be engineered and manufactured like any other product."
There is a lot of confirmation bias in the industry around estimates. If an estimate is ever wrong, it's never taken as evidence that estimation in general is a flawed practice. The failure is always attributed elsewhere, usually to a lack of effort or focus. But if an estimate happens to prove true (even broken clocks are correct twice a day), everyone pats themselves on the back and says, "look how predictable we are. We are so good at estimation."
The autonomy you gain from starting a business is also a big deal. You can work long and hard with low stress when you are the one making decisions. Burnout comes from being powerless and at the mercy of someone else.
That's for people outside the Scrum team, meaning that no external stakeholder may manipulate the backlog by bypassing the POs.
Internally the team must act as a cohesive unit, which means that a Scrum dev should have the same objectives as a PO, with regards to both priorities and goals. That may mean adding items in the backlog pertaining to technical debt, and so forth
Thank you for listing out the cancer that is Scrum.
Anecdotal story:
Are product owners engineers? Typically not, and this is where the problem starts for most engineering teams doing "Scrum".
I work with overzealous POs (who aren't engineers) and since they "own" the product and the backlog (as according to that definition), they own everything. Since the PO is supposed to manage the backlog, we have POs running backlog refinement sessions where the vast majority of topics and tickets being discussed are of an engineering/technical nature (lots of stuff that is periphery to the product features such as security enhancements, build pipelines, performance, code quality and tests, etc.) and since the PO "owns" it all it is their decision/opinion what counts, i.e. they pretend to be tech-leads.
I even work with a PO who insists of writing all the engineering tickets (title/description), where no engineer can understand what the ticket is about since this person simply doesn't have the ability to clearly and concisely articulate engineering tasks.
Now, obviously people will point out that this is not a problem with Scrum but a culture issue with the organisation, and I agree (my organisation has many ingrained and deeply embedded institutional dysfunctions) but the Scrum definitions aren't helping engineering teams.
I think we have been stuck with this for so long we have forgotten that any other way is possible. Scrum is just a complicated way to divide labor. No one else at a software company needs a formal process to partition their work. They just sit down, talk and then go do the work. They come up with a light weight way -- a informal process -- to get stuff down. They would never even think to ask themselves, "I wonder what formal process we can follow to assign work?" They just do it.
Sure, and that works great if you have a small, motivated, experienced, and disciplined team with a deep understanding of customer needs. But that is not the reality in most real organizations that must reliably deliver software. They often need to get productive work out of inexperienced, undisciplined developers who don't personally know what the customers even want. So Scrum or something like it ends up being the least bad option that actually works.
You might argue that companies should hire more competent developers who fully understand the business domain issues. But there are few of those available at any price, especially in complex fields like healthcare.
Even though I made it as a toy/proof of concept, it's turned out to be pretty useful for small to medium size projects. As I've used it, I've found all kinds of interesting benefits and helpful usage patterns. I've tried to document some; I hope to do more soon.
--https://rethinkingsoftware.substack.com/p/the-joy-of-literat...
--https://rethinkingsoftware.substack.com/p/organic-markdown-i...
--https://rethinkingsoftware.substack.com/p/dry-on-steroids-wi...
--https://rethinkingsoftware.substack.com/p/literate-testing
--https://www.youtube.com/@adam-ard/videos
The project is at a very early stage, but is finally stable enough that I thought it'd be fun to throw out here and see what people think. It's definitely my own unique spin on literate programming and it's been a lot of fun. See what you think!