Until now I had not found an eloquent way to express that emotion, but you come close.

Somewhere around the time node got very popular, I started to notice a lot of impeccably branded (is that the right word? trendy maybe?) websites with tutorials that used all the right buzzwords to get me interested. Once I'd step through the content, it'd be very low quality. Despite the smooth lines and round edges, a lot of them were riddled with inaccuracies, assumptions, unabashed spelling and grammar issues, stolen content and lots of candid offtopic observations.. Not to mention they all loaded very slowly.

I used to joke about it mockingly during the early node days that you could judge the trajectory of a project based on the ratio of bytes dedicated to persona and branding vs actual content. Anything passing the 1:2 ratio generally didn't last long.

I am optimistic though. We are still in the early days of server-side javascript. It took 20 years for the greater php community to coalesce around a handful of quality libraries, frameworks, and solutions to things like user management instead of everyone rolling their own or using things like '$_GET["pass"] == "foo"' they they grabbed from a google search. Node is barely over a decade old. We are already seeing patterns mature in the node ecosystem and I hope things keep progressing that way.

Many frameworks or languages were fairly well developed by a decade's time. Node is 13 years old; compare it to Rails circa 2018.

I don't think PHP makes for a great comparison, because the web ecosystem was still growing. Node was birthed in the age of Github, social media, Stack Overflow, and Google. PHP was released in 1995, when search engines were still stuck in their infancy and only a small percentage of the world was even using the WWW. 20 years in PHP's time is like maybe 5 years from 2009.

Another point about PHP: Wordpress, the largest deployed PHP app, was released in 2003, PHP's 8 year mark.

> It took 20 years ... Node is barely over a decade old. We are already seeing patterns mature in the node ecosystem and I hope things keep progressing that way.

But... much of the earliest web work (PHP, etc) occurred in the early days of search. Example of quality code, best security practices, etc all were fairly .. rudimentary and hard to find.

Just because it took 20 years starting 20 years ago doesn't justify 20 years starting from 10 years ago. There are infinitely more and better resources for just about everything these days.

> early days of server-side javascript

Err ... Netscape's LiveWire introduced server-side Javascript in 1996 even before Java became widely used on the server side. The module convention, the (synchronous) core modules of Node.js, and its canonical http middleware API and express.js/Connect/JSGI is from the CommonJS initiative, a co-op of 2000's SSJS framework developers [1].

[1]: https://www.commonjs.org/

This is an "um ackshually" reply. Nobody used LiveWire and it died a swift death.

And don't "um ackshually" me about saying "nobody." You know what I mean.

Classic asp also supports JavaScript, it is old and wide spread in the bowels of legacy sites.

I thought of node as a victory for async more than anything else.

> I thought of node as a victory for async more than anything else.

In terms of server-side development? Async is rather useless on the server side. Can't generate a web page or JSON blob with the results of a database query until you actually get those results from the database. I can see the application for something like websockets but I presume most Node sites aren't using those.

I figured its success was due to V8 being reasonably fast, plus a generation of front-end-centric developers coming along who had learned to use JS quite heavily on the front end and didn't realize or didn't care that better options already existed on the back end.

> Async is rather useless on the server side [...] its success was due to [...] a generation of front-end-centric developers [who] didn't realize [...] that better options already existed

You'd be wrong, again. According to Ryan Dahl, creator of Node.js [1]:

> I think the combination of Chris Neukirchen’s rack plus how Nginx structured its web server with non-blocking IO led me to start thinking about how those two things could be combined. [...] So, when V8 came out, I started poking around with it, and it looked fascinating [...] and suddenly, I clicked that: oh! Javascript is single-threaded, and everybody is already doing non-blocking. [...] I think JavaScript plus asynchronous IO plus some HTTP server stuff would be a cool thing. And I was so excited about that idea that I just worked on it non-stop for the next four years.

[1]: https://mappingthejourney.com/single-post/2017/08/31/episode...

> In terms of server-side development? Async is rather useless on the server side. Can't generate a web page or JSON blob with the results of a database query until you actually get those results from the database.

This is precisely why async (well, cooperative concurrency generally) is useful, whether on a server or otherwise. Blocking for things you can’t do yet is a waste of idle resources. There are plenty of other concurrency models which are also useful, but yielding/suspending when idle is almost always better than not. And the cases where it’s not generally have other problems.

I don't understand. What can a Node app do in between making a database request and when the response arrives?

Make other database requests, or return the responses derived from prior requests. Or just anything else waiting in the event loop queue, including queueing more work.

You can think of the concurrency model like green threads with only one thread available, or actors in a single system process.

Edit to elaborate: yield/suspend is key here. The way it works is yielding/suspended calls have their outstanding work put behind the extant queue which proceeds until each next queued routine suspends or returns, and once that set of blocking work completes the routine which suspended can proceed until it yields again or returns. The obvious pathological case is long running work which doesn’t suspend. Which if it’s I/O or idle-heavy should be refactored to yield, if it’s compute-heavy it should be treated as an optimization or scaling target.

I'm trying to envision a situation where you'd want to make more than one database request at the same time. Like I guess on a blog or something, you could load the user record of the current user and then load the blog post details at the same time, but then there's a chance that the user doesn't have permission to view that blog post and then the request for the post has gone to waste, right? It goes against my instincts to kill a request as quickly and directly as possible if something can't be accessed.

I don't know, I'm willing to accept that maybe I'm just not used to thinking of developing back-end web apps that way. But async still just doesn't seem as useful as it would in a GUI desktop app.

It is about multi request concurrency/parallelism.

If 20 users log in at the same time node and php would behave very differently.

Php would start 20 processes/threads and execute each in independently.

Node would dispatch 20 Request events to the event loop in the same process.

I will not argue the advantage of server-side async, but here it is clear that node needs it, as those 20 database queries will be executed from the same thread. Without async the 20th user would have 20 times as much latency as the first.

> I'm trying to envision a situation where you'd want to make more than one database request at the same time.

Because I have two requests active at the same time and the second one which arrived shouldn’t wait for me to stand around doing nothing while I wait for the database to serve the first.

> Like I guess on a blog or something, you could load the user record of the current user and then load the blog post details at the same time, but then there's a chance that the user doesn't have permission to view that blog post and then the request for the post has gone to waste, right? It goes against my instincts to kill a request as quickly and directly as possible if something can't be accessed.

Here one user represents 2+ events, both of which should fail. This wasn’t part of my original explanation but yielding does give you the opportunity to fail both simultaneously. Albeit in this case it’s just concurrency broadly. Why is that against your instincts, if you know that one request’s failure implies the other will also fail? Why should that user wait in line to get the same error after another cycle of the same trial?

> I'm just not used to thinking of developing back-end web apps that way. But async still just doesn't seem as useful as it would in a GUI desktop app

Yep I get that. And async takes a bit to get used to in general. The way I think of it is: I have two kinds of work which need to be done, and a long line asking me to work. Some of the work needs labor (CPU), some needs patience and communication (IO). Everyone in line needs some of both.

If I do the first person’s immediate labor and wait for further communication, persons 1 and 2 are waiting for me to proceed. If I do the first person’s labor, send off an asynchronous request for feedback on that, I can do N people’s labor before I get any response. As long as no one spends too long making anyone else wait in queue everyone moves as fast as the queue’s capacity.

> Here one user represents 2+ events, both of which should fail. This wasn’t part of my original explanation but yielding does give you the opportunity to fail both simultaneously. Albeit in this case it’s just concurrency broadly. Why is that against your instincts, if you know that one request’s failure implies the other will also fail? Why should that user wait in line to get the same error after another cycle of the same trial?

I really have no idea what you're saying here, so let me put it this way. I would code this hypothetical blog page access situation this way in PHP: Get user data from database. Do access checks on user data. See user does not have permissions to view blog article. Show 403 page.

Assuming I'm understanding what you're saying about async database requests, this is apparently "correct" in Node: Request user data and blog post data from database. If the blog post data comes first, wait. When the user data comes, do access check and show 403 page. If the blog post data hasn't come yet… it just gets ignored or something, I guess. Otherwise the data just gets thrown out…?

What's against my instinct is to make the request for the blog article data unless and until it is known that it will be needed.

A typical Node service would probably be designed very similarly to your description of the PHP equivalent. The difference in a typical scenario is that, should another request come in while the first query for user data is in flight, the same process would initiate its first query for user data concurrently. Both requests are still pending but suspended until the IO they depend on is available (and until whatever other synchronous work on the event loop yields or completed).

The scenario you describe, performing multiple queries for the same request in tandem, certainly isn’t incorrect. It’s a less common design, but one which might make sense depending on the workload. An example where I’d choose that design: both queries are slow individually, but not significantly slower queried concurrently. Even if the query for blog post data turns out to be wasted work for one request, it might halve the time to serve another request.

When I say which is more common, it’s probably more a matter of the mental model of the human who implemented it. Fulfilling a request as a series of steps versus a set of concurrent steps only becomes a matter of correctness when one of those steps is dependent on another (and even then only if there’s no other reasonable way to address their dependency afterwards).

I’ll add that designing for concurrency this way often leads to better design overall. It helps guide towards minimizing dependencies (or at least minimizing shared state) between functions. Which in turn isn’t just easier to reason about, but also easier to scale should you want (say) to move some expensive function out to its own service or worker.

Okay, then it sounds like we've been using a different definition of "async" all along then.

Why do you say that? It’s definitely async on Node, even if a single request/response is entirely sequential.

Maybe some pseudocode will help illustrate. Here is a hypothetical[1] implementation of the Node equivalent of the PHP design you described:

  const handleRequest = async (request) => {
    try {
      const userData = await getUserData(request)
      const blogData = await getBlogData(request)

      return whatever(userData, blogData)
    } catch (error) {
      return errorResponse(error)
    }
  }

In this function, both `await` expressions suspend the current call and yield to the event loop, until the `Promise` returned by the awaited function resolves. During this time, Node will be idle and able to accept additional requests on the same thread.

But because, in this hypothetical implementation, `getBlogData` doesn’t depend on `getUserData`, it could be written this way too:

  const handleRequest = async (request) => {
    try {
      const [ userData, blogData ] = await Promise.all([
        getUserData(request),
        getBlogData(request)
      ])

      return whatever(userData, blogData)
    } catch (error) {
      return errorResponse(error)
    }
  }

Both implementations are functionally equivalent, but the latter speculatively risks wasted work for the potential benefit of IO concurrency in the non-error case. This is a common performance optimization technique for single-threaded runtimes like Node, where `await` might represent a significant portion of the response time.

To your original wondering, why `await` at all? Well, for one because you have no choice. With very few exceptions, all IO is asynchronous in Node. And async colors[2] JS functions. More importantly, because JS is (mostly) single threaded and generally spawning new processes is either not available or expensive to do on demand. Getting the most processing out of a single process is the whole point.

Other environments will either accomplish something similar with threads, or spin up a process per request and depend on the OS for scheduling work. They’re all slightly different trade offs along the same general theme: make idle resources available while waiting. Node’s trade-off is that it has to handle a global (or process-wide anyway) task queue. Threaded approaches and similar put more scheduling control, resources, and complexity, in the hands of devs. Process-per-request eliminates the need to think about any of this, at the expense of cold start times and memory.

All of those trade-offs are reasonable depending on the workload and depending on the environment. Part of the reason `async` (the keyword and/or language facility) has gained popularity across a wide variety of use cases is that the combination of concurrent state complexity, cold start time, and memory pressure makes it appealing to model the problem in a way that almost looks synchronous.

1: Node devs stumbling upon this: yes I realize this isn’t the typical interface for a request callback. That’s intentional to keep it simple, but you can do it this way if the traditional callback hell interfaces are not your cup of tea… you just need a simple wrapper.

2: http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y...

Interesting. Based on this series of responses, you moved from "async is useless" to "actually, I don't actually know what async processes can do, and what happens between each process." It implies to me that your initial comment was written with little understanding of what you purported to denigrate.

Well, I understand what they can do in desktop and mobile GUI apps. But server-side web requests don't work that way; there's a beginning and an end and a pretty straight line between the two.

Cooperative multitasking is incredible for server side applications.

This isn’t helpful. They very clearly and frankly asked. Don’t shame people for not knowing something you already know, they’ll very probably be turned off from learning it.

It is a different story if they asked in good faith, but they clearly started their initial comment as a derisive declaration of something being useless, so it sounded to me that they had no real intention of actually learning.

You may want to follow the thread where we continue to discuss it in good faith. I think they’re sincerely inquisitive, and sincere (and impressively candid!) in expressing their understanding gap.

I think the various “assume best intent” rules and adages would benefit greatly from a huge “assume limited familiarity when dismissiveness seems out of place” addendum. At least for technical topics where the stakes are “whoops we actually just disagree on the merits”.

There aren't many good reasons to make good web content like that today. Money? It's way oversaturated with people who can live off $400/y. Recognition? Your content will be stolen by the previous group a thousand times over. To make a difference? There are more rewarding ways to do that if you've got the deep industry knowledge.

It's almost always been like that. When NoSql was the new hotness, there was blogspam and fistfights everywhere. When DI frameworks were the new hotness, there was blogspam and fistfights everywhere. I could go on. Devs see something new, learn enough to be annoyingly dangerous and blog like crazy. Other devs will glazed-eyed follow along because it makes their resume shiny.

Very accurate. There was a time that I enjoyed this stuff but now it just takes too much effort to filter.

this 100%

sometimes i Google stuff i already know just to verify i remembered it correctly

lately i've been refreshing my memory on Vue JS and this website comes up a lot: https://thewebdev.info

the solutions i read there were often incorrect and misleading

i can't imagine becoming a developer today, when most of the stuff you read on the internet bullsh*t

of course you can browse the docs, but some docs are tedious to comprehend when all you're looking for is a simple one-line answer

Maybe that's the fault of those "who know" but don't write and contribute online ?

I don't think so. If you know where to look then good content is available, but it doesn't get as widely shared because it isn't as interesting to as many people.

Those who know are fewer, they get outnumbered by the low quality content.