NixOS is annoying because everything is weird and symlinked and so I find myself fairly frequently making the mistake of writing `#!/bin/bash`, only to be told it can't find it, and I have to replace the path with `/run/current-system/sw/bin/bash`.
Or at least I thought I did; apparently I can just have done `#!bash`. I just tested this, and it worked fine. You learn something new every day I guess.
/bin/sh is NOT required by POSIX, they explicitly warn that it may not exist[1].
> Applications should note that the standard PATH to the shell cannot be assumed to be either /bin/sh or /usr/bin/sh, and should be determined by interrogation of the PATH returned by getconf PATH , ensuring that the returned pathname is an absolute pathname and not a shell built-in.
I don't know if this is still a thing, but I distinctly remember when playing around with NixOS years ago that `env` was the only thing in /usr/bin, which I assumed was pretty much exactly for this reason.
Because /bin is the standard location for bash. The only one that breaks that expectation is NixOS (and maybe GuixSD?), apparently. I'm surprised they didn't symlink /bin or put a stub. Last time I tried NixOS was like 10 years ago. I thought there was a /bin/bash, but maybe it was just a /bin/sh?
Other interpreters like python, ruby, etc. have more likelyhood of being used with "virtual environments", so it's more common to use /usr/bin/env with them.
/bin is the "standard" location for bash on a subset of Linux distributions and basically no other Unix...
So it's not really a standard.
/bin/sh is a much more common convention but once again, not a standard.
There really isn't a truly portable shebang, but the same can be said about executables themselves. As part of the build or install step of whatever thing you're making, you should really be looking these up and changing them.
It's only really NixOS as far as I know that doesn't ever put bash in /bin/bash (as far as Linux distributions go). But, on the other hand, there are quite a few distros (or at least flavours of distros) which don't ship bash by default (alpine, minimal versions of most distros, and embedded-Linux focused stuff if you count it). I imagine the most common "installation" of Linux is userspace in a container (yeah I know there's no kernel there, but nobody who talks about "Linux" broadly speaking specifically cares about the kernel) and a good chunk of those will be minimal with no bash.
Bash has to be explicitly installed on OpenBSD, FreeBSD, NetBSD (I think, haven't used it in a while) and probably a bunch of others. And in all of those cases (that I know of) it doesn't end up in /bin/bash once installed.
The default bash shipped on macs is so abhorrently ancient that it would be strictly better if it didn't exist because it would reduce the number of people who think bash scripts I write are broken (they're not broken, they just inevitably depend on some bash 4+ feature). Moreover, hardcoding /bin/bash as your shebang in this case will prevent anyone from remediating this problem by installing a non-ancient bash because the old one in /bin/bash will still get used.
Will not wok on OpenBSD where the shell that comes with the system is ksh at /bin/ksh and /bin/sh and if you want bash it is a third party package and correspondingly gets installed as /usr/local/bin/bash
It does get awkward, especially when porting. all your third party libraries and includes are in /usr/local/lib /usr/local/include but at least it is better than freebsd which also insists on putting all third party configs under /usr/local/etc/
They do symlink /bin/sh to be fair, and that's very often good enough for a lot of scripts. That's what I usually do if I don't need anything bash offers.
Thing is, a few years ago when Debian changed its default sh from bash to ... either ash or dash, I forget which, I got into the habit of always writing `#!/bin/bash` at the top of my scripts, in case I didn't realize that something I was using was a bashism not found in classic /bin/sh. So if I used Nix (I don't, since for my particular use cases the juice isn't worth the squeeze), I would get seriously messed up by that.
It's guaranteed to work provided that Bash is in the path.
It's very common for Python. Less so for Bash for two reasons: because the person who writes the script references /bin/sh instead (which is required to be there) even when they are writing bash-isms, or because the person who writes the script assumes that Bash is universally available as /bin/bash.
I think you're mixing two concepts: relative paths (which are allowed after #! but not very useful at all) and file lookup through $PATH (which is not done by the kernel, maybe it's some shell trickery).
> and file lookup through $PATH (which is not done by the kernel, maybe it's some shell trickery)
It's libc. Specifically, system(3) and APIs like execvp(3) will search $PATH for the file specified before passing that to execve(2) (which is the only kernel syscall; all the other exec*() stuff is in libc).
> /usr/bin/env and /bin/sh are part of the POSIX standard, this is why NixOS has those available.
Contrary to popular belief, those aren't in the POSIX standard.
The following are not in the POSIX standard, they are just widely implemented:
- "#!" line.
- /bin/sh as the location of a POSIX compliant shell, or any shell.
- /usr/bin/env as the location of an env program.
- The -S option to env.
I think you are using "not required by the POSIX standard" when you say "not in" which is not an accurate shorthand.
#! is certainly in the POSIX standard as the exact topic of "is /bin/sh always a POSIX" shell is a discussion point (it is not guaranteed since there were systems that existed at the time that had a non-POSIX shell there)
Are they in POSIX? I do not think they are. All of them is a convention from what I remember.
Shebang is a kernel feature, for example, and POSIX does define the sh shell language and utilities, but does not specify how executables are invoked by the kernel.
Similarly, POSIX only requires that sh exists somewhere in the PATH, and the /bin/sh convention comes from the traditional Unix and FHS (Filesystem Hierarchy Standard), but POSIX does not mandate filesystem layout.
You're definitely correct. "#!" is reserved (see Rationale C.2.1), but not required, though it's described as "ubiquitous" (see Rationale C.1.7). "/bin/sh" isn't required either, but arguably ubiquitous in that there's always some shell located there. The proper way to find the POSIX-conformant shell is with `command -v sh` (which is equivalent to using `getconf PATH` and then searching for sh), and POSIX counsels to discover the path and substitute it inline when installing scripts (see Application Usage in sh utility specification.)
IME /bin/sh is invariably sufficiently POSIX conformant to bootstrap into a POSIX shell (or your preferred shell), even on Solaris and AIX. And if you're willing to stick to simple scripts or rigorously test across systems, sufficient for most tasks. Outside Linux-based systems it's usually ksh88, ksh93, pdksh, or some derivative. OTOH, for those who are only familiar with bash that may not be particularly helpful.
I've had more trouble, including bugs, with other utilities, like sed, tr, paste, etc. For shell portability it's usually niche stuff like "$@" expansion with empty lists, for example how it interacts with nounset or IFS, independent of POSIX mode.
Linux is UNIX in the context of my question. On Linux the shebang is actually handled by the kernel. When you load a binary and attempt to execute it with a syscall, the kernel reads the first few bytes of the binary. If it is an ELF header, it executes the machine code as you would expect. If the first two bytes are "#!", then it interprets it as a shebang header and loads the specified binary to interpret it.
Again, this is kernel code. I admit I'm a bit confused as to why it didn't work on your system. This shouldn't be handled at the shell level.
It is possible for the shell to handle it. From zshall(1):
> If the program is a file beginning with ‘#!', the remainder of the first line specifies an interpreter for the program. The shell will execute the specified interpreter on operating systems that do not handle this executable format in the kernel.
I did a little digging and found that the `|| eno == ENOENT` was added quite a bit earlier[1] than the actual pathprog lookup[2]. While I could find the "issue discussion" for the pathprog change[3] I wasn't able to find it for the ENOENT addition, which was kind of interesting and frustrating--[4] is the `X-Seq` mentioned in the commit but that seems to be inconsistent or incorrect for the actual cross-reference, and nearby in time wasn't helpful either.
I think it makes it to calling open_exec but there's a test for BINPRM_FLAGS_PATH_INACCESSIBLE, which doesn't seem relevant since 'bash' isn't like '/dev/fd/<fd>/..', but does provoke an ENOENT.
env bash is all well and good for normies, but if you're already on NixOS did you know you can have nix-shell be your interpreter and back flip into any reproducible interpreted environment you like?
Or any other system with Nix installed. I use this at work to provide scripts with all their dependencies specified that work across any Linux distro & MacOS. First execution is slow since it has to fetch everything, but after that it's fast and just works.
The script you posted is only portable in theory and not in practice. Executing it while using a non-POSIX shell like Elvish (even without having it as a default shell) makes it immediately fail.
Meanwhile, `#!/usr/bin/env` is completely portable in the practical sense. Even systems with non-standard paths like NixOS, Termux and GoboLinux patch in support for it specifically.
NixOS is annoying because everything is weird and symlinked and so I find myself fairly frequently making the mistake of writing `#!/bin/bash`, only to be told it can't find it, and I have to replace the path with `/run/current-system/sw/bin/bash`.
Or at least I thought I did; apparently I can just have done `#!bash`. I just tested this, and it worked fine. You learn something new every day I guess.