Appreciating what Lisp is capable of doing (think macros), and having worked through SICP some twenty years ago, and after having tried to make a deep dive in CL and Emacs Lisp two years ago, I come to the conclusion that there is no silver bullet in Lisp-land. Python is a good enough Lisp, as Peter Norvig has concluded. And it’s got all batteries included. Ain‘t nothing it can’t do. Building websites, doing maths, automating, except building compilers and OSes. But Lisp won’t compete in that field anyway.
Programming languages are there to express ideas and solve problems. I can think of more elegant ways to express ideas (functional languages for instance), but Python is an excellent ecosystem for solving problems.
> Python is a good enough Lisp, as Peter Norvig has concluded. And it’s got all batteries included. Ain‘t nothing it can’t do.
I almost agree, but Python fails in one area, which happens to be my current area of interest. It doesn't have sufficiently strict encapsulation to support object capabilities[0] at the language level.
JavaScript, on the other hand, does[1] - so you don't necessarily need Lisp for this, but I do enjoy the parentheses. :)
First, it has no macros. The ability to metaprogram in Python is practically nonexistent compared to Lisp.
Second, it has no usable lambda (being limited to a single expression in a non-expression oriented language. Most lisps heavily rely on the ability to pass around arbitrary functions whether named or not.
Third, it is slow. Your toy scheme implementation is probably going to be about as fast as Python and something like SBCL/Common Lisp will be literally 3-400x faster and that’s without mentioning that Python is limited to green threads. There’s no reason to limit yourself to such a slow language
Fourth, it lacks the interactive development environment of Common Lisp. Sue it has a REPL, but you can’t do all your coding against a live instance with fine-grained control of your updated code. This is hard to describe, but it’s a massive differentiator.
Python does have plenty of metaprogramming capabilities, they just are more complex to use than a macro system. One simple, but highly limited system is metaclasses. Similarly weak but existing features include creating classes at runtime, and adding new methods to classes at runtime. A far more powerful system is compiling source to AST, which can then be manipulated, and finally compiled to bytecode. This allows arbitrary metaprogramming, but is certainly fairly complex.
One could even use it to implement a macro-like system by coding up a new finder that uses a new SourceFileLoader subclass that overrides the (accidentally undocumented!?) `source_to_code` method. In there, one can compile to ast, transform the ast, and then compile the ast to bytecode. (The method was documented back in 3.4 by way of being documented on a ancestor abstract base class, but that documented ancestor was changed to be static in 3.5, leaving this method seemingly accidentally undocumented.)
For the transform the AST step, one might look for apparent function calls to special macro functions, and instead call those macro methods as AST time passing in the AST of its arguments, receiving back as AST that you place in the tree replacing the function call. There is some trickiness here. For example, it is much easier if the macros are defined in another module so that they can already be compiled, and thus available while importing a module that uses them.
It should also be possible to handle macros defined in the same file by stripping out non-macro function definitions, and non-imports from top level statements, compiling that, and then using the result while processing the whole (unstripped) AST, and then finally compiling the result.
Of course such a system is significantly more complicated than with a lisp, but it is still possible. Its load time performance is also not likely to be especially wonderful, but python is not exactly a performance powerhouse.
SBCL is written in Lisp, yes? Except the runtime, which is C + asm.
I've heard people wrote some OSes in the past, like Genera. Or if you prefer recent attempt, try https://github.com/froggey/Mezzano. Never tried it, though.
The true killer feature of lisp is s-expressions and the structural editing affordances they provide. “If I can’t slurp and barf, I don’t want to be part of your revolution.” ~Emma Goldman(ish)
I think Forth beats Lisp out in these domains. To program at all in Forth is to write a DSL and to metaprogram. It is extremely common to come into a Forth codebase in an enterprise environment, and it looks nothing like anything you've seen before. Hand an intermediate Lisp programmer a codebase where the first 1000 lines are completely replacing all the built-ins and then swapping to some non-sexpr language, you're going to get a lot of headscratches and frustration. A Forth programmer will have expected this and have a large bottle of APAP in their hand for the ensuing task ahead of them. In this same way, Forth is also an important lesson as to why these things are more "anti-features" when you base your entire language's identity around them.
It seems like, if anything, a language built around creating and traversing tree structures would be perfect for making compilers. But, hey, what do I know? :)
Programming languages are there to express ideas and solve problems. I can think of more elegant ways to express ideas (functional languages for instance), but Python is an excellent ecosystem for solving problems.