Proposal for Lisp as the Extension Mechanism #14539
Replies: 6 comments 10 replies
-
|
When it comes to ease of learning and use, I think ruby is also a good option. Personally, I feel that ruby's chained calls are more elegant than bracketed expressions. When I look at elisp code, I always feel like I'm struggling to understand it, whereas ruby code is very intuitive and beautiful. But I'm not sure ruby is suitable as a language embedded in an editor! |
Beta Was this translation helpful? Give feedback.
-
|
Ruby should be possible. Sublime Text made a similar decision. They use Python. But my proposal goes in a different direction. Lisp has some unique selling points that I don't see with other languages. In my opinion these matter for the purpose of being an embedded language. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, Lua is a more lightweight scripting language. It has significant advantages as a configuration and scripting language, and it's also easy for developers to maintain. Neovim is a great example of this. Lua has a well-established ecosystem and is simple and easy to use." |
Beta Was this translation helpful? Give feedback.
-
|
The problem with Lua is that it does not seem to have a very clear direction, and it has been forked several times as a result. Python and Ruby are more suitable. Ideally, the scripting language should be portable, support multi-threading well, maybe even compile to WASM. Guile is in my opinion better for the purpose than a full CL, and has a WASM compiler (Hoot). @habruening has mentioned Clojure — it would be a great choice, but I fear that the WASM backend is not mature enough? If it has reached sufficient maturity, then it would be my first choice. |
Beta Was this translation helpful? Give feedback.
-
|
Lua is one of the few options. I don't know Lua so much. No doubt that Lua for NeoVim is a great success. I would be a bit concerned if Lua is really versatile enough. It is nice to have a real programming language that addresses all sorts of problems. But the web page of Lua clearly sais, that it is a scripting language. I am sceptical if a scripting language is appropriate for real GUI heavy stuff. I suppose we want this. This will be a bit more complicated for Zed than for Emacs or NeoVim, because Zed is not a console program. So you need a language that addresses the challenges, including multi threading. Lua looks to me like it has a bit of everything, but not enough of everything. For example it does not have map and reduce from functional programming or no multimethods or no native OOP. Some people really need these things to be productive. I prefer a real general purpose language, so that the editor can get so powerful that I can live the whole day in the editor like Emacs people do it. Assume you want to implement an E-Mail client in Zed. What language do you want to use? Obviously things like Scala are not an option. In my opinion Lisp is the hottest candidate. And the performance of Lisp is quite often still much faster than Lua. Lua is normally compared to other scripting languages und has the reputation to be very fast in this comparison. But Lisp is often compared to C. But I suppose, the Zed team goes more into the direction of Web Assembly. They already use Web Assembly for Tree Sitter. With Web Assembly they also get a good solution for safety. With Web Assembly they are language independent. The Zed team may consider this as a benefit. I personally see it as a disadvantage, because it causes much fragmentation in the community. Everybody would use his favourite language. With Lisp you don't get such a fragmentation. You find tons of projects where programming language A is implemented in programming language B. Lua in Python. Lisp in Lua. Python in Java. Everything in everything. Because people are never happy with what they have. But what you don't find, people who try to implement Lua in Lisp, Python in Lisp, JavaScript in Lisp. Lisp developers are normally very happy with their language (my observation). If you take Lisp, you get one language that can rule everything. What you get with Lisp is not only a language for extensions. You also get a dynamic environment. See here what I mean. That Clojure language (which is a Lisp dialect) enables impressive dynamic features in VS Code by exposing the JavaScript API to the user space. The whole infrastructure that enables a comfortable working experience with this foreign API already existed. That stuff implements the idea that in the middle of an arbitrary text file you can have pieces of code, which you can execute in your editor. You want to write something like 30 + 12 and quickly evaluate it without the need to install a math evaluation plugin. Or you want to copy something into the clipboard, filter it, sort it and paste it back. This is quickly done in any programming language. But you need a programming language that is available for you in that moment and that has access to your editor. You want to be able to have a text file that contains its own GUI, which helps editing itself. If you plan to enable this with Lua, fine. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for your clarification! It's fun to discuss these topics, and your points make sense. I just want to emphasize that Lua is one of the choices available. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, you convinced me, that Lua is a good and fits. |
Beta Was this translation helpful? Give feedback.
-
|
Just my experience, I gave up on emacs for two motives: it is too slow and elisp. Elisp, and functional languages in general, it is interesting, amazing, but too hard for the majority of devs, and for me. If zed wants to be a major force in editors wars, he needs to have a very engaged community that develops a multitude of plugins. Unfortunately lisp does not promote this because to be productive in that language takes too much time to deeply understand. As far I understand that is one major of the problems of emacs: a heavily engaged community, that builds an impressive software that makes everything, but that is still a small community because not everyone understands lisp. Lua, is fast, and a script language, so it is easy to understand. We can see this in the neovim: a big and engaged community. I believe that it is not merely a software decision but a choice that will impact the development of the community. And if zed chooses lisp I will give up too unfortunately. Well, I prefer python, but I understand that is a really bad software decision for obvious reasons. @habruening , I believe maybe you love lisp, and I understand that, it is a amazing language, but functional languages, that are beautiful, are hard and not for everyone unfortunately. |
Beta Was this translation helpful? Give feedback.
-
|
Maybe a good option to plugin language will Rhai a Rust script language. |
Beta Was this translation helpful? Give feedback.
-
|
I personally love Emacs and use it all the time but honestly I also think Lua may be a better option, Emacs uses elisp because it's old, not because its the best option. |
Beta Was this translation helpful? Give feedback.
-
|
I looked a bit what options exist in the world. Unfortunatelly not so many. Let me summarise what I found out. My personal conclusion at the moment is that Lua is indeed the best candidate and the fastest you can get. But in my opinion Lua is not perfect. Just one example. Emacs has an XML outline. NeoVIM not. Why not? We need a language, where such things are easy to implement. What is the ideal way for XML editing? You need a view with the XML file, a view with your user configurable WYSIWYG editing (e.g. something like https://www.altova.com/de/authentic) and one view for an outline. All these three parts must be editable and responsive. If you change something in one part (e.g. drag and drop something in the outline, the other parts must update simultaneously at the speed of thought. That is hard to achieve. That works only if the thing that you edit is not ASCII text. ASCII text is the enemy of effective programming. That is hard to say in a discussion about text editors. But we must face the reality. You cannot parse and update ASCII text everywhere, because that is hard to implement. You want data models instead. Otherwise you will never be better than VS Code. So you need a real powerful programming language, that enables this. I don't think, that Lua is that powerful. The web page clearly sais, it is a scripting language. Also I am a bit sceptical if Lua has answers for the next 30 years. It looks to me more like a trendy language for now. But that is maybe unly until someone comes with a new alternative. Lua is from 1993. The authors of Lua in my opinion are a bit too pragmatic and do not seem to have a long term strategy. On the one side they say "We did not consider LISP or Scheme because of their unfriendly syntax." But then they say "Semantically, Lua has many similarities with Scheme, even though these similarities are not immediately clear because the two languages are syntactically very different. The influence of Scheme on Lua has gradually increased during Lua’s evolution." I am not sure, how serious I can take Lua. That reminds me at JavaScript, where many people say, it is a dialect of Lisp. But many people are actually not happy wiht JavaScript and try to replace it by something else. Or why does nobody consider JavaScript for Zed? And now look how people use JavaScript: React is Functional Programming. What I want to say: There is still a lot happening in the world of programming languages. Betting on a language is a very hard decision. But, what I can clearly see, is that Emacs with Elisp made the right decision 50 years ago. Although not very popular, there is nobody who seriously wants to replace Elisp with something else. So what is the right decision for the next 50 years? A few alternatives (but none of them viable):
Unfortunatelly I have no good proposal. But the Zed team will have to make a decision. If they decide for WASM, I could understand that. But in my opinion it would only be an escape. I don't think, that Emacs or NeoVIM became so successful if there would not have been the one programming language that unifies everything. I don't think, that WASM would reproduce that "your text editor is a whole operating system" experience. That Emacs idea of being a whole operating system has a long tradition in the Lisp world (https://www.youtube.com/watch?v=7RNbIEJvjUA). But anyway, Zed would be a reason for me to learn Lua, even if I prefer Lisp. |
Beta Was this translation helpful? Give feedback.
-
|
I gave this some thought, and I also looked at what other editors do. Even though, as a lisp person, I have my preference for languages like Guile, I think Lua is better, for three reasons:
Therefore, my vote for Lua. |
Beta Was this translation helpful? Give feedback.
-
|
User red-co replied with
... then apparently deleted the message. I agree that the wasm VM in Zed allows us to use any language that can be compiled to wasm, BUT I interpreted this discussion as having a first class scripting language. Having more choices is nice, but if somebody wants to modifiy an extension, than another, chances are that they have to learn more than one language. One language should have a special status and have a rich set of bindings for all sorts of manipulations and written by the Zed team. These bindings then would be ported to other languages, possibly by others. |
Beta Was this translation helpful? Give feedback.
-
|
A few observations.
What would be missing? Is for instance a node.js runtime necessary in the wasm runtime for wasmoon compiled programs? |
Beta Was this translation helpful? Give feedback.
-
On the contribution page you invite users to make proposals. I have a proposal how the extension mechanism could be. Sorry, it is nearly an essay. But the proposal contains very unusual ideas. It needs a few more words to explain why I make this proposal.
Problem Statement
The challenge is to provide an extension mechanism with the following properties.
Solution Proposal
These requirements are hard to fulfil with standard programming languages.
I propose the programming language Common Lisp for the extension mechanism.
Rationale
Lisp is easy to learn. If you compare Lisp to other languages, Lisp has fewer language concepts. The only structuring concept are parenthesis. If you look at typical JavaScript code, for example an anonymous function with curley braces as a function argument in parenthesis and commata and simocolon, you wish you had a syntax that makes this easier and not more complicated than it is. With Lisp You get this easier.
Lisp is an interactive programming environment. A text editor is interactive text processing. Both fit well together. Example: With multi cursor editing you edit a Java file. You place a cursor at each variable assignment. Now you want to reduce this selection to those assignments, where the target variable is a global variable. Normally this is impossible with text editors. You would have to ask the LSP in order to find this out, and then you have to reduce the selections dependong on what comes out of this query. You need a small programming interface for such such editing problems. Lisp gives you this.
Lisp is very high level. You only have parenthesis. You can write an expression like
(open-file :file-name)and it can be anything. It can be an option in a configuration file. It can be en entry in a menu. It can be a function that you call. Or maybe it is a list. It can be all together, because it is just a symbol followed by a keyword in parentheses. You can define a function with the nameopen-fileand then execute the expression. These ideas allow us to write program code that completely abstracts away the implementation details. And that is what we need for an extension mechanism. If someone writes(open-file :file-name)in the configuration file that configures for example a menu, everybody knows what this means, but nobody cares what it technically is. But if you look at a typical C thing likeopen_file("filename")everybody in the world recognises this as a function call, and nobody would like to have such things written in a human readable configuration file. So if you want to make this configurable you are forced to invent an ad-hoc language, while in Lisp you all the time work like this.The reason why I propose Lisp is that Lisp feels like interpreted, but technically it is a compiled language. This is what an extension mechanism meeds. You get the advantage of an interpreted language, where extensions are deliverd as source code only. But you also get the advantage of a compiled language with natively compiled fast code. This works with a concept that no other programming language offers: When you create an executable or a library with Lisp, the whole compiler is normally baked into it. The whole program execution is a dynamic thing. That allows you to dynamically at runtime compile more things, evaluate things or change things. This would mean in practice, when you load an extension for the first time, it gets automatically compiled. When you load it for the second time, it is already compiled. Only when you change the source code of the an extension it would be recompiled in that moment, so that not even a restart is required. This is what Lisp implementations typically provide, so you get this for free.
Many Lisp implementations provide a native interface to foreign languages (https://cffi.common-lisp.dev/), into Rust or C that is used by many people.
Drawbacks
The Lisp runtime environment is not a small library. If we talk only about a small extension mechanism with the only purpose to hook into your Rust world, you may not want such a huge mechanism. But if you take into account how much of the functionality of nowadays text editors are provided by extensions, it may be worth it.
Motivation
One reason why Visual Studio Code gained a lot of popularity is the programming language in which it is written. JavaScript performs very well and JavaScript attracts developers, who contribute all sorts of extensions. The succcess of JavaScript is not a random coincidence. Many people regard JavaScript as a dialect of Lisp (https://raganwald.com/2013/07/19/javascript-is-a-lisp.html, https://www.crockford.com/javascript/javascript.html). If we look at two important editors: VS Studio Code uses JavaScript and Emacs uses Lisp. The Emacs community has proven, that Lisp is appropriate for the best editors, and the VS Studio Code community has proven that ideas of dynamic programming matter for masses. But both editors have certain usability problems.
TreeSitter by the way already uses S-Expressions, which is the Lisp syntax. But TreeSitter is probably a special topic anyway, because it comes as C code. This proposal has no answer to that problem. But if Zed becomes successful and more popular it may be worth to think about a TreeSitter implementation in a way that is not based so much on binaries. That could be Lisp or whatever mechanism Zed in the future has. Lisp is an option, because it is a real programming language.
Perspective: Dynamic Behaviour
With dynamic behaviour I mean something like the following ideas.
These ideas are quite fantastic, because there is no editor that supports such things. It is hard to imagine something which we have never seen. It requires a very dynamic extension mechanism. You don't get this with a extension mechanism that requires a dedicated build or process. You want that you extension language is pure text, because this fits everywhere. You achieve this with Lisp or JavaScript or Lua. There are very heavy and impressive tools like Jetbrains MPS that try to achieve such a user experience. MPS is a great piece of software. But it is based on Java, which makes it very complex. With Lisp, as extension mechanism, where code is data you can achieve this much easier.
Beta Was this translation helpful? Give feedback.
All reactions