Skip to content

Commit e9e6af9

Browse files
jeayejeaye
authored
Add jank intro post (#278)
1 parent e9bf975 commit e9e6af9

File tree

3 files changed

+235
-0
lines changed

3 files changed

+235
-0
lines changed

‎_posts/2024-12-20-jank.md

+235
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,235 @@
1+
---
2+
title: "The jank programming language"
3+
layout: post
4+
excerpt: "Learn about jank, the native Clojure dialect on LLVM,
5+
and how it uses Clang and CppInterOp to stand out from the pack.
6+
Discover how Clojure users develop and how jank brings that to
7+
the native world."
8+
sitemap: false
9+
author: Jeaye Wilkerson
10+
permalink: blogs/jank_intro/
11+
banner_image: /images/blog/jank_intro/logo.png
12+
date: 2024-12-20
13+
tags: jank clojure clang clang-repl cppinterop
14+
---
15+
16+
# Intro
17+
Hi everyone! I'm [Jeaye Wilkerson](https://github.com/jeaye), creator of the
18+
[jank programming language](https://jank-lang.org). jank is a dialect of
19+
[Clojure](https://clojure.org) which I've been working on for around eight years
20+
now. To start with, jank was an exploration of what my ideal programming
21+
language would be. Coming from a background of systems programming and game
22+
development, I was very comfortable with C++, but I also was exploring the
23+
world of functional programming. All of this led me to try Clojure, which
24+
ultimately refined the direction of jank altogether.
25+
26+
My work on jank is made possible by Clang, LLVM, and the tools coming out of the
27+
[Compiler Research Group](https://compiler-research.org/), including Cling/Clang
28+
REPL and [CppInterOp](https://github.com/compiler-research/CppInterOp).
29+
30+
The past two years of jank development have been exciting as I've focused on
31+
building jank into a native Clojure dialect on LLVM. This marries the lispy
32+
world of REPL-driven, data-oriented, interactive programming and the C++ world
33+
of light, fast, native runtimes. jank isn't production ready yet, but it's come
34+
a long way in the past two years.
35+
36+
37+
<img src="/images/blog/jank_intro/star-history.png" style="display: block; margin-left: auto; margin-right: auto;" width="50%" />
38+
39+
# What is Clojure?
40+
Clojure is a modern, practical take on Lisp. It's functional-first, has
41+
persistent, immutable data structures by default, but allows for adhoc side
42+
effects. As it's a lisp, it has a powerful macro system which allows devs to
43+
transform their own code using the same functions they use to transform any
44+
other data. Clojure is dynamically typed and also garbage collected.
45+
46+
Clojure has a rich tooling story for interactive development. Clojure devs
47+
generally connect their text editor directly to their running programs, via a
48+
REPL (read, eval, print, loop) client/server. This allows us to send code from
49+
our editor to our program to be evaluated; the results come back as data which
50+
we can then continue to work on in our editor. This whole workflow allows any
51+
function to be redefined at any time during development and it allows for a
52+
development iteration loop like nothing else.
53+
54+
Clojure is a hosted language, which means it has a symbiotic relationship with
55+
the host environment on which it runs. By default, Clojure is on the JVM.
56+
Clojure JVM enables seamless interop with other JVM languages like Java, Kotlin,
57+
Scala, etc. However, Clojure runs on many other hosts. For example,
58+
ClojureScript runs on JavaScript (browser or node). ClojureCLR runs on the CLR
59+
(where C#, F#, etc run). There's also ClojureDart, ClojErl (BEAM), and many
60+
others. Each of these dialects provides seamless interop with its host. Since
61+
Clojure is always hosted, things like socket libs, filesystem libs, etc are
62+
never included in Clojure; they just come from the host.
63+
64+
Clojure on a native host, though, is where jank comes in.
65+
66+
# What makes jank special?
67+
jank provides all of the interactive functionality of Clojure, which involves
68+
JIT compiling native code with LLVM, while also providing seamless interop with
69+
the native world. For those who are familiar, jank for Clojure is basically
70+
analogous to Clasp for Common Lisp. However, compared to Clasp, jank aims to
71+
provide even more seamless native interop which doesn't require making "bridge"
72+
files.
73+
74+
## Interop features
75+
In jank, when you require a module (like including a header in C++ or importing
76+
a module in Python), that module may be backed by a jank file or by a C++ file.
77+
If it's backed by a C++ file, jank will JIT compile that file and then bootstrap
78+
it by invoking a well-known function. This allows you to vender C++ code
79+
alongside your jank code within the same project seamlessly.
80+
81+
For example, take a look at this jank file which requires both a jank dependency
82+
and a C++ dependency. They're both required in the same way. When
83+
`clojure.pprint` is required, jank will find that the module is backed by a
84+
`.jank` file, so it will JIT compile that file. However, when `sleep-native` is
85+
required, jank will find that it's backed by a C++ file and it will JIT compile
86+
that using `clang::Interpreter`. In either of these cases, if a `.o` is present
87+
and up to date, jank will load that instead.
88+
89+
```clojure
90+
(ns nap-time
91+
(:require [clojure.pprint :as p] ; JIT loads a jank module
92+
[sleep-native :as s])) ; JIT loads a C++ module
93+
94+
(defn main []
95+
(s/sleep 500)
96+
(p/pprint "napped!"))
97+
```
98+
99+
The `sleep-native` module can be backed by a C++ file which looks something like
100+
this. jank will look for a special `jank_load_sleep_native` function in the
101+
module, based on the name of the module itself. Note, the `-native` suffix is
102+
just a common pattern used for modules backed by C++ files.
103+
104+
```cpp
105+
object_ptr jank_load_sleep_native()
106+
{
107+
/* Create the sleep-native namespace so it's exposed to the jank world. */
108+
auto const ns{ _rt_ctx->intern_ns("sleep-native") };
109+
auto const intern_fn{ [=](native_persistent_string const &name, auto const fn) {
110+
auto const boxed_fn{ make_box<obj::native_function_wrapper>(fn) };
111+
ns->intern_var(name)->bind_root(boxed_fn);
112+
} };
113+
114+
/* Create a var holding a pointer to the sleep function. */
115+
intern_fn("sleep", [](object_ptr const ms) -> object_ptr {
116+
auto const duration(std::chrono::milliseconds(to_int(ms)));
117+
std::this_thread::sleep_for(duration);
118+
return nil_const;
119+
});
120+
121+
return nil_const;
122+
}
123+
```
124+
125+
Overall, this automatic module support allows for arbitrarily complex C++ files
126+
to be loaded, which will generally be used to wrap some existing native code and
127+
use jank's C or C++ API to expose that code to the rest of the jank system.
128+
However, this is all still roughly equivalent to "bridge files" and I aim to do
129+
better.
130+
131+
On top of that, jank will also support interop directly from jank code into C++
132+
land, allowing for the instantiation of native C++ objects as locals, calling
133+
native functions, and also instantiating C++ templates on the fly. This is where
134+
[CppInterOp](https://github.com/compiler-research/CppInterOp) comes in. At this
135+
point, it's only in design phase, but I aim to utilize CppInterOp to its fullest
136+
so that we can allow for arbitrary C++ values within jank, calling C++
137+
functions, and working with C++ data types. The working design looks something
138+
like this:
139+
140+
```clojure
141+
; Feed some C++ into Clang so we can start working on it.
142+
; Including files can also be done in a similar way.
143+
(c++/declare "struct person{ std::string name; };")
144+
145+
; `let` is a Clojure construct, but `c++/person.` creates a value
146+
; of the `person` struct we just defined above, in automatic memory.
147+
(let [s (c++/person. "sally siu")
148+
; We can then access structs using Clojure's normal interop syntax.
149+
n (.-name s)
150+
; We can call member functions on native values, too.
151+
; Here we call std::string::size on the name member.
152+
l (.size n)]
153+
; When we try to gives these native values to `println`, jank will
154+
; detect that they need boxing and will automatically find a
155+
; conversion function from their native type to jank's boxed
156+
; `object_ptr` type. If such a function doesn't exist, the
157+
; jank compiler fails with a type error.
158+
(println n l))
159+
```
160+
161+
All of this will work with RAII, allow auto-boxing, but also give complete
162+
access to C++ from within jank.
163+
164+
## AOT features
165+
With jank, you can AOT compile your programs to two different runtimes:
166+
167+
1. A dynamic runtime which enables REPL usage, further JIT compilation,
168+
redefining functions, etc
169+
2. A static runtime which strips out all JIT capabilities, enables LTO, and
170+
directly links functions instead of going through vars
171+
172+
Static runtime binaries will also be able to be statically linked, providing a
173+
portable binary which will very easily run on any distro or in any docker
174+
container.
175+
176+
# Why would people use jank?
177+
There are two main audiences for jank.
178+
179+
## Clojure users
180+
jank's runtime is significantly lighter than the JVM. Since its runtime is
181+
specifically crafted for jank, rather than being a generic VM, it's also able to
182+
compete with the JVM in terms of runtime performance while keeping memory usage
183+
lower.
184+
185+
On top of that, anyone who is using Clojure and wanting easier access to native
186+
libraries, tighter AOT compilation, etc, without sacrificing the fundamental
187+
Clojure interactive development workflows will choose jank.
188+
189+
## Native users
190+
In game development, where my career has been focused, we build our engines and
191+
games in C++, generally. However, we very often include a scripting language
192+
like Lua in the engine so that we don't need to write all of our game logic in
193+
C++. jank is an excellent fit here, too. Not only does it provide seamless
194+
interop with the existing C++ engine/game code, but the REPL-driven workflows it
195+
supports will allow you to spin up your game once and then send code to it, get
196+
data back, and continue iterating on the code and data shapes without ever
197+
restarting your game.
198+
199+
Naturally, this is a game development focused use case, but the same thing
200+
applies to any existing native code base which wants to utilize a higher level
201+
language for a tighter iteration loop.
202+
203+
# Next steps
204+
jank is not yet released in production, but I aim to have it out the door in 2025.
205+
In fact, as of January 2025, I'll have quit my job at Electronic Arts to
206+
focus on jank full-time, unpaid aside from some small sponsors. I'm aiming to
207+
get jank released, gather initial feedback, and help existing Clojure users
208+
onboard jank into their systems.
209+
210+
The three big tasks that are remaining before jank is released are:
211+
212+
1. Improved error handling (following the recent work on Elm, Rust, and others)
213+
2. Finalized C++ interop support (using the CppInterOp library)
214+
3. Finalized AOT compilation support
215+
216+
# Future plans
217+
In the long term, since I'm building my dream language, I won't stop at just a
218+
native Clojure dialect. jank will always be able to just be that, but I'll also
219+
be extending it to support gradual typing (maybe linear typing), explicit memory
220+
management, stronger pattern matching, value-based errors, and more. This will
221+
allow another axis of control, where some parts of the program can remain
222+
entirely dynamic and garbage collected while others are thoroughly controlled
223+
and better optimized. That's exactly the control I want when programming.
224+
225+
# Thank you!
226+
For everyone who has worked on Cling, Clang REPL, and CppInterOp: Thank you!
227+
jank is possible because of the magic you folks have created. To Vassil, in
228+
particular, thanks for all of the support over the past couple of years as I
229+
onboarded with Cling, ported to Clang, and then ultimately picked up
230+
CppInterOp as a means to tackle seamless C++ interop.
231+
232+
## Would you like to join in?
233+
1. Join the jank community on [Slack](https://clojurians.slack.com/archives/C03SRH97FDK)
234+
2. Join the jank design discussions or pick up a ticket on [GitHub](https://github.com/jank-lang/jank)
235+
3. Join the Compiler Research community on [Discord](https://discord.gg/Vkv3ne4zVK)

‎images/blog/jank_intro/logo.png

159 KB
Loading

‎images/blog/jank_intro/star-history.png

83.4 KB
Loading

0 commit comments

Comments
 (0)