Racket: ConjureEvalBuf / ConjureEvalFile failures

[sogaiu]

Description

For certain .rkt files that work [1] upon opening in a buffer, using :ConjureEvalBuf or :ConjureEvalFile subsequently can lead to errors (at least for my setup [2] (^^; ).

Examples of such errors include:

E5108: Error executing lua: Vim:Error executing Lua callback: ...r/.local/share/nvim/plugged/conjure/lua/conjure/eval.lua:144: Conjure client '' doesn't support function: eval-str                                                             
stack traceback:                                                                
        [builtin#19]: at 0x55fe27da5020                                         
        ...r/.local/share/nvim/plugged/conjure/lua/conjure/eval.lua:144: in function <...r/.local/share/nvim/plugged/conjure/lua/conjure/eval.lua:132>          
        [C]: at 0x55fe27b45e60                                                  
stack traceback:                                                                
        [C]: at 0x55fe27b45e60   

E5108: Error executing lua: Vim:Error executing Lua callback: Conjure client '' doesn't support function: eval-file
stack traceback:                       
        [builtin#19]: at 0x55c2b5a25020                         
        [C]: at 0x55c2b57c5e60         
stack traceback:                       
        [C]: at 0x55c2b57c5e60

Note that I haven't yet experienced these errors for some other .rkt files. For the files I have examined, it appears that those that start with:

#lang <language-name>

are fine.

However, those that start with things that look like either:

#lang s-exp "something.rkt"

#lang reader "other.rkt"

seem to lead to the sorts of errors described above.

I haven't tested that many files though.

How to Reproduce

The code I tried with is from the online book "Beautiful Racket".

I followed the "full setup" instructions here which might be translated as:

• install a recent racket (I compiled v8.15 from source on a Linux box)
• install the beautiful-racket package via raco: raco pkg install --auto beautiful-racket
• check that the package works by examining the output of: racket -l br/test

To obtain the code to reproduce the issue with, choose one of the following:

• download a zip file linked from this page -- there should be two files contained within (stacker.rkt and stacker-test.rkt), -OR-
• copy-paste the code from the same page into appropriately named files -OR-
• enter the following into relevantly named files

stacker.rkt

#lang br/quicklang

(define (read-syntax path port)
  (define src-lines
    (port->lines port))

  (define src-datums
    (format-datums '(handle ~a) src-lines))

  (define module-datum 
    `(module stacker-mod "stacker.rkt"
       ,@src-datums))

  (datum->syntax #f module-datum))

(provide read-syntax)

(define-macro (stacker-module-begin HANDLE-EXPR ...)
  #'(#%module-begin
     HANDLE-EXPR ...
     (display (first stack))))

(provide (rename-out [stacker-module-begin #%module-begin]))

(define stack empty)

(define (pop-stack!)
  (define arg (first stack))
  (set! stack (rest stack))
  arg)

(define (push-stack! arg)
  (set! stack (cons arg stack)))

(define (handle [arg #f])
  (cond
    [(number? arg) (push-stack! arg)]
    [(or (equal? * arg) (equal? + arg))
     (define op-result
       (arg (pop-stack!) (pop-stack!)))
     (push-stack! op-result)]))

(provide handle)

(provide + *)

stacker-test.rkt

#lang reader "stacker.rkt"
4
8
+
3
*

To experience the issue:

• Open stacker-test.rkt (the second of the two files above) in neovim
• Observe 36 appearing in the HUD (there may be a subsequent error message after as well -- not sure what that is about)
• Invoke either :ConjureEvalBuf or :ConjureEvalFile
• Observe an error message at the bottom of the screen

---

[1] By "work", I mean the HUD shows the expected evaluation result when the file is opened in the buffer initially.

[2] I'm using:

• neovim v0.11.0-dev-1623+g34d808b73c
• conjure v4.53.0

[russtoku]

I can replicate the error message when running :ConjureEvalBuf or <localleader>eb after editing stacker-test.rkt.

Here's my log output:

; eval (buf): ...et/Beautiful_Racket/stacker-test.rkt
; Dropping #lang, only supported in file evaluation.
"stacker.rkt"
4
8
#<procedure:+>3#<procedure:*>
; flush-output: undefined;
;  cannot reference an identifier before its definition
;   in module:
;     "/Users/russ/Projects/Programming_Languages/Scheme/Racket/Beautiful_Racket/stacker-test.rkt"
; [,bt for context]

What's happening is when evaluating the buffer, the #lang ... line is dropped and the rest of the code is sent to a Racket REPL. Unfortunately, the Racket REPL can't handle #lang lines so the Racket client strips those lines out before sending code to the REPL.

I was able to run the stacker-test.rkt program at a shell command line.

$ racket stacker-test.rkt
36$

So, while you can edit stacker-test.rkt in Neovim, you can't run it by evaluating the buffer or the file. For the purpose of learning with Beautiful Racket, it should be OK to edit in Neovim and run the files with Racket at the shell command line. However, for other programs this is probably less than ideal.

There may be hope for evaluating files or buffers with Conjure because there is a racket-mode for Emacs which appears to be able to deal with the peculiarities of #lang. I haven't tried to look at the details. It will take a bunch of work to figure out how improve the Racket client so that it can evaluate a file or buffer. For now, we'll need to run the file directly with a command line Racket like racket stacker-test.rkt or racket stacker.rkt.

[russtoku]

See #456 for related issue.

[Olical]

Thanks for providing additional context @russtoku, I can reproduce this issue and agree the solution probably isn't a straightforward fix in Conjure. From my investigation, my hunch was that the #lang with this reader keyword is replacing some key REPL features we use from Conjure like ,enter and ,require-reloadable.

I got this module working and reloading by removing the automatic calls to ,enter from Conjure's client, then I can run ,ef as much as I want. I just have to remember that we never call ,enter on the module.

So I'm thinking maybe we only use ,enter if the #lang line matches some pattern maybe defined by a new function like simple-lang?. So it will determine which kind of #lang we can safely call ,enter on - and which would break the REPL mechanisms in such a way that the REPL is now useless to us (which is what we see currently).

[Olical]

I've added an option which isn't perfect but might be good enough for you for now and gives us some good data on what a good solution looks like. Try setting g:conjure#client#racket#stdio#auto_enter to false (or v:false in VimL) - this disables the automatic calls to ,enter which means your ,ee will no longer be automatically in the context of the module you're editing, but your ,ef should keep working even if you're loading a module that overrides core features of the REPL like the reader.

Not perfect, but a lever you can pull. Then maybe we work out exactly which #langs should be safe to ,enter and which probably aren't then we add that as another heuristic for the auto-enter. ,enter is super useful 95% of the time I think, it just breaks your REPL and requires a restart if you ,enter something very different.

I was also trying to see if I could load xrepl into the stacker module somehow, so even when we enter we still have access to xrepl commands, but I'm not sure how to go about that.

[russtoku]

@Olical, your fix works!

With g:conjure#client#racket#stdio#auto_enter set to false, I was able to evaluate stacker-test.rkt with <localleader>ef while the buffer is editing stacker-test.rkt. The log reports:

; eval (file): ...cket/Beautiful_Racket/stacker-test.rkt
; reloading "stacker-test.rkt"
36

A minor nit would be that the virutal text reads:

=> ; reloading "stacker-test.rkt" 36

It should be noted that while you can <localleader>ef with stacker.rkt, you can't evaluate the define-macro form unless you change the g:conjure#client#racket#stdio#command to racket -I br/quicklang. Doing this is equivalent to setting #lang br/quicklang which defines define-macro.

[russtoku]

BTW, I think that it's safe to not append (flush-output) with the Racket code being sent to the REPL.

diff --git a/fnl/conjure/client/racket/stdio.fnl b/fnl/conjure/client/racket/stdio.fnl
index 7bbabd22..52798cd8 100644
--- a/fnl/conjure/client/racket/stdio.fnl
+++ b/fnl/conjure/client/racket/stdio.fnl
@@ -61,7 +61,7 @@
             (log.append [(.. comment-prefix "Dropping #lang, only supported in file evaluation.")])
             (s:gsub lang-line-pat ""))
           s)]
-    (.. code "\n(flush-output)")))
+    (.. code "\n")))

 (fn eval-str [opts]
   (with-repl-or-warn

[Olical]

I think it's used for when users print without a newline? I've had people complain about that in the past IIRC 🤔

…

On Sun, 2 Feb 2025, 22:00 Russ Tokuyama, ***@***.***> wrote: BTW, I think that it's safe to not append (flush-output) with the Racket code being sent to the REPL. diff --git a/fnl/conjure/client/racket/stdio.fnl b/fnl/conjure/client/racket/stdio.fnl index 7bbabd22..52798cd8 100644 --- a/fnl/conjure/client/racket/stdio.fnl +++ b/fnl/conjure/client/racket/stdio.fnl @@ -61,7 +61,7 @@ (log.append [(.. comment-prefix "Dropping #lang, only supported in file evaluation.")]) (s:gsub lang-line-pat "")) s)] - (.. code "\n(flush-output)"))) + (.. code "\n"))) (fn eval-str [opts] (with-repl-or-warn — Reply to this email directly, view it on GitHub <#643 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACM6XM3V24FO66RRFJHWLD2N2IRPAVCNFSM6AAAAABWHWYUBCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMRZGU3TKNBYGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[russtoku]

Ahhhh...

[sogaiu]

Thank you both for your investigation and work on this issue!

After some additional help from russtoku on Discord, <localleader>ef now leads to evaluation results instead of an error.

I had some difficulty in getting there, but ultimately what worked for me was to:

• Add the following to init.vim:

  let g:conjure#client#racket#stdio#auto_enter = v:false
  

• Disable the following plugin in init.vim:

  Plug 'benknoble/vim-racket'
  

Without the disabling, I still got what looked like the same (or similar?) errors as seen earlier:

[russtoku]

I can confirm that using benknoble/vim-racket causes an error. It's beyond my current skill level as to why. I checked the keymap for <localleader>ef and it appears to be OK with and without the vim-racket plugin.

The interesting thing is that benknoble/vim-racket is distributed with Vim and Neovim but it's the 2022 Aug 29 version. The current version is 2024 Jun 01. While there are some differences, the plugin appears to be written so that you can use a more recent version by installing it. It's very nicely done.

[Olical]

I wonder if it's overriding the filetype based on the #lang header somehow? Conjure relies on the filetype to work out which client to delegate to, if that option is being played with by something else it could cause these issues where the client is just an empty string / null at eval time.

[russtoku]

The benknoble/vim-racket plugin has a filetype detection script that gets run when *.rkt, *.rktl, and *.zuo files are loaded. It does look at the#lang in the first line of the file and sets the filetype according to its rules.

:se ft?
filetype=reader

Something to be aware of. I'm sure how one would configure Neovim to stop that.

[Olical]

Ah that'll do it! I have a feeling there's a way to do some sort of filetype hierarchy? Maybe that's what this plugin is using so it goes reader -> racket so it inherits other racket filetype stuff but has a unique filetype of reader on the surface. I wonder if there's a way we can query the filetype to get it's "root" of racket rather than the superficial "reader" which currently doesn't map to a client. I don't want to have to teach Conjure all the possible filestypes that could map to racket because that could be infinite. I have a feeling there's a way to tell we're in racket still somehow. And ideally not just by the extension of the file.

…

On Tue, 4 Feb 2025, 19:45 Russ Tokuyama, ***@***.***> wrote: The benknoble/vim-racket plugin has a filetype detection script that gets run when *.rkt, *.rktl, and *.zuo files are loaded. It does look at the #lang in the first line of the file and sets the filetype according to its rules. :se ft? filetype=reader Something to be aware of. I'm sure how one would configure Neovim to stop that. — Reply to this email directly, view it on GitHub <#643 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACM6XLZNTPA6RBN7MCRVPL2OEKGBAVCNFSM6AAAAABWHWYUBCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMZUHEYTCMZXGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[russtoku]

I've come up with a work-around for people who want to use benknoble/vim-racket for the latest version of the Racket support that comes with Neovim.

benknoble-vim-racket's filetype detection sets autocommands that trigger when *.rkt, *.rktl, or *.zuo files are loaded.

I created an after/ftplugin Lua script that is execute after Racket-related scripts are executed. Since it the last compared to scripts from benknoble/vim-racket, it removes the autocommands that set the filetype for Racket files based on the #lang line.

In your config/nvim directory where ever that is, create a subdirectory, after/ftplugin. In the config/nvim/after/ftplugin directory, create racket.lua with:

if (vim.fn.exists("#filetypedetect#BufRead#*.rkt")) then
  vim.cmd("au! filetypedetect BufRead *.rkt,*.rktl,*.zuo")
end

I think that this is one of those things that belongs in the wiki instead of in the source code.

[sogaiu]

I've given the after/ftplugin idea a try.

So far it seems to be working :)

Thanks @russtoku!

[Olical]

So Neovim actually has support for multi part filetype strings, so does Conjure. If the vim-racket plugin set the filetype to racket.scribble for example, instead of just scribble then Conjure would work just fine I think. I wonder if that'd be possible. Maybe we should ask the author of the racket plugin here or open an issue with them to see what they think.