Fixing max tokens for Anthropic + context reset + initial command edition + adding -d query

[cryptax]

Checklist

• Fixing issues (but they weren't opened)
• Mark this if you consider it ready to merge
• I've tested
• I wrote some documentation

Description

This merge is fixing several issues:

1. max_tokens. (BUG FIX). This setting is present in the configuration, but it was not set in the requests sent to Anthropic. The value for Anthropic was hard-coded to a low max. So, it was impossible to take advantage of Claude 3.7 64,000 max output tokens!

2. context reset. (BUG FIX). When we issued r2ai -R the context was not reset. Somehow, the useful code had been partly commented out. Now, context reset works fine in auto mode. In direct mode, we currently do not save context, so the option is useless: context is reset automatically each time.

3. command edition (BUG FIX). In auto mode, a first initial command is sent along with the question. The initial commands are in a configuration settings, but they can be modified live by the end user. In the context which was sent, the command remaining not modified, hence not matching the output. This has been fixed. Note that there is a still a bug to fix when a tool command is edited. The context is not changed, and it should.

4. query in direct mode (NEW FEATURE). I added the possibility to ask a question on the current function. It's -d query. It was very simple to implement, and quite helpful in the end.

5. max_input_tokens. I introduced earlier a new setting to tell r2ai the maximum input token, and truncate any request above this limit. However, my commit was incorrectly assuming a token was a character, which is wrong. A token is a word, a syllab... it really depends. So, actually, the setting max_input_tokens does not make sense. Consequently I removed it.

The rest is documentation, or minor modifs, or merge commits.

[cryptax]

Forgot to mention I added a simple binary to examples for documentation purposes.

[cryptax]

I see the CI build is failing on Linux, but it works at my place. Would it be somehow possible that your build-linux isn't updated with the latest r2 lib?

/usr/include/libr/r_cons.h:954:39: note: expected ‘const char *’ but argument is of type ‘RCons *’ {aka ‘struct r_cons_t *’}
  954 | R_API void r_cons_println(const char* str);
      |                           ~~~~~~~~~~~~^~~
r2ai.c:433:33: error: too many arguments to function ‘r_cons_println’
  433 |                                 r_cons_println (cons, "/help    show this help");
      |                                 ^~~~~~~~~~~~~~

On my host:

axelle@alligator:~/git/r2ai/src$ make
cc -I/usr/local/include/libr  -I. -fPIC -Wall -g -DUSE_R2_CURL=1 -DUSE_LIBCURL=0   -c -o r2ai.o r2ai.c
cc -I/usr/local/include/libr  -I. -fPIC -Wall -g -DUSE_R2_CURL=1 -DUSE_LIBCURL=0   -c -o auto.o auto.c
cc -I/usr/local/include/libr  -I. -fPIC -Wall -g -DUSE_R2_CURL=1 -DUSE_LIBCURL=0   -c -o vdb.o vdb.c
cc -I/usr/local/include/libr  -I. -fPIC -Wall -g -DUSE_R2_CURL=1 -DUSE_LIBCURL=0   -c -o tools.o tools.c
cc -I/usr/local/include/libr  -I. -fPIC -Wall -g -DUSE_R2_CURL=1 -DUSE_LIBCURL=0   -c -o messages.o messages.c
cc -I/usr/local/include/libr  -I. -fPIC -Wall -g -DUSE_R2_CURL=1 -DUSE_LIBCURL=0   -c -o anthropic.o anthropic.c
cc -I/usr/local/include/libr  -I. -fPIC -Wall -g -DUSE_R2_CURL=1 -DUSE_LIBCURL=0   -c -o r2ai_http.o r2ai_http.c
cc -I/usr/local/include/libr  -I. -fPIC -Wall -g -DUSE_R2_CURL=1 -DUSE_LIBCURL=0   -c -o openai.o openai.c
cc -I/usr/local/include/libr  -I. -fPIC -Wall -g -DUSE_R2_CURL=1 -DUSE_LIBCURL=0   -c -o markdown.o markdown.c
cc -fPIC -shared -o r2ai.so r2ai.o auto.o vdb.o tools.o messages.o anthropic.o r2ai_http.o openai.o markdown.o -I/usr/local/include/libr  -I. -fPIC -Wall -g -DUSE_R2_CURL=1 -DUSE_LIBCURL=0 -L/usr/local/lib -lr_core -lr_config -lr_debug -lr_bin -lr_lang -lr_anal -lr_bp -lr_egg -lr_asm -lr_arch -lr_esil -lr_flag -lr_reg -lr_search -lr_syscall -lr_fs -lr_io -lr_socket -lr_cons -lr_magic -lr_muta -lr_util -ldl 

[trufae]

cherry picked by hand. do not merge commits pls :(