0.23.13

Minor changes to 0.23.12 for Go target. #531

0.23.12

This point release is to change the code in trgen to allow different parser generators. We now have Antlr-ng, which is port of Antlr4 to TypeScript. I am planning to make Trash not so Antlr-specific.

Fixes #518, #524, #525, #527.

0.23.11

This is a point release for the Trash Toolkit. There are only a couple of minor changes to correct some bugs.

• Fix trgen to handle names like star.g4. The code did have a bug in considering those files as template files, e.g., st.build.sh, where the name should begin with st. to indicate that it is a template file. That error caused a cascade of problems in generating the driver.
• Fix trgen to accept template files in the "target-specific" code directories for a file. Prior to this change, target-specific code was just copied. This is necessary to have a st.CMakeLists.txt template file so it can be instatiated with OS specific C++ compiler options, for Windows, Linux, and Mac OS. antlr/grammars-v4#4362

0.23.9

This is a minor change.

• Fixed trgen problems with grammar files that end with st.g4, e.g., tst.g4. Trgen would not create a good driver for the grammar. I included a test to make sure this doesn't regress with a future change.
• Fixed the built-in grammar for Bison, so that instead of recursion, kleene closure is used. This cuts back on excessive stack size for parsing some Bison grammars. Here are some example rules that were changed.

token_decls_for_prec
    : token_decl_for_prec+
    | TAG token_decl_for_prec+
    | token_decls_for_prec TAG token_decl_for_prec+
    ;
symbol_decls
    : symbol+
    | TAG symbol+
    | symbol_decls TAG symbol+
    ;
bison_grammar
    : rules_or_grammar_declaration
    | bison_grammar rules_or_grammar_declaration
    ;
params
    : params actionBlock
    | actionBlock
    ;

vs.

token_decls_for_prec
    : (token_decl_for_prec+ | TAG token_decl_for_prec+) (TAG token_decl_for_prec+)*
    ;
symbol_decls
    : (symbol+ | TAG symbol+) (TAG symbol+)*
    ;
bison_grammar
    : rules_or_grammar_declaration+
    ;
params
    : actionBlock+
    ;

• trclonereplace was fixed up to perform proper rule cloning and replacing.
• trextract was added. _I haven't really refind what this command should do. But, essentially, I want to have it extract out actions and replace with macros. I can then add a tool to expand the macros. This code is intended to fix the "target agnostic problem."
• tragl now has "multi-tree" display. Prior to this change, each tree passed via a parsing result set would have to be viewed in order, and closed off each time. Instead, we can now view all the trees simultaneously, and compared. There are two modes for trparse: one to not group, and the other to group. This affects how tragl displays the trees, whether in one "doc" window, or multiple. The default is "ungrouped", i.e., multiple.

12/14-20:44:10 ~/issues/g4-current/c/Generated-CSharp
$ trparse --ambig ../examples/TypeCast.c | tragl
CSharp 0 ../examples/TypeCast.c success 0.071354
12/14-20:44:40 ~/issues/g4-current/c/Generated-CSharp

• trgen templates for performance. The templates for the drivers where changed a little to help in tracing performance issues. For example, instead of Total Time: 123.123 being displayed, a prefix can be passed in to display group Total Time: 123.123. This helps with another change for testing performance and extracting out information for statistical analysis. Some of the old double-minus -- syntax for dotnet calls were fixed as well, as this is not accepted by the dotnet tool anymore. The getAllPossibleParseTrees() function was changed in the CSharp templates to now return a string corresponding to the "decision/alt" for an ambiguous parse tree that was computed. This helps one to understand where grammar ambiguity resides. This could be considered a "major change", but since most don't use Trash, this isn't a problem.
• trglob now produces a list of files and directories with relative paths instead of absolute.

0.23.10

This release is a minor change.

The trtree app was updated with block tree output, and it is now the default. This output style incorporates the Unicode characters for edges. This helps to visually identify what children belong to which parent.

So, for example, with the Arithmetic grammar, and the input string 1+2*3, the following block tree is produced.

$ trparse -i '1+2*3' | trtree
CSharp 0 string success 0.0237195
file_
├── expression
│   ├── expression
│   │   └── atom
│   │       └── scientific
│   │           └── SCIENTIFIC_NUMBER
│   │               └── "1"
│   ├── PLUS
│   │   └── "+"
│   └── expression
│       ├── expression
│       │   └── atom
│       │       └── scientific
│       │           └── SCIENTIFIC_NUMBER
│       │               └── "2"
│       ├── TIMES
│       │   └── "*"
│       └── expression
│           └── atom
│               └── scientific
│                   └── SCIENTIFIC_NUMBER
│                       └── "3"
└── EOF
    └── ""


12/13-11:05:02 ~/temp7/Generated-CSharp

An interesting note: The code for this was derived from ChatGPT (https://chatgpt.com/share/675ed066-4994-8007-aeee-db755431f6b3). The code had a bug (it listed directories twice, and even when pointed out, could not correct it), but I fixed that, and adapted it from files and directories to parse trees.

In the trgen app, the templates were adjusted to compute and output token stream indices for tokens. The ambiguity for a performance test was updated--ambiguity testing is only available with trperf, and that cannot be computed for any target other than CSharp.

Again, with the Arithmetic grammar, the raw driver produces the following.

$ ./bin/Debug/net8.0/Test.exe -input '1+2*3' -tokens
[@0,0:0='1',<2>,1:0]
[@1,1:1='+',<5>,1:1]
[@2,2:2='2',<2>,1:2]
[@3,3:3='*',<7>,1:3]
[@4,4:4='3',<2>,1:4]
[@5,5:4='<EOF>',<-1>,1:5]

CSharp 0 string0 success 0.0159526
Total Time: 0.1538565
12/13-11:08:00 ~/temp7/Generated-CSharp

0.23.8

This release is a minor update.

• Fixes #494.
• Fixes #496.
• Fixes #497. This change adds trnullable to the toolkit. Note, it outputs a list of pairs symbol name x is nullable. This may change in the future to output a PRS so this can be processed by trcaret.

0.23.7

This release is a minor update for trgen and trcover.

• Updates the analysis of Antlr4 grammar files. Tuples are created for each recognizer created, including two for combined grammars. It relates all these properly now.
• Fixes a minor problem with trcover when the current directory is not the "Generated" directory.
• Removes the "double-minus" from the dotnet tool commands in all the trgen templates. Apparent it is no longer required. E.g., dotnet trgen -- --version was outputting an error rather than the version of the tool.
• Minor change to the Github Actions for installing dotnet. I don't know why, but setup-dotnet was failing on making a directory for dotnet.
• Minor changes for testing the toolkit better.

Release 0.23.6

This is a minor release of the Trash Toolkit for build regressions over in grammars-v4 due to changes in trgen.

• Fixed build regressions in trgen. I was changing some logic for the analysis of grammars, and ended up breaking the code for some test cases.
• Added selected grammars from grammars-v4 to testing of Trash with every commit.
• Changed templates for trgen to always use "local" dotnet tool for Trash. The templates between Trash and that in grammars-v4 are now identical.
• Updated how TypeScript drivers for trgen are called.

0.23.5

Minor release, which adds features for tracking ambiguity within a grammar.

• Added the command-line option --ambig to trparse to output parse trees of ambiguity. Each tree is printed with a unique file name that is composed from the original file name plus the decision number.
• Added file name to output of trtree when there are multiple parsing result sets.
• Added Antlr4 parse tree output to trtree.
• Added indented parenthesis-less output to trtree.
• Refactored some of the underlying code for parse tree output.
• Updated the CSharp templates in target for collecting information on parse tree ambiguity.
• Updated the dependencies for the trgenvsc-generated extension.

0.23.4

Minor fix and new feature.

• trperf. The value for column "a" (ambiguities) was incorrectly implemented, counting the number of errors.
• trgenvsc. The settings.rc JSON file is now initialized when "install.sh" is run. It is set up with syntactic highlighting categories.