Problem
The parser fails to parse library/numerics/testutils.pl because it uses Unicode characters in predicate names:
Error:
syntax_error(No terminal matches 'δ' in the current parser context)
,δ_inverses_t/5
^
Root Cause
The Lark grammar in vibeprolog/parser.py defines atoms/identifiers using ASCII character patterns only. Greek letters and other Unicode characters are not recognized as valid atom characters.
Solution
Extend the parser's atom pattern to include Unicode letter categories:
- Update the ATOM token pattern in the Lark grammar to accept Unicode letters
- Use Unicode character classes in the regex: [\p{L}] (letters) or similar
- Ensure the pattern still maintains valid Prolog identifier semantics
- Test against both ASCII and Unicode predicate names
Implementation Details
- Review vibeprolog/parser.py for the ATOM token definition
- Update the regex pattern to include Unicode letters
- Ensure backward compatibility with ASCII-only identifiers
- Handle Unicode in quoted and unquoted atoms consistently
Testing Requirements
-
Create comprehensive test file tests/test_unicode_identifiers.py covering:
- Greek letters in predicate names (α, β, γ, δ, etc.)
- Other Unicode letters (é, ñ, ü, etc.)
- Unicode in compound terms
- Unicode in quoted atoms
- Unicode in variable names (should still be uppercase or underscore)
- Mixed ASCII and Unicode identifiers
- Ensure proper distinction between atoms and variables
-
Verify library/numerics/testutils.pl loads successfully
-
Ensure no regressions in existing parser tests
-
Test that semantics are unchanged (Unicode atoms still unify correctly, etc.)
Documentation
Update FEATURES.md to document:
- Parser now supports Unicode letters in atom identifiers
- Examples of valid Unicode atom names
- Any restrictions on which Unicode characters are supported
Problem
The parser fails to parse library/numerics/testutils.pl because it uses Unicode characters in predicate names:
Error:
Root Cause
The Lark grammar in vibeprolog/parser.py defines atoms/identifiers using ASCII character patterns only. Greek letters and other Unicode characters are not recognized as valid atom characters.
Solution
Extend the parser's atom pattern to include Unicode letter categories:
Implementation Details
Testing Requirements
Create comprehensive test file tests/test_unicode_identifiers.py covering:
Verify library/numerics/testutils.pl loads successfully
Ensure no regressions in existing parser tests
Test that semantics are unchanged (Unicode atoms still unify correctly, etc.)
Documentation
Update FEATURES.md to document: