Merge pull request #301 from fglock/feature/defer-blocks

Implement Perl defer blocks (use feature 'defer')

Author: fglock

.cognition/skills/debug-perlonjava/SKILL.md

@@ -60,6 +60,32 @@ perl dev/tools/perl_test_runner.pl perl5_t/t/op
 perl dev/tools/perl_test_runner.pl --jobs 8 --timeout 60 perl5_t/t
 ```
 
+### Test runner environment variables
+The test runner (`dev/tools/perl_test_runner.pl`) automatically sets environment variables for specific tests:
+
+```perl
+# JPERL_UNIMPLEMENTED="warn" for these tests:
+re/pat_rt_report.t | re/pat.t | re/regex_sets.t | re/regexp_unicode_prop.t
+op/pack.t | op/index.t | op/split.t | re/reg_pmod.t | op/sprintf.t | base/lex.t
+
+# JPERL_OPTS="-Xss256m" for these tests:
+re/pat.t | op/repeat.t | op/list.t
+
+# PERL_SKIP_BIG_MEM_TESTS=1 for ALL tests
+```
+
+To reproduce what the test runner does for a specific test:
+```bash
+# For re/pat.t (needs all three):
+cd perl5_t/t && JPERL_UNIMPLEMENTED=warn JPERL_OPTS=-Xss256m PERL_SKIP_BIG_MEM_TESTS=1 ../../jperl re/pat.t
+
+# For re/subst.t (only PERL_SKIP_BIG_MEM_TESTS):
+cd perl5_t/t && PERL_SKIP_BIG_MEM_TESTS=1 ../../jperl re/subst.t
+
+# For op/bop.t (only PERL_SKIP_BIG_MEM_TESTS):
+cd perl5_t/t && PERL_SKIP_BIG_MEM_TESTS=1 ../../jperl op/bop.t
+```
+
 ### Interpreter mode
 ```bash
 ./jperl --interpreter script.pl
@@ -123,39 +149,32 @@ git checkout branch && mvn package -q -DskipTests
 ./jperl -e 'failing code'
 ```
 
-### 2. Bisect to find the bad commit
-**IMPORTANT**: Always rebuild after switching commits!
-```bash
-git log master..branch --oneline
-git checkout <commit> && mvn package -q -DskipTests && ./jperl -e 'test'
-```
-
-### 3. Create minimal reproducer
+### 2. Create minimal reproducer
 Reduce the failing test to the smallest code that demonstrates the bug:
 ```bash
 ./jperl -e 'my $x = 58; eval q{($x) .= "z"}; print "x=$x\n"'
 ```
 
-### 4. Compare with system Perl
+### 3. Compare with system Perl
 ```bash
 perl -e 'same code'
 ```
 
-### 5. Use --parse to check AST
+### 4. Use --parse to check AST
 When parsing issues are suspected, compare the parse tree:
 ```bash
 ./jperl --parse -e 'code'                    # Show PerlOnJava AST
 perl -MO=Deparse -e 'code'                   # Compare with Perl's interpretation
 ```
 This helps identify operator precedence issues and incorrect parsing.
 
-### 6. Use disassembly to understand
+### 5. Use disassembly to understand
 ```bash
 ./jperl --disassemble -e 'minimal code'                      # JVM bytecode
 ./jperl --disassemble --interpreter -e 'minimal code'        # Interpreter bytecode
 ```
 
-### 7. Profile with JFR (for performance issues)
+### 6. Profile with JFR (for performance issues)
 ```bash
 # Record profile
 $JAVA_HOME/bin/java -XX:StartFlightRecording=duration=10s,filename=profile.jfr \
@@ -166,14 +185,14 @@ $JAVA_HOME/bin/jfr print --events jdk.ExecutionSample profile.jfr 2>&1 | \
   grep -E "^\s+[a-z].*line:" | sed 's/line:.*//' | sort | uniq -c | sort -rn | head -20
 ```
 
-### 8. Add debug prints (if needed)
+### 7. Add debug prints (if needed)
 In Java source, add:
 ```java
 System.err.println("DEBUG: var=" + var);
 ```
 Then rebuild with `mvn package -q -DskipTests`.
 
-### 9. Fix and verify
+### 8. Fix and verify
 ```bash
 # After fixing
 mvn package -q -DskipTests
@@ -258,6 +277,50 @@ Both backends share the parser (same AST) and runtime (same operators, same Runt
 
 All paths relative to `src/main/java/org/perlonjava/`.
 
+## CRITICAL: Investigate JVM Backend First
+
+**When fixing interpreter bugs, ALWAYS investigate how the JVM backend handles the same operation before implementing a fix.**
+
+The interpreter and JVM backends share the same runtime classes (`RuntimeScalar`, `RuntimeArray`, `RuntimeHash`, `RuntimeList`, `PerlRange`, etc.). The JVM backend is the reference implementation - if the interpreter handles something differently, it's likely wrong.
+
+### How to investigate JVM behavior
+
+1. **Disassemble the JVM bytecode** to see what runtime methods it calls:
+   ```bash
+   ./jperl --disassemble -e 'code that works'
+   ```
+
+2. **Look for the runtime method calls** in the disassembly (INVOKEVIRTUAL, INVOKESTATIC):
+   ```
+   INVOKEVIRTUAL org/perlonjava/runtime/runtimetypes/RuntimeList.addToArray
+   INVOKEVIRTUAL org/perlonjava/runtime/runtimetypes/RuntimeBase.setFromList
+   ```
+
+3. **Read those runtime methods** to understand the correct behavior:
+   - How does `setFromList()` handle different input types?
+   - What methods does it call internally (`addToArray`, `getList`, etc.)?
+
+4. **Use the same runtime methods in the interpreter** instead of reimplementing the logic with special cases.
+
+### Example: Hash slice assignment with PerlRange
+
+**Wrong approach** (special-casing types in interpreter):
+```java
+if (valuesBase instanceof RuntimeList) { ... }
+else if (valuesBase instanceof RuntimeArray) { ... }
+else if (valuesBase instanceof PerlRange) { ... }  // BAD: special case
+else { ... }
+```
+
+**Correct approach** (use same runtime methods as JVM):
+```java
+// JVM calls addToArray() which handles all types uniformly
+RuntimeArray valuesArray = new RuntimeArray();
+valuesBase.addToArray(valuesArray);  // Works for RuntimeList, RuntimeArray, PerlRange, etc.
+```
+
+The JVM's `setFromList()` → `addToArray()` chain already handles `PerlRange` correctly via `PerlRange.addToArray()` → `toList().addToArray()`. The interpreter should use the same mechanism.
+
 ## Common Bug Patterns
 
 ### 1. Context not propagated correctly
@@ -335,10 +398,6 @@ perl -MO=Deparse -e 'code'
 # Compare output
 diff <(./jperl -e 'code') <(perl -e 'code')
 
-# Bisect
-git log master..HEAD --oneline
-git checkout <sha> && mvn package -q -DskipTests && ./jperl -e 'test'
-
 # Git workflow (always use branches!)
 git checkout -b fix-name
 # ... make changes ...

.cognition/skills/interpreter-parity/SKILL.md

@@ -59,6 +59,18 @@ JPERL_INTERPRETER=1 ./jperl script.pl
 JPERL_INTERPRETER=1 ./jperl -e 'code'
 ```
 
+**CRITICAL: eval STRING uses interpreter by default!**
+Even when running with JVM backend, `eval STRING` compiles code with the interpreter.
+This means interpreter bugs can cause test failures even without `--interpreter`.
+
+To trace eval STRING execution:
+```bash
+JPERL_EVAL_TRACE=1 ./jperl script.pl 2>&1 | grep -i interpreter
+```
+
+Fallback for large subs (`JPERL_SHOW_FALLBACK=1`) does NOT show eval STRING usage.
+One-liners won't trigger fallback - test with actual test files!
+
 ## Architecture: Two Backends, Shared Everything Else
 
 ```
@@ -154,6 +166,53 @@ All paths relative to `src/main/java/org/perlonjava/`.
 
 ## Debugging Workflow
 
+### CRITICAL: Save Master Baselines ONCE, Don't Rebuild Repeatedly
+
+**Save master baseline to files FIRST** (do this once per debugging session):
+```bash
+# Switch to master and build
+git stash && git checkout master
+mvn package -q -DskipTests
+
+# Save master test output for JVM backend
+cd perl5_t/t && ../../jperl re/subst.t 2>&1 > /tmp/master_subst.log
+grep "^not ok" /tmp/master_subst.log > /tmp/master_subst_fails.txt
+
+# ALSO save interpreter baseline!
+cd perl5_t/t && ../../jperl --interpreter re/subst.t 2>&1 > /tmp/master_subst_interp.log
+
+# Switch back to feature branch
+git checkout feature-branch && git stash pop
+```
+
+**After making changes**, compare against saved baselines:
+```bash
+mvn package -q -DskipTests
+
+# Test JVM backend
+cd perl5_t/t && ../../jperl re/subst.t 2>&1 > /tmp/feature_subst.log
+diff /tmp/master_subst_fails.txt <(grep "^not ok" /tmp/feature_subst.log)
+
+# MUST ALSO test with interpreter!
+cd perl5_t/t && ../../jperl --interpreter re/subst.t 2>&1 > /tmp/feature_subst_interp.log
+```
+
+### CRITICAL: Always Test with BOTH Backends
+
+A fix that works for JVM backend may break interpreter, or vice versa.
+
+**For quick tests (one-liners):**
+```bash
+./jperl -e 'test code'               # JVM backend
+./jperl --interpreter -e 'test code' # Interpreter backend
+```
+
+**For test files (use env var so require/do/eval also use interpreter):**
+```bash
+./jperl test.t                        # JVM backend
+JPERL_INTERPRETER=1 ./jperl test.t    # Interpreter backend (full)
+```
+
 ### 1. Reproduce with minimal code
 ```bash
 # Find the failing construct
@@ -162,6 +221,18 @@ JPERL_INTERPRETER=1 ./jperl -e 'failing code'
 ./jperl -e 'failing code'
 ```
 
+**CRITICAL: Save baselines to files!** When comparing test suites across branches:
+```bash
+# On master - save results so you don't have to rebuild later
+git checkout master && mvn package -q -DskipTests
+cd perl5_t/t && JPERL_INTERPRETER=1 ../../jperl test.t 2>&1 | tee /tmp/test_master.log
+JPERL_INTERPRETER=1 ../../jperl test.t 2>&1 | grep "^ok\|^not ok" > /tmp/test_master_results.txt
+grep "^ok" /tmp/test_master_results.txt | wc -l  # Save this number!
+
+# Return to feature branch - now you can compare without rebuilding master
+git checkout feature-branch && mvn package -q -DskipTests
+```
+
 ### 2. Use --disassemble to see interpreter bytecode
 ```bash
 JPERL_INTERPRETER=1 ./jperl --disassemble -e 'code' 2>&1

AGENTS.md

@@ -80,3 +80,53 @@ See `.cognition/skills/` for specialized debugging and development skills:
 - `interpreter-parity` - JVM vs interpreter parity issues
 - `debug-exiftool` - ExifTool test debugging
 - `profile-perlonjava` - Performance profiling
+
+## Regression Tracking (feature/defer-blocks branch)
+
+### Summary
+
+All reported regressions have been investigated. The issues fall into two categories:
+1. **Fixed in this branch**: goto-related issues
+2. **Pre-existing on master**: MethodHandle conversion errors, regex octal escape parsing
+
+### Regression Status
+
+| Test | Status | Details |
+|------|--------|---------|
+| op/die_goto.t | **FIXED** (5/5) | `goto &sub` in `$SIG{__DIE__}` handlers now works |
+| uni/goto.t | **FIXED** (2/4) | Tests 1-2 pass (goto &{expr}). Tests 3-4 fail due to pre-existing regex octal escape bug |
+| op/bop.t | Pre-existing | MethodHandle conversion error - fails on master too |
+| op/warn.t | Pre-existing | MethodHandle conversion error - fails on master too |
+| re/subst.t | Pre-existing | MethodHandle conversion error - fails on master too |
+| re/pat_rt_report.t | Pre-existing | MethodHandle conversion error - fails on master too |
+| lib/croak.t | Pre-existing | `class` feature incomplete |
+
+### Fixes Applied in This Branch
+
+1. **EmitControlFlow.java**: Added `handleGotoSubroutineBlock()` for `goto &{expr}` tail call support
+2. **CompileOperator.java**: Added `goto &{expr}` support to bytecode interpreter  
+3. **RuntimeControlFlowList.java**: Added validation for undefined subroutines in tail calls
+4. **RuntimeCode.java**: Added `gotoErrorPrefix()` for "Goto undefined subroutine" error messages
+5. **CompileAssignment.java**: Added `vec` lvalue support for interpreter
+6. **OpcodeHandlerExtended.java**: Fixed `|=` and `^=` to use polymorphic `bitwiseOr`/`bitwiseXor`
+7. **WarnDie.java**: Added TAILCALL trampoline for `goto &sub` in `$SIG{__DIE__}` handlers
+
+### Pre-existing Issues (on master too)
+
+- **MethodHandle conversion errors**: Affects op/warn.t, re/subst.t, re/pat_rt_report.t, op/bop.t
+- **Regex octal escapes**: `\345` in patterns is parsed as backreference `\3` + `45`
+- **op/bop.t**: `new version ~$_` crashes in Version.java
+- **String bitwise ops**: Interpreter uses numeric ops instead of string ops
+
+### How to Check Regressions
+
+```bash
+# Run specific test
+cd perl5_t/t && ../../jperl <test>.t
+
+# Count passing tests
+../../jperl <test>.t 2>&1 | grep "^ok" | wc -l
+
+# Check for interpreter fallback
+JPERL_SHOW_FALLBACK=1 ../../jperl <test>.t 2>&1 | grep -i fallback
+```