Bad optimization by `wasm-opt`

[mkustermann]

I think there's some optimization bug in wasm-opt

See the attached wasmopt_bug.tar.gz file. Reproduction instructions:

# Unpack
% tar xvzf wasmopt_bug.tar.gz 
pkg/dart2wasm/bin/run_wasm.js
dowhile.O1.mjs
dowhile.O1.wasm
dowhile.O1.bad.mjs
dowhile.O1.bad.wasm

# Run the unoptimized version
% third_party/d8/linux/x64/d8  $PWD/pkg/dart2wasm/bin/run_wasm.js -- $PWD/dowhile.O1.mjs $PWD/dowhile.O1.wasm
=> works

# Optimize via wasm-opt
% out/ReleaseX64/wasm-opt --enable-gc --enable-reference-types --enable-multivalue --enable-exception-handling --enable-nontrapping-float-to-int --enable-sign-ext --enable-bulk-memory --enable-threads '--no-inline=*<noInline>*' --closed-world --traps-never-happen --type-unfinalizing -Os --type-ssa --gufa -Os --type-merging -Os --type-finalizing --minimize-rec-groups -g dowhile.O1.wasm -o dowhile.O1.bad.wasm

# Run the optimized version
% third_party/d8/linux/x64/d8  $PWD/pkg/dart2wasm/bin/run_wasm.js -- $PWD/dowhile.O1.bad.mjs $PWD/dowhile.O1.bad.wasm
=> spins infinitely

[kripken]

Thanks for the report, fix in #8014

[kripken]

Side note, wasm-reduce did a really nice job here. Sometimes reducing hanging testcases is annoying (how long do you wait, and if it just happens to run slowly it can be a false positive), but I just put a timeout on the last line of the instructions above and wasm-reduce turned the original 91,440 bytes into just 120 bytes. And it took just a few minutes.

The final testcase was nice enough to easily see the problem quickly:

(module  
 (rec   
  (type $0 (sub (struct)))
  (type $1 (sub final $0 (struct (field i64)))) 
  (type $2 (func))
 )   
 (type $3 (func (param (ref extern))))
 (export "$invokeMain" (func $0))
 (func $0 (param $0 (ref extern))
  (call $1)
 )
 (func $1 (type $2)
  (call $2)
 )
 (func $2 (type $2)
  (local $0 (ref null $0))
  (local $1 i64)
  (loop $label
   (drop
    (ref.as_non_null
     (local.tee $0
      (struct.new $1
       (local.get $1)
      )
     )
    )
   )
   (local.set $1
    (i64.const 1)
   )
   (br_if $label
    (i32.eqz
     (block $block1 (result i32)
      (block $block
       (drop
        (br_on_null $block
         (local.get $0)
        )
       )
       (br $block1
        (i32.wrap_i64
         (struct.get $1 0
          (ref.cast (ref $1)
           (local.get $0)
          )
         )
        )
       )
      )
      (i32.const 0)
     )
    )
   )
  )
 )
)

(Running the optimizer on that removed far too much code, before the fix.)