adding UI test to autodiff

[KMJ-007]

r? @ZuseZ4

[ZuseZ4]

You'll need the PrintTA flag, look in the rustc-dev-guide for the exact spelling. That should at least give you some typetree metadata.

You likely also want to first merge support for the more specific flag we discussed where you can pass a specific function name. Look into enzyme to find out the name of the flag and grep for how enzyme handles it internally. We likely want to pass that flag to enzyme not via the command line, since we call libEnzyme, which doesn't accept command line flags.
Instead, you should pass it just like all the other flags I'm passing to Enzyme, you can find the handling in rustc_codegen_llvm/src/back/write/lto.rs. First try to pass a hard-coded function name to enzyme, and make it print TA just for that function. We can later generalize it.

I'd focus for the next days on first getting the flags tested and supported, then you can continue with the tests here.

[KMJ-007]

this changes are for fixing PrintTAFn flag, which was not working, should i create new PR for this?

[ZuseZ4]

No need for it, just make it it's own commit and that the history is clean.
E.g. end up with a first commit fixing the Wrapper, and a second commit introducing the tests.

[KMJ-007]

Yeah, I will clean the history after I am done writing tests at the end

[ZuseZ4]

I'd assume that the dbg/noundef numbers like 384, 385 and 14 will change soon enough. Best case you replace all fixed constants like 384 and 385 with different variables, there's probably a good regex for it. I.e. on the first use of 384 you assign a name to it, and on all following usages of 384 you assure that it's the same name (value) showing up. There should be examples in Enzyme and the LLVM lit docs on how to do that.
Otherwise at least make sure that all of these two numbers are replaced with an anonymous placeholder, so if they change tests don't break.

Edit: now that we have a UI test ignore the comment about LLVM's lit test, still check if there's a way to avoid test failures if the numbers change. cc @jieyouxu who might have a suggestion.

[jieyouxu]

I'm not sure how you would meaningfully normalize these. Maybe instead, use FileCheck inside a run-make test?

Otherwise, this is going to break

[KMJ-007]

i am thinking of using this regex:

//@ normalize-stderr: "!(dbg|noundef) ![0-9]+" -> "!$1 !N"
//@ normalize-stderr: "%[0-9]+" -> "%X"
//@ normalize-stdout: "!(dbg|noundef) ![0-9]+" -> "!$1 !N"
//@ normalize-stdout: "%[0-9]+" -> "%X"

[jieyouxu]

That normalization can still be fragile, because can't the instruction ordering also change?

[workingjubilee]

Yes, any time there are multiple subsequent loads, they're independent enough that, while LLVM usually generates them in a reasonably reliable order, we do change things enough in our codebase that sometimes we change those orderings around. So we would want CHECK-DAG for those lines.

[ZuseZ4]

@workingjubilee @jieyouxu these are currently ui tests since we test stderr/stdout output. Should they be something else? I don't think we can use the LLVM LIT test infrastructure in ui, or do we have a Rust equivalent which makes CHECK-DAG work?

In general, I want to make sure that we see all the type analysis info which enzyme deduces written out, even if it's looking fragile for now. His next PRs will start lowering Rust information to llvm metadata, which should be more reliable if we do it properly. In that case Enzyme will need to deduce less types, and we can narrow down what we're actually testing for (which is the metadata generated by us).

[workingjubilee]

We have "run arbitrary code to execute the test" tests, called "run-make" because they used to be makefiles, but they actually are Rust executables now: https://github.com/rust-lang/rust/blob/master/tests/run-make/README.md

You can use that to run filecheck:

rust/src/tools/run-make-support/src/external_deps/llvm.rs

Line 23 in 4e97337

pub fn llvm_filecheck() -> LlvmFilecheck {

[jieyouxu]

Yeah, I think using FileCheck within a run-make test is going to be least fragile, because you can also "don't care" about attributes and whatever and only match on stuff you do care about.

[rustbot]

Some changes occurred in src/tools/enzyme

cc @ZuseZ4

[rustbot]

⚠️ Warning ⚠️

• Some commits in this PR modify submodules.

• The following commits have merge commits (commits with multiple parents) in your changes. We have a no merge policy so these commits will need to be removed for this pull request to be merged.

  • 3c0c1a6

  You can start a rebase with the following commands:

  $ # rebase
  $ git pull --rebase https://github.com/rust-lang/rust.git master
  $ git push --force-with-lease
  

[KMJ-007]

@ZuseZ4 what should i do about UI test?

[ZuseZ4]

I'd follow the suggestions from jieyouxu and jubilee to move to run-make tests, so you can match more precisely on the error messages. There are a lot of lines in this PR, and we don't want to spend too much time maintaining it.
You'll also want to use run-make on your actual TypeTree PR since there you'll often need to make sure that one specific line looks the way we want it, so learning how to get llvm's test infra to work here should help you with the next PR too.