Use env variables or not

[chentong319]

Sometimes I was puzzled by the behavior of some execution or failed to reproduce something because of the hidden env variable. I am wondering whether we should use env variable. If yes, what are the general rules of using env variables in onnx-mlir project?
First of all, I asked someone. Here is the answer:

✅ When It's a Good Approach:

• Configuration Outside of Code:
  Environment variables allow you to control settings without changing code or configuration files. This makes deployment easier, especially in cloud environments (e.g., AWS, Heroku, Docker).

• Sensitive Data Handling:
  Ideal for credentials, API keys, and secrets since they aren't stored in code or version control.

• Cross-Environment Flexibility:
  Useful for configuring behavior across dev, staging, and production environments (e.g., switching databases, toggling debug modes).

• Stateless Applications:
  In containerized environments (like Docker or Kubernetes), environment variables help maintain statelessness by providing configuration at runtime.

❌ When It's Problematic:

• Lack of Visibility:
  If someone runs the program without knowing the necessary environment variables, they may encounter confusing errors.

• Not Always Portable:
  Moving the program between systems can be tricky if the required environment variables aren't documented or set correctly.

• Security Risks:
  If not managed carefully, sensitive data in environment variables can be exposed (e.g., via process lists or misconfigured shells).

• Complexity in Large Projects:
  Heavy reliance on environment variables can lead to "configuration sprawl," making it hard to manage and debug.

I tried to summarize all the uses of env variable in onnx-mlir:

1. Control Cmake: ZDNN_LIBRARY_DIR
2. onnx-mlir execution: IMPORTER_FORCE_DYNAMIC, MallocStackLogging,
3. execution of the compiled model:

• instrumentation: ONNX_MLIR_INSTRUMENT_FILE, ONNX_MLIR_NO_INSTRUMENT_TIME
• NNPA debugging: OM_STATUS_MESSAGES_ENABLED,
• python driver: ONNX_MILR_HOME, MODELS_PATH, PY_LIB, VERBOSE, TEST_NO_FLOAT16, TEST_CASE_BY_USER, and a lot!

What should we do about them and what is the guideline/plan for future?

[AlexandreEichenberger]

Another big one is ONNX_MLIR_FLAGS that are used to put default flags to the onnx-mlir compiler.

Thanks @chentong319 for starting this discussion. I agree that we have added flags left and right. I agree that there are 3 kinds

1. ones to govern the building of the compiler (aka to drive cmake...)
2. ones to govern the execution of the compiler
3. one to govern the execution of the binary

And for each, a sub-category of what is potentially needed by the users vs to enable certain testing scripts.

Is your suggestion to make an exhaustive list, sort them in these categories, and see if we can remove some?