[Relax][PyTorch] Support torch.export.ExportedProgram in Relax PyTorch Frontend

[mshr-h]

This is a tracking issue for supporting PyTorch's torch.export.ExportedProgram in the Relax PyTorch frontend.

TODOs:

• Cleanup the existing TorchFXImporter
  • Cleanup unary op converters [Relax][PyTorch] Cleanup unary op converters #17356
  • Cleanup binary op converters [Relax][PyTorch] Cleanup binary op converters #17366
  • Cleanup Neural Network op converters [Relax][PyTorch] Cleanup Neural Network op converters #17369
  • Cleanup Statistical op converters [Relax][PyTorch] Cleanup Statistical, Search and DataType op converters #17372
  • Cleanup Search op converters [Relax][PyTorch] Cleanup Statistical, Search and DataType op converters #17372
  • Cleanup Tensor Manipulation op converters [Relax][PyTorch] Cleanup Tensor Manipulation and Creation op converters #17376
  • Cleanup Tensor Creation op converters [Relax][PyTorch] Cleanup Tensor Manipulation and Creation op converters #17376
  • Cleanup DataType op converters [Relax][PyTorch] Cleanup Statistical, Search and DataType op converters #17372
  • Cleanup Other op converters [Relax][PyTorch] Cleanup Tensor Manipulation and Creation op converters #17376
• Upgrade PyTorch [CI] Upgrade PyTorch to 2.4.1 #17338
• Upgrade CI image
  • [CI] Update image tag to 20240917-153130-9f281758 #17397
  • [CI] Upgrade unity image tag to 20240917-153130-9f281758 #17410
• Introduce ExportedProgramImporter [Relax][PyTorch] Add support for torch.export.ExportedProgram in Relax PyTorch Frontend #17396
• Expand ops supports
  • unary ops [Relax][PyTorch] Support more unary ops for ExportedProgram importer #17421
  • binary ops [Relax][PyTorch] Support binary, statistical and search ops for ExportedProgram importer #17424
  • neural network ops [Relax][PyTorch] Support neural network ops for ExportedProgram importer #17426
  • statistical ops [Relax][PyTorch] Support binary, statistical and search ops for ExportedProgram importer #17424
  • search ops [Relax][PyTorch] Support binary, statistical and search ops for ExportedProgram importer #17424
  • tensor manipulation ops [Relax][PyTorch] Support tensor manipulation and creation ops for ExportedProgram importer #17429
  • tensor creation ops [Relax][PyTorch] Support tensor manipulation and creation ops for ExportedProgram importer #17429
• Documentation [Relax][PyTorch][Docs] Use torch.export insteamd of fx.symbolic_trace for tutorial #17436

refs:

• [Frontend][PyTorch] torch2.0 ExportedProgram PoC #16531
• torch.export IR Specification
• torch.export Nightly Tutorial
• Definition of the Core ATen Operator Set
• IRs

cc @junrushao @shingjan @yelite

[mshr-h]

This is a comparison of the number of lines of source code.

Current fx_graph_translator

$ tokei fx_translator.py 
===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Python                  1         1947         1691           48          208
===============================================================================
 Total                   1         1947         1691           48          208
===============================================================================

New fx_graph_translator andexportedprogram_translator

$ tokei base_fx_graph_translator.py exportedprogram_translator.py fx_translator.py 
===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Python                  3         2342         1975          113          254
===============================================================================
 Total                   3         2342         1975          113          254
===============================================================================

[tqchen]

we can decouple TVMC support here, since the new relax flow focuses on more modular python based flow. So we can conclude as long as the importer part finishes