[0035] Revisit Per-Element Matrix Operations

[mapodaca-nv]

Which proposal does this relate to?
0035 - linalg::Matrix

Describe the issue or outstanding question.
The current list of unary operations provided by ApplyUnaryFunction and binary operations provided by Matrix class operators are not sufficient for activation and reduction functions. Furthermore, a programmable method would be preferred over a set of built-in functions.

The possible solutions could be:

1. Preferred A map(func, Matrix) function which takes a function reference as an argument. The arguments of the mapped function would provide row and col information, or an element index that can be converted to row and col through an intrinsic function.
2. A Matrix::Length function, which returns the number of elements per-thread, for iterating over, along with a Matrix::operator[] for accessing the elements using a per-thread index.
3. Extend the current enum class UnaryOperation with additional unary operations (e.g., sqrt, ceil, reciprocal, relu, sigmoid, tanh, etc.) and add a ApplyBinaryOperation function with enum class BinaryOperation (e.g., max, pow, atan2, hypot, etc.).

Option 1 provides the most flexibility and also enables additional use cases. While Option 3 is the most limiting for developers and requires constant updates to the specification.

Additional context
See also PR 598 comment from @nhaehnle .

[llvm-beanz]

The problem with option 1 is that it requires changes to the core HLSL language, not just an API exposure. That means requiring HLSL 202x as well as figuring out the language rules for function pointers in addition to the DXIL related changes to support function pointers.

While option 3 is pretty limited, I think it is extremely unlikely we can implement option 1 in a reasonable timeline with our current resourcing.