[WIP][GPU] Expand matmul decomp cases

[kealan-barbieri]

Description

Expands cases where inputs are decompressed to include integer activations as well as weights. Integer weights are still required but in the presence of fpmath setting activations will also be upconverted.

This means that cases intending to use integer accumulation must not supply eg attr-fpmath=f16:true as all such cases must be upconverted per https://jira.devtools.intel.com/browse/MFDNN-13380.

Fixes # (github issue)

Checklist

General

• Do all unit and benchdnn tests (make test and make test_benchdnn_*) pass locally for each commit?
• Have you formatted the code using clang-format?

[rjoursler]

This is invalid, it will just lead to out-of-register issues like fixed in #2467 as kernels optimized for B may not have enough spare registers for upconversion. If some of our kernel strategies on B are functional for [OB], then they should just be marked as such.

Additionally, we should not need multiple calls to add_mode_matches, instead, we should get

add_mode_matches(fpmath_bf16, [](Type dt) -> const char * {
        if (dt == Type::f32) { return "[SB]"; }
        if (dt.isInt8() || dt.isInt4()) return "[OB]";
        if (dt.isF8()) return "B"; // This seems invalid and could could lead to out of registers, Need to determine appropriate specifier for GEMM.
}

[kealan-barbieri]

I see what you mean, but we need to strike a balance between only supporting specifically optimized cases and out of box functionality. There is an issue with the current approach - there is no fallback from "[OB]" type strategies to "[B]" type due to the selection process.

I suggest we take the opposite approach and mark strategies that will not tolerate upconversion. Optimized strategies with "[OB]" tags should be prefered on a performance basis when theyre useable.

[rjoursler]

I suggest we take the opposite approach and mark strategies that will not tolerate up conversion.

This is what we are doing already, it is just that B is how we mark that the kernel will not support up conversion. 😉

[kealan-barbieri]

Refactored, renamed all "O", "H", "S" strategies to `"[OH]", "H", "S"' to conform with naming convention.

[kealan-barbieri]

make test
disable device_cpu
enable device_gpu
disable benchdnn_all
enable benchdnn_matmul