concat2 internal padding

[syurkevi]

Description

This PR improves performance for concat with blocked internal padding, cases such as aBc32b:aBc32b 2x33x2:2x32x2 that contain padding inside the concat dimension that must be removed. Better bandwidth utilization has been achieved through a new specialized kernel that attempts to form fully utilized write cache lines from overlapping aligned reads.

Currently, the kernel is specialized to only n=2 inputs and has some restrictions wrt/simd size. The change should address a large portion of the problem layers identified in issue MFDNN-9174.

Since the approach is fairly generic, a geomean speedup of roughly ~30% is observed across multiple platforms. Peak speedup of certain problem layers reaches ~70%.

The kernel has been kept within the reusable_simple_concat implementation. This may need to be revisited to determine the optimal code organization.

Checklist

General

• Do all unit and benchdnn tests (make test and make test_benchdnn_*) pass locally for each commit?
• Have you formatted the code using clang-format?

Performance improvements

• Have you submitted performance data that demonstrates performance improvements?

[syurkevi]

make test perf-gpu
set primitive=concat

[dzarukin]

That's too much to validate a pretty narrow new use case.
Please reduce and try to incorporate into existing files.
And this file won't be picked for GPU validation besides that.

Additionally, would be nice to understand where the blocked layout comes from because, AFAIK, activations must move to nxc format for whole model since Xe2.

[syurkevi]

Reduced the number of tests and incorporated into gpu validation. Do let me know if there are still too many.

There has been some back and forth on the necessity of block support. There were talks it'll still be needed with future platforms but don't recall the details for which customer. Would like to close this narrow case performance gap regardless and will revisit with more improvements(ex: 3+ inputs, large block sizes w/small simd, flexible conf, etc) if there is demand again.

[syurkevi]

make test
disable test_device_cpu
disable benchdnn_all
enable benchdnn_concat

[atkassen]

make test perf-gpu
set primitive=concat

[syurkevi]

make test perf-gpu
set primitive=concat

[syurkevi]

make test perf-gpu
set primitive=concat

[syurkevi]

make test perf-gpu
set primitive=concat