Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: benchdnn: support --mode-modifer=M for graph driver #2880

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

wzt1997
Copy link
Contributor

@wzt1997 wzt1997 commented Mar 14, 2025

General

The PR enables --mode-modifer=M for benchdnn graph driver:

  1. Avoid creating reference primitives as well as the memories and customized data filling.
  2. For customized operations, handle integral inputs specially for indexing values.
  3. Otherwise, use fixed value( 0x3F3F3F3F (0.747059f) ) for data filling.
./tests/benchdnn/benchdnn --graph --mode=F --batch=test_graph_all
Output template: perf,%engine%,%prb%,%-time%,%0time%
perf,cpu,--mode=F --graph --case=op/f32/abs.json,0.0561523,0.139608
perf,cpu,--mode=F --graph --case=op/f32/abs_bwd.json,0.0541992,0.064628
perf,cpu,--mode=F --graph --case=op/f32/add.json,0.0571289,0.0702186
perf,cpu,--mode=F --graph --in-shapes=1:- --case=op/f32/add.json,0.0563965,0.0670133
perf,cpu,--mode=F --graph --case=op/f32/avgpool.json,4.81152,10.1638
perf,cpu,--mode=F --graph --op-attrs=0:auto_pad:SAME_UPPER*kernel:2x2 --case=op/f32/avgpool.json,4.58423,6.78208
//....

Sanitizer test:

        Start 201: test_benchdnn_modeR_graph_ci_cpu
199/220 Test #201: test_benchdnn_modeR_graph_ci_cpu ........................   Passed   21.13 sec

        Start 233: test_benchdnn_modeR_graph_ci_gpu
233/252 Test #233: test_benchdnn_modeR_graph_ci_gpu .........................   Passed   73.62 sec

@wzt1997 wzt1997 added the component:graph-api Codeowner: @oneapi-src/onednn-graph label Mar 14, 2025
@wzt1997 wzt1997 self-assigned this Mar 14, 2025
@github-actions github-actions bot added the component:tests Codeowner: @oneapi-src/onednn-arch label Mar 14, 2025
@wzt1997 wzt1997 force-pushed the zhitao/grpah-enable-mode-modifer-m branch from 250f997 to 633dd37 Compare March 14, 2025 03:05
@wzt1997
Copy link
Contributor Author

wzt1997 commented Mar 17, 2025

make test
set test_scope=NIGHTLY
disable benchdnn_all
enable benchdnn_graph

@wzt1997 wzt1997 force-pushed the zhitao/grpah-enable-mode-modifer-m branch from 633dd37 to 13d0901 Compare March 18, 2025 01:46
@wzt1997 wzt1997 changed the title [WIP]tests: benchdnn: support --mode-modifer=M for graph driver tests: benchdnn: support --mode-modifer=M for graph driver Mar 18, 2025
@wzt1997 wzt1997 marked this pull request as ready for review March 18, 2025 02:39
@wzt1997 wzt1997 requested review from a team as code owners March 18, 2025 02:39
@wzt1997 wzt1997 force-pushed the zhitao/grpah-enable-mode-modifer-m branch from 13d0901 to c58a631 Compare March 18, 2025 07:14
@wzt1997
Copy link
Contributor Author

wzt1997 commented Mar 19, 2025

make test
set test_scope=NIGHTLY
disable benchdnn_all
enable benchdnn_graph

@wzt1997 wzt1997 force-pushed the zhitao/grpah-enable-mode-modifer-m branch 2 times, most recently from b1ca911 to 562593f Compare March 19, 2025 03:11
@wzt1997 wzt1997 force-pushed the zhitao/grpah-enable-mode-modifer-m branch from 562593f to 162bff8 Compare March 19, 2025 06:50
lt_id_2_mems_.at(id), lt_id_2_lt_.at(id), true));
} else if (has_bench_mode_modifier(mode_modifier_t::no_ref_memory)) {
partition_mem_map.emplace(
id, dnn_graph_mem_t(lt_id_2_lt_.at(id), true));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that internal mem_ can be constructed without lt_id_2_mems_.
I suggest to change the logic to following:

    // Emplace a memory object first.
    partition_mem_map.emplace(
                    id, dnn_graph_mem_t(lt_id_2_lt_.at(id), true));
    if (lt_id_2_mems_.at(id)) {
        SAFE(fill_underlying_mem_with_data(partition_mem_map[id], lt_id_2_mems_.at(id)), WARN);
    }

This way the flow is always the same and no_ref_memory will affect only the if part transparently without hurting a general logic.

Copy link
Contributor Author

@wzt1997 wzt1997 Mar 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For correctness check, we need to create graph memories based on primitive memory layouts, as benchdnn will reorder the outputs from both paths to abx during comparison. Therefore, the memory tags of outputs from two paths have to be identical.

For simplicity, I have done the following changes:

  1. Move the data filling part to an independent function.
  2. Remove the function that creates graph memories from graph logical tensors, and keep the primitive memory parameter.

@wzt1997 wzt1997 force-pushed the zhitao/grpah-enable-mode-modifer-m branch from 162bff8 to 79d435e Compare March 20, 2025 14:39
@wzt1997
Copy link
Contributor Author

wzt1997 commented Mar 24, 2025

make test
set test_scope=NIGHTLY
disable benchdnn_all
enable benchdnn_graph

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:graph-api Codeowner: @oneapi-src/onednn-graph component:tests Codeowner: @oneapi-src/onednn-arch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants