Skip to content

Commit d8a40b5

Browse files
MenookerMenooker
authored
graph: backend: introduce graph compiler (#5103)
1 parent 51c40c7 commit d8a40b5

File tree

644 files changed

+172522
-15
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

644 files changed

+172522
-15
lines changed

‎README.md

+1
Original file line numberDiff line numberDiff line change
@@ -328,6 +328,7 @@ terms.
328328

329329
Apache License Version 2.0:
330330
* [Xbyak_aarch64](https://github.com/fujitsu/xbyak_aarch64)
331+
* [LLVM](https://llvm.org)
331332

332333
Boost Software License, Version 1.0:
333334
* [Boost C++ Libraries](https://www.boost.org/)

‎THIRD-PARTY-PROGRAMS

+225-1
Original file line numberDiff line numberDiff line change
@@ -467,7 +467,7 @@ limitations under the License.
467467
END OF TERMS AND CONDITIONS
468468

469469
--------------------------------------------------------------------------------
470-
6. Boost C++ Libraries (src/common/primitive_hashing.hpp)
470+
6. Boost C++ Libraries (src/common/primitive_hashing.hpp, src/graph/backend/graph_compiler/core/src/util/hash_utils.hpp)
471471
Copyright 2005-2014 Daniel James.
472472

473473
Boost Software License - Version 1.0 - August 17th, 2003
@@ -581,3 +581,227 @@ THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
581581
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
582582
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
583583
IN THE SOFTWARE.
584+
585+
10. LLVM (src/graph/backend/graph_compiler/core/src/util/array_ref.hpp)
586+
==============================================================================
587+
The LLVM Project is under the Apache License v2.0 with LLVM Exceptions:
588+
==============================================================================
589+
590+
Apache License
591+
Version 2.0, January 2004
592+
http://www.apache.org/licenses/
593+
594+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
595+
596+
1. Definitions.
597+
598+
"License" shall mean the terms and conditions for use, reproduction,
599+
and distribution as defined by Sections 1 through 9 of this document.
600+
601+
"Licensor" shall mean the copyright owner or entity authorized by
602+
the copyright owner that is granting the License.
603+
604+
"Legal Entity" shall mean the union of the acting entity and all
605+
other entities that control, are controlled by, or are under common
606+
control with that entity. For the purposes of this definition,
607+
"control" means (i) the power, direct or indirect, to cause the
608+
direction or management of such entity, whether by contract or
609+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
610+
outstanding shares, or (iii) beneficial ownership of such entity.
611+
612+
"You" (or "Your") shall mean an individual or Legal Entity
613+
exercising permissions granted by this License.
614+
615+
"Source" form shall mean the preferred form for making modifications,
616+
including but not limited to software source code, documentation
617+
source, and configuration files.
618+
619+
"Object" form shall mean any form resulting from mechanical
620+
transformation or translation of a Source form, including but
621+
not limited to compiled object code, generated documentation,
622+
and conversions to other media types.
623+
624+
"Work" shall mean the work of authorship, whether in Source or
625+
Object form, made available under the License, as indicated by a
626+
copyright notice that is included in or attached to the work
627+
(an example is provided in the Appendix below).
628+
629+
"Derivative Works" shall mean any work, whether in Source or Object
630+
form, that is based on (or derived from) the Work and for which the
631+
editorial revisions, annotations, elaborations, or other modifications
632+
represent, as a whole, an original work of authorship. For the purposes
633+
of this License, Derivative Works shall not include works that remain
634+
separable from, or merely link (or bind by name) to the interfaces of,
635+
the Work and Derivative Works thereof.
636+
637+
"Contribution" shall mean any work of authorship, including
638+
the original version of the Work and any modifications or additions
639+
to that Work or Derivative Works thereof, that is intentionally
640+
submitted to Licensor for inclusion in the Work by the copyright owner
641+
or by an individual or Legal Entity authorized to submit on behalf of
642+
the copyright owner. For the purposes of this definition, "submitted"
643+
means any form of electronic, verbal, or written communication sent
644+
to the Licensor or its representatives, including but not limited to
645+
communication on electronic mailing lists, source code control systems,
646+
and issue tracking systems that are managed by, or on behalf of, the
647+
Licensor for the purpose of discussing and improving the Work, but
648+
excluding communication that is conspicuously marked or otherwise
649+
designated in writing by the copyright owner as "Not a Contribution."
650+
651+
"Contributor" shall mean Licensor and any individual or Legal Entity
652+
on behalf of whom a Contribution has been received by Licensor and
653+
subsequently incorporated within the Work.
654+
655+
2. Grant of Copyright License. Subject to the terms and conditions of
656+
this License, each Contributor hereby grants to You a perpetual,
657+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
658+
copyright license to reproduce, prepare Derivative Works of,
659+
publicly display, publicly perform, sublicense, and distribute the
660+
Work and such Derivative Works in Source or Object form.
661+
662+
3. Grant of Patent License. Subject to the terms and conditions of
663+
this License, each Contributor hereby grants to You a perpetual,
664+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
665+
(except as stated in this section) patent license to make, have made,
666+
use, offer to sell, sell, import, and otherwise transfer the Work,
667+
where such license applies only to those patent claims licensable
668+
by such Contributor that are necessarily infringed by their
669+
Contribution(s) alone or by combination of their Contribution(s)
670+
with the Work to which such Contribution(s) was submitted. If You
671+
institute patent litigation against any entity (including a
672+
cross-claim or counterclaim in a lawsuit) alleging that the Work
673+
or a Contribution incorporated within the Work constitutes direct
674+
or contributory patent infringement, then any patent licenses
675+
granted to You under this License for that Work shall terminate
676+
as of the date such litigation is filed.
677+
678+
4. Redistribution. You may reproduce and distribute copies of the
679+
Work or Derivative Works thereof in any medium, with or without
680+
modifications, and in Source or Object form, provided that You
681+
meet the following conditions:
682+
683+
(a) You must give any other recipients of the Work or
684+
Derivative Works a copy of this License; and
685+
686+
(b) You must cause any modified files to carry prominent notices
687+
stating that You changed the files; and
688+
689+
(c) You must retain, in the Source form of any Derivative Works
690+
that You distribute, all copyright, patent, trademark, and
691+
attribution notices from the Source form of the Work,
692+
excluding those notices that do not pertain to any part of
693+
the Derivative Works; and
694+
695+
(d) If the Work includes a "NOTICE" text file as part of its
696+
distribution, then any Derivative Works that You distribute must
697+
include a readable copy of the attribution notices contained
698+
within such NOTICE file, excluding those notices that do not
699+
pertain to any part of the Derivative Works, in at least one
700+
of the following places: within a NOTICE text file distributed
701+
as part of the Derivative Works; within the Source form or
702+
documentation, if provided along with the Derivative Works; or,
703+
within a display generated by the Derivative Works, if and
704+
wherever such third-party notices normally appear. The contents
705+
of the NOTICE file are for informational purposes only and
706+
do not modify the License. You may add Your own attribution
707+
notices within Derivative Works that You distribute, alongside
708+
or as an addendum to the NOTICE text from the Work, provided
709+
that such additional attribution notices cannot be construed
710+
as modifying the License.
711+
712+
You may add Your own copyright statement to Your modifications and
713+
may provide additional or different license terms and conditions
714+
for use, reproduction, or distribution of Your modifications, or
715+
for any such Derivative Works as a whole, provided Your use,
716+
reproduction, and distribution of the Work otherwise complies with
717+
the conditions stated in this License.
718+
719+
5. Submission of Contributions. Unless You explicitly state otherwise,
720+
any Contribution intentionally submitted for inclusion in the Work
721+
by You to the Licensor shall be under the terms and conditions of
722+
this License, without any additional terms or conditions.
723+
Notwithstanding the above, nothing herein shall supersede or modify
724+
the terms of any separate license agreement you may have executed
725+
with Licensor regarding such Contributions.
726+
727+
6. Trademarks. This License does not grant permission to use the trade
728+
names, trademarks, service marks, or product names of the Licensor,
729+
except as required for reasonable and customary use in describing the
730+
origin of the Work and reproducing the content of the NOTICE file.
731+
732+
7. Disclaimer of Warranty. Unless required by applicable law or
733+
agreed to in writing, Licensor provides the Work (and each
734+
Contributor provides its Contributions) on an "AS IS" BASIS,
735+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
736+
implied, including, without limitation, any warranties or conditions
737+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
738+
PARTICULAR PURPOSE. You are solely responsible for determining the
739+
appropriateness of using or redistributing the Work and assume any
740+
risks associated with Your exercise of permissions under this License.
741+
742+
8. Limitation of Liability. In no event and under no legal theory,
743+
whether in tort (including negligence), contract, or otherwise,
744+
unless required by applicable law (such as deliberate and grossly
745+
negligent acts) or agreed to in writing, shall any Contributor be
746+
liable to You for damages, including any direct, indirect, special,
747+
incidental, or consequential damages of any character arising as a
748+
result of this License or out of the use or inability to use the
749+
Work (including but not limited to damages for loss of goodwill,
750+
work stoppage, computer failure or malfunction, or any and all
751+
other commercial damages or losses), even if such Contributor
752+
has been advised of the possibility of such damages.
753+
754+
9. Accepting Warranty or Additional Liability. While redistributing
755+
the Work or Derivative Works thereof, You may choose to offer,
756+
and charge a fee for, acceptance of support, warranty, indemnity,
757+
or other liability obligations and/or rights consistent with this
758+
License. However, in accepting such obligations, You may act only
759+
on Your own behalf and on Your sole responsibility, not on behalf
760+
of any other Contributor, and only if You agree to indemnify,
761+
defend, and hold each Contributor harmless for any liability
762+
incurred by, or claims asserted against, such Contributor by reason
763+
of your accepting any such warranty or additional liability.
764+
765+
END OF TERMS AND CONDITIONS
766+
767+
APPENDIX: How to apply the Apache License to your work.
768+
769+
To apply the Apache License to your work, attach the following
770+
boilerplate notice, with the fields enclosed by brackets "[]"
771+
replaced with your own identifying information. (Don't include
772+
the brackets!) The text should be enclosed in the appropriate
773+
comment syntax for the file format. We also recommend that a
774+
file or class name and description of purpose be included on the
775+
same "printed page" as the copyright notice for easier
776+
identification within third-party archives.
777+
778+
Copyright [yyyy] [name of copyright owner]
779+
780+
Licensed under the Apache License, Version 2.0 (the "License");
781+
you may not use this file except in compliance with the License.
782+
You may obtain a copy of the License at
783+
784+
http://www.apache.org/licenses/LICENSE-2.0
785+
786+
Unless required by applicable law or agreed to in writing, software
787+
distributed under the License is distributed on an "AS IS" BASIS,
788+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
789+
See the License for the specific language governing permissions and
790+
limitations under the License.
791+
792+
793+
---- LLVM Exceptions to the Apache 2.0 License ----
794+
795+
As an exception, if, as a result of your compiling your source code, portions
796+
of this Software are embedded into an Object form of such source code, you
797+
may redistribute such embedded portions in such Object form without complying
798+
with the conditions of Sections 4(a), 4(b) and 4(d) of the License.
799+
800+
In addition, if you combine or link compiled forms of this Software with
801+
software that is licensed under the GPLv2 ("Combined Software") and if a
802+
court of competent jurisdiction determines that the patent provision (Section
803+
3), the indemnity provision (Section 9) or other Section of the License
804+
conflicts with the conditions of the GPLv2, you may retroactively and
805+
prospectively choose to deem waived or otherwise exclude such Section(s) of
806+
the License, but only in their entirety and only with respect to the Combined
807+
Software.

‎cmake/options.cmake

+12
Original file line numberDiff line numberDiff line change
@@ -180,6 +180,18 @@ option(DNNL_EXPERIMENTAL_SPARSE
180180
independetly from DNNL_EXPERIMENTAL."
181181
OFF) # disabled by default
182182

183+
184+
option(ONEDNN_EXPERIMENTAL_GRAPH_COMPILER_BACKEND
185+
"builds oneDNN Graph API graph-compiler backend" OFF)
186+
set(ONEDNN_EXPERIMENTAL_GRAPH_COMPILER_CPU_LLVM_CONFIG "AUTO" CACHE STRING
187+
"graph-compiler's llvm-config path")
188+
set(ONEDNN_EXPERIMENTAL_GRAPH_COMPILER_CPU_JIT "llvm;builtin" CACHE STRING
189+
"the optional JIT backends for graph-compiler: llvm;c;builtin")
190+
191+
if(DNNL_EXPERIMENTAL)
192+
set(ONEDNN_EXPERIMENTAL_GRAPH_COMPILER_BACKEND ON FORCE)
193+
endif()
194+
183195
# ======================
184196
# Profiling capabilities
185197
# ======================

‎doc/advanced/experimental.md

+9-3
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,10 @@ Both kinds of experimental features can be enabled simultaneously.
2424
|:-----------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------|
2525
| ONEDNN_EXPERIMENTAL_BNORM_STATS_ONE_PASS | Calculate mean and variance in batch normalization(BN) in single pass ([RFC](https://github.com/oneapi-src/oneDNN/tree/rfcs/rfcs/20210519-single-pass-bnorm)). |
2626

27-
| Build time option | Description |
28-
|:---------------------------|:-------------------------------------------------------------|
29-
| ONEDNN_EXPERIMENTAL_SPARSE | Enable experimental API and functionality for sparse domain. |
27+
| Build time option | Description |
28+
|:-------------------------------------------|:-------------------------------------------------------------------|
29+
| ONEDNN_EXPERIMENTAL_SPARSE | Enable experimental API and functionality for sparse domain. |
30+
| ONEDNN_EXPERIMENTAL_GRAPH_COMPILER_BACKEND | Enable experimental graph compiler backend of the graph component. |
3031

3132
## Features details
3233

@@ -141,3 +142,8 @@ calculated as max(4 * 1000000 * (1 - 0.99)), 1).
141142
@warning
142143
- Enabling experimental features does not guarantee that the library will utilize them
143144
- Enabling experimental features might change the accuracy of oneDNN primitives
145+
146+
### ONEDNN_EXPERIMENTAL_GRAPH_COMPILER_BACKEND
147+
This option extends the coverage scope of the graph API to cover larger fusion
148+
patterns apart from primitive patterns. Refer to
149+
[Graph Compiler](@ref dev_guide_graph_compiler) for more details.

‎doc/build/build_options.md

+25-6
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ oneDNN supports the following build-time options.
1212
| ONEDNN_BUILD_TESTS | **ON**, OFF | Controls building the tests |
1313
| ONEDNN_BUILD_GRAPH | ON, **OFF** | Controls building graph component (experimental) |
1414
| ONEDNN_ENABLE_GRAPH_DUMP | ON, **OFF** | Controls dumping graph artifacts |
15+
| ONEDNN_EXPERIMENTAL_GRAPH_COMPILER_BACKEND | ON, **OFF** | Enables the [graph compiler backend](@ref dev_guide_graph_compiler) of the graph component (experimental)|
1516
| ONEDNN_ARCH_OPT_FLAGS | *compiler flags* | Specifies compiler optimization flags (see warning note below) |
1617
| ONEDNN_ENABLE_CONCURRENT_EXEC | ON, **OFF** | Disables sharing a common scratchpad between primitives in #dnnl::scratchpad_mode::library mode |
1718
| ONEDNN_ENABLE_JIT_PROFILING | **ON**, OFF | Enables [integration with performance profilers](@ref dev_guide_profilers) |
@@ -278,16 +279,34 @@ explicitly specify the path to the SDK using `-DOPENCLROOT` CMake option.
278279
$ cmake -DONEDNN_GPU_RUNTIME=OCL -DOPENCLROOT=/path/to/opencl/sdk ..
279280
~~~
280281

282+
@anchor component_limitation
281283
## Graph component limitations
282284

283285
The graph component can be enabled via the build option `ONEDNN_BUILD_GRAPH`.
284286
But the build option does not work with some values of other build options.
285287
Specifying the options and values simutanously in one build will lead to a CMake
286288
error.
287289

288-
| CMake Option | Value |
289-
|:------------------------|:---------------|
290-
| ONEDNN_GPU_RUNTIME | OCL |
291-
| ONEDNN_GPU_VENDOR | NVIDIA |
292-
| ONEDNN_ENABLE_PRIMITIVE | PRIMITIVE_NAME |
293-
| ONEDNN_ENABLE_WORKLOAD | INFERENCE |
290+
| CMake Option | Unsupported Values |
291+
|:------------------------|:-------------------|
292+
| ONEDNN_GPU_RUNTIME | OCL |
293+
| ONEDNN_GPU_VENDOR | NVIDIA |
294+
| ONEDNN_ENABLE_PRIMITIVE | PRIMITIVE_NAME |
295+
| ONEDNN_ENABLE_WORKLOAD | INFERENCE |
296+
297+
## Graph Compiler Backend Limitations
298+
299+
As a backend of the graph component, besides the options described in
300+
[Graph component limitations](@ref component_limitation), graph compiler
301+
backend has some extra limitations. Specifying unsupported build options will
302+
lead to a CMake error.
303+
304+
| CMake Option | Unsupported Values |
305+
| :-----------------------| :------------------|
306+
| ONEDNN_CPU_RUNTIME | THREADPOOL, SYCL |
307+
| ONEDNN_GPU_RUNTIME | OCL, SYCL |
308+
309+
Besides, the instructions contained in the kernels generated by the graph
310+
compiler backend are [AVX512_CORE](@ref dev_guide_cpu_dispatcher_control) or
311+
above, so these kernels will not be dispatched on systems that do not have
312+
corresponding instruction sets support.

0 commit comments

Comments
 (0)