Executorch initial support #28425

cavusmustafa · 2025-01-14T01:38:43Z

Details:

OV side changes for initial ExecuTorch OV backend

Tickets:

ticket-id

rkazants

I have several question about this feature:

Will is support ExecuTorch binary compiled to OV ExecuTorch backend?
What other ExecuTorch binaries will be supported: default, xnnpack?
We won't support ExecuTorch binary inference using native OpenVINO workflow, right?
Do you have PR/branch with ExecuTorch OV backend implementation?

Best regards,
Roman

ynimmaga · 2025-01-14T05:44:55Z

src/bindings/python/src/openvino/frontend/pytorch/torchdynamo/backend_utils.py

@@ -83,3 +83,10 @@ def _is_testing(options) -> Optional[Any]:
            return True
    return False

+def _executorch(options) -> Optional[Any]:


If executorch is added in options, this will also be available to torch.compile users and may break the flow if the option is set to True as ExecuTorchPythonDecoder will be used. Instead, can we add it as a kwarg to openvino_compile and set it to False by default? This can be set to True in Executorch openvino backend. That way it is not visible to the user. Please let me know your thoughts.

I see, I was trying not to increase the number of parameters for openvino_compile at first but it makes sense not to expose this feature to users. I think it is a better idea to pass this as kwarg. I will update accordingly.

cavusmustafa · 2025-01-14T08:14:17Z

I have several question about this feature:

Will is support ExecuTorch binary compiled to ExecuTorch backend?

What other ExecuTorch binaries will be supported: default, xnnpack?

We won't support ExecuTorch binary inference using native OpenVINO workflow, right?

Do you have PR/branch with ExecuTorch OV backend implementation?

Best regards, Roman

Yes, the changes included in this PR to support ExecuTorch OpenVINO backend.
POC will support OpenVINO backend only but we are also working on enabling fallback to default backend for unsupported ops.
ExecuTorch binary inference will be using OpenVINO workflow similar to inference execution with torch.compile. The partition compiled for OpenVINO execution (which is stored in .pte file) is executed in ExecuTorch OpenVINO backend which uses OpenVINO C++ API.
Branch: https://github.com/ynimmaga/executorch/tree/openvino_backend

Please let me know for additional questions and clarifications needed.

rkazants · 2025-01-16T05:15:59Z

src/bindings/python/src/openvino/frontend/pytorch/fx_decoder.py

+                    self._inputs.append(i)
+                    self._input_signature.append(value.name)
+                    if hasattr(value, "meta") and ('tensor_meta' in value.meta.keys()) and value.meta['tensor_meta']:
+                        found_shapes.append(value.meta['tensor_meta'].shape)


can you please comment what type of this input that requires special branch?
for else, we need a comment as well

rkazants · 2025-01-16T05:17:17Z

src/bindings/python/src/openvino/frontend/pytorch/fx_decoder.py

+        self._input_signature = []
+        self._example_input = None
+
+        if issubclass(type(pt_module), torch.fx.graph_module.GraphModule):


Suggested change

if issubclass(type(pt_module), torch.fx.graph_module.GraphModule):

if isinstance(pt_module, torch.fx.graph_module.GraphModule):

can we do it?

please create separate internal function in this class to handle torch.fx.graph_module.GraphModule case
and separate function to handle torch.fx.Node because now it is quite long for constructor

rkazants · 2025-01-16T05:21:09Z

src/bindings/python/src/openvino/frontend/pytorch/fx_decoder.py

+            for idx, shape in enumerate(found_shapes):
+                if shape is not None:
+                    new_shape = []
+                    for dim in shape:
+                        if (dynamic_shapes or type(dim).__name__ == "SymInt"):
+                            new_shape.append(-1)
+                        else:
+                            new_shape.append(dim)
+                    found_shapes[idx] = torch.Size(new_shape)


I believe we should have helper function for this. Can you please check or create?
Why can't you reformat found_shapes[idx] right away when you were adding elements above? Now you have to re-iterate through it again

rkazants · 2025-01-16T05:25:05Z

src/frontends/pytorch/src/op_table.cpp

@@ -789,6 +789,7 @@ const std::unordered_map<std::string, CreatorFunction> get_supported_ops_fx() {
        {"aten.argmin.default", op::translate_argmin},
        {"aten.as_strided.default", op::translate_as_strided},
        {"aten.as_strided_.default", op::translate_as_strided},
+        {"aten.as_strided_copy.default", op::translate_as_strided},


can you please add layer tests for newly added translators?

rkazants · 2025-01-16T05:26:32Z

src/bindings/python/src/openvino/frontend/pytorch/torchdynamo/compile.py

-from openvino.frontend.pytorch.fx_decoder import TorchFXPythonDecoder
-from openvino import Core, Type, PartialShape, serialize
+from openvino.frontend.pytorch.fx_decoder import TorchFXPythonDecoder, ExecuTorchPythonDecoder
+from openvino.runtime import Core, Type, PartialShape, serialize


Suggested change

from openvino.runtime import Core, Type, PartialShape, serialize

from openvino import Core, Type, PartialShape, serialize

openvino.runtime gets deprecated this release

rkazants · 2025-01-16T05:28:11Z

src/bindings/python/src/openvino/frontend/pytorch/torchdynamo/compile.py

@@ -78,7 +78,7 @@ def openvino_compile_cached_model(cached_model_path, options, *example_inputs):

    return compiled_model

-def openvino_compile(gm: GraphModule, *args, model_hash_str: str = None, options=None):
+def openvino_compile(gm: GraphModule, *args, model_hash_str: str = None, options=None, executorch=False):
    core = Core()

    device = _get_device(options)


for ExecuTorch, will you cache ov::Model or ov::CompiledModel?

rkazants · 2025-01-16T05:29:06Z

src/bindings/python/src/openvino/frontend/pytorch/torchdynamo/compile.py

@@ -13,8 +13,8 @@
 from torch.fx import GraphModule


Don't we need to have separate file for executorch implementation? Just to avoid mix with torch.compile

ynimmaga added 3 commits November 12, 2024 22:48

Support for executorch OV backend

af9d22d

Merge branch 'openvinotoolkit:master' into executorch_ov_backend

41109cb

Added missing executorch copy ops

f393338

github-actions bot added category: Python API OpenVINO Python bindings category: PyTorch FE OpenVINO PyTorch Frontend labels Jan 14, 2025

cavusmustafa requested review from ynimmaga and suryasidd January 14, 2025 02:43

cavusmustafa added 2 commits January 13, 2025 18:49

ExecuTorchPythonDecoder added

01e7517

Merge branch 'master' into executorch_initial_support

d94df45

cavusmustafa marked this pull request as ready for review January 14, 2025 02:54

cavusmustafa requested review from a team as code owners January 14, 2025 02:54

cavusmustafa requested a review from PiotrKrzem January 14, 2025 02:54

cavusmustafa added the Code Freeze label Jan 14, 2025

rkazants reviewed Jan 14, 2025

View reviewed changes

ynimmaga reviewed Jan 14, 2025

View reviewed changes

cavusmustafa added 3 commits January 14, 2025 01:10

Fix op conversion issues due to merging master

b0e0e29

Use argument to enable executorch instead of setting option

c369635

Merge branch 'master' into executorch_initial_support

fa160e6

ynimmaga approved these changes Jan 16, 2025

View reviewed changes

cavusmustafa added 2 commits January 15, 2025 19:05

Temporary solution for ExecuTorch and GPTQ conflict

291379d

Merge branch 'master' into executorch_initial_support

d2de31f

rkazants reviewed Jan 16, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Executorch initial support #28425

Executorch initial support #28425

cavusmustafa commented Jan 14, 2025 •

edited

Loading

rkazants left a comment •

edited

Loading

ynimmaga Jan 14, 2025

cavusmustafa Jan 14, 2025

cavusmustafa commented Jan 14, 2025

rkazants Jan 16, 2025

rkazants Jan 16, 2025

rkazants Jan 16, 2025

rkazants Jan 16, 2025

rkazants Jan 16, 2025

rkazants Jan 16, 2025

rkazants Jan 16, 2025

rkazants Jan 16, 2025

	if issubclass(type(pt_module), torch.fx.graph_module.GraphModule):
	if isinstance(pt_module, torch.fx.graph_module.GraphModule):

	from openvino.runtime import Core, Type, PartialShape, serialize
	from openvino import Core, Type, PartialShape, serialize

Executorch initial support #28425

Are you sure you want to change the base?

Executorch initial support #28425

Conversation

cavusmustafa commented Jan 14, 2025 • edited Loading

Details:

Tickets:

rkazants left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cavusmustafa commented Jan 14, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cavusmustafa commented Jan 14, 2025 •

edited

Loading

rkazants left a comment •

edited

Loading