Skip to content

Commit 8c9d4be

Browse files
authoredSep 5, 2024
[CPU] Add interface to release compiled model internal memory (#26390)
### Details: This PR introduces an `ov::CompiledModel` level interface that allows to release memory allocated by the compiled model. In this PR the interface is only supported by the CPU plugin. ### Tickets: - CVS-145873
1 parent ad3f51b commit 8c9d4be

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

75 files changed

+1429
-825
lines changed
 

‎src/inference/dev_api/openvino/runtime/icompiled_model.hpp

+6
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,12 @@ class OPENVINO_RUNTIME_API ICompiledModel : public std::enable_shared_from_this<
134134
*/
135135
ov::SoPtr<ov::IRemoteContext> get_context() const;
136136

137+
/**
138+
* @brief Release intermediate memory
139+
*
140+
*/
141+
virtual void release_memory();
142+
137143
virtual ~ICompiledModel() = default;
138144

139145
private:

‎src/inference/include/openvino/runtime/compiled_model.hpp

+9
Original file line numberDiff line numberDiff line change
@@ -200,6 +200,15 @@ class OPENVINO_RUNTIME_API CompiledModel {
200200
return get_property(property.name()).template as<T>();
201201
}
202202

203+
/**
204+
* @brief Release intermediate memory.
205+
*
206+
* This method forces the Compiled model to release memory allocated for intermediate structures, e.g. caches,
207+
* tensors, temporal buffers etc., when possible
208+
*
209+
*/
210+
void release_memory();
211+
203212
/**
204213
* @brief Returns pointer to device-specific shared context
205214
* on a remote accelerator device that was used to create this CompiledModel.

0 commit comments

Comments
 (0)