Changing layout of weights as desired for NPUW.

[ujjayant-kadian]

Details:

• item1
• ...

Tickets:

• ticket-id

[TolyaTalamanov]

Do we have already existing function for permute/tranpose?

Naive implementation can be done even simpler by just copying data to the tensor with different strides.
e.g let's assume original layout is NCHW [1, 3, 10, 15] but you need NHWC

1. Create tensor with NHWC layout: ov::Tensor tmp([1, 10, 15, 3], ...);
2. Copy data from existing tensor: orig.copy_to(tmp); <- this basically transpose data as tmp has different strides
3. orig.set_shape([1, 10, 15, 3])
4. tmp.copy_to(orig)

[ujjayant-kadian]

Tried this implementation particularly:

    const auto& to_shape = to->get_shape();
    size_t dim1 = to_shape[0];
    size_t dim2 = to_shape[1];
    size_t dim3 = to_shape[2];
    ov::Shape new_shape = {dim2, dim3, dim1};
    ov::Tensor tmp(to->get_element_type(), new_shape);
    ov::SoPtr<ov::ITensor> tmp_itensor = ov::get_tensor_impl(tmp);
    to->set_shape(new_shape);
    to->copy_to(tmp_itensor._ptr);
    tmp_itensor->copy_to(to._ptr);
    std::cout << "To new shape: " << to->get_shape() << std::endl; 

But when I verified, by testing if the actual data is transposed or not. It turns out not. So moved back to earlier implementation.

[ujjayant-kadian]

After these changes, I get the following error:

[ NPUW: ERR ] Subgraph [1] - FAILED to run infer request:
Check 'is_dynamic || port.get_shape() == tensor->get_shape()' failed at src/inference/src/dev/isync_infer_request.cpp:277:
The input tensor size is not equal to the model input type: got [32,128,12288] expecting [12288,32,128].