LLM inference on Lunar Lake 258v causes system reboot

[endomorphosis]

it did not appear to have anything to do with running out of system ram, I the only difference between the vanilla implementation from the examples list, and the code that I wrote in python that causes the system crash on the ov_model.generate() function, is that I have also have cuda dependencies, because I am writing some code that auto-loads models regardless of hardware platform and model architecture, and multiplexes the inference endpoints from api providers.

https://github.com/endomorphosis/ipfs_accelerate_py/blob/a1cb9ca8e0d8623bf8ddc66daed350ff2cf27dfd/ipfs_accelerate_py/worker/skillset/hf_llava.py#L325

[endomorphosis]

The computer has rebooted from a bugcheck. The bugcheck was: 0x00000124 (0x0000000000000000, 0xffff830fc8602028, 0x00000000b2000000, 0x0000000008210402). A dump was saved in: C:\windows\MEMORY.DMP. Report Id: 3599d5d5-5f01-4290-8f01-34ac5c70de4f.

https://huggingface.co/datasets/endomorphosis/LunarLake_Crash_MemDump/resolve/main/MEMORY.DMP

[Wan-Intel]

Could you please share the following information with us to further investigate the issue?

• Python version
• OpenVINO™ GenAI version
• Hardware specifications
• Host Operating System
• List any steps we should take to reproduce the error you are seeing
• Additional environment information
• If Other Deep Learning Framework, please specify
• If applicable, Deep Learning Framework used

[endomorphosis]

Can you please download this file so that I can remove it from huggingface, given that the memory dump has (most likely) intel github credentials in it.

[endomorphosis]

Could you please share the following information with us to further investigate the issue?

• Python version
  3.10.11
• OpenVINO™ GenAI version
  2024.6
• Hardware specifications
  Lunar Lake PC, on the Tiber Devcloud
• Host Operating System
  Windows 11
• List any steps we should take to reproduce the error you are seeing

I have provided the code where the breakpoint should be placed.

https://github.com/endomorphosis/ipfs_accelerate_py/blob/a1cb9ca8e0d8623bf8ddc66daed350ff2cf27dfd/ipfs_accelerate_py/worker/skillset/hf_llava.py#L325

Here is the entrypoint to that code.

https://github.com/endomorphosis/ipfs_accelerate_py/blob/a1cb9ca8e0d8623bf8ddc66daed350ff2cf27dfd/ipfs_accelerate_py/ipfs_accelerate.py#L1627

The code occurs intermittently once every 3 or 4 runs.

• Additional environment information
  Memory dump information was provided.

#1435 (comment)

• If Other Deep Learning Framework, please specify
  https://github.com/endomorphosis/ipfs_accelerate_py/blob/a1cb9ca8e0d8623bf8ddc66daed350ff2cf27dfd/requirements.txt

[endomorphosis]

I no longer have access to this lunar lake system, but I was provided this feedback by another dev.

The error message from windows indicates its a drivers issue, and I will leave this for you all to figure out.

[Wan-Intel]

I've copied your repository and setup the environment with the following steps on a local Windows system:

git clone https://github.com/endomorphosis/ipfs_accelerate_py.git
cd ipfs_accelerate_py
python -m venv openvino_env
openvino_env\Scripts\activate
python -m pip install --upgrade pip
pip install -r requirements.txt

I ran the Python script, and it didn't reboot the system. However, I encountered an error: AttributeError: 'IndexError' object has no attribute 'keys'

cd ipfs_accelerate_py
python ipfs_accelerate.py

Do you encounter the issue when using a local Windows system?

[endomorphosis]

oh, sorry, its a repository that I am actively committing to, its 330AM right now, but i will get that corrected when i wake up, but apparently the issue isnt so much with my code, but something to do with the driver instability.

[Wan-Intel]

I've cloned the latest repository and the LLM inference worked on a 12th gen Windows system.

I'll escalate the case to relevant team to further investigate the issue on Lunar Lake system.

[endomorphosis]

I've cloned the latest repository and the LLM inference worked on a 12th gen Windows system.

I'll escalate the case to relevant team to further investigate the issue on Lunar Lake system.

btw, the model that you did inference on was T5, whereas the problem that I had was loading llava 7b, a considerably larger model.

I am currently integration testing a wide variety of models, because this is meant to be a general purpose model server, for a rather large peer to peer based mlops infrastructure project built from scratch.