Update requirements.txt

[dpurandare]

Can you update the requirements.txt file from your setup? Looks like you updated the pytorch version however the requirements.txt shows older dependencies. Couple of threads shows reference to the requirements.txt however it would be great to get updated requirements.txt to avoid unnecessary issues while running the code as is.

[rasbt]

Thanks for the comment! I just looked at the requirements.txt but no version number is hardcoded with a upper version limit (except numpy) there unless I am overlooking something.

E.g., currently, the requirements for PyTorch are set to torch >= 2.0.1, which means that this is the minimum required version to run PyTorch. However, if you don't have PyTorch installed, it will install the most recent version. The CI in this repo checks the code for compatibility with both 2.0.1 and 2.5.

Did you run into any issues with the requirements/code as is?

[dpurandare]

Thank you. I have not faced any challenges yet. Just started with Chapter 2 and hope things will work.

[piyushkr082]

Hello - I am still getting below error and not ableto launch jupyter lab beacuse of it. I tried running it without uv as well. Could you please suggest alternate solution to run code.
(LLMs-from-scratch-main) C:\Users\pc\Desktop\LLMs-from-scratch-main>uv run jupyter lab
error: Distribution tensorflow-io-gcs-filesystem==0.37.1 @ registry+https://pypi.org/simple can't be installed because it doesn't have a source distribution or wheel for the current platform

hint: You're on Windows (win_amd64), but tensorflow-io-gcs-filesystem (v0.37.1) only has wheels for the following platforms: manylinux_2_17_aarch64, manylinux_2_17_x86_64, manylinux2014_aarch64, manylinux2014_x86_64, macosx_10_14_x86_64, macosx_12_0_arm64

(LLMs-from-scratch-main) C:\Users\pc\Desktop\LLMs-from-scratch-main>pip install -r requirements.txt
Audited 9 packages in 53ms

[rasbt]

Interesting! Thanks for this! So the flow to recommend would be as follows?

rmdir /s /q .venv
uv venv --python=python3.11
.venv\Scripts\activate
setx UV_PIP_OPTS "--no-binary tensorflow-io-gcs-filesystem"
uv pip install -r requirements.txt
uv tool run jupyter lab

[d-kleine]

I just have tested the standalone version with uv tool run jupyter lab - it runs, but doesn't recognize the venv because it's isolated then.

With jupyter lab, it works for me on Windows, this is practically .venv\Scripts\jupyter lab. Same command on Linux would be .venv/bin/jupyter lab, see https://docs.astral.sh/uv/guides/integration/jupyter/#using-jupyter-with-a-non-project-environment

@rasbt issue solved?

[rasbt]

Totally forgot to follow up here. I need to get a cheap Windows laptop for testing one of these days...

[d-kleine]

The thing about Windows (and other OS like Linux) is that you can have different instances of tools located differently. For example, when running jupyter lab , this would launch my JLab from my Anaconda installation which is prioritized in my system environment variables settings.

So, in the venv I also had the same issue with uv run jupyter lab that it does not work in Windows (maybe got fixed meanwhile?). jupyter lab only runs as intended as when you have Jlab only installed there. Starting directly from the venv (Windows: .venv\Scripts\jupyter lab; Linux: .venv/bin/jupyter lab) is the savest way imo.

[rasbt]

Ah yes, that makes sense. When you run

jupyter lab only runs as intended

though, then you have to be careful that it actually uses the packages from .venv.

.venv\Scripts\jupyter lab; Linux: .venv/bin/jupyter lab

Fair! Maybe a good thing to add to the docs.

[anneleonore]

Hi there,

I'm commenting on this thread, because there might be other users with a similar issue. I am following your set-up instructions https://github.com/rasbt/LLMs-from-scratch/tree/main/setup/01_optional-python-setup-preferences. However I am running into a similar error as @piyushkr082 when I try to install the requirements on a MacBook Pro.

My system specifications are:
ProductName: macOS
ProductVersion: 14.6.1
BuildVersion:23G93

I am using:
Python version: 3.10.16
uv version: 0.6.11

The error I am seeing (when running uv pip install -r requirements.txt)

No solution found when resolving dependencies:
╰─▶ Because only the following versions of torch are available:
torch<=2.3.0
torch==2.3.1
torch==2.4.0
torch==2.4.1
torch==2.5.0
torch==2.5.1
torch==2.6.0
and torch>=2.3.0 has no wheels with a matching platform tag (e.g., macosx_14_0_x86_64), we can conclude that torch>=2.3.0 cannot be used.
And because you require torch>=2.3.0, we can conclude that your requirements are unsatisfiable.
hint: Wheels are available for torch (v2.6.0) on the following platforms: manylinux_2_28_aarch64, manylinux1_x86_64, macosx_11_0_arm64, win_amd64

Unfortunately, I also see an error when I run pip install -r requirements.txt:
ERROR: Could not find a version that satisfies the requirement torch>=2.3.0
ERROR: No matching distribution found for torch>=2.3.0

I then ran the similar command but for macOS export UV_PIP_OPTS="--no-binary tensorflow-io-gcs-filesystem", but this unfortunately did not solve the errors.

Maybe I just need to wait until there is a wheel available for the macOS 14_0_x86_64 system's architecture, then this is just a comment that people with similar computer system architectures might run into difficulties when running your current set-up instructions.

[rasbt]

Hi there. That's an interesting one, and sorry about the hassle.

Actually, if I see it correctly, you are using an Intel MacBook? Maybe that's the culprit, and the PyTorch team stopped supporting Intel MacBooks after PyTorch 2.3. The book should work well on PyTorch 2.0, 2.1, and 2.2. I think I just updated it to 2.3 in the requirements because of some other dependency.

You could update the current requirements.txt:

torch >= 2.3.0             # all
jupyterlab >= 4.0          # all
tiktoken >= 0.5.1          # ch02; ch04; ch05
matplotlib >= 3.7.1        # ch04; ch06; ch07
tensorflow >= 2.18.0       # ch05; ch06; ch07
tqdm >= 4.66.1             # ch05; ch07
numpy >= 1.26, < 2.1       # dependency of several other libraries like torch and pandas
pandas >= 2.2.1            # ch06
psutil >= 5.9.5            # ch07; already installed automatically as dependency of torc

to

torch >= 2.0.1        # all
jupyterlab >= 4.0     # all
tiktoken >= 0.5.1     # ch02, ch04, ch05
matplotlib >= 3.7.1   # ch04, ch05
numpy >= 1.24.3       # ch05
tensorflow >= 2.15.0  # ch05
tqdm >= 4.66.1        # ch05, ch07
pandas >= 2.2.1       # ch06

and give that a try. Please let me know if it works!