Initial support for BitTorrent distribution of ISOs

[mskiptr]

Hello everyone! Before even becoming a user, I present you with this patchset. The aim here is to make BitTorrent an alternative supported way of obtaining Bazzite ISO images for installation.

The PR is marked as a draft because not everything here is ready to be merged just yet. While the first patch should work (and result in .torrent files being published for each built ISO), I wasn't able to test it on my own. When I try to run the GH action from a cloned repo, the build fails at the "Determine Flatpak Dependencies" step (docker: Error response from daemon: Head "https://ghcr.io/v2/mskiptr/bazzite/manifests/stable": denied.).

The second patch is a work-in-progress. I would like to mark each generated .torrent with the information which build number | version | revision it corresponds to. But how do I obtain that? I couldn't find it anywhere among all variables in that YAML file.

Finally, the third patch adds the official Fedora tracker. It isn't necessary, since there's always the DHT and peer exchange, so the torrent should work without a tracker but adding one might still be beneficial. The patch is marked as a draft because using that tracker will have to be coordinated with the Fedora infrastructure admins most likely. (And btw, if this really needs a tracker, we can always look for some general-purpose one.)

Now, how will the swarm start up? At first I thought that for each variant and version someone will have to obtain the ISO manually and inject it into a torrent client. That would certainly be not ideal to say the least, but at least it should be simpler than getting someone else to send you parts of the file when the transfer keeps failing for you.

But it turns out we have a much better option. There's this thing called Web Seeds. It basically means that a .torrent file can contain a list of URIs that should be tried automatically, side-by-side with any actual BitTorrent peers.

The apparent lack of support for HTTP range requests from the current CDN might throw a wrench in that. But even then having an official .torrent available would still be an improvement here. Another problem might be that currently the web download link is not unique between consecutive Bazzite releases. So if someone tries to download the ISO using a .torrent file from an older release, the software might get confused.

[AzemaViator]

For reference, you can't just use the Fedora tracker blindly. The Fedora Infrastructure Team must submit every torrent.
https://docs.stg.fedoraproject.org/en-US/infra/release_guide/torrentrelease/

For the web seed this is blocking range support on the downloads. https://community.cloudflare.com/t/public-r2-bucket-doesnt-handle-range-requests-well/434221

[mskiptr]

For reference, you can't just use the Fedora tracker blindly. The Fedora Infrastructure Team must submit every torrent. https://docs.stg.fedoraproject.org/en-US/infra/release_guide/torrentrelease/

Indeed. It would need to be coordinated with them somehow. But also automated and overall handled very differently than their regular procedure, which means it would require even more assistance from them. So that patch definitely needs to wait. (It's a draft and it shows.)

But if that turns out impossible (e.g. because of licensing issues), we can either forgo a tracker entirely or use some general-purpose one instead. I guess we will want to avoid ones even remotely associated with piracy, so something "Linux-related" would be the best. Maybe https://linuxtracker.org/? The documentation is pretty sparse, so from a quick glance I can't really tell if it's suitable. It seems to rely on using the web interface and accounts to submit new torrents (which could probably be automated with a simple curl incantation from GH actions). But can we upload just the .torrents and not entire ISOs? And how will it handle name collisions every single week? These need further research I guess.

For the web seed this is blocking range support on the downloads. https://community.cloudflare.com/t/public-r2-bucket-doesnt-handle-range-requests-well/434221

Yeah. I've now found a few reports like that. Seems to be a Cloudflare bug (or at least an edge-case in the HTTP specification which doesn't mesh well with many clients). Nothing conclusive tho

I was hoping to test how well Transmission handles it. That's why I would love to see that first patch tested in action. It would let me try downloading an ISO with just the web seed so I could check how reliable it is for me. But it has just occurred to me I can simply create this .torrent file on my end. (That's assuming I can make it before the next ISOs are generated. What an awful timing lol)

[AzemaViator]

From my understanding, the web seed will not function at all because it depends on the Range header being respected and working correctly. I have not tried making a torrent to test webseeding yet, but I have confirmed the range header does not result in expected returns from the download cdn.

There does seem to be fosstorrents, which has an open tracker; they also have a discord for having official partnerships. One of which is ultramarine

On the topic of whether or not to have a tracker, DHT is not something everyone can use. Depending on NATs, some clients require a tracker to function at all.

[castrojo]

There does seem to be fosstorrents

I joined their discord in december with the intent on investigating - seems like a good fit for us, might be worth investigating and then dropping over some cash to help for infra?

[AzemaViator]

skopeo inspect docker://${{ steps.registry_case.outputs.lowercase }}/${{ matrix.image_name }}:${{ steps.generate-tag.outputs.tag }} | jq -r '.Labels["org.opencontainers.image.version"]'

This can be used to get the release version. ie

$ skopeo inspect docker://ghcr.io/ublue-os/bazzite:stable | jq -r '.Labels["org.opencontainers.image.version"]'
41.20250106.3

I am not sure if skopeo or jq are available in this job, but you should be able to use it to get something in the comments like Installer for bazzite-deck-nivida version 41.20250106.3

[mskiptr]

From my understanding, the web seed will not function at all because it depends on the Range header being respected and working correctly. I have not tried making a torrent to test webseeding yet, but I have confirmed the range header does not result in expected returns from the download cdn.

I've now tested this with Transmission 4.0.6. I couldn't get it to work with https://download.bazzite.gg/bazzite-deck-stable.iso as the web seed, but whether R2 reliably supports range requests seems to vary between people, so it might work for someone else. Then I tried the same, but with the image served by Python's http.server. It does not support range requests either, so it didn't work. Finally, I used (RangeHTTPServer)[https://github.com/danvk/RangeHTTPServer] – a drop-in replacement, specifically created to support this feature and Transmission picked it up right away. I didn't even need to restart anything.

There does seem to be fosstorrents, which has an open tracker; they also have a discord for having official partnerships. One of which is ultramarine

On the topic of whether or not to have a tracker, DHT is not something everyone can use. Depending on NATs, some clients require a tracker to function at all.

Yeah. I meant that in case we couldn't find a suitable tracker, getting this done would still be useful to some people.

But while testing these things out I found one more reason not to use trackerless torrents: file and xdg-open do not understand them! (I should probably report that…)

[castrojo]

I've reached out to fosstorrents and applied for support for our images.

[AzemaViator]

I've now tested this with Transmission 4.0.6. I couldn't get it to work with https://download.bazzite.gg/bazzite-deck-stable.iso as the web seed, but whether R2 reliably supports range requests seems to vary between people, so it might work for someone else. Then I tried the same, but with the image served by Python's http.server. It does not support range requests either, so it didn't work. Finally, I used (RangeHTTPServer)[https://github.com/danvk/RangeHTTPServer] – a drop-in replacement, specifically created to support this feature and Transmission picked it up right away. I didn't even need to restart anything.

From what I linked before in the Cloudflare community support thread, R2, where the ISOs are stored, does support range requests. However, when using a custom domain, it applies Cloudflare's cache to the files, which does not support range headers on files over 500 MB. Setting a page rule for the (sub)domain to bypass Cloudflare's cache fixes the issue, as all requests directly hit R2, which will handle the Range header right.

[mskiptr]

Here are two examples for anyone who wants to test them out.

• https://0x0.st/8oqG.torrent/bazzite-deck-stable.iso.torrent
• https://0x0.st/8oqk.torrent/bazzite-deck-stable.iso_trackerless.torrent

They were created with these two commands:

mktorrent --announce={udp,http}"://fosstorrents.com:6969/announce" --piece-length=20 --web-seed=http{,s}"://download.bazzite.gg/bazzite-deck-stable.iso" .../bazzite-deck-stable.iso
mktorrent --piece-length=20 --web-seed=http{,s}"://download.bazzite.gg/bazzite-deck-stable.iso" .../bazzite-deck-stable.iso

What to check for:

1. Does the current CDN works as a web seed for you?
2. Does the trackerless version connect to any other peers? (I'm probably the only one for now. And don't worry if it's quite slow – I have the upload speed limited to 80 KiB/s as to not overwhelm and clog up my uplink.)
3. Can you connect to the tracker? (Doesn't seem to work for me, but I haven't looked into it yet.)

[AzemaViator]

For the trackers alone I can connect to the UDP but not the http, I am not sure why that is the case.

The CDN setup as it is currently for webseeding just ends up in it trying to connect making a connection and then failing then retrying over and over.

I do not have a working DHT setup on my network so I can not test if that functions.