tasks: Bind the host's podman API socket

[martinpitt]

tasks: Bind the host's podman API socket

This paves the way for spawning per-job tasks containers from the
container (via job-runner).

Getting the permissions right is unfortunately annoyingly complicated, as the
host's socket has 660 permissions, but the tasks container runs as uid 1111.
Ideally we could use something like

-v "${XDG_RUNTIME_DIR:-/run}/podman/podman.sock:/podman.sock:idmap=gids=$(id -g)-1111-1"

but that fails with "invalid mappings", and is generally not well documented.
--mount=type=bind,[...],idmap --uidmap [...] does not work either.

So resort to file permissions for the host's podman.sock:

• For local developers in run-local.sh, make the socket world
  writable. This does not actually hurt for a human developer: Its
  directory (/run/user/uid) is not accessible for any other user.

• For production, set the socket group to 1111. That doesn't matter
  much, the secrets are all already chmod'ed to the container user, and
  these machines don't do anything else. setfacl would be more
  targeted, but it isn't installed in Fedora IoT/CoreOS.

---

I rolled this out to one production machine:

ansible -i inventory -m include_role -a name=tasks-systemd -e instances=1 -l rhos-01-1 openstack_tasks

Et voilà:

❱❱❱ ssh rhos-01-1 sudo podman exec -it cockpit-tasks-1 podman-remote ps
Warning: Permanently added '10.0.203.194' (ED25519) to the list of known hosts.
CONTAINER ID  IMAGE                         COMMAND               CREATED        STATUS        PORTS                 NAMES
51ef06c0f09c  quay.io/minio/minio:latest    server /data          4 days ago     Up 4 days     0.0.0.0:80->9000/tcp  cockpit-s3
ea0bd1fbcc99  quay.io/cockpit/tasks:latest  /usr/local/bin/co...  5 minutes ago  Up 5 minutes                        cockpit-tasks-1

Once this lands, I'll apply it to all our production machines.

[martinpitt]

Ugh, running the test on GitHub is difficult. First of all, the setfacl did succeed, but the container still gets an EPERM. If I hack this with chmod 666, then it fails with

Error: unable to connect to Podman socket: server API version is too old. Client "4.0.0" server "3.4.4"

containers/podman#17362 doesn't make me very hopeful, but there is somebody who backported the Debian bookworm packages to Ubuntu 22.04: https://launchpad.net/~quarckster/+archive/ubuntu/containers . This works very well indeed!

I don't want to waste too much time on the EPERM on github, so I just hacked it.

[martinpitt]

Rebased after #585

[allisonkarlitskaya]

Why not use the "normal" location on the inside, and save ourselves the envvar?

[martinpitt]

Heh, I spent about two hours on that. There's no $XDG_RUNTIME_DIR in the tasks container, so podman defaults to some /tmp/podman-1111/whatever path which doesn't feel very predictable. Plus, if you actually bind-mount it there, it then wants a /work/rundir/podman/... directory. This is by far the easiest and most robust version that I found.

[allisonkarlitskaya]

I mean /run/podman/podman.sock. I guess that's only the default when you're root, though?

[martinpitt]

But we aren't, we are uid 1111 😁

[martinpitt]

@allisonkarlitskaya Simplified as discussed, fixed the rebase error, and I redeployed it on rhos-01-1. I re-validated that I can access podman there.

[allisonkarlitskaya]

Better, thanks!