Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create GitHub Actions workflows to build and push Docker Images #212

Merged
merged 15 commits into from
Dec 12, 2024

Conversation

kabilar
Copy link
Member

@kabilar kabilar commented Dec 11, 2024

Fix #141

  • Add GitHub Actions workflow to test the image builds upon pull request
  • Add GitHub Actions workflow to build and push to Docker Hub upon merge to main
    • Use existing DOCKERHUB_TOKEN and DOCKERHUB_USERNAME GitHub Actions secrets
    • Rename Dockerfile tag from latest to latest-base
    • Confirm that the images build on my fork and push to my Docker Hub. See successful GitHub Actions run and Docker Hub.
  • Disable Build configuration on Docker Hub since we will now be pushing from GitHub

cc @satra

@kabilar kabilar requested a review from asmacdo December 11, 2024 20:48
Copy link
Member

@satra satra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall looks good to me. it may be good to see if the build caches so that we don't rebuild all each time this workflow is run.

@asmacdo
Copy link
Member

asmacdo commented Dec 12, 2024

LGTM

I don't think it will do any build caching since it will fire up a totally fresh instance each push. That shouldn't be a problem for dockerhub if the builds are deterministic, but it probably will waste some power.

IIUC these images don't actually make use of anything in the repo other than the Dockerfiles themselves, so maybe something like...

on:
  push:
    paths:
      - 'Dockerfile.*'

https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#example-including-paths

@kabilar
Copy link
Member Author

kabilar commented Dec 12, 2024

overall looks good to me. it may be good to see if the build caches so that we don't rebuild all each time this workflow is run.

Good idea. I was able to add a caching mechanism. For reference, here are the docs:

@kabilar
Copy link
Member Author

kabilar commented Dec 12, 2024

IIUC these images don't actually make use of anything in the repo other than the Dockerfiles themselves, so maybe something like...

Good idea. I have added the paths filter to both workflows.

@kabilar
Copy link
Member Author

kabilar commented Dec 12, 2024

Proceeding with the merge here since all suggestions have been addressed.

@kabilar kabilar merged commit 731e0ad into dandi:main Dec 12, 2024
@kabilar
Copy link
Member Author

kabilar commented Dec 12, 2024

Hi @asmacdo, did you set an expiration date on Docker Hub for the DOCKERHUB_TOKEN secret? I was able to successfully run the Build Docker Images workflow but the Build and Push Docker Images failed with the following error (link):

#19 ERROR: failed to push ***/dandihub:latest-matlab: failed to authorize: failed to fetch oauth token: unexpected status from GET request to https://auth.docker.io/token?scope=repository%3A***%2Fdandihub%3Apull%2Cpush&service=registry.docker.io: 401 Unauthorized

If the token is expired, I can replace the current one. Thanks.

@asmacdo
Copy link
Member

asmacdo commented Dec 12, 2024

@kabilar I used a repo specific key, you'll need a new one.

@kabilar
Copy link
Member Author

kabilar commented Dec 12, 2024

Thanks @asmacdo.

All images have been built and pushed to Docker Hub. See latest GA run and images on Docker Hub.

cc @satra

@satra
Copy link
Member

satra commented Dec 12, 2024

@kabilar - thank you. as part of this should we disable autobuilds on dockerhub?

@kabilar
Copy link
Member Author

kabilar commented Dec 12, 2024

Hi @satra, I disabled the autobuilds on dockerhub.

I have noticed an issue with the GA builds. Only the images/Dockerfile is being built for all entries in matrix strategy. Trying to resolve now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Review (and possibly update) the automated Docker image build and publishing processes
3 participants