Replies: 1 comment
-
Yes I think so, as for a local agent the Dagster project files must be available in the local file system. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi
I'm evaluating Dagster+ Hybrid
I will install two Dagster Agent, one on Windows the other on Linux
The one on Windows is specific to execute some powershell and Active Directory requests that must be executed on a Windows Machine for network and domain security reasons
For this specific use case, it seems I'm forced to install a local agent (native windows python installation, no docker) on my windows host where powershell and official active directory powershell module will be installed
For the code location I have to declare a code location as python module/file/package that will be available locally on my windows Host
My question is, if my python module containing definitions and assets are available from Git Repository, is it possible my Windows Dagster local agent pull automatically my Python definitions package from the git repository when materializing it ?
I've tried with a Docker Dagster Agent and a gitlab workflow, build+push and the Docker agent pull automatically the newest image. My problem with Docker agent is that it is not available natively on windows and I have to use WSL distribution and so I have no longer access to native powershell tools and native/certifiied Active Directory powershell module (the Open Active Directory available on Linux is not officially certified)
So with Local Agent I have access directly to powershell tools and certified ActiveDirectory Module BUT I have not automatic reinstallation of the newest python package/definitions from git repository
Do I have to use gitlab pipeline to connect to my windows server with SSH to update the local python package before running my Dagster pipeline ? Is this the only solution ?
Beta Was this translation helpful? Give feedback.
All reactions