egeria-workspaces provides a runtime environment for learning, experimenting and using Egeria. The default configuration sets up a full Egeria system and provides pre-built content to help you get started.
This environment is not designed for enterprise-wide use. Please see the Planning Guide for more information about designing bespoke Egeria deployments, such as Cloud Native approaches and the use of Helm charts to configure Kubernetes clusters. For further help and advice, please feel free to engage with the community on our slack channel - we'd love your feedback and participation.
egeria-workspaces consists of a number of artifacts reflected by the folder structure itself. Here is a quick tour:
Subfolders contain artifacts for different deployments of Egeria along with optional runtimes often used with Egeria. The deployments provide docker compose scripts to orchestrate the building, configuration and startup of the components needed. Here is the break-down of the configurations:
This provides the standard Egeria deployment, suitable for most use cases.
It contains the egeria-quickstart.yaml docker compose script and supporting files. The result of running this compose script sets up:
- Egeria running on port 9444
- Apache Kafka running on port 9192
- Postgres running on port 5442
- Jupyter running on port 8888
- Open Lineage Proxy running on port 6000
There are several additional features of this environment - including sharing of select folders between Jupyter and Egeria, and externalization of the configuration and runtime information to make it available outside of the docker containers. More details can be found in the README.md file within this directory.
This folder contains some sample docker compose scripts to start some other runtimes that we often use with Egeria. Currently this includes:
- airflow & marquez - Apache Airflow is a popular open source workflow runtime and marquez offers some very nice visualization of open lineage graphs.
- superset - Apache Superset is an open source reporting and dashboarding tool.
While the egeria-quickstart environment is a good starting point for most folk, we've also included some other docker scripts to support some simpler deployments. The available deployments are:
- egeria-platform-compose - deploys Egeria with an XTDB file based repository along with Kafka.
- egeria-platform-jupyter-compose - additionally adds a Jupyter server
- egeria-platform-postgres-compose - deploys the postgres database for use with Egeria, and Kafka.
These simpler configurations do not externalize their configurations and only share a subset of the folders. They provide configurations for these servers:
- active-metadata-store
- simple-metadata-store
- integration-daemon
- engine-host
- view-server
More details can be found in the README.md files within this folder.
The exchange folder is to support exchange of file-based information between the Egeria running in a docker container, the Jupyter environment, and the host file-system.
The distribution hub is where Egeria can place information and results that it generates so that they are easily visible to the users and Jupyter, This information currently includes:
- logs - Egeria audit logs (if file based event logging has been configured)
- surveys - Survey reports generated by Egeria based on user request or automation.
The landing-area directory (or any of its subdirectories) are monitored by the qs-integration-daemon server. If you add files under this directory, they will be automatically classified using their file name and file extension, and then catalogued into the qs-metadata-store metadata repository as assets.
The landing-bay directory is where users place information to be ingested by Egeria. There are sub-directories for different kinds of information:
- glossary - for importing and exporting glossary terms
- open-metadata-archives - for importing open-metadata-archives
- secrets - to configure local secrets; default secrets are provided, yet will often be replaced with local information.
The information in these folders are used by the Runtimes. They are not for the general user to use. Externalizing runtime information here, rather than embedded within the containers, means that if containers are upgraded or destroyed, the environment can still be recovered. Currently there are sub-directories here for:
- airflow-volumes
- egeria-pg
- egeria-platform-data
This folder is meant for you to put your own private working files for use with Egeria and Jupyter. The directory is mounted and visible within both Egeria and Jupyter runtimes.
This set of folders contains examples, samples, utilities and other artifacts useful to getting started with Egeria. Please explore. Extend if desired, and if you want to contribute your own content to the community feel free to contact us via Slack or email.
License: CC BY 4.0, Copyright Contributors to the ODPi Egeria project.