Data Platform/Systems/DataHub/Upgrading

The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

The upstream DataHub repository is: https://github.com/linkedin/datahub/

At the moment we maintain a fork of DataHub here: https://gerrit.wikimedia.org/r/admin/repos/analytics/datahub

Update - We are about to unfork from the upstream project and we will shortly migrate our build pipeline to GitLab.

Deploy datahub CLI tool

The version of the CLI tool has to match the server version, so we have to:

Update the datahub-cli version on the packaged virtual environment
Build and publish to Archiva: Analytics/Systems/Archiva#Uploading_Dependency_Artifacts
Update the artifact version and metadata ingestion jobs in the airflow jobs repository