Search code examples
gitlabgitlab-cigitlab-omnibus

Managing (older) CI artifacts in GitLab CE Omnibus

Good day all,

I run a GitLab CE Omnibus (8.4.3) installation for my company. We recently started using CI, but still mainly for generating documentation. Binary builds is in the process of being added.

As part of the system, I run a daily backup task (using gitlab-rake gitlab:backup:create). Over the last couple of days, these artifacts have started becoming extremely large, even though it is just documentation at this stage (doxygen generated images seem to be the major source of the problem). Because artifacts are included in the backup, daily backups (current strategy keeps 2 weeks' worth) have become unwieldy in size.

I've searched a fair bit for some answers, but most of the hits I get seem to be related to feature requests at this stage. I'm aware that CI is still relatively new, but I would imagine artifact management is reasonably high on the requirements list.

I have three questions:

  1. Is there a way to manage/delete older artifacts in GitLab, short of manually deleting them from the disk (and possibly breaking links in the process)? It would be ideal if this process could be automated.

  2. Following on from 1, is it possible to target specific classes of artifacts in the cleaning strategy (ie, only delete older documentation targets, but not binaries, etc)?

  3. Is it possible to either entirely, or based on CI target types, exclude artifacts from the recommended gitlab-rake gitlab:backup:create backup procedure?

Any links, tips or advice would be highly appreciated!

Kind regards,

[UPDATE] Some more reading, yielded the following:

  1. As of Gitlab 8.5, I can manually delete single build artifacts. This helps, but is not scalable. Timeline for proper artifact management (including expiry dates, etc) seems to be Gitlab 8.7.

  2. There seems to be no clear request for handling of different build target artifacts in different ways yet.

  3. No information on removing artifacts from the backup task.

Solution

  • I would be very interested in finding out answers to 1 & 2 too.

    For excluding items on the backup task try the SKIP argument:

    gitlab-rake gitlab:backup:create SKIP=artifacts,builds

    You can also SKIP repositories,lfs,uploads. Just supply them as a comma separated list. See also https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/raketasks/backup_restore.md