Publish artifacts to S3 only
Hey there, we generate fairly sizeable artifacts (~2-3GB) on every build, and I'd like these to be backed up on our GCP S3 bucket, but not locally. Is this possible?
I only recently realised that the artifacts were being stored both locally and on S3, when I noticed that we were rapidly running out of space on our self-hosted TC server. Ideally, in addition to the above, I'd also like to figure out a cleanup rule to delete all the current artifacts on our local machine.
Please sign in to leave a comment.
What are your Project > Artifacts Storage (https://www.jetbrains.com/help/teamcity/configuring-artifacts-storage.html) settings for the affected project? Please share the screenshot.
Best regards,
Anton
I'm attaching two pictures of our setup, which our project inherits from.
I assume that the project itself doesn't override these inherited settings and doesn't have the built-in storage active. Please correct me if I'm wrong.
What do you see in <TeamCity Data Directory>/system/artifacts? Do you see actual published artifacts? Or internal artifacts and metadata? As per documentation:
When external storage for artifacts is enabled, the TeamCity internal artifacts (including build logs) will still be published to the TeamCity server and stored in the TeamCity Data Directory in the built-in artifacts' storage. The same applies to the metadata about artifacts mappings, which will be published to the artifacts directory of the TeamCity Data Directory.
Could you provide some details on what your build publishes as artifacts and what is published to <TeamCity Data Directory>/system/artifacts? If you want to share a screenshot you may do so with https://uploads.jetbrains.com/ and share the upload ID.
Best regards,
Anton
Apologies for the delay getting back to you, & full disclosure: I had to delete a lot of our older artifacts, as our TC server was rapidly running out of space, so I wasn't able to look back at historical builds.
However, I did just run a new test build, and indeed it looks like the internal artifacts (logs, settings, &c.) were stored internally, while the external artifact (the .zip file containing the build of our game) was correctly sent to our GCP S3 storage. I know some of the older log folders seem to be fairly large (on the order of 100s of MB), so it's possible those were piling up.
In any case, with this new build verifying what I thought was the expected behaviour (user-defined artifacts being stored externally), I think that does it for me. I'll make sure to check our cleanup rules for the internal artifacts, but for now I think my issue is resolved.
Thanks for your help!