clean-up not removing artifacts

Hi, according to our documentation (https://confluence.jetbrains.com/display/TCD8/Clean-Up#Clean-Up-CleanupRules) the following builds (and their artifacts won't be deleted):

• pinned builds;
• builds used as a source for artifact dependency in other builds when the "Prevent clean-up" option for dependency artifacts is enabled. See Clean-Up for Dependent Builds below. Such builds are marked with icon in the build history list;
• builds used as snapshot dependency in other not yet deleted builds;
• builds of build configurations that were deleted less than one day ago.

Also, you can find information about cleanup process in teamcity-server.log.

Thanks for the hints, Sergey, however

1) I don't think the undeleted artifacts belong to builds that satisfy any of the 4 requirements you list (pinned, source for artificial dependency in builds that 'prevent cleanup...', snapshot dependency of undeleted build, belong to a recently deleted configuration).   (Although I confess I'm not sure how to check for the 2nd or 3rd ones).

2) I don't find a "teamcity-server.log", either on my server or in the documentation.  Did you mean teamcity-agent.log?  If so, that file does have lines include "artifact" and lines with "clean", but the only lines that contain both say: "INFO - ages.freeSpace.CleanHandleImpl - Removing files under /Users/teamcity/teamcity_agent_home/system/.artifacts_cache with Artifacts cache cleaner".

3) Note, I have turned on logging, via Administration > Server Administration > Diagnostics > Active logging preset: <debug-all>.   

So my question stands: is there any way to debug the cleanup process, such as seeing the reason a given (directory or file) was or wasn't deleted?

Cleanup rules are applied only to TC server. Cleanup on TC agent is not configurable via WebUI. I'm a bit confused, because the original path you mentioned (.BuildServer/system/artifacts/) belongs to TC server, but the second (/Users/teamcity/teamcity_agent_home/system/.artifacts_cache) is an agent's folder.

Server's log teamcity-server.log is located under <TC installation>/logs/. Also, you can view (and download this log) from Web UI: Administration > Server Administration > Diagnostics > Server Logs.

There are some logging during cleanup process, which is usually enough to understand the reason.

Thanks for the explanation of how to find "teamcity-server.log" in the WebUI.  I also found it inside "/Library/TeamCity/logs" on the server.

There are many build-artfifacts directories that appear in the log, but still exist at the end of the cleanup.  Using just one example, ".BuildServer/system/artifacts/FOO_IOS/Enterprise\ Configuration/14614", I see in the log...

    [2015-01-14 03:00:00,004]   INFO -  jetbrains.buildServer.CLEANUP - Preparing for clean-up...
    [2015-01-14 03:00:00,004]   INFO -  jetbrains.buildServer.CLEANUP - Clean-up started...
    [2015-01-14 03:00:00,035]   INFO -  jetbrains.buildServer.CLEANUP - Collecting builds depending on other builds...
    [2015-01-14 03:00:00,097]   INFO -  jetbrains.buildServer.CLEANUP - Done collecting depending builds for each cleanup level
    ...
    [2015-01-14 03:00:00,100]   INFO -  jetbrains.buildServer.CLEANUP - Processing Foo :: FOO iOS :: Enterprise Configuration {id=SixFlags_SixFlagsIOS_EnterpriseConfiguration, internal id=bt102}
    [2015-01-14 03:00:00,100]   INFO -  jetbrains.buildServer.CLEANUP -  Applying policy: clean ARTIFACTS more than 25 days older than the last build; artifact patterns: +:**/*
    ...
    ...
    [2015-01-26 03:00:04,962]   INFO -  jetbrains.buildServer.CLEANUP - Processing builds pack #46 (6 left), build ids: [  <snip>...., 14614, ....<snip>]
    [2015-01-26 03:00:04,975]   INFO -  jetbrains.buildServer.CLEANUP - Cleaning extensions data...
    [2015-01-26 03:00:04,975]   INFO -  jetbrains.buildServer.CLEANUP - Deleting build artifacts...

But on the disk, that folder still exists, and contains these items.

     136 Jan 22 09:10 14614
    6148 Jan 22 09:10 14614/.DS_Store
     136 Dec  8 10:25 14614/.teamcity
     136 Dec  8 10:25 14614/.teamcity/properties
    2604 Dec  8 10:25 14614/.teamcity/properties/build.finish.properties.gz
    2604 Dec  8 10:25 14614/.teamcity/properties/build.start.properties.gz
     102 Dec  8 10:25 14614/.teamcity/settings
      40 Dec  8 10:25 14614/.teamcity/settings/digest.txt

Shouldn't this directory have been deleted?  Does TeamCity get tricked by the existence of the .DS_Store file (an OS X artifact)?

Files under .teamcity contain build-related information. They will only be deleted, when the build itself is deleted.