rsync to server using uploaded SSH key

I want to use rsync to deploy build artifacts to a Linux server (both for speed and to delete files from the server that no longer exist in the build output). There doesn't seem to be a plugin for this. I can, of course, write it as a command line build step, but is there a way to authenticate using a public key uploaded to TeamCity (like when using SSH Upload)?