Git Private Key

I'm trying to setup TeamCity for the first time. The configuration is git (hosted by unfuddle), MSBuild VS2008 or 2010.
When testing the connection I get an error - unable to load identity file.
I have the the private key path set to C:\Users\Craig\.ssh
This works for the standard git bash environment.
I'm no expert with git and ssh, any clues would be appreciated!

I've setup teamcity on my laptop as a trial.

I would love to get it all work before setting up on our server at work.

Thanks,
Craig.