Need DLLs for agent build process but do not want to deploy them

Hi - 

We have some base DLLs that will never change but are required as references within the Visual Studio project for compiling.

We do not want to deploy these base DLLs every time.  (They add 45 min to the deploy process).

Also, we clean the build directory for each build/deploy.

Is there a way we can retrieve the base DLLs from source control and have the build agent compile with them but not deploy them (as they already exist on the target server)?

We use TeamCity, SVN, and one build step for the solution in Visual Studio 2015.

Thanks for any assistance.