AWS credentials management system (multiple accounts)

I tried the new AWS credentials management system (Teamcity 2022.10). I think it is a real need feature, that makes it possible to make connections to AWS more secure. I found the documentation somewhat lacking, for e.g. how to use de generated credentials file in combination with e.g. the aws CLI. The biggest showstopper for me using it is how to use it I'm combination with multiple AWS accounts.

E.g. I have pipelines that deploy to multiple AWS environments depending on the selected environment. If I add multiple AWS credentials to a pipeline I only see one file generated with AWS credentials and the generated file only contains the temporary credentials for one AWS account.

Is it possible to make this feature compatible with using multiple AWS accounts? Or am I overlooking something and is this already possible?