FTP Plugin - Unable to get Passive FTPES working

Created April 05, 2023 10:00

We're trying to upload artifacts to a Windows VM hosted in Azure.

We've been trying to setup an FTP server for TeamCity to upload to.

We tried using FileZilla FTP Server, however TeamCity errors with the message "425 Unable to build data connection: TLS session of data connection was not resumed." This is somewhat expected as we've not been able to use FileZilla FTP Server except with the FileZilla FTP Client.

So we tried using the Windows FTP Service instead. However, it responds to "PBSZ 16384" with "501" - i.e. 16384 is too large. Having done some research, it seems that FTP(E)S generally sticks with "PBSZ 0" because TLS is a transport-layer encryption protocol. Having looked at the source code, it seems that 16384 is hard coded in the plugin.

To be clear, we were able to connect to both FTPs with the FileZilla FTP Client, but neither worked with the TeamCity FTP Plugin.

Is there something obvious that we're missing that would allow us to use either of the approaches above?

If not, what's the recommended way to securely transfer files to a remote server?

4 comments

Anatoly Cherenkov

Created April 06, 2023 16:26

Hi! The buffer of 16384 bytes shouldn't be too large because the supported range of values for the PBSZ command is from 0 to 2^32, according to RFC 2228. The 501 response might also mean that the server couldn't parse the argument for another reason.

   PROTECTION BUFFER SIZE (PBSZ)

      <...>  This number shall be no greater than can be
      represented in a 32-bit unsigned integer.

      This command allows the FTP client and server to negotiate a
      maximum protected buffer size for the connection.  There is no
      default size; <...>

      <...>

      If the server cannot parse the argument, or if it will not fit in
      32 bits, it should respond with a 501 reply code.

More information about establishing a connection should be available in the full build log (go to the Build Log page > Download log > select Full build log or Zip). Feel free to share it via https://uploads.jetbrains.com/ (also share the upload ID if you do so).

Daniel Heard

Created April 11, 2023 08:48

According to https://enterprisedt.com/products/edtftpjssl/doc/manual/html/ftpscommands.html :

The PBSZ command is intended by RFC2228 to define the buffer-size to be used by the security mechanism when it is encrypting data on the data-channel. However, since TLS is a transport-layer protocol and therefore doesn't require explicit encryption of data by the application layer, this buffer-size is redundant. Ford-Hutchinson therefore requires that a value of '0' is always passed as a parameter.
> PBSZ 0
While this call is redundant (as it is effectively implied by the AUTH command), it is required and must precede the PROT command.

While this is one implementation, we can imagine the reasoning being used with other implementations.

Relevant TeamCity log lines:

E: Step 2/17: Upload (FTP Upload)
i: [Step 2/17] > USER *******
i: [Step 2/17] < 331 Password required

i: [Step 2/17] > PASS *******
i: [Step 2/17] < 230 User logged in.

i: [Step 2/17] > PBSZ 16384

i: [Step 2/17] < 501 Server cannot accept argument.

E: [Step 2/17] Step Upload (FTP Upload) failed

E: Unknown build problem reported on agent
: Build finished

Relevant Windows FTP log lines:

DATETIME REMOTE_ADDR - LOCAL_ADDR 21 ControlChannelOpened - - 0 0 58c4 -
DATETIME REMOTE_ADDR - LOCAL_ADDR 21 AUTH TLS 234 0 0 58c4 -
DATETIME REMOTE_ADDR - LOCAL_ADDR 21 USER teamcity 331 0 0 58c4 -
DATETIME REMOTE_ADDR teamcity LOCAL_ADDR 21 PASS *** 230 0 0 58c4 /
DATETIME REMOTE_ADDR teamcity LOCAL_ADDR 21 PBSZ 16384 501 87 0 58c4 -
DATETIME REMOTE_ADDR teamcity LOCAL_ADDR 21 ControlChannelClosed - - 0 0 58c4 -

If we use FileZilla FTP Client to connect, the client issues PBSZ 0 to which the server responds with 200. Similarly if we use IntelliJ to connect.

Anatoly Cherenkov

Created April 14, 2023 12:52

Thanks for the details. Unfortunately, the buffer size is currently hard-coded. We will add an option to change the buffer size. Please track the related bug here: https://youtrack.jetbrains.com/issue/TW-80496

Daniel Heard

Created April 14, 2023 12:58

Thank you for the update!

Please sign in to leave a comment.