Perforce and corrputed files

Has anyone ever seen the following error within there builds?  Exception include below:

We started getting this error in the middle of the night.  The build worked fine before midnight, but afterwards it failed.  Local installs of TC on windows work fine.  The install on Linux is now causing issues.

We checked the files within the buildagent ( those checked out from perforce ) and we discovered that binary files ( *.jar, *.doc, etc) are corrupted.  You can still open the jar with winrar, however there is an extra root directory within it ( named the same as the jar minus the extension)

Any ideas....we are baffled.

We are using build 3.0 5985

Buildfile: build.xml

[14:23:13]:   [taskdef] java.util.zip.ZipException: Could not find End Of Central Directory

[14:23:13]:   [taskdef] at java.util.zip.ZipFile.open(Ljava.lang.String;I)I(Native Method)

[14:23:13]:   [taskdef] at java.util.zip.ZipFile.<init>(Ljava.io.File;I)V(Unknown Source)

[14:23:13]:   [taskdef] at java.util.zip.ZipFile.<init>(Ljava.io.File;)V(Unknown Source)

[14:23:13]:   [taskdef] at org.apache.tools.ant.AntClassLoader.getResourceURL(AntClassLoader.java:919)

[14:23:13]:   [taskdef] at org.apache.tools.ant.AntClassLoader$ResourceEnumeration.findNextResource(AntClassLoader.java:126)

[14:23:13]:   [taskdef] at org.apache.tools.ant.AntClassLoader$ResourceEnumeration.<init>(AntClassLoader.java:88)

[14:23:13]:   [taskdef] at org.apache.tools.ant.AntClassLoader.findResources(AntClassLoader.java:869)

[14:23:13]:   [taskdef] at java.lang.ClassLoader.getResources(Ljava.lang.String;)Ljava.util.Enumeration;(Unknown Source)

[14:23:13]:   [taskdef] at org.apache.tools.ant.taskdefs.Definer.resourceToURLs(Definer.java:267)

[14:23:13]:   [taskdef] at org.apache.tools.ant.taskdefs.Definer.execute(Definer.java:211)

[14:23:13]:   [taskdef] at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:275)

[14:23:13]:   [taskdef] at org.apache.tools.ant.Task.perform(Task.java:364)

[14:23:13]:   [taskdef] at org.apache.tools.ant.Target.execute(Target.java:341)

[14:23:13]:   [taskdef] at org.apache.tools.ant.helper.ProjectHelper2.parse(ProjectHelper2.java:142)

[14:23:13]:   [taskdef] at org.apache.tools.ant.ProjectHelper.configureProject(ProjectHelper.java:91)

[14:23:13]:   [taskdef] at org.apache.tools.ant.Main.runBuild(Main.java:653)

[14:23:13]:   [taskdef] at org.apache.tools.ant.Main.startAnt(Main.java:187)

[14:23:13]:   [taskdef] at org.apache.tools.ant.launch.Launcher.run(Launcher.java:246)

[14:23:13]:   [taskdef] at org.apache.tools.ant.launch.Launcher.main(Launcher.java:67)

13 comments
Comment actions Permalink

Update....

I have confirmed that all binary files extracted with Perforce within Teamcity are being wrapped with GZip archive.

So it looks like this:

     ReadMe.doc  ( GZip archive)
          |----ReadMe ( word document)

where before it was
     ReadMe.doc ( word document )

The original file ReadMe.doc ( a word doc) is converted to GZip.  It contains one entry ( ReadMe ) if i extract this file using winrar and open within Word it succeeds.

very frustrating

0
Comment actions Permalink

Hello,

  Could you please check, what is the file type for these files in Perforce.
  Are these files gzipped manually?
 
  Regards,
  KIR

0
Comment actions Permalink

The perforce filetype is binary.

No they are not manually gzipped.  We tracked the exception down to a specific jar file within an ant script.  When we opened the jar file we noticed that it was a jar file that was wrapped with a gzip archive. Hence the ZipException: Could not find End Of Central Directory. 
It turns out that all the binary files are now exhibiting this behaviour when checked out from Perforce.

This issue only seemed to arise within our Teamcity/Linux environment

p4-commandline/Linux     --> ok
p4-commandline/Windows -->ok
Teamcity/Windows          -->ok
Teamcity/Linux               --fail

Also, it has been working ( this specific build configuration ) without issue for close to a year now.  The errors only started to appear in the early morning hours of Nov 26th.  We have confirmed with our Perforce team that no maintenance was done on their side.

Any thoughts as to why or how behaviour appeared and how can we solve it?

0
Comment actions Permalink

As I found in Perforce docs, 
"Some file types are compressed to gzip format when stored in the depot. The compression occurs when you submit the file, and decompression happens when you sync (copy the file from the server to the workspace). The client workspace always contains the file as it was submitted."

But this doesn't explain why do we get gzipped content from perforce.

Did you try to reset source on the linux build agent? There is an action to do it under "Actions" on the build configuration page, near Run button in toolbar.

What version of Perforce do you use?

Regards,
KIR

0
Comment actions Permalink

yes, i reset the source on the linux machine, however the synced files were still gzipped.


Perforce versions
Client:
     Rev. P4/LINUX24X86/2007.2/131394 (2007/08/14).
Server:
     Proxy Version:      P4P/LINUX26X86_64/2008.1/158777 (2008/07/09)
     Server Version:      P4D/LINUX26X86_64/2008.1/164042 (2008/09/08)

Here's a weird thing.  Yesterday our Windows machines worked...today they do not.  We do have some custom code implemented with TeamCity ( nothing to do with vcs/perforce ) so we decided to test TeamCity with a fresh install ( clean, straight out of the box with no custom add-ons/plugins).  The result -> we still get the gzipped archives.

Is there anything specific or unique with the way you guys take the files from perforce and write them to disk?  I'm completely baffled by all of this.

Any help would be appreciated.

Thanks

J

0
Comment actions Permalink

Hello,

   All of this is really weird. It looks like Perforce forgets to ungzip content before returning it to TeamCity.

   Could you please do the following:
   - setup latest P4 client on a machine with TeamCity
   - setup TeamCity 4.0 release as a fresh installation
   - enable debug logs for perforce in teamcity-server-log4j.xml as described in docs (this makes sense only for TC 4.0).
     I'm interested in logs for category jetbrains.buildServer.VCS.P4
     Please send us teamcity-vcs.log and teamcity-server.logs after reproducing the problem.

   Kind regards,
   KIR

0
Comment actions Permalink

OK....

I setup as follows:
     TC 4.0 - fresh install...straight out of the box - no custom code
     P4 client: Rev. P4/NTX86/2008.1/168182 (2008/10/10).
     Perforce Infrastructure:
     Perforce Server: P4D/LINUX26X86_64/2008.1/164042 (2008/09/08)
     Perforce Proxy: P4P/LINUX26X86_64/2008.1/158777 (2008/07/09)

I updated the log config files to generate the debug output. I am sending the files to teamcity-feedback@jetbrains.com as they are too big to attach here.


I noticed the line within the vcs logs as follows:

[2008-11-28 14:14:16,003]  DEBUG -   jetbrains.buildServer.VCS.P4 - run p4 -P xxx -c atf -u joverton -p <perforce-port> -G print //depot/dev/apps/tools/ATF/...@1847192

i ran this command within my cmd prompt and directed to a file...similar to what you are doing within the agent ( i presume).  How do you parse the file afterwards as it is made up of all the files within the perforce view. And is this the spot where the file is not being uncompressed?  I ask as all the methods i've tried to replicate the compressed file outside of teamcity has failed ( I mean the files are not corrupted).  It is only via Teamcity-perforce config does the file end up being corrupted.

Thanks for your help

0
Comment actions Permalink

Hello,

   You've found the command we use to obtain the file content. We store this content to a temporary file
   and then parse it. While parsing, we don't assume the content to be gzipped.

   If possible, I'd appreciate if you send us the result of the command execution for a file with gzipped output.

   I've created an issue in our tracker for the problem: TW-6366

   Thanks!

   KIR

0
Comment actions Permalink

Hello,

   I think I've found it. From the Perforce change log:


Bugs fixed since 2008.1/156517 (beta)
    #175236 **
           When using the proxy, 'p4 print' of binary files without
           the '-o' flag would be output as compressed and possibly
           output twice in a row.  Fixed.  (Bug #31596)

  Looks like this is Perforce problem.

  Regards,
  KIR
0
Comment actions Permalink

i saw that withoin the release notes...however, we are using a perforce version greater than the one specified
The bug is found within 2008.1/156517; we are using 2008.1/164042 .

0
Comment actions Permalink

Hello,

  In fact, this bug is marked with revision #175236.

  And the current downloadable version of P4D server is 2008.1/176084

  So I believe the problem still persists in your version of the server.

  Kind regards,
  KIR

0
Comment actions Permalink

ahhhhhh yes, it is clear now.....i saw that number and assumed it was a bug id.  Makes sense now.  I guess i have to investigate the possibility of an upgrade from our side.


Thanks for your help

Jeremy

0
Comment actions Permalink

ahhhhhh yes, it is clear now.....i saw that number and assumed it was a bug id.  Makes sense now.  I guess i have to investigate the possibility of an upgrade from our side.


Thanks for your help


Jeremy

0

Please sign in to leave a comment.