JetBrains: Please rethink VcsSupport!

All,

Over the last 2 weeks I have been working on getting MKS fully integrated into TeamCity with no luck. I would like this post to serve as both a warning to others and a plea to JetBrains to please rethink VcsSupport for TeamCity.

For those of you holding your breath for an MKS plugin:
Dont. After wasting 2 weeks trying to integrate with VcsSupport, I can tell you that the best you can hope for is broken support at best - which I refuse to do.

JetBrains:
It seems to me that VcsSupport was built entirely around functionality present in SVN, CVS, and to a lesser degree, Perforce rather than common VCS concepts (ie: a VCS will at least checkout/resynchronize/etc.). This makes it all but impossible to integrate VCS that do not follow that usage pattern.

Please consider the following suggestions:

1) Refactor VcsSupport to support primitive VCS operations only, such as checkout/resyncrhonize and returning a modification set based upon that rather than using awkward API's such as collectBuildChanges() and buildPatch(). If additional tracking is needed to support personal changelists etc. these should be handled soley by TeamCity.

2) Do not make VCS operations time-sensitive. If it is neccessary to track filesystem changes by time, TeamCity should track these changes internally rather than inflicting the work on VcsSupport authors. Some VCS (such as MKS) do not support collecting build changes or even project resynchronization by date which makes it impossible to use with TeamCity.

3) Do away with getContent(). Not all VCS correctly support fetching content based on dates. TeamCity should be intelligent enough to track changes itself, and look at the filesystem for the most current revision (or issue a checkout).

4) Documentation! I have been able to figure out how VcsSupport works in the wild only through bits from forum posts, decompilations, and lots of trial by fire. Please document these API's in a public javadoc rather than distributing a source .jar!


Regards,

Steven Stallion

12 comments
Comment actions Permalink

Hello Steven,

1) Refactor VcsSupport to support primitive VCS operations only, such as
checkout/resyncrhonize and returning a modification set based upon that
rather than using awkward API's such as collectBuildChanges() and
buildPatch().


Current VcsSupport has to work on the server but sources are on the agent.
So there should be some way to collect all changed files and send them to an
agent.

>If additional tracking is needed to support personal changelists etc. these
>should be handled soley by TeamCity.

We want to view changes for all of our builds not for personal ones only.

2) Do not make VCS operations time-sensitive.
You don't have to use dates at all. All you need is define repository
version. It can be any string value

3) Do away with getContent().

So what can you suggest for sending patch to an agent?

>TeamCity should be intelligent enough to track changes itself
Excuse me, but where should TeamCity treack this changes? For all current
version controls there are no project files on the server at all.

We can write support for such kind of VCSes, I guess, which will track file
system and collect changes itself, but it relates to implementation not to
API itself, doesn't it?

--
Olesya Smirnova
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"


"Steven Stallion" <no_reply@jetbrains.com> wrote in message
news:5172367.1162911630715.JavaMail.itn@is.intellij.net...

All,

>

Over the last 2 weeks I have been working on getting MKS fully integrated
into TeamCity with no luck. I would like this post to serve as both a
warning to others and a plea to JetBrains to please rethink VcsSupport for
TeamCity.

>

For those of you holding your breath for an MKS plugin:
Dont. After wasting 2 weeks trying to integrate with VcsSupport, I can
tell you that the best you can hope for is broken support at best - which
I refuse to do.

>

JetBrains:
It seems to me that VcsSupport was built entirely around functionality
present in SVN, CVS, and to a lesser degree, Perforce rather than common
VCS concepts (ie: a VCS will at least checkout/resynchronize/etc.). This
makes it all but impossible to integrate VCS that do not follow that usage
pattern.

>

Please consider the following suggestions:

>

1) Refactor VcsSupport to support primitive VCS operations only, such as
checkout/resyncrhonize and returning a modification set based upon that
rather than using awkward API's such as collectBuildChanges() and
buildPatch(). If additional tracking is needed to support personal
changelists etc. these should be handled soley by TeamCity.

>

2) Do not make VCS operations time-sensitive. If it is neccessary to track
filesystem changes by time, TeamCity should track these changes internally
rather than inflicting the work on VcsSupport authors. Some VCS (such as
MKS) do not support collecting build changes or even project
resynchronization by date which makes it impossible to use with
TeamCity.

>

3) Do away with getContent(). Not all VCS correctly support fetching
content based on dates. TeamCity should be intelligent enough to track
changes itself, and look at the filesystem for the most current revision
(or issue a checkout).

>

4) Documentation! I have been able to figure out how VcsSupport works in
the wild only through bits from forum posts, decompilations, and lots of
trial by fire. Please document these API's in a public javadoc rather than
distributing a source .jar!

>
>

Regards,

>

Steven Stallion



0
Comment actions Permalink

And there is at least one reason why we just cannot use scheme you suggested
for all vcses.
I mean do update on file system, rescan it, find what was changed.

For cvs and svn it is important to use proper line feeds, but line feeds can
differ on windows and linux. So if server is under windows it is impossible
to change line separators in text files just because we don't know which
ones are text.

Also in this sample we don't know which files are executable and cannot send
this information to the linux agent.

So we can do helper which will work with files on file system, but
information about file type and mode will be lost. Is it suitable for you?
Also information about user and commit message for the change list will be
lost also.

As for getContent you can do not implement them at all, they are used on
plugin side only. I think we can change VcsSupport so that there will be
nullable getContentProvider getter.


--
Olesya Smirnova
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"


"Olesya Smirnova" <Olesya.Smirnova@jetbrains.com> wrote in message
news:eiqh93$7au$1@is.intellij.net...

Hello Steven,

>
>> 1) Refactor VcsSupport to support primitive VCS operations only, such
>> as checkout/resyncrhonize and returning a modification set based upon
>> that rather than using awkward API's such as collectBuildChanges() and
>> buildPatch().
>

Current VcsSupport has to work on the server but sources are on the agent.
So there should be some way to collect all changed files and send them to
an agent.

>
>>If additional tracking is needed to support personal changelists etc.
>>these should be handled soley by TeamCity.
>

We want to view changes for all of our builds not for personal ones only.

>

2) Do not make VCS operations time-sensitive.
You don't have to use dates at all. All you need is define repository
version. It can be any string value

>
>> 3) Do away with getContent().

So what can you suggest for sending patch to an agent?

>
>>TeamCity should be intelligent enough to track changes itself

Excuse me, but where should TeamCity treack this changes? For all current
version controls there are no project files on the server at all.

>

We can write support for such kind of VCSes, I guess, which will track
file system and collect changes itself, but it relates to implementation
not to API itself, doesn't it?

>

--
Olesya Smirnova
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"

>
>

"Steven Stallion" <no_reply@jetbrains.com> wrote in message
news:5172367.1162911630715.JavaMail.itn@is.intellij.net...

>> All,
>>
>> Over the last 2 weeks I have been working on getting MKS fully integrated
>> into TeamCity with no luck. I would like this post to serve as both a
>> warning to others and a plea to JetBrains to please rethink VcsSupport
>> for TeamCity.
>>
>> For those of you holding your breath for an MKS plugin:
>> Dont. After wasting 2 weeks trying to integrate with VcsSupport, I can
>> tell you that the best you can hope for is broken support at best - which
>> I refuse to do.
>>
>> JetBrains:
>> It seems to me that VcsSupport was built entirely around functionality
>> present in SVN, CVS, and to a lesser degree, Perforce rather than common
>> VCS concepts (ie: a VCS will at least checkout/resynchronize/etc.).
>> This makes it all but impossible to integrate VCS that do not follow that
>> usage pattern.
>>
>> Please consider the following suggestions:
>>
>> 1) Refactor VcsSupport to support primitive VCS operations only, such
>> as checkout/resyncrhonize and returning a modification set based upon
>> that rather than using awkward API's such as collectBuildChanges() and
>> buildPatch(). If additional tracking is needed to support personal
>> changelists etc. these should be handled soley by TeamCity.
>>
>> 2) Do not make VCS operations time-sensitive. If it is neccessary to
>> track filesystem changes by time, TeamCity should track these changes
>> internally rather than inflicting the work on VcsSupport authors. Some
>> VCS (such as MKS) do not support collecting build changes or even project
>> resynchronization by date which makes it impossible to use with
>> TeamCity.
>>
>> 3) Do away with getContent(). Not all VCS correctly support fetching
>> content based on dates. TeamCity should be intelligent enough to track
>> changes itself, and look at the filesystem for the most current revision
>> (or issue a checkout).
>>
>> 4) Documentation! I have been able to figure out how VcsSupport works in
>> the wild only through bits from forum posts, decompilations, and lots of
>> trial by fire. Please document these API's in a public javadoc rather
>> than distributing a source .jar!
>>
>>
>> Regards,
>>
>> Steven Stallion
>



0
Comment actions Permalink

Olesya,

The biggest hurdle is having to do collectBuildChanges() and buildPatch() between two project versions. MKS does not support practical project versioning during development, so dates are the only real option (like CVS). Unfortunately dates arent supported in very many operations. I might be able to work around this, but I have a question:

When a call to collectBuildChanges() and buildPatch() are made, would there be a problem in just grabbing the most recent copy of the repository? Will collectBuildChanges() or buildPatch() ever be called for a version other than the current?

If it is possible to get away with that, I could have the MKS plugin complete within a couple of days and fully functioning.

Thanks again!

Steve

0
Comment actions Permalink

>Will collectBuildChanges() or buildPatch() ever be called for a version
>other than the current?

At the moment they are called for current version, but not for version which
is current for the vcs repository but for last returned current version.
They can differ.

We're trying to pay special attention to finding person who broke the build.
That is why it would be better if changes in the build are calculated
correctly.
If you send the latest sources they can contain changes haven't been
collected yet. For example they were made just before the build start.

Also it is necessary for building history builds (you want to build
something not with latest version but with version which has been used for
building some specified build). But this feature is planned only...

--
Olesya Smirnova
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"


"Steven Stallion" <no_reply@jetbrains.com> wrote in message
news:33115110.1162993347219.JavaMail.itn@is.intellij.net...

Olesya,

>

The biggest hurdle is having to do collectBuildChanges() and buildPatch()
between two project versions. MKS doesnt support project versions, so
dates are the only real option (like CVS). Unfortunately dates arent
supported in very many operations. I might be able to work around this,
but I have a question:

>

When a call to collectBuildChanges() and buildPatch() are made, would
there be a problem in just grabbing the most recent copy of the
repository? Will collectBuildChanges() or buildPatch() ever be called for
a version other than the current?

>

Thanks!

>

Steve



0
Comment actions Permalink

Olesya,

Thanks for the clarification.

The best I can do, is to perform a resync on a sandbox (checkout the latest copy of the repository). When this is done, I can pull back the change type (add/delete/update), and the file type (archive/binary). I can also return the revision that was checked out and changelist information.

Is there a way to make this work with TeamCity?

Steve

0
Comment actions Permalink

>I can also return the revision that was checked out and changelist
>information.
So, can you provide information about change has been made? Actually you can
send all sources instead of patch so there is no problem with finding
differences. It will take more time but it will work.

The best I can do, is to perform a resync on a sandbox (checkout the
latest copy of the repository).

Does your vcs provide some information about change during update? As an
output for example.
If you can collect this information only during update it is also possible
to save it and return from inside "collectChanges".

--
Olesya Smirnova
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"


"Steven Stallion" <no_reply@jetbrains.com> wrote in message
news:23133435.1163077790411.JavaMail.itn@is.intellij.net...

Olesya,

>

Thanks for the clarification.

>

The best I can do, is to perform a resync on a sandbox (checkout the
latest copy of the repository). When this is done, I can pull back the
change type (add/delete/update), and the file type (archive/binary). I can
also return the revision that was checked out and changelist information.

>

Is there a way to make this work with TeamCity?

>

Steve



0
Comment actions Permalink

So, can you provide information about change has been
made? Actually you can
send all sources instead of patch so there is no
problem with finding
differences. It will take more time but it will work.


How would this impact looking at changes between builds? Does this mean that each user will have Build Changes?

Does your vcs provide some information about change
during update? As an
output for example.
If you can collect this information only during
update it is also possible
to save it and return from inside "collectChanges".


It does, but it is only for the current version of the sandbox. With mks, it is impossible to tell the difference between projects for a given time interval correctly.

0
Comment actions Permalink

How would this impact looking at changes between builds? Does this mean
that each user will have Build Changes?

Sorry, I'm not sure I do understand your question.

I want to suggest a way how can we match our api and your vcs capabilities.
I'll describe what TeamCity needs from a vcs support and I want you to
imaginate what api can you provide.

I'll add something like getCapabilities to VcsSupport and you will be able
do not provide file content or sources for an old build if it is impossible
for your vcs (at least we can zip project files before build and store the
zip as an build artifact to support building some "history" builds).

1) VcsSupport has to provide changes since last collectChanges call (using 2
version were convenient, but we can change it.)
2) VcsSupport has to build patch. At the moment last patch version is not
current vcs version, but in you can we can support "building 1 patch in a
time", so the second version will be always the current one. You can send
the whole project sources so there will not necessity to find changes files
between current version and some other version.

So actually the main api can be like
collectChanges(vcsroot) - changes between last detected and current version
sendSources (vcsroot, patchbuilder) - send all project files
getCapabilities

Or (just for safety) we can split VcsSupport into several interfaces and
other capabilities will be implemented with different ones. So if you cannot
support some capability you can do not implement the corresponding
interface.

What do you think?

--
Olesya Smirnova
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"


"Steven Stallion" <no_reply@jetbrains.com> wrote in message
news:457498.1163276143023.JavaMail.itn@is.intellij.net...
>> So, can you provide information about change has been
>> made? Actually you can
>> send all sources instead of patch so there is no
>> problem with finding
>> differences. It will take more time but it will work.
>

How would this impact looking at changes between builds? Does this mean
that each user will have Build Changes?

>
>> Does your vcs provide some information about change
>> during update? As an
>> output for example.
>> If you can collect this information only during
>> update it is also possible
>> to save it and return from inside "collectChanges".
>

It does, but it is only for the current version of the sandbox. With mks,
it is impossible to tell the difference between projects for a given time
interval correctly.



0
Comment actions Permalink

Olesya,

I think that would work out perfectly. Thanks for all of your hardwork on this!

0
Comment actions Permalink

Oleysa,

Has there been any progress made to change VcsSupport since release 1.1 ?

Thanks!

Steve

0
Comment actions Permalink

Hi Steven,
unfortunatelly I cannot change such global changes in the next release
(which is being planned to be released this spring).

What I can do right now is I can try to create some helper which will find
changed files for the directory (build a patch and collect changes).
Could you please specify if it is enougth for your vcs support?



--
Olesya Smirnova
JetBrains, Inc
http://www.jetbrains.com
"Develop with pleasure!"


"Steven Stallion" <no_reply@jetbrains.com> wrote in message
news:26794672.1169230598517.JavaMail.itn@is.intellij.net...

Oleysa,

>

Has there been any progress made to change VcsSupport since release 1.1 ?

>

Thanks!

>

Steve



0
Comment actions Permalink

It should as long as I do not have to checkout changes between a given time (MKS does not support this).

Thanks!

Steve

0

Please sign in to leave a comment.