Hello List,
I am very new to Lisp and come from a python background. The very
dynamic nature of Lisp fascinates me but I wonder how people handle
Team work and Version Control in Lisp.

If I understand things correctly you have just 1 running image. What is
to do if there are several programmers ? How can incoherent changes be
prevented ?

One possible solution is 1 'Production' image, several 'QA' and
'Development' Images. If that is the way to go, how are changes
documented and transferred between the images ?

In older threads in c.l.l. I read about the Version Control Systems
people use, but I wonder what exactly makes up a file which these tools
can control.

If I change a Function definition dynamically, what belongs in the file
which shall be put under the control of the SCM system ? What is the
source code if I delete a function ? I feel that there is a mismatch
between the dynamic nature of Lisp and the concept of source code in
files, isn't it ?

Thanks for listening !

Norbert Klamann

From:	Andreas Thiele
Subject:	Re: Version Control and Team Work in Lisp
Date:	Wed, 19 Jan 2005 12:50:12 +0100

schrieb im Newsbeitrag
news:1106122685.781525.65050@z14g2000cwz.googlegroups.com...
....
> If I understand things correctly you have just 1 running image. What is
> to do if there are several programmers ? How can incoherent changes be
> prevented ?
....
>

I think you are considering a special case, something like a web or
application server, a case were you only have one running instance of an
application. If you consider a stand alone application, in this respect
nothing is different from any other language. You have sources, compile
sources and you might create executables (depending on your implementation).

Lispers are not centered around images. This is like dumping a database and
later restoring the dump. Possible, but not the 'standard' way of working.

Even in a running application, most people will change their sources and
're-load' part of the sources or compiled sources into the running process.
So you might use something like CVS.

It seems you are assuming a dozen developers working on a 1,000,000 lines of
code application. This is not typical in the lisp world. Lisp teams are
rather small and code is rather compact if compared to typical imperative
languages. So I'd expect about 3 lispers with perhaps 50,000 lines doing the
same (my very personal estimation :-)) ).

Andreas

From:	Paolo Amoroso
Subject:	Re: Version Control and Team Work in Lisp
Date:	Wed, 19 Jan 2005 14:00:23 +0100

"Andreas Thiele" writes:

> Lispers are not centered around images. This is like dumping a database and

Smalltalk development is more image-centered.

Paolo
--
Why Lisp? http://alu.cliki.net/RtL%20Highlight%20Film
Recommended Common Lisp libraries/tools (see also http://clrfi.alu.org):
- ASDF/ASDF-INSTALL: system building/installation
- CL-PPCRE: regular expressions
- UFFI: Foreign Function Interface

From:	Pascal Bourguignon
Subject:	Re: Version Control and Team Work in Lisp
Date:	19 Jan 2005 09:43:49 +0100

Norbert.Klamann@klamann-software.de writes:

> Hello List,
> I am very new to Lisp and come from a python background. The very
> dynamic nature of Lisp fascinates me but I wonder how people handle
> Team work and Version Control in Lisp.

Exactly like in other languages, because we keep the sources in source files.

> If I understand things correctly you have just 1 running image. What is
> to do if there are several programmers ? How can incoherent changes be
> prevented ?
>
> One possible solution is 1 'Production' image, several 'QA' and
> 'Development' Images. If that is the way to go, how are changes
> documented and transferred between the images ?
>
> In older threads in c.l.l. I read about the Version Control Systems
> people use, but I wonder what exactly makes up a file which these tools
> can control.
>
> If I change a Function definition dynamically, what belongs in the file
> which shall be put under the control of the SCM system ? What is the
> source code if I delete a function ? I feel that there is a mismatch
> between the dynamic nature of Lisp and the concept of source code in
> files, isn't it ?
>
> Thanks for listening !

Images are implementation, target and version specific. Therefore
it's not too practical to keep important state stored only in an
image. I use images only to save the current session, as a
checkpoint, when I have to quit the lisp or reboot the hardware.

--
__Pascal_Bourguignon__ _ Software patents are endangering
() ASCII ribbon against html email (o_ the computer industry all around
/\ 1962:DO20I=1.100 //\ the world http://lpf.ai.mit.edu/
2001:my($f)=`fortune`; V_/ http://petition.eurolinux.org/

From:	Marcin 'Qrczak' Kowalczyk
Subject:	Re: Version Control and Team Work in Lisp
Date:	Sun, 23 Jan 2005 14:28:49 +0100

From:	Paolo Amoroso
Subject:	Re: Version Control and Team Work in Lisp
Date:	Wed, 19 Jan 2005 18:54:38 +0100

William Bland writes:

> Every source control system I've ever seen uses a line-based diff
> algorithm. Has anyone over made one that understands ps and can do
> better diffs on Lisp code?

See these tools at the CMU Common Lisp Repository:

Programs for comparing versions of source code
ftp://ftp.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/lisp/code/tools/src_cmp/0.html

Paolo
--
Lisp Propulsion Laboratory log - http://www.paoloamoroso.it/log
Recommended Common Lisp libraries/tools (see also http://clrfi.alu.org):
- ASDF/ASDF-INSTALL: system building/installation
- CL-PPCRE: regular expressions
- UFFI: Foreign Function Interface

From:	Christopher C. Stacy
Subject:	Re: Version Control and Team Work in Lisp
Date:	Wed, 19 Jan 2005 16:24:10 GMT

Pascal Bourguignon writes:

> Norbert.Klamann@klamann-software.de writes:
>
> > Hello List,
> > I am very new to Lisp and come from a python background. The very
> > dynamic nature of Lisp fascinates me but I wonder how people handle
> > Team work and Version Control in Lisp.
>
> Exactly like in other languages, because we keep the sources in source files.
>

> Images are implementation, target and version specific. Therefore
> it's not too practical to keep important state stored only in an
> image. I use images only to save the current session, as a
> checkpoint, when I have to quit the lisp or reboot the hardware.

On the Lisp Machine, we had a pretty fancy patch-version-control
system (not to be confused with a source control system, nor with
the file versioning system) that was integrated with DEFSYSTEM.
Images were built and delivered to serve as the base against which
patches (FASL+source) would be applied (when the machine was booted
or logged into, and thereafter in real time under user control).
This is how things were done both internally by the vendor's own
system developers, and by customers.

If everybody is using the same Lisp implementation, I don't
see what the problem would be in doing the same thing on
conventional computers. After all, that's how the vendor
shipped their Lisp implementation to you in the first place.
Each image is probably going to be a 12-20 MB binary, of course.
The patches to those base images will be numerous small source/binary
files in the familiar fashion from other languages.

Periodically, a new base image is copiled and built from a
snapshot/release of the source files, which are checked into
the source control system. There could be computed data sets
stored in the image which do not correspond to program source files.

From:	William Bland
Subject:	Re: Version Control and Team Work in Lisp
Date:	Wed, 19 Jan 2005 17:07:06 GMT

On Wed, 19 Jan 2005 09:43:49 +0100, Pascal Bourguignon wrote:

> Norbert.Klamann@klamann-software.de writes:
>
>> Hello List,
>> I am very new to Lisp and come from a python background. The very
>> dynamic nature of Lisp fascinates me but I wonder how people handle
>> Team work and Version Control in Lisp.
>
> Exactly like in other languages, because we keep the sources in source files.

Every source control system I've ever seen uses a line-based diff
algorithm. Has anyone over made one that understands ps and can do
better diffs on Lisp code?

Cheers,
Bill.

From:	Cameron MacKinnon
Subject:	Re: Version Control and Team Work in Lisp
Date:	Wed, 19 Jan 2005 13:09:15 -0500

William Bland wrote:
> Every source control system I've ever seen uses a line-based diff
> algorithm. Has anyone over made one that understands ps and can do
> better diffs on Lisp code?

Better? If you mean that the diff files are smaller, then I've got two
thoughts on that. Using a text compression algorithm on the output of
traditional diff will likely get the file size down to as small or
smaller than an optimal pr-diff without compression.

Also, if it is storage space you're worried about, wasted residual space
in the last disk block is likely to dominate for typical short diffs,
unless the diffs are packed into a database or filesystem that doesn't
waste partial blocks.

Consider also that traditional line based diffs, with a bit of context,
are fairly easy to read by humans. This is a feature that another system
may not have.

I suspect that minimal difference algorithms for trees exist. Sexprs are
trees, right? Sounds like a fun little hack.

Of course, you could just write a filter to put a newline between each
atom and each pair of opening or closing parentheses, run THAT through
diff... :-)

From:	William Bland
Subject:	Re: Version Control and Team Work in Lisp
Date:	Wed, 19 Jan 2005 18:22:49 GMT

On Wed, 19 Jan 2005 13:09:15 -0500, Cameron MacKinnon wrote:

> William Bland wrote:
>> Every source control system I've ever seen uses a line-based diff
>> algorithm. Has anyone over made one that understands ps and can do
>> better diffs on Lisp code?
>
> Better? If you mean that the diff files are smaller, then I've got two
> thoughts on that. Using a text compression algorithm on the output of
> traditional diff will likely get the file size down to as small or
> smaller than an optimal pr-diff without compression.

[snip...]

Hmm. This is something that I'm having to drum into the heads of some of
the people I'm working with at the moment. Diffs don't need to be small
for machine-related reasons. They need to be small so that, when I'm
ploughing through them (e.g. to find causes of merging conflicts, or to
find out exactly what happened between two versions to cause something to
stop working), I don't have to read pages and pages of crap.

I think Lisp probably wins over other languages already in this regard
though, by having a very widely accepted format for code, so it's unlikely
there would be much code in a diff that differs only in whitespace for
example.

Cheers,
Bill.

From:	Trent Buck
Subject:	Re: Version Control and Team Work in Lisp
Date:	Thu, 20 Jan 2005 13:02:07 GMT

Up spake William Bland:
> ...so it's unlikely
> there would be much code in a diff that differs only in whitespace for
> example.

I assume you and your cow-orkers know about the (probably GNU only)
--ignore-all-space option?

--
-trent
The problem with America is stupidity. I'm not saying we
should kill idiots, but why not just take the safety labels
off everything and let the problem solve itself?

From:	Adrian Kubala
Subject:	Re: Version Control and Team Work in Lisp
Date:	Thu, 20 Jan 2005 13:20:02 -0600

Trent Buck schrieb:
> Up spake William Bland:
>> ...so it's unlikely
>> there would be much code in a diff that differs only in whitespace for
>> example.
>
> I assume you and your cow-orkers know about the (probably GNU only)
> --ignore-all-space option?

I don't think this is the issue so much as the fact that diff will group
changes in weird ways, so that for example the end of one function and
the start of a new one are grouped as one change, or the ending bracket
of one function becomes the ending bracket of a different function. I
guess I've never had obvious problems because of this but it makes
intuitive sense that a tree-based diff could be smarter about complex
merges because it could localize changes better.

Not to mention you could do some pretty intelligent patches. DARCS for
example can express a simple token-replace as a patch, and knows to
apply the token replace to new code coming from merges -- p- instead
of token-based patches could potentially do even more interesting
refactorings.

From:	Ivan Boldyrev
Subject:	Re: Version Control and Team Work in Lisp
Date:	Thu, 20 Jan 2005 18:09:17 +0600

On 8994 day of my life William Bland wrote:
> Every source control system I've ever seen uses a line-based diff
> algorithm. Has anyone over made one that understands ps and can do
> better diffs on Lisp code?

GNU tla can use any external diff/merge tool.

--
Ivan Boldyrev

Sorry my terrible English, my native language is Lisp!

From:	Kaz Kylheku
Subject:	Re: Version Control and Team Work in Lisp
Date:	19 Jan 2005 11:11:12 -0800

Norbert.Klamann@klamann-software.de wrote:
> Hello List,
> I am very new to Lisp and come from a python background. The very
> dynamic nature of Lisp fascinates me but I wonder how people handle
> Team work and Version Control in Lisp.
>
> If I understand things correctly you have just 1 running image. What
is
> to do if there are several programmers ? How can incoherent changes
be
> prevented ?

A running image is the basic unit of deployment of a Lisp application.
There is no requirement that developers must use a single shared image
for actually writing, debugging and testing code.

What is a Lisp image? It's the same thing as any other kind of program
image.

If you have a machine that is programmed from scratch in Lisp, then it
may be the case that the entire operating system and all the
applications running in it are one image: exactly as would be the case
in a system written in assembly language and C, without memory
protection.

Or a Lisp image may be found in the form of a process underneath an
operating system which virtualizes the machine and protects processes
from each other. Lisp is commonly run this way now on unixes.

> One possible solution is 1 'Production' image, several 'QA' and
> 'Development' Images. If that is the way to go, how are changes
> documented and transferred between the images ?

You check your sources into the version control system, in the normal
way.

> In older threads in c.l.l. I read about the Version Control Systems
> people use, but I wonder what exactly makes up a file which these
tools
> can control.
>
> If I change a Function definition dynamically, what belongs in the
file
> which shall be put under the control of the SCM system ?

If you want to maintain a non-stop running image of your system, that
is certainly possible. What happens is that you develop changes in the
normal way on your development machine and test them in your own
private image. When they are good, you push them into the running
image.

> What is the
> source code if I delete a function ? I feel that there is a mismatch
> between the dynamic nature of Lisp and the concept of source code in
> files, isn't it ?

If you delete a function, you edit it out of your source code. If you
compile the resulting module and push it into the running image, the
existing function in the image does not disappear. Indeed, it must not,
because there could still be threads running over it.

You could get into the image and explicitly dissociate the function
from its symbol. Then when the last reference to the actual function
object is dropped (no variables refer to it and no threads execute its
code), it will become garbage, subject to collection.

Otherwise if you delete or rename functions, your running image can end
up with these unused functions that are still there, but nobody calls
them. That isn't a big deal.

Of course, any new image you generate from the sources won't have those
functions.

Your configuration management is based on tagged source code revisions.
If you want to be absolutely clean about your correspondence between
image and source code baseline, then you have to regenerate the image
for each new baseline and re-start everything from scratch. If you
incrementally update a running image with new binary code, then things
are a little more wobbly. A given current configuration may have
left-over material in it from a previous configuration. That could give
rise to bugs whereby there are left-over dependencies on the old stuff
(such that a cleanly generated image *won't* work due to these dangling
dependencies, but the running one happens to work because the left-over
material satisfies those dependencies).