Actions

Difference between revisions of "Contributing Patches to Gambit Source Code"

From Gambit wiki

m (Copied from old wiki)
 
(use `git format-patch` (one might want to also use `git send-email`))
 
(8 intermediate revisions by 4 users not shown)
Line 1: Line 1:
 
__NOTOC__ __NOEDITSECTION__
 
__NOTOC__ __NOEDITSECTION__
 
The Gambit [[Source code repository | source code repository]] is
 
The Gambit [[Source code repository | source code repository]] is
managed with the [http://www.selenic.com/mercurial/wiki/ Mercurial]
+
managed with the [http://git.or.cz/ git]
 
distributed version control system.
 
distributed version control system.
  
Line 7: Line 7:
 
copies can be edited independently using any tool (editors, utility
 
copies can be edited independently using any tool (editors, utility
 
programs, etc).  When a developer completes making a set of related
 
programs, etc).  When a developer completes making a set of related
changes they are recorded as a ''patch'' (also called a
+
changes they are recorded as a ''commit''.  The developer can submit
''changeset'').  The developer can then contribute the patch by
+
these changes to the Gambit maintainers as a ''patch''.  If the patch is accepted
submitting it to the Gambit maintainers.  If the patch is accepted
 
 
then it will be applied to the repository by the Gambit maintainers
 
then it will be applied to the repository by the Gambit maintainers
 
and will become part of the central repository.
 
and will become part of the central repository.
  
This page only gives the main Mercurial commands that are needed
+
This page only gives the main procedures and git commands that are needed
 
to manage the local copy of the central repository.
 
to manage the local copy of the central repository.
Please read the [http://hgbook.red-bean.com/ Mercurial manual] for
+
Please read the [http://www.kernel.org/pub/software/scm/git/docs/gittutorial.html git tutorial] for
detailed usage instructions.
+
typical usage instructions.
  
==== Installing Mercurial ====
+
==== Installing git ====
  
Prebuilt distributions of Mercurial and a source code distribution are
+
Git was created for the development of the Linux kernel, and
available
+
is sometimes preinstalled on Linux distributions.
[http://www.selenic.com/mercurial/wiki/index.cgi/BinaryPackages here].
+
Prebuilt distributions of git and a source code distribution are
 +
available [http://git.or.cz/#download here].
 
Please follow the installation instructions on the
 
Please follow the installation instructions on the
[http://www.selenic.com/mercurial/wiki/ Mercurial web site].
+
[http://git.or.cz/ git web site].
Note that version 0.9.1 is not recent enough; but 0.9.3 and 0.9.4
 
should work!
 
  
To simplify usage of Mercurial the configuration file '''$HOME/.hgrc'''
+
To simplify usage of git the git configuration file '''$HOME/.gitconfig'''
 
should be created and contain at least your username.  There are
 
should be created and contain at least your username.  There are
 
many settings that can be specified
 
many settings that can be specified
 
such as the editor to use for entering the comment attached
 
such as the editor to use for entering the comment attached
to a patch.  For example, to use emacs:
+
to a commit.  For example, to use emacs:
  
 
<pre>
 
<pre>
# Mercurial configuration file.
+
[user]
[ui]
+
        name = Marc Feeley
username = John Doe <jdoe@acme.com>
+
        email = feeley@iro.umontreal.ca
editor = emacs -nw
+
[core]
 +
        editor = emacs -nw
 +
[color]
 +
        ui = auto
 
</pre>
 
</pre>
 
If you want to use emacs' '''ediff''' to merge conflicting changes follow
 
[http://www.selenic.com/mercurial/wiki/index.cgi/MergingWithEmacs these instructions].
 
  
 
==== Obtaining a local copy of the Gambit source code ====
 
==== Obtaining a local copy of the Gambit source code ====
  
 
The best way to obtain a local copy of the Gambit source code is to
 
The best way to obtain a local copy of the Gambit source code is to
download and unpack a recent source code release.
+
download and unpack a recent source code release.  You need
Normally the most recent source code release should be downloaded.
+
the '''developer''' version, which has a '''-devel''' in the tarball name.
 +
Normally the most recent source code release should be downloaded
 +
and you need at least v4.3.2 (the first version to support git).
 +
After that you can use the '''make update''' command to
 +
update your local copy with all the changes committed to
 +
the central source code repository.
 +
 
 +
Get the most recent tarball from the
 +
[http://dynamo.iro.umontreal.ca/~gambit/wiki/index.php/Main_Page main Gambit page].
  
 
<pre>
 
<pre>
wget http://www.iro.umontreal.ca/~gambit/download/gambit/v4.0/source/gambc-v4_0_0.tgz
+
wget http://www.iro.umontreal.ca/~gambit/download/gambit/v4.3/source/gambc-v4_3_2-devel.tgz
tar zxf gambc-v4_0_0.tgz
+
tar zxf gambc-v4_3_2-devel.tgz
mv gambc-v4_0_0 gambit
+
mv gambc-v4_3_2-devel gambit
 
</pre>
 
</pre>
  
Line 65: Line 71:
 
<pre>
 
<pre>
 
cd gambit
 
cd gambit
./configure
+
./configure --enable-single-host
 
make bootstrap
 
make bootstrap
 
</pre>
 
</pre>
Line 75: Line 81:
 
==== Synchronizing with the central source code repository ====
 
==== Synchronizing with the central source code repository ====
  
The Mercurial '''pull''' command gets the local copy in synch with the
+
The local source code files can then be updated to contain the latest commits
central source code repository.
+
on the central source code repository with the command:
  
 
<pre>
 
<pre>
hg pull
+
make update
 
</pre>
 
</pre>
  
The local source code files can then be updated to contain the latest source code
+
This will call the '''git fetch''' and '''git checkout''' commands in a loop, so that the Gambit compiler
with the Mercurial '''update''' command.
+
for each release between the local copy's release and the most recent release are
 +
built in succession. This is necessary because the Gambit system is bootstrapped
 +
with itself, and to compile the runtime system for a specific version the Gambit
 +
compiler for that version must be used.  If for some reason you need to do this
 +
manually the following procedure should be used.  Assuming the local copy is currently
 +
at release v1.2.3 and that v1.2.5 is the most recent release, do this:
  
 
<pre>
 
<pre>
hg update
+
git checkout v1.2.4-bootstrap
 +
make bootstrap
 +
git checkout v1.2.4
 +
make bootclean bootstrap
 +
git checkout v1.2.5-bootstrap
 +
make bootstrap
 +
git checkout v1.2.5
 +
make bootclean bootstrap
 +
git checkout master
 +
make bootstrap
 
</pre>
 
</pre>
  
Line 93: Line 113:
  
 
The local source code files can be updated to a specific version
 
The local source code files can be updated to a specific version
with the '''-r''' option to the Mercurial '''update''' command.
+
with the '''git checkout''' command.
  
 
<pre>
 
<pre>
hg update -r v4.0.0
+
git checkout v4.2.1
 
</pre>
 
</pre>
  
==== Creating a patch ====
+
After this command it is not possible to build Gambit with
 +
a '''make''' because the local '''gsc-comp''' is not the appropriate
 +
version to compile the runtime system.  One way around this problem
 +
is to download the release for that version in a different directory, do a
 +
'''make bootstrap''' in that directory, and then copy '''gsc-comp'''
 +
back here.
 +
 
 +
==== Creating a commit ====
 +
 
 +
A source code change may simply be modifications of existing
 +
files, deletions, and addition of new files.
 +
If a new file has been created git should be informed by entering:
 +
 
 +
<pre>
 +
git add newfile
 +
</pre>
  
After making a set of related changes to the local copy of the
+
After making a set of related changes it is customary to review which files have changed
repository, a patch is created with the command:
+
by executing the command:
  
 
<pre>
 
<pre>
hg commit
+
make status
 
</pre>
 
</pre>
  
This command will pop up an editor to compose a comment describing the nature of the patch.
+
If all checks out, a commit is created with the command:
Please use a short but descriptive comment.
 
You can save some typing by adding the comment on the command line:
 
  
 
<pre>
 
<pre>
hg commit -m "Fixed bug in ,b command"
+
make commit
 
</pre>
 
</pre>
  
To get a list of the patches (starting with the latest, i.e. the '''tip''') use the '''log''' command:
+
This command will pop up an editor to compose a comment describing the nature of the patch.
 +
Please use a short but descriptive comment on the first line of text entered.
 +
 
 +
To get a list of the commits (starting with the latest, i.e. the '''HEAD''') use:
  
 
<pre>
 
<pre>
hg log
+
make log
 
</pre>
 
</pre>
  
 
==== Contributing a patch ====
 
==== Contributing a patch ====
  
A patch file can be created using the '''export''' command.  To include only the
+
A patch file can be created using the '''git diff''' command.  To include only the
most recent patch use:
+
most recent commit use:
  
 
<pre>
 
<pre>
hg export tip > my-patch
+
git format-patch -1
 
</pre>
 
</pre>
  
A range of patches can be included like this:
+
A range of patches can be specified like this:
  
 
<pre>
 
<pre>
hg export 4:6 > my-patch
+
git format-patch -4 HEAD~3
 
</pre>
 
</pre>
  
The patch file can then be submitted to the Gambit maintainers
+
The patch file(s) can then be submitted to the Gambit maintainers
 
by sending the file by email to '''gambit@iro.umontreal.ca'''.  For example:
 
by sending the file by email to '''gambit@iro.umontreal.ca'''.  For example:
  
Line 147: Line 183:
  
 
This section contains some notes for the Gambit maintainers.  It explains the steps to follow to release a new version of Gambit.  '''The steps must be followed carefully or you may end up with a wedged system!'''
 
This section contains some notes for the Gambit maintainers.  It explains the steps to follow to release a new version of Gambit.  '''The steps must be followed carefully or you may end up with a wedged system!'''
 +
 +
===== Manual procedure =====
  
 
Assume the latest release of Gambit is v1.2.3 and the new release will be v4.5.6.
 
Assume the latest release of Gambit is v1.2.3 and the new release will be v4.5.6.
Line 153: Line 191:
 
Then the '''gambit.h''' file, '''configure.ac''' file, and other files with an embedded version number must be updated so they refer to version v4.5.6.
 
Then the '''gambit.h''' file, '''configure.ac''' file, and other files with an embedded version number must be updated so they refer to version v4.5.6.
  
===== 1. Creating a Gambit compiler for version v4.5.6 =====
+
'''1. Creating a Gambit compiler for version v4.5.6'''
  
 
Make sure you have a working bootstrap compiler ('''gsc-comp''') and save a copy just in case:
 
Make sure you have a working bootstrap compiler ('''gsc-comp''') and save a copy just in case:
Line 173: Line 211:
 
<pre>
 
<pre>
 
make bootstrap
 
make bootstrap
hg commit -m "[COMPILER CHANGES NEEDED FOR v4.5.6] Changed version in compiler"
+
git commit -a -m "[COMPILER CHANGES NEEDED FOR v4.5.6] Changed version in compiler"
 +
git tag v4.5.6-bootstrap
 
</pre>
 
</pre>
  
 
Note that at this very point the new compiler will not pass the tests because the runtime still expects the old version number.
 
Note that at this very point the new compiler will not pass the tests because the runtime still expects the old version number.
  
===== 2. Upgrading the runtime files from version v1.2.3 to v4.5.6 =====
+
'''2. Upgrading the runtime files from version v1.2.3 to v4.5.6'''
  
 
For the C files generated from Scheme files the version numbers can easily be upgraded by compiling them from scratch using the new Gambit compiler.
 
For the C files generated from Scheme files the version numbers can easily be upgraded by compiling them from scratch using the new Gambit compiler.
This will be done automatically by a '''make''' after a '''make clean'''.  The other changes to the files are made by running the script '''misc/changev''' (this must be done first because it changes the file '''configure.ac''' which produces the '''configure''' script which must be run again).
+
This will be done automatically by a '''make''' after a '''make bootclean'''.  The other changes to the files are made by running the script '''misc/changev''' (this must be done first because it changes the file '''configure.ac''' which produces the '''configure''' script which must be run again).
  
 
<pre>
 
<pre>
 
misc/changev 102003 405006
 
misc/changev 102003 405006
make clean
+
make bootclean
 
./configure
 
./configure
 
make
 
make
Line 191: Line 230:
 
mv tests/mix.c tests/test5.ok
 
mv tests/mix.c tests/test5.ok
 
make check  # this time all tests should pass
 
make check  # this time all tests should pass
hg commit -m "[RUNTIME CHANGES NEEDED FOR v4.5.6] Changed version of runtime using misc/changev"
+
git commit -a -m "[RUNTIME CHANGES NEEDED FOR v4.5.6] Changed version of runtime using misc/changev"
hg tag v4.5.6
+
git tag v4.5.6
hg push
 
 
</pre>
 
</pre>
  
===== 3. Creating prebuilt installers for version v4.5.6 =====
+
'''3. Creating prebuilt installers for version v4.5.6'''
  
 
To create prebuilt installers for Mac OS and Windows you need a Mac with Xcode and Parallels workstation with Windows and the MinGW and Microsoft Visual C Express environments.  The following command will build all variants of the installers (it takes about 2 hours in total):
 
To create prebuilt installers for Mac OS and Windows you need a Mac with Xcode and Parallels workstation with Windows and the MinGW and Microsoft Visual C Express environments.  The following command will build all variants of the installers (it takes about 2 hours in total):
Line 211: Line 249:
  
 
This will automatically send an announcement on the Gambit mailing list.  You must manually update the main page of the Gambit Wiki so that it refers to the latest version.
 
This will automatically send an announcement on the Gambit mailing list.  You must manually update the main page of the Gambit Wiki so that it refers to the latest version.
 +
 +
===== Automated procedure =====
 +
 +
Step 1 and 2 in the above instructions are executed with the '''change-version''' make target:
 +
 +
<pre>
 +
make NEW_VERSION=v4.5.6 change-version
 +
</pre>
 +
 +
Alternatively, the '''new-revision''', '''new-minor''', or '''new-major''' make targets can be used
 +
to compute the new version number from the current version by incrementing one
 +
of the version number fields:
 +
 +
<pre>
 +
make new-revision
 +
</pre>
 +
 +
==== Troubleshooting ====
 +
 +
A user was having trouble using trouble building an up-to-date version of Gambit starting from v4.3.2.  '''make update''' was failing.  Marc Feeley responded with an explanation and a solution.
 +
 +
The solution is to download the latest version of Gambit and use that to run the '''make update''' in the git repository.
 +
 +
Here's the [http://article.gmane.org/gmane.lisp.scheme.gambit/4399 reply]:
 +
 +
<blockquote>
 +
Let me explain the problem.  If you're only interested in a solution: get v4.6.0 and a make update from that.
 +
</blockquote>
 +
 +
<blockquote>
 +
Gambit is a self-hosting compiler.  That means that it is bootstrapped using itself.  Most of the system is
 +
implemented in Scheme (not just the libraries, but the compiler itself).  So when a change is made in the
 +
compiler's source code, it must be compiled with the current version of the compiler.  Roughly speaking,
 +
version 2 must be compiled with version 1.  Then when changes are made (to get the source code of version 3),
 +
then version 3 must be compiled with version 2.  And so on.  When you "make update" the build process executes
 +
the chain of self compilations that led to the most recent version of Gambit.  In other words, if the most
 +
recent Gambit is v4 and you currently have v1 installed, then the "make update" will: compile the v2 source
 +
with the v1 executable to get the v2 executable, then it will compile the v3 source with the v2 executable to
 +
get the v3 executable, and finally compile the v4 source with the v3 executable to get the v4 executable.
 +
This chain must be followed because features that are used in the source code of the most recent version of
 +
the compiler may have appeared (i.e. were implemented) in an intermediate version of the compiler.
 +
</blockquote>
 +
 +
<blockquote>
 +
This self dependency is not easy to handle in the build process and makefiles, and currently there is a bug.
 +
In your particular case, Gambit v4.3.2 does not support the same command line options as more recent
 +
version of Gambit.  So somewhere in this chain of commits, the makefiles were not consistent with the then
 +
current compiler.  But because the commits can't be "taken back" the self compilation sequence that "make
 +
update" executes has some version transitions that will forever be broken.  I haven't looked into ways
 +
(automatic tool, or better version changing habits) to avoid this problem in the future.
 +
</blockquote>
 +
 +
[[Category: Development]]

Latest revision as of 19:36, 11 July 2010

The Gambit source code repository is managed with the git distributed version control system.

Each developer uses a local copy of the central repository. The copies can be edited independently using any tool (editors, utility programs, etc). When a developer completes making a set of related changes they are recorded as a commit. The developer can submit these changes to the Gambit maintainers as a patch. If the patch is accepted then it will be applied to the repository by the Gambit maintainers and will become part of the central repository.

This page only gives the main procedures and git commands that are needed to manage the local copy of the central repository. Please read the git tutorial for typical usage instructions.

Installing git

Git was created for the development of the Linux kernel, and is sometimes preinstalled on Linux distributions. Prebuilt distributions of git and a source code distribution are available here. Please follow the installation instructions on the git web site.

To simplify usage of git the git configuration file $HOME/.gitconfig should be created and contain at least your username. There are many settings that can be specified such as the editor to use for entering the comment attached to a commit. For example, to use emacs:

[user]
        name = Marc Feeley
        email = feeley@iro.umontreal.ca
[core]
        editor = emacs -nw
[color]
        ui = auto

Obtaining a local copy of the Gambit source code

The best way to obtain a local copy of the Gambit source code is to download and unpack a recent source code release. You need the developer version, which has a -devel in the tarball name. Normally the most recent source code release should be downloaded and you need at least v4.3.2 (the first version to support git). After that you can use the make update command to update your local copy with all the changes committed to the central source code repository.

Get the most recent tarball from the main Gambit page.

wget http://www.iro.umontreal.ca/~gambit/download/gambit/v4.3/source/gambc-v4_3_2-devel.tgz
tar zxf gambc-v4_3_2-devel.tgz
mv gambc-v4_3_2-devel gambit

Note that in anticipation of making changes to the source code the directory has been renamed to gambit to avoid thinking it is a pristine copy of a particular release.

The system should then be prepared for modifications of the Scheme source code by creating a compiler for bootstrapping:

cd gambit
./configure --enable-single-host
make bootstrap

This will create the gsc-comp compiler, which is executed when a Scheme source code file in the Gambit system must be compiled following a change.

Synchronizing with the central source code repository

The local source code files can then be updated to contain the latest commits on the central source code repository with the command:

make update

This will call the git fetch and git checkout commands in a loop, so that the Gambit compiler for each release between the local copy's release and the most recent release are built in succession. This is necessary because the Gambit system is bootstrapped with itself, and to compile the runtime system for a specific version the Gambit compiler for that version must be used. If for some reason you need to do this manually the following procedure should be used. Assuming the local copy is currently at release v1.2.3 and that v1.2.5 is the most recent release, do this:

git checkout v1.2.4-bootstrap
make bootstrap
git checkout v1.2.4
make bootclean bootstrap
git checkout v1.2.5-bootstrap
make bootstrap
git checkout v1.2.5
make bootclean bootstrap
git checkout master
make bootstrap

Note that the latest source code may be unstable. It is wise to check that it passes basic consistency checks by running the command make check.

The local source code files can be updated to a specific version with the git checkout command.

git checkout v4.2.1

After this command it is not possible to build Gambit with a make because the local gsc-comp is not the appropriate version to compile the runtime system. One way around this problem is to download the release for that version in a different directory, do a make bootstrap in that directory, and then copy gsc-comp back here.

Creating a commit

A source code change may simply be modifications of existing files, deletions, and addition of new files. If a new file has been created git should be informed by entering:

git add newfile

After making a set of related changes it is customary to review which files have changed by executing the command:

make status

If all checks out, a commit is created with the command:

make commit

This command will pop up an editor to compose a comment describing the nature of the patch. Please use a short but descriptive comment on the first line of text entered.

To get a list of the commits (starting with the latest, i.e. the HEAD) use:

make log

Contributing a patch

A patch file can be created using the git diff command. To include only the most recent commit use:

git format-patch -1

A range of patches can be specified like this:

git format-patch -4 HEAD~3

The patch file(s) can then be submitted to the Gambit maintainers by sending the file by email to gambit@iro.umontreal.ca. For example:

mail -s "[PATCH] ,b command fix" gambit@iro.umontreal.ca < my-patch

Releasing a new version of Gambit

This section contains some notes for the Gambit maintainers. It explains the steps to follow to release a new version of Gambit. The steps must be followed carefully or you may end up with a wedged system!

Manual procedure

Assume the latest release of Gambit is v1.2.3 and the new release will be v4.5.6. Because Gambit is bootstrapped with itself, releasing a new version of Gambit is a two step process. The Gambit compiler must first be modified so that it generates C code which references the gambit.h file version v4.5.6. Then the gambit.h file, configure.ac file, and other files with an embedded version number must be updated so they refer to version v4.5.6.

1. Creating a Gambit compiler for version v4.5.6

Make sure you have a working bootstrap compiler (gsc-comp) and save a copy just in case:

make check
make bootstrap
cp gsc-comp gsc-comp.old

Now modify the definition of compiler-version in the file gsc/_parms.scm so that it refers to v4.5.6 (using a single integer encoding of the version number):

(define (compiler-version) 405006) ;; 100000*major + 1000*minor + revision

Now create the new bootstrap compiler and commit the change with an easily identifiable comment:

make bootstrap
git commit -a -m "[COMPILER CHANGES NEEDED FOR v4.5.6] Changed version in compiler"
git tag v4.5.6-bootstrap

Note that at this very point the new compiler will not pass the tests because the runtime still expects the old version number.

2. Upgrading the runtime files from version v1.2.3 to v4.5.6

For the C files generated from Scheme files the version numbers can easily be upgraded by compiling them from scratch using the new Gambit compiler. This will be done automatically by a make after a make bootclean. The other changes to the files are made by running the script misc/changev (this must be done first because it changes the file configure.ac which produces the configure script which must be run again).

misc/changev 102003 405006
make bootclean
./configure
make
make check   # will fail test5 because version numbers have changed
mv tests/mix.c tests/test5.ok
make check   # this time all tests should pass
git commit -a -m "[RUNTIME CHANGES NEEDED FOR v4.5.6] Changed version of runtime using misc/changev"
git tag v4.5.6

3. Creating prebuilt installers for version v4.5.6

To create prebuilt installers for Mac OS and Windows you need a Mac with Xcode and Parallels workstation with Windows and the MinGW and Microsoft Visual C Express environments. The following command will build all variants of the installers (it takes about 2 hours in total):

make release

If no errors were reported then you can upload the new release to the Gambit repository using:

make publish-release

This will automatically send an announcement on the Gambit mailing list. You must manually update the main page of the Gambit Wiki so that it refers to the latest version.

Automated procedure

Step 1 and 2 in the above instructions are executed with the change-version make target:

make NEW_VERSION=v4.5.6 change-version

Alternatively, the new-revision, new-minor, or new-major make targets can be used to compute the new version number from the current version by incrementing one of the version number fields:

make new-revision

Troubleshooting

A user was having trouble using trouble building an up-to-date version of Gambit starting from v4.3.2. make update was failing. Marc Feeley responded with an explanation and a solution.

The solution is to download the latest version of Gambit and use that to run the make update in the git repository.

Here's the reply:

Let me explain the problem. If you're only interested in a solution: get v4.6.0 and a make update from that.

Gambit is a self-hosting compiler. That means that it is bootstrapped using itself. Most of the system is implemented in Scheme (not just the libraries, but the compiler itself). So when a change is made in the compiler's source code, it must be compiled with the current version of the compiler. Roughly speaking, version 2 must be compiled with version 1. Then when changes are made (to get the source code of version 3), then version 3 must be compiled with version 2. And so on. When you "make update" the build process executes the chain of self compilations that led to the most recent version of Gambit. In other words, if the most recent Gambit is v4 and you currently have v1 installed, then the "make update" will: compile the v2 source with the v1 executable to get the v2 executable, then it will compile the v3 source with the v2 executable to get the v3 executable, and finally compile the v4 source with the v3 executable to get the v4 executable. This chain must be followed because features that are used in the source code of the most recent version of the compiler may have appeared (i.e. were implemented) in an intermediate version of the compiler.

This self dependency is not easy to handle in the build process and makefiles, and currently there is a bug. In your particular case, Gambit v4.3.2 does not support the same command line options as more recent version of Gambit. So somewhere in this chain of commits, the makefiles were not consistent with the then current compiler. But because the commits can't be "taken back" the self compilation sequence that "make update" executes has some version transitions that will forever be broken. I haven't looked into ways (automatic tool, or better version changing habits) to avoid this problem in the future.