fortran

Rebuilding the whole Debian archive using the Open64 compiler

I got bored recently, so I rebuilt the whole Debian archive on one of my machines. To make this not a completely useless excercise, I used the Open64 compiler instead of gcc and created build logs for your perusal.

So what is Open64?

From the Wikipedia page:

Open64 is an open source, state-of-art, optimizing compiler for the Intel IA-64 (Itanium), AMD Opteron and Intel IA-32e architecture. It derives from the SGI compilers for the MIPS R10000 processor. It was released under the GPL in 2000, and now mostly serves as a research platform for compiler and computer architecture research groups. Open64 is licensed under the GPL. Open64 supports Fortran 77/95 and C/C++, as well as the shared memory programming model OpenMP. It can conduct high-quality interprocedural analysis, data flow analysis, data dependence analysis and array region analysis.

Open64 installation

The installation is pretty easy fortunately:

$ wget http://ovh.dl.sourceforge.net/sourceforge/open64/open64-4.0-src.tar.bz2
$ tar xfvj open64-4.0-src.tar.bz2
$ cd open64-4.0
$ export TOOLROOT=/opt/open64
$ make
$ make install (as root)

I think you need gcc-3.4 (gcc 4.x is not yet supported), and for some odd reason you also need csh as one of the install scripts seems to use it.

It would be nice if someone could package Open64 for Debian, I definately don't have the time to maintain such a huge package (a whole maintainer team would probably be good here).

Rebuilding the Debian archive

There are several possible ways (and tools) to rebuild the Debian archive; I've used pbuilder/cowbuilder with the rebuild scripts from Bastian Venthur, which are now included in pbuilder.

First we need to install the required packages, setup a cowbuilder base chroot, and get the list of packages:

$ apt-get install cowdancer grep-dctrl wget devscripts sudo
$ cowbuilder --create --distribution lenny --basepath /var/cache/pbuilder/testing-base.cow
$ cp -r /usr/share/doc/pbuilder/examples/rebuild .
$ cd rebuild
$ ./getlist lenny

Now we add Open64 into the cowbuilder chroot and fix up the chroot by pointing the gcc/g++ symlinks to Open64:

$ cp -a /opt/open64 /var/cache/pbuilder/testing-base.cow/opt
$ chroot /var/cache/pbuilder/testing-base.cow
$ cd /usr/bin
$ mv gcc gcc.orig
$ ln -s /opt/open64/bin/opencc gcc
$ mv g++ g++.orig
$ ln -s /opt/open64/bin/openCC g++
$ exit

In addition, we set the CC and CXX environment variables to Open64, which will make 90% of all (autoconf-using) packages automatically use Open64. We need a small script for that:

$ cat c.cfg:
export CC="/opt/open64/bin/opencc -m32"
export CXX="/opt/open64/bin/openCC -m32"

Now edit the buildall script. Change the Debian mirror used there (optional) and make it use our c.cfg script by adding the --configfile /path/to/rebuild/c.cfg option in the "pdebuild" line.

We can now finally start building the archive:

./buildall list.lenny.i386 lenny

You can also run multiple buildall instances at once to speed up the archive rebuild on SMP/multicore machines, and you can even abort the command and simply restart it later. The script will continue where it left off.

Results

The whole rebuild (with 2 instances of buildall running at the same time) took ca. 9 days on an AMD64 Athlon64 X2 (dual core, 1.8 GHz each) machine with 1 GB of RAM.

I really should have used something like apt-proxy to speed up the rebuild and save some bandwidth, but I read about apt-proxy too late...

All log files from my rebuild are available for detailed analysis if anybody is interested (you can browse the logfiles online or download all of them as tarball). I didn't perform any detailed analysis, just some rough numbers here:

  • Succeeded package builds: 8425
  • Failed package builds: 2509
  • Total number of packages rebuilt: 10934

If anybody does some more elaborate analysis, please let me know.

Syndicate content