Plasma GitLab Archive
Projects Blog Knowledge

GODI Project:
Home 
Highlights 
Screenshots 
Get GODI 
Docs 
Mailing List 
Resources 
Packages 
Workgroups 
OASIS 
Why Ocaml? 
GODI is discontinued. This is the archived content of the former site godi.camlcity.org.
  up  

README about GODI

GODI - The O'Caml source distribution

$Id: README.xml 940 2011-07-17 21:39:57Z gerd $

GODI is a distribution of the Objective Caml language as packages that can be automatically built from their sources. GODI does not include pre-compiled binaries, but it is possible to create them, even in packaged form.

Basically, GODI is a framework to download, compile, install, and package O'Caml software. This software is not included in GODI, but GODI knows from which Internet sites it can be downloaded. To do so, GODI provides small archives with build instructions, the build.tgz archives. These instructions (basically Makefiles) are dynamically added to the GODI framework, and extend it with the information needed to deal with a certain piece of software.

After the software has been installed, it is always archived as so-called binary packages. The binary packages can be easily transferred to other computers with the same operating system, where they can be added to the local GODI installations without having to rebuild them again.

GODI knows about software dependencies, and automatically builds prerequisites first. Furthermore, GODI can upgrade hierarchies of dependent libraries by rebuilding the libraries in the right order.

Requirements

On which operating systems does GODI run?

This is one of the questions that have not yet really been answered. I am developing GODI on Linux and Solaris, and I am very sure it runs on most Unix-style operating systems without or only with very few changes. The software GODI bases on is known to be quite portable, especially the BSD make environment, the NetBSD packaging tools, and Objective Caml itself. I would like to hear if somebody gets GODI running on a certain platform, because I cannot test it for platforms I don't have access to.

Here is a (preliminary) list of systems (on which the GODI base system runs, but not necessarily all packages):

  • Linux: Debian, Ubuntu, SuSE, Fedora

  • Solaris: Solaris 7-10, Open Solaris

  • FreeBSD: FreeBSD 8

  • NetBSD: NetBSD 5

  • MacOS X: MacOS X 10.3 (Panther)

  • HP-UX: HP-UX 11i (= 11.11)

There is a good chance that it runs on other Unix-type OS, too, provided that Ocaml supports the CPU, and the GNU toolchain is available.

There is also some support for Windows, both as Cygwin and as MinGW port. However, this port is less maintained, and is often broken.

Which other software do I need?

This depends on the operating system. In general, you need gcc, and the usual C toolchain (assembler, linker, preprocessor). Other C compilers are not supported. The toolchain may be GNU or non-GNU, depending on the OS (see below).

Furthermore, you need GNU make, GNU gzip/gzcat, and GNU patch (or compatible, like the versions found in BSD systems). bzip2/bzcat is recommended, some packages need it.

You need the standard Unix tools for shell programming and file manipulation.

If PCRE is coming with your operating system, GODI can take advantage from this. Otherwise you can also enable to install GODI's own version of PCRE.

For packages beyond the bootstrap, you may need additional libraries that are not part of GODI. For example, to install the godi-lablgtk package, you need the gtk library. It is recommended to install at least tcl/tk, as some tools of the O'Caml core distribution base on it, e.g. ocamlbrowser. These additional libraries are not necessary for the GODI bootstrap, and can be installed later when demanded.

Special notes for the operating systems:

  • Linux: The Linux distributions are very different in what they install by default. Normally, the Unix tools are all present, and there is nothing to say about gcc and the C toolchain. Many distros package the C header files separately in "dev" packages. Ensure you have these packages installed, at least for libc and for the libraries you want to use. Recent versions of the bootstrap script have built-in support for checking all this.

  • Solaris: Solaris 8 has GNU gzip and GNU patch, Solaris 7 does not have these tools. You can get the missing tools (gcc, gmake, gzip, gpatch) from sunfreeware.com. Solaris 9 is not tested, but I would not expect problems.

  • FreeBSD: Install GNU make from the package collection.

  • NetBSD: Install GNU make from the package collection.

  • Cygwin: Install bc, binutils, bzip2, diffutils, file, fileutils, findutils, gawk, gcc, gdbm, grep, gzip, m4, make, ncurses, patch, patchutils, sed, sh-utils, tar, tcltk, wget.

  • MacOS X: (Seems to work out of the box, but I did not set up the machine where I ran the tests.)

  • HP-UX: You need gcc, gas (GNU assembler), gmake, gzip/gzcat, gpatch.

How do I install GODI?

Instructions

You need the bootstrap archive godi-bootstrap-<VERSION>.tar.gz (hyperlinks at the end of the text). Unpack it, and run the bootstrap script with the right options, e.g.

gzip -d  godi-bootstrap-<VERSION>.tar.gz
tar xf  godi-bootstrap-<VERSION>.tar
cd godi-bootstrap-<VERSION>
./bootstrap
The script will ask for all install options it needs to know. (There is also the possibility to run it non-interactively with the -batch switch.)

Note that all detail-level log messages are not written to the terminal, but to the file bootstrap.log.

It is not necessary to install GODI as root. Actually, I would not even recommend it, because there is no part that would have advantages from this.

Special notes for Cygwin

  • It is not possible to install GODI onto a network filesystem. Neither it is possible that the bootstrap script is located on such a filesystem. Only local disks are allowed. (The reason is that Cygwin's mkdir does not work properly for this case.) You can, however, create a GODI installation on a local directory /godi on computer A, and use this directory via network from another computer B by creating a symlink from /godi to //A/.../godi such that it points to the same directory. Of course, all installation procedures must be executed on A and never on B.

  • The PATH environment variable must not contain spaces. This practise does not work well for many Unix scripts. For the bootstrap script you must manually ensure this, as it refuses to start when there are spaces. godi_console is a bit more intelligent and simply deletes PATH components that contain spaces. However, this might lead to further problems, e.g. that DLLs are not found. In this case it is recommended to either (1) fix the Windows installation such that important things are not installed under problematic file names, or (2) to create symlinks to these problematic directories and to put the (space-less) symlinks instead of the original names into PATH.

  • Microsoft Windows has stricter filesystem semantics than Unix, especially it often locks files and directories that are not locked by Unix systems in the same situation. For example, it is not possible to delete the current working directory. Of course, GODI deletes a lot of directories when it builds software, and the user should take care not to prevent this by accident (e.g. do "cd /" before starting godi_console). Another problem is that it is not possible to delete executables that are currently running. This is rather annoying (well, not only for GODI, because it is the reason that you must frequently restart the computer when you install programs under Windows), because there are actually packages in GODI that replace themselves. Especially, if you build a new version of godi-tools (containing the program godi_console) the situation arises that the running instance of godi_console would like to replace itself. This is now resolved for the special cases where it is necessary.

Troubleshooting (bootstrap)

Preamble: GODI, and this means both the bootstrap procedure and the final installation, is quite sensitive regarding to environmental influences. The reason is that GODI drives compilers, and these are deeper anchored in the operating system than "normal" software. Furthermore, lots of scripts are used to control the build processes, and these depend more on correct environments than binary software. That means, changing environment variables, or replacing base software of the operating system, may break GODI. It is almost impossible for GODI to improve this situation, because this is caused by the nature of the development environments of the underlying operating systems. -- Anyway, here are some important hints.

The "bootstrap" script writes a log file bootstrap.log containing all the stdout messages. Remember to look into this file when analyzing errors.

After an error the bootstrap script can usually be restarted. Just add the switch -restart. It continues then at the last restart point that was successfully passed.

Most errors are resulting from missing prerequisite software. Some of the often missing programs are dc, file, m4. GODI is a driver for compiling, so you need also the C compiler gcc and the whole toolchain (as, ld, ar, ranlib). Last but not least you need the header files (ending in .h), often available as "development packages" from your operating system.

If required programs are available, but not found, extend the PATH variable before starting "bootstrap". The bootstrap procedure has some built-in intelligence where to find the right versions of programs for the various OS it knows. This usually works, but in special configurations you might want to have more detailed control. The option --prepend-path of "bootstrap" forces that your PATH has higher precedence than the built-in knowledge where to look for programs. Use --search-path arg to completely override the built-in knowledge, and to use arg as (colon-separated) path where to search for programs.

It is sometimes not possible to upgrade the operating system while keeping the GODI installation. One should at least recompile all packages.

I have also reports about environment variables confusing GODI, especially PKGDIR and DISTDIR on Gentoo systems. Ensure that these variables are not set before starting the bootstrap, or before starting godi_console. (I hope I can resolve environment issues more systematically in the future.)

The system clock should not go too wrong, because "make" does not like that. If GODI is installed on NFS, the clocks of the sever and the client must be synchronized using ntp. If not, "make" sometimes skips required compilation steps, and the build will be incomplete, resulting in strange errors.

GODI sometimes misses to download files properly ("Checksum error"). In this case, you can recover from the error as follows: Identify the file name from the error message. Look into /home/joe/godi/build/distfiles, and delete the wrong file. Finally restart the failing action.

And a system-wide installation?

I recommend that you create a system user ID for this purpose, e.g. "godi", that is primarily used as owner of the files. You can cross-login as this user if you want to install or deinstall packages.

GODI does not support group-writeable installations properly, and it is unlikely that it ever will.

Installing and deinstalling GODI packages

The GODI console

When the bootstrap is done, there is a program godi_console allowing you to manage your installation. This is a menu-based application with a simple user interface structured into lots of menus.

You can invoke functions by typing a letter on the keyboard; see the foot lines of the menus displaying which letter is bound to which function. Another way to interact with the program is to enter a number (followed by RETURN), to enter a sub menu.

What can you do with godi_console:

  • Update the build instructions: This is sub menu 1 of the main menu. This function downloads new build instructions, if available. This is usually the first step to update your installation.

  • Select packages for build: This can be done in sub menu 2 of the main menu. There is a list of all available software, and by entering the number of the package you can go to a detailed description. Type then 'b' to select the package.

  • Rebuild packages: Of course, you can also select packages that are already installed. In this case, the old version is removed before the new version is built.

  • Remove packages: This is also possible in the sub menu 2. Go into the sub menu for the package, and press 'r' to select it for removal.

  • Upgrade packages: This is a convenience function; it selects all packages for rebuild for which newer versions are available. (Type 'u' in sub menu 2.) This is usually done after "update".

  • Dependency analysis and the start of installation: Pressing 's' in sub menu 2 performs a dependency analysis, and suggests the next step in the installation process. If the result of the analysis is that all dependencies are fulfilled, the installation (or deinstallation) can start. Otherwise, further packages are added to the selection by godi_console, either for build, rebuild, or removal, and the selection list appears again (adjust the selection, or press again 's' to confirm).

  • Watch the build process: After the installation has started, godi_console monitors the build process. The build messages are printed on the screen, but you can also save them to a file.

Note that godi_console has also a command-line interface, to be used in scripts. There is a manual page explaining it.

How do I install an O'Caml package?

Just for demonstration how simple this usually is:

  • Start godi_console

  • Select the first sub menu by entering 1 and RETURN

  • A lot of messages appear, and the build instructions are updated for the newest version. After that, the main menu appears again.

  • Select the second sub menu by entering 2 and RETURN

  • The list of packages appears. Scroll with 'n' and 'p' up and down (or use the cursor keys). Select your package by entering its number, and press RETURN.

  • Press 'b' to select the package for build.

  • Press 'x' to exit the package dialog. The list of packages is displayed again.

  • Press 's' to do the dependency analysis. Often, further packages are now selected (look for the 'b' flag in the FL column). In this case, press 's' again.

  • A screen is displayed explaining the build plan, i.e. how many packages will be built. Press 'o' to start.

  • Lots of build messages scroll over the screen ...

  • Press 'x' to quit the build process, and exit godi_console.

Is there a command-line tool for installation?

In previous versions of GODI I said "no" to this question, and in very old versions I explained a tool called "godi_build" intended for the command line. These answers are all no longer up to date. There is such a tool, and it is godi_console itself which can also be used on the command-line.

For example, to build the package godi-xstr just type

godi_console perform -build godi-xstr
and all actions are carried out to build or rebuild this package. Of course, everything is done without asking further questions. Because of this, I strongly recommend to first do

godi_console perform -dry-run -build godi-xstr
to have a look at the build plan without executing it. It is also a good idea to first set up the wish list (with "godi_console wish") before executing plans with "godi_console perform -wishes", because godi_console becomes interruptible by this. This means, in case of an error it knows which wishes have already been fulfilled and which are still open, i.e it knows where to continue. (The wish list is actually the column with the "r" and "b" flags in the interactive mode of godi_console.)

There is a manual page explaining the command-line mode of godi_console in detail.

Note that there is still a tool called "godi_build", but it should only be used for bootstrapping. A number of package options have not been implemented in this script, and there are probably lots of errors in it.

Traps

There are now a number of "conf" packages: conf-gdbm, conf-x11, conf-tcltk, conf-pcre.

These packages configure where system libraries can be found. For example, conf-x11 tries to find the X11 tree. For all of these packages it is possible to override the corresponding compiler switches by adding options to godi.conf. (godi_console allows you to set these options in a very convenient way; press 'c' in the package menu.)

For example, the package godi-ocaml-graphics (containing the Graphics library) requires that conf-x11 is installed first. conf-x11 searches the X11 options, and saves them into a file, where godi-ocaml-graphics expects them. The consequence is that changes of godi.conf do not have an immediate effect on the build of godi-ocaml-graphics, because this package reads its configuration from the file created by conf-x11; this package must be built again to make the change effective. I hope you never run into such a trap, because I think the conf-* packages can only be created for configurations that will work (they should fail to build otherwise).

The O'Caml system has been divided up into four GODI packages:

  • godi-ocaml-src:

    Downloads the source code, configures it, and makes it available to other packages.

  • godi-ocaml:

    This pkg creates the runtime system, and the compilers.

  • godi-ocaml-graphics:

    The graphics library (requires X Windows)

  • godi-ocaml-labltk:

    The labltk library (including ocamlbrowser)

If you want to change one of the _basic_ O'Caml options, you have to do that for godi-ocaml-src, and have to rebuild the other packages individually (e.g. if you change which C compiler is used).

How do I update my installation?

With godi_console, this is very simple:

  • Start godi_console

  • Select the first sub menu by entering 1 and RETURN

  • A lot of messages appear, and the build instructions are updated for the newest version. After that, the main menu appears again.

  • Select the second sub menu by entering 2 and RETURN

  • The list of packages appears. Just press 'u' and confirm the following question ('o'). Now all packages are selected for which new versions are available.

  • Press 's' to do the dependency analysis. Often, further packages are now selected (look for the 'b' flag in the FL column). In this case, press 's' again.

  • A screen is displayed explaining the build plan, i.e. how many packages will be built. Press 'o' to start.

  • Lots of build messages scroll over the screen ...

  • Press 'x' to quit the build process, and exit godi_console.

Note that upgrades often select many packages because of dependencies, because all _dependent_ packages are also rebuilt if a prerequisite package is upgraded!

How do I upgrade to the newest O'Caml system?

The godi.conf variable GODI_SECTION determines which section of the available packages is selected. The section name is normally the version number of the O'Caml system, e.g. GODI_SECTION=3.07. By changing the section name, you can upgrade to a newer O'Caml version (when GODI provides such a version...).

It is a good idea to save your old binary packages first, because some of them will be overwritten. (Because libraries with the same version numbers may be part of several sections.)

Note that you can have several GODI installations, provided that only one installation is part of your PATH variable. (Actually, GODI looks up itself by running the command godi_confdir which outputs the absolute path of the etc directory with the godi.conf file.)

How do I manage the binary packages?

(Management of binary packages is not yet supported by godi_console, so the following is still the only possibility.)

You can find out which packages are installed by running

godi_info -a
This command can also be used to get detailed information about packages (see man godi_info).

You can delete packages by running

godi_delete <packagename>
If the package cannot be deleted because of dependencies, do not use the -f (force) option, because this creates a mess. Better also delete the dependent packages with the -r option. (Or use godi_console, as it is already possible to remove packages.)

You can add packages by running

godi_add <packagefile>.tgz
All of these commands have manual pages.

These commands are derived from the pkg_* commands of NetBSD, and are very similar. The -F option does not work, however, because there is no file index that would allow fast lookup of files.

What is the directory layout of the O'Caml installation?

We have (ASCII art):

/home/joe/godi = <prefix>
  |
  +- etc
  |  configuration files, e.g. godi.conf, findlib.conf, ld.conf
  |
  +- bin
  |  binaries for normal applications, and development
  |
  +- sbin
  |  binaries for GODI management and administrative tasks
  |
  +- man
  |  manual pages
  |
  +- doc
  |  |
  |  +- <package name>
  |  |  documentation for a package
  |  |
  |  +- cgi-bin
  |  |  CGI programs for documentation
  |  |
  |  +- GODI
  |     documentation for GODI itself
  |
  +- lib
     C libraries, plus:
     |
     +- ocaml
     |  |
     |  +- std-lib
     |  |  O'Caml standard library, "the big mess"
     |  +- pkg-lib
     |  |  Findlib-managed libraries installed with GODI
     |  +- site-lib
     |  |  Findlib-managed libraries manually installed
     |  +- compiler-lib
     |     some .cmi files of the O'Caml compilers
     |
     +- godi
        |
        +- saved-src
           +- ocaml.tar.gz
              The archived sources of O'Caml, after configuration
              (to apply patches)
There are some more directories not depictured here, but they are either standard Unix, or have to do with special software (e.g. share/mk is required by bmake).

Another omitted directory is "build", used for the build framework (see above).

Troubleshooting (installing packages)

A system library is not found

Often, there is a "conf" package for the system library, e.g. conf-pcre configures where libpcre is installed. The "conf" packages have some built-in intelligence where to search for system libraries. Of course, these packages cannot look into every directory of the system, and it is also possible that they find the wrong version of the library (if you have installed several versions). So the question is how to exactly specify where to expect a system library.

For every library there are options for godi.conf. By setting them, the search algorithm of the "conf" package is modified such that a certain location is tried before all others. From godi_console, you can set these options in the configure dialog (after you have selected a certain package of the list, type 'c' to enter this dialog). This dialog explains which options are available for the package, and allows you to change them.

For example, for conf-pcre the configure dialog outputs:

"Options:
[ 1]         GODI_BASEPKG_PCRE = no
[ 2]          GODI_PCRE_INCDIR =
[ 3]          GODI_PCRE_LIBDIR =

Build message:
Available options for godi.conf:

- GODI_BASEPKG_PCRE: Whether to build GODI's version of pcre
  (yes/no). Defaults to "no".

- GODI_PCRE_INCDIR: Where pcre.h can be found. Set this option
  if GODI_BASEPKG_PCRE=no, and the already installed PCRE
  library is not found.

- GODI_PCRE_LIBDIR: Where libpcre can be found. Set this option
  if GODI_BASEPKG_PCRE=no, and the already installed PCRE
  library is not found."
You have the option to set GODI_BASEPKG_PCRE to "yes", and to use the version of libpcre that is distributed with GODI. In this case, you need not to set the other variables, because GODI knows where to search its own components. If you leave GODI_BASEPKG_PCRE="no", it is expected that libpcre is already installed somewhere. You can try whether GODI can find this location: Just press 't' to test this. If this fails, you must set GODI_PCRE_INCDIR and GODI_PCRE_LIBDIR to the directories where pcre.h and libpcre.so are installed, respectively. Test your options before you leave the configure dialog.

Pitfall: Sometimes not the directories need to be entered, but the flags for the C compiler (e.g. -I/directory, -L/directory). This is usally indicated in the build message.

It is also possible to modify the search algorithm of the "conf" packages globally. This is interesting if you have your private location where add-on libraries are installed, and you do not want to configure this location again for every package. The variable SEARCH_LIBS specifies the locations where to search. If you set it in godi.conf, e.g.

SEARCH_LIBS=/my/path
your locations are prepended to the search path. If you have several such locations, use this syntax:

SEARCH_LIBS=/my/path1
SEARCH_LIBS+=/my/path2
The default path includes /usr, /usr/local, /usr/pkg, /opt/pkg, /opt/sfw, /sw, and /cw. (If this path should be extended because an operating system or software distribution installs libraries at another location, please tell me.)

The "conf" packages, once configured and installed, save the successful options into another file, such that further changes of godi.conf do not have any effect. If it becomes necessary to change library options again, you must rebuild the "conf" package first.

Why is the system-wide PCRE library not accepted?

Because it is too old. godi-pcre needs at least PCRE 4.0.

I am behind a firewall. How do I enforce passive FTP/a proxy?

Passive FTP is enabled by default.

To use a proxy, you need to set the environment variables ftp_proxy and http_proxy, e.g.

ftp_proxy=http://proxy.company.com
http_proxy=http://proxy.company.com
export ftp_proxy http_proxy
These variable must be set in the environment , not in godi.conf.

Links


  up  
This web site is published by Informatikbüro Gerd Stolpmann
Powered by Caml