---------------------------------------------------------------------- Finding shared libraries in a portable way ---------------------------------------------------------------------- $Id: finding_libraries.txt 393 2004-05-01 22:25:31Z gerd $ GODI supports a number of operating systems, and unfortunately, different methods of looking up shared libraries can be found. At the time of writing this note, we have: - ELF-based systems (Linux, FreeBSD, much of NetBSD, Solaris) - PE-based systems (Cygwin) - Mach-O-based systems (Darwin/OS X) - enhanced a.out systems (some NetBSD platforms) - HP-UX From earlier projects, I also know: - XCOFF (AIX) although this platform is not yet supported by GODI. In principle, there is also the case that shared libraries are not available, or are not supported by GODI. In this note, only shared libraries are discussed, i.e. libraries that are specified at link time, and are automatically loaded when the program starts. In contrast to this, dynamic libraries are loaded at runtime on behalf of the running program. For some systems, both types of libraries are generalized to a common type, but for other systems, the types are independently implemented. ---------------------------------------------------------------------- Details of the systems ---------------------------------------------------------------------- (1) ELF There are usually three ways of looking up libraries at runtime: (a) by the environment variable LD_LIBRARY_PATH This is a colon-separated path. (b) by the RPATH stored in the executable that needs the library as dependency The RPATH is part of the "dynamic" header of the executable. There can be several RPATHs, which are tried in order. Important: The RPATH is stored in the file pointing to the library, not in the library itself. (c) by a fixed or configurable set of default paths This includes usually /lib and /usr/lib, and maybe further locations. For some systems, the default paths are configurable (e.g. in /etc/ldconfig for Linux). For GODI, (a) is not an option, because: Environment variables cannot always be set, especially when programs are started by daemons. There is no way to resolve conflicts, when in the same environment two programs must be started that need different library paths. Programs relying on LD_LIBRARY_PATH cannot be setuid/setgid. Method (a) has too many restrictions for a general-purpose SDK. Method (c) works very well for libraries with system-wide scope that are potentially available for every program on the system. This includes the libraries coming with the OS, and also local additions when they are carefully managed (often found in the /usr/local tree). Method (b) is best for additions that don't have system-wide scope, and when it is required to override libraries looked up by (c). When GODI installs libraries, these are intended to be used for programs created with GODI, and should not affect other programs. For these reasons, such libraries must be looked up using method (b), i.e. with an RPATH. When GODI searches libraries, it is often the case that they are of type (c), but we cannot exclude type (b). The algorithm searching for libraries must take both methods into account. SYNTAX Method (c) is the default when an executable foo is linked with a library bar, and no special option is specified: gcc -o foo -lbar When there are -L options their values don't have any effect on the way of finding the library at runtime. Method (b) can be forced by adding an RPATH. This requires a linker option -R or -rpath, like in gcc -o foo -Wl,-R,/path/to/dir -lbar (-Wl,arg,arg,... simply passes the comma-separated options arg to the linker.) Alternatively, there is usually also the environment variable LD_RUN_PATH, a colon-separated path listing the RPATH entries. This is just another way of setting these entries. DEBUGGING On systems with GNU toolchain, use objdump -p -j .dynamic foo The printed header block contains all RPATH entries added to the executable. For Solaris, call dump -Lv foo for a similar output. Of course, one can also call the ldd tool, but it does not print why a library is taken from a certain directory: ldd foo FURTHER INFORMATION - Filename pattern: Shared libraries have names like lib<name>.so.<VERSION> (where .<VERSION> is optional). Here, <name> is the identifier passed to the C compiler/linker with the -l switch. - Handling different versions of the same library: The linker looks into the file lib<name>.so, and the entry SONAME contains the real name, usually with a version number. At runtime, the SONAME determined at link time is actually be used to search the library file. This allows very flexible setups (but the details are beyond the scope of this note). ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ (2) PE + Cygwin Shared libraries have two parts: - The import library, which is a static library added to the executable linking with the shlib - The dynamic library, a .dll file At compile time, one only needs the import library which is some kind of wrapper around the code doing the dynamic link in reality, so the executable needs not to cope with the low-level details. At runtime one only needs the dll file. Finding the dll file is the critical part. As far as I know, there is only one way of finding dll files: The directories in the PATH environment variable are searched one after the other. (Recall that there are no absolute installation locations under Windows: At installation time, the user can enter the directory where to install the software (even for Windows itself). So it does not make any sense to have something like RPATH under Windows.) Cygwin puts the DLL parts of all system libraries into /bin or /usr/bin (depending on the Cygwin version) - from the point of view of Windows this might be something like D:\Cygwin\bin or D:\Cygwin\usr\bin, and the chosen directory is in PATH. The import libraries are in /lib or /usr/lib. As there is no other method, GODI must use this way of finding libraries. Because there is no benefit but lots of trouble, GODI must not install additional shared libraries. Static libraries are to be used. As O'Caml under Cygwin does not support dynamic loading of stubs, there is no striking advantage from using shared libraries here. When searching for libraries, GODI simply looks at the import libraries, and ignores the DLL part. The user is responsible for setting the PATH variable correctly to find the necessary DLLs. SYNTAX With gcc -o foo -lbar the static library bar is added to the executable foo. bar may be really static, or an import library for a DLL, one does not know this. (However, there is a naming convention under Cygwin, see below.) The -L options only affect searching the import libraries. DEBGUGGING The command cygcheck foo.exe outputs which DLL files are loaded at runtime. FURTHER INFORMATION - Import libraries have the file pattern (under Cygwin): lib<name>.dll.a Static libraries: lib<name>.a (without .dll) DLL files have the pattern: cyg<name>.dll or do not follow any pattern (except the suffix .dll) when they are coming from outside of Cygwin. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ (3) Mach-O There are three ways of looking up shared libraries: (a) by the environment variable DYLD_LIBRARY_PATH This is a colon-separated path, works like LD_LIBRARY_PATH under ELF (b) by a path stored in the executable that is automatically collected when linking the executable with the library This means: The library foo has an entry that specifies the official location of the library, this entry is called the install_name. It should be an absolute path to the library. This path is copied to the executable, and used to load the library. Note: This is similar to the SONAME of ELF with the addition that the install_name may contain the full path of the library. (c) by the environment variable DYLD_FALLBACK_LIBRARY_PATH, which defaults to $(HOME)/lib:/usr/local/lib:/lib:/usr/lib Note: There is also a more abstract view on libraries, called frameworks. When a library is part of a framework, another set of paths is used (I think). Much of the discussion for ELF also applies to Mach-O. Interestingly, the method (b) of storing the lookup path in the executable is solved differently, and this method is regarded as the usual way for application-specific libraries. For GODI this means that (b) is the method of choice when it installs additional shared libraries, and that it must pay attention to (b) when searching libraries. SYNTAX gcc -o foo -lbar looks up libbar.dylib at link time, pulls the install_name from this file (if not present, uses libbar.dylib instead), puts the install_name into the executable foo for lookup at runtime. -L only affects the search at link time. It is possible to override the install_name at link time: gcc -o foo -Wl,-dylib_file,/orig/install_name:/subst/install_name \ -lbar The /orig/install_name (found in any library) is replaced by /subst/install_name. This may be useful to create executables when the library bar is not (yet) at the location indicated by the install_name. DEBUGGING There is otool, don't know how to use it. FURTHER INFORMATION: - Shared libraries have the file pattern: lib<name>.<version>.dylib where .<version> is an optional three-level version number. The same symlink tricks are used as for ELF. References: http://fink.sourceforge.net/doc/porting/porting.en.html http://www.freebsd.org/cgi/man.cgi?query=ld&apropos=0&sektion=1&manpath=Darwin+7.0.1+PPC&format=html http://www.freebsd.org/cgi/man.cgi?query=dyld&apropos=0&sektion=1&manpath=Darwin+7.0.1+PPC&format=html ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ (4) XCOFF This is again a totally different story. The loader tries these methods: (a) Search the directories enumerated in the environment variable LIBPATH So, LIBPATH works at runtime like LD_LIBRARY_PATH for ELF. However, LIBPATH is also consulted at link time, like LD_RUN_PATH for ELF (i.e. both aspects are not separated). (b) Search the directories of a path that have been recorded in the executable during link time There are numerous ways of setting this path (also called "Index 0 path"), it depends on the combination of: * LIBPATH * -L switches * -blibpath switches * -noblibpath switches * the presence of import files The usual way of doing things is described below. Note that there is no system-wide path. The "Index 0 path" usually contains /lib, /usr/lib, etc. GODI must not create shared libraries. It is a painful procedure, one has to create an export file containing all symbols to export, and there are lots of traps. For example, shared libraries must sometimes be manually unloaded from memory to force that a new version will be used in the future. It is very difficult to create a cascade of two dependent libraries without installing the first library at its final location before making the second. When searching shared libraries, the -L linker switches are crucial. See below to see why. When -blibpath and -noblibpath are not used, the Index 0 path will be set correctly. SYNTAX gcc -o foo -lbar -L/path/to/bar Here, the -L option is not only used to search bar at link time, but this path is also recorded in foo as Index 0 path for runtime lookup. -L/lib and -L/usr/lib are magically added, so these system directories are always part of the Index 0 path. It is also possible to set the Index 0 path directly: gcc -o foo -lbar -L/path/to/bar -Wl,-blibpath,/path/to/bar:/lib:/usr/lib -blibpath overrides all other rules to set the Index 0 path. DEBUGGING dump -HTv foo outputs the recorded runtime paths It is also possible to see what is really happening at load time. Run foo under dbx, and execute the "map" command. FURTHER INFORMATION - File pattern: A single shared library has the suffix .o as any other object file. One has to look into the file to find out whether it is shared or not: dump -ov file.o It is common practice to put several versions of the same shared library into an archive (created by ar). For example, libc.a is a collection of shared libraries. Pattern: lib<name>.a, without version number. There are also files ending in .so. This is a different format, enabled for run-time loading (dlopen). References: http://www-106.ibm.com/developerworks/eserver/pdfs/aix_ll.pdf (especially Appendix D) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ (5) a.out as used by NetBSD Short version: There is LD_LIBRARY_PATH, no RPATH, and a configurable default ld.so.conf. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ (6) HP-UX For 32-bit programs, the file format SOM is used, for 64-bit programs, the ELF format is used. Currently, ocamlopt generates 32 bit output. (6a) HP-UX 32 bit There are two ways of looking up shared libraries: (a) by the environment variable SHLIB_PATH This is a colon-separated path, works like LD_LIBRARY_PATH under ELF (b) by a path stored in the executable that is automatically collected when linking the executable with the library HP-UX simply stores the -L directories in the generated executable, like AIX. GODI must not create shared libraries. SYNTAX gcc -o foo -lbar -L/path/to/bar Here, the -L option is not only used to search bar at link time, but this path is also recorded in foo for runtime lookup. DEBUGGING n/a (6b) HP-UX 64 bit NOTE: This mode is not used for GODI! For lookup, the usual ELF rules apply: - LD_LIBRARY_PATH - The RPATH stored in the executable - A system-wide default SYNTAX gcc -o foo -lbar -L/path/to/bar -Wl,+b,COLON:SEPARATED:PATH DEBUGGING ldd elfdump FURTHER INFORMATION - Shared library names have the pattern libNAME.sl or libNAME.NUMBER where NUMBER is a simple decimal number, e.g. libm.2 References: "HP-UX Linker and Libraries User's Guide": http://docs.hp.com/hpux/onlinedocs/B2355-90655/B2355-90655.html ld(1): http://docs.hp.com/cgi-bin/onlinedocs.py?mpn=B2355-90680&service=hpux&path=../B2355-90680/00/01/152&title=HP-UX%20Reference%20Volume%201%3A%20Section%201 ====================================================================== GODI ARCHITECTURE TO FIND LIBRARIES ====================================================================== [Note: Actually, this section is a suggestion to fix the various problems of the current version of GODI.] +++ Requirements +++ The libraries to find are either: (1) additions by GODI in $LOCALBASE/lib (2) additions by the user found anywhere (but the user has to say where) (3) system-wide libraries For (1) and (2), path-based library lookup is preferred if available. For (3) the system default is preferred (whatever it is). The method to choose is controlled by Makefile variables. These are usually set by defs.<OS>.mk and godi.conf, and interpreted by bsd.pkg.mk, and the driver Makefiles in the packages. GODI can always assume that a library is installed in its final place before an executable is created linking with it. In the PLISTs, the various filename conventions are a problem. A syntax is needed to point to a library in a portable way. Libraries are usually searched by the conf packages. The user can manually specify the library location (or compile/link options). If not, the library is searched in a number of defined places (OS-dependent). +++ Makefile variables +++ CREATE_SHLIBS Values: "yes", "no" = undefined Meaning: If "yes", it is allowed to create shared libraries for this system (usually passed as option to a "configure" script using autoconf, so GODI needs not to know how to really create a shlib) Default: set by defs.<OS>.mk SHLIB_TYPE Values: "ELF", "PE", "MachO", "XCOFF", "NA", maybe "a.out" Meaning: Which file format the OS uses for shared libraries. NA=not available, the system does not support shlibs, or better, all library handling is done as for static libraries Default: set by defs.<OS>.mk SEARCH_LIBS Values: Space-separated list of paths Meaning: The directories where to look for system-wide libraries and user additions. GODIs own location $LOCALBASE needs not to be listed. The "base directories" are listed in this variable, i.e. "lib" is appended to look for libraries, and "include" is appended to look for .h files Default: XXX SEARCH_LIBS_OVERRIDE Values: Space-separated list of paths Meaning: SEARCH_LIBS is set to a default list, and it is only possible to prepend to it in godi.conf. If it is required to override the default list completely, one can set SEARCH_LIBS_OVERRIDE instead. Default: unset ELF_RPATH Values: "yes", "no" = undefined Meaning: If "yes", options are passed to the linker setting the RPATH This variable must not be "yes" if SHLIB_TYPE != ELF Default: set by defs.<OS>.mk ELF_RPATH_FLAG Compat name: RPATH_FLAG Values: "-R", "-rpath", undefined Meaning: This must be set when ELF_RPATH=yes. This is the name of the linker option setting the RPATH. Default: set by defs.<OS>.mk LDCONFIG: Values: C compiler switches for link stage Meaning: The make framework adds the RPATH switches to this variable when they are required. LDCONFIG is passed to subsequent "configure" and "make" invocations. Default: set by bsd.pkg.mk +++ Variables typically set by conf packages +++ Definitions for these variables are written to the files $LOCALBASE/lib/godi/foo.mk by the configure script of the conf-foo package. There is no strict specifications for these variables, and packages can deviate from the following suggestion if needed: CONF_foo_INCDIR Values: Directory Meaning: Where to look for .h files CONF_foo_LIBDIR Values: Directory Meaning: Where to look for shared libraries CONF_foo_DEFS or CONF_foo_CFLAGS Values: C compiler options for preprocessing and compilation stage Meaning: Can be used instead of CONF_foo_INCDIR when it is possible that more than a single -I option must be passed to the C compiler. CONF_foo_LIBS Values: C compiler options for linker stage Meaning: Can be used instead of CONF_foo_LIBDIR when it is possible that more than a single -L option must be passed to the compiler. This variable must not contain RPATH-related options (or better, _either_ the RPATH options must be part of this variable, or the package sets CONF_foo_NEED_RPATH to signal whether these options must be added) CONF_foo_STATIC_LIBS Values: C compiler options for linker stage Meaning: When different (more) options are necessary for static linking, these options should go to this variable. For static linking, often more dependent libraries must be listed. If this distinction does not make sense, this variable can be omitted. CONF_foo_NEED_RPATH Values: "yes", "no" = undefined Meaning: Whether to pass an RPATH option to the linker for _this_ library. This can only be "yes" if ELF_RPATH=yes. This must be "no" if the library can be linked without RPATH entry. The RPATH option is created for the directory CONF_foo_LIBDIR, or for every directory of a -L switch in CONF_foo_LIBS. CONF_foo_CONFIG Values: Path to an executable Meaning: There are now often foo-config scripts that return the necessary options to link with a library. If such a script is used, this variable contains the _absolute_ path to it. Nothing is specified how the script is called, and what does it return. +++ How conf packages work +++ The conf packages should look into: - The directory provided by the GODI user - The directories listed by SEARCH_LIBS or SEARCH_LIBS_OVERRIDE (in this order). To check whether a directory works: a. Test whether the necessary .h files can be found b. Test whether a sample object file can be created using these .h files c. It MUST NOT be tested whether the necessary libraries can be found (by looking at the file names) - don's assume file suffixes here d. Test whether a sample program can be created that links with the library e. Test whether the program can be started f. If necessary: Further tests, e.g. whether the version of the library is right If all tests are passed, the directory can be accepted. If not, another test should be done by adding RPATH options (if enabled), i.e. repeat tests d.-f. +++ PLIST syntax +++ @library path/to/dir name selects all files in path/to/dir that are part of the library <name>. Both static and shared libraries are matched. The <name> must not contain prefixes like "lib", and it must not contain suffixes, i.e. it is the same identifier as passed to the -l linker option. If present, .la files (GNU libtool) are also selected. Note: This does not work when several packages install the same library in different versions. But this is not possible anyway, because not all OS support this.