3.1.1.1 Portable FASL ¶

ECL provides two compilers (bytecodes compiler, and C/C++ compiler). Portable FASL files are built from source lisp files by the bytecodes compiler. Generally FASC files are portable across architectures and operating systems providing a convenient way of shipping portable modules. Portable FASL files may be concatenated, what leads to bundles. FASC files are faster to compile, but generally slower to run.

;; install bytecodes compiler
(ext:install-bytecodes-compiler)

;; compile hello.lisp file to hello.fasc
(compile-file "hello1.lisp")
(compile-file "hello2.lisp")

;; reinitialize C/C++ compiler back
(ext:install-c-compiler)

;; FASC file may be loaded dynamically from lisp program
(load "hello1.fasc")

;; ... concatenated into a bundle with other FASC
(with-open-file (output "hello.fasc"
                        :direction :output
                        :if-exists :supersede)
  (ext:run-program
   "cat" '("hello1.fasc" "hello2.fasc")  :output output))

;; ... and loaded dynamically from lisp program
(load "hello.fasc")

3.1.1.2 Native FASL ¶

If you want to make a library which is loaded dynamically from a lisp program, you should choose the fasl file format. Under the hood native fasls are just shared library files.

This means you can load fasl files with dlopen and initialize it by calling a init function from C programs, but this is not an intended usage. The recommended usage is to load fasl files by calling the load lisp function. To work with Native FASL files ECL has to be compiled with --enable-shared configure option (enabled by default).

Creating a fasl file from one lisp file is very easy.

(compile-file "hello.lisp")

To create a fasl file from more lisp files, firstly you have to compile each lisp file into an object file, and then combine them with c:build-fasl.

;; generates hello.o
(compile-file "hello.lisp" :system-p t)
;; generates goodbye.o
(compile-file "goodbye.lisp" :system-p t)

;; generates hello-goodbye.fas
(c:build-fasl "hello-goodbye"
              :lisp-files '("hello.o" "goodbye.o"))

;; fasls may be built from mix of objects and libraries (both shared and
;; static)
(c:build-fasl "mixed-bundle"
              :lisp-files '("hello1.o" "hello2.a" "hello3.so"))

3.1.1.3 Object file ¶

Object files work as an intermediate file format. If you want to compile more than two lisp files, you might better to compile with a :system-p t option, which generates object files (instead of a fasl).

On linux systems, ECL invokes gcc -c to generate object files.

An object file consists of some functions in C:

• Functions corresponding to Lisp functions
• The initialization function which registers defined functions on the lisp environment

Consider the example below.

(defun say-hello ()
  (print "Hello, world"))

During compilation, this simple lisp program is translated into the C program, and then compiled into the object file. The C program contains two functions:

• static cl_object L1say_hello: ’say-hello’ function
• ECL_DLLEXPORT void _eclwm2nNauJEfEnD_CLSxi0z(cl_object flag): initialization function

In order to use these object files from your C program, you have to call initialization functions before using lisp functions (such as say-hello). However the name of an init function is seemed to be randomized and not user-friendly. This is because object files are not intended to be used directly.

ECL provides other user-friendly ways to generate compiled lisp programs (as static/shared libraries or executables), and in each approach, object files act as intermediate files.

3.1.1.4 Static library ¶

ECL can compile lisp programs to static libraries, which can be linked with C programs. A static library is created by c:build-static-library with some compiled object files.

;; generates hello.o
(compile-file "hello.lsp" :system-p t)
;; generates goodbye.o
(compile-file "goodbye.lsp" :system-p t)

;; generates libhello-goodbye.a
(c:build-static-library "hello-goodbye"
                        :lisp-files '("hello.o" "goodbye.o")
                        :init-name "init_hello_goodbye")

When you use a static/shared library, you have to call its init function. The name of this function is specified by the :init-name option. In this example, it is then init_hello_goodbye. The usage of this function is shown below:

#include <ecl/ecl.h>
extern void init_hello_goodbye(cl_object cblock);

int
main(int argc, char **argv)
{
    /* setup the lisp runtime */
    cl_boot(argc, argv);

    /* call the init function via ecl_init_module */
    ecl_init_module(NULL, init_hello_goodbye);

    /* ... */

    /* shutdown the lisp runtime */
    cl_shutdown();

    return 0;
}

Because the program itself does not know the type of the init function, a prototype declaration is inserted. After booting up the lisp environment, it invokes init_hello_goodbye via ecl_init_module. init_hello_goodbye takes an argument, and ecl_init_module supplies an appropriate one. Now that the initialization is finished, we can use functions and other stuff defined in the library.

DEPRECATED read_VV - equivalent to ecl_init_module

3.1.1.5 Shared library ¶

Almost the same as with a static library. The user has to use c:build-shared-library:

;; generates hello.o
(compile-file "hello.lsp" :system-p t)
;; generates goodbye.o
(compile-file "goodbye.lsp" :system-p t)

;; generates libhello-goodbye.so
(c:build-shared-library "hello-goodbye"
                        :lisp-files '("hello.o" "goodbye.o")
                        :init-name "init_hello_goodbye")

3.1.1.6 Executable ¶

ECL supports the generation of executable files. To create a standalone executable from a lisp program, compile all lisp files to object files. After that, calling c:build-program creates the executable:

;; generates hello.o
(compile-file "hello.lsp" :system-p t)
;; generates goodbye.o
(compile-file "goodbye.lsp" :system-p t)

;; generates hello-goodbye
(c:build-program "hello-goodbye"
                 :lisp-files '("hello.o" "goodbye.o"))

Like with native FASL, the program may be built also from libraries.

3.1.1.7 Summary ¶

In this section, some file types that can be compiled with ECL were introduced. Each file type has an adequate purpose:

• Object file: intermediate file format for others
• Fasl files: loaded dynamically via the load lisp function
• Static library: linked with and used from C programs
• Shared library: loaded dynamically and used from C programs
• Executable: standalone executable

ECL provides a high-level interface c:build-* for each native format. In case of Portable FASL the bytecodes compiler is needed.

3.1.2.1 Example code to build ¶

An example project is included in the ECL source distribution in the examples/asdf_with_dependence/ directory.

This project depends on the alexandria library and consists of a system definition (example-with-dep.asd), package definition (package.lisp), and the actual library code (example.lisp).

Before following the steps below, you must configure ASDF to find your systems. You can either copy or symlink the example directory in one of the standard ASDF locations, or push the path of the example directory to your asdf:*central-registry*, for example:

(push "./" asdf:*central-registry*)

3.1.2.2 Build it as an single executable ¶

Use this in the REPL to make an executable:

(asdf:make-build :example-with-dep
                 :type :program
                 :move-here #P"./"
                 :epilogue-code '(progn (example:test-function 5)
                                        (si:exit)))

Here the :epilogue-code is executed after loading our library; we can use arbitrary Lisp forms here. You can also put this code in your Lisp files and directly build them without this :epilogue-code option to achieve the same result. Running the program in a console will display the following and exit:

Factorial of 5 is: 120

3.1.2.3 Build it as shared library and use in C ¶

Use this in the REPL to make a shared library:

(asdf:make-build :example-with-dep
                 :type :shared-library
                 :move-here #P"./"
                 :monolithic t
                 :init-name "init_dll_example")

Here :monolithic t means that ECL will compile the library and all its dependencies into a single library named example-with-dep--all-systems.so. The :move-here parameter is self-explanatory. :init-name sets the name of the initialization function. Each library linked from C/C++ code must be initialized, and this is a mechanism to specify the initialization function’s name.

To use it, we write a simple C program:

/* test.c */
#include <ecl/ecl.h>
extern void init_dll_example(cl_object);

int main (int argc, char **argv) {
  
  cl_boot(argc, argv);
  ecl_init_module(NULL, init_dll_example);

  /* do things with the Lisp library */
  cl_eval(c_string_to_object("(example:test-function 5)"));

  cl_shutdown();
  return 0;
}

Compile the file using a standard C compiler (note we’re linking to libecl.so with -lecl, which provides the lisp runtime³):

gcc test.c example-with-dep--all-systems.so -o test -lecl

If ECL is installed in a non-standard location you may need to provide flags for the compiler and the linker. You may read them with:

ecl-config --cflags
ecl-config --libs

Since our shared object is not in the standard location, you need to provide LD_LIBRARY_PATH pointing to the current directory to run the application:

LD_LIBRARY_PATH=`pwd` ./test

This will show:

Factorial of 5 is: 120

You can also build all dependent libraries separately as a few .so files and link them together. For example, if you are building a library called complex-example, that depends on alexandria and cl-fad, you can do the following (in the REPL):

(asdf:make-build :complex-example
                 :type :shared-library
                 :move-here #P"./"
                 :init-name "init_example")

(asdf:make-build :alexandria
                 :type :shared-library
                 :move-here #P"./"
                 :init-name "init_alexandria")

(asdf:make-build :cl-fad
                 :type :shared-library
                 :move-here #P"./"
                 :init-name "init_fad")

(asdf:make-build :bordeaux-threads
                 :type :shared-library
                 :move-here #P"./"
                 :init-name "init_bt")

Note that we haven’t specified :monolithic t, so we need to build bordeaux-threads as well because cl-fad depends on it. The building sequence doesn’t matter and the resultant .so files can also be used in your future programs if these libraries are not modified.

We need to initialize all these modules using ecl_init_module in the correct order. (bordeaux-threads must be initialized before cl-fad; cl-fad and alexandria must be initialized before complex-ecample.)

Here is a code snippet (not a full program):

extern void init_fad(cl_object);
extern void init_alexandria(cl_object);
extern void init_bt(cl_object);
extern void init_example(cl_object);

/* call these *after* cl_boot(argc, argv); 
   if B depends on A, you should first init A then B. */
ecl_init_module(NULL, init_bt);
ecl_init_module(NULL, init_fad);
ecl_init_module(NULL, init_alexandria);
ecl_init_module(NULL, init_example);

3.1.2.4 Build it as static library and use in C ¶

To build a static library, use:

(asdf:make-build :example-with-dep
                 :type :static-library
                 :move-here #P"./"
                 :monolithic t
                 :init-name "init_example")

This will generate example-with-dep--all-systems.a in the current directory which we need to initialize with the init_example function. Compile it using:

gcc test.c example-with-dep--all-systems.a -o test-static -lecl

Then run it:

./test-static

This will show:

Factorial of 5 is: 120

Note we don’t need to pass the current path in LD_LIBRARY_PATH here, since our Lisp library is statically bundled with the executable. The result is the same as the shared library example above. You can also build all dependent libraries separately as static libraries.

3.1.3.1 Getting started with cross compilation ¶

To get started, follow the steps described below:

1. Cross compile ECL itself following the steps described in Building ECL.
2. Step one will create a Lisp file called target-info.lsp containing all the information necessary for the ECL compiler to create compiled files for the chosen target system. Load this information into a running ECL process on the host system by calling c:read-target-info on this file.
3. Supply the resulting target information to the :target option of compile-file or with-compilation-unit.

For an example, consider two files a.lisp and b.lisp which are supposed to be linked into a shared library. This can be accomplished with the following steps:

(defvar *target* (c:read-target-info "/path/to/ecl/installation/lib/ecl-xx.x.x/target-info.lsp"))

(compile-file "a.lisp" :target *target* :system-p t)
(load "a.lisp") ; make macro definitions in "a.lisp" accessible to "b.lisp"
(compile-file "b.lisp" :target *target* :system-p t)

(with-compilation-unit (:target *target*)
  (c:build-shared-library "example"
                          :lisp-files '("a.o" "b.o")
                          :init-name "init_example"))

Cross compilation using ASDF is not supported yet but is planned for the future.

3.1.3.2 Limitations and pitfalls for cross compiling Common Lisp ¶

The ubiquity with which typical Common Lisp programs run code at compile time in macros or other similar constructs makes cross compilation somewhat more challenging than in other programming languages. Since compilation happens on a different system than the one in which the code generated by a macro is run, differences between the host and target environment can lead to bugs if the macro hasn’t been written with cross compilation in mind. Consider for instance the following example macro iterating over prime number in the range from a to b:

(defmacro do-prime-numbers ((a b) p &body body)
  `(loop with ,p = (next-prime ,a)
         until (> ,p ,b)
         do ,(if (typep b 'fixnum)
                 `(locally (declare (fixnum ,p))
                    ,@body)
                 `(progn ,@body))
            (setf ,p (next-prime (1+ ,p)))))

While this macro works fine in a standard setting, it will produce an incorrect declaration when cross compiling if the size of a fixnum is larger in the host than in the target and the upper bound b is larger than most-positive-fixnum in the target but smaller than most-positive-fixnum in the host. In general, any observable difference between target and host system may contribute to such issues. In the list below, we collect a number of such potential issues and explain strategies to solve them.

• Different type or subtype definitions. This may arise from differing word sizes, as in the fixnum example above, or different feature support such as complex floating point number support in the C compiler leading to differing subtype relationships for complex numbers in Lisp. There are two main strategies for dealing with this problem.
  1. Avoid the problem by rewriting the code not to rely on target specific information. For instance, in the above example we can change the declaration as follows to avoid relying on fixnums:

     (defmacro do-prime-numbers ((a b) p &body body)
       `(loop with ,p = (next-prime ,a)
              until (> ,p ,b)
              do (locally (declare (type (integer ,a ,b) ,p))
                   ,@body)
                 (setf ,p (next-prime (1+ ,p)))))
     

     If the declared integer type is a subtype of fixnum, the ECL compiler will automatically take this into account and optimize the same as if a fixnum declaration had been made.

  2. Use the lexical environment parameter to typep and subtypep. Both functions take an optional parameter which can be used to supply information about type relationships in the target system. In the above example, this works as follows:

     (defmacro do-prime-numbers ((a b) p &body body &environment env)
       `(loop with ,p = (next-prime ,a)
              until (> ,p ,b)
              do ,(if (typep b 'fixnum env)
                      `(locally (declare (fixnum ,p))
                         ,@body)
                      `(progn ,@body))
                 (setf ,p (next-prime (1+ ,p)))))
     

     It is important, however, not to blindly modify every occurrence of typep and subtypep in this way. Some calls to these functions may check for types in the host environment, in which case they should not receive an environment parameter. It is necessary to know for which environment the call is meant in order to decide what to do.

• Different value of *features*, leading to mismatched #+ and #- read time conditionals. During cross compilation, *features* will be rebound to the value in the target system.⁴ However, macros that are defined in the host system will see the host *features* during read time. Consider again the example given above where a.lisp was cross compiled, then loaded before b.lisp was cross compiled. If a.lisp contains a macro definition

  (defmacro my-macro (...)
    #+android `(do-something ...)
    #-android `(do-something-else ...))
  

  then my-macro will expand to do-something in a.lisp but do-something-else in b.lisp if cross compiling for the android target. There are several ways to deal with this issue:

  1. Load source files containing macro definitions in an environment where *features* has the same value as in the target system. This can be accomplished either by wrapping load in a with-compilation-unit call with a :target option given or by using the :load keyword option of compile-file in conjunction with :target. In the above example, we would compile as follows:

     (compile-file "a.lisp" :target *target* :system-p t :load t)
     (compile-file "b.lisp" :target *target* :system-p t)
     

     Note that this will not work if the file contains read time conditionals selecting between different compile time code paths due to differences in the host system (let’s say for example some code generation from a file that is located in a different location for different compilation hosts).

  2. Replace read time conditionals by lookups that run during macro expansion, e.g.

     (defmacro my-macro (...)
       (if (member :android *features*)
           `(do-something ...)
           `(do-something-else ...)))
     

  3. Add feature keywords for the target system to *features* in the host before compilation. This is best suited for conditionals which only add but not subtract code (keywords which only appear in #+ conditionals). In our example, we would call

     (push :android *features*)
     

     before starting cross compilation.

• Floating point accuracy issues. ECL relies on the C standard library for numerical functions which may return slightly different values for different systems. If your code depends sensitively on numerical functions returning bitwise identical results, it is best not to run any numerical code at compile time and restrict yourself to only running run time code.

Another option to avoid the aformentioned issues entirely is to use emulation instead of cross compilation. Even if full-blown emulation is too complicated, simply cross compiling from a version of ECL compiled for the same word size as the target system would for instance avoid the fixnum issues in our example macro alltogether.

Whether the compiler is switched to cross compilation mode or not can be diagnosed from the presence of a :cross keyword in *features*. If necessary, this can be used to select different code paths as well.

3.1.4.1 Compiler flags ¶

It is not required to surround the compiler flags with quotes or use slashes before special characters.

Variable: string c:*user-cc-flags* ¶

Flags and options to be passed to the C compiler when building FASL, shared libraries and standalone programs.

Variable: string c:*user-linker-flags* ¶

Flags for options (e.g. -Wl,foo flags, usually in the $LDFLAGS variable in autoconf) to be passed to the linker when building FASL, shared libraries and standalone programs.

Variable: string c:*user-linker-libs* ¶

Flags for libraries (e.g. -lfoo flags, usually in the $LIBS variable in autoconf) to be passed to the linker when building FASL, shared libraries and standalone programs.

Variable: string c:*cc-optimize* ¶

Optimize options to be passed to the C compiler.

Variable: string c:*user-ld-flags* ¶

DEPRECATED Flags and options to be passed to the linker when building FASL, shared libraries and standalone programs.

3.1.4.2 Compiler & Linker programs ¶

Variable: string c::*cc* ¶

This variable controls how the C compiler is invoked by ECL. One can set the variable appropriately adding for instance flags which the C compiler may need to exploit special hardware features (e.g. a floating point coprocessor).

Variable: string c::*ld* ¶

This variable controls the linker which is used by ECL.

Variable: string c::*ranlib* ¶

Name of the ‘ranlib’ program on the hosting platform.

Variable: string c::*ar* ¶

Name of the ‘ar’ program on the hosting platform.

Variable: string c::*ecl-include-directory* ¶

Directory where the ECL header files for the target platform are located.

Variable: string c::*ecl-library-directory* ¶

Directory where the ECL library files for the target platform are located.