The Common Lisp Cookbook – Interfacing with your OS

The ANSI Common Lisp standard doesn’t mention this topic. (Keep in mind that it was written at a time where Lisp Machines were at their peak. On these boxes Lisp was your operating system!) So almost everything that can be said here depends on your OS and your implementation.

Accessing Environment variables

ASDF comes with a function that’ll allow you to look at Unix/Linux environment variables on a lot of different CL implementations:

* (uiop:getenv "HOME")
  "/home/edi"

Below is an example implementation:

* (defun my-getenv (name &optional default)
  "Obtains the current value of the POSIX environment variable NAME."
  (declare (type (or string symbol) name))
  (let ((name (string name)))
    (or #+abcl (ext:getenv name)
        #+ccl (ccl:getenv name)
        #+clisp (ext:getenv name)
        #+cmu (unix:unix-getenv name) ; since CMUCL 20b
        #+ecl (si:getenv name)
        #+gcl (si:getenv name)
        #+mkcl (mkcl:getenv name)
        #+sbcl (sb-ext:posix-getenv name)
        default)))
MY-GETENV
* (my-getenv "HOME")
"/home/edi"
* (my-getenv "HOM")
NIL
* (my-getenv "HOM" "huh?")
"huh?"

You should also note that some of these implementations also provide the ability to set these variables. These include ECL (si:setenv) and AllegroCL, LispWorks, and CLISP where you can use the functions from above together with setf. This feature might be important if you want to start subprocesses from your Lisp environment.

Also note that the Osicat library has the method (environment-variable "name"), on POSIX-like systems including Windows. It is also fset-able.

Accessing the command line arguments

Basics

Accessing command line arguments is implementation-specific but it appears most implementations have a way of getting at them. Roswell or external libraries (see next section) make it portable.

SBCL stores the arguments list in the special variable sb-ext:*posix-argv*

$ sbcl my-command-line-arg

….

* sb-ext:*posix-argv*

("sbcl" "my-command-line-arg")
*

More on using this to write standalone Lisp scripts can be found in the SBCL Manual

LispWorks has system:*line-arguments-list*

CL-USER> system:*line-arguments-list*
("/Users/cbrown/Projects/lisptty/tty-lispworks" "-init" "/Users/cbrown/Desktop/lisp/lispworks-init.lisp")

CMUCL has interesting extensions for manipulating the arguments

Here’s a quick function to return the argument strings list across multiple implementations:

(defun my-command-line ()
  (or
   #+SBCL *posix-argv*
   #+LISPWORKS system:*line-arguments-list*
   #+CMU extensions:*command-line-words*
   nil))

Now it would be handy to access them in a portable way and to parse them according to a schema definition.

Parsing command line arguments

We have a look at the Awesome CL list#scripting section and we’ll show how to use unix-opts.

(ql:quickload "unix-opts")

We can now refer to it with its opts nickname.

We first declare our arguments with opts:define-opts, for example

(opts:define-opts
    (:name :help
           :description "print this help text"
           :short #\h
           :long "help")
    (:name :level
           :description "The level of something (integer)."
           :short #\l
           :long "level"
           :arg-parser #'parse-integer))

Everything should be self-explanatory. Note that #'parse-integer is a built-in CL function.

Now we can parse and get them with opts:get-opts, which returns two values: the first is the list of valid options and the second the remaining free arguments. We then must use multiple-value-bind to catch everything:

(multiple-value-bind (options free-args)
    ;; There is no error handling yet (specially for options not having their argument).
    (opts:get-opts)

We can explore this by giving a list of strings (as options) to get-opts:

(multiple-value-bind (options free-args)
                   (opts:get-opts '("hello" "-h" "-l" "1"))
                 (format t "Options: ~a~&" options)
                 (format t "free args: ~a~&" free-args))
Options: (HELP T LEVEL 1)
free args: (hello)
NIL

If we put an unknown option, we get into the debugger. We refer you to unix-opts’ documentation and code sample to deal with erroneous options and other errors.

We can access the arguments stored in options with getf (it is a property list), and we can exit (in a portable way) with opts:exit. So, for example:

(multiple-value-bind (options free-args)
    ;; No error handling.
    (opts:get-opts)

  (if (getf options :help)
      (progn
        (opts:describe
         :prefix "My app. Usage:"
         :args "[keywords]")
        (exit))) ;; <= exit takes an optional return status.
    ...

And that’s it for now, you know the essential. See the documentation for a complete example, and the Awesome CL list for useful packages to use in the terminal (ansi colors, printing tables and progress bars, interfaces to readline,…).

Running external programs

uiop has us covered.

Synchronously

uiop:run-program either takes a string as argument, denoting the name of the executable to run, or a list of strings, for the program and its arguments:

(uiop:run-program "firefox")

or

(uiop:run-program '("firefox" "http:url"))

This will process the program output as specified and return the processing results when the program and its output processing are complete.

This function has the following optional arguments (the documentation is in the source):

run-program (command &rest keys &key
                         ignore-error-status
                         (force-shell nil force-shell-suppliedp)
                         input
                         (if-input-does-not-exist :error)
                         output
                         (if-output-exists :supersede)
                         error-output
                         (if-error-output-exists :supersede)
                         (element-type #-clozure *default-stream-element-type* #+clozure 'character)
                         (external-format *utf-8-external-format*)
                       &allow-other-keys)

It will always call a shell (rather than directly executing the command when possible) if force-shell is specified. Similarly, it will never call a shell if force-shell is specified to be nil.

Signal a continuable subprocess-error if the process wasn’t successful (exit-code 0), unless ignore-error-status is specified.

If output is a pathname, a string designating a pathname, or nil (the default) designating the null device, the file at that path is used as output. If it’s :interactive, output is inherited from the current process; beware that this may be different from your *standard-output*, and under slime will be on your *inferior-lisp* buffer. If it’s t, output goes to your current *standard-output* stream. Otherwise, output should be a value that is a suitable first argument to slurp-input-stream (qv.), or a list of such a value and keyword arguments. In this case, run-program will create a temporary stream for the program output; the program output, in that stream, will be processed by a call to slurp-input-stream, using output as the first argument (or the first element of output, and the rest as keywords). The primary value resulting from that call (or nil if no call was needed) will be the first value returned by run-program. E.g., using :output :string will have it return the entire output stream as a string. And using :output '(:string :stripped t) will have it return the same string stripped of any ending newline.

if-output-exists, which is only meaningful if output is a string or a pathname, can take the values :error, :append, and :supersede (the default). The meaning of these values and their effect on the case where output does not exist, is analogous to the if-exists parameter to open with :direction :output.

error-output is similar to output, except that the resulting value is returned as the second value of run-program. t designates the *error-output*. Also :output means redirecting the error output to the output stream, in which case nil is returned.

if-error-output-exists is similar to if-output-exist, except that it affects error-output rather than output.

input is similar to output, except that vomit-output-stream is used, no value is returned, and T designates the *standard-input*.

if-input-does-not-exist, which is only meaningful if input is a string or a pathname, can take the values :create and :error (the default). The meaning of these values is analogous to the if-does-not-exist parameter to open with :direction :input.

element-type and external-format are passed on to your Lisp implementation, when applicable, for creation of the output stream.

One and only one of the stream slurping or vomiting may or may not happen in parallel in parallel with the subprocess, depending on options and implementation, and with priority being given to output processing. Other streams are completely produced or consumed before or after the subprocess is spawned, using temporary files.

run-program returns 3 values:

  • the result of the output slurping if any, or nil
  • the result of the error-output slurping if any, or nil
  • either 0 if the subprocess exited with success status, or an indication of failure via the exit-code of the process

Asynchronously

With

uiop:launch-program

Its signature is the following:

launch-program (command &rest keys
                         &key
                           input
                           (if-input-does-not-exist :error)
                           output
                           (if-output-exists :supersede)
                           error-output
                           (if-error-output-exists :supersede)
                           (element-type #-clozure *default-stream-element-type*
                                         #+clozure 'character)
                           (external-format *utf-8-external-format*)
                           directory
                           #+allegro separate-streams
                           &allow-other-keys)

If output is a pathname, a string designating a pathname, or nil (the default) designating the null device, the file at that path is used as output. If it’s :interactive, output is inherited from the current process; beware that this may be different from your *standard-output*, and under Slime will be on your *inferior-lisp* buffer. If it’s T, output goes to your current *standard-output* stream. If it’s :stream, a new stream will be made available that can be accessed via process-info-output and read from. Otherwise, output should be a value that the underlying lisp implementation knows how to handle.

if-output-exists, which is only meaningful if output is a string or a pathname, can take the values :error, :append, and :supersede (the default). The meaning of these values and their effect on the case where output does not exist, is analogous to the if-exists parameter to open with :DIRECTION :output.

error-output is similar to output. T designates the *error-output*, :output means redirecting the error output to the output stream, and :stream causes a stream to be made available via process-info-error-output.

launch-program returns a process-info object, which look like the following (source):

(defclass process-info ()
    (
     ;; The advantage of dealing with streams instead of PID is the
     ;; availability of functions like `sys:pipe-kill-process`.
     (process :initform nil)
     (input-stream :initform nil)
     (output-stream :initform nil)
     (bidir-stream :initform nil)
     (error-output-stream :initform nil)
     ;; For backward-compatibility, to maintain the property (zerop
     ;; exit-code) <-> success, an exit in response to a signal is
     ;; encoded as 128+signum.
     (exit-code :initform nil)
     ;; If the platform allows it, distinguish exiting with a code
     ;; >128 from exiting in response to a signal by setting this code
     (signal-code :initform nil)))

See the docstrings.

Forking with CMUCL

Here’s a function by Martin Cracauer that’ll allow you to compile a couple of files in parallel with CMUCL. It demonstrates how to use the UNIX fork system call with this CL implementation.

(defparameter *sigchld* 0)

(defparameter *compile-files-debug* 2)

(defun sigchld-handler (p1 p2 p3)
  (when (> 0 *compile-files-debug*)
    (print (list "returned" p1 p2 p3))
    (force-output))
  (decf *sigchld*))

(defun compile-files (files &key (load nil))
  (setq *sigchld* 0)
  (system:enable-interrupt unix:sigchld #'sigchld-handler)
  (do ((f files (cdr f)))
      ((not f))
    (format t "~&process ~d diving for ~a" (unix:unix-getpid)
            `(compile-file ,(car f)))
    (force-output)
    (let ((pid (unix:unix-fork)))
      (if (/= 0 pid)
          ;; parent
          (incf *sigchld*)
          ;; child
          (progn
            (compile-file (car f) :verbose nil :print nil)
            (unix:unix-exit 0)))))
  (do () ((= 0 *sigchld*))
    (sleep 1)
    (when (> 0 *compile-files-debug*)
      (format t "~&process ~d still waiting for ~d childs"
              (unix:unix-getpid) *sigchld*)))
  (when (> 0 *compile-files-debug*)
    (format t "~&finished"))
  (when load
    (do ((f files (cdr f)))
        ((not f))
      (load (compile-file-pathname (car f))))))

Page source: os.md