[Openmcl-devel] How to use lisp program in unix pipeline

[Gary Byers]

On Mon, 12 May 2003, Frank Sonnemans wrote:

> Hello all,
>
> I wrote a small lisp application which takes an xml file, filters it and
> writes a html file. What I now want to do is use this lisp application in
> several ways:
>
> Myfilter infile outfile
>
> Or
>
> Myfilter < infile > outfile
>
> Or
>
> Cat infile | Myfilter > outfile
>
> How do I do this with openmcl. As the startup time of openmcl is very short
> I would basically like to use it to replace my unreadable perl scripts.
>
> So far I got some results using a construct like:
>
> Cat Myfilter.lisp | openmcl -b

The support that exists for creating applications (especially applications
that don't behave like lisp development systems) is poorly modularized,
mostly undocumented, and in some cases poorly implemented.  (Other than
that, it's just fine.)

The general idea is that there's a CLOS class called CCL::APPLICATION
(and an interesting subclass of that class, called
CCL::LISP-DEVELOPMENT-SYSTEM); the special variable CCL:*APPLICATION*
is assumed to be bound to an instance of some type of APPLICATION
object, and certain application-specific behaviors (how arguments
are processed, how errors are handled, how command-line arguments
are processed) are determined by calling generic functions which
have methods specialized on the current application class (the value
of CCL:*APPLICATION*.)

The function SAVE-APPLICATION is ordinarily used to write a heap
image file (either the initial "dppccl.image"/"PPCCL" image created
during bootstrapping or a lisp image that's been customized and/or
augmented in some way.)  SAVE-APPLICATION additionally takes an
:APPLICATION-CLASS argument, whose value can be a class (presumed
to be a subclass of APPLICATION) or the name of such a class.
SAVE-APPLICATION arranges that when the saved image starts up,
*APPLICATION* will be bound to an instance of  that specified class
and will start up and behave according to methods specialized on
that class.

As straightforward as that might sound, reality's a lot messier:
there are some obviously customizable behaviors that don't go through
the *APPLICATION* protocol, there are ways of customizing behavior
that conflict with it, and there are probably things in
LISP-DEVELOPMENT-SYSTEM that should be in APPLICATION and things
in APPLICATION (or outside the protocol) that are probably specific
to LISP-DEVELOPMENT-SYSTEM.

The only example of any of this that I can think of is in
"ccl:examples;cocoa-application": it says that a COCOA-APPLICATION
(perhaps it'd be better to call it a COCOA-LISP-DEVELOPMENT-SYSTEM)
is a LISP-DEVELOPMENT-SYSTEM that doesn't try too hard to parse its
command-line arguments (when launched from the Finder, an application's
sole (?) argument is of the form "-psn_XXXX" : a process serial number
used to register the application with the window server.)

Some command-line arguments (things having to do with memory initialization
and with specifying the heap image to load, mostly) are processed by
the lisp kernel.  Those arguments that the kernel doesn't handle (including
argv[0], the kernel pathnmame) are used to initialize the variable
CCL::*COMMAND-LINE-ARGUMENT-LIST*.

When an application starts up, it does some initialization (some of
which could maybe be - but isn't - governed by *APPLICATION*) and then
calls the application's TOPLEVEL-FUNCTION method with the value
returned by calling the application's INIT-FILE method as an argument.
The TOPLEVEL-FUNCTION method for APPLICATION (defined in
"ccl:level-1;l1-readloop.lisp") parses and then processes the
command-line arguments.  An APPLICATION subclass is presumed to
have a COMMAND-LINE-ARGUMENTS instance variable, which is a getopt-like
table that defines the arguments that the application recognizes.



>
> Problem is that openmcl outputs it's welcome message and prompt before the
> output of my lisp program (see bottom of this message for example). This is
> unsuitable for the type of use I have in mind. Can I suppress the prompt and
> welcome message?

The TOPLEVEL-FUNCTION method for LISP-DEVELOPMENT-SYSTEM does
a CALL-NEXT-METHOD to handle the arguments, then runs some code
which finishes argument processing and then starts a read-eval-print
loop.  The "Welcome ..." banner gets printed unless the variable
CCL::*INHIBIT-GREETING* is set.

It arguably doesn't make much sense for the read-eval-print loop
to print a prompt if the -b/--batch option is in effect; there's
currently no easy way to suppress the printing of the prompt.

One could argue that the -b flag basically affects the behavior
of LISP-DEVELOPMENT-SYSTEM.  It's actually processed by the kernel,
in the belief that it affects how some standard streams are initialized
and therefore has to be handled early.  (It's desirable that streams
be initialized as early as possible, so that errors don't go into the
Stack Overflow Death Spiral: it's very hard to tell you that *ERROR-OUTPUT*
isn't a valid stream without trying to write to *ERROR-OUTPUT*)

>
> Secondly how do I access the command line parameters from my lisp program?
>

The built-in stuff uses the PARSE-APPLICATION-ARGUMENTS and PROCESS-
APPLICATION-ARGUMENTS.  PARSE-APPLICATION-ARGUMENTS returns 3 values:
an indication of whether or not an error occurred during parsing,
a "normalized" list of option specifiers and associated parameters,
and a list of any other arguments that aren't associated with options.
(As a LISP-DEVELOPMENT-SYSTEM, OpenMCL doesn't accept any such
"other arguments"; the idea is that some other application classes
might.

PROCESS-APPLICATION-ARGUMENTS takes the values returned by PARSE-
APPLICATION-ARGUMENTS as its arguments.  The method specialized
on the class APPLICATION exits with a "USAGE" message if an error
was returned from the parser of if the --help option was present.
The PROCESS-APPLICATION-ARGUMENTS method for LISP-DEVELOPMENT-SYSTEM
does some checking and uses the options returned by the parser
to set some special variables that're referenced later in the
startup process.

I don't believe that there's currently any way for an application
subclass to inherit command-line argument specifiers from its
superclasses.  That's kind of stupid: it may mean that a subclass
of LISP-DEVELOPMENT-SYSTEM that wants to add an argument or two
has to copy LISP-DEVELOPMENT-SYSTEM's COMMAND-LINE-ARGUMENTS
instance variable and push a thing or two on it.

If this protocol is inappropriate, you can avoid it by making
a PARSE-APPLICATION-ARGUMENTS method that returns three NIL
values and accessing CCL::*COMMAND-LINE-ARGUMENTS-LIST* later
in your TOPLEVEL-FUNCTION method.

> Regards,
>
>
> Frank
>
> Example command line session showing openmcl printing prompt:
>
> [Frank-Sonnemanss-Computer:~] frank% cat test.lisp
> (print "hello world")
> [Frank-Sonnemanss-Computer:~] frank% cat test.lisp | openmcl -b
> Welcome to OpenMCL Version (Beta: Darwin) 0.13.4!
> ?
> "hello world"
> "hello world"
>

It's possible to suppress the banner, and it should be possible
to suppress the r-e-p-l's prompt (perhaps via the --batch option.)

It sounds like your XML-HTML filter doesn't really want or need
the read-eval-print loop.

You -should- therefore be able to do something like:

(defclass xml-html-filter (ccl::application) ())

(defmethod ccl::toplevel-function ((app xml-html-filter) init-file)
  (declare (ignore init-file))
  ;;
  (let* ((parsed-xml-object (read-xml-file *standard-input)))
     (write-html-file parsed-xml-file *standard-output*)))

(defmethod ccl::application-error ((app xml-html-filter)
                                   condition error-pointer)
  (declare (ignore error-pointer)) ;; the stack frame where the error occurred
  (format *error-output* "~&Fatal error: ~a" condition)
  (quit -1))

? (save-application "xml-html-filter.image"
     :application-class 'xml-html-filter)

% openmcl xml-html-filter.image <xmlfile >htmlfile

The operative word is "should": I can't think of a reason that would
prevent that from working, but don't trust this stuff 100% and would
not be shocked if such a reason existed.


>
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/cgi-bin/mailman/listinfo/openmcl-devel
>
>

_______________________________________________
Openmcl-devel mailing list
Openmcl-devel at clozure.com
http://clozure.com/cgi-bin/mailman/listinfo/openmcl-devel