Subject: [clisp-list] How to suppress / redirect standard error?

From: Thomas Wager <[email-masked]>
Date: Fri Aug 23 04:46:05 2002

Hi,

I am thinking about the following problem for a couple of hours now, but
it seems I cannot get it work. I want to call a Unix command from within
CLISP and return its output as a list of strings. The following
definition does just what I want:

(defun system-call (cmd &rest args)
  (let ((in (ext:run-program cmd :arguments args
                                 :input nil
                                 :output :stream)))
    (let ((res
      (loop as l = (read-line in nil)
        while l
          do
        collect l)))
      (close in)
      res)))

However, if the Unix command produces an error message to standard error
I cannot catch (or discard it):

[16]> (system-call "/usr/bin/ls" "-l" "/nothing")
/nothing: No such file or directory
NIL

Any ideas?

Thanks,

Thomas

---

From: "John K. Hinsdale" <[email-masked]>
Date: Fri Aug 23 05:24:02 2002

> However, if the Unix command produces an error message to standard
 > error I cannot catch (or discard it):

One option that is kludgy but will at definitely work is to make the
following 2-line program (call it "combine_output"):

---- Cut here ---
#!/bin/sh
$* 2>&1
---- Cut here ---

Then use SYSTEM-CALL as you have defun'ed it, but using
"combine_output" as the program, and your real program as the first
argument:

(system-call "/path/to/comine_output" "/usr/bin/ls" "-l" "/nothing" "/more-nothing")

This worked for me (I got a list of two strings) on Linux.

If you want to _discard_ the errors (hmmmmmmm :) make a script called
"discard_errors" that does:
---- Cut here ---
#!/bin/sh
$* 2>/dev/null
---- Cut here ---

Then call as above.

BTW, what that "2" refers to is the Unix "standard error" stream.
Unix assigns the numbers 0,1,2 to three default streams, "input,"
"output" and "errors" (respectively) which it creates with every
program.  In the shell, you can re-direct where these go by referring
to the streams by number as above.  The directive "2>1" means "make
stream 2 (errors) go wherever stream 1 (normal output) goes."

In the case of merging the outputs, they will get interleaved in no
predictable order - this may create problems for you parsing them.
Because of buffering, Unix won't even guarantee that they appear in
the merged output in the order they were written to by the program
(!).  If you need access to the output and the error streams
separately, you may have to get uglier and save the errors to a
temp. file and read that.  If you're discarding the error stream
(hmmmm :) there's no problem.

There might be a better way but I thought I'd at least give you an
"out" to get through the day.

Good day!

---
John Hinsdale, Alma Mater Software, Inc., Tarrytown, NY 10591-3710 USA
[email-masked] | http://www.alma.com/staff/hin | +1 914 631 4690

---

From: Randall R Schulz <[email-masked]>
Date: Fri Aug 23 06:59:06 2002

John, Thomas,

A slight improvement on John's solution is to use the "exec" shell 
built-in. This command causes the shell to overlay or replace itself (using 
the Unix / POSIX "exec" system call) with the specified command. Otherwise, 
the shell will "fork" and create a sub-process to run that command. When 
there's only one command, this is a wasted overhead.

Thus the "comine_output" script would look like this:

-==--==--==--==-
#!/bin/sh
exec $* 2>&1
-==--==--==--==-

It's also a very good idea to quote the arguments individually. To do that, 
use this alternate variable expansion syntax:

-==--==--==--==-
#!/bin/sh
exec "$@" 2>&1
-==--==--==--==-

If you don't do this, arguments with spaces or shell metacharacters (there 
are many) will causes error or misinterpretations of your invocation.

The "exec" command should be available in all shells. Only 
bourne-shell-like shells have "$@". That list includes (at least) the 
Bourne Shell, ASH, BASH and KSH.

Good luck.

Randall Schulz
Mountain View, CA USA

At 05:23 2002-08-23, John K. Hinsdale wrote:

>John Hinsdale,

---

From: Marco Baringer <[email-masked]>
Date: Fri Aug 23 07:30:03 2002

>   (let ((in (ext:run-program cmd :arguments args
         ((in (ext:run-shell-command (format nil "~S ~{ \'~S\'~} 2>&1"
                                             cmd args)
                                     :input nil
                                     :output :stream)))
>       (loop as l = (read-line in nil)

this avoids having to create a seperate program, but will you will
have problems if elements of args contains #\' chars.

1) wouldn't it be nice to have an :ERROR-STREAM keyword arg ala :INPUT
   and :OUTPUT? (Sam, if you don't have objections i'd be happy to
   take a whack at this.)

2) Why can't one specify a stream as the arg to :OUTPUT or :INPUT?

-- 
-Marco
Ring the bells that still can ring.
Forget your perfect offering.
There is a crack in everything.
That's how the light gets in.
     -Leonard Cohen

---

From: "John K. Hinsdale" <[email-masked]>
Date: Fri Aug 23 07:51:02 2002

>   ((in (ext:run-shell-command (format nil "~S ~{ \'~S\'~} 2>&1"

Note that when Lisp is launching the command it needs to use a shell
that understands the "2>&1" syntax.  So /bin/csh and /bin/tcsh will
NOT work.  The shell used can probably be ensured by setting
environment varialbe SHELL to /bin/sh or un-setting it.

There's no ambiguity w/ the wrapper script as the shell is set
explitly with the first "#!/bin/sh" line which says what interpereter
to use to run the script.

I agree w/ all of what Randall Schultz said too.  More important than
the "exec" (forking's pretty cheap these days) is the use of the
syntax to prevent un-escaping of the shell arguments.

 >    and :OUTPUT? (Sam, if you don't have objections i'd be happy to

the right way.  If the underlying O/S has a two-output-stream model,
and the point of the call is to interface to the underlying O/S
program exec facility, both streams should be accessible explicitly.

Things get even worse when you start opening command pipes and want to
route their separate output/error streams to different places using
only the shell redirection syntax ...

---

---

From: Sam Steingold <[email-masked]>
Date: Fri Aug 23 08:03:02 2002

> * In message <[email-masked]>
> * On the subject of "Re: [clisp-list] How to suppress / redirect standard error?"
> * Sent on 23 Aug 2002 16:29:41 +0200
> * Honorable Marco Baringer <[email-masked]> writes:
>

go ahead!

> 2) Why can't one specify a stream as the arg to :OUTPUT or :INPUT?

a missing feature?

thanks!

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat7.3 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
Lisp: its not just for geniuses anymore.

---

From: Marco Baringer <[email-masked]>
Date: Fri Aug 23 08:44:03 2002

> only the shell redirection syntax ...

i'm confused as to what you mean by "the right way." say we add an
:ERROR-STREAM arg to EXT:RUN-SHELL-COMMAND (EXT:RUN-PROGRAM get's it
too). We also allow the args to :INPUT, :OUTPUT and :ERROR-STREAM to
be stream objects (or :TERMINAL, :STREAM, NIL or pathname objects with
the current semantics). As far as i can see this would allow for
arbitrary manipulation of pipes between programs. yes?

---

From: "John K. Hinsdale" <[email-masked]>
Date: Fri Aug 23 09:07:04 2002

> i'm confused as to what you mean by "the right way."

Right, by "right way" i meant exactly having the explicity additional
param, error-stream arg as has been proposed.  this in contrast with
the "less then right" way (that I admit I suggested :) of using the
shell's redirect syntax, which has the problems of:

	1 - is shell dependent for merging streams
	2 - doesn't quite work for pipes
	3 - incurs the argument escaping/quoting issues mentioned

 > say we add an
 > nbe stream objects (or :TERMINAL, :STREAM, NIL or pathname objects with

Yes, would solve (2) above in a way that is more flexible than can be
done even w/ Unix own shells.  I'm only speaking w/ knowledge of
csh/tcsh and sh, not bash which may have solved the problem.  But not
everyone will have bash. (e.g., Solaris)

BTW, if it's intened to allow "merging" the output and error streams,
there will still be the problem I mentioned earlier with the streams'
data being interleaved unpredicatbly because of how Unix buffers its
standard streams. I don't believe there's a solution to that other
than Lisp's I/O library changing the buffering policies, which is
perhaps not its business to do.  This is a long-standing Unix problem.

Hope this helps!!!

	---

---

From: Pascal Bourguignon <[email-masked]>
Date: Fri Aug 23 17:31:07 2002

> From: Marco Baringer <[email-masked]>
> Date: 23 Aug 2002 17:44:15 +0200
> 

While shortcuts like  :INPUT :OUTPUT and :ERROR may  be nice syntactic
suggar, why not design the facility to be able to handle any number of
file descriptor.  

What about something like:

     (ext:run-program "gpg"
                      :arguments '("--status-fd"     "3"
                                   "--logger-fd"     "4"
                                   "--passphrase-fd" "5" 
                                   "--command-fd"    "6"
                                   ;;; ... )         
                       :input  clear-text-stream
                       :output encrypted-stream
                       :error  std-err-stream
                       :fd (3 :out  status-stream)
                       :fd (4 :out  logger-stream)
                       :fd (5 :in   password-stream)
                       :fd (6 :in   command-stream)
                       )

-- 
__Pascal_Bourguignon__                   http://www.informatimago.com/
----------------------------------------------------------------------
 The name is Baud,...... James Baud.

---

From: Thomas Wager <[email-masked]>
Date: Sun Aug 25 23:56:02 2002

All,
thanks a lot for your feedback. I'm going with shell scripts now. For
the future I would also prefer to have an extra keyword argument :error
to handle the error stream seperate.
Thanks!

Thomas 

"John K. Hinsdale" wrote:
> -------------------------------------------------------
> This sf.net email is sponsored by: OSDN - Tired of that same old
> cell phone?  Get a new here for FREE!
> https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390
> _______________________________________________
> clisp-list mailing list
> [email-masked]
> https://lists.sourceforge.net/lists/listinfo/clisp-list