Subject: [clisp-list] CGI & binary i/o with stdio

From: Sam Steingold <[email-masked]>
Date: Sat May 3 14:42:40 2003

some users want to be able to do binary i/o on *standard-input* and
*standard-output* (CGI pretty much requires this).
There appears to be no portable way to do this in CLISP.
The usual way on Unix is (open "/dev/fd/0").
It should not be too hard to make this portable.
The question is: what should the interface be?
I.e., what argument to OPEN should open stdin (stdout, stderr)?
E.g., (OPEN :TERMINAL )?

CMUCL uses something like (fd-to-stream (dup 0)).
but 0/1/2 is a unixism.

Suggestions?

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat9 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
Growing Old is Inevitable; Growing Up is Optional.

---

From: Arseny Slobodjuck <[email-masked]>
Date: Sat May 3 19:29:46 2003

Hello Sam,

> Suggestions?
Command-line option ?

-- 
Best regards,
 Arseny

---

From: Sam Steingold <[email-masked]>
Date: Sun May 4 07:31:12 2003

> * In message <[email-masked]>
> * On the subject of "Re: [clisp-list] CGI & binary i/o with stdio"
> * Sent on Sun, 4 May 2003 13:30:59 +1000
> * Honorable Arseny Slobodjuck <[email-masked]> writes:
>

no - it should be possible to do

(with-open-stream (s (open-stdin-in-binary-mode))
  ...)

that's why I want a dup() there - so that the user will be able to
close the stream safely
  
-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat9 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
Daddy, why doesn't this magnet pick up this floppy disk?

---

From: Arseny Slobodjuck <[email-masked]>
Date: Sun May 4 14:53:13 2003

Hello Sam,

> that's why I want a dup() there - so that the user will be able to
  
(coerce s 'binary-terminal-stream) ? I'm full of ideas.
Btw the distinction between text and binary modes in lisp is in
element-type, right ?

-- 
Best regards,
 Arseny

---

From: Sam Steingold <[email-masked]>
Date: Sun May 4 15:36:07 2003

Hi Arseny,

> * In message <[email-masked]>
> * On the subject of "Re[2]: [clisp-list] CGI & binary i/o with stdio"
> * Sent on Mon, 5 May 2003 08:56:59 +1000

we are dealing with a stream opening paradigm.
what is a `handle' in win32?
is it just a glorified file descriptor (an unsigned int)?
in amiga?

if yes, we could just allow

        (open 0 :direction :input :element-type 'unsigned-byte)

(but what if the user would do :direction :output?)

> Btw the distinction between text and binary modes in lisp is in

yes.

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat9 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
nobody's life, liberty or property are safe while the legislature is in session

---

From: Arseny Slobodjuck <[email-masked]>
Date: Mon May 5 03:03:06 2003

Hello Sam,

> is it just a glorified file descriptor (an unsigned int)?
AFAIK, no. To support this:
(uint)GetStdHandle(STD_INPUT_HANDLE) == 3 (in my case).

> (but what if the user would do :direction :output?)

What about making make-terminal-stream user-level function with
parameters (as terminal-stream is not file-stream which open should
return) ? What about using *terminal-io* as filename argument to
open (barely legal) ? BTW, is *terminal-io* still of strmtype_terminal
when io is redirected ? I wasn't able to check it fast - result is
confusing. How about changing element-type on-the-fly
(setf element-type) ?

-- 
Best regards,
 Arseny

---

From: Sam Steingold <[email-masked]>
Date: Mon May 5 10:52:02 2003

> * In message <[email-masked]>
> * On the subject of "Re[4]: CGI & binary i/o with stdio"
> * Sent on Mon, 5 May 2003 21:05:55 +1000

yuk - so stdin is not 0?!
what are stdout and stderr?

> return) ?

good idea!
(make-stream object &key &allow-other-keys)
will make a stream out of OBJECT (a handle, :INPUT, :OUTPUT, :ERROR, or
:IO) with the &key's the same as for OPEN sans :DIRECTION.
actually, no, we do need :DIRECTION because there is no way to figure
out whether the handle is open for input or output or both.
or is there?!

> BTW, is *terminal-io* still of strmtype_terminal when io is redirected
yes.

> How about changing element-type on-the-fly (setf element-type) ?

changing the element-type of the terminal i/o on the fly is considered
dangerous because it screws up the user interface.

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat9 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
Don't ascribe to malice what can be adequately explained by stupidity.

---

From: Arseny Slobodjuck <[email-masked]>
Date: Mon May 5 14:11:19 2003

Hello Sam,

> what are stdout and stderr?
7 and b  (3+4, 3+4+4)

> good idea!
Thank you!

> :IO) with the &key's the same as for OPEN sans :DIRECTION.
Cool. But where to get handle in lisp ?

> out whether the handle is open for input or output or both.
I am not aware of it.

-- 
Best regards,
 Arseny

---

From: Sam Steingold <[email-masked]>
Date: Mon May 5 15:59:14 2003

> * In message <[email-masked]>
> * On the subject of "Re: [clisp-list] Re: Re[4]: CGI & binary i/o with stdio"
> * Sent on Tue, 6 May 2003 08:14:26 +1000

linux:open and posix:duplicate-handle.

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat9 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
Failure is not an option. It comes bundled with your Microsoft product.

---

From: Arseny Slobodjuck <[email-masked]>
Date: Tue May 6 06:55:12 2003

Hello Sam,

> linux:open and posix:duplicate-handle.

Wouldn't it be better to use streams as a source of handles ? I even
have written a function for it already (stream_lend_handle).

Unfortunately, I see no way to duplicate HANDLE in win32. Of course
I can be wrong here, but I have taken a (short) look to dup2 in MS CRT and
found support for this. DuplicateHandle creates new handle and it is being
written into structure referenced by int handle (remaining the same).
Had they been able to duplicate HANDLES they wouldn't have to do it
this way. IMHO...

-- 
Best regards,
 Arseny

---

From: Sam Steingold <[email-masked]>
Date: Tue May 6 07:04:07 2003

> * In message <[email-masked]>
> * On the subject of "Re[2]: [clisp-list] Re: Re[4]: CGI & binary i/o with stdio"
> * Sent on Wed, 7 May 2003 00:18:58 +1000

sure, we can use files too!

> Unfortunately, I see no way to duplicate HANDLE in win32. Of course I
> can be wrong here, but I have taken a (short) look to dup2 in MS CRT
> and found support for this. DuplicateHandle creates new handle and it
> is being written into structure referenced by int handle (remaining
> the same).  Had they been able to duplicate HANDLES they wouldn't have
> to do it this way. IMHO...

see LISPFUN(duplicate_handle) in pathname.d.
is it incorrect?

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat9 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
MS: our tomorrow's software will run on your tomorrow's HW at today's speed.

---

From: Arseny Slobodjuck <[email-masked]>
Date: Tue May 6 15:53:07 2003

Hello Sam,

> see LISPFUN(duplicate_handle) in pathname.d.
As far as I understand dup and dup2, it works only as dup, but not
as dup2 on win32. I.e. one cannot point existing handle to another
file, new handle should be allocated. Second parameter is ignored on
win32.

-- 
Best regards,
 Arseny

---

From: Sam Steingold <[email-masked]>
Date: Tue May 6 16:24:05 2003

hi Arseny & Marco,

> * In message <[email-masked]>
> * On the subject of "Re[4]: [clisp-list] Re: Re[4]: CGI & binary i/o with stdio"
> * Sent on Wed, 7 May 2003 09:57:06 +1000
> As far as I understand dup and dup2, it works only as dup, but not as
> dup2 on win32. I.e. one cannot point existing handle to another file,
> new handle should be allocated. Second parameter is ignored on win32.

linux:

[1]> (posix:duplicate-handle 1)
4
[2]> (posix:duplicate-handle 1)
5
[3]> (posix:duplicate-handle 1 5)
5
[4]> (posix:duplicate-handle 1 7)
7

what do you get on win32?

if the second argument is indeed ignored, we are in trouble as far as
compatibility goes...

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat9 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
To a Lisp hacker, XML is S-expressions with extra cruft.

---

From: Sam Steingold <[email-masked]>
Date: Tue May 6 16:39:06 2003

> * In message <[email-masked]>
> * On the subject of "Re: Re[4]: CGI & binary i/o with stdio"
> * Sent on 05 May 2003 13:42:43 -0400
> * I write:
> 

fcntl(fd,F_GETFL) will return that on unix.
what is the win32 and amiga (and acorn) analogue?

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat9 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
Stupidity, like virtue, is its own reward.

---

From: Arseny Slobodjuck <[email-masked]>
Date: Tue May 6 19:34:07 2003

Hi,

>> new handle should be allocated. Second parameter is ignored on win32.

Sam> linux:

[1]>> (posix:duplicate-handle 1)
Sam> 4
[2]>> (posix:duplicate-handle 1)
Sam> 5
[3]>> (posix:duplicate-handle 1 5)
Sam> 5
[4]>> (posix:duplicate-handle 1 7)
Sam> 7

Sam> what do you get on win32?

Sam> if the second argument is indeed ignored, we are in trouble as far as
Sam> compatibility goes...

*** - Win32 error 6 (ERROR_INVALID_HANDLE): The handle is invalid.
1. Break [2]> :a

[3]> (posix:duplicate-handle 3)
19
[4]> (posix:duplicate-handle 3)
23
[5]> (posix:duplicate-handle 3 7)
27

-- 
Best regards,
 Arseny                            mailto:[email-masked]