Subject: [clisp-list] socket-connect's timeout

From: [email-masked]
Date: Fri May 24 11:36:01 2002

Hello!

I've got a little problem with sockets.  When I use
socket:socket-connect and the site I want to connect to is down or
non-existent it takes a forever till my program can continue.

I think a solution would be for me if I could specify a timeout to
socket-connect.  Is that a missing feature or am I off the track?  (I
don't have experience in socket programming in any other language so
it's possible I'm missing something obvious here.)

How can I solve this situation?

Thanks in advance.

-- 
ignotus
                             Microsoft -- programs so large they have weather

---

From: Sam Steingold <[email-masked]>
Date: Fri May 24 12:21:02 2002

> * In message <[email-masked]>
> * On the subject of "[clisp-list] socket-connect's timeout"
> * Sent on Fri, 24 May 2002 20:34:30 +0200
> * Honorable [email-masked] writes:
>
> socket-connect.  Is that a missing feature or am I off the track?

At the moment, there is no way to do this, which, I think, is the only
operation where blocking is inevitable now (we have socket-wait,
socket-status and listen, which will help you avoid blocking in all
other cases).

This is a nice feature to have, so I would appreciate it if someone
could tell me how to set connection timeout for a socket.  I suspect
that there is a setsockopt() option for that (SO_RCVTIMEO?), but I am
not sure.  Do we have a "dedicated TCP/IP guru" here?

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat7.2 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
Bus error -- please leave by the rear door.

---

From: Jason Lowdermilk <[email-masked]>
Date: Fri May 24 13:01:05 2002

I don't think you can specify the connection timeout, exactly. You can
roll your own by setting the socket to non-blocking (ioctl w/ FIONBIO)
before calling "connect"; then if connect sets errno to EINPROGRESS you
can use "select" with a timeout to implement your own connection
timeout.

Jason

On Fri, 2002-05-24 at 13:20, Sam Steingold wrote:
[snip]
> is a nice feature to have, so I would appreciate it if someone

---

From: John Jorgensen <[email-masked]>
Date: Fri May 24 13:30:03 2002

> not sure.  Do we have a "dedicated TCP/IP guru" here?

Not that I'm an expert on this, but the last time I had to do this in C on
linux, I used an alarm. The SO_SNDTIMEO and SO_RCVTIMEO options may or may
not work in linux 2.4.x... I read conflicting reports. But I thought those
timeouts were for use after the connection is opened.

J*

---

From: Sam Steingold <[email-masked]>
Date: Fri May 24 13:42:03 2002

> * In message <[email-masked]>
> * On the subject of "Re: [clisp-list] socket-connect's timeout"
> * Sent on 24 May 2002 15:20:27 -0400
> * I write:
>
> could tell me how to set connection timeout for a socket. 
> Do we have a "dedicated TCP/IP guru" here?

Thanks to Jason and John for their insights!
The fact that there are two very different suggestions is quite scary:
it basically means that there is no standard way to do this.

I wonder if any application actually does this.

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat7.2 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
Don't use force -- get a bigger hammer.

---

From: John Wiseman <[email-masked]>
Date: Fri May 24 16:39:02 2002

--- Sam Steingold <[email-masked]> wrote:

> Thanks to Jason and John for their insights!  The fact that there
> are two very different suggestions is quite scary: it basically
> means that there is no standard way to do this.
> 

Yes.  I implemented it for cmucl.  See
<http://lemonodor.com/archives/000019.html>.

John Wiseman

__________________________________________________
Do You Yahoo!?
LAUNCH - Your Yahoo! Music Experience
http://launch.yahoo.com

---

From: John Jorgensen <[email-masked]>
Date: Fri May 24 17:27:05 2002

> <http://lemonodor.com/archives/000019.html>.

And somewhere in a Stevens book, I saw the alarm technique. I'll try to
find an example. It looks like John Wiseman's suggestion is better though.

J*

---

From: Arseny Slobodjuck <[email-masked]>
Date: Fri May 24 17:44:02 2002

Hello Sam,

>> Do we have a "dedicated TCP/IP guru" here?

Sam> Thanks to Jason and John for their insights!
Sam> The fact that there are two very different suggestions is quite scary:
Sam> it basically means that there is no standard way to do this.

IANATCPIPG, but Jason's approach seems standard - it's already used in
socket-wait. MSDN:

connect
.....

If  no  error  occurs,  connect  returns  zero.  Otherwise, it returns
SOCKET_ERROR,  and  a  specific error code can be retrieved by calling
WSAGetLastError.

On a blocking socket, the return value indicates success or failure of
the connection attempt.

With  a nonblocking socket, the connection attempt cannot be completed
immediately.  In  this  case,  connect  will  return SOCKET_ERROR, and
WSAGetLastError  will  return  WSAEWOULDBLOCK. In this case, there are
three possible scenarios:

--  Use  the  select  function  to  determine  the  completion  of the
connection request by checking to see if the socket is writeable.
-- ... (windows specific)
-- ... (windows specific)

-- 
Best regards,
 Arseny                            mailto:[email-masked]

---

From: Todd Sabin <[email-masked]>
Date: Sat May 25 08:18:02 2002

> not sure.  Do we have a "dedicated TCP/IP guru" here?

As others have suggested, I think the right way to do this is by
putting the socket in non-blocking mode for the connect call.
(Coincidentally, I mentioned the same issue a couple days ago on the
-devel list in the thread about shutdown.)  If you're going to go that
route, can I suggest that the actual timeout be done in lisp with
socket-status instead of in C?  I.e., do something like make
a socket-connect% which takes a non-blocking option, and then
have (very rougly)

(defun socket-connect (... &key timeout)
   ...
   (let ((stream (socket-connect% ....)))
     (when timeout
       (socket-status (stream . output) timeout))
     stream))

The point being to make it possible to do a non-blocking
socket-connect, but letting the user have the option of doing his own
timeout error handling later.

Todd

p.s.  Even with non-blocking connect, there's still the possibility of
blocking during name resolution.  Sigh.

---

From: Sam Steingold <[email-masked]>
Date: Sun May 26 14:16:02 2002

all interested parties please try the appended patch.
I tested it only on linux, so I am especially interested in what other
platform users think.
Specifically win32: what's the MS-way to do ioctl/FIONBIO?

basically, I added the :timeout keyword arg to SOCKET-ACCEPT and
SOCKET-CONNECT so that

  (socket-accept s :timeout x) ==

  (if (socket-wait s x)
      (socket-accept s)
      (error 'timeout))

and similarly for SOCKET-CONNECT.

Todd, this is not what you want, I know, you prefer the UNIX way of
returning NIL instead of signaling an error.  The lisp way (as I
understand it) is to return a valid object or to signal an error.

You can get the behavior you want using IGNORE-ERRORS, while a careless
user might get stuck with SOCKET-CONNECT returning NIL here:

  (let ((s (socket-connect 80 "www.foo.com" :timeout 1)))
    (format s "GET /~2%")
    (read-all s))

if S is NIL, format will succeed, while READ-ALL will probably block
trying to read from *STANDARD-INPUT*.

Live free or die.

Index: socket.d
===================================================================
RCS file: /cvsroot/clisp/clisp/src/socket.d,v
retrieving revision 1.58
diff -u -w -b -r1.58 socket.d
--- socket.d	22 May 2002 17:39:34 -0000	1.58
+++ socket.d	26 May 2002 20:08:37 -0000
@@ -308,13 +308,10 @@
 
 # Look up a host's IP address, then call a user-defined function taking
 # a `struct sockaddr' and its size, and returning a SOCKET.
-typedef SOCKET (*socket_connect_fn) (struct sockaddr * addr, int addrlen);
-local SOCKET with_hostname (const char* host, unsigned short port, socket_connect_fn connector);
-local SOCKET with_hostname(host,port,connector)
-  var const char* host;
-  var unsigned short port;
-  var socket_connect_fn connector;
-  {
+typedef SOCKET (*socket_connect_fn_t) (struct sockaddr * addr, int addrlen,
+                                       void* opts);
+local SOCKET with_hostname (const char* host, unsigned short port,
+                            socket_connect_fn_t connector, void* opts) {
     #ifdef HAVE_INET_PTON
     #ifdef HAVE_IPV6
     {
@@ -322,7 +319,8 @@
       if (inet_pton(AF_INET6,host,&inaddr.sin6_addr) > 0) {
         inaddr.sin6_family = AF_INET6;
         inaddr.sin6_port = htons(port);
-        return connector((struct sockaddr *) &inaddr, sizeof(struct sockaddr_in6));
+      return connector((struct sockaddr *) &inaddr,
+                       sizeof(struct sockaddr_in6), opts);
       }
     }
     #endif
@@ -331,7 +329,8 @@
       if (inet_pton(AF_INET,host,&inaddr.sin_addr) > 0) {
         inaddr.sin_family = AF_INET;
         inaddr.sin_port = htons(port);
-        return connector((struct sockaddr *) &inaddr, sizeof(struct sockaddr_in));
+      return connector((struct sockaddr *) &inaddr,
+                       sizeof(struct sockaddr_in), opts);
       }
     }
     #else
@@ -344,7 +343,8 @@
         inaddr.sin_family = AF_INET;
         inaddr.sin_addr.s_addr = hostinetaddr;
         inaddr.sin_port = htons(port);
-        return connector((struct sockaddr *) &inaddr, sizeof(struct sockaddr_in));
+      return connector((struct sockaddr *) &inaddr,
+                       sizeof(struct sockaddr_in), opts);
       }
     }
     #endif
@@ -361,7 +361,8 @@
         inaddr.sin6_family = AF_INET6;
         inaddr.sin6_addr = *(struct in6_addr *)host_ptr->h_addr;
         inaddr.sin6_port = htons(port);
-        return connector((struct sockaddr *) &inaddr, sizeof(struct sockaddr_in6));
+      return connector((struct sockaddr *) &inaddr,
+                       sizeof(struct sockaddr_in6), opts);
       } else
       #endif
       if (host_ptr->h_addrtype == AF_INET) {
@@ -370,9 +371,10 @@
         inaddr.sin_family = AF_INET;
         inaddr.sin_addr = *(struct in_addr *)host_ptr->h_addr;
         inaddr.sin_port = htons(port);
-        return connector((struct sockaddr *) &inaddr, sizeof(struct sockaddr_in));
-      } else {
-        sock_set_errno(EPROTOTYPE); return INVALID_SOCKET; # Not an Internet host!
+      return connector((struct sockaddr *) &inaddr,
+                       sizeof(struct sockaddr_in), opts);
+    } else { # Not an Internet host!
+      sock_set_errno(EPROTOTYPE); return INVALID_SOCKET;
       }
     }
   }
@@ -435,21 +437,19 @@
 #endif
 
 #ifdef TCPCONN
-local SOCKET connect_to_x_via_ip (struct sockaddr * addr, int addrlen);
-local SOCKET connect_to_x_via_ip(addr,addrlen)
-  var struct sockaddr * addr;
-  var int addrlen;
-  {
+local SOCKET connect_to_x_via_ip (struct sockaddr * addr, int addrlen,
+                                  void* ignore) {
     var SOCKET fd;
     var int retries = 3; # number of retries on ECONNREFUSED
+  (void)(ignore); # no options -- ignore
     do {
       if ((fd = socket((int) addr->sa_family, SOCK_STREAM, 0)) == INVALID_SOCKET)
         return INVALID_SOCKET;
       #ifdef TCP_NODELAY
-        # turn off TCP coalescence (the bandwidth saving Nagle algorithm)
-        {
+    { # turn off TCP coalescence (the bandwidth saving Nagle algorithm)
           int tmp = 1;
-          setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, (SETSOCKOPT_ARG_T)&tmp, sizeof(int));
+      setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, (SETSOCKOPT_ARG_T)&tmp,
+                 sizeof(int));
         }
       #endif
       # Connect to the socket.
@@ -562,9 +562,9 @@
     var unsigned short port = X_TCP_PORT+display;
     if (host[0] == '\0') {
       get_hostname(host =);
-      fd = with_hostname(host,port,&connect_to_x_via_ip);
+      fd = with_hostname(host,port,&connect_to_x_via_ip,NULL);
     } else {
-      fd = with_hostname(host,port,&connect_to_x_via_ip);
+      fd = with_hostname(host,port,&connect_to_x_via_ip,NULL);
     }
     if (fd == INVALID_SOCKET)
       return INVALID_SOCKET;
@@ -717,13 +717,12 @@
 #   This can (and should) be done multiple times for the same
 #   socket_handle.
 
-global SOCKET create_server_socket (host_data *hd, SOCKET sock, unsigned int port);
-local SOCKET bindlisten_via_ip (struct sockaddr * addr, int addrlen);
-local SOCKET bindlisten_via_ip(addr,addrlen)
-  var struct sockaddr * addr;
-  var int addrlen;
-  {
+global SOCKET create_server_socket (host_data *hd, SOCKET sock,
+                                    unsigned int port);
+local SOCKET bindlisten_via_ip (struct sockaddr * addr, int addrlen,
+                                void* ignore) {
     var SOCKET fd;
+  (void)(ignore); # no options -- ignore
     # Get a socket.
     if ((fd = socket((int) addr->sa_family, SOCK_STREAM, 0)) == INVALID_SOCKET)
       return INVALID_SOCKET;
@@ -733,7 +732,8 @@
     # will still yield an error.)
     {
       var unsigned int flag = 1;
-      if (setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, (SETSOCKOPT_ARG_T)&flag, sizeof(flag)) < 0) {
+    if (setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, (SETSOCKOPT_ARG_T)&flag,
+                   sizeof(flag)) < 0) {
         saving_sock_errno(CLOSESOCKET(fd)); return INVALID_SOCKET;
       }
     }
@@ -756,7 +756,8 @@
     var SOCKET fd;
     if (sock == INVALID_SOCKET) {
       # "0.0.0.0" allows connections from any host to our server
-      fd = with_hostname("0.0.0.0",(unsigned short)port,&bindlisten_via_ip);
+      fd = with_hostname("0.0.0.0",(unsigned short)port,&bindlisten_via_ip,
+                         NULL);
     } else {
       var sockaddr_max addr;
       var SOCKLEN_T addrlen = sizeof(sockaddr_max);
@@ -773,7 +774,7 @@
           break;
         default: NOTREACHED;
       }
-      fd = bindlisten_via_ip((struct sockaddr *)&addr,addrlen);
+      fd = bindlisten_via_ip((struct sockaddr *)&addr,addrlen,NULL);
     }
     if (fd == INVALID_SOCKET)
       return INVALID_SOCKET;
@@ -796,32 +797,56 @@
   }
 
 # Creation of sockets on the client side:
-# SOCKET fd = create_client_socket(hostname,port);
+# SOCKET fd = create_client_socket(hostname,port,void* timeout);
 #   creates a connection to a server (which must be waiting
 #   on the specified host and port).
 
-global SOCKET create_client_socket (const char* hostname, unsigned int port);
-local SOCKET connect_via_ip (struct sockaddr * addr, int addrlen);
-local SOCKET connect_via_ip(addr,addrlen)
-  var struct sockaddr * addr;
-  var int addrlen;
-  {
+global SOCKET create_client_socket (const char* hostname, unsigned int port,
+                                    void* timeout);
+local SOCKET connect_via_ip (struct sockaddr * addr, int addrlen,
+                             void* timeout) {
+  # <http://cr.yp.to/docs/connect.html>:
+  # - make a non-blocking socket, connect(), select() for WR
     var SOCKET fd;
     if ((fd = socket((int) addr->sa_family, SOCK_STREAM, 0)) == INVALID_SOCKET)
       return INVALID_SOCKET;
    #ifdef WIN32_NATIVE
     if (!lingerize_socket(&fd)) return INVALID_SOCKET;
    #endif
+ #ifdef FIONBIO
+  if (timeout) {
+    var int non_blocking_io = 1;
+    if (ioctl(fd,FIONBIO,&non_blocking_io) != 0) { return INVALID_SOCKET; }
+  }
+ #endif
     if (connect(fd, addr, addrlen) >= 0)
       return fd;
+ #ifdef FIONBIO
+  if (sock_errno_is(EINPROGRESS)) {
+   restart_select:
+    var fd_set handle_set;
+    FD_ZERO(&handle_set); FD_SET(fd,&handle_set);
+    var int ret = select(FD_SETSIZE,NULL,&handle_set,NULL,
+                         (struct timeval*)timeout);
+    if (ret < 0) {
+      if (sock_errno_is(EINTR)) goto restart_select;
+      saving_sock_errno(CLOSESOCKET(fd)); return INVALID_SOCKET;
+    }
+    if (ret == 0) { # timeout
+      CLOSESOCKET(fd); errno = ETIMEDOUT;
+      return INVALID_SOCKET;
+    }
+    if (connect(fd, addr, addrlen) >= 0)
+      return fd;
+  }
+ #endif
     saving_sock_errno(CLOSESOCKET(fd));
     return INVALID_SOCKET;
   }
-global SOCKET create_client_socket(hostname,port)
-  var const char* hostname;
-  var unsigned int port;
-  {
-    return with_hostname(hostname,(unsigned short)port,&connect_via_ip);
+global SOCKET create_client_socket (const char*hostname, unsigned intport,
+                                    void* timeout) {
+  return with_hostname(hostname,(unsigned short)intport,
+                       &connect_via_ip,timeout);
   }
 
 # ==================== miscellaneous network related stuff ====================
Index: stream.d
===================================================================
RCS file: /cvsroot/clisp/clisp/src/stream.d,v
retrieving revision 1.280
diff -u -w -b -r1.280 stream.d
--- stream.d	23 May 2002 19:26:59 -0000	1.280
+++ stream.d	26 May 2002 20:08:44 -0000
@@ -14730,15 +14730,80 @@
   skipSTACK(1);
 }
 
+# parse timeout argument
+# sec = posfixnum or (SEC . USEC) or (SEC USEC) or float or ratio
+#       or nil/unbound
+# usec = posfixnum or nil/unbound
+# can trigger GC
+local struct timeval * sec_usec (object sec, object usec, struct timeval *tv) {
+  if (eq(sec,unbound) || eq(sec,NIL)) {
+    return NULL;
+  } else if (consp(sec)) {
+    if (!nullp(Cdr(sec)) && eq(usec,unbound))
+      usec = (consp(Cdr(sec)) ? Car(Cdr(sec)) : Cdr(sec));
+    sec = Car(sec);
+  } else if (floatp(sec) || ratiop(sec)) { # sec = sec mod 1
+    pushSTACK(sec); funcall(L(floor),1);
+    sec = value1;
+    if (eq(usec,unbound)) { # usec = round(sec*1000000)
+      pushSTACK(subr_self); # save subr_self
+      pushSTACK(value2); pushSTACK(fixnum(1000000)); funcall(L(mal),2);
+      pushSTACK(value1); funcall(L(round),1);
+      subr_self = popSTACK(); # restore subr_self
+      usec = value1;
+    }
+  }
+  if (!posfixnump(sec))
+    fehler_posfixnum(sec);
+  tv->tv_sec = posfixnum_to_L(sec);
+  if (eq(usec,unbound) || eq(usec,NIL)) {
+    tv->tv_usec = 0;
+  } else {
+    if (!posfixnump(usec))
+      fehler_posfixnum(usec);
+    tv->tv_usec = posfixnum_to_L(usec);
+  }
+  return tv;
+}
+
+#if defined(HAVE_SELECT) || defined(WIN32_NATIVE)
+# wait for the socket server to have a connection ready
+# returns true iff socket_accept will return immediately
+local bool socket_server_wait (object sose, struct timeval *tvp) {
+  var SOCKET handle = TheSocket(TheSocketServer(sose)->socket_handle);
+ #if defined(WIN32_NATIVE)
+  return interruptible_wait(handle,tvp);
+ #else
+ restart_select:
+  begin_system_call();
+  var int ret;
+  var fd_set handle_set;
+  FD_ZERO(&handle_set); FD_SET(handle,&handle_set);
+  ret = select(FD_SETSIZE,&handle_set,NULL,NULL,tvp);
+  if (ret < 0) {
+    if (sock_errno_is(EINTR)) {
+      end_system_call(); goto restart_select;
+    }
+    SOCK_error();
+  }
+  end_system_call();
+  return (ret != 0);
+ #endif # WIN32_NATIVE
+}
+#endif
+
 extern SOCKET accept_connection (SOCKET socket_handle);
 
-# (SOCKET-ACCEPT socket-server [:element-type] [:external-format] [:buffered])
-LISPFUN(socket_accept,1,0,norest,key,3,
-        (kw(element_type),kw(external_format),kw(buffered)) ) {
+# (SOCKET-ACCEPT socket-server [:element-type] [:external-format] [:buffered]
+#                [:timeout])
+LISPFUN(socket_accept,1,0,norest,key,4,
+        (kw(element_type),kw(external_format),kw(buffered),kw(timeout)) ) {
   var SOCKET sock;
   var decoded_el_t eltype;
   var signean buffered;
   var SOCKET handle;
+  var struct timeval tv;
+  var struct timeval *tvp = sec_usec(popSTACK(),unbound,&tv);
 
   test_socket_server(STACK_3,true);
 
@@ -14753,6 +14818,12 @@
   # Check and canonicalize the :EXTERNAL-FORMAT argument:
   STACK_1 = test_external_format_arg(STACK_1);
 
+ #if defined(HAVE_SELECT) || defined(WIN32_NATIVE)
+  if (tvp && !socket_server_wait(STACK_3,tvp)) { # handle :TIMEOUT
+    skipSTACK(4); sock_set_errno(ETIMEDOUT); OS_error();
+  }
+ #endif
+
   sock = TheSocket(TheSocketServer(STACK_3)->socket_handle);
   begin_system_call();
   handle = accept_connection (sock);
@@ -14766,52 +14837,13 @@
   skipSTACK(4);
 }
 
-local struct timeval * sec_usec (object sec, object usec, struct timeval *tv) {
-  if (eq(sec,unbound) || eq(sec,NIL)) {
-    return NULL;
-  } else {
-    if (!posfixnump(sec))
-      fehler_posfixnum(sec);
-    tv->tv_sec = posfixnum_to_L(sec);
-    if (eq(usec,unbound) || eq(usec,NIL)) {
-      tv->tv_usec = 0;
-    } else {
-      if (!posfixnump(usec))
-        fehler_posfixnum(usec);
-      tv->tv_usec = posfixnum_to_L(usec);
-    }
-    return tv;
-  }
-}
-
 # (SOCKET-WAIT socket-server [seconds [microseconds]])
 LISPFUN(socket_wait,1,2,norest,nokey,0,NIL) {
+  test_socket_server(STACK_2,true);
  #if defined(HAVE_SELECT) || defined(WIN32_NATIVE)
-  var SOCKET handle;
   var struct timeval timeout;
   var struct timeval * timeout_ptr = sec_usec(STACK_1,STACK_0,&timeout);
-  test_socket_server(STACK_2,true);
-  handle = TheSocket(TheSocketServer(STACK_2)->socket_handle);
- #if defined(WIN32_NATIVE)
-  value1 = interruptible_wait(handle,timeout_ptr) ? T : NIL;
- #else
- restart_select:
-  begin_system_call();
-  {
-    var int ret;
-    var fd_set handle_set;
-    FD_ZERO(&handle_set); FD_SET(handle,&handle_set);
-    ret = select(FD_SETSIZE,&handle_set,NULL,NULL,timeout_ptr);
-    if (ret < 0) {
-      if (sock_errno_is(EINTR)) {
-        end_system_call(); goto restart_select;
-      }
-      SOCK_error();
-    }
-    end_system_call();
-    value1 = (ret == 0) ? NIL : T;
-  }
- #endif # WIN32_NATIVE
+  value1 = socket_server_wait(STACK_2,timeout_ptr) ? T : NIL;
  #else
   value1 = NIL;
  #endif
@@ -14819,15 +14851,19 @@
   skipSTACK(3);
 }
 
-extern SOCKET create_client_socket (const char* host, unsigned int port);
+extern SOCKET create_client_socket (const char* host, unsigned int port,
+                                    void* timeout);
 
-# (SOCKET-CONNECT port [host] [:element-type] [:external-format] [:buffered])
+# (SOCKET-CONNECT port [host] [:element-type] [:external-format] [:buffered]
+#                 [:timeout])
 LISPFUN(socket_connect,1,1,norest,key,3,
-        (kw(element_type),kw(external_format),kw(buffered)) ) {
+        (kw(element_type),kw(external_format),kw(buffered),kw(timeout)) ) {
   var char *hostname;
   var decoded_el_t eltype;
   var signean buffered;
   var SOCKET handle;
+  var struct timeval tv;
+  var struct timeval *tvp = sec_usec(popSTACK(),unbound,&tv);
 
   if (!posfixnump(STACK_4))
     fehler_posfixnum(STACK_4);
@@ -14851,7 +14887,7 @@
     fehler_string(STACK_3);
 
   begin_system_call();
-  handle = create_client_socket(hostname,posfixnum_to_L(STACK_4));
+  handle = create_client_socket(hostname,posfixnum_to_L(STACK_4),tvp);
   if (handle == INVALID_SOCKET) { SOCK_error(); }
   end_system_call();
   value1 = make_socket_stream(handle,&eltype,buffered,
Index: subr.d
===================================================================
RCS file: /cvsroot/clisp/clisp/src/subr.d,v
retrieving revision 1.92
diff -u -w -b -r1.92 subr.d
--- subr.d	21 May 2002 14:57:05 -0000	1.92
+++ subr.d	26 May 2002 20:08:45 -0000
@@ -964,12 +964,12 @@
 LISPFUN(socket_server,0,1,norest,nokey,0,NIL)
 LISPFUNN(socket_server_port,1)
 LISPFUNN(socket_server_host,1)
-LISPFUN(socket_accept,1,0,norest,key,3,\
-        (kw(element_type),kw(external_format),kw(buffered)) )
+LISPFUN(socket_accept,1,0,norest,key,4,\
+        (kw(element_type),kw(external_format),kw(buffered),kw(timeout)) )
 LISPFUN(socket_wait,1,2,norest,nokey,0,NIL)
 LISPFUN(socket_status,1,2,norest,nokey,0,NIL)
-LISPFUN(socket_connect,1,1,norest,key,3,\
-        (kw(element_type),kw(external_format),kw(buffered)) )
+LISPFUN(socket_connect,1,1,norest,key,4,\
+        (kw(element_type),kw(external_format),kw(buffered),kw(timeout)) )
 LISPFUNN(socket_stream_port,1)
 LISPFUNN(socket_stream_host,1)
 LISPFUN(socket_stream_peer,1,1,norest,nokey,0,NIL)
Index: subrkw.d
===================================================================
RCS file: /cvsroot/clisp/clisp/src/subrkw.d,v
retrieving revision 1.22
diff -u -w -b -r1.22 subrkw.d
--- subrkw.d	2 Apr 2002 21:03:42 -0000	1.22
+++ subrkw.d	26 May 2002 20:08:46 -0000
@@ -167,7 +167,7 @@
 #endif
 #endif
 #ifdef SOCKET_STREAMS
-v(3, (kw(element_type),kw(external_format),kw(buffered)) )
+v(4, (kw(element_type),kw(external_format),kw(buffered),kw(timeout)) )
 s(socket_accept)
 s(socket_connect)
 #endif
Index: constsym.d
===================================================================
RCS file: /cvsroot/clisp/clisp/src/constsym.d,v
retrieving revision 1.144
diff -u -w -b -r1.144 constsym.d
--- constsym.d	21 May 2002 14:57:06 -0000	1.144
+++ constsym.d	26 May 2002 20:08:46 -0000
@@ -888,6 +888,7 @@
 LISPSYM(socket_stream_host,"SOCKET-STREAM-HOST",socket)
 LISPSYM(socket_stream_peer,"SOCKET-STREAM-PEER",socket)
 LISPSYM(socket_stream_local,"SOCKET-STREAM-LOCAL",socket)
+LISPSYM(Ktimeout,"TIMEOUT",keyword)
 #ifdef HAVE_SHUTDOWN
 LISPSYM(socket_stream_shutdown,"SOCKET-STREAM-SHUTDOWN",socket)
 #endif

---

From: Todd Sabin <[email-masked]>
Date: Sun May 26 16:49:03 2002

>   (socket-accept s :timeout x) ==

Given SOCKET-STATUS, this isn't really necessary for SOCKET-ACCEPT.
Doesn't hurt, though.

> trying to read from *STANDARD-INPUT*.

You've misunderstood what I want.  I have no problem with
(socket-connect ... :timeout N) signaling an error if the timeout
expires, and I agree that's the right thing.  But what I want is
something different, namely some form of socket-connect which doesn't
block, even in the success cases.  The only way to do that is to have
the C socket-connect not handle the timeout, but to always return a
stream (which may not be fully connected yet).  Then in lisp, you can
either do (socket-status (stream . output) N) and signal an error if
the stream isn't fully connected within your timeout, or you can put
the not-fully-connected stream into your state machine, if you happen
to be trying to write a non-blocking TCP server and you don't have
threads.

Either way, the C socket-connect always either returns a valid object
or signals an error.  If a user were to try to use the not fully
connected stream, they'd just end up blocking at that point.  But you
wouldn't expose users to this by default.  Since the non-blocking
semantics are different from the existing behavior, I was suggesting
that the C SOCKET-CONNECT be renamed to SOCKET-CONNECT%, and then
write the normal SOCKET-CONNECT in lisp.  That way users would see the
behavior you're talking about, but people can still get a true
non-blocking socket-connect if they want it.

Does that make any more sense?

Todd

---

From: Arseny Slobodjuck <[email-masked]>
Date: Sun May 26 20:03:01 2002

Hello,

Monday, May 27, 2002, 9:48:03 AM, you wrote:

Todd> You've misunderstood what I want.  I have no problem with
Todd> (socket-connect ... :timeout N) signaling an error if the timeout
Todd> expires, and I agree that's the right thing.  But what I want is
Todd> something different, namely some form of socket-connect which doesn't
Todd> block, even in the success cases.

By analogy with socket-wait it can be (socket-connect .. :timeout 0)
(but socket-wait has &optional, not &keyword timeout).

But Sam's patch turns off blocking - doesn't it result in wrong
recv/read/send/write handling ?

BTW I believe, Berkeley guys wasn't so stupid to not implement TCP/IP
on existing file API if it was possible. I mean, I think, if we
working with sockets as with streams we should be ready wor some
lack of functionality. OTOH that lack of functionality can result in
big gain in generality thus speed of development, maintainability (at
Lisp layer) etc.

---

From: Todd Sabin <[email-masked]>
Date: Sun May 26 22:11:02 2002

> (but socket-wait has &optional, not &keyword timeout).

Correct me if I'm wrong, but using socket-connect with :timeout 0 will
(virtually) always signal an error, since you'll never get connected
instantly.  (Perhaps you would on loopback.)  That is not what I want.

> recv/read/send/write handling ?

If it doesn't turn blocking back on, which it seems it doesn't, then
yes, that's a problem that needs to be fixed regardless.

> big gain in generality thus speed of development, maintainability (at

The flip side of that is that they give you all those socket options,
non-blocking handles, etc., if you didn't actually _need_ them
sometimes.  I have no problem with the normal interfaces in clisp
picking sane defaults for that stuff.  I do have a problem with the
fact that there's no way to workaround them without completely
abandoning the provided interfaces, and starting from scratch with
linux:socket, linux:connect, etc.

Todd

---

From: Arseny Slobodjuck <[email-masked]>
Date: Mon May 27 05:09:02 2002

Hello Sam,

Monday, May 27, 2002, 7:16:21 AM, you wrote:

Sam> Specifically win32: what's the MS-way to do ioctl/FIONBIO?
They call it WSPIoctl. IIUC it should be

errorp = WSPIoctl(socket,FIONBIO,&unsigned_long_nonblockp,
  sizeof(unsigned_long_nonblockp),NULL,0,NULL,NULL,NULL,NULL,&errno_returned);

But it needs experimenting.

---

From: Arseny Slobodjuck <[email-masked]>
Date: Mon May 27 05:09:02 2002

Hello Todd,

>> (but socket-wait has &optional, not &keyword timeout).

Todd> Correct me if I'm wrong, but using socket-connect with :timeout 0 will
Todd> (virtually) always signal an error, since you'll never get connected
Todd> instantly.  (Perhaps you would on loopback.)  That is not what I want.
Yes, you're right. But socket-wait doesn't return an error - it polls (and
it's convenient). What if we'll have analog of socket-wait but for client
side - call it, say connect-wait or wait-connect or client-wait... It will
turn blocking off, starts connecting (if connection wasn't started yet),
waits checking whether connection ready and returns nil or t depending on
result. Then socket-connect (when called) checks the status, turns on the
blocking and signals an error or returns stream in full harmony with Lisp
spirit balancing Client and Server sides of Connection... How do you
like it ?

---

From: Todd Sabin <[email-masked]>
Date: Mon May 27 08:43:02 2002

> spirit balancing Client and Server sides of Connection... How do you

Well, I'm not very clear on how your proposal would work.  What would
you pass to SOCKET-CONNECT to indicate that you want to use whatever
it is you get from CLIENT-WAIT?  And I don't really want a separate
function for the waiting.  I want to be able to take <whatever it is>
and use in with socket-status, along with all the other open
connections my server is processing.

What about the idea of having a SOCKET-CONNECT% didn't you like?

I don't suppose it makes sense to re-suggest the stuff in:
http://www.geocrawler.com/archives/3/1124/2002/1/100/7517822/ does it?
Sam already shot that down, but it's another way of dealing with this,
as well as providing UDP sockets, unix domain sockets, etc.  To tell
the truth, I've already implemented most of that locally.  It needs
some refinement, but works just fine.

Todd

---

From: Sam Steingold <[email-masked]>
Date: Mon May 27 09:21:02 2002

> * In message <[email-masked]>
> * On the subject of "Re: Re[4]: [clisp-list] socket-connect's timeout"
> * Sent on 27 May 2002 11:41:21 -0400
> * Honorable Todd Sabin <[email-masked]> writes:
>

KISS - keep it simple.

first, this is not really necessary.
if we really will introduce the "half-baked streams" (i.e., streams that
will start any i/o operation with a connect() syscall), they can be
created when the timeout is 0 (i.e. when otherwise we are guaranteed to
get an error).

second, I am not sure these "half-baked streams" are a good isea.
what do you need them for?

> http://www.geocrawler.com/archives/3/1124/2002/1/100/7517822/ does it?

here is the problem.
I _know_ what you want: you want, basically, bindings/linuxlibc6 in
CLISP at all times on all platforms (this is actually what I wanted when
I added --with-export-syscalls).  Therefore anything short of that will
not satisfy you, and a lot of that will not be portable (even across
unixes) and will not work too well with the traditional Lisp paradigms,
therefore this SOCKET-CONNECT% (which should actually be
%SOCKET-CONNECT! :-) will not be enough for you and will be too much
for "lisp purists" :-(

> some refinement, but works just fine.

good.
first, wrap it into `#ifdef EXPORT_SYSCALLS'.
second, please do try to go a step up from bindings/linuxlibc6: a lot
of stuff here can be unified in one function (e.g., FILE-STAT unifies
lstat() and fstat()).
third, provide a way to get back into the Lisp world, i.e., something
like `sys:make-fd-stream' in CMUCL to convert a socket to a stream.
(BTW, since the word `socket' is already overused, please call your new
type `handle').

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat7.2 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
Press any key to continue or any other key to quit.

---

From: Sam Steingold <[email-masked]>
Date: Mon May 27 09:32:02 2002

> * In message <[email-masked]>
> * On the subject of "Re[2]: [clisp-list] socket-connect's timeout"
> * Sent on Mon, 27 May 2002 22:00:57 +1000
> * Honorable Arseny Slobodjuck <[email-masked]> writes:
>

great!  please make win32.d and win32aux.d export a non-interruptible
ioctl, just like unix.d and unixaux.d do.

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat7.2 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
If your VCR is still blinking 12:00, you don't want Linux.

---

From: Arseny Slobodjuck <[email-masked]>
Date: Mon May 27 15:24:01 2002

Hello Sam,

>> But it needs experimenting.

Sam> great!  please make win32.d and win32aux.d export a non-interruptible
Sam> ioctl, just like unix.d and unixaux.d do.
Ok, in a day or two.

---

From: Arseny Slobodjuck <[email-masked]>
Date: Mon May 27 15:24:02 2002

Hello Todd,

>> spirit balancing Client and Server sides of Connection... How do you

Todd> Well, I'm not very clear on how your proposal would work.  What would
Todd> you pass to SOCKET-CONNECT to indicate that you want to use whatever
Todd> it is you get from CLIENT-WAIT?
You're right again. Then we need a CLIENT-SOCKET datatype analogous to
SERVER-SOCKET and initializer for it (taking port and host parameters).
Then initializer starts connection, CLIENT-WAIT waits or polls and
CLIENT-CONNECT creates the stream or throws an error.

Todd> What about the idea of having a SOCKET-CONNECT% didn't you like?
Think about it as about your half-connected streams splitted into
half-connected part (CLIENT-SOCKET) and connected SOCKET-STREAM.
That's just in more agreement with things existing before (as far I can
see).

Todd> I don't suppose it makes sense to re-suggest the stuff in:
Todd> http://www.geocrawler.com/archives/3/1124/2002/1/100/7517822/ does it?
I think those people who don't want to know about blocking, UDP
sockets, ioctl etc have right to it. So let advanced interface exists,
but if it hurts basic interface - maybe make it separate from SOCKET: ?

---

From: Todd Sabin <[email-masked]>
Date: Tue May 28 09:21:07 2002

Sam Steingold <[email-masked]> writes:
> will start any i/o operation with a connect() syscall), 

I think you may have misunderstood again.  They wouldn't start any i/o
with a connect call.  The connect call would already have been made,
but in non-blocking mode.  If you were to try to use the socket/stream
without waiting (via SOCKET-STATUS) for the connect to finish, then it
would just block at that point.  There'd be no need to modify any of
the current read/write paths.

>                                                          they can be

This would work for me.

> what do you need them for?

I've already explained several times, but I'll try again.  Imagine
you're writing a proxy server.  You accept connections from clients,
and in the process of servicing them, you have to open connections to
the resources they're requesting.  Unless you consider it acceptable
for your proxy to be totally non-responsive for large periods of time,
you've got to have a non-blocking connect.  The only representation in
clisp for a socket connection is a socket-stream.  Therefore, you need
a non-blocking way of getting a socket-stream which you can then use
with socket-status.  Is that clear enough?

> unixes) and will not work too well with the traditional Lisp paradigms,

Please stop assuming you know what I want.  What I'm saying in that
email is that I want _sockets_, not all of linuxlibc6, on all
platforms.  Are there any clisp-supported platforms that don't
support, say, UDP sockets?  If not, why not make them standard in
clisp?  Why should people be forced to resort to FFI?

I think there is a basic set of socket functionality which is
available on all platforms (socket, bind, listen, accept, connect,
recvfrom, sendto, e.g.).  And that providing that, and then building
higher level stuff on top of it would provide people with far more
functionality and options than they currently have.  Do you disagree
with the feasability of that?  If so, are there specific problems you
know of?

> first, wrap it into `#ifdef EXPORT_SYSCALLS'.

I think #ifdef FULL_SOCKETS or something would be more appropriate.

> of stuff here can be unified in one function (e.g., FILE-STAT unifies

As I said, this is a non-issue for me.

> like `sys:make-fd-stream' in CMUCL to convert a socket to a stream.

Yes, this is really the crux of the problem.  There's currently no way
to get back into the normal lisp world.  What I did for now was to
expose make_socket_stream as MAKE-SOCKET-STREAM% (should be %MAKE...).
If you agree that (or something similar) is necessary, and if you
agree that the basic socket functionality can be exposed portably
(though perhaps you don't), then you're 95% of the way to what I was
suggesting in that email.  At least, maybe you understand where I'm
coming from now?

E.g., given the above, I wouldn't care about clisp not providing the
non-blocking connect, because I could easily add it myself at the lisp
layer.  And the same would be true of lots of other socket
functionality.

Todd

---

From: Todd Sabin <[email-masked]>
Date: Tue May 28 09:35:03 2002

> That's just in more agreement with things existing before (as far I can

Ah.  This is actually extremely close to what I mention in the email
referenced below.  Your CLIENT-SOCKET would just be a normal SOCKET,
and what you are calling CLIENT-CONNECT would be more like a
MAKE-SOCKET-STREAM.  I.e., something that takes a socket and makes a
stream from it.

> but if it hurts basic interface - maybe make it separate from SOCKET: ?

I don't understand why you think this would force complexity on people
who don't want it.  If you want to keep using the high level stuff
that's there now, you can.  People who want to use the current stuff
aren't forced to look at the C implementation, are they?  All I'm
saying is to move some of the C code to lisp.  But it'd still be
"implementation details" people can ignore if they want.

Todd

---

From: Sam Steingold <[email-masked]>
Date: Tue May 28 11:03:36 2002

> * In message <[email-masked]>
> * Sent on 28 May 2002 12:19:34 -0400
> They wouldn't start any i/o with a connect call.  The connect call
> would already have been made, but in non-blocking mode.

are you saying that the second connect() call in connect_via_ip() is
wrong?  why didn't you say that?!

> If you were to try to use the socket/stream without waiting (via
> SOCKET-STATUS) for the connect to finish, then it would just block at
> that point.  There'd be no need to modify any of the current
> read/write paths.

when do I put the socket back into the blocking mode?
right after connect() but before the first i/o operation?

> > created when the timeout is 0 (i.e. when otherwise we are guaranteed to

you got it - try the appended patch with the current CVS
(preferably _before_ replying to this message :-).
please comment on the code - you know more about TCP/IP than I do.

> clisp?  Why should people be forced to resort to FFI?

so I exaggerated a bit.  you only want sockets.  fine.

BTW, does a stream based on a UDP socket make sense?
if yes, could we make it an option for socket-connect and socket-accept?

> with the feasability of that?  If so, are there specific problems you

win32.
will your code work _the same way_ on all Unixes and on w32?

> I think #ifdef FULL_SOCKETS or something would be more appropriate.

EXPORT_SYSCALLS corresponds to a configure option already.

> suggesting in that email.  At least, maybe you understand where I'm

okay - send a patch doing what you want.
it should work both on UNIX and win32.

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat7.2 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
Failure is not an option. It comes bundled with your Microsoft product.

--- socket.d.~1.59.~	Tue May 28 04:11:10 2002
+++ socket.d	Tue May 28 13:03:18 2002
@@ -823,6 +823,8 @@
     return fd;
  #ifdef FIONBIO
   if (sock_errno_is(EINPROGRESS)) {
+    var struct timeval *tv = (struct timeval*)timeout;
+    if (tv==NULL || tv->tv_sec!=0 || tv->tv_usec!=0) { # wait
    restart_select:
     var fd_set handle_set;
     FD_ZERO(&handle_set); FD_SET(fd,&handle_set);
@@ -836,8 +838,8 @@
       CLOSESOCKET(fd); errno = ETIMEDOUT;
       return INVALID_SOCKET;
     }
-    if (connect(fd, addr, addrlen) >= 0) {
-      # connected - restore blocking IO
+    }
+    { # connected - restore blocking IO
       var int non_blocking_io = 0;
       if (ioctl(fd,FIONBIO,&non_blocking_io) == 0)
         return fd;

---

From: Sam Steingold <[email-masked]>
Date: Tue May 28 11:28:17 2002

> * In message <[email-masked]>
> * On the subject of "Re: Re[6]: [clisp-list] socket-connect's timeout"
> * Sent on 28 May 2002 12:33:07 -0400

the exported socket API will have many more functions,
and this _will_ be confusing for an average user.

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat7.2 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
Binaries die but source code lives forever.

---

From: "Thomas F. Burdick" <[email-masked]>
Date: Tue May 28 12:42:02 2002

> and this _will_ be confusing for an average user.

Only if you document it as one big API.  The other option is to
document it in layers.  Have a "High-Level Sockets API" section
followed by a "Low-Level Sockets API" section.  I'm pretty sure that
most users will stop reading at the end of the high-level section,
when they figure out they can do what they want with it.

-- 
           /|_     .-----------------------.                        
         ,'  .\  / | No to Imperialist war |                        
     ,--'    _,'   | Wage class war!       |                        
    /       /      `-----------------------'                        
   (   -.  |                               
   |     ) |                               
  (`-.  '--.)                              
   `. )----'

---

From: Arseny Slobodjuck <[email-masked]>
Date: Tue May 28 15:49:03 2002

Hello, Sam,

>> would already have been made, but in non-blocking mode.

Sam> are you saying that the second connect() call in connect_via_ip() is
Sam> wrong?  why didn't you say that?!
I tried your patch on win32 (with ioctl changed to win32 equivalent -
I found 'ioctlsocket(socked,cmd,arg)' function in API) and it works
not very well. I am quite busy to debug it fast, but we're not in a hurry,
are we ? Or better yet I will debug it after unix version will work
(basically it works, but when I'm emulating slow server responce, it fails.
I patched it and now it locks at all).