Subject: RE: RE: [clisp-list] CLISP an German charcters

From: [email-masked]
Date: Wed Sep 11 01:00:02 2002

Hi,

Thanks for your reply.

> -----Original Message-----
> From: Sam Steingold [mailto:[email-masked]]
> Sent: Tuesday, September 10, 2002 7:01 PM
> To: Jens Himmelreich
> Cc: [email-masked]
> Subject: Re: [clisp-list] CLISP an German charcters
> [...]
> what is your specific problem:
>         what do you do?

I try to explain what I do.

I have a function:
(defun get-content-from-file (filename)
  (with-open-file 
   (stream filename)
    (let ((lines)) 
      (do ((line (read-line stream) (read-line stream nil 'eof)))
          ((eq line 'eof) 'eof)
        (setq lines (append lines (list (trim line)))))
      (reduce #'(lambda (s1 s2) (concatenate 'string s1 " " s2)) lines))))

(defun trim (s)
  "Spaces, Tabs and newlines stripped from the beginning and the end"
  (string-trim '(#\Space #\Tab #\Newline) s))

and get a text with german chars like this: 'L\204nge'. If I type I get
a 'Länge' in this emacs-buffer.

With some texts I get an error:
*** - Character #\u2022 cannot be represented in the character set
CHARSET:CP850
May be it's this character: '\225'

Because I see #\u I guess unicode and try the following function:
(defun get-content-from-file (filename)
  (with-open-file 
   (stream filename :external-format (ext:make-encoding :charset "UTF-8"
                                      :input-error-action :ignore))
    (let ((lines)) 
      (do ((line (read-line stream) (read-line stream nil 'eof)))
          ((eq line 'eof) 'eof)
        (setq lines (append lines (list (trim line)))))
      (reduce #'(lambda (s1 s2) (concatenate 'string s1 " " s2)) lines))))

Now I don't get an error, but I don't get any non-ascii charcter. They
are ingnored (I think because :input-error-action :ignore)

>         what do you see?

*** - Character #\u2022 cannot be represented in the character set
CHARSET:CP850
or no non-ascii character

>         what do you expect/want to see?

I hope to see the characters like in a normal text-editor
It's no problem, if the special (unicode) characters are
ignored, but I need the normal german character, the 
quotation-marks an so on.

Could you help me?

best regards
jens himmelreich

---

From: Sam Steingold <[email-masked]>
Date: Wed Sep 11 06:34:07 2002

> * In message <A159B2B20DBCD211B722006008964A7501AB1D77@EXCH-SVR>
> * On the subject of "RE: RE: [clisp-list] CLISP an German charcters"
> * Sent on Wed, 11 Sep 2002 09:59:57 +0200
> * Honorable [email-masked] writes:
>
> I have a function:

replace APPEND with PUSH and then NREVERSE for improved performance.

> May be it's this character: '\225'

when do you get the error?
during input from the file? (type "help" at the break prompt).
if yes, you should look at CUSTOM:*DEFAULT-FILE-ENCODING*
<http://clisp.cons.org/impnotes.html#def-file-enc> or supply
:EXTERNAL-FORMAT argument to WITH-OPEN-FILE.

if the error occurs when the output of GET-CONTENT-FROM-FILE is printed
to the *STANDARD-OUTPUT*, you need to modify CUSTOM:*TERMINAL-ENCODING*
<http://clisp.cons.org/impnotes.html#term-enc>.

See also <http://clisp.cons.org/clisp.html#opt-enc>.

>                                       :input-error-action :ignore))

never, ever use :*-ERROR-ACTION :IGNORE unless you know for certain
that this is what you really want.

> are ingnored (I think because :input-error-action :ignore)

are you sure your file is encoded with UTF-8 and not, say ISO-8859-1?

-- 
Sam Steingold (http://www.podval.org/~sds) running RedHat7.3 GNU/Linux
<http://www.camera.org> <http://www.iris.org.il> <http://www.memri.org/>
<http://www.mideasttruth.com/> <http://www.palestine-central.com/links.html>
Who is General Failure and why is he reading my hard disk?