[Openmcl-devel] encoding/decoding clozure strings to/from UTF-8
ron at flownet.com
Tue Dec 30 19:46:36 UTC 2014
You’re working way too hard. Character streams automatically do work for you. All character streams in CCL have a parameter that sets their character encoding, and the default is UTF-8. So unless you want a different encoding, you should just be able to output characters to a stream and the Right Thing will happen. If it doesn’t, then either your default encoding has changed somehow, or you’ve found a bug in CCL.
On Dec 30, 2014, at 11:35 AM, Mark Klein <m_klein at MIT.EDU> wrote:
> Karsten, Matthew,
> Thanks for the pointers, that makes sense. My next challenge is to change a bunch of web page generation methods so that they output UTF-8. I wrote a little macro that I think will do the job. Am I on the right track, can it be made substantially faster or reliable somehow?
> (defmacro c2utf8 (&rest body)
> "Takes everything body sends to stream, and converts it to UTF-8"
> `(let ((list (encode-string-to-octets (with-output-to-string (stream) , at body) :external-format :utf-8)))
> (loop for c across list do (write-char (code-char c) stream))))
> It might be better, for example, if I could somehow redefine the stream so the UTF-32 text that goes in comes out as UTF-8?
>> On Dec 30, 2014, at 2:13 PM, Karsten Poeck <karsten.poeck at gmail.com> wrote:
>> On 30.12.14 19:46, R. Matthew Emerson wrote:
>>>> On Dec 30, 2014, at 1:19 PM, Mark Klein <m_klein at MIT.EDU> wrote:
>>>> I have Clozure 1.10 running on a Mac (OS 10.10.1). I'd like to be able to encode strings to and from UTF-8 (to handle multi-language text in a web application I am developing). Can someone give me some pointers on how I can do that?
>>> Basically, use (encode-string-to-octets string :external-format :utf-8) to encode a lisp string into a vector of octets containing UTF-8-encoded data, and use (decode-string-from-octets vector :external-format :utf-8) to create a lisp string from a vector of octets containing UTF-8-encoded data.
>> Since Matthew already answered the original question, another example with 1,2 and 3 byte encoding in utf-8 from http://en.wikipedia.org/wiki/UTF-8
>> (map 'vector #'(lambda(byte)(write-to-string byte :base 16)) (encode-string-to-octets "$¢€" :external-format :utf-8 :use-byte-order-mark nil))
>> #("24" "C2" "A2" "E2" "82" "AC")
>> To read an utf-8 encoded file use
>> (defun read-utf-8-file (file)
>> (with-open-file (stream file :direction :input :external-format :utf-8)
>> Openmcl-devel mailing list
>> Openmcl-devel at clozure.com
> Mark Klein
> Principal Research Scientist
> Center for Collective Intelligence
> Massachusetts Institute of Technology
> Visiting Researcher
> Dynamic and Distributed Information Systems Group
> University of Zurich
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 455 bytes
Desc: Message signed with OpenPGP using GPGMail
More information about the Openmcl-devel