[Openmcl-devel] *default-external-format* and encoding and decoding strings.

[Gary Byers]

On 03/07/2016 08:30 PM, Ron Garret wrote:
> On Mar 7, 2016, at 7:16 PM, Gary Byers<gb at clozure.com>  wrote:
>
>> On 03/07/2016 04:34 PM, Ron Garret wrote:
>>> On Mar 7, 2016, at 3:23 PM, Gary Byers<gb at clozure.com>  wrote:
>>>
>>>> On 03/07/2016 03:36 PM, Ron Garret wrote:
>>>>> On Mar 7, 2016, at 1:30 PM, R. Matthew Emerson<rme at clozure.com>  wrote:
>>>>>
>>>>>>> On Mar 5, 2016, at 12:03 PM, Ron Garret<ron at flownet.com>  wrote:
>>>>>>>
>>>>>>>
>>>>>>> On Mar 5, 2016, at 8:39 AM, Dmitry Igrishin<dfigrish at gmail.com>  wrote:
>>>>>>>
>>>>>>>> 2016-03-05 16:25 GMT+03:00 Dmitry Igrishin<dfigrish at gmail.com>:
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> The *default-external-parameter* doesn't considered by
>>>>>>>> count-characters-in-octet-vector, decode-string-from-octets,
>>>>>>>> encode-string-to-octets, string-size-in-octets functions which
>>>>>>>> has the :external-format parameter. I believe that
>>>>>>>> *default-external-parameter* should affect the behaviour of
>>>>>>>> all functions with :external-format parameter, right?
>>>>>>>> Sorry, I meant the *default-external-format* special variable...
>>>>>>> This is a bug in ccl::lookup-character-encoding.  Here’s a patch:
>>>>>>>
>>>>>>> (in-package :ccl)
>>>>>>>
>>>>>>> (let ((ccl::*warn-if-redefine-kernel* nil))
>>>>>>>   (defun lookup-character-encoding (name)
>>>>>>>     (gethash (or name *default-external-format*) *character-encodings*)))
>>>>>> I don't think I can apply this.
>>>>>>
>>>>>> The issue is that nil is a valid character encoding name: it's a documented synonym for :iso-8859-1.
>>>>> That’s a bug in the documentation.  NIL should be a synonym for *default-character-encoding*.  (I’m not joking.  That part of the docs was written before the introduction of *default-character-encoding*.)
>>>> when that part (what part is it, by the way?) was written, :iso-8859-1 was the default character encoding.
>>> Yes, I know.  That’s why I thought it was wrong, because the introduction of *default-external-format* had rendered it obsolete.  But then I read it more closely and remembered my history...
>>>
>>>> Changing the functions in question to use (e.g.)
>>>>
>>>> (defun encode-string-to-octets (string &key (external-format :default) ....) ...
>>>>
>>>> seems to be another approach
>>> Yes, just more work.
>> Unless I'm missing something, we are talking about 4 functions here - ENCODE-STRING-TO-OCTETS, DECODE-STRING-FROM-OCTETS,
>> COUNT-OCTETS-IN-STRING, and COUNT-CHARACTERS-IN-OCTET-VECTOR.
> That sounds right, though to be sure you’d have to go through a lot more of the code base to make sure you had gotten every function with an external-format argument.
At the risk of incurring the wrath of the SPCDH:

How many functions accept an :external-format arg that defaults to NIL?

I get 5: the 4 that I mentioned, and CCL:MAKE-VECTOR-INPUT-STREAM.

All functions in CCL that accept an :external-format argument either 
call CCL::NORMALIZE-EXTERNAL-FORMAT on that
argument or pass the argument to something that does. 
CCL::NORMALIZE-EXTERNAL-FORMAT recognizes :DEFAULT.

All 5 of the functions in question are documented to accept an 
external-format argument, but the default  value of that
argument is not stated. If the lambda lists of 5 functions were changed from

  ,,, &key external-format ...

to

... &key (external-format :default) ...

the ticket would be closed with minimal impact on the documentation or 
any existing code.  (yes, anyone calling MAKE-VECTOR-INPUT-STREAM
expecting the stream's external-format to default to NIL/iso-8859-1 
would have to change their expectation.  What justifies that expectation ?)