[Openmcl-devel] "potential numbers" as symbol names

[Dan Knapp]

   At	 first I thought that this was perfectly well-defined.  Remember 
that
a token is a "textual representation" (glossary), not an actual symbol 
or
number.  Section 2.3.4 seems to say that a token is not a potential 
number
if it contains a package marker, which means that the rules for handling
potential numbers don't apply to things such as :3.  However, section
2.3.4 defers to section 2.3.5 as to what is a token to begin with, and 
it is
the interpretation of 2.3.5 that causes trouble for us.

   2.3.5 is somewhat ambiguous; it seems to say that :3 is a "valid 
pattern
for a token", since it's listed in the table titled "valid patterns", 
but that the
interpretation of that pattern is undefined.  If so, this is directly 
at odds
with 2.3.4, which does define its interpretation.  However, if :3 is 
not a
token at all, there is no conflict, because 2.3.4 describes rules for 
the
interpretation of tokens.

   If :3 were considered a token, it would be a symbol.  It is not a
potential number as defined by 2.3.1.1 (clause 1), because it contains
a character which is not a digit, number marker, etc (section 2.1.4.2,
"Character Traits").

   I was confused by clause 3 of 2.3.1.1.  It starts by unambiguously
defining the result of having a package marker (which is redundant, 
since
the prior clauses already define the same result), then states that that
result is not well-defined.  Then I realized that clause 3 comes into 
play
with non-base characters which are both package markers and number
markers, not that there are likely to be any such.  See section 2.1.4.2
again.

   Another minor point: It's incorrect to say that "\3" is a legit 
symbol while
"3" is not.  Both are valid as symbol names in the sense that you can 
call
INTERN on those strings directly (and they are two different symbols).  
The question is only whether :3 and package:3 are tokens which are to 
be
interpreted as symbols.

   So we're left with two interpretations.  Either :3 is a token and a 
symbol,
with cautionary wording in a couple places that it really isn't, or it's
undefined whether :3 is a token at all.

   Now that I've worked all that out, I guess I don't much care either 
way. :)
It comes down to whether it's better to tolerate marginal behaviour, or 
to
"soundly trounce" it.  Giving an error is probably better, assuming 
there's
no significant performance penalty.  I would think that the tokenizer 
is a
tight inner loop, so we should look at performance before deciding.

   Links for convenience:

http://www.lispworks.com/documentation/HyperSpec/Body/02_adb.htm
"2.1.4.2 Constituent Traits"

http://www.lispworks.com/documentation/HyperSpec/Body/02_caa.htm
"3.2.1.1 Potential Numbers as Tokens"

http://www.lispworks.com/documentation/HyperSpec/Body/02_cd.htm
"2.3.4 Symbols as Tokens"

http://www.lispworks.com/documentation/HyperSpec/Body/02_ce.htm
"2.3.5 Valid Patterns for Tokens"

http://www.lispworks.com/documentation/HyperSpec/Issues/iss055_w.htm
"Issue COLON-NUMBER Writeup"

-- Dan Knapp