[Openmcl-devel] "potential numbers" as symbol names

Dan Knapp dankna at accela.net
Wed Apr 20 12:43:53 PDT 2005

   At	 first I thought that this was perfectly well-defined.  Remember 
a token is a "textual representation" (glossary), not an actual symbol 
number.  Section 2.3.4 seems to say that a token is not a potential 
if it contains a package marker, which means that the rules for handling
potential numbers don't apply to things such as :3.  However, section
2.3.4 defers to section 2.3.5 as to what is a token to begin with, and 
it is
the interpretation of 2.3.5 that causes trouble for us.

   2.3.5 is somewhat ambiguous; it seems to say that :3 is a "valid 
for a token", since it's listed in the table titled "valid patterns", 
but that the
interpretation of that pattern is undefined.  If so, this is directly 
at odds
with 2.3.4, which does define its interpretation.  However, if :3 is 
not a
token at all, there is no conflict, because 2.3.4 describes rules for 
interpretation of tokens.

   If :3 were considered a token, it would be a symbol.  It is not a
potential number as defined by (clause 1), because it contains
a character which is not a digit, number marker, etc (section,
"Character Traits").

   I was confused by clause 3 of  It starts by unambiguously
defining the result of having a package marker (which is redundant, 
the prior clauses already define the same result), then states that that
result is not well-defined.  Then I realized that clause 3 comes into 
with non-base characters which are both package markers and number
markers, not that there are likely to be any such.  See section

   Another minor point: It's incorrect to say that "\3" is a legit 
symbol while
"3" is not.  Both are valid as symbol names in the sense that you can 
INTERN on those strings directly (and they are two different symbols).  
The question is only whether :3 and package:3 are tokens which are to 
interpreted as symbols.

   So we're left with two interpretations.  Either :3 is a token and a 
with cautionary wording in a couple places that it really isn't, or it's
undefined whether :3 is a token at all.

   Now that I've worked all that out, I guess I don't much care either 
way. :)
It comes down to whether it's better to tolerate marginal behaviour, or 
"soundly trounce" it.  Giving an error is probably better, assuming 
no significant performance penalty.  I would think that the tokenizer 
is a
tight inner loop, so we should look at performance before deciding.

   Links for convenience:

" Constituent Traits"

" Potential Numbers as Tokens"

"2.3.4 Symbols as Tokens"

"2.3.5 Valid Patterns for Tokens"

"Issue COLON-NUMBER Writeup"

-- Dan Knapp

More information about the Openmcl-devel mailing list