[Openmcl-devel] ns-date initializers on OSX 10.8

[Paul Krueger]

Gary, this was extremely useful; thanks. I can only seem to reliably create one of the two tagged date types that you discovered and any number of different NSDate class methods seem to create one every time in my environment. For my purposes I'm using dateWithTimeIntervalSinceReferenceDate: since I already use it as part of translating lisp dates to NSDates, but simpler things like #/distantFuture seem to reliably generate tagged nsdate instances for me too.

I was guessing that the two tagged NSDate classes you found might correspond to the regular NSDate and NSCalendarDate classes, but no amount of manipulation of NSCalendarDate instances resulted in a second type of tagged date, so maybe that guess was wrong. I even tried to get an NSDateFormatter to generate the other tagged type, but no go.

I made a few minor tweaks to your code (mainly to follow your suggestions) to get:

(defun setup-class-info-for-tagged-instance (instance)
 ;; "instance" is a pointer known to be an ObjC instance that isn't recognized as an ObjC
 ;; object, presumably because its address has some of its low 4 bits set.
 (let* ((tag (tagged-objc-instance-p instance)))
   (when (and tag (not (objc-tagged-instance-class-index tag)))
     ;; We should only bother with this if tag isn't already on the alist
     (let* ((class (objc-message-send instance "class")))
       (unless (%null-ptr-p class)
        (install-foreign-objc-class class nil)
        (push (cons tag (objc-class-or-private-class-id class)) *tagged-instance-class-indices*)))
     ;; return a copy of the pointer that will be interpreted as an ns-object if this was successful
     (%inc-ptr instance 0))))

and using this on an instance that I can reliably create at load time seems to do the trick so far. I'm sure I'll run into the other tagged class at some point and then perhaps I can figure out how to reliably create an instance of that one too.

On Jan 6, 2013, at 8:08 AM, Gary Byers <gb at clozure.com> wrote:

> For the terminally curious:
> 
> In 10.8, the ObjC runtime uses a 16-element table to map the low 4 bits of a tagged
> instance pointer to a class.  (Only odd tag values can have entries, and when I use GDB
> to look at that table in a running CCL process it looks like:
> 
> 0x7fff76a4d148 <_objc_tagged_isa_table>:	0x0000000000000000	0x0000000000000000
> 0x7fff76a4d158 <_objc_tagged_isa_table+16>:	0x0000000000000000	0x0000000000000000
> 0x7fff76a4d168 <_objc_tagged_isa_table+32>:	0x0000000000000000	0x0000000000000000
> 0x7fff76a4d178 <_objc_tagged_isa_table+48>:	0x0000000000000000	0x00007fff75642eb0
> 0x7fff76a4d188 <_objc_tagged_isa_table+64>:	0x0000000000000000	0x00007fff756434c8
> 0x7fff76a4d198 <_objc_tagged_isa_table+80>:	0x0000000000000000	0x0000000000000000
> 0x7fff76a4d1a8 <_objc_tagged_isa_table+96>:	0x0000000000000000	0x00007fff756434c8
> 0x7fff76a4d1b8 <_objc_tagged_isa_table+112>:	0x0000000000000000	0x0000000000000000
> (gdb) po 0x00007fff75642eb0
> __NSCFNumber
> (gdb) po 0x00007fff756434c8
> __NSTaggedDate
> (gdb)
> 
> __NSCFNumber and __NSTaggedDate are the names of private/internal subclasses of NSNumber and
> NSDate, respectively.  The table is that the tag 7 is used to represent (some) instances of
> __NSCFNumber and that the tags 9 and 13 are both (by luck or design) used to represent immediate
> instances of __NSTaggedDate.  I have no idea why two tags are used in this case; this stuff just
> doesn't look finished to me ...
> 
> If we could get our hands on that table, then it'd be straightforward to use it to determine
> the class of a tagged pointer (just use the low 4 bits of the pointer as an index into the table.)
> Unfortunately, the symbol "_objc_tagged_isa_table" (the symbol actually has two leading underscores)
> isn't exported from libobjc.  (Yes, there are arguments against exposing too much implementation
> detail, but some interface - something that maps a tag value to a class or NIL - would be useful,
> and no interface at all is pretty useless.)
> 
> I said earlier that I didn't see the use of tagged NSDate instances in 10.8.2.  What I'm now
> seeing is that they're used sometimes and not others: within the same lisp session, something
> like (#/date ns:ns-date) sometimes returns a tagged pointer and sometimes returns a tradition
> pointer (with 0 tag bits).  I have no idea why this is, either.
> 
> Since 10.8, it seems to be the case that there are reliable ways of creating an immediate/tagged
> NSNumber instance; specifically
> 
> (#/initWithInt: (#/alloc ns:ns-number) 0)
> 
> returns a pointer whose low 4 bits are non-zero.  Startup code that runs when the bridge is loaded
> (and whenever a saved image that contains that code is loaded) (re-)initializes an alist that maps
> from ObjC tag values to infornation about the ObjC class that uses those tag bits to represent at
> least some of its instances.  This happens in the "(defloadvar *tagged-instance-class-indices* ..."
> form in "ccl:objc-bridge;objc-support.lisp", and the effect of that code is to initialize
> the alist CCL::*TAGGED-INSTANCE-CLASS-INDICES* so that it contains an entry for at most one
> tag value (the tag of NSNumber instances that were created with the #/initWithInt: call above.)
> 
> If we could reliably create a tagged NSDate instance, we could repeat that process and create
> another entry on that alist.  I don't know how to do that reliably: sometimes I see tagged instances
> and sometimes I see traditional aligned pointers.
> 
> If you need a short-term workaround so that tagged ObjC pointers will be recognized by the type
> system, code like the following (which is mostly stolen from the DEFLOADVAR form above) should
> enable that:
> 
> 
> ;;; This is all in the CCL package.
> (defun setup-class-info-for-tagged-instance (instance)
>  ;; "instance" is a pointer known to be an ObjC instance that isn't recognized as an ObjC
>  ;; object, presumably because its address has some of its low 4 bits set.
>  (let* ((tag (tagged-objc-instance-p instance)))
>    (when tag
>      ;; We should only bother with this if tag isn't already on the alist
>      (let* ((class (objc-message-send instance "class")))
>        (unless (%null-ptr-p class)
>         (install-foreign-objc-class class nil)
>         (push (cons tag (objc-class-or-private-class-id class)) *tagged-instance-class-indices)))))
> ))
> 
> That's pretty horrible, but may get you unwedged.
> 
> Note that when dealing with MACPTRs, the type system tries to cache type information in the pointer
> itself; the code above might only get called if the system has decided that the pointer is just
> a generic pointer (and not an ObjC object) and cached that information.  One workaround would be
> make the code above return a copy of its "instance" argument via something like:
> 
> (%inc-ptr instance 0)
> 
> or
> 
> (%int-to-ptr (%ptr-to-int instance))
> 
> The copy wouldn't have any cached type info, so the type system would try to determine that
> info and should be influenced by the new alist entry.
> 
> 
> 
> 
> On Sat, 5 Jan 2013, Paul Krueger wrote:
> 
>> For the record I'm using vanilla 10.8.2, not one of the newer pre-release osx versions. Although as I think about it, I wonder if it's possible that Xcode knows that I'm a developer and automatically downloaded a newer development version of the libraries? As I said in my followup, the basic constructor returns an NSDate, but others are (I now suppose) returning some sort of tagged object. I'll see what I can find to deal with tagged objects in the current CCL code (pointer to that welcomed) and I am a paid-up authorized developer, so I'll also go see what I can find on Apple's site that might help (pointers to their relevant docs are also appreciated). Between those, perhaps I can create a work-around of some sort. Suggestions are always appreciated and if there is anything I can do to help you guys figure this out, please let me know. I'm sort of dead-in-the-water until I figure out something.
>> 
>> On Jan 5, 2013, at 1:23 PM, Gary Byers <gb at clozure.com> wrote:
>> 
>>> As you may or may not recall, Apple invented tagging within the last year.
>>> 
>>> Most NSObjects are represented as an aligned pointer (the low N bits
>>> are 0, where N=4 IIRC) whose first word is an ObjC class or metaclass
>>> object; with 10.8, they started representing some kinds of NSNumber
>>> instances as immediate objects where the low nibble was some
>>> distinguished non-zero value (that meant "some kind of NSNumber instance)
>>> and the upper 60 bits or so contained some immediate value and possibly some
>>> other type info.  I'm fairly sure that I looked and found that as of 10.8 only
>>> one of the ~15 available values was used (for this NSNumber case.)
>>> 
>>> That (using "low tags" to represent immutable objects) is a reasonable
>>> thing to do; I somehow missed the memo where they announced that such a change
>>> was forthcoming but managed to get support for this change into CCL before the
>>> 10.8 release.  (Or maybe just a little after.)
>>> 
>>> Your followup message suggested that this may have just been some sort of artifact,
>>> but it's possible that Apple has decided that at least some kinds of NSDate could
>>> also benefit from an immediate representation.  (If so, the memo's late again.)
>>> 
>>> I don't see this as of 10.8.2.
>>> 
>>> 
>>> 
>>> 
>>> On Sat, 5 Jan 2013, Paul Krueger wrote:
>>> 
>>>> When I try to create ns:ns-date objects in CCL (1.9-dev-r15559M-trunk  (DarwinX8664)) while running on OSX 10.8 I get the following:
>>>> 
>>>> ? (#/init (#/alloc ns:ns-date))
>>>> #<A Foreign Pointer #x41B69891A426BEFD>      <=== Not recognized as an ns-date instance
>>>> ? (#/init (#/alloc ns:ns-object))
>>>> #<NS-OBJECT [uninitialized] (#x25EDA860)>
>>>> 
>>>> All ns:ns-date initializers also return macptrs that CCL doesn't recognize as instances of ns-date or any other type of ns-object.
>>>> 
>>>> If I pass a pointer created by any ns-date initializer to an interface object that is expecting an ns-date object, it seems to take it and display it correctly. I just can't interact with that object in Lisp. So for example if it is changed in the interface and I get it back, I can't tell what was changed.
>>>> 
>>>> Is this because I'm running on a 10.8 system with an objective-c bridge for 10.6 or is something else going on? This isn't new functionality since 10.6, but did something internally change that screwed things up? This is the first class where I've run into this problem.
>>>> 
>>>> Recommendations?
>>>> _______________________________________________
>>>> Openmcl-devel mailing list
>>>> Openmcl-devel at clozure.com
>>>> http://clozure.com/mailman/listinfo/openmcl-devel
>>>> 
>>>> 
>> 
>>