[Openmcl-devel] Apple OKs other dev tools for iOS

[Gary Byers]

On Sat, 11 Sep 2010, peter wrote:

>
> So it seems we may use a 3 layered system. The full development environment 
> as now in CCL on computer, some sort of CCL development environment on iOS 
> which would include compiler and editor, and the final minimal end 
> deliverable application.  The application itself carrying some non essential 
> but useful debugging info. The style of development perhaps along the lines 
> of MCLIDE's.
>

It might be neat (and on an iPad or some other device with a large form
factor it might even be usable), but I have a lot of trouble imagining
a usable editor running on a phone or an iPod touch.
>
> Will we need to jailbreak iOS to get CCL on the device (seems 3GS 4.1 
> version's just out)?
>

One of the things that you generally do a lot of during a lisp
development session is to put machine code in memory and execute it.
Sometimes that code comes from FASL files, sometimes it's just
compiled interactively.   In CCL, objects that contain machine code
aren't (usually) conceptually different from other things (strings,
vectors, CLOS instances, ...): the GC can move them around in memory
or decide that they're unreachable.  (On some platforms, they need
some special treatment to ensure cache coherency, but other than that
a CODE-VECTOR or a FUNCTION that contains machine instructions is otherwise
treated like any other kind of lisp object.)

We generally don't think that "putting code in memory and executing it"
is a particularly expensive operation, and we do it all the time and expect
to be able to do it all the time as a normal part of the development cycle.

CCL allocates code in the same kind of memory that other kinds of objects
are allocated in.   (SAVE-APPLICATION can copy code to other memory areas,
but during development it's usually just in the same heap with everything
else.)  The memory pages that all of this dynamic stuff is sloshing around
in need to have read, write, and execute permissions.  (Not all CPUs/MMUs
support the concept of separate execute permissions; modern ARM processors
do.)

iOS generally wants to exercise control over what pages in a process's
memory map have execute permission.  Pages that're mapped from a
digitally- signed executable file or library can have execute
permissions; it's otherwise at least somewhat tricky to put code in
memory (which requires write permission to the pages in question) and
make it executable.  Whether it's possible at all may depend on device
and OS version; in the case where I saw that it was possible, it might
be prohibitively expensive and is certainly not something that one would
do casually, to the point where I'd been thinking "Hmm.  Let's look at Android
and Meego and Symbian and other platforms; this is ridiculous."

(As I understand it, some environments like Gambit Scheme approach this
issue by compiling to C, having the C compiler on a Mac generate and digitally
sign a shared library, downloading that library to the device, and having
the lisp open that shared library.  That's clever and I'm sure that it works
and works well;  that sort of thing is probably on the wrong side of
"prohibitively expensive" to be attractive for a number of reasons.)

Note that if we've gotten a bunch of code in memory and save an image (so
that all of that code is mapped from the image), then at least in theory
it should be possible to digitally sign that image and the resulting
application should be able to execute its code (what a concept!) and do
so with Apple's permission.  Whether Apple would approve such an application
for distribution in the App Store or not is hard to say ("when we said that
it was OK to use other development tools, we meant 'other tools that we
understood' !"), but I'm not aware of anything about such an application that
would necessarily violate the newly-stated acceptance criteria.

Jailbreaking a device removes or loosens the digital-signing
restrictions.  It incidentally makes the memory-permission behavior a
lot closer to normal (though not quite there), so that "putting code
in memory and executing it" isn't prohibitively difficult.  It also
voids the device's warranty, makes some people uncomfortable, and
makes Apple so uncomfortable that they start making ridiculous claims
to US regulatory agencies.  (It's estimated that several million
iPhones had been jailbroken; the Earth is still spinning on its axis
and the cellular network is apparently as secure as it ever was, but
we'd probably rather not require jailbreaking for development if we
didn't have to.)

I'm glad that the correct answer to your question is "that issue's
still being reviewed", and it's an honest answer: there might be
solutions/approaches that haven't been considered.  (If I had to
answer the question right now - and thankfully I don't - I'd probably
say that the memory protection restrictions on a non-jailbroken device
are too onerous to make interactive, lisp-like development on the device
pleasant, and I think that a jailbroken device would probably support that
kind of incremental development a lot better.)

These restrictions affect incremental code generation, which is something
that a delivered application doesn't need to do, so (AFAICT) there's no
requirement that a device be jailbroken in order to run such an application.