[Openmcl-devel] The M1 port and the future of CCL

[David Ritchie]

I’m using CCL professionally and for occasional personal things too, and have for many years - getting on for 20 all in all. I’m running a web based system on SmartOS (Solaris/illumos based), so it’s surprising but good that CCL is still maintained on that system. I guess I have a way of choosing things that exceptionally few other people are using!

I’m also now using an M2 Mac, and I like CCL. I’d like to be able to run the system I’m working on using SBCL as well, just to have options, but CCL has a very easy and convenient way of mmap-ing files as arrays of 64 bit integers which I’m using for a memory based database. So SBCL isn’t an option without quite a lot of work. The main reason I started using CCL was because it compiled a lot faster, which was very useful at the time.

I haven’t really used the CCL IDE for anything serious. I’ve experimented with it, and the Cocoa bridge does seem very nice. I even tried firing up the old MCL IDE on a ‘classic’ Mac to see what that was like. The manual for it seems good. I imagine it probably wouldn’t be a bad choice for teaching CL in a graphical environment. BUT - as you say Ron - it’s a constant battle keeping up with macOS changes. I actually just got it running on my new Mac, which is nice. Oh - it just crashed!

I do all my CL development, and many other things, in GNU Emacs. The combination of emacs+slime for remote development, magit, org-mode, tramp etc are really good. It also means it doesn’t matter what OS I use. I’m doing my development in a Debian VM running on said M2 mac. I don’t want to be tied to a proprietary OS. In an ideal alternate reality world I’d like to have a similar setup using an emacs written in CL, but with all the org/magit/tramp/whatever goodness. Emacs is basically a poor-man’s lisp machine with a very large user base. 

In answer to the question of whether it’s worth keeping the CCL IDE alive - I wish there was more interest in it. In the olden days when MCL was used at Apple I can see that it could have taken off. I don’t suppose there would be many objective C programmers or Xcode users if not for it having been the standard Apple (and Next) IDE. For your personal projects I definitely think the path of least resistance would be to switch to emacs/slime. Maybe there would be more interest in the CCL IDE if it could be freed from MacOS. The only obvious way I can see to do that easily would be with GNUStep. I did try the old Windows port of CCL which used cocotron. It basically worked. That would solve the problem with Apple breaking things, and if it worked well enough to do some graphical things there might be some interest.

As to just porting the compiler without the IDE and Cocoa bridge: that would certainly be useful for me. Linux/arm64 would be very useful. I wonder how much work is involved in doing that. If I had time to work on it (which I definitely don’t at the moment) I’d love to do that, though I assume there would be somewhat of a learning curve. 

— David


> On 22 Nov 2023, at 19:27, Ron Garret <ron at flownet.com> wrote:
> 
> Tim McNerny and I had a long discussion on the future of CCL this morning and I wanted to write up some of the things we talked about and solicit feedback from the community.
> 
> I have been using CCL since the first C stood for Coral.  My first CCL ran on a Mac Plus with 1 MB of RAM and an 800k floppy drive.  That was nearly 30 years ago, and I've been using it more or less continuously ever since, both in professional setting and for my personal projects.  I have a lot invested in it.
> 
> For me personally, there are three main sources of value in CCL:
> 
> 1.  The IDE and MacOS integration.  When it first came out, CCL blew everything else out of the water, and IMHO it's still the best available.  I really like having an editor that is well integrated into the Mac UI/UX that I can customize in CL rather than elisp, and I really like being able to develop native Mac applications interactively using the ObjC bridge and Cocoa integration.
> 
> 2.  Seamless and reliable migration to production environments.  I can run the exact same code in a Linux production environment as on my Mac development machine.  That allows me to have very high confidence that things will Just Work when I push production code.  CCL is also rock-solid.  I run server processes in CCL that run for months without needing any attention.
> 
> 3.  The compiler is blazingly fast, and the code it produces is plenty fast enough for my needs.  On the rare occasions when I need peak performance, I'll write some C code and call it via the FFI.  And on very, very rare occasions I can actually tweak the compiler if I really need to.
> 
> The problem, of course, is that the Mac environment is and always has been a moving target.  CCL successfully made the transition from MC68k to PPC to x86 to x86-64, and now we're facing ARM64.  Unfortunately, the processor architecture is not the only thing that is moving this time.  The MacOS architecture is also undergoing significant changes away from its unix roots and becoming more like iOS.  AFAICT the writing is on the wall for Objective C.  Swift is the future on Macs.  And more generally, Apple as a company just doesn't value stability, and never has.
> 
> This is not necessarily a bad thing.  Apple's willingness to jettison old standards is one of the things that allows them to stay at the cutting edge.  I really like my M1 Macbook Air.  More importantly, my wife likes her M1 Macbook Air.  I can't see her switching over to Linux, and being on the same environment as she is on has significant advantages when it comes time to put on my domestic tech support hat.
> 
> So on the one hand, I would really love to see CCL continue to be a viable development environment for me.  On the other hand, I'm really starting to wonder whether it's really worth the effort.  Even if we manage to port the compiler and kernel to the M1, the ObjC bridge and IDE will probably need some significant additional work, and some day we will probably need a Swift bridge.  And God only knows what will happen in ten years.  For a system that has been useful for 30 years, that is a time horizon that matters.
> 
> All of which has led to wonder if a better long-term solution for me might be to go to a different CL implementation.  I've tried SBCL and ECL on the M1 and both seem to work quite well, though I have not leaned very hard on either one.  ECL is particularly attractive because it generates C code, so porting it to a new processor architecture is a lot easier than a native compiler.  That is potentially a big win if one's goal is long-term stability.
> 
> I'm putting this out there because CCL is still at a crossroads.  Is it really worth trying to keep it alive?  Is there a bit enough user community to make it worthwhile?  What does that community want?  I would love to see CCL continue, but it's far from clear that porting it is the path of least resistance to my own personal goals.  It might be better for me to use ECL or SBCL and Emacs/Slime, or maybe try to port Hemlock directly to one of those implementations, maybe work on an ECL/Swift bridge, or something like that.
> 
> I'm very much hoping that the answer will turn out to be that yes, it's worth keeping CCL alive.  But at the moment it is not clear to me, and it's particularly unclear whether just porting the compiler will be useful by itself without bringing the rest of the system along as well.
> 
> Thoughts?
> 
> rg
>