[Openmcl-devel] Speed, compilers and multi-core processors

[Dan Weinreb]

Justin Grant wrote:
>
> Haskell has many mutable data structures too
It does?  I didn't know that; I thought it didn't.  (I'm only up
to chapter 9 in "Real World Haskell".)
> (as does Clojure, it's built on Java).
In my previous mail, when I discussed Clojure, I wasn't
including the "arbitrary calls into Java" part of Clojure.
In fact, I'm looking forward to see how Clojure sews
together its approach with the use of Java libraries.
>
> Memory-management is a completely different problem that requires a 
> different solution than does concurrent programming. Memory allocation 
> is always required by any program.
Well, actually, the way that GPGPUs and DSM chips are programmed
does not use malloc/free, as I understand it.  It's just a question of
the right tool for the job.
> Who's to say that the user will always be writing concurrent programs ?
Sure, I agree.

On the other hand, soon nearly every personal computer (assuming
that that continues to have a meaning!) will have multiple cores.
In fact, we're pretty close to that now.  Sometimes it's easy
to keep cores busy simply because there are many background
processes, but the number of cores continues to rise.  I recently
was shopping for a new desktop and found that there are
consumer computers using the Intel i7 with 8 cores.  So
it seems not unreasonable to assume that there will be more
and more need for languages that handle concurrency.
As you say, not every job needs to worry about that kind
of close concurrency; but more and more jobs are going
to.  At least, many people think that's what's coming.
> 'Everything looking like a nail' and all that.
> This problem exists at another level and is best handled by the 
> programmer. Sure it's nice to make such an assumption at a language 
> design level if you must but make it optional and in doing so allow 
> the programmer more power.
There is always a tradeoff between giving an application,
or a subsystem, more, or less, power.

Here's a simple example.  Our Lisp system at ITA has many
configuration parameters.  What we used to do (we have
not quite gotten rid of this yet) is to create named
configurations, which worked by having a file full
of Lisp forms that was loaded.  That's a lot of power;
a config file could do anything doable from Lisp.

But for many reasons, we've decided that a declarative
form of configuration has a lot of advantages, and
we now use structured configuration files, e.g.
each line has a key, a colon, and a value (but
a bit more complicated than that, JSON in some
cases).  We reduce the power, but now programs
can interpret config files and do various useful
operations on them.

In my experience, I've seen this kind of architectural
tradeoff come up over and over again.  More power
is not necessarily better; it depends on a lot of things.
>
> Clojure has intentionally ignored many approaches taken in CL (not a 
> judgment, just an observation).
> Many choices were made because of Clojure's foundation (the JVM) and 
> this shows through when programming in the language beyond any trivial 
> examples. Customer approved platforms, existing investments and the 
> 'industry standard' all factored heavily in Clojure's design at some 
> cost to giving the user more power (it's all on the Clojure website). 
Yes, that's quite true.
> This makes for Clojure being the 'industry standard' Lisp, for better 
> or worse. Clueless managers may end up praising it the way they did 
> Java after reading about it in some magazine.
Well, I'm not sure that's very important.  There are extremely
good reasons to build a Lisp on top of a JVM, and additional
extremely good reasons to expose that as part of the language
definition.  It's not what you'd do if you were designing a
new language in a vacuum, but these days it's no longer
realistic to do that.  The ability to take advantage of the existing,
and always improving, JIT compiler and garbage collector
technology, the portability, and the ability to tie into the
huge number of both general and highly specialized
JVM libraries, has enormous practical value.  The idea
that we'd ever get enough firepower behind Lisp that
we could replicate all of that is entirely impractical.
I had been advocating this kind of approach for at
least a year before I ever heard of Clojure!
>
> That's all fine, but another problem that Clojure faces is the 
> marriage of a language trying to be 'purely' functional on top of a 
> platform(including libraries) that will always be imperative.
> Java was never just a language, it's always been a language, the jvm 
> and the libraries.
> Clojure inherits the legacy of all this because of it's design.
> The magnitude and scope of these problems is yet to be discovered by 
> the Java/Clojure community when they get into building real systems 
> and extending current systems on the JVM with existing libraries.
I agree with all of that.  Although there are an impressive number of
users (given that it's so new and it's based on Lisp and so on) of
Clojure, I agree with you that the issues you mention above have
probably not been explored sufficiently that we can come to
a very clear judgement about how it all works out or doesn't
and what to do about it.
> My bet is that you'll probably see lots of users effectively 
> programming in Java (but using s-exps) when they realize that using 
> the language with existing Java libs is a complete impedance mismatch. 
> You'll likely hear the more experienced Clojure users suggest wrapping 
> these problems up in macros or something. Who wants that job ?
I guess I'm more optimistic than you are, but only time will tell.

-- Dan
>
> Good luck ...
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clozure.com/pipermail/openmcl-devel/attachments/20090521/03d19f95/attachment.htm>