[Openmcl-devel] Speed, compilers and multi-core processors

[Dan Weinreb]

Justin,

Justin Grant wrote:
>
> That being said, I really do like some of the features that I've seen 
> in Clojure (speaking from actual experience using Clojure and CL). 
> Forcing immutable data structures on users to simplify the language 
> implementation issues
I would say that the immutable data structure make the language 
implementation
more difficult, not easier.  The point is for the language to make 
things easier
for the programmer, by taking on hard jobs (such as efficient GC, 
optimization
of generated code, etc).
> or to make concurrent programming 'easier'
I don't see the reason for the quotation marks around "easier".

Here at ITA, we had a big debate about whether each Lisp
process should have multiple request-handling threads, or
each just one.  Many pros and cons were identified.  But
one of the major points made (by some extremely experienced
programmers) is that using concurrent threads was much too
dangerous.  In a complex system that uses shared mutable
data structures, you need to get the concurrency control
absolutely right.

For example, we might have each Lisp
process maintain some kind of cache.  Rather than
having N copies of the cache, one for each thread,
we might want to save both memory (only one copy)
and time (fewer fetches) by sharing the cache between
threads.    Other things would be shared as well.
It also hurts that Common Lisp does not define
thread-safety on various operations, since Common
Lisp's spec is written as if there were only ever
one thread.  This is one of the main reasons we
decided to create a whole Lisp process for each
request-handling thread.

In contrast, database management systems do NOT
get into trouble when applications use concurrency.
This is because their concurrency control is based
on a transaction model.  True ACID transactions
are very easy to think about and reason about.
It is much more easy to "prove" correctness.
(By "prove", I mean "convince yourself for
sure that it works".)

We know now that the wonderful concept of
transactions can be adopted from the DBMS
world into the programming language world.
I consider this a truly major breakthrough
in the progress of modern computer language
technology.  Of course, not every language
will find this appropriate, but it's quite impressive
how many of the new languages that are
currently being created have adopted the
same paradigm: mostly use immutable objects
and also provide transactions.

Given the choice of being "forced" to use
this to do concurrent programming, versus
being given the "power" to do any old
kind of locking I want, I'll gladly take
the former, just as I am happy to give
up the ability to write my own malloc,
and I am happy to pay the cost of
having my array bounds checked
automatically.


> is neither of them. The one thing I would change in Clojure is this 
> very thing. Even Haskell has it's monads
You can do monads in Clojure.  Monad in Haskell are just a library,
not part of the very heart of the language.  It's a design pattern,
and Clojure has all the fundamental power needed to implement
that pattern.  (There was discussion of this at the Int'l Lisp
Conference.  I am not quite familiar enough with this all to
defend what I just said, but ask Ravi Nanatavi of Bluespec
who writes actual product code in Haskell every day.)


> and Lisp allows for functional and imperative styles to be mixed.
Those words can be used in various ways, so I won't
try to address this question at a direct level.  But the
greater points are: Clojure does give you a wide variety
of ways to program.  But if Clojure's "mostly immutable
with transactions" paradigm does limit me, that's OK,
particularly if the job at hand calls for close and
relatively complex concurrency needs.

That's not to say that Clojure is the Right Language for
All Problems.  Here at ITA, we do plenty of Java,
and we use C/C++ in several places where it's
the most appropriate tool for the job.
>
> There may be an important lesson here ...
At the very least, I think discussing these issues can
be enlightening and educational for all of us, including
me of course.

-- Dan
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clozure.com/pipermail/openmcl-devel/attachments/20090521/10463ea7/attachment.htm>