[Openmcl-devel] Speed, compilers and multi-core processors

[Justin Grant]

Thanks for the responses Dan, Mikel :

>
>  Haskell has many mutable data structures too
>
> It does?  I didn't know that; I thought it didn't.  (I'm only up
> to chapter 9 in "Real World Haskell".)
>

You're ahead of me in RWH but what a great read hey ?


>  Who's to say that the user will always be writing concurrent programs ?
>
> Sure, I agree.
>
> On the other hand, soon nearly every personal computer (assuming
> that that continues to have a meaning!) will have multiple cores.
> In fact, we're pretty close to that now.  Sometimes it's easy
> to keep cores busy simply because there are many background
> processes, but the number of cores continues to rise.  I recently
> was shopping for a new desktop and found that there are
> consumer computers using the Intel i7 with 8 cores.  So
> it seems not unreasonable to assume that there will be more
> and more need for languages that handle concurrency.
> As you say, not every job needs to worry about that kind
> of close concurrency; but more and more jobs are going
> to.  At least, many people think that's what's coming.
>

I agree that this is coming (and are already here to a certain degree).
I'd love to get an 8 or 16 way machine under my desk !

Haskell already seems to do an awesome job of utilizing all cores in a
machine or allowing the programmer to configure explicitly how the cores
will be used (affinity etc.). Annotating code for concurrent behaviour is
fairly easy in Haskell too :
http://cgi.cse.unsw.edu.au/~dons/blog/2007/11/29#smoking-4core
Haskell is also flexible, there are a few options available for concurrent
programming (including software transactional memory) :
http://cgi.cse.unsw.edu.au/~dons/blog/2007/11/26#no-headaches



>
> This makes for Clojure being the 'industry standard' Lisp, for better or
> worse. Clueless managers may end up praising it the way they did Java after
> reading about it in some magazine.
>
> Well, I'm not sure that's very important.  There are extremely
> good reasons to build a Lisp on top of a JVM, and additional
> extremely good reasons to expose that as part of the language
> definition.  It's not what you'd do if you were designing a
> new language in a vacuum, but these days it's no longer
> realistic to do that.  The ability to take advantage of the existing,
> and always improving, JIT compiler and garbage collector
> technology, the portability, and the ability to tie into the
> huge number of both general and highly specialized
> JVM libraries, has enormous practical value.  The idea
> that we'd ever get enough firepower behind Lisp that
> we could replicate all of that is entirely impractical.
> I had been advocating this kind of approach for at
> least a year before I ever heard of Clojure!
>


I agree that it's important to build a Lisp on top of a JVM in spite of the
potential issues that I mention.
I'm a fan of the JVM too, not as much the language Java or some of it's
libraries but hey "worse is better" and I use those when necessary.
Pragmatism often overrides my desire to write beautiful code in a more
beautiful language.

For all my playing devil's advocate regarding Clojure I really do appreciate
all the work that's gone into it.
It's one of the best Lisp's on the JVM to date. I alternate between thinking
I could use Clojure for a few years happily to actually being turned off by
some of it's design, not to mention the issue of multiple layers of types
which the user has to juggle in their heads (Java types vs. Clojure types
etc.) which I believe Mikel has pointed out elsewhere.

I worked on a project over a year ago where we implemented a prototype(very
solid actually) search server(in Clojure) that used the Lucene Java libs. It
was interesting because this prototype made use of the concurrent features
of Clojure and every step of the way we were benchmarking the performance so
that we could gauge just how scalable the service could be. We wanted to use
all cores in our server. This what at a time before Clojure was really very
popular and even now I have not heard of anyone else that's built something
like this (they may have).
We also built a similar prototype in CL and in C. The C version obviously
took a very long time to build compared with the CL and Clojure versions. It
also was more difficult to make changes to.  The interop in Clojure with the
Java libs felt really clumsy at times and the language itself seemed to get
in the way in places.
The Clojure version was comparable to CL in performance and lines of code.
The end result though was that  Clojure was rock solid in performance and
scaled very well for our needs. In the end we didn't see any compelling
reason to choose Clojure over CL, in fact CL had more power to offer in the
language, so we stuck with it.
We also got bored(didn't see the point) with mapping all the commonly known
names in CL to their Clojure equivalents.

I may just make the source code for the Clojure version available at some
point.
I'll need to check and see if that's ok.


-Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.clozure.com/pipermail/openmcl-devel/attachments/20090521/a0eaa3e7/attachment.htm>