#clojure – 2008-06-04

2008-06-04

01:54	albino	this? http://vim.sourceforge.net/scripts/script.php?script_id=221
02:03	Ycros	could be. I should have bookmarked it
02:05	Ycros	actually no, I think it was Limp (which is linked to from the one you found)
10:49	Chouse1	after I picking that little fight yesterday, I lost track of one of my initial questions...
10:49	Chouse1	Is it better to use (. Class forName "org.postgresql.Driver") or (. java.sql.DriverManager registerDriver (org.postgresql.Driver.))
10:50	Chouse1	...or is there a better way altogether?
10:53	rhickey	I didn't get time to look at why simply naming org.postgresql.Driver was insufficient. But I kind of dislike Class forName - I'd go with the latter
10:53	rhickey	(even in Java I've encountered this with JDBC drivers)
10:54	rhickey	there is also more of a declarative, preferences-based registry I think
10:54	Chouse1	Ok. I hadn't seen any examples doing the later. I didn't know if there might be bad side effects or consequences.
10:54	rhickey	the whole thing is built on the driver self-registering as a side effect of loading - ugh
10:55	Chouse1	yeah. I'm sure whoever made it up thought it was rather clever.
10:56	rhickey	from the DriverManager docs:
10:56	rhickey	As part of its initialization, the DriverManager class will attempt to load the driver classes referenced in the "jdbc.drivers" system property. This allows a user to customize the JDBC Drivers used by their applications. For example in your ~/.hotjava/properties file you might specify:
10:56	rhickey	jdbc.drivers=foo.bah.Driver:wombat.sql.Driver:bad.taste.ourDriver
10:56	rhickey
10:56	rhickey	might be a better approach
10:57	Chouse1	ok
13:02	Chouse1	I just wrote a proxy that maps a vector of vectors to an AbstractTableModel.
13:03	Chouse1	About half of the Clojure method bodies are fewer characters than the Java method names.
13:03	rhickey	heh
13:03	Chouse1	(getColumnName [i] (nth cols i))
13:04	rhickey	doing some Swing?
13:04	Chouse1	I am. First time, really.
13:04	Chouse1	also first time using java.sql, as yesterday's outburst indicated.
13:04	rhickey	:)
13:06	Chouse1	Using doto, I can build up a swing widget tree using code that mimics the hierarchy pretty well, and without naming most of the intermediate panels.
13:06	Chouse1	...which is pretty nice. The Java examples I'm looking at quickly become a tangle of awkwardly named local vars.
13:07	rhickey	using (JThis. ...) makes that pretty declarative too
13:08	Chouse1	Yeah I converted all my (new...)s to (JFoo. ...)s this morning. I'm still pretty ambivalent.
13:08	Chouse1	I think if I had my syntax highlighter pick out the JFoo.'s in a different color, that might make the difference.
13:08	rhickey	Groovy has 'builders' for Swing just for that, Clojure comes close with no special support
13:08	rhickey	highlighting would be excellent, yes
13:10	la_mer	Has anyone really pushed around enclosure yet?
13:10	Chouse1	btw, I wrote my own result-seq because I needed a vector for each row instead of a struct map. With (.method obj) and #() syntax, is significantly shorter.
13:11	Chouse1	sorry, resultset-seq
13:11	Chouse1	la_mer: I'm meaning to, but haven't yet. I get the impression the creators are using it pretty heavily.
13:11	Chouse1	lunchtime.
16:23	Chouse1	for clojure to work in javascript, you might not need a lispreader in js, but surely you'd at least need the compiler, wouldn't you?
16:23	Chouse1	otherwise how would (eval) work?
16:27	rhickey	It depends, no eval is distinctly possible
16:29	Chouse1	as in, clojure might work even if it didn't have eval?
16:29	la_mer	wow, gen-class is a whopper
16:29	la_mer	incredibly cool, though :-)
16:30	rhickey	there are 2 distinct ways to think about ClojureScript, one is a Clojure interpreter in JavaScript, the other is a compiler from Clojure to JavaScript (written in Clojure, and doing macroexpansion there)
16:32	Chouse1	right. They would have distinct feature sets or usage profiles, right?
16:32	rhickey	I think a Clojure (subset) to js compiler is more interesting then sending Clojure code to the browser for interpretation, YMMV
16:33	Chouse1	that would mean some kind of (to-js) function or macro that consumes a file or clojure code/data structures and produces a string, right?
16:34	rhickey	yes
16:34	Chouse1	someone mentioned starting that already.
16:34	Chouse1	cgrand
16:35	Chouse1	http://clojure-log.n01se.net/date/2008-05-29.html#17:56
16:35	rhickey	there are js correlates for many of the special ops, then just do macroexpansion on the clojure side to get more language, plus something like the Java syntax for talking to the js stuff
16:36	Chouse1	I'm trying to think if there are any features I think are really useful that would require a compiler/interpreter on the js side.
16:36	rhickey	a very early Clojure prototype was written in CL and emitted js
16:37	rhickey	the thinking then was that js could be compiled in both java/.net, but .net's jscript had too many small differences
16:38	rhickey	but a small lisp->js is pretty easy.
16:40	rhickey	interesting quote here: http://bc.tech.coop/blog/030920.html
16:41	rhickey	"Those of you who are familiar with more traditional functional languages, such as Lisp or Scheme, will recognize that functions in JScript are fundamentally the Lambda Calculus in fancy dress. (The august Waldemar Horwat -- who was at one time the lead Javascript developer at AOL-Time-Warner-Netscape -- once told me that he considered Javascript to be just another syntax for Common Lisp. I'm pretty sure he was being serious; Waldemar's a
16:41	Chouse1	that got cut off, but I've heard it elsewhere.
16:42	Chouse1	In fact, I'm pretty sure I've seen posts from the guy who started JavaScript that he was hired to create a "scheme with Java-like syntax"
16:44	Chouse1	http://weblogs.mozillazine.org/roadmap/archives/2008/04/popularity.html
16:44	rhickey	cross-compiling Clojure is very doable. I compiled to both Java and C# for a long time. The only thing that doesn't map well is blocks (do) as expressions - if only the had kept C's comma operator!
16:45	rhickey	and loops as expressions
16:45	Chouse1	"Whether that language should be Scheme was an open question, but Scheme was the bait I went for in joining Netscape"
16:46	Chouse1	developing code for use in the browser would either require a compilation step, or clojure running in a server.
16:47	rhickey	I imagine most Clojure web apps would have Clojure running in a server
16:47	Chouse1	I suppose if you had the latter, you might with some effort be able to load updated functions without reloading the whole app in the browser.
16:47	rhickey	as cgrand is doing, js and html gen could happen together
16:48	rhickey	most of the servlet containers have comet support now
16:50	rhickey	funny, this just came by - ecmascript 4 to js compiler written in server-side Python:
16:50	rhickey	http://ecmascript4.com/
16:51	Chouse1	huh.
16:54	rhickey	I think the important thing that makes the Clojure->js job small is that the macros get expanded by and in Clojure, so you never have js versions of cond, when etc, just if
16:54	Chouse1	I assume cgrand is just writing functions to do the conversion. Any reason to think that's not the best way?
16:55	la_mer	rhickey: Am I right in thinking that, assuming each namespace has appropriate metadata lined up, that gen-and-save-class could be used to dump a pile of loaded clojure to disk?
16:56	rhickey	la_mer: no, the Clojure code behind a gen-ed class is not in the .class file, just the stub
16:57	rhickey	Compiled Clojure code can have data literals in it, more complex to pre-compile.
16:58	rhickey	at some point I might look at an ahead-of-time compiler
16:58	la_mer	Ah, that's why I'm getting "UnsupportedOperationException: XXXX not defined" when I try to load up the generated class -- it's not finding a matching .clj file to load the impl's from.
16:59	rhickey	the impl can be loaded pretty late, but must be before it's called
16:59	rhickey	can also be changed after objects exist
17:00	rhickey	Chouser: I haven't looked too closely at cgrand's js stuff
17:01	rhickey	but bringing up a Lisp is always the same - a handful of special ops to bootstrap
17:02	Chouse1	right, I'm wondering if you think implementing the conversion in macros and clojure funcs would be better or worse than some other option, like an alternate Compiler class.
17:04	la_mer	Yeah. I just caught wind of the gen-and-save-class utility, and thought that perhaps it would enable source-less distribution.
17:04	rhickey	Chouser: ah, well, certainly not a Compiler class in Java
17:05	Chouse1	:-) You would have written yours in clojure if you could have, would you.
17:05	rhickey	definitely, it was always a hope to bootstrap it, but practically, once you have a working compiler, you're pretty tired :)
17:06	Chouse1	heh
17:07	rhickey	Take the Clojure time machine: http://clojure.svn.sourceforge.net/viewvc/clojure/trunk/src/lisp/clojure.lisp?view=markup&pathrev=85
17:08	Chouse1	hm, looks like CL ;-)
17:08	rhickey	CL to Java/C# compiler for an early Clojure
17:11	rhickey	I don't think you'd need a full Clojure in js, but the ability to gen js from sexprs with macros would be really nice
17:13	rhickey	so you use macros to get a tree of special ops, then walk the tree and emit js
17:13	Chouse1	I've written quite a bit of js (we've got over 38K lines for our project at work), and I'd be pretty frustrated if the environment were crippled or any less dynamic than js is by itself.
17:14	Chouse1	yeah, that doesn't sound too hard, does it? I wonder how far cgrand has gotten.
17:14	rhickey	wow! 38k
17:14	Chouse1	yeah, scary, right?
17:15	Chouse1	that doesn't include the 3rd-party libs we use (zimbra, google maps, openlayers, etc.)
17:15	rhickey	keeping all of the js goodness would have to be a focus
17:15	rhickey	but you'd take the same approach as with the Java - define minimal syntax support for direct calls
17:16	Chouse1	I guess one question would be -- how much would the data consumed by js* have to look like clojure source?
17:16	rhickey	do you eval your json?
17:16	Chouse1	yeah, but that could stay json.
17:17	rhickey	then either way, I think a js reader for Clojure data format wouldn't be too much
17:17	rhickey	or translate to json at the gateway
17:17	Chouse1	you'd want js* to understand literal hash and vector syntax of course, but it clearly wouldn't need to do sync or agents.
17:18	Chouse1	...so would it need persistent data or not?
17:18	rhickey	right, there isn;t a need to bring semantics to js it doesn;t already have
17:18	rhickey	I don;t think persistent
17:18	Chouse1	I guess the clojure idioms might start to break down if you have js arrays instead of vectors. ...not sure.
17:19	rhickey	depending on your data size you can just copy on write
17:20	rhickey	but it is an issue for Clojure idioms
17:21	Chouse1	I don't think having conj modify an existing array would be acceptable, for example. If you wanted to do non-persistent you'd have to use a different verb.
17:21	rhickey	yes, definitely
17:21	rhickey	but Clojure has aset for Java arrays
17:22	Chouse1	ah, good point.
17:22	Chouse1	but being forced to use aset all the time would be painful.
17:22	rhickey	the main problem is data size - copy on write preserves the semantics but doesn't scale
17:22	rhickey	but you may not need to scale
17:23	Chouse1	...so I think that answers it (in my mind anyway) -- we'd need some kind of immutable list and hash on the JS side.
17:23	rhickey	list is easy
17:23	Chouse1	right, it might not need to scale, but it'd have to be there in order for it to feel anything like clojure.
17:24	rhickey	js has everything you need to implement the hash map
17:24	rhickey	just arrays
17:25	Chouse1	hm. It'd be interesting to port hashmap to clojure so that it would work in js.
17:25	rhickey	Clojure leans heavily on Java's good ephemeral memory handling, not a known attribute of js
17:25	Chouse1	yeah, but again that mainly has to do with scaling. ..and the JS VMs will keep getting better.
17:26	rhickey	I think you'd write it by hand in js and preload is as runtime support
17:26	Chouse1	why?
17:27	rhickey	because you don't have a Clojure->js compiler yet
17:27	Chouse1	but you could have a COW vector to get you by... ;-)
17:28	Chouse1	or even just JS array support.
17:28	rhickey	object models
17:29	rhickey	I'm not arguing against it
17:29	rhickey	Just that, I would have had to have had Clojure very far along in order to have it generate PersistentHaspMap
17:30	rhickey	ymmv
17:30	Chouse1	hm. Yeah, I guess the JS situation isn't any further along than you would have been with the Java version.
17:31	rhickey	It depends a lot on what the js would look like - functional or an object?
17:32	Chouse1	which in my experience hinges mainly on whether you want to use JS inheritence. If you don't need inheritance, closures are great for creating objects.
17:33	Chouse1	gah!
17:33	rhickey	It leverages Java, as I think it should given the layer it is at.
17:33	rhickey	when in Rome
17:34	Chouse1	what's missing?
17:34	Chouse1	I guess the flip side is having created hashmap in the host language, you can use it outside of clojure if you want.
17:35	rhickey	classes with typed fields, primitive bit math, static methods
17:36	rhickey	yes, the Clojure Java lib has utility outside of Clojure, and can leverage all kinds of things I don't intend to add to Clojure
17:36	rhickey	lest Clojure inherit the complexity of Java
17:37	Chouse1	but that's because it's a Java class. surely that kind of functionality could be designed for and implemented in clojure.
17:37	Chouse1	if not, then isn't that a problem?
17:39	rhickey	no, because in order to make a language go all the way down and match the efficiency it has to have primitive types
17:39	rhickey	that was no to - isn't that a problem
17:40	rhickey	yes you can implement abstractions like that, but they will have some cost due to their higher-level nature
17:41	Chouse1	oh, ok.
17:42	Chouse1	As long as the barrier is performance and not expressiveness, I'm ok. ;-)
17:43	rhickey	I think Clojure holds it own on expressiveness
17:44	Chouse1	yes
17:44	rhickey	OTOH, I'm pretty stunned that I was able to get the primitive math as fast as Java without emitting math primitive bytecodes
17:45	Chouse1	It feels a little odd to have to use java interop (proxy, gen-*-class) to create data structures that have compatible APIs with the buildins, but I guess that makes sense in light of your performance statement.
17:47	rhickey	a big benefit of that is that the resulting thing is native. virtual method dispatch is also very fast. Some other JVM dynamic langs interpose a wrapper/bridge class between everything
17:47	rhickey	to map to their object models
19:24	rhickey	Chouser: what's the time for this on your machine:
19:24	rhickey	(loop [acc (double 0) ind (int 0)]
19:24	rhickey	(if (== ind (int 100000000))
19:24	rhickey	acc
19:24	rhickey	(recur (+ acc
19:24	rhickey	(. java.lang.Math (sqrt ind)))
19:24	rhickey	(unchecked-inc ind))))
19:24	rhickey	vs what you posted?
19:28	Lau_of_DK	Anybody know what the reason for implementing primitive support in Clojure was?
19:28	rhickey	speed
19:29	rhickey	some of us do audio/image processing
19:29	rhickey	nice to stay in Clojure
19:30	arbscht	avoids the "glue language" trap neatly
19:30	Lau_of_DK	oh - I just heard a talk you gave, rhickey, to a group of Lispers from NYC and just near the end I heard the sentences "there will never ever be primitives in Clojure" ?
19:31	rhickey	and there aren't, really. It doesn't generate the bytecodes for primitive arithmetic itself
19:32	rhickey	It isn't something I expected to work this well. Never say never I guess :)
19:32	Lau_of_DK	Alright, thanks for clearing it up :)
19:32	Lau_of_DK	Good talk though - You got quite the applause
20:01	Ycros	rhickey: hey, I saw an IRC log of you saying that Clojure doesn't need conditions/restarts because it can be implemented using dynamic function binding?
20:02	Ycros	rhickey: is there something I can read, or can you shed some light on how this might be done?
20:03	rhickey	http://groups.google.com/group/clojure/msg/842e8bb058015282
20:03	Ycros	thanks :)
20:04	rhickey	sure. note I said similar, not exactly CL conditions
20:05	Ycros	oh I see, that's fairly simple
20:05	rhickey	simple meaning easy or simplistic?
20:10	Ycros	easy I guess
20:11	rhickey	you can build richer abstractions on that if needed
21:26	Chouser	ah, I was trying to use a primitve ind, but was missing the comparison literal.
21:28	Chouser	but still, my posted loop ranges from 3132 to 3293 msecs, while rhickey's with unchecked-inc ranges from 3121 to 3293
21:37	Chouser	rhickey: you missed my answer, and nothing else.
21:37	Chouser	I had tried to use a primitve ind, but missed the comparison literal.
21:37	Chouser	but still, my posted loop ranges from 3132 to 3293 msecs, while yours with unchecked-inc ranges from 3121 to 3293
21:38	rhickey	unchecked-inc doesn't matter, but I think declaring ind int does - yours went from 5secs to 3.5 here
21:39	Chouser	oh! hm. I wonder why the difference between the improvement here and there.
21:40	Lau_of_DK	this might be a really dumb question, but is the swank-clojure any different from a regular swank ?
21:43	rhickey	Chouser: I don't see how they could be the same speed - hmmm....
21:44	rhickey	since both the compare and the inc have to be generic without the hints
21:44	rhickey	are you java 6?
21:46	Chouser	right. When I tried (int 0) for ind but forgot (int 1000...), the performance was worse, I assume because of extra boxing and unboxing.
21:46	rhickey	you are on java 6?
21:48	Chouser	I think so. I'm trying to verify that.
21:49	rhickey	java -version
21:49	Chouser	ah. yeah, 1.6.0-b09
21:50	rhickey	interesting. I'm stuck on 1.5, I've heard the optimizations are much better on 6, looking forward to that. I guess the main point from the benchmark was that Clojure is the fastest ;)
21:51	Chouser	:-)
21:51	rhickey	To be fair I think Kawa supports some hints that would make it faster
21:52	rhickey	And I'm not sure that is the maximum opt from SBCL
21:52	Chouser	yeah, it's hard to be fair in a comparison like that without knowing each language pretty well.
21:52	rhickey	but Clojure should always be == Java in this area now, which means plenty fast
21:52	Chouser	I thought a C version might be interesting.
21:53	Chouser	for some value of interesting.
21:53	rhickey	sqrt is bad, some arithmetic would be better
21:53	rhickey	rms or something
21:54	Chouser	Java's sqrt is bad, or what?
21:54	rhickey	sqrt is not primitive arithmetic
21:54	Chouser	ah, true.
21:56	rhickey	and dominates his benchmark
21:56	rhickey	fyi, from REPL:
21:56	rhickey	(-> System .getProperties (get "java.runtime.version"))
22:01	Chouser	thanks
22:04	Chouser	(reduce + (map #(. java.lang.Math sqrt %) (range 10000000)))
22:06	Chouser	That's just hopeless, isn't it.
22:06	rhickey	2.1081849486439312E10
22:07	Chouser	If I replaced #() with something defined using definline, range would still be returning boxed numbers, right?
22:07	Chouser	and + would be generic?
22:12	Lau_of_DK	rhickey, what are the terms for building commercial software off clojure?
22:14	rhickey	Chouser: takes 4 secs here
22:15	Chouser	yeah, it's 1/10 the benchmark. sorry for the confusion
22:16	Chouser	10000000 instead of 100000000. how could you have missed that? ;-)
22:16	Chouser	Lau_of_DK: http://clojure.org/license.html
22:16	rhickey	sorry, doing 2 things at once here
22:18	Lau_of_DK	Thanks Chouser :)
22:20	rhickey	Chouser: laziness in map will dominate:
22:20	rhickey	(time (reduce #(+ %1 (. java.lang.Math sqrt %2)) 1 (range 10000000)))
22:24	Chouser	that's still 898 msecs instead of 327 msecs for the loop version.
22:29	rhickey	I consider that pretty good, considering range is a seq abstraction
22:31	Chouser	oh, sure! I'm not saying it's bad, I'm just kicking around with more idomatic forms.
22:38	rhickey	the cool thing about the primitive math is that it is amenable to being the target of higher-level macros, like amap and areduce
22:39	rhickey	I can imagine whole DSLs for different formula-building patterns
22:39	rhickey	but it will be macrology, the higher-level constructs have some fixed costs
22:44	rhickey	and 10 million steps of reduce and range in a second is fine perf for most apps. Only serious crunchers need get into this micro-optimization
22:44	Chouser	yep, good point.
22:45	JamesIry	rhickey: This is somewhat off topic of Clojure the language and more about its implementation. I've been reading papers on various STM impls lately. I remember you writing that Clojure uses MVCC. Do you have any links about the kinds of workload that MVCC based STM is good or not good for? Or has there even been any research into MVCC STM?
22:47	rhickey	JamesIry: I read a bunch of papers on STM while I was designing Clojure's. They all left me wondering a bit about the context of use...
22:48	rhickey	i.e. they seemed to focus on dropping STM under otherwise mutation-oriented apps...
22:48	rhickey	fine grained field-level refs...
22:48	JamesIry	Agreed. Many of the benchmarks they user are hashtables or arrays where everything is a mutable ref.
22:48	JamesIry	s/user/us
22:48	rhickey	while I always was thinking about refs to immutable persistent data structures, more of a record orientation
22:48	JamesIry	s/user/use!
22:49	rhickey	so turned to databases for inspiration
22:49	rhickey	as I consider STM to be the same problem in many ways
22:49	JamesIry	Very much so!
22:50	rhickey	spent lots of time with Gray and Reuter's "Transaction Processing"
22:50	rhickey	and whatever info I could get on Oracle and Postgresql
22:53	rhickey	I wasn't aware of another MVCC STM when I did my work, but this came up recently:
22:53	rhickey	http://www.esw.inesc-id.pt/~jcachopo/jvstm/
22:53	JamesIry	I think it would be interesting to investigate the kinds of workload where the overheads of MVCC are small compared to the overheads of redoing the work, vs where the overhead is large. One interesting possibility is to have a pluggable STM impl so that application writers could do their own perf testing to determine which fits best.
22:54	JamesIry	Thanks for the link
22:55	rhickey	after reading Gray and Reuter I would never want anything other than MVCC, in an STM or a DBMS
22:55	JamesIry	:)
22:55	rhickey	not that they advocate it, my own conclusion
22:55	Lau_of_DK	rhickey, "never" ?
22:55	rhickey	for some value of never
22:55	JamesIry	Yeah, how 'bout a very low contention scenario
22:56	rhickey	I just don't see where MVCC loses
22:56	rhickey	low contention everything works well
22:57	rhickey	Gray and Reuter do show some serious disdain for optimistic concurrency
22:57	rhickey	That's why Clojure has real blocking
22:58	JamesIry	One paper I've been studying (http://www.cs.rochester.edu/~vmarathe/research/papers/2008_PPoPP.pdf) has an algorithm specifically optimized so that low contention is cheap. It's C based, though, and may be too expensive to translate into JVM terms.
23:01	rhickey	will read, thanks
23:01	JamesIry	The central part is based on 64 bit CAS - and that's not something that you can rely on being reasonable on every JVM
23:03	JamesIry	I'll have to pick up the Gray and Reuter book
23:03	rhickey	I have a philosophical problem with allowing more work to proceed than can succeed
23:04	JamesIry	rhickey: I hear you. It's okay if the work is cheap compared to the costs of blocking...but at the datase level that's almost certainly not true. When dealing with in memory refs to large immutable structures it probably still isn't true.
23:05	JamesIry	That's one problem with the benchmarks that many STM papers use. They pound the hell out of the refs, but few actually do much work in between reads and writes.
23:06	rhickey	I just see no correlation to the reality I;ve experienced writing MT programs in those papers
23:06	JamesIry	It's a classic microbenchmark miopia.
23:06	rhickey	For me, STM is a way to automate and make correct the same kind of locking that would be done manually in a real program
23:07	rhickey	for super-contended caches the stuff Cliff Click is doing will work fine, I wouldn't try to do that with STM
23:07	JamesIry	Sure...but in some cases doing CAS and retries makes sense when you do things manually.
23:08	JamesIry	Yeah...his stuff is definitely the kind of thing where special purpose constructs like concurrent maps make more sense.
23:08	rhickey	he's in the CAS world
23:09	rhickey	there's likely no one-size-fits all
23:10	JamesIry	That's why it would be interesting to see a suite of benchmarks againt various STMs. Some benchmarks could do real work so that there's a cost to any retries. Others could just pound crap out of the refs so that blocking itself becomes a major overhead.
23:11	JamesIry	Users and designers of STMs could then evaluate which work load is more nearly like their own - or, as I was thinking earlier, empirically try several impls.
23:12	rhickey	It's an area (realistic benchmarks) that needs attention
23:13	JamesIry	One thing I'm excited about is languages like Clojure taking concurrency seriously as a first class concept.
23:14	rhickey	It's got to be more than an add-on or it won't work. I don't think people can just drop STM into the same old approach. Immutability has to be taken seriously, for instance
23:15	JamesIry	Agreed.
23:16	rhickey	I would hope Clojure offers a model (persistent data structures in refs) that is compatible with it's STM approach
23:16	JamesIry	It's going to be a learning curve, though. It will take a while for people to wake up from the illusion that state can be encapsulated.
23:17	JamesIry	The only way to encapsulate state is to limit the scope of what it can possibly affect - OO doesn't do that at all well. The state of one object affects the transitive closure of all objects with references to it.
23:18	rhickey	A recent thing I added to Clojure allows you to make Java objects whose state was a ref - transactional objects if you will. Still thinking through the implications...
23:19	JamesIry	Not sure I follow
23:19	rhickey	the problem with OO is the direct references to mutable things. Making them indirect and transactional, to immutable things, can let it feel quite similar while ensuring that people can only ever see consistent objects, without needing locks
23:19	JamesIry	I got ya
23:24	rhickey	one of the nice things about using a ref to an immutable map, say, for an object's state, vs a bunch of fields each of which was an STM ref, is that you can, outside of a transaction, get a consistent object
23:27	JamesIry	Yup. The context in which I've been exploring STM is to create one for Scala. Scala's story on immutability isn't as good as Clojure's, though. It's got immutable persistent structures in the standard libary, but it doesn't really discourage mutable objects either. So I forsee much trouble with people trying to play with ordinary mutable objects in transactions and getting strange results.
23:28	Chouser	Scala doesn't have stong immutable vectors or hashes, does it?
23:29	JamesIry	Vector, no. Set, hash map, etc yes
23:29	rhickey	biting the bullet on immutability for all of the core data structures is one of the best things I did for Clojure. It's worked out better than I thought and I've gotten very little pushback from users, surprisingly.
23:29	Chouser	hm. I must have missed that.
23:30	JamesIry	http://www.scala-lang.org/docu/files/api/scala/collection/immutable$package.html
23:32	rhickey	what are the perf characteristics for the hash map O(log N) ?
23:32	rhickey	immutable != persistent
23:32	JamesIry	Most of their structures are persistent
23:33	Chouser	are those copy-on-write?
23:33	JamesIry	I think there's some ugliness in the way hashmap is done, though. I vaguely recall that that it was high on the list to get reimplemented.
23:33	JamesIry	Chouser: I don't think they used copy on write
23:34	rhickey	persistent just meaning old versions still available, or persistent meaning with perf characteristics intact? Amortized or not?
23:34	rhickey	if you happen to know...
23:34	JamesIry	I don't know all the perf characteristics. Some are amortized using laziness and whatnot.
23:35	JamesIry	It would be interesting to get you and them in a room. You guys are solving similar problems on the same platform.
23:35	JamesIry	Room could be virtual, of course :-)
23:35	rhickey	Yes, I'd love to. I have to admit to having spent almost no time with Scala :(
23:36	JamesIry	Well...it's a bit like Java and ML had a love child.
23:36	JamesIry	Very different design philosophy from Clojure!
23:37	rhickey	Clojure owes some bits to ML
23:37	rhickey	no type system though
23:37	JamesIry	Yeah...I gathered the explicit ref types was a bit inspired by ML
23:37	rhickey	exactly
23:38	JamesIry	One of the things on my list is to investigate the Clojure/Scala boundary. Both will encode closures differently, for instance, but it seems like it might be easy enough to translate.
23:39	rhickey	Well, Clojure is pretty transparent to Java - there are interfaces for everything, it uses the same stack and calling conventions, and there are no magic arguments, trampolines, CPS etc
23:40	rhickey	no wrappers
23:40	JamesIry	Same with Scala - that's one area where both languages took the same philosophy: embrace the platform don't try to beat it into submission.
23:40	rhickey	right
23:41	JamesIry	In fact, with Scala if you stick to the Java-like subset you get pretty much the same bitecode as you'd get from Java. Things like closures just compile to instances of interfaces called FunctionN (where N is the number of arguments).
23:42	JamesIry	s/bitecode/bytecode
23:42	JamesIry	Sigh...fat fingers tonight
23:42	rhickey	Clojure has a single interface, IFn, with overloaded invoke method for different arities
23:42	JamesIry	Ah
23:43	rhickey	allows you to pass a variadic function object to map et al
23:44	JamesIry	Right...good point. Scala's variadic functions are more like Java - the variadic argument becomes a single argument. The difference is Java uses and array, Scala uses an immutable list.
23:44	JamesIry	I suspect there are a fair number of differences like that. Many of them largely driven by the dynamic vs static nature of the languages.
23:45	rhickey	yes
23:46	rhickey	I expect between the two we cover a lot of what people with this, er, mindset might want
23:46	JamesIry	Yup...the languages aren't going to attract the same crowd.
23:46	Chouser	I was attracted to both.
23:47	Chouser	...though I guess I'm pretty firmly here now. ;-)
23:47	JamesIry	Well, me too :)
23:47	rhickey	I do think they both attract people interested in programming at a higher level than Java.
23:48	rhickey	People will divide on static vs dynamic as always
23:48	JamesIry	Of course. Plus the standard Lispy macro stuff to which Scala has no answer at all right now.
23:49	rhickey	I see the next divide being about mutability, where Clojure and Scala may end up on the same side, the FP guys having gotten there first, but Java and the scripting langs having no good story
23:49	JamesIry	Well, I shouldn't say no answer, things like optional laziness and a reasonbly flexible syntax allow Scala to do SOME of the things I might want to whip up a macro for in a lisp. But only some.
23:50	JamesIry	rhickey: that is THE divide. the static vs dynamic flame wars are silly when compared to something as fundamental as that.
23:51	rhickey	and yet, most people still don't even see that coming
23:52	rhickey	multicore is going to smash into imperative programming like a train wreck
23:54	Chouser	every method of scala's hashmap is synchronized -- that means locked, right?
23:54	JamesIry	The hashmap impl is the one that sucks right now. I know its on their list
23:54	Chouser	even the get method, apparently because a get can trigger a mutation.
23:54	Chouser	they should just use rhickey's.
23:54	Chouser	;-)
23:54	JamesIry	:)
23:55	JamesIry	Damn straight!
23:55	rhickey	Chouser: that's not inherently wrong, if they yield effective immutability
23:55	Chouser	I can't think why they shouldn't. Same license, I believe. rhickey's is Java, so they could just start using it.
23:55	Chouser	rhickey: I don't think it's incorrect, just not as efficient.
23:55	JamesIry	Chouser: The api would be wrong, but the core algo would be fine
23:56	JamesIry	rhickey: it's not wrong, it's just silly. Plus, if I expect I'm using a normal immutable and persistent structure I'm not going to think about things like deadlock
23:57	Chouser	it looks like they keep a singly-linked list of changes to the structure, and occasionally re-collect into a new hash table. ...but I could be reading it wrong.
23:57	rhickey	those are all possible implementations, delta chains etc
23:58	rhickey	would need to be synchronized
23:58	Chouser	JamesIry: sure, but tidy little scala wrapper around PersistentHashMap would be easy.
23:58	rhickey	not necessarily a deadlock risk, if there are no nested locks
23:58	rhickey	I tried a bunch of things like that
23:59	JamesIry	rhickey: that's a good point. I suppose if some equals or hashmap method on a key was synchronized there could be risk of deadlock, but that's a pretty pathological case.