#clojure – 2009-06-01

2009-06-01

04:21	cads	hey, would a multilanguage VM running java do tailcall optimization?
04:30	cgrand	cads: http://wikis.sun.com/display/mlvm/TailCalls the experimental branch of the JVM does TCO
04:35	cads	cgrand I could smooch you but I won't
04:36	cads	this is great!
08:04	hsuh	hi... whats the current recipe for installing slime+swank? svn versions aren't working for me... anyone?
08:07	jdz	hsuh: you need swank-clojure
08:07	jdz	hsuh: from the git i think
08:07	hsuh	yes, all that
08:08	hsuh	so what i currently get is: the REPL starts and is functional, but i get some errors (java.lang.Exception: Unable to resolve symbol: lazy-seq in this context (core.clj:70), java.lang.Exception: No such var: swank.swank/ignore-protocol-version (NO_SOURCE_FILE:3), java.lang.Exception: No such var: swank.swank/start-server (NO_SOURCE_FILE:5)) and slime keeps polling or something
08:21	hsuh	hm... here we are.. replacing lazy-seq (not found) with seq makes things work.. anyone knows if that was removed from clojure or i need to require a library before calling it ?
08:25	Chouser	hsuh: yeah, lazy-seq has been gone awhile.
08:25	cemerick	it sounds like you have an old version of clojure. lazy-seq is definitely in clojure, since before v1.0 was marked.
08:26	Chouser	oh
08:26	cemerick	wha? :-/
08:26	Chouser	uh
08:26	Chouser	sorry. I blame monday.
08:26	Chouser	cemerick is, of course, correct.
08:26	cemerick	whew, I thought I was going bananas!
08:26	cemerick	Chouser: you were probably thinking of lazy-cons
08:27	Chouser	cemerick: yes, of course. Thanks for making my mistake sound reasonable. :-)
08:27	cemerick	Chouser: heh. How goes the new gig, anyways? I hope all's well in general.
08:27	Chouser	yep, thinks are well, can't complain, etc. :-)
08:28	Chouser	checked in some clojure code last week, so we'll see how that goes.
08:29	cemerick	is clojure still in skunk-works mode there?
08:30	Chouser	a little higher profile than that, but only in "prototype" code for now -- slated to be replaced by something that performs better later. ...again, we'll see. :-)
08:32	cemerick	Good luck. We actually just made some huge strides in performance. We now have a use-case for pmap, which really helps -- our older data structures weren't parallelizable, but starting early last week, we switched to a new set of impls that are...and wow, the difference is fun :-)
08:37	Chouser	ah, great.
08:38	Chouser	did you see Rich's comment about pmap and chunked seqs?
08:39	cemerick	uh, no. My list inbox is oh-so-neglected.
08:39	Chouser	nah, it was a passing irc comment.
08:39	cemerick	ah. log link?
08:40	Chouser	http://clojure-log.n01se.net/date/2009-05-29.html#07:40
08:41	cemerick	oh, so chunking (not that I'm clear about what chunks are at all, of course) makes seqs almost naturally parallelizable, in that the work partitioning is almost done for you....?
08:42	Chousuke	hmm
08:42	cemerick	Our two usages of pmap happen to be in cases where we always have to invoke a particular fn over a dataset with a couple of different parameters, making for an obviously-parallelizable process.
08:43	Chouser	not quite. more like the overhead of doing 'next' (calling down the stack, allocating a new seq, etc.) is enough that in a lot of cases pmap doesn't make sense right now.
08:43	cemerick	Oh, I see. Reaching for the stars, I am :-)
08:44	Chouser	but with a chunked seq (such as a seq over a vector, for example) that overhead is required much less often, so pmap may make sense of less heavy funcs.
08:44	Chouser	:-)
08:45	hsuh	cemerick: i just compiled from svn...
08:45	cemerick	I was thinking of how seq producer might know how to partition a dataset better than its consumer, so the chunks could be sized appropriately for pmap-ing.
08:45	Chouser	seq producers do the chunking, yes.
08:45	cemerick	hsuh: hrm. I don't work off of trunk, but I'd be very surprised if rich pulled something like lazy-seq...
08:46	hsuh	maybe i need to require something (lazy) ?
08:46	Chouser	hsuh: can you check at a repl if you've got lazy-cons?
08:46	hsuh	ser=> lazy-seq -> java.lang.Exception: Unable to resolve symbol: lazy-seq in this context (NO_SOURCE_
08:46	Chouser	if you do, something's definitely amiss.
08:46	cemerick	hsuh: no, it's in the std lib, as it were. I'd grab the latest release jar, start with that to get a baseline, and then move on to build from sources if you're so inclined.
08:46	hsuh	user=> lazy-cons java.lang.Exception: Can't take value of a macro: #'clojure.core/lazy-cons (NO_SOURCE_FILE:0)
08:46	hsuh
08:46	cemerick	yeah, that's very old
08:46	hsuh	cemerick: k thanks
08:47	hsuh	i've done something wrong :)
08:47	cemerick	hsuh: what svn URL are you using?
08:47	Chousuke	hsuh: did you get it from sourceforge? :P
08:47	cemerick	man, it's a groupthink in here :-)
08:47	hsuh	hmm err :)
08:47	hsuh	yes...
08:47	Chousuke	heh
08:47	hsuh	ops
08:47	cemerick	hsuh: http://code.google.com/p/clojure/
08:47	Chousuke	clojure is on google code nowadays
08:48	hsuh	and i thought i was on the edge
08:48	hsuh	hehe
08:48	cemerick	wow, I didn't think the SF svn was still up and running. Seems like it should be switched off.
08:48	hsuh	i'll ping jochu to update github page for swank-clojure
08:48	hsuh	because github feels updated... so i blindly followed it..
08:49	hsuh	well, thanks a lot
08:50	cemerick	hsuh: hth :-)
10:58	cemerick	hrm, I'm getting odd pauses at the end of compilation cycles -- I suspect dangling agent threads. Anyone hit this before?
11:02	cemerick	ah, removing the pmap calls I just added makes the compilation process finish immediately as before....
11:25	hiredman	http://groups.google.com/group/clojure/browse_thread/thread/b6f192eca22f6820 <-- the kind of maddess that insues if you combine quote and syntax-quote
11:27	Chouser	cemerick: didn't see your posts, sorry. If you use futures or agents (which pmap does), you'll have to shut down the agent pools to exit cleanly.
11:27	drewr	..and even then maybe not :-)
11:28	cemerick	Chouser: I'm about to post to the group with a patch. To be clear though, none of the pmap calls were invoked -- but their usage in the code probably caused the loading of c.l.Agent, which sets the threadpools off on their way.
11:28	Chouser	well, I've been doing a lot of that, and haven't had issues. I saw there was a thread, but didn't read into it deeply enough to spot the difference.
11:28	Chouser	hm.
11:29	cemerick	Chouser: the patch I'm proposing adds an Agent.shutdown() call to finally blocks in the entry points of clojure.main and clojure.lang.Compile. Does that seem reasonable?
11:29	Chouser	I don't think your diagnosis is correct.
11:30	Chouser	clj -e "(fn [] (agent nil))" ;exits promptly
11:30	Chouser	clj -e "(agent nil)" ; exits promptly
11:30	Chouser	clj -e "(send-off (agent nil) identity)" ; hangs
11:30	cemerick	ah, that code is run, so yes, the agents are being used. My bad.
11:31	Chouser	cemerick: and so you know, agent shutdown calls and/or System/exit calls have been added and removed from the various entry points a few times. People always complain either way.
11:31	cemerick	(creating a default EMPTY value)
11:31	cemerick	hrm.
11:32	Chouser	the problem the other direction being if you have an agent or something that shuts down the pool when it's the last one out, and your main program just falls off the end, an automatic pool shutdown would break things.
11:32	cemerick	well then, certainly no one could complain about a shudown call in clojure.lang.Compile, which would fix my immediate problem.
11:32	Chouser	hm.
11:35	cemerick	In general, it doesn't seem like everyone should keep tripping over this problem in different directions. It's a frustratingly difficult thing to debug if you're not attuned to it already. :-[
11:39	Chouser	in my experience, for any particular app it's pretty hard to know when all other threads are done and it's safe to exit.
11:39	drewr	cemerick++ It's extremely difficult to debug. I think I have mine figured out and then I see different behavior.
11:39	Chouser	it's hard for me to imagine how it could be solved automatically.
11:39	rhickey	hmm... #clojure on the plane - awesome
11:41	Chouser	rhickey: nice.
11:41	Chouser	heading to CA?
11:41	drewr	Chouser: Yep, I finally had to remove agent activity from my script. Couldn't determine even for my own program what the right behavior was.
11:41	replaca	rhickey: cool. best wishes for the script bowl tomorrow!
11:41	rhickey	yup
11:41	cemerick	Chouser: why not declare that agents are shutdown when the main thread exits, period, and if you don't want that to happen, join as necessary?
11:41	rhickey	replaca: thanks!
11:41	cemerick	rhickey: Indeed, good luck :-)
11:41	rhickey	the 4 minute limit is quite short
11:41	cemerick	I just don't see why everyone that works with agents should have to trip over this particular issue.
11:41	replaca	rhickey: do you want me to make a hand-out/poster for you for the clojure meetup, or do you want to just let that be?
11:41	cemerick	or actually, not even agents, just a simple pmap usage ropes you into the same area...
11:42	cemerick	(as in my case)
11:42	rhickey	replaca: I don't know that I could hand them out, but you could
11:43	replaca	rhickey: I won't be there - I guess I could come and see if there are some public places to post
11:43	rhickey	replaca: ah, right
11:43	replaca	I think they'll only check at the entrance to the sessions, so that would probably work
11:43	rhickey	I imagine those who would come are on the ggroup or here
11:44	replaca	rhickey: I'll do that.
11:44	replaca	rhickey: Yeah, I think you're right + anyone you sufficiently inspire
11:44	replaca	The twittersphere has had a bit of buzz about Stuart's book!
11:46	Chouser	cemerick: I guess that may be reasonable. For the cases where my code is carefully collecting the threads before agent-shutdown, it would be no harder to do that work and then rely on automatic agent-shutdown. shrug
11:46	triddell	rhickey: I'm sure you have other things on your mind but I wondered if Issue #100 on the tracker had any plans to get attention. It's a bug that basically stops compiled classes from working on IBM JVMs.
11:47	djpowell	Is there any way to dispatch on array types in multi-methods - or to use hierarchies to derive from array types
11:47	cemerick	Chouser: yeah, I'm not at all clear on what the solution is, but I can't imagine how frustrating it would be as a complete noob to hit this issue. That's why I'm agitating :-)
11:47	rhickey	triddell: I'd welcome a patch
11:48	rhickey	cemerick: do you have a proposal?
11:49	triddell	rhickey: A patch was attached to the post that describes the original issue (it wasn't my post/patch so I didn't attach it to the issue)
11:49	clojurebot	rhickey is a T-1000 sent from the future
11:49	cemerick	rhickey: well, an immediate fix to keep compilation runs from hanging for 60s (add Agent.shutdown() in the finally block of main). I don't think anyone could complain about that.
11:49	triddell	rhickey: http://groups.google.com/group/clojure/browse_thread/thread/e64719d716c29ce0
11:50	triddell	rhickey: the patch works for me but I didn't know if it might have any other ramifications
11:50	cemerick	rhickey: On the solution in general, I'm generally aiming for something that will result in sane behaviour for noobs, and something that experienced users can easily work with (my current idea being, shutdown on exit from clojure.main in every case, and if you're clever enough to know that that's not what you want, make your main thread join until you're ready to be done).
11:51	cemerick	Chouser says that's gone back and forth a couple of times, but the current state of play doesn't seem reasonable for the common case, or for new users.
11:51	cemerick	(i.e. difficult-to-explain "hanging")
11:52	rhickey	triddell: someone with a CA needs to attach it to the issue
11:53	djpowell	Is there any way of writing an array class literal? Something like Ljava.lang.Object;?
11:54	rhickey	cemerick: I think there needs to be a recipe for both scenarios, by join you mean what?
11:55	rhickey	djpowell: you need the class object or just a type hint?
11:56	djpowell	well, I wanted to do multi-method dispatch, with 'class' as my dispatch function, and have it handle array types
11:56	djpowell	or - alternatively, to use derive to handle array types the same as lists - with a seq wrapper or something
11:57	cemerick	rhickey: sorry, that was a poor term choice. I just mean that if you want to delay your program's exit until your agents are done doing their thing, you should be responsible for holding the main thread until you're ready to exit. Lots of ways to do that.
11:57	triddell	rhickey: ok, I'll send a CA off and ping you about the issue after you've received it
11:58	cemerick	djpowell: we're just using (defmethod foo (class (make-array Character/TYPE 0)), etc
11:59	rhickey	,(Class/forName "[Ljava.lang.Object;")
11:59	clojurebot	[Ljava.lang.Object;
12:00	djpowell	ah - I had some strange idea that the dispatch values had to be literals for some reason
12:01	rhickey	cemerick: we need to automate that, lest we make a change and the other side complains again, round and orund
12:01	djpowell	that will work ok - or I could just change the dispatch function
12:01	replaca	rhickey: on second thought, I may not sell the meetup too hard - we've already got about 35 people signed up and the space will be crammed with 50. Feel free to mention it in your talk or to interested folks and just give them my email/number for info.
12:01	rhickey	djpowell: dispatch value do not need to be literals
12:01	rhickey	replaca: fair enough - I'm looking forward to it
12:02	replaca	rhickey: us too!
12:02	djpowell	rhickey: yeah cool. although on the subject of arrays, I did have some trouble type hinting that something was a char-array
12:02	cemerick	rhickey: I can see people wanting to do it in a lot of different ways. I hesitate to think I'd have enough perspective to propose a general-enough solution.
12:03	cemerick	rhickey: A sneaky option would be to trim back the keepalive of the threads in Agent's Executors' threadpools. It's 60s now, and could be a lot lower.
12:05	rhickey	hit 2000 members on group today!
12:08	cemerick	really? It says 1983 right now...
12:08	rhickey	cemerick: I don't know how often they update that, the admin screens say 2000
12:08	cemerick	ah
12:09	rhickey	cemerick: what about letting people set auto-shutdown-agents, default true? puts it in their hands without establishing a policy
12:12	stuhood	rhickey: would it be possible to model them like threads, where each agent could be a 'daemon'?
12:12	cemerick	rhickey: yeah, that's fine by me. :-) I think c.l.Compile should just have it hardcoded, though -- there's no user code in involved in that path that could customize the behaviour, anyway.
12:20	rhickey	biab
12:30	replaca	dysinger: are you in the channel?
12:35	dysinger	yes
12:38	replaca	dysinger: are you coming to SF for the clojure meetup? I saw you said something like that on Twitter.
12:38	replaca	dysinger: If so, I want to make sure you're on the rsvp list
12:38	dysinger	y
12:38	dysinger	I wasn't able to get logged in and rsvp yet
12:39	dysinger	yes thank you - put myself and phil hagelberg on there
12:39	dysinger	Tim Dysinger
12:39	replaca	dysinger: Don't worry, consider it done. I already had Phil on my list.
12:40	replaca	Anyone else coming can just ping me here or email me and I'll put you on the list
12:40	dysinger	thanks replaca
12:43	replaca	dysinger: np
12:46	Roux	anyone here got the 'programming clojure' book?
12:46	dnolen_	yes
12:47	triddell	triddell: yes
12:47	Roux	does it do a good job of explaining functional programming to people not familiar with that paradigm?
12:47	triddell	Roux: oops, yes
12:47	danlarkin_	replaca: wish I was on the west coast :) consider me there in spirit!
12:47	Roux	triddell: does it do a good job of explaining functional programming to people not familiar with that paradigm?
12:48	triddell	Roux: I think it does a very good job... I was new to it before clojure and it was the book I used to "get" it
12:48	Roux	great
12:48	Roux	:)
12:49	Roux	but with only 200 pages i would think it only has time to explain clojure library itself and no time to help ppl grok functional programming
12:50	triddell	in fact, I'm writing a review on Amazon right now and I'd encourage other readers to do the same and help to promote the book
12:51	replaca	danlarkin_: we all wish you were here too. I envision a future where we'll be able to have our own national events like the Ruby guys do.
12:52	Roux	replaca: you a python or a ruby guy?
12:53	replaca	Roux: both, to a certain extent
12:53	Roux	which softens your heart most of all
12:54	Roux	?
12:54	triddell	well, it's two hundred pages of functional programming! because clojure is a Lisp it can cover what would take another language 200 pages in chapter or two... the rest is practical examples and api specifics (the majority of which highlight its functional nature)
12:54	dnolen_	Roux: also compared to Python and Ruby, Clojure is fairly miniscule. Daunting at first but then you realize how small Clojure actually is.
12:54	triddell	Roux: arg, well, it's two hundred pages of functional programming! because clojure is a Lisp it can cover what would take another language 200 pages in chapter or two... the rest is practical examples and api specifics (the majority of which highlight its functional nature
12:54	Roux	triddell: yeah bro! i bought the scala book, and it's about 700 pages!!!
12:55	replaca	Roux: well, Clojure's my favorite right now cause I feel like I can really rock'n'roll in it
12:55	dnolen_	compared to Scala, Clojure is nanoscopic.
12:55	Roux	Scala strikes me as very complex
12:55	Roux	scarily so
12:55	Roux	but quite clever and nifty too :)
12:56	replaca	Roux: python and ruby are a little different, but overall I prefer Ruby. I end up using python more because at work we're based on the CLR and IronPython gives me a nice interface to that
12:56	triddell	Roux: bought that too, was interested and then found clojure, never read the Scala book after it arrived
12:56	replaca	Roux: I've just begun learning scala
12:57	replaca	I'm interested in some of the more advanced communication stuff that's being implemented in scala (esp. by the twitter folks) and would like to bring some of that tech into the Clojure fold
12:57	replaca	since I'm mostly a distributed systems guy
12:58	Roux	cool
12:58	Roux	there is alot to like in scala
12:58	Roux	esp. if you come from a c++/java background
12:59	replaca	Roux: yeah, I'm not of the "one language to rule them all" school. Clojure feels best to me, but I don't want to be chauvinistic about it.
12:59	Roux	scala feels quite rubyish too
12:59	Roux	i mean you iterate through a list like this in scala: myList.foreach(x => println(x))
13:00	dnolen_	scala doesn't work via messaging tho, so that seems like an odd comparison.
13:00	Roux	it's a pity they didn't credit ruby as an influence on the wikipedia page, cos as a rubyist myself i see alot of ruby in scala and i dont think it's a coincidence
13:01	triddell	Roux: the other day I wrote a simple Rest/Json service for use on Google App Engine in under a hundred lines using Clojure... it was the most fun I'd had programming in quite a while... I'm not a language snob either but fun is fun!
13:01	Roux	yeah i cant wait to get my head around a lisp, the most i've done so far is just muck around with e-lisp a teeny tiny bit and it was alot of fun thinking and programming recursively
13:03	dnolen_	well it seems to me that Ruby -> Clojure is natural, since Ruby was Matz's attempt to merge his two favorite languages, Lisp Smalltalk with the UNIX shell convenience of perl.
13:04	Roux	yeah that also probably explains the similarity of ruby to scala too, in so far as ruby is an amalgum of the oop/functional paradigmns in a similar way that scala purports to be (though it's true that scala values immutability higher than ruby)
13:06	dnolen_	well there's definitely a connection Gilad Bracha (of StrongTalk fame, and developer of NewSpeak) said he's had many conversations with Odersky about Scala.
13:08	Roux	odersky probably just doesn't give ruby as an influence so he doesn't scare off the python dudes, who love to spit on the ground of anything connected to ruby
13:11	hiredman	clojurebot: have you heard the word about scala?
13:11	clojurebot	"you can usually accomplish a lot by laboriously enumerating type arguments everywhere." -- seen in #scala
13:11	Roux	clojurebot: have you heard the word about ruby?
13:11	clojurebot	Excuse me?
13:12	Roux	:D
13:13	Chousuke	clojurebot: ruby is <reply>Chunky bacon!
13:13	clojurebot	Roger.
13:14	Roux	nice one :)
13:15	Chousuke	I hope someone will trigger clojurebot's random auto reply with "ruby" and be thoroughly confused as a result
13:16	hiredman	clojurebot: outback steak house?
13:16	clojurebot	I don't understand.
13:16	hiredman	bah
13:16	hiredman	clojurebot: steak?
13:16	clojurebot	I want to go to there
13:16	Roux	hehe although i dont know a single rubyist who hasn't read _why's guide ;)
13:25	Roux	triddell: finished writing your review yet?
13:25	Roux	im curious to see it
13:53	cemerick	replaca: coming in late, but I've begun to ponder what a couchdb clone would look like implemented with clojure. That'd be a big (but interesting) task.
13:55	danlarkin_	cemerick: beware NIHS
13:56	hiredman	a clojureql type deal for couchdb might be nice
13:56	cemerick	danlarkin_: oh, I quiver in terror in the face of NIHS :-)
13:56	ataggart	~google NIHS
13:56	cemerick	couch-in-clojure is fancy more than anything else
13:56	clojurebot	First, out of 166000 results is:
13:56	clojurebot	NIHS
13:56	clojurebot	http://www.nihs.go.jp/
13:57	cemerick	Our Pacific friends will be happy to know the terror they instill in clojure programmers. :-)
13:57	hiredman	:P
13:57	hiredman	hmmm
13:57	ataggart	lol
13:59	dysinger	cemerick - why re-write couchdb in clojure ?
13:59	dysinger	it's awesome as is
14:00	cemerick	dysinger: I have no good reason right now, which (among other reasons) is why I'm not.
14:00	dysinger	I think it would be more interesting to write a couchdb view server for clojure
14:00	dysinger	so you can write your map/reduce in clojure :)
14:00	cemerick	dysinger: that's something we'll probably be tackling in the next month or two, unless someone else gets there first :-)
14:00	stuartsierra	I already do that with Hadoop.
14:00	dysinger	yucky
14:01	cemerick	we couldn't handle the admin complexity of hadoop, unfortunately.
14:01	dysinger	we are looking at that too - thanks for your example code.
14:01	stuartsierra	welcome
14:01	dysinger	I just don't like how rube goldberg hadoop is at the moment.
14:02	dysinger	I am leaning towards straight clojure and amqp
14:02	stuartsierra	It's not so bad once you get into it - most of the complexity is for edge-case optimizations.
14:02	dysinger	probably
14:02	hiredman	:\|
14:03	dysinger	other thing I want to look at is terracotta and clojure for map/reduce in native lang.
14:03	replaca	cemerick: well, even if we don't need to recreate the wheel, it is interesting to think about how you'd tackle a problem like couchdb in Clojure and whether a Clojure implementation would be a good thing
14:03	dysinger	using agents ore something
14:03	stuartsierra	Does terracotta provide data locality?
14:03	dysinger	no
14:03	dysinger	all our data is on s3 anyway
14:03	dysinger	so import to hdfs or disk - doesn't seem to make much diff
14:04	stuartsierra	Most of Hadoop's performance comes from data locality.
14:04	dysinger	y
14:04	dysinger	true
14:04	dysinger	in order to keep hadoop happy hadoop must also be your storage engine
14:05	dysinger	tis the nature of the map/reduce paper....
14:05	stuartsierra	Well, not necessarily. You do have to copy things in and out of HDFS, but you don't have to store it there permanently. I store everything on S3.
14:05	dysinger	er "as in the paper"
14:05	replaca	more generally, it seems right that a good set of clojure primitives would allow clojure to enter the problem space currently dominated by erlang in a nicer package
14:05	dysinger	doesn't swap in and out of s3 take a long time with large data sets ?
14:05	dysinger	that's been my experience
14:06	stuartsierra	Not really, b/c you can parallelize the copy.
14:06	dysinger	across the nodes
14:06	hiredman	replaca: the main problem is all the cool message buses use erlang
14:06	replaca	hiredman: and maybe we need a cooler message bus of our own :-)
14:06	dysinger	still yeah - copying 4TB to each node is going to take a long time.
14:07	hiredman	replaca: maybe just use terracotta?
14:07	stuartsierra	Well, yes. I get around 1GB/min from S3 on an 18-node cluster.
14:07	dysinger	teracotta would be much more fun if it was dynamic language - capable
14:07	replaca	hiredman: could be. I don't understand terracotta well enough to know if it's the answer
14:08	dysinger	you have to AOT compile, instrument things and then describe each generated closure class in their XML files
14:08	dysinger	and last I checked it required a hack to clojure
14:08	hiredman	dysinger: oh, I did not realize
14:08	dysinger	but still it seems to work
14:09	hiredman	the repl works
14:09	hiredman	I am not sure how that would work if "dynamic" stuff doesn't
14:09	dysinger	paul stadig did these tests http://paul.stadig.name/2009/02/clojure-terracotta-yeah-baby.html a while back and he has a project on github
14:09	hiredman	yeah
14:09	hiredman	I've seen them
14:09	dysinger	that's all I've been playing with.
14:09	dysinger	would be cool to have it working
14:10	dysinger	that would eliminate the need for hadoop and rabbitmq for me.
14:10	dysinger	In theory
14:10	dysinger	and move the problem into straight clojure code
14:10	clojurebot	"There is no problem in computer programming which cannot be solved by an added level of indirection." -- Dr Maurice Wilkes
14:10	dysinger	heh
14:10	hiredman	~botsnack
14:10	clojurebot	thanks; that was delicious. (nom nom nom)
14:12	cemerick	dysinger: what would you use for primary storage, though?
14:13	hiredman	openmq is java
14:14	cemerick	openmq is jms, jms is painful :-(
14:15	dysinger	we use s3 and xfer to local disk
14:15	cemerick	maybe rhickey's forthcoming queue abstraction will help there
14:15	hiredman	cemerick: :\|
14:15	dysinger	nothing beats erlang for messaging
14:16	dysinger	I don't have any stats but if I had money to bet - I'd bet rabbitmq could take a better beating than any jms server
14:16	hiredman	dysinger: but having everything on the jvm would be nice
14:16	dysinger	I used to think that but it's not such a comfortable thing for me anymore.
14:16	dysinger	anyway - it's not a requirement for me anyomer.
14:16	dysinger	anymore
14:17	dysinger	Our plan is to use rabbitmq & json to make the messaging language neutral
14:18	hiredman	http://clojure-log.n01se.net/date/2008-09-08.html#14:26
14:18	dysinger	we are connecting to rabbitmq with ruby, python, java and soon erlang
14:20	dysinger	and if we can tie in the messaging primitives in clojure to it -> that would be nice.
14:20	dysinger	(the currently being thought about but not yet existing primitives)
14:20	hiredman	I just going to say...
14:23	dysinger	hey technomancy - ready for some SF meetup ?
14:25	danlarkin_	cemerick & tashafa: my response on twitter didn't come off as light hearted as it did in my head before I typed it :)
14:25	cemerick	danlarkin_: heh, you worry too much :-)
14:25	technomancy	dysinger: woo!
14:26	cemerick	tashafa: hey, another Bostonite :-)
14:26	cemerick	(not that I'm in Boston, but it's close, anyway)
14:40	dysinger	technomancy took a bunch of twiddling but I got emacs 23 working with mac environment.plist
14:40	dysinger	now am happy eshell user
14:41	technomancy	ugh; I'm sorry you had to do that
14:41	dysinger	It's fine
14:41	dysinger	we should add it to the starter kit
14:41	technomancy	manually editing plist XML files is not a task fit for a human
14:41	Chousuke	there's the plist editor :p
14:42	dysinger	I have a rake task to push to the plist file
14:42	technomancy	yeah, if you can automate it, I'm sure it would save poor future users a lot of pain.
14:42	dysinger	anytime my env vars change
14:42	dysinger	I have never even used the environment.plist file before -> and I have been OS X since 2002
14:42	technomancy	would prefer a standalone solution, but that's definitely better than the current situation.
14:43	dysinger	I always just used ~/.profile
14:43	dysinger	but emacs carbon ignores that
14:43	dysinger	retarded
14:43	dysinger	!
14:44	technomancy	dysinger: it's because it's launched from the finder/dock
14:44	dysinger	y
14:44	technomancy	I submitted a bug report about that a few years ago
14:44	technomancy	not exactly holding my breath
14:44	dysinger	eshell should just read .profile like other terminal apps.
14:44	triddell	Roux: my review is up on Amazon... it's mostly a "feel good" piece than a technical review though
14:44	dysinger	s/read/source
14:45	technomancy	it should be easier to source .profile than it is now (as a workaround for the broken OS X behaviour), but the correct behaviour is to inherit
14:46	ataggart	ah nice! the meetup is 2 blocks from my office
14:47	dysinger	I just eneded up including osx-plist.el and adding a (if.... (eq system-type 'darwin)
14:47	dysinger	block
14:47	dysinger	it works
14:48	technomancy	dysinger: great; would love a patch
15:25	cemerick	hrm, what's the incantation to get a type hint to emit properly from a macro? I've tried `(.foo #^Bar someBar), `(.foo #^{:type Bar} someBar) so far...
15:26	Chouser	try `(.foo ~(with-meta `someBar {:type Bar}) ...)
15:26	hiredman	interesting
15:26	cemerick	Actually, #^Bar works according to (meta (second (macroexpand ...))), but I still get reflection warnings.
15:27	cemerick	I meant :tag above, sorry
15:27	rhickey	#^Bar should be fine unless Bar is a variable or something
15:27	Chouser	right, me too. :-)
15:28	rhickey	does a non-macro version of the #^ work?
15:31	cemerick	rhickey: I was actually just trying to type-hint defmacro so as to limit the noise when printing reflection warnings :-)
15:32	rhickey	?
15:32	cemerick	sorry, defmethod
15:33	Roux	rhickey: you look a bit like wittgenstein
15:34	rhickey	cemerick: ah
15:37	rhickey	presuming multifn is a symbol is likely but not required...
15:38	leon	Hello
15:38	rhickey	you'll need to add metadata to the form, as Chouser showed, since you are unquoting it, but given it could be an arbitrary expression yielding a
15:39	rhickey	MultiFn, you might want to wrap it in a call to identiy and add metadata to that call
15:39	rhickey	identity
15:39	leon	I have a short question: is there a 'length' function in clojure for lists or any seqs? I cannot find any in the api/wiki
15:39	Chouser_	leon: count
15:39	leon	thanks Chouser
15:39	rhickey	Chouser - fastest gun in the west
15:39	ataggart	,(count [1 2 3 4])
15:39	clojurebot	4
15:40	leon	could go into the lisp/clojure differences :-)
15:40	hiredman	,(counted? [1 2 3 4])
15:40	clojurebot	true
15:40	ataggart	,(doc counted?)
15:40	clojurebot	"([coll]); Returns true if coll implements count in constant time"
15:40	ataggart	nice
15:40	Chouser	it helps when I have #clojure up in mutliple windows on the same desktop. :-P
15:40	cemerick	rhickey: why the call to identity?
15:41	Chouser	aslo wehn I quit caring abuot my tpying
15:45	rhickey	leon: what's the length of a hash table in CL?
15:46	leon	no idea
15:46	rhickey	CL/CL differences
15:47	rhickey	hash tables don't have length in CL, they have... counts
15:48	rhickey	only count works across collection types IMO
15:49	rhickey	cemerick: I guess I'm afraid that somehow the expr used for multifn won't support metadata, just paranoia, but defmethod doesn't dictate it be a symbol - maybe it should
15:52	Chouser	I had hoped I wouldn't have to think about races in Clojure. But you throw enough agents, blocking sockets, and atoms in there and poof complexity.
15:52	Roux	rhickey: http://agbellotti.files.wordpress.com/2006/11/wittgenstein.jpg
15:53	rhickey	Chouser: is it inherent in blocking sockets, or is there something missing?
15:54	rhickey	Chouser: did you look at the executors framework?
15:54	Chouser	If I had a coherent suggestion, I would have described it by now.
15:55	Chouser	agents are my executor framework api. :-)
15:55	danlarkin_	oh wow! He does really look like wittgenstein
15:55	Chouser	I think the kinds of complexity I'm seeing can be had as soon as you mix agents and atoms.
15:56	Chouser	or I guess agents examining/responding to any external state
15:56	Roux	danlarkin_: hehehe :D
15:56	triddell	triddell: I hope Rich brought his tweed jacket for JavaOne ;-)
15:56	rhickey	Chouser: yeah, but maybe they aren't a good fit, or need to expose some more of the support, e.g. interruption
15:57	Roux	danlarkin_: quite similar features but he lacks the 1000-yard stare of our friend ludwig: http://peernet.lbpc.ca/Philosophy/images/Wittgenstein.jpeg
15:59	Chouser	Roux: also Rich isn't quite as grainy
15:59	Roux	;)
16:02	Roux	clojure is in good hands: Described by Bertrand Russell as "the most perfect example I have ever known of genius as traditionally conceived, passionate, profound, intense, and dominating", Wittgenstein is considered by many to be the greatest philosopher of the 20th century.
16:04	stuhood	rhickey,Chouser: at the risk of trying to solve a problem that isn't fully defined, perhaps agents should be able to be shutdown individually? such that they will finish any (sends) in the queue, but throw Rejected for new (sends)
16:05	Chouser	stuhood: I very nearly said something like that earlier today. I was thinking more that once all agents were in the finished state, the pools could shut down automatically.
16:05	Chouser	but I don't know what "all agents" means
16:09	stuhood	yea... it seems like agents really just need to be a thread+queue abstraction: so each agent could be marked as a daemon or not, so that shutting down the pool would wait for all non-daemon agents
16:12	hiredman	(actor)
16:14	stuhood	they execute Actions, so why not? =P
16:15	Chouser	I think rhickey nailed (as I'm suprised) ... replacing several of my agents with single-thread exectors may remove a lot of my complexity.
16:15	hiredman	java.util.concurrent is pretty great
16:18	hiredman	I feel sort of cheated never having been introduced to it before
16:29	cemerick	aaah, :tag's value needs to be a symbol, not a class! :-(
16:29	Chouser	wow
16:30	Chouser	sorry, I didn't even consider that.
16:30	cemerick	yup, I'm that slow :-/
16:30	cemerick	that should really be changed to be more accepting, no?
16:35	cemerick	ah, no, nevermind, that's just the way it has to be.
16:47	Chouser	gah. why does every different lib have to have it's own callback class (or two or three)
16:47	technomancy	what's up with libraries always requiring the JAVA_HOME env variable to be set?
16:49	Chouser	oh, because they each have different signatures. :-P
16:49	technomancy	anyone want to recommend an alternative to mvnrepository.com that isn't down most of the time?
16:57	cemerick	ugh, beating gensyms into getting type-hinted isn't proving much easier than the last challenge
17:05	cemerick	ah, blast from the past from cgrand saves the day :-) http://groups.google.com/group/clojure/browse_thread/thread/776f65f8a2f134f7
17:05	cemerick	speak of the devil :-)
19:45	gcv	when it comes to handling file paths, is there an idiomatic Clojure alternative to using java.io.File? I just typed (.. (java.io.File. "/my/path/here") getParentFile getParentFile) in a source file and it looks verbose.
19:46	technomancy	gcv: c.c.java-utils/file works for me
19:46	technomancy	though... only a few chars less verbose
19:46	technomancy	you could do (file "/my/path/here/" ".." "..")
19:49	gcv	technomancy: thanks, that's pretty good. although it doesn't quite return the same thing: #<File /my> for my original example, and #<File /my/path/here/../..> using the file function.
19:49	dysinger	clojure never promised magic/non-verbose java interop :P
19:49	dysinger	it's still java
19:50	technomancy	gcv: yeah, that's lousy. it'd be cool if "file" would automatically get the canonical version instead of just allowing crazy crap like that
19:51	dysinger	http://java.sun.com/j2se/1.4.2/docs/api/java/io/File.html#getCanonicalFile()
19:51	gcv	yeah, I was just wondering why file doesn't call that before returning
19:52	hiredman	well, it ain't a final class...
19:53	dysinger	(str .getCanonicalPath (file "/my/path/here/" ".." ".."))
19:53	hiredman	uh
19:53	hiredman	no?
19:54	gcv	hiredman: getCanonicalPath already returns a string, I don't think you need the str call.
19:54	hiredman	and str is not comp, and you cannot compose method calls
19:54	dysinger	trie
19:54	dysinger	true
19:54	dysinger	true
19:54	dysinger	also
19:54	dysinger	I was just goofing around
19:54	hiredman	I know
19:55	dysinger	I am rusty - it's been a week since clojure coding :P
19:55	dysinger	need back on horse
19:55	dysinger	(.getCanonicalPath (file "/my/path/here/" ".." ".."))
19:55	hiredman	dysinger: I saw some video where some guy from australia was presenting about openmq and he said in his testing it beat activemq
19:55	dysinger	cool
19:55	hiredman	of course this was just some random video on the internet
19:56	dysinger	activemq isn't hard to beat - I think it uses derby by default for presistent message queueu
19:56	dysinger	queue
19:56	hiredman	huh
19:56	hiredman	well that makes sense
19:56	dysinger	java db
19:56	dysinger	but you can configure it faster
19:56	hiredman	for some reason I was thinking activemq was erlang
19:56	dysinger	no that's rabbitmq
20:01	achin	I'm trying to port someone's pre-1.0 Clojure code to 1.0. I'm having trouble figuring out why you would test for something being an instance of clojure.lang.LazyCons. Would I want to change this to test for LazySeq? just use seq?
20:03	Chouser	I've never felt the need to test for LazyCons
20:03	Chouser	what do that do with it if it's a LazyCons?
20:03	Chouser	what if it's not?
20:03	achin	I'm trying to figure that out. I don't know enough about the library I'm porting.
20:04	hiredman	Chouser: if you want, for example, lazy code to be forced on a sandboxed thread
20:04	achin	This is clj-html. It's the bit of code that dispatches bits the DSL to expand it.
20:04	hiredman	achin: (instance? clojure.lang.LazySeq s) or something like that
20:05	hiredman	achin: erm, I think clj-html has been updated
20:05	hiredman	just pull from github
20:05	achin	hiredman: that was my guess, but I don't know enough about Clojure to know if LazySeq is a drop-in replacement for LazyCons.
20:05	achin	hiredman: I just pulled.
20:05	hiredman	h,,,
20:06	hiredman	hmm
20:06	achin	hiredman: It's still using lazy-cons.
20:06	achin	hiredman: And stuff like rrest that's gone now.
20:06	hiredman	http://github.com/mmcgrana/clj-html/tree/master from this?
20:06	Chouser	rrest --> nnext
20:06	achin	chouser: right.
20:06	achin	hiredman: I pulled from git://github.com/mmcgrana/clj-html.git
20:06	hiredman	weird
20:07	achin	Last updated 19 Jan.
20:07	hiredman	oh
20:07	hiredman	heh
20:07	hiredman	I fixed it locally
20:07	achin	Heh.
20:07	hiredman	no wonder it works for me(TM(TM)
20:08	achin	Would you mind sharing? I'm just getting into Clojure and I was using this as an exercise to get oriented.
20:08	hiredman	http://github.com/hiredman/clj-html/tree/master
20:09	hiredman	wow
20:09	achin	Thanks.
20:09	hiredman	this was just sort of a quick fix, so some of it is not the best
20:10	achin	It's not like mine's going to be any prettier.
20:18	achin	Hrm. If I write, (.toUpperCase "text"), everything's okay. If I wrap it in a macro, I get an unrecognized form exception. http://pastebin.com/mcbf21fd
20:22	achin	Heh. That never even worked to begin with.
23:31	Chouser	Using the Executors API helped, I think, to solve the complexity I had, making it easier to see the (different) complexity I need. :-/
23:39	rhickey	heh
23:46	Chouser	If I need to serialize small pieces of work from n different threads, is there a performance consequence for using a single-threaded executor vs. an atom vs. a plain lock?
23:47	Chouser	I suppose I should read that concurrency book I've seen waved around...
23:48	Chouser	well, that's not a problem for tonight.
23:49	Chouser	rhickey: hope you have a great day tomorrow. Knock 'em dead, and all that. :-)
23:49	rhickey	thanks!
23:57	twopoint718	I'm curious about the role of the "this" in the arguments to "impl-foo" toward the bottom of defn impl-foo [this]
23:57	twopoint718	(str (class this)))
23:57	twopoint718
23:57	twopoint718	(defn impl-bar [this]
23:57	twopoint718	Sorry about that, this was the page I meant to reference: http://clojure.org/compilation
23:58	twopoint718	In particular, it looks like these clojure-generated methods are getting the class as their first argument. Is that correct?