#clojure – 2008-11-12

2008-11-12

01:58	sohail	uhhh... what the f: http://commons.apache.org/jelly/
02:00	tWip	that's the logical conclusion to xml-mania... making it turing complete
02:01	Nafai	sohail: Yes, jelly is very painful
02:01	Nafai	I've actually written a bit in it in the past
02:03	sohail	incroyable
02:04	Nafai	I've also done a bit in a workflow language called XPDL
02:05	Nafai	It took 7 printed pages of XML to do a simple method implementation
02:05	sohail	wow
02:05	sohail	I bet that stuff pays well though
07:33	Lau_of_DK	Good afternoon gentlemen
08:07	gnuvince	morning
09:58	Cark	is there any equivalent to CL's labels in clojure ?
09:58	Cark	so that i can recursively call a let function
09:59	rhickey	Cark: self-recursion is possible with (fn my-name [] ... (my-name ...))
09:59	rhickey	mutual local recursion a la labels/letrec is not currently supported
10:00	Cark	ah too bad
10:00	Cark	thanks though
10:00	rhickey	on the todo list
10:02	gnuvince	rhickey: would that mean that the order in which functions are defined would become unimportant?
10:03	rhickey	gnuvince: this is just about local/let-bound fns, but yes
10:04	gnuvince	ah ok.
10:37	rhickey	Chouser: http://clojure-log.n01se.net/ is down?
10:40	abrooks	rhickey: It appears to be. I'll check it out.
10:42	Chouser	abrooks: thanks
10:46	abrooks	Chouser: Shoot. I need to go interview a candidate now. I'll continue after lunch.
10:50	Chouser	rhickey: back up
10:50	rhickey	Chouser: thanks!
10:51	abrooks	Chouser: Was it Ruby?
10:52	Chouser	abrooks: maybe. I turned off mod_rails for clojurescript.n01se.net, and now it seems to work. Could be coincidence.
10:52	Chouser	I'll have to try fast-cgi or something instead.
11:05	Chouser	Seems like binding/set! might work for early termination of an enumerator
11:05	rhickey	Chouser: sounds scary
11:12	Chouser	also Java uses enum to mean something completely different
11:18	rhickey	cgrand: thanks for the LFE post!
11:22	cgrand	rhickey: you're welcome
11:27	cgrand	chouser: I also thougt about using binding/set! and dismissed it immediatly
11:27	cgrand	too scary
11:30	rhickey	cgrand: you spent a lot of effort to support map, I wonder if it is as important as 'chunking', i.e. reducing at other than 1:1 ratio
11:31	rhickey	for me the inversion of control is much more about resource management than having someone else run the iteration
11:32	cgrand	yeah I also wondered about the practicality of suporting map (and came to no conclusion and since I was on my way...)
11:32	rhickey	map is definitely a challenge
11:35	rhickey	as far as supporting ordinary functions - the underlying protocol can be richer as adapters are easy
11:37	cgrand	What du you mean by "reducing at other than 1:1 ratio" : reducing N items at a time (N:1) or , for example, transforming an "enum" of char[]into an enum of lines (N:M)?
11:38	rhickey	the 'slides' passed a stream to the reducing fn, not individual items
11:38	rhickey	opens the door to consuming more than one item per step
11:39	Chouser	My (very weak!) attempt at doing this passed a seq to the step function, which then could return however much of the seq it chose to not consume.
11:39	rhickey	chars to lines would be an example
11:41	rhickey	Chouser: yes, that's closer to the slides
11:42	rhickey	there's a risk of leaking the seq, but that could be in the "don't do that" category AFAIC
11:42	rhickey	the enum would close the resource and subsequent use of the seq would fail
11:43	Chouser	oh, if you passed the chunk-seq to outside the enum somehow?
11:43	rhickey	sure, it could be your return even
11:44	rhickey	or passed to something in your worker code
11:44	rhickey	the latter would still be in scope of enumeration
11:44	Chouser	ok, yep, I see that.
11:46	leafw	rhickey, try: wn enumeration -synsn
11:46	rhickey	These internal iterations have the potential to be very fast, much faster than seqs (i.e. allocation-free), that's why I had the io/stream primitive notion under them, in my model that's what I passed. Would then go back and rebuild all the seqs on io/streams
11:46	leafw	(wordnet)
11:48	rhickey	reducer?
11:49	rhickey	supporting map would blow my model :(
11:49	leafw	distiller?
11:49	rhickey	very tricky
11:49	Chouser	only multi-seq map, right?
11:49	rhickey	right
11:50	Chouser	similar problem in ruby iterators, which I was able to solve with continuations. :-/
11:52	cgrand	rhickey: we had different goals: mine was to come up with a general collection "framework", yours is better IO. Fun vs practical :-)
11:52	rhickey	cgrand: I want both
11:56	rhickey	this would be a very important addition to Clojure
11:58	cgrand	Working on MFE made me thought wonder about replacing seqs by reducers as the basic building block...
11:58	cgrand	LFE
11:59	rhickey	early termination is not laziness
11:59	rhickey	I do think the seqs can be built on ios
11:59	rhickey	should be
12:00	rhickey	some ios are only available in resource managed contexts - reducers
12:00	rhickey	those are the building blocks in my mind
12:01	rhickey	I had played with seqs on iterators early on, but iterators are MT-broken
12:01	rhickey	ios can be made atomic
12:01	rhickey	more primitive than iterators, just need EIO protocol
12:07	cgrand	I'm lost between ios and reducers. Are reducers specialized ios or two different things (ios being generators)?
12:08	rhickey	ios are simple streams/generators, reducers can be thought of as things that make ios available in a resource-managed context
12:09	cgrand	so ios aren't directly exposed in user code?
12:43	cddr	general Java question coming up...
12:43	cddr	To make a library threadsafe, must one wrap all uses of iterators inside a synchronized block?
12:45	roblally	I think it is worse than that. Some Collections can't be modified whilst you iterate over them. So you have to protect the underlying collection as well as the iterator. Otherwise you get ( IIRC ) ConcurrentModificationExceptions thrown.
12:45	cddr	yeah those are what I'm trying to fix
12:46	cddr	although the underlying collections do seem to be Vectors rather than lists so maybe not so bad
12:47	roblally	Vector is one of the old collection classes that are synchronised by default. But that doesn't protect the iterators.
12:47	roblally	If it isn't too expensive, I copy the collection before iteration 'just to be sure'
12:48	roblally	Of course that may not give you the right answer in every case.
12:50	cddr	I tried the copy (clone) approach before but it didn't seem to work (i.e. still got the same exception)
12:52	roblally	I guess that you need to lock during the copy process ( since copying to another class that takes a Collection parameter in it's constructor will probably fall back to an internal iterator.
12:53	cddr	apologies for non #clojure related spam but I have asked this stuff on c.l.j and got nothing
12:53	cddr	I'll shut-up now, thanks roblally
12:54	roblally	But locking during copy should be less painful than locking during the whole iteration process. Basically you need to lock on every operation that touches the collection. Without the copy you need to lock on the collection plus the iterator ( which is much nastier in Java )
12:54	roblally	Happy to try to help.
12:55	roblally	(As an aside, having a conversation that shows the pain of not using Clojure on a Clojure list will probably just make everyone here feel all warm inside)
12:56	cddr	lol
12:57	Cark	well sometimes the java non-sense leaks to clojure too ...
12:57	Cark	those tons of different and incompatible listener interfaces .... i hate these
12:58	Cark	don't know how i could abstract that to a simple function call
12:59	Cark	so in the end, i make a proxy for ListSelectionListener, then another one for ChangeListener etc ... they all have the same single event parameter.. grrr
13:11	Chousuke	Cark: maybe you could use a multimethod?
13:12	Chousuke	Cark: that way, there would still be multiple functions for creating proxies but at least they'd be an implementation detail
13:19	rhickey	cgrand: sorry, got called away - ios could be exposed where safe
13:20	rhickey	for instance an io could be passed to the fn inside a reducer
13:21	Cark	chousuke : yep that makes sense
13:23	Cark	chousuke : but i still need to declare the type of listener for each callback invocation
13:24	Cark	still shorter than a full proxy form tho
14:07	Chouser	Cark: could you write a macro to abstract away whatever's in common between those proxies?
14:14	Cark	chouser : yes i'm in the process of doing so
14:15	Cark	actually i'll do that tomorrow, time to kick back and watch a movie ... see you all tomorrow
14:51	drewr	I've got a lazy stream that is exhausting my heap space.
14:51	drewr	I have a (count) in front of it. Would that be keeping objects around so that they're not GC'ed?
14:51	rhickey	drewr: are you holding on to the head?
14:52	rhickey	yeah, count will run it out
14:52	drewr	OK, let me get rid of that.
14:52	drewr	I've got a ref that I update with the count so I don't even need that.
15:03	drewr	What's the right way to kick something off asynchronously that itself uses agents to do other asynchronous things?
15:04	drewr	Clojure doesn't seem to allow nested send-offs.
15:05	rhickey	drewr: it does support nested actions
15:06	rhickey	they don't fire until the sender's action completes
15:07	rhickey	so lexically nested, but temporally chained
15:11	drewr	Is there any way that I could mess that up lexically so that they don't get chained temporally?
15:12	rhickey	nope, but if you send then block, those sends aren't going anywhere
15:12	drewr	OK.
15:17	rhickey	there's always (.run (Thread. f))
15:18	rhickey	I guess I could add a release-pending-actions, waiting until action complete is usually a feature
15:22	drewr	Removing (count ...) kept my JVM from crashing, but now something is just blocking.
15:22	drewr	Ugh.
15:35	Chouser	oh, cool. You can look at queued agents: (.getQueue Agent/pooledExecutor)
15:35	Chouser	well, queued actions.
15:36	abrooks	Ah, nifty. I've wanted to do that.
15:36	Chouser	but that's only for send, not send-off
15:36	Chouser	I wonder why pooledExecutor is public but soloExecutor is not.
15:38	rhickey	Chouser: probably only public so I could debug something
15:38	Chouser	and the action has its agent, but that's also private.
15:38	Chouser	ah. seems like runtime viewing of that stuff could be generally useful.
15:39	Chouser	wouldn't want people calling .shutdown I suppose.
15:39	rhickey	I don't want people marrying implementation details
15:39	Chouser	I only want to flirt with them.
15:39	rhickey	I've swapped out the pools and queues a few times
15:41	Chouser	are you at all sympathetic to a higher-level view-only feature? a way to get a sense of how backed up the agent queue is, for example.
15:42	rhickey	Chouser: yes, definitely, both agents and refs need more introspective capabilities
15:46	kotarak	In how far, is it voodoo to access a private function from a macro via ns-resolve? Say foo.bar/macro uses foo.bar/private. Do I have to make private public? %)
15:47	Chouser	kotarak: excellent question. I've wondered that myself.
15:48	kotarak	One can use ns-resolve. But that feels hacky like hell...
15:48	Chouser	So far I've always just made such functions public, documented them, and assumed nobody would ever use the directly.
15:48	kotarak	Also my way up to now.
15:49	kotarak	(ns foo) (defn- foo [] :Private) (in-ns 'use) ((ns-resolve 'foo 'foo)) => :Private, that works, but well, ... not really...
15:59	rhickey	AOT shaping up, in 1094:
15:59	rhickey	AOT compiler support
15:59	rhickey	breaking change to load - no longer takes extension
15:59	rhickey	load will load from classfile if newer than source
15:59	rhickey	to compile, source dir and compile dir must be in classpath
15:59	rhickey	(compile 'my.cool.ns)
15:59	rhickey	will compile my/cool/ns.clj and anything it loads directly or indirectly
16:00	rhickey	require/use system should use classfiles if present and newer
16:00	rhickey	just move your files up a dir
16:00	Chouser	sweet
16:01	kotarak	thumbs up, that saves my lots of directories containing a single file...
16:01	rhickey	just need to sync clojure.contrib and shake out anything we find
16:01	duck1123_	rhickey: when it's all done, will you be posting anything describing what needs to be changed if you haven't updated since before the AOT work?
16:01	rhickey	one thing I was able to preserve was a ns defined by multiple files - clojure.core is
16:02	duck1123_	that's a very nice fix, much more Java-like
16:02	rhickey	duck1123_: if you're using contrib you should let those guys shake that out first
16:02	rhickey	other than moving your files up a dir, and dropping the extension on any manual load calls, AOT shouldn't change anything
16:03	rhickey	the binding form changes are bigger from a source-level perspective
16:03	duck1123_	do you /have/ to move them up a level?
16:03	rhickey	yup, namespaces now correspond to classes, not packages
16:04	Chouser	hm I just created a namespace dir last night and was happy to be able to put a README and support files in there.
16:04	Chouser	dumb, since I knew this was coming.
16:05	kotarak	Chouser: you still can, no? Then you have foo/bar.clj and foo/bar/README.txt foo/bar/support.clj...
16:05	rhickey	Chouser: those related resources are the only thing that doesn't work well here, was an argument against this arrangement initially
16:05	Chouser	yeah, I suppose. don't know if anyone will think to look in the subdir for a readme.
16:06	duck1123	this will mean less typing when C-x C-f ing from file to file
16:06	kotarak	Hmm.... How many projects, does a jar contain? Maybe /README.txt at the root is sufficient?
16:06	cemerick	rhickey: this will be huge. Many thanks.
16:07	rhickey	cemerick: genclass subsumption still to go
16:08	rhickey	one thing to note is that this compile is a 'same-world' compiler, a side effect of compiling is loading, this to make macros and the fns they use avaiable
16:10	rhickey	compile-files will be true when compiling and can be used like this: (def a (when-not compile-files (my-big-init)))
16:10	cemerick	oh, that'll be very nice, as well
16:10	rhickey	for stuff you only want at runtime, or just put that work in function bodies
16:11	rhickey	so no eval-when yet
16:13	rhickey	back later...
17:14	rhickey	everyone ported to 1094?
17:14	rhickey	:)
17:14	duck1123	was it all for nothing? :)
17:14	tomhickey	=)
17:15	kotarak	Hmm.. The hg mirror is still at 1091.
17:17	duck1123	github is up to date
17:22	AWizzArd	is 1094 the new 'stable' version?
17:22	rhickey	AWizzArd: nothing has been unstable, but recent version have had breaking changes
17:23	AWizzArd	seems that AOT made a lot of progress so far thumbs up
17:24	rhickey	looking for testers/feedback/bug reports
17:25	AWizzArd	sure, I'm in
17:25	kotarak	will try also
17:35	Chouser	user=> (find-ns 'clojure.xml)
17:35	Chouser	#<Namespace #<Namespace: clojure.xml>>
17:36	Chouser	clojure.contrib mmap and gen-interface are on 1094, seem to be working fine.
17:36	rhickey	Chouser: I saw that the other day, still chewing on default print currently #<(class x) (str x)>
17:37	Chouser	ah, ok.
17:37	rhickey	Namespace's toString does too much
17:38	rhickey	fixed
17:38	rhickey	user=> (find-ns 'clojure.core)
17:38	rhickey	#<Namespace clojure.core>
17:39	rhickey	Chouser: thanks for the report on mmap and gen-interface
17:42	Chouser	.class files aren't generated unless asked for?
17:42	rhickey	right, you have to (compile 'clojure.contrib.xxx)
17:43	Chouser	main() methods will come later?
17:43	rhickey	main goes with the genclass fold-in
17:44	rhickey	an easy option is to map main to main
17:45	Chouser	(compile ...) creates a classes dir in the CWD, not necessarily the one in your classpath.
17:47	rhickey	classes is just a default, you can set compile-path to whatever you want
17:47	rhickey	is bound in repl, you can set! or use binding
17:47	Chouser	wild. So I just compiled a namespace, then imported it and used Java reflection on it.
17:48	rhickey	cool!
17:48	rhickey	I want to get genclass folded in so it's just as easy
17:48	rhickey	also AOT proxy
17:49	rhickey	basically want to be able to deliver Clojure apps w/o any need for custom classloader
17:49	SimonAdameit	Hi, I have some problems with setting up clojure/slime. See http://paste.lisp.org/display/70249 for output. Any Idea whats going wrong?
17:50	Chouser	I have no idea what this is. a whole lot of const__xx that appear to vars and symbols. oh, return values from the top-level forms in my namespace .clj?
17:50	Chouser	rhickey: yeah, that's a truly worth goal.
17:53	Chouser	SimonAdameit: sorry, I don't use emacs, but many others here do. I'm sure someone will be along to help you.
17:54	Chouser	rhickey: hm, but if I (compile foo) without the right classes dir in my classpath, it looks like Clojure is unhappy about being able to find the class.
17:56	SimonAdameit	rhickey: My congrats to you, Clojure looks alot like the lisp in my daydreams :) .. I mean realy great.
17:57	rhickey	SimonAdameit: great - thanks!
17:57	rhickey	Chouser: right, you can put your classes wherever you want, and that dir has to be a)in your classpath, and b) set as compiler-path
17:59	rhickey	Chouser: the const__xxs are any vars or literals from compilation, they turn into static members
17:59	Chouser	zip-filter and zip-filter.xml required no changes other than file renames.
17:59	rhickey	Chouser: great!
18:00	rhickey	are you compiling everything as you move it?
18:00	Chouser	scgilardi is hard at it, too.
18:00	Chouser	no
18:00	Chouser	should I?
18:00	rhickey	Chouser: if you can, that way we know that everything works the same when compiled
18:01	Chouser	and then test the compiled version.
18:01	Chouser	yeah, ok
18:01	Chouser	gotta go. bbl.
18:03	rhickey	me too
18:04	SimonAdameit	Yay! - going back to revision 1088 solved the problem. as recommended in the group
18:46	rhickey	hmm... why didn't my VM complain about a class called core-print?
20:05	gnuvince_	Is there a prettier way to do something like: (loop [coll coll, n n] ...)?
20:05	gnuvince_	(instead of inventing a bunch of names
21:41	_Jordan_	SLIME won't connect, just polls indefinitely even after the Clojure REPL prompt shows up... anyone seen this before?
21:48	yangsx	_Jordan_: you're using an interim subversion checkout, it seems
21:49	_Jordan_	yangsx: bummer... do I just wait then?
22:28	drewr	_Jordan_: No, just back up a few revs.
22:31	_Jordan_	drewr: thanks!
22:59	yangsx	_Jordan_: Now that rhickey completed AOT, you can fetch the latest. I actually can run it now, but a few changes to .emacs is necessary.
23:00	_Jordan_	yangsx: Thanks--I ended up just backing up a few revs as suggested, which I think will be sufficient for me to start having fun learning :)
23:01	yangsx	for a quick trial, in your inferior-lisp buffer: (add-classpath "path-to-clojure-classes") (add-classpath "path-to-clojure/src/clj") and M-x slime-connect, it works for me
23:06	_Jordan_	This is neat! Now I just need to get it into my fingers to not spell it with an 's', and I'm golden :D
23:43	larrytheliquid	is there a let-like function that accepts a list of symbol/value pairs? ie: bindings-list => '(one 1); (let-like-function bindings-list one) => 1