#clojure – 2008-06-30

2008-06-30

08:19	StartsWithK	anyone knows did yrb maybe changed his nick or something? two week ago we had discussion about wrapper for jogl lib and he expressed interest in using it.
08:20	StartsWithK	repo is still at http://freehg.org/u/kresimir/neman/ not finished, but there are now 6 examples and i uploaded couple of other thigs.
08:20	StartsWithK	so.. if anyone is interested
10:20	cemerick	So, I'd like to use a namespace via some alias -- e.g. say there's a foobarbaz namespace that I want to use, but its name is too long, or it conflicts with some name I want to use, and I don't want to refer it (i.e. (refer 'foobarbaz)). What I'd like to do is something like (alias 'foobarbaz 'someprettyname), and then be able to use foobarbaz defs and such via someprettyname/*.
10:21	cemerick	I put together a macro that sorta-kinda works, but:
10:22	cemerick	(a) I'm not sure I'm touching everything I need to in order to "ball up" the referred-to namespace into one that has the more preferable name,
10:22	cemerick	and (b) this route leads to serious namespace pollution (i.e. lots of top-level namespaces that exist only as aliases)
10:23	rhickey	there will be namespace aliases at some point
10:23	cemerick	OK. I won't plumb down that dead end anymore, then.
10:23	rhickey	I can't imagine a good user-level solution
10:23	cemerick	Yeah, it got pretty messy pretty quick for me.
10:24	cemerick	FWIW, I'm putting this out there to resolve the long-namespace problem related to aligning namespaces with Java packages (brought up here: http://groups.google.com/group/clojure/browse_frm/thread/4b875918d3255119?hl=en)
10:26	rhickey	I think Clojure will move towards Java-style namespaces, which are long, so aliases will be needed. Also to putting clj files in directories mirroring what Java does. This will allow me to compile bytecode with the Clojure namespaces and have Java tooling be able to find it
10:28	cemerick	Nice. I wasn't sure where you were heading with that, as previous comments you made in the group sounded skeptical of having one-to-one relationships between directories and packages/namespaces. +1 to that, IMO.
10:32	rhickey	when in Rome...
10:39	Chouser	Coming from C++-land, I've never been sure I liked Java's mapping of namespaces to directories, so I appreciate that hasn't been assumed all along here.
10:40	Chouser	Ending up that way for good reasons, though, is entirely fine by me. :-)
10:41	rhickey	There are lots of things that leverage classpath-relativity
10:42	Chouser	Did create-struct used to act like defstruct does now?
10:43	rhickey	it's used by defstruct
10:44	elg	n i
10:44	elg	-chat then i
10:44	elg	oh, wrong channel sorry. (stupid client)
10:44	rhickey	Chouser: did you see seque?
10:46	rhickey	I was wondering why you used Deque? (not in Java 1.5)
10:48	Chouser	rhickey: I did, I'm still picking through it.
10:49	Chouser	Handling of nil and false is very nice.
10:50	Chouser	The reason I did enqueue the way I did is because my initial use-case will have a sax parser as the producer, so I won't have the opportunity to return my seq members. Hence enqueue.
10:51	Chouser	But perhaps it would be better to decompose those two parts (seque and enqueue) somehow.
10:51	rhickey	You can't make a lazy-seq on the parser?
10:51	Chouser	I'm gratified you caught the purpose @agt :-)
10:52	Chouser	(to propogate exceptions)
10:52	rhickey	thanks for the docs :)
10:52	rhickey	I didn't at first
10:52	Chouser	this is how I'm making a lazy-seq of the parser!
10:52	rhickey	oh, sax parser is callback based...
10:53	Chouser	right.
10:53	rhickey	yuck
10:53	Chouser	there are pull-parsers out there, which is a much nicer model, but they're not bundled with Java.
10:54	rhickey	if we standardize the sentinels, we could have a queue-seq
10:54	rhickey	then just have your callback fill a queue
10:55	rhickey	seque could use queue-seq
10:55	Chouser	I think I'd still need a separate thread for the parser and callbacks.
10:55	rhickey	QNIL, QEOS
10:57	Chouser	I did have a question that pertains to seque and my original ... if nobody consumes to the end of the seq, we've got a blocked thread hanging out there.
10:57	Chouser	Will the thread be GC'ed?
10:57	rhickey	nope
10:58	Chouser	yeah, that was my guess. so it'd be easy to use the seq in a lazy context and leak threads. :-/
10:58	rhickey	yeah, I thought about a weak ref to the queue in the agent fn
11:00	rhickey	java.lang.ref.WeakReference
11:01	abrooks	http://www.usenix.org/events/usenix08/tech/full_papers/tang/tang_html/index.html
11:02	Chouser	Are threads ever GC'ed while they're still running (or blocked)? I kinda assumed they wouldn't be.
11:02	abrooks	Only tangentially related but I thought it was interesting and perhaps of general interest.
11:03	abrooks	Er, this is a better link: http://www.usenix.org/events/usenix08/tech/tang.html
11:03	rhickey	abrooks: very neat. What's nice for Clojure is as things like this end up in the JVM, Clojure gets them for free
11:03	abrooks	rhickey: Right.
11:10	Chouser	rhickey: I used Deque because I'm a complete Java noob. I wanted "Linked" instead of "Array", and didn't notice there was a "Queue" version. I spent a couple minutes trying to figure out what the difference was, but since it worked for me I just gave up and used it.
11:10	Chouser	I see now that Deque is a Double Ended Queue, and I don't need that at all of course.
11:11	rhickey	ah. I just use the 1.5 JavaDoc so I don't even see things I can't use
11:11	rhickey	trying to keep Clojure 1.5 compatible
11:13	drewr	rhickey: Those of us still on OS X 10.4 thank you. :-)
11:13	rhickey	that would be me too :)
11:14	Chouser	Hm, and my wife. So I guess I should also be greatful.
11:15	drewr	What are the big improvements 1.5 -> 1.6?
11:16	Chouser	Well, mainly these nifty double-ended queues.
11:16	Chouser	;-)
11:17	drewr	Interesting. I've never seen native support for deques in any language before.
11:22	abrooks	drewr: Python has deques: http://docs.python.org/lib/deque-objects.html
11:22	abrooks	Python 2.4
11:23	abrooks	Google reminded me.
11:23	drewr	abrooks: By the beard of Zeus! I've never noticed that.
11:23	abrooks	:)
11:24	lisppaste8	rhickey pasted "seque with reclamation" at http://paste.lisp.org/display/63042
11:25	drewr	I check out PEPs too, but generally after the fact to see the motivation for a particular feature.
11:26	lisppaste8	rhickey annotated #63042 with "seque" at http://paste.lisp.org/display/63042#1
11:26	abrooks	drewr: http://www.python.org/dev/peps/peps.rss
11:28	rhickey	Chouser: that one cleans up threads when no one is using the seq anymore - even works for unbounded seques on infinite seqs, if you don't run out of memory first
11:28	drewr	abrooks: Thanks.
11:28	Chouser	rhickey: very cool.
11:29	Chouser	is it using weak-NIL to notice when it's no longer needed?
11:29	Chouser	it == agent, I guess.
11:29	rhickey	right - the consuming seq has a strong reference to NIL, the agent doesn't
11:30	Chouser	if you constrain the size of the queue and cause the agent to block, might it get stuck there and no go around again to discover it's not needed?
11:31	cemerick	We still need to keep PDFTextStream JDK 1.4 compatible -- there's a lot of people still on 1.3/1.4, and not planning on moving any time soon.
11:31	rhickey	Chouser: ugh, yup
11:31	Chouser	rhickey: sorry to snipe. :-/
11:33	Chouser	WeakReference doesn't seem to have a way to get some code run when the reference is broken. :-(
11:34	rhickey	that wouldn't work - but it can put them on a queue
11:34	rhickey	I could use timeouts...
11:34	Chouser	oh! a timeout wouldn't be too bad.
11:35	Chouser	.poll
11:36	rhickey	no, on the offer side
11:37	Chouser	but for all your work to do my use case any good, I've got to figure out how get callbacks in there.
11:37	Chouser	oh, right. .offer
11:38	Chouser	I'd rather not have 2 extra threads when I really only need 1.
11:49	Chouser	hm, ([s] ... [[... :as s] s]) ... you'd never know s is immutable. ;-)
12:05	kotarak	is creating a closure with fn cheap?
12:19	rhickey	kotarak: yes, the fn gets compiled once and the fn value is a simple ctor call
12:22	kotarak_	ok. the problem was (is?) a macro which is needed because not all arguments are to be evaluated immediately.
12:22	kotarak_	now I wondered whether to put everything in a macro or into a driver function with a small macro wrapper.
12:23	rhickey	the latter
12:23	kotarak_	The wrapper creates a fn which may (or may not) be called by the driver
12:23	kotarak_	ah. ok.
13:01	lisppaste8	rhickey annotated #63042 with "seque w/timeout" at http://paste.lisp.org/display/63042#2
13:23	Chouser	rhickey: cool. Now for my sax parser... ;-)
13:24	Chouser	my general approach of providing enqueue as a binding that my callbacks can use is sound, right?
13:24	rhickey	can we not just Lego-ize the seque code?
13:25	Chouser	I hope so, but I want to make sure I'm headed in the right direction.
13:26	Chouser	I don't see how to get sax+callbacks to populate a seq or queue without it having its own thread.
13:26	Chouser	my callbacks have to push data somewhere, and the place that makes sense is an LBQ
13:26	rhickey	can you just send an agent off with the sax parse method?
13:27	rhickey	yes, push onto LBQ
13:27	Chouser	yes, that's what I was using the original seque (or with-bg-queue) agent and LBQ for.
13:29	Chouser	I guess what I think I have to do is have something almost exactly like your seque (hopefully built out of the same parts) with the only difference being the agent fn
13:29	rhickey	when I look at seque, I see <-seq<-queue<-seq
13:30	rhickey	and your parse: <-seq<-queue<-parser
13:30	rhickey	so, queue-seq: <-seq<-queue seems to be a common part
13:32	Chouser	yes, but seque is controls the loop for the right-hand <-seq part, and is calling rest on it. I can't make my parser behave like that without essentially doing my original with-bg-queue stuff again.
13:33	Chouser	so I guess I'd like to be able to ask seque (or a slightly different version of it) to still provide enqueque or lbq
13:33	rhickey	(defn seque [n s]
13:33	rhickey	(queue->seq (seq->queue n s)))
13:34	Chouser	in fact enqueue ought to now do the keep-alive detection stuff and throw an exception or something the unwind the sax parser if its not needed.
13:35	Chouser	s/the unwind/to unwind/
13:37	rhickey	(defn sax-seq [n xml]
13:37	rhickey	(let [q (LBQ n)]
13:37	rhickey	(send-off agent sax->queue xml q)
13:37	rhickey	(queue->seq q)))
13:38	rhickey	pseudocode of course, keep-alive the trick
13:40	rhickey	queue/seq protocol will have to define eos/nil/exception sentinels
13:41	rhickey	i.e. put exceptions in queue rather than tie consumer to producer agent
13:47	Chouser	ok, maybe I'm beginning to understand how that might work.
14:00	Chouser	of course somebody in that agent needs to know about the sentinels and how to use them. If LBQ is the stock Java class, then sax->queue itself would have to be dealing with keep-alive, NIL, .offer, etc. which seems a shame.
14:01	rhickey	a function can hide some of that
14:01	Chouser	yes
14:02	Chouser	I think we're actually approaching the same solution. But I'm still trying to understand if my enqueue is Bad or not.
14:27	Lau_of_DK	Chouser: Are you working on lazy xml ?
14:31	Chouser	Lau_of_DK: yep
14:31	Lau_of_DK	Interesting - How close are you to finishing ?
14:31	Chouser	well "working" is a bit of a stretch, but yeah that's the goal.
14:31	Chouser	quite a ways.
14:31	Lau_of_DK	Please let me know when you complete, I'd love to see it
14:32	Chouser	once we've got seque hammered out, it'll be easy to generate a lazy seq of sax events. That's very close to done.
14:32	Lau_of_DK	Its it buggy or just not running ?
14:33	Chouser	but then I've got to build a lazy tree from that seq. I have a plan, but it's another whole stage.
14:33	Chouser	the pieces just aren't written yet. no bugs. ;-)
14:33	Lau_of_DK	k
14:51	lisppaste8	Chouser annotated #63042 with "seque and with-seque" at http://paste.lisp.org/display/63042#3
14:52	Chouser	rhickey: I'm sure you'll hate the ugliness of using enqueue internally in seque, but it gets the job done without extra threads or duplicating code.
16:45	Chouser	I can't use :when and :while in the same for?
16:46	rhickey	no, do you need to?
16:47	bpattison	should (pr "\u0022") produce "\"" with an extra backslash? I was expecting just a double quote (")
16:49	rhickey	pr prints readably, try print
16:50	Chouser	yeah, I want (filter p1 (take-while p2 s))
16:50	bpattison	okay -- that works -- thanks
16:51	Chouser	I was hoping to write that (for [i s] :when p1 :while p2)
16:52	Chouser	no big deal. the classic formation's nearly as good as the for-comprehension would be.
16:53	kotarak	clojure is so much fun: I'm trying to understand monads and I implemented a let-bind which is in no way more wordy than Haskell's >>= notation. :)
16:54	rhickey	what does let-bind do?
16:56	kotarak	it does basically simplify: (bind m1 (fn [x1] (bind m2 (fn [x2] (do-something-with x1 x2)))))) to (let-bind [x1 m1 x2 m2] (do-something-with x1 x2))
16:58	rhickey	what does bind do?
16:59	kotarak	bind does: (defn bind [m k] (fn [x] (let [y (m x)] ((k y))))
16:59	kotarak	oops. just a second. confusion
17:01	kotarak	ok. It depends on the monad, I suppose (as I as said: just trying to understand). For the identity monad it is: (defn bind [a k] (k a))
17:04	kotarak	For the error monad (whatever this may be): (defn [[t a] k] (cond (= t 'success) (k a) (= t 'failure) ['failure a]))
17:04	kotarak	defn bind, of course
18:52	meredydd	What's the current status on regexps?
18:52	meredydd	I heard rhickey talking about the possibility of making #"patterns" no longer instances of java.util.Pattern
18:53	meredydd	(for the purpose of making them invokable)
18:53	meredydd	And what's the situation wrt escapes?
19:18	rhickey	meredydd: no changes near term on regex
19:19	meredydd	Okay. So, it's still good old-fashioned leaning toothpicks?
19:19	rhickey	yup
19:20	meredydd	Native reader syntax for regexps is such a win to start with, it feels churlish to complain :)
19:21	rhickey	there have been several threads but each has petered out before consensus
20:29	slava	rhickey: does clojure allow reader macros?
20:33	rhickey	slava: no
21:58	Chouser	is having both drop and nthrest intentional?
22:01	Chouser	more importantly, is there a function that takes a seq and returns the same lazy seq except shorter by one?
22:03	slava	rest?
22:04	Chouser	ah. :-) right, but shorter on the other end.
22:05	Chouser	Sorry I didn't make that clear.
22:05	blackdog	butlast?
22:05	Chouser	right!
22:05	Chouser	oh, it's there already. perfect, thanks.
22:05	blackdog	but it's not lazy?
22:05	slava	it can't be lazy, it has to evaluate the sequence to get the length
22:08	Chouser	it could be lazy, just ahead by one. Which is what I need, so I guess I'll write it.
22:10	Chouser	(defn but-last [[n & r]] (when r (lazy-cons n (but-last r))))
22:10	Chouser	not too painful I guess
22:10	Chouser	rhickey: any reason the builtin butlast should be eager?
22:15	Chouser	hm, I guess mine chokes if you give it a zero-length seq. that's ok for my use case though.
22:30	rhickey	Chouser: no reason, cgrand pointed that out, will fix at some point
23:33	Chouser	well, I've got lazy xml parsing working. a drop-in replacement for src/xml.clj
23:51	Chouser	It's still pretty rough, but here it is in case anyone wants to throw rotten tomatoes: http://n01se.net/paste/nRFa