#clojure – 2008-07-16

2008-07-16

09:06	rhickey	Anyone XMPP savvy?
09:39	Chouser	you might need an xml parser that doesn't strip whitespace
09:39	rhickey	Chouser: ?
09:42	Chouser	xml.clj trims whitespace around text regardless of whether it's meaningful or not.
09:45	rhickey	Chouser: what's an example?
09:50	Chouser	(xml/parse (org.xml.sax.InputSource. (java.io.StringReader. "<a><b>1</b> <c>2</c></a>")))
09:50	Chouser	That loses the space between 1 and 2
09:51	Chouser	emit then inserts newlines where they don't belong.
09:52	Chouser	There may be many XML formats where that doesn't matter, but there certainly are some where that'll break things, xhtml being one of them.
09:52	Chouser	and xhtml can be used in xmpp
09:53	rhickey	would you want a flag toggling the ws dumping?
09:53	rhickey	because otherwise in those other format you get a lot of useless child elements
09:53	Chouser	that's what I've got in my (still unreleased) lazy-xml lib
09:54	rhickey	emit is a toss off, admittedly
09:54	Chouser	but I think a "better" solution is to use a mechanism for interacting with the parsed tree that makes it easy to ignore whitespace you don't care about
09:55	Chouser	same for xml comments. It's nice to leave those things in there if your going to modify something a little and write it back out.
09:55	Chouser	yeah, I added a :pad option to my emit in case you want the newlines.
09:56	rhickey	in any case, I don't expect to be using xml.clj for XMPP, as it's streamed
09:57	Chouser	hm, good point.
09:57	rhickey	might try the smack lib first: http://www.igniterealtime.org/projects/smack/index.jsp
09:57	rhickey	I'm thinking about xmpp for distributed Clojure
09:58	Chouser	oh, of course you don't want to deal with the xml directly. what was I thinking. :-)
09:59	Chouser	well, for what it's worth we actually used xmpp for inter-app communication at work.
09:59	Chouser	we got it working, but ended up ditching it in favor of our own custom XML protocol over TCP.
10:00	Chouser	xmpp had a lot of features we didn't care about, but that the servers tried hard to support. We ended up with hard-to-debug events where the server appeared to be at fault, but it was hard to be sure.
10:01	Chouser	We were happy to remove our depenence on a Jabber server.
10:05	Chouser	But who knows what we may have been doing wrong. YMMV
10:10	meredydd	Is there an Official Answer to what (alter) returns?
10:11	meredydd	Right now, it returns the new value
10:12	meredydd	as does (ref-set) and (commute) - but the docs don't say anything official, and I'm leery of building on undocumented functionality :)
10:15	meredydd	Chouser: yeah...that was my (admittedly arms-length) impression of using XMPP for IPC.
10:15	rhickey	Chouser: thanks for the info on your experience, that's why I asked, but I don't want to build from nothing. Other option is JXTA, but far fewer usingthat
10:15	meredydd	That said, I've just spent a happy week rolling my own IPC system for this product in Clojure, supporting exactly what I want and nothing more.
10:16	Chouser	it's a pity because it seems like it'd be nice: xml, highly available servers, etc.
10:17	Chouser	we also considered UPnP, but that never got past the design stage
10:18	rhickey	I really like extending the chat metaphor to applications, including group messages, pub/sub, presence etc
10:19	Chouser	yep. I think we used direct messaging and pub/sub. Not sure if we used presence or not, and I don't think we used group messaging.
10:20	rhickey	as you said, xmpp has more than is needed, but rebuilding 1/3 of xmpp is still a big job
10:20	Chouser	I didn't implement any of this, but I seem to remember pub/sub being poorly supported or somehow causing us trouble.
10:20	Chouser	I wonder if you built your own server (or at least had more complete control or command of it) if that would make the difference.
10:21	rhickey	I like the availability of openfire and smack from the same source, both Java. I would hope for good client/server compatibility there
10:21	Chouser	We're C++ shop using a Java-based server -- it a mysterious black box to us.
10:22	rhickey	meredydd: yes, they return the new value
10:22	meredydd	rhickey: cool, thanks. Could I request a doc-patch to make that explicit?
10:29	rhickey	meredydd: docs fixed but svn won't let me check in - aargh
10:29	meredydd	rhickey: Eh-oh. Server problem, or SVN internal error?
10:30	Chouser	hm, when SVN at sourceforge was failing the other day, I couldn't even do a fetch, but it worked for me just now.
10:31	meredydd	Chouser: Where's work, by the way?
10:31	Chouser	http://www.fourddev.com/
10:32	Chouser	website's pretty sparse and out of date, but there ya go. ;-)
10:34	rhickey	Chouser: 403 Forbidden on checkin
10:35	Chouser	rhickey: :-( sorry.
10:39	rhickey	Chouser: what jabber server(s) were you using?
10:45	Chouser	I belive we used wildfire
10:48	Chouser	oh, didn't notice you were gone. I believe we used wildfire
10:49	rhickey	Chouser: and for a client lib?
10:50	Chouser	http://delta.affinix.com/iris/
10:52	rhickey	Chouser: in spite of your experience I think I'll continue to explore xmpp - how long ago did you try it?
10:56	Chouser	looks like we gave up on jabber in January of 2007
10:56	Chouser	I hope it works better for you. :-)
10:56	rhickey	me too!
11:00	Chouser	it looks like we used "rosters" and "presence" earlier, and switched to "pubsub" and "status" later. I don't really know what those mean.
11:01	Chouser	I seem to recall the main issue being flakey presence. In our application, we really couldn't have users "disappearing" if they weren't actually disconnected, but we chased just such problems a lot.
11:14	cemerick	open Q: since clojure has its own very expressive structured data format (sexprs/vectors/maps), why use an XML wire protocol? Or would XMPP only be an envelope for readable clojure?
11:15	rhickey	cemerick: the latter, message bodies would be readable Clojure
11:17	Chouser	rhickey: looked at lazy-seq closing at all yet?
11:27	rhickey	Chouser: I planted it in my head before ECOOP, but it really poses profound philosophical problems...
11:27	rhickey	close being inherently not functional
11:27	Chouser	hm, yep, good point.
11:28	Chouser	I don't have a great need for it, but you mentioned you might look at it after ECOOP, so I thought I'd ask.
11:28	rhickey	I have on my agenda making lazy-cons a special op, because it is so heavily used
11:28	rhickey	doing so will allow me to make it one alloc instead of three
11:29	Chouser	ok
11:29	rhickey	and at that time I might extend the interface to include close, but haven't worked out sound semantics
11:30	rhickey	e.g. what if you've aliased a closeable seq and one consumer closes it before the other is done?
11:30	Chouser	yeah, it really messes with the value semantics of seqs, doesn't it.
11:30	rhickey	right
11:34	rhickey	potentially at least, but I'm still thinking about it. In some ways, someone calling close simply makes as much of the seq as anyone has read its effective length. As long as everyone sees the same elements it might not matter. But the threading aspects are tricky too
11:38	Chouser	hm. And the use case is so that func x can return a line-seq to func y which can then close any underlying file that might or might not be there?
11:39	rhickey	Chouser: that's one, but the general problem is one of non-memory resources associated with a seq source - files, threads etc
11:39	rhickey	need to be released when no one cares about the seq anymore
11:41	Chouser	right, but specifically where the resource is created by the seq-creator, not passed in?
11:41	Chouser	or do I still have that wrong?
11:42	rhickey	doesn't really matter, as soon as you pass off the seq
11:42	rhickey	to some code that doesn't know about the resource
11:42	Chouser	yeah, ok.
14:33	jgrant	does clojure support something like common lisps keyword args ?
14:34	rsynnott	you could probably use a hash and destructure in the function declaration
14:36	Chouser_	jgrant: no direct support, but it's not uncommon to say something like:
14:37	Chouser	(defn foo [a b & opts] (let [opthash (apply hash-map opts)] ...))
14:38	Chouser	or with destructuring like rsynnott suggested:
14:38	jgrant	thx
14:39	Chouser	(defn foo [a b & opts] (let [{:keys [c d e]} (apply hash-map opts)] ...))
14:39	Chouser	either of those could be called like: (foo 1 2 :d 5)
15:26	rapido	anyone seen http://will.thimbleby.net/misc/ ?
15:27	rapido	maps instead of lists as the primitive structure
15:27	rapido	(for code and data)
15:27	albino	I looked at it yesterday, but didn't glean much
15:27	rsynnott	"simplicity of the syntax and design of LISP... and a syntax similar to that of Smalltalk"
15:27	rsynnott	weird
15:28	rapido	it's interesting as an idea
15:28	rapido	and it is immutable to the core
15:29	rapido	it seems that more and more (research) languages and frameworks are leaning towards immutability
15:31	rapido	and we also won't need no stinking loops (www.nsl.com)
15:31	albino	what's the perceived benefit of a map instead of a list as the default structure?
15:32	rapido	unification of concepts?
15:32	rapido	although i think a multi-map is even more general
15:32	albino	huh, he's listing xml as one of the benefits
15:34	rapido	xml is not fundamental. but key/value pairs are
15:35	albino	"or example maps can easily represent lists but not vice versa." <- is that really true?
15:35	albino	It isn't easy to go from (1 "a" 2 "b") to a map of 1->"a", 2->"b" ?
15:36	rapido	albino: not so. a list can encode a map. but a map implementation is more efficient for key/value lookup
15:36	rapido	we agree
15:36	rapido	in my language enchilada, everything is a list ... but ....
15:37	albino	rapido: I was gonna ask you what you were implementing your language in these days, but for some reason I couldn't remember the name
15:37	rapido	i flag every (sub) list if its sorted (ascending)
15:37	rapido	if its sorted, its considered a multi-map
15:38	albino	why call it multi-map instead of sorted-map ?
15:38	rapido	certain operations perform faster (sorting, lookup, etc) when a list is sorted
15:38	rapido	multi-map allows multiple keys
15:39	rapido	so (1 "a" 1 "b" 2 "c" 3 "d" ) is allowed
15:39	albino	so non-unique keys then
15:39	rapido	yep
15:40	albino	so when you do a get on 1 do you get ("a" "b") ?
15:40	rapido	yep
15:41	rapido	it's very regular
15:41	rapido	in enchilada, every 'key' and 'value' is an expression
15:41	albino	there needs to be a #exotic-langs or something where all these cool language creators hang out and piss each other off
15:41	rapido	lol
16:13	ericthorsen	rhickey: question on the debug extensions namespace and pathing... given a clojure namespace 'org.me.package in source file stuff.clj, would the resulting file path be org/me/package/stuff.clj ... assuming you moved to the java package/path style for namepaces?
16:16	rhickey_	ericthorsen: I was just thinking about that. If the ns is org.me.package then the file should be package.clj in org/me
16:17	ericthorsen	rhickey: so the compiler will ignore the physical file name (if it does not match)?
16:17	rhickey_	but then a fn foo defined therein will be of class org.me.package.foo, so there's a mismatch with Java's class->file relationship
16:19	ericthorsen	rhickey: but there is no file package in that case. It would be looking for org/me/package/foo.class yes?
16:19	rhickey_	ericthorsen: there are 2 aspects - the source and source path in the debug info and the classnames of the generated fns
16:19	ericthorsen	understood
16:20	rhickey_	so the source and source path would always be based on the file/dirs
16:22	rhickey_	so maybe it would be as you said org/my/namespace/whatever.clj
16:23	rhickey_	so saying org.my.namespace.foo was in whatever.clj would match up classpath-wise
16:23	rhickey_	hmm, but not good for the folks doing the require stuff...
16:26	rhickey_	although leaving the namespace as a dir would allow for multiple files defining a ns, maybe some load order issues
16:33	rhickey_	any lib/require authors/users want to chime in?
18:07	meredydd	Argh. Another compiler bug.
18:10	meredydd	Ooh, wow. I can only assume this to be a reader bug.
18:10	meredydd	(It really doesn't help that the reader exceptions give you no hints as to what file it's reading when it encounters its errors)
18:12	Chouser	meredydd: but maybe you can give us a hint? :-)
18:13	meredydd	Ha. Nope, not a reader bug, just a spectacularly unhelpful error message.
18:13	meredydd	Which reminds me - the next time rhickey can bear to see my face without running away, I want to ask him about the general state of backtraces in Clojure
18:17	Chouser	I have found the stack trace dumps to be rather opaque, but with a bit of practice I'm getting better at picking out the useful info from among the noise.
18:18	Lau_of_DK	It takes a little practice - but I dont see why Clojure it self could provide that useful information ?
18:18	meredydd	Chouser: I agree, and the same thing's happening to me.
18:18	Lau_of_DK	Find a bug in Clojure takes 50x the time it would do in Vs2005
18:18	meredydd	But so much of the time, the info Just Isn't There
18:18	Chouser	clojure's not generating the stack trace, the JVM is.
18:18	meredydd	or, worse, it's in about four levels of "Caused By:"
18:18	meredydd	which is monstrously unhelpful, because you have to do the picking-through-the-noise bit four times
18:18	Lau_of_DK	Chouser: ,then forget the stack trace, there's probably a better way to go about it
18:19	Chouser	meredydd: well that's not really worse is it. having the information buried is better than not having it.
18:19	meredydd	True. Invert that "so much of the time" and "worse" :P
18:19	Chouser	;-) sorry to pedantic
18:20	meredydd	...well, given the context, that missing verb is too tempting to pass up :^D
18:20	Chouser	I wonder if one could write an out try/catch block that then picked through the stack information for you to highlight the most useful bits
18:20	Chouser	s/out/outer/
18:20	meredydd	Not really helpful if you do run-time introspection of exceptions
18:20	Chouser	it's not? why not?
18:21	meredydd	Oh, just that you'd need to do your try/catch magic for every try/catch expression
18:22	Chouser	you can't pick through the whole trace from an outer catch?
18:22	meredydd	I suppose my stack-trace-munging code could have a good go at it...
18:22	meredydd	Chouser: Most of the time, it doesn't make it to an outer catch
18:22	Chouser	oh, in your code base that may be. :-)
18:23	meredydd	But, as I say, I funnel most of it through a single piece of code that pretties it up to send somewhere else, so I guess I could do it there
18:23	Chouser	well the rest of us want to use it too, such as at a repl
18:23	meredydd	uh-huh. Probably put it together as a library function of some sort
18:24	meredydd	give it an exception, and it'll give you a "clojure native" backtrace.
18:24	Chouser	yeah, something like that
18:24	meredydd	To a first approximation, I guess you could just "grep -v clojure.lang"
18:24	Chouser	yep. i tend to look first for .clj files
18:25	Chouser	you'd also want to try to detect "compile time" exceptions an display them differently
18:25	meredydd	Ah...again, my own biases - this is an app with large hunks of Java code, so it's likely the call stack will be weaving in and out between the two languages
18:26	Chouser	hm. I suspect this would be the problem in general -- what each of us considers "important" varies by just enough.
18:26	meredydd	Chouser: Hmm...aren't they all subclasses of CompilerException?
18:26	Chouser	for example, I would imagine Rich generally wants to soo the clojure.lang items. :-)
18:27	Chouser	meredydd: nope, not when macros are involved
18:27	meredydd	Oh, joy.
18:27	Chouser	(defmacro foo [] `(nil))
18:27	Chouser	(foo)
18:27	meredydd	I'd imagine Rich (or anyone else hacking core) to be a very marginal case, though.
18:28	Chouser	java.lang.NullPointerException, but it's at compile time so the stack trace doesn't really go through any .clj code
18:28	meredydd	For comparison, it's not as though ordinary Java backtraces go through interesting bits of the JVM just in case there's a core dev watching :P
18:28	Chouser	hmm
18:29	meredydd	Chouser: Hmm...you could probably detect that case.
18:29	meredydd	"if there's no Clojure code in here, treat it like an ordinary Java backtrace"
18:32	Chouser	actually, you may be right. it says NullPointerException at the top, but it may actually be a CompilerException. I'm not quite sure.
18:33	meredydd	...and when even you're still not quite sure, that's a strong indication that the error message system is, ahem, insufficiently clear :)
18:40	meredydd	In any case, if you're interested, the compiler bug has just vanished into the ether.
18:40	meredydd	Maybe might have been a downstream product of the reader bug, which at a certain point was claiming to successfully read an ill-formed file, and perhaps there was some data-structure brokenness that propagated to the compiler
18:41	meredydd	Unfortunately, I didn't think to save it, so I can't reproduce :S
18:42	Chouser	ah, well. you didn't happen to be doing any regex stuff, were you?
18:43	meredydd	no, why?
18:44	Chouser	something like #"\w+" will kill your reader quick
18:45	Chouser	...although my test of it just now had more robust results than when I've tried in the past.
21:31	koneill	morning all ... it's there a log for this c
21:31	koneill	hannel ? ... sorry about the line break
21:32	arbscht	http://clojure-log.n01se.net/date/
21:32	koneill	thanks