#clojure – 2008-06-12

2008-06-12

00:32	jcrites	hey... I just saw a neat presentation about Clojure
00:33	jcrites	I was wondering -- if I were going to convert some Java code to Clojure, how would I implement the visitor pattern? is there a better Clojure idiom?
08:12	cgrand	rhickey: wow, thanks for working on reduce
08:13	rhickey	cgrand - wait, let me check it in...
08:14	rhickey	ok, now up
08:14	rhickey	reduce problem was a bug from my perspective
08:14	cgrand	cool, thanks
08:15	rhickey	but I would caution that you beware the REPL itself can hold onto arg exprs, so when you want to test this, wrap in a let/fn/times or something
08:18	cgrand	ok
08:18	cgrand	btw I ran into the reduce problem while playing with Wide Finder 2
08:19	rhickey	ah, I've been meaning to look at that, glad you are
08:22	cgrand	I translated Bray's ruby script to Clojure nearly line by line -- it surprised me to not have to fully rewrite
08:23	cgrand	It's certainly not the most idiomatic clojure code...
08:23	rhickey	I fiddled with WF1 a bit, wrote about it in the group
08:24	rhickey	I think Clojure's strength will be in the ease-of-parallelization area
08:24	rhickey	http://groups.google.com/group/clojure/browse_frm/thread/7f2180bba2bb67c3/ca161d86b3d8045f?lnk=gst&q=wide+finder#ca161d86b3d8045f
08:24	rhickey	many new features make that even easier now
08:25	rhickey	pre regex support
08:25	rhickey	pre destructuring
08:25	cgrand	yup, right now I have a parallel version which is the serial one with a pdoseq macro instead of a doseq
08:25	rhickey	that is very cool
08:26	cgrand	but it not performs pretty well on the T2000
08:26	cgrand	-> it doesn't perform
08:26	rhickey	wf1 wasn't very parallelizable
08:27	rhickey	if you want input let me know
08:27	cgrand	you still recommend to use agents to dispatch lines ?
08:28	rhickey	not sure about the granularity, lines may be too small
08:29	rhickey	there are 2 aspects to performance here - one is absolute perf, the other is speedup due to parallelization
08:29	rhickey	if you get the latter, it's still a benefit
08:31	rhickey	the former can be a sinkhole as people do block i/o etc
08:31	rhickey	really a waste of time
08:31	rhickey	but should try to beat Ruby :)
08:31	lisppaste8	cgrand pasted "Wide Finder 2 (serial code)" at http://paste.lisp.org/display/62110
08:49	cgrand	this code is already faster (but not that much) than Ruby: on my box for 10M lines I get 3'45" (Ruby) and 2'45" (Clojure)
08:51	blackdog	cgrand, is the regex compilation cached?
08:52	cgrand	afaik the compiler turns regex literals into constants
08:54	cgrand	(the constants being the compield java.util.regex.Pattern
08:54	cgrand	)
08:56	blackdog	ok, that's interesting, so one doesn't need to re-pattern in advance then
08:59	blackdog	cgrand, is that jruby or cruby your testing on?
08:59	cgrand	cruby
09:09	rhickey	blackdog: yes, regex literals #"..." are compiled on read
11:40	Lau_of_DK	Any body here got some experience with the 5 Guess Mastermind algorithm ?
11:48	rhickey	Lau_of_DK: please try to keep the dialog here focused on Clojure, thanks
11:50	Lau_of_DK	rhickey, I'm entering (or trying to) a 5 guess algorithm written in clojure in a contest - does that make it relevant ?
11:51	rhickey	only if related to the Clojure part
11:55	Lau_of_DK	rhickey, I understand that you want to keep it relevant - but this is one of the only places I get to hang out with Clojure coders, and everybody here code in Clojure so I imagine any general discussion about an algorithm would end up in actual code talk
11:55	Lau_of_DK	Personally I would really enjoy it, if there were freedom for that kind of discussions, without keeping a tight censorship
12:18	yrb	is the (Class.) syntax to create a new instance only in the svn branch? and is a good idea to follow the svn or better to stick with releases?
12:19	rhickey	(Classname. ...) is svn only
12:20	rhickey	I think many here track svn
12:25	Lau_of_DK	Questions about the emacs stuff: When I start up swank clojure, I have 2 seperate REPLs running, one is inferior-lisp and one is slime-repl clojure - is this the same for you guyS?
12:38	Lau_of_DK	If I do (recur (inc cnt) (take cnt fibs)) and cnt start on 1 - will this then take 2 ? Or is cnt not incremented until the loop restarts?
12:47	Lau_of_DK	nvm
14:12	albino	rhickey: watching the "clojure for java programmers", thanks lots of good info. One thing if I may make a suggestion, next time if possible can you restate the audience questions? Or maybe talk the hosts into giving audience member mics.
14:13	rhickey	Yes, I'm terrible with that... sorry
14:13	albino	rhickey: np, still way worth the time to watch
14:19	rsynnott	Lau_of_DK: that's normal; you want to use the slime-repl one
14:19	Lau_of_DK	rsynnott, okay thanks - and yea thats what Im doing, just wondering why I had a second
14:22	rsynnott	I think the other is the lisp's stdin/out)
14:22	rsynnott	or the clojure's, in this case :)
14:23	Lau_of_DK	oh ok. One of the gabs in my "Lisp education" is that I never really found any good explanations of all the workings of emacs, slime, swank and all that. Got any good links? :)
14:24	rsynnott	nope
14:24	rsynnott	slime is more or less a link to the remote repl, with convenience stuff and a fancy inspector and so on
14:24	rsynnott	it uses data from the remote lisp to do indentation and completion and similar (which is very nice)
14:25	rsynnott	I wonder will someone do CUSP for Clojure? :)
14:25	rsynnott	(CUSP is a Java replacement for the Emacs bit of slime)
14:26	Lau_of_DK	rsynnott, thanks for the explanation, youre a link in your own right :)
14:27	Lau_of_DK	I'm not sure how difficult it is to build a editor/repl interaction, but not taking that into account, it wouldnt be a huge task writing a good IDE for clojure
14:30	rhickey	have you tried enclojure?
14:31	Lau_of_DK	Yes, I've tried it
14:32	rsynnott	I've tried one netbeans UI for non-java-language before; it isn't an experience I'd care to repeat :)
14:32	rsynnott	I'm happy with SLIME
14:32	rsynnott	(it'd be nice to have an inspector and so on, of course)
14:33	Lau_of_DK	I never thought I'd say this, but I'm actually pretty happy with the emacs integration. If Enclojure gets a better interaction with the REPL and a few other details sorted out, I might switch to it though
14:33	ericthorsen	rsynott: what problems did you encounter? We are building the enclojure plugin for clojure. Have a REPL, code browser, debugging support, etc.etc.
14:34	ericthorsen	There is still much work to do but Netbeans has taken care of quite a bit of heavy lifting.
14:34	ericthorsen	http://www.enclojure.org
14:36	rsynnott	I just didn't like netbeans very much, really :)
14:40	Lau_of_DK	ericthor, can you evaluate single defn's directly to the REPL in Enclojure yet?
14:56	Lau_of_DK	In C#, if I wanted "Bits" to become "Bitz" I could just do str[3] = "z" - Is there anyway I can do that operation from Clojure on a java string?
14:57	Chouser_	you want to mutate the string!?! ;-)
14:59	Chouser_	(str (.substring "Bits" 0 3) "z")
14:59	Lau_of_DK	Yes Chouser, I have this one stupid assignment that I've spent an hour on now, its driving me crazy
15:00	Chouser_	(.replace "Bits" "s" "z")
15:05	rsynnott	assignment?
15:05	rsynnott	are you using this for college stuff or something?
15:06	Lau_of_DK	No its just a project
15:07	Lau_of_DK	I'm trying to do a Mastermind puzzle solver
15:27	rsynnott	ah
15:27	rsynnott	cool :)
15:27	rsynnott	I did one of those years ago... in C
15:27	rsynnott	yuk
15:27	rsynnott	no, actually, it was in turbo pascal :S
15:34	blackdog	jumping in, i have a start of a clojure plugin for jedit,
15:34	blackdog	it has a console and can run jedit macros, but i'm having difficulty evaling a buffer
15:35	blackdog	i'll post what i have if anyone is interested
15:38	blackdog	well not evaling, but it doesn't seem to use the same classloader as the console, cos i can't find the vars in there
15:38	Chouser_	blackdog: clojure uses its own classloader
15:39	rhickey	but that classloader is a child of the one used to load Clojure
15:40	blackdog	i'm not sure, i was going well, with clojure macros being able to populate the clojure console,
15:40	blackdog	with stuff, but doing the same kind of thing by evaling the buffer didn't work, maybe slava will help :)
15:40	blackdog	jedit is his baby
16:36	Lau_of_DK	I have a long function that is throwing a Typecast error - Is there anyway I could get a linecount that fires this exception, instead of just 20 lines of java dump?
16:51	rhickey	new release, new site up
17:00	la_mer	oooh, shiny :-)
17:04	la_mer	rhickey: regarding the license -- have you gotten any more feedback on it w.r.t. commercial use of clojure? (I'm remembering now the slight dust-up over the license some months ago on google groups)
17:05	la_mer	(FWIW, I agree with your interpretation of the CPL, I'm just wondering if anyone else has expressed concerns / disagreement with your interpretation)
17:10	Chouser_	Lau_of_DK: aren't there line numbers on each line of the stack trace?
17:11	Lau_of_DK	Yes, but to files like boot.clj, swank-clojure.clj and so on
17:12	Lau_of_DK	I just want something like Cast exception: Line 5, Col 2, in my own file
17:14	Chouser_	Lau_of_DK: perhaps you can nopaste the function and stack trace?
17:14	Lau_of_DK	nopaste?
17:14	Chouser_	http://paste.lisp.org/new/clojure
17:17	Lau_of_DK	thanks, i'll get to a little later, I have to finish something before bed
17:26	rhickey	la_mer: no complaints for commercial use. May be some GPL incompatibilities
17:28	slava	rhickey: nice site
17:28	slava	what CMS are you using?
17:31	rhickey	that's wikispaces
17:31	slava	hopefully one day it will be something written in clojure :)
17:31	rhickey	I guess, if someone's interested in that domain
17:37	la_mer	rhickey: that's a plus, IMO ;-)
17:37	slava	why did you pick CPL and not BSD?
17:40	rhickey	because I want derived work to give back
18:01	slava	i rewrote your persistent-vector implementation in factor; is this code bound by the CPL?
18:01	slava	does a rewrite count as a derived work
19:20	rhickey	slava: I think that depends on how much of a copy it is and how much original - it might
19:27	slougi	hi. is there an example somewhere on how to use external java libraries from clojure? I'd like to play with using it in conjunction with Qt Jambi.
19:28	rhickey	if they are in your classpath, you can import them just like built-in Java libs
19:29	slougi	how would that be? (I am not entirely fluent in Java)
19:29	rhickey	how to put in classpath or how to import?
19:30	slougi	how to import :)
19:30	rhickey	parallel.clj, which comes with Clojure. imports the forkjoin library - you could look at that
19:31	slougi	thanks for the pointer. and sorry for the newbie questions, i honestly didn't find a good answer before.
19:32	slava	rhickey: have you considered adding compile time type checks of any kind to clojure?
19:32	rhickey	a little bit - since I track types for reflection elimination
19:33	rhickey	I like the idea of optional checks that generate warnings
19:36	rhickey	slougi: doc for import is in: http://clojure.org/namespaces
19:45	slougi	rhickey: cool, i missed that before. I was looking in java interop. thanks =)
19:50	slougi	wow, this is very cool
19:50	slougi	the first lisp that actually interacts with the toolkit i want to use in a decent way
19:52	slava	clojure has a lot of potential, it is well-designed and it will hopefully get more people interested in lisp in general
19:52	Ycros	oh wow, when did the site get a redesign
19:52	slava	my interest in clojure stems from its use of persistent data structures and iterators. i don't care about java interop so much :)
19:52	slougi	it's the first lisp that i like the feel of to be certain
19:53	njbartlett	I want to call the Compiler.load method from Java, but I need to specify a classloader. Can I somehow put it into the LOADER static field?
19:53	rhickey	Ycros: redesign is still underway, but had to cut over with the release since that's where the new docs are
19:56	rhickey	njbartlett: could you explain a bit why?
19:57	njbartlett	rhickey: OSGi integration. I want to compile some Clojure code contributed by a bundle, using that bundle's classloader
19:57	rhickey	for use only by that bundle?
19:58	njbartlett	rhickey: Not necessarily, the result could be shared
20:00	njbartlett	rhickey: Hmm I guess the clojure-generated classes are never going to form exportable API from that bundle, but instances could find themselves in other bundles as services
20:01	rhickey	I know nothing about OSGi, but wonder how cross-sibling sharing works, normally couldn't be seen
20:02	njbartlett	OSGi doesn't use hierarchical classloaders. Instead they're arranged in a graph, with explicit imports and exports of Java packages
20:04	rhickey	The classes Clojure generates on the fly should never be seen directly by name, only through interfaces in the Clojure libs, e.g. IFn
20:04	njbartlett	But just thinking aloud here... to be able to write exportable classes in Clojure, I would have to get the compiler to generate a class from a clj in response to a class load event, which means I need a special classloader
20:04	njbartlett	rhickey: Hmm but you can generate interface instances as dynamic proxies right?
20:05	rhickey	exactly - I can;t depend on the supplied classloader's ability to load bytecode from memory - I must create my own - but could parent from supplied loader
20:05	njbartlett	Right that's what i need to work out, how to supply you a parent classloader
20:05	rhickey	interface instances yields instances of the interfaces, at least with proxy, so there too name is not exposed
20:05	njbartlett	I assume your classloader does normal parent-first delegation
20:05	rhickey	it does
20:06	rhickey	I sued to use the context classloader, which would have given you a way...
20:06	rhickey	used
20:06	njbartlett	Right. But now?
20:07	rhickey	now it's hardwired to RT.ROOT_CLASSLOADER
20:07	rhickey	you could, as an experiment, make that non-final and set it temporarily to try your idea
20:08	njbartlett	Uhuh. I was already thinking this wasn't going to work without submitting back patches
20:08	rhickey	let's see if it works at all first. I'm willing to work on making this hookable
20:09	njbartlett	BTW my codename for this little project is Closgiure :-)
20:09	rhickey	yikes
20:09	rhickey	:)
20:10	rhickey	I think there are some fundamental conflicts between these graph-like classloaders (and need to re-parent all the time) and gen-and-load-class
20:10	njbartlett	So ROOT_CLASSLOADER is static, like everything else... that means if I set it temporarily I can only compile in a single thread
20:11	rhickey	you can try it out and I can make it thread-local with vars
20:11	njbartlett	Okay
20:12	rhickey	if you see compiler.load does a lot of that already
20:13	njbartlett	Yeah I'm just trying to get my head around your, um, unorthodox Java coding style!
20:14	rhickey	I still question why you need to use that other classloader
20:14	rhickey	what happens if you don't?
20:17	njbartlett	Say the Clojure code imports package org.foo. That package will be visible to the containing bundle because it imports org.foo at the bundle level, but it won't be visible to the Clojure bundle itself. Essentially I want to make a Clojure bundle that performs a "compilation service" for other bundles.
20:17	rhickey	I don't think that is sound
20:18	njbartlett	Okay, I don't know enough about Clojure yet. It's an experiment.
20:18	rhickey	Clojure is a compiled language, whose compiled named entities are static and global (per classloader), just like Java's
20:19	rhickey	i.e. there's no difference between clojure/rest and Math.max
20:19	njbartlett	Right, but your compiler is executed at runtime
20:19	rhickey	doesn't matter, it's atill trying to obtain compiled performance, so names must be bound, not looked up every time
20:20	rhickey	again, like Java, code that uses Math.max gets linked/bound once to a global entity
20:20	njbartlett	What do you mean, names must not be looked up every time?
20:21	rhickey	The alternative to my unorthodox static style is to put names in instance environment objects in which things are looked up
20:21	njbartlett	Well no it doesn't. Somebody asks their classloader to load the Math class, and that gets delegated up to the boot classloader, and then the static method if invoked.
20:22	njbartlett	In theory there could be an alternative implementation of Math. Actually not because of a special rule (only the boot classloader is permitted to define classes in the java.* namespace)
20:22	rhickey	once only, it's not looked up every time, and it's looked up in a global place
20:22	rhickey	per classlader
20:22	rhickey	classloader
20:23	njbartlett	Right, I don't see what this has got to do with my approach to integration with OSGi
20:24	rhickey	Having one Clojure and many unrelated classloaders puts the onus on Clojure to work like OSGi
20:24	rhickey	which provides some non-hierarchical sharing
20:24	njbartlett	I don't believe it does
20:25	rhickey	let's set the class part aside. If you load some fn ns/foo, that is globally available to all modules using the same Clojure
20:26	rhickey	Which wouldn't conflict with the Java hierarchical model, since they would all be beneath whatever loaded Clojure
20:26	njbartlett	Ah I see what you mean about the sharing your do
20:27	njbartlett	How do you make these things visible? It's clearly not just via classloaders
20:27	rhickey	it kind of is ,since I use Java's static semantics
20:28	njbartlett	Right but you assume you always have the same parent classloader
20:28	rhickey	so the module that loads Clojure is like Clojure's version of the System classloader
20:29	rhickey	it's a root for everything running under that Clojure
20:29	njbartlett	I see.
20:30	rhickey	using statics ensures that if you loaded a second Clojure under another clasloader, it would be independent
20:30	rhickey	completely
20:30	njbartlett	Okay, I have tried separate instances of Clojure, one per bundle, but it's just too damn slow for that.
20:30	njbartlett	Like 5 seconds startup time
20:31	njbartlett	Fine if you're only doing it once for the whole JVM...
20:32	rhickey	then I would say that you need to take a different approach to modularity for using a shared Clojure than classloaders - using Clojure's namespaces for instance, because the OSGi separate classloaders aren't really giving you any isolation in this case
20:32	njbartlett	Right. In other words, Clojure doesn't add any value to OSGi and vice versa. Or so it seems.
20:34	rhickey	possibly not, but with understanding of how it works we might be able to address your objectives in using OSGi
20:35	rhickey	because you certainly can share an instance, and Clojure has very good thread independence
20:36	rhickey	and namespaces
20:36	njbartlett	I'm committed to OSGi so the question for me is whether Clojure can be useful to me in that environment.
20:37	rhickey	I've talked with people using NetKernel about using UUID named tear-off namespaces
20:37	njbartlett	Right I followed some of that thread
20:38	rhickey	your loader service could load code in a guaranteed unique namespace, and deliver IFns to consumers, dropping the ns after load
20:40	njbartlett	I'm going to have to learn more about Clojure and come back to this. I think I've come at it with some invalid preconceptions from doing Scala etc
20:41	rhickey	feel free to follow up on the group
20:41	njbartlett	Will do, thanks
20:49	Chouser	or something.
20:52	rhickey	ruby?
20:54	Chouser	yep. on rails. It was my choice as or a year or two ago. Now I have to live with it.
20:55	Chouser	"as of"
20:55	rhickey	would you want a full-stack rails-alike for Clojure?
20:57	Chouser	not sure. The main benefit I was hoping for from Rails was an architecture with pluggable features.
20:57	Chouser	need a user authentication system with email-reset of password? plug it in. that sort of thing.
20:58	Chouser	If properly done, I would think a modular (non-full-stack) system could allow for that.
20:58	Chouser	...and in practice, rails doesn't even do a particularly good job. "plug it in" includes generating code in your app source tree, which rather complicates upgrades.
20:58	dudleyf	Chouser: Have you looked at Merb?
20:59	dudleyf	I'd like to see something more like that for Clojure
20:59	dudleyf	Working with ActiveRecord has made me hate ORM, too
20:59	Chouser	I haven't done enough of this kind of app to know how to design an API that cuts across at the "right" places. I was hoping Rails would be an easy way to import that kind of expertice, but I'm not overly impressed.
21:00	Chouser	I've always been a bit cold to ORM. If I knew of a good replacement for the relational database, I'd definitely look into that.
21:00	Chouser	dudleyf: peeked at it, but it's way too late now.
21:01	rhickey	heh, I think the problem is the O part
21:01	dudleyf	I like the database part, It's the objects that bother me
21:01	rhickey	not the R
21:01	dudleyf	Heh, yeah
21:01	rhickey	I've done OODBMS, no mapping, it was not a good experience
21:02	Chouser	they both seem a poor fit. My data doesn't really fit neatly into tables, and after forcing it there both I have to work pretty hard to express what I mean, and the DB has to work pretty hard to make it performant.
21:02	dudleyf	I don't have much experience with JDBC, is it possible (easy) to get back something that looks like a seq of maps?
21:02	rhickey	that can happen too
21:02	JamesIry	dudleyf: that would be very easy
21:02	Chouser	hm, not having ever used an OODBMS I thought maybe they were the Solution.
21:02	dudleyf	Chouser: CouchDB?
21:02	rhickey	(doc resultset-seq)
21:03	JamesIry	dudleyf: JDBC result sets are more or less the Java equivalent of seq of maps
21:03	Chouser	peeked at that too.
21:03	Ycros	activerecord is the worst part of rails (I'm also a rails developer)
21:03	dudleyf	Thanks
21:03	rhickey	The problem with OODBMS is that information is not objects
21:03	JamesIry	For a completely alternative way to interface to a SQL DB, see also HaskellDB. Probably not appropriate for a dynamically typed language, though.
21:04	rhickey	it can be sliced an indefinite number of ways
21:04	Chouser	tuplespace
21:04	rhickey	but an OODBMS bakes an applications view of the data in to the data
21:04	Chouser	I've been flirting with RDF too.
21:05	Chouser	see, this is part of why I picked rails -- I wasn't confident enough in any particular alternative. Oh well. For now I just have to live with it.
21:05	Chouser	...learn as much as I can about what I would really want.
21:07	Ycros	merb is actually a lot better
21:08	Ycros	it has cleaner code, less magic, runs faster, uses less memory
21:08	Ycros	and it doesn't tie you down to activerecord
21:08	Ycros	of course, you do lose the ability to simply drop in a lot of rails plugins (some of the more generic ones still work)
21:09	dudleyf	rhickey: Do you have large projects in Clojure?
21:09	rhickey	Clojure is a large project
21:09	rhickey	Not yet, starting to use it in my work
21:09	dudleyf	Well, right :-)
21:14	dudleyf	Are namespaces enough to keep the various modules of code separated from each other in a large project?
21:14	rhickey	works for JAva
21:15	dudleyf	But Java has classes as well as packages
21:16	rhickey	and lots of the functions in those classes are getName/setName etc, not even needed with maps
21:17	rhickey	namespace names can be multi-component as well, so you could go one level further to mimic class scope
21:19	blackdog	rhickey, how does that work? multi-component?
21:20	rhickey	foo-bar-baz/fred, or foo.bar.baz/fred
21:20	rhickey	just convention
21:20	blackdog	ok