#clojure – 2008-08-27

2008-08-27

07:19	achim_p	hi!
07:20	achim_p	is there a way of concatenating vectors which is more efficient than (vec (concat v1 v2)) ?
07:29	achim_p	(vec (concat vec1 vec2)) takes linear time, as i understand, because concat only uses the sequential views of the vectors, right?
07:29	achim_p	i thought there might be a faster way, depending on the implementation of the vector type ...
08:33	rhickey	achim_p: yes, sequential, there's no deque or virtual vectors yet
09:30	achim_p	rhickey: okay, thanks!
09:31	achim_p	what are clojure vectors based on? are they bagwell vlists? i believe you mentioned it in one of the videos, but i can't find it now
09:31	rhickey	achim_p: they are not vlists, but bitmapped hash tries like the maps, the difference being the keys are implied and the range is full
09:38	Chouser	I wouldn't think it'd be too hard to use proxy to whip up a virtually concatonated pair of vectors
09:42	achim_p	rhickey: i'll have to dig deeper into this, i'm new to this functional stuff ;)
09:42	achim_p	thanks again - also for clojure, it's really exciting!
09:42	rhickey	you're welcome!
09:44	blackdog	rhickey, did you see this from jrose@sun on the jvm list? Folks, invokedynamic, continuations and tailcalls are brewing (in
09:44	blackdog	three different locations) and interface injection will (I hope)
09:44	blackdog	become real after method handles go mainstream.
09:45	blackdog	tail calls? is that the tail call optimiation stuff?
09:46	rhickey	blackdog: yes, that will be the real tailcalls. I'll be speaking at the JVM Languages Summit and hope to get the scoop on where things are at.
09:47	blackdog	yea, glad to see you going and getting the recognition
09:47	blackdog	java luminary i think they say :)
09:47	Chouser	It's pretty amazing how little tailcalls are needed in Clojure. I guess it'll be nice for mutual recursion.
09:48	rhickey	Chouser: well, I designed based on what was possible
09:49	Chouser	sure, but even if there were real tailcalls, I would still prefer loop/recur when it would work -- I think it's actually clearer on a couple levels.
09:49	rhickey	even with all the talk now, so far that work is relegated to the playground JVM, and who knows when it will be in a JDK, and then how long before companies are willing to move to that JDK
09:51	drewr	What would be the advantage of real TCO over the tricks loop/recur does now? What it incur less overhead because you're having to mimic the behavior?
09:52	rhickey	drewr: real TCO lets you call another (non-self) function without growing the stack, loop/recur is limited to internal calls
09:54	drewr	Ah, I guess I've only ever needed internal calls.
09:55	achim_p	Chouser: like a list/vector of vectors? that wouldn't be much better than a sequence. you'd have to visit each subvector and determine its count to figure out which element of which subvector a given (global) index maps to
09:56	rhickey	you need TCO when you are trying to create a concrete result using recursion, and at each step you will conditionally call a different function to process the rest. There are some elegant algorithms that require TCO.
09:58	Chouser	achim_p: Well it's probably not a good general solution, but if you had a particular case with a small number of very large vectors, you might be able to write something that performs better than just appending them all into a single vector.
10:00	Chouser	so is there any interest or desire to use something like setf in Clojure?
10:00	Chouser	(def m {:a1 {:b1 [:c1 :c2 {:d3 5}]}})
10:00	Chouser	(setf ((nth ((m :a1) :b1) 2) :d3) 9) => {:a1 {:b1 [:c1 :c2 {:d3 9}]}}
10:02	rhickey	Chouser: setf is based upon there being a 'place' that is changed, there's no place there as it is functional, so setf is not the model. We've talked about <- (analogous to ->) here before
10:02	Chouser	hm.
10:03	rhickey	and yes, I'm waiting for <-, but no one has gotten desperate enough. I;'m surprised, as once you get nested it gets pretty arduous)
10:04	rhickey	(<- m :a1 :b1 2 :d3 9)
10:04	Chouser	http://clojure-log.n01se.net/date/2008-07-15.html#09:22
10:05	Chouser	Parth Malwankar has posted related ideas to the forum recently.
10:11	achim_p	Chouser: i'm looking for a data structure that suppports a fairly efficient (sub-linear) "drop-nth". my initial idea was concatenating two subvecs, but this is really slow on large vectors. a vector-of-vectors kind of thing will fragment very fast in my case, because i'll be dropping lots of nths ;) (all of them, eventually)
10:11	rhickey	(assoc-> m :a1 :b1 2 :d3 9) ?
10:11	Chouser	Currently: (-> m :a1 :b1 (nth 2) :d3) => 5
10:11	cemerick	it'd be nice to be able to invoke Callables as fns...
10:12	Chouser	So perhaps: (<- m :a1 :b1 (nth 2) :d3 9) ?
10:12	rhickey	cemerick: that would add a conditional to all invocations where there is none now
10:13	cemerick	yeah, I know it's unworkable -- just musing away
10:13	Chouser	achim_p: ah, I see. What about a sorted-map with integer keys?
10:15	Chouser	(<- m :a1 :b1 (nth 2) :d3 (do 9))
10:15	rhickey	achim_p: you want drop nth and the result to still have fast indexed lookup?
10:17	achim_p	yeah, not sure whether that's possible at all ...
10:17	rhickey	Chouser: that's why I renamed to assoc->, the macro needs to know if can rebuild with assoc, so arbitrary accessors can't be supported (without getting into defsetf and places)
10:17	rhickey	achim_p: not as far as I know
10:18	Chouser	rhickey: ok, so it could assume "get" on the way in and "assoc" on the way out.
10:19	rhickey	Chouser: maybe best not to get stuck on -> analogy, but idea is to just take keys followed by val and do nested assoc
10:19	rhickey	right
10:19	Chouser	what of the final item -- a value or a function?
10:20	Chouser	It looks like Parth's pretty close to this already.
10:22	rhickey	in the original discussion there was the possibility of the last expression being either a value or a parenthesized function-expression of the (presumed existing) value, missing first arg, like: (+ 5) or (assoc k v)
10:24	rhickey	or could have separate alter-> assoc->
11:01	parth_m_	rhickey: Just as Chousers mail on updating nested structures on the group.
11:01	parth_m_	I was wondering if you had a chance to see the field-write function here http://groups.google.com/group/clojure/msg/c312ba23efcf4c17
11:02	parth_m_	I needed it quite badly as I have started using 3-4 levels of nesting for a program I am writing.
11:03	parth_m_	The improved field-read posted by Graham Fawcett also seemed nice to me.
11:05	Chouser	parth_m_: I didn't understand your explanation for why field-write can't be a macro.
11:05	Chouser	oh, nm, I got it now. :-)
11:05	rhickey	there's no need for field-read: (-> nx :a :b :c)
11:06	Chouser	you want to be able to pass in a computed access-spec.
11:08	Chouser	oh cool -- with -> you can even provide default values: (-> nx :a (:b {}) :c)
11:08	parth_m__	sorry I got dropped off
11:08	rhickey	-> rocks
11:09	Chouser	although if you have vectors mixed in you need to name "nth" explicitly.
11:10	parth_m__	I would be happy to see field-write (or something like it) in clojure or contrib.
11:11	parth_m__	The use case I had for field-read was that as structures become nested, I don't really hand code the access-specification.
11:12	Chouser	did you see you can use the -> macro?
11:12	parth_m__	I have a separate function get-access-path that takes a keyword and returns a vector that is the access-spec.
11:13	parth_m__	I am sorry, my client died and lost the messages.
11:13	Chouser	11:05 < rhickey> there's no need for field-read: (-> nx :a :b :c)
11:14	parth_m__	Is there a way to pass a "apply" a vector to ->? Maybe there is but I couldn't figure it out.
11:15	Chouser	I think you'd have to use (eval), which isn't too happy.
11:15	parth_m__	Oh.
11:16	rhickey	parth_m_: (reduce get nx [:a :b :c])
11:17	parth_m__	Oh. Thats nice ... thanks :)
11:22	Chouser	"reduce get" works on vectors, but you lose the default-value feature of ->
11:24	parth_m__	I am not really thinking of (reduce get ..) as a replacing ->. Just a complement for field-write which has a similar use and feel.
11:25	Chouser	oh, of course. -> is much more general than (reduce get ...)
11:26	Chouser	It just seems like it'd be nice to have a symmetrical get and set, but for that to work you need a comprehensive get.
11:27	Chouser	if the different ways to do nested get have different feature sets, it's harder to have a symmetrical set to go with them.
11:28	rhickey	Chouser: CL's places are the general solution, but for Clojure without the in-place aspect
11:28	parth_m__	Chouser: Are you thinking of something like nested sets etc?
11:30	parth_m__	Specifically for field-write/read struct, accepting vectors is an important use case in my view as access-path [:a :b :c] etc. may be generated at runtime based on the final key. At least thats how I am using it.
11:31	parth_m__	It allows me to build layers on top of the field-write ... e.g. (def reg-write [core reg value ] (field-write core :content value (reg-path reg)))
11:42	Chouser	being a function (instead of a macro) would be helpful for integrating with commute and alter as well.
11:44	mac__	Hello, can someone help me out with gen-and-load-class? I'm trying to generate a class in order to specify a finalizer (yes I know that's considered bad java but humor me) but I can't get it right. This is what I got: (in-ns 'test)
11:44	mac__	(clojure/refer 'clojure)
11:44	mac__	(gen-and-load-class 'test.Hello)
11:44	mac__	(defn Hello-finalize []
11:44	mac__	(println "oh no!"))
11:44	mac__	And it works if I make up some own method and put it in with a :method parameter to gen-class but this does not seem to work. Is it because finalize is protected or something?
11:44	mac__	oops, sorry for the linebreaks, just pasted in some stuff :)
11:45	Chouser	gen-class doesn't generate finalize methods
11:45	mac__	I see, so I would have to do that in java then?
11:46	Chouser	It used to (for all classes), but it made all gen-class classes much slower at runtime, so that feature was backed out.
11:46	mac__	Well that's sound I guess since .. .yeah ^^ the gc would go crazy
11:46	mac__	Could be an optional thing though
11:47	Chouser	I guess for now you would have to use Java, though I don't think it'd be too hard to give gen-class -- right, an option to generate finalize.
11:47	mac__	like :finalize or something in the arguments
11:47	Chouser	right
11:47	mac__	ok thanks
11:48	Chouser	rhickey: what do you think, would you accept a ":gen-finalize true" patch for gen-class?
11:50	rhickey	Chouser: can you get one by using :methods currently?
11:50	mac__	I tried but just got exceptions.. don't know if I got the syntax right though
11:50	Chouser	Hm, I may have jumped to a conclusion there. Hang on.
11:50	rhickey	if not, yes, patch ok
11:51	mac__	Would it look like this? :methods [['finalize []]]
11:51	mac__	Cause that won't work for me :)
11:53	Chouser	mac__: try: (gen-and-load-class 'foo.Hello :methods [['finalize [] Void/TYPE]])
11:54	mac__	ah Void/TYPE, cool didn't know about that. Trying...
11:56	Chouser	I seem to be able to define the class and instantiate a bunch, but I'm not sure if finalize is getting called or not.
11:56	Chouser	I don't really know Java well enough to be sure.
11:56	cemerick	finalize is never guaranteed to be called
11:57	mac__	Yeah it's hard to verify if that worked :) but at least there is a finalize method on that class now.
11:57	Chouser	you'd think after making 10000 foo.Hellos, that some would go away...
11:58	mac__	Yeah if you don't hang on to them
11:58	mac__	Try doing (System/gc), not guaranteed to do anything but it may help
11:58	cemerick	my understanding is that, even if a foo.Hello is gc'd, the finalize method isn't guaranteed to be called
11:58	mac__	I got nothing when I did it
11:59	Chouser	I'm also not sure of the context finalize would be called in -- will it still have access to stdout?
11:59	cemerick	as long as the jvm isn't in the process of exiting, then yes
11:59	mac__	Yeah you can do anything from a finalize pretty much.
12:01	Chouser	The finalize method generated this way is "public" -- does that signature differ enough from the "protected" one provided by Object that it won't be called by the gc?
12:01	mac__	And they are not guaranteed to be called before exit of the jvm but in a lenghty process they should get called before the gc reclaims the objects memory.
12:02	mac__	Chouser: I don't know but it doesn't seem to work anyway. I did a java test earlier today and got printouts from finalize when doing a System.gc
12:02	mac__	So I smell a difference
12:02	Chouser	hm
12:03	parth_m	rhickey: Do you see value in field-write (and possibly its complement field-read - for usability) being added to clojure? or contrib?
12:05	rhickey	parth_m: yes, but the interface matters a lot to me, field and write are bad names
12:06	parth_m	Agreed. This was just something I started doing because I needed it.
12:06	parth_m	Any suggestion? On names. Possibly any implementation improvements?
12:13	Chouser	mac__: I'm not sure what's going on. A quick hack to generate protected finalize methods still doesn't generate any output.
12:16	Chouser	oh, got it.
12:16	mac__	Solved it :) Did you put any argument in your clojure finalize method? I didn't at first and it will get sent the java object... so it worked for me now that I added an argument to it
12:16	Chouser	right :-)
12:16	Chouser	(defn Hello-finalize [this] (println "finalize"))
12:16	mac__	yep
12:17	Chouser	without the arg it was throwing an exception which is of course ignored in finalize.
12:17	mac__	Exactly. So in conclusion, it works with :methods if you just do it right ;)
12:17	Chouser	:-)
12:20	mac__	Well thanks for the help anyway, might have just given up and done it in java if I didn't have someone to discuss it with.
12:22	Chouser	never give up and do it in java. ;-)
12:24	mac__	That's a good quote
12:25	hoeck	rhickey: hi, i made the multimethod version of print, dispatching on class
12:26	hoeck	sorry, i mean hi all :)
12:30	parth_m	Hello :)
12:35	rhickey	hoeck: cool - have you sent in a CA?
12:38	hoeck	rhickey: yes, about 2 weeks ago
12:38	rhickey	hoeck: ok - I've been on vacation and haven't checked the box since returning
12:40	hoeck	rhickey: ah, should i email you the file or add it to clojure-contrib or the like?
12:41	rhickey	you can put it as an attachment in a message to the group
12:43	hoeck	rhickey: okay
12:45	lisppaste8	parth_m_ pasted "field-read/field-write" at http://paste.lisp.org/display/65959
12:46	parth_m	rhickey: any suggestions on the names ... I am quite bad with names. I have pasted the latest version.
13:08	parth_m	rhickey: Got to go. Please mail any suggestions you might have on the group and I can roll them in. Good day!
13:29	mac__	I need help with gen-class again.. How do I define a constructor that has a different signature from the superclass (Object in my case). Is this not correct if I want a constructor that takes an Object as it's argument? :constructors {[Object] []} I'm getting this exception when trying to instantiate it: java.lang.UnsupportedOperationException: nth not supported on this type: PersistentHashMap
13:32	mac__	hmm might be something else
13:37	mac__	yep, I made a mistake with the return value from init.
14:00	notyouravgjoel	java -cp jline-0.9.91.jar:clojure.jar jline.ConsoleRunner clojure.lang.Repl << will this still work with the newest clojure?
14:01	Chouser	I haven't used jline, but I can't think of any recent changes that would break that if it used to work.
14:02	notyouravgjoel	k; getting started has it listed, but it isnt working
14:04	Chouser	hm. you're getting errors?
14:05	notyouravgjoel	'Exception in thread "main" java.lang.NoClassDefFoundError: jline/ConsoleRunner
14:05	notyouravgjoel	java.lang.NoClassDefFoundError: jline/ConsoleRunner
14:06	Chouser	Does it work without the jline stuff?
14:07	notyouravgjoel	yep
14:08	notyouravgjoel	ah, just figured it out
14:08	notyouravgjoel	sorry, was rather stupid on my part
14:11	Chouser	np.
14:23	rhickey	lisppaste8: url
14:23	lisppaste8	To use the lisppaste bot, visit http://paste.lisp.org/new/clojure and enter your paste.
14:24	lisppaste8	rhickey pasted "Multi-key get/assoc" at http://paste.lisp.org/display/65964
14:25	cemerick	rhickey: ooh, that looks nice
14:27	rhickey	cemerick: the other option is to force the 'multi-key' into a vector
14:28	cemerick	it seems quite pleasant the way it is
14:29	rhickey	I agree, Parth's not here to explain his use-case, but this method is much more efficient than his whatever the interface
14:31	Chouser	I wonder if it would be useful to be able to provide defaults to allow building out the initial structure.
14:31	Chouser	Parth may be content with (apply mk-assoc nx v)
14:35	lisppaste8	rhickey annotated #65964 with "mk-assoc that builds" at http://paste.lisp.org/display/65964#1
14:39	Chouser	that's fine for maps
14:39	Chouser	foo=> (mk-assoc {:a [:b0 :b1 {:c 9}]} :a 2 :c 7)
14:39	Chouser	{:a [:b0 :b1 {:c 7}]}
14:39	Chouser	foo=> (mk-assoc {} :a 2 :c 7)
14:39	Chouser	{:a {2 {:c 7}}}
14:41	rhickey	Chouser: what do you want the second one to do?
14:42	Chouser	I'd have to specify that I want to build a vector at that level instead of a map
14:42	Chouser	I haven't thought of a good API
14:42	Chouser	(mk-assoc {} :a '(2 []) :c 7)
14:42	Chouser	ick
14:42	rhickey	I don't think it's a real problem
14:42	Chouser	ok