#clojure – 2008-08-26

2008-08-26

08:17	dabd	hi anyone knows if the video from the clojure and the semantic web meeting will be available?
09:42	rhickey_	dabd: I've been on vacation and haven't had a chance to look at the semantic web video yet
09:44	dabd	rhickey: ok thanks
14:36	mbeau	I'd like to simulate keyword parameters in a function; it seems that this is not directly supported in clojure so it seems that I should be able to easily do this myself
14:37	mbeau	one idea I had was to take the 'rest' parameter list and turn that into a hashmap - that gives me pretty much what I'm looking for
14:37	mbeau	how can I take a list and convert that into a map?
14:37	mbeau	seems like it should be obvious, but I can't figure it out
14:38	erochester	try this: (apply hash-map [:a 1 :b 2])
14:39	mbeau	ok, thanks
14:40	erochester	no problem.
15:33	cemerick	...it's too bad that stuff like map, reduce, etc., aren't functions on ASeq. That'd make Java a whole lot more sane.
15:35	rhickey	cemerick: if Java had closures
15:35	rhickey	that implemented IFn :)
15:36	rhickey	Var map = RT.var("clojure","map");
15:36	rhickey	map.invoke(...);
15:36	rhickey	Clojure is definitely callable from Java
15:37	cemerick	yeah, I know I could do that -- I don't think this particular chunk of Java will be sticking around for long though, so I don't want to prod at it too much.
15:37	cemerick	I'd be happy enough to drop in anonymous AFn subclasses, really.
15:37	cemerick	Don't take the suggestion seriously, though -- just a side-effect of having a sizable Java codebase that has clojure stuff starting to filter in.
15:44	lisppaste8	Chouser pasted "starved transaction?" at http://paste.lisp.org/display/65898
15:45	Chouser	when the transaction in slow completes, you see ":show-returned" and then the function dead completes.
15:45	rhickey	ok
15:46	Chouser	However, as long as the fast transactions keep committing, slow keeps resetting and never seems to finish.
15:46	rhickey	right
15:46	rhickey	what is your expectation?
15:46	Chouser	I was expecting some Magic to how up and fix my problem. :-)
15:46	Chouser	sorry, brb...
15:46	rhickey	I see
15:48	blackdog	has anyone been thinking of a ruby gem like package system?
15:51	cemerick	rhickey: why is Delay no longer a fn? (sorry if I missed some prior discussion)
15:52	rhickey	cemerick: there is now a proper force, which is a no op if the arg isn't a delay
15:53	Chouser	I'm back. I was under the impression that at some point new transactions would be held up to allow a very old transaction to complete.
15:55	cemerick	rhickey: yeah, I saw that, and that's fine, but there are definite advantages to being able to treat delays as fn's
15:56	rhickey	Chouser: no, only if there is a live conflict will an older transaction 'win', your slow doesn't acquire anything until after its delay - try putting (ensure c) before sleep
15:58	rhickey	cemerick: I'm on the fence about how rich to make delays
15:58	rhickey	when would you use them as fns?
15:59	cemerick	anytime I have a map containing fns for values; some functions may be very involved, so caching their results transparently is super-handy
16:00	cemerick	(handy enough that I'd use my own fn-delay impl to get the same results, FWIW)
16:01	rhickey	cemerick: why wouldn't you just delay the invocation?
16:02	rhickey	then force = call
16:03	rhickey	delays as fns was always just sugar
16:03	cemerick	because I want to wrap up fns into a map and toss it over the wall to a caller -- I don't have control over when those functions are pulled out and invoked
16:04	rhickey	but the caller doesn't pass args
16:04	rhickey	so they could call force instead of invoke
16:04	rhickey	or I'm confused about the use case
16:05	cemerick	sure, but then they'd have to know about delays as well as fns -- I'd end up littering (if (delay? fn) (force fn) (fn)) all over the place.
16:05	rhickey	no - force does that for you
16:06	cemerick	so you're suggesting making all of the values in such maps delays?
16:06	rhickey	no, force on non-delay just works
16:06	rhickey	user=> (force 42)
16:06	rhickey	42
16:07	rhickey	From Java, Delay.force(42)
16:07	cemerick	oh, I see; my initial reaction to that is "yuck"
16:09	cemerick	I guess I don't see the advantage of having a split between fns and delays to begin with; using (force fn) all the time instead of simply (fn) isn't obviously better
16:09	rhickey	yes it is - because you can't say (42)
16:10	rhickey	now you can have mixed delayed and non-delayed values in a mixed environment and handle uniformly
16:14	rhickey	but (d) was just sugar for (get d), no difference in behavior
16:17	cemerick	the problem with that is that forcing a fn returns the fn, not the value of invoking the fn; a mixed environment of delays and fns doesn't yield the results I'd expect in that case (not that those results are incorrect, just not what I'd expect). I view delays as essentially transparent wrappers around fns, so treating them as values themselves trips me up.
16:17	cemerick	I suppose my perspective could be fundamentally faulty shrug
16:19	rhickey	delays delay expressions, the fact that they are implemented using closures is an implementation detail - you could always have said (delay 42) - where's the fn in that? :)
16:20	cemerick	touch� :-) Of course, any (delay x), where x is already a strict value, doesn't really have any application (that I can think of).
16:22	rhickey	even when not a constant, it's still an expression that's delayed, not a fn: (delay (+ 21 21))
16:23	cemerick	sure
16:23	rhickey	but without force that detects delays, people did/do have to say (delay 42) sometimes when creating a data structure with mixed delayed/non-delayed values. Now they don't have to
16:25	cemerick	I'd much rather put the onus on the creator of the data structure, and not the caller.
16:26	rhickey	there is always some onus on the caller, since there isn't a value without force/get/call
16:27	cemerick	surely fn invocation is more fundamental than force/get, certainly in the overwhelmingly common cases?
16:27	rhickey	pretending all the values are fns doesn't change that
16:28	rhickey	and it precludes uniform handling
16:28	rhickey	that's the key point
16:31	cemerick	I guess we have different common use-cases in mind. Doing (force x) and then conditionally invoking the result seems pretty backwards to me, but then I can't think of the last time I put delays and numbers (for example) into the same map or list.
16:32	cemerick	My final point would be: why not permit the old behaviour? Delays-as-fns is sugar, for sure, but makes things a lot simpler (in my use cases, anyway).
16:32	rhickey	> Doing (force x) and then conditionally invoking the result - I don't see when that happens ever. force is the exact same operation as call was, except it works on non-delays too
16:34	rhickey	invoke called get(), force calls get()
16:35	rhickey	iff it's a delay
16:35	cemerick	the problem is this; consider a vector: (def v [(fn [] 5) (delay 5)])
16:36	cemerick	(map force v) => (user.fn__2127@1c2812 5)
16:37	cemerick	if delays are fn's, you can do (map #(%) v) => (5 5)
16:38	cemerick	that's what I mean by conditionally invoking the result -- that fn is its own value which then needs to be invoked after being forced in order to get to the same place
16:40	rhickey	your example has no purpose to using delay, so it's hard to see the point
16:41	rhickey	if the second 5 is some expensive expression, then (def v [5 (delay 5)])
16:42	cemerick	yes, it's watered down to make an example; consider {:root root-node :size (delay (fn [] ....))}, a tree structure with a :size attribute that will walk the tree and return and cache the size of the tree rooted at root-node. You want the value of :size to be a fn, so it can be optionally delayed depending upon the specific costs associated with the size calculation.
16:43	rhickey	then it's not (delay (fn [] ...) but (delay ...)
16:44	cemerick	right, sorry -- the point's the same, though
16:44	rhickey	no it's not, once you have an expression, then fns have nothing to do with it
16:46	rhickey	all delays-as-fns do is superficial, and if you've been wrapping values in fns to make them match delays you'll no onger have to
16:46	rhickey	longer
16:47	rhickey	check the diff on Delay.java - it's not functionally different
16:47	cemerick	yeah, I'm looking at it right now; no longer being an AFn is a significant difference
16:48	rhickey	not in what happens during invoke/force or the nature of delayed values
16:49	cemerick	you can't invoke a delay any more -- that's quite different, implementation detail or not
16:50	cemerick	yeah, I mistyped the (delay (fn []...)) bit -- I meant (delay ...)
16:51	rhickey	the reason I took out delays as fns and delays as IRef is that they do not have the same behavior when the contained _value_ is itself a fn, e.g. invoking a var casts its value to fn and invokes it, but a delay didn't do that
16:56	cemerick	I don't currently use refs much (if at all); I wouldn't think that their requirements are tied to delays, though.
16:56	rhickey	cemerick: delays really aren't functions of anything, not taking args makes that plain
16:57	rhickey	I understand it's a breaking change for you, and I'm sorry about that, but I made delay public to help you along and have subsequently refined its design - I've got to be able to do that between releases
16:58	rhickey	uniform force is plainly a better interface than invocation
16:58	cemerick	oh, no need to apologize, please :-)
17:00	rhickey	but there is a transformation between what you've been doing and the new way, that is easier on the producer side and, let's say different on the consumer side, but strictly more powerful even if you don't currently leverage that
17:06	Chouser	I guess what bothers me about my starved transaction is that I could have a large system that's functioning just fine, but by adding some new code that innocently pokes away too quickly at a ref, I could cause some existing slow transaction to start hanging for long stretches at a time.
17:07	Chouser	I suppose you'd just have to test for that and fix your slow transactions -- split them up, add ensure, or whatever?
17:09	rhickey	Chouser: that's not a problem I can solve for you. There will always be issues with contention that are part of a concurrent design. STM and agents handle the correctness part, but you will still have to deal with transaction granularity, degree of overlap, transaction duration etc.
17:10	rhickey	one general rule is not to have long-running transactions compete with short running transactions for the same refs
17:11	rhickey	(writing)
17:15	rhickey	Chouser: did you try the ensure before sleep?
17:16	Chouser	yep, wokred like a charm
17:16	rhickey	so you do have some tools to work with
17:16	Chouser	yes
17:17	cemerick	rhickey: thanks for clarifying things earlier; I hope I didn't come off as irritated or whatever...
17:18	rhickey	np, I understand it sucks to have something changed from underneath you
17:18	cemerick	nah, it's to be expected when you're running off of HEAD :-D
17:19	cemerick	besides, nothing will change much; I expect I'll create my own DelayFn class (or whatever) to retain the same behaviour for myself. Oh, the joys of interfaces...
17:21	cemerick	also besides, such things are a very, very small price to pay to be able to use something as truly phenomenal as clojure
17:21	rhickey	Chouser: I meant 'can solve for you automatically' before, of course I'm always willing to help you
17:22	Chouser	rhickey: heh, I understood.
21:12	jamii	how does clojure find namespaces when refer is used? do I have to load the file first?
21:13	Chouser	yes, refer only works with names that have already been loaded
21:27	Chouser	(apply concat (partition 1 2 [1 2 3 4 5])) => (1 3 5)
21:27	Chouser	but there's got to be a better way, right?
22:22	arohner	I'm finding myself doing
22:23	arohner	(let [foo (assoc foo key val)]...)
22:23	rhickey	user=> (take-nth 2 [1 2 3 4 5])
22:23	rhickey	(1 3 5)
22:23	arohner	that doesn't feel idiomatic
22:23	arohner	is there a better way to accomplish that?
22:25	rhickey	arohner: what feels wrong - reusing foo?
22:25	arohner	yeah
22:25	rhickey	you could call it bar :)
22:25	arohner	hah
22:25	arohner	I'm immediately recurring on foo
22:25	arohner	so for more context
22:25	arohner	(let [foo (assoc foo key val)] (recur foo))
22:26	arohner	but it still feels slightly weird
22:26	rhickey	the first foo is in a loop?
22:26	arohner	yes
22:27	rhickey	I'd just (recur (assoc foo key val))
22:27	arohner	ok, I guess that makes it idiomatic :-)
22:28	arohner	hmm, maybe that's not the source of my hesitation
22:28	rhickey	but consider reduce
22:28	arohner	ah
22:28	arohner	maybe that's what I'm looking for
22:29	arohner	I didn't think of treating the problem that way, but that might be a better fit
22:30	arohner	thanks
22:32	rhickey	sure
22:36	Chouser	rhickey: take-nth. I feel dumb now.
22:37	rhickey	Chouser: don't worry - I had to look it up myself :)
22:37	Chouser	arohner: and if you're using reduce to build a map, you might like into
22:38	rhickey	yes, if you are not doing any other logic
22:38	arohner	what does conjoined mean in this context?
22:39	rhickey	user=> (conj {} [:a 1])
22:39	rhickey	{:a 1}
22:39	rhickey	user=> (conj {} [:a 1] [:b 2])
22:39	rhickey	{:b 2, :a 1}
22:40	arohner	ah, that conjoined :-)
22:40	rhickey	maps are collections of entries, so you can conjoin an entry
22:41	rhickey	or another map:
22:41	rhickey	user=> (conj {:a 1} {:b 2 :c 3})
22:41	rhickey	{:c 3, :b 2, :a 1}
22:42	rhickey	Clojure is like Lego
22:42	arohner	hah
22:42	arohner	that wouldn't be a bad irc topic
23:00	arohner	are there any functions for working with ratios?
23:01	arohner	i.e. get the numerator, get the denominator, increment the fraction?
23:03	rhickey	(.numerator 22/7)
23:04	rhickey	and .denominator
23:04	arohner	ah, cool
23:05	Chouser	huh. Why don't I see those with (doseq x (.getMethods (class 3/4)) (prn x))
23:06	rhickey	because they are fields
23:06	Chouser	ah
23:06	rhickey	public final
23:06	arohner	Chouser: that is a cool trick
23:07	arohner	and yet again clojure is a better java than java
23:07	arohner	not to say that's only what it's good for!
23:08	Chouser	arohner: one of the tricks I use to give the impression I know things when I don't.