#clojure – 2008-04-15

2008-04-15

00:23	abrooks	Chouser, frodwith: -> turns (-> x c b a) into (a (b (c x))). x is placed into a list if it's not a list already.
03:56	vincenz	What does . do?
04:43	vincenz	Dumb question, how do I clear the repl variables if I'm doing interactive programming with SLIME?
04:58	vincenz	clojure is great )
04:58	vincenz	:)
05:18	cgrand	. is a pecial form for java interop
05:19	cgrand	java: myObject.myMethod(arg1, arg2)
05:19	cgrand	clojure: (. myObject myMethod arg1 arg2)
05:27	vincenz	Oh I thought the syntax was
05:27	vincenz	(. myObject (myMethod arg arg2))
05:27	vincenz	that's what I've been using, since it complained n (. myObject method)
05:50	cgrand	It depends which version you are using
05:51	cgrand	see http://groups.google.com/group/clojure/browse_thread/thread/dd64bd49191f8062/9ef373abfb6e5b57
05:52	vincenz	Thank you
05:52	vincenz	Btw, does the designer of clojure ever come in here?
05:55	cgrand	He connects to this channel every day but right now it's a bit early.
05:56	vincenz	Alright thank you, I've watched the two movies, one one clojure and concurrency and one on clojure squences. Ihaven't dug in yet into actually using it but overall it gave me a good feel for the language, and I agreed with most of his design decisions.
05:57	cgrand	On which decisions do you disagree?
05:57	vincenz	ne
05:57	vincenz	Wel more like the lack of one
05:58	vincenz	Anyways, in regards to sequences, I very much agree with the interface he chose for, and in my own research I have come to conclusions about a similar interace
05:58	vincenz	What I feel is missing...
05:58	vincenz	Hmm
05:58	vincenz	Do you mind if I delay this conversation? colleagues are asking me for lunch
05:59	cgrand	(no, pb we seems to be in the same timezone... lunch time!)
05:59	vincenz	Ttyl
06:58	vincenz	re
07:26	vincenz	rhickey: Hi
07:27	vincenz	rhickey: You're the author of clojure?
07:27	rhickey	yes
07:27	vincenz	I saw your two presentations on the concepts behind it and must admit I'm rather interested, and wanted to throw an idea at you.
07:28	vincenz	(The one on clojure and concurrency, and the one on sequences)
07:28	rhickey	ok
07:30	vincenz	If I were to analogize clojure with an existing lisp, I would look at scheme. However scheme has something which clojure does not have: continuations. There are several types of continuations: the old noes, then the direct ones (shift/reset). I doubt the first would be well-behaved in the presence of the STM-stuff you have. The secon might be, but only limitied to some contexts. However, there is a third type coof continuation, 'sub-con
07:32	rhickey	That did get cut off - seems like a good question for the google group
07:32	vincenz	Alright, I will write a post tonight then. I am not certain how feasible it would be on the JVM.
07:33	vincenz	But it would be an interesting excercise to find out how this meshes with the concurrency model of clojure.
07:33	vincenz	Is there some white paper on the core primitives?
07:33	rhickey	Just the web site
07:33	vincenz	Ok
07:40	vincenz	Just so I get it correctly: functions passed to agents enter into the agent's thread, and threads block on sync-blocks until such sync-blocks succeced, and only then continue with any possible code that may fall after it?
07:43	rhickey	Not quite. Each agent doesn't have a thread, but actions sent to agents are asynchronous and will happen on 'another' thread eventually, i.e. send returns right away. sync is synchronous, but not necessarily blocking at all - but a sync block may be run more than once until it succeeds
07:46	vincenz	Just like STM in Haskell :)
07:46	rhickey	sort of, there is no or-else
07:47	vincenz	I presume side-effects are delayed just as agent-calls are?
07:47	vincenz	(E.g. IO)
07:47	rhickey	the only side effects you can use are agent sends
07:47	vincenz	What about 'println'
07:47	rhickey	but that is by convention - I can't control Java
07:49	vincenz	Actually, it would be interesting to make an IO agent. That way IO happens similarly to how all other agent stuff works. Maybe you could bottle the java-stuff in that IO agent as well. Sort of like an IO monad, except in this case it's a separate thread.
07:49	rhickey	I don't like monads
07:49	vincenz	You wouldn't have monads
07:49	rhickey	And not all Java stuff has side-effects
07:49	vincenz	But that way you wouldn't have something happen twice in case a sync-block is rerun.
07:50	rhickey	People can use agents for that, but I'm not going to automate since I can't determine when the Java call would need that isolation
07:50	vincenz	E.g. (dosync (print "Hello") (readsomevariable) (writesomevariable))
07:50	rhickey	(dosync (send an-agent print "Hello") (readsomevariable) (writesomevariable))
07:51	rhickey	DIY
07:51	vincenz	Fair enough :)
07:51	vincenz	I wasn't here to critique that choice, just an interesting topic to talk about :)
07:52	rhickey	Java is a hole I'm not going to plug - instead I've focused on giving safe, correct, efficient alternatives
07:52	vincenz	Well I wasn't targetting Java specifically, more the IO stuff, since that's well defined as a library as is. But indeed, a DIY approach there would suffice.
07:53	vincenz	Besides, the JAva-side would probably make coding some stuff really painful if you needed data to come back.
07:54	rhickey	yes, transactional IO is not straightforward in any case
07:54	rhickey	especially the I
07:55	rhickey	input/transaction/output will be a common pattern
07:55	vincenz	Either way, I would block your thread and thus block your sync pattern.
07:55	vincenz	So it's probably a poor choice to do I insidea transaction
07:55	rhickey	blocking in sync is a no-go
07:55	vincenz	Continuations and transcations are also an open problem
07:56	vincenz	I'll try to reread your exact threading model tonight as well as that paper I have in mind
07:56	vincenz	and try to figure out an early possible solution, open for discussion
07:56	vincenz	Though I wonder if this would even be possible on the JVM :\|
08:00	vincenz	Finally, I must say that your abstraction for sequences is really elegant. I've -vaguely- envisioned something similar (I work on optimizations of data-types, concretely sequences), but this was a very nice crystalization :)
08:01	rhickey	It's just cons/car/cdr freed from cons cells :)
08:01	vincenz	Sure, but you
08:01	vincenz	'
08:02	vincenz	Whoops, but you need a persistent model of a vector :)
08:02	rhickey	yes, extending to vectprs and maps is probably Clojure's contribution, though I don't claim novelty for any of Clojure
08:04	vincenz	And in the end, this is the model you need
08:04	vincenz	It's a zipper like model
08:04	vincenz	Iterator models are fundamentally broken
08:05	vincenz	(And are completely useless, as soon as you alter your sequence through one of the iterators, you invalidate the other ones..)
08:06	vincenz	And working in clojure almost makes me want to work in java again, as now you acn use emacs + clojure to do online coding + introspection quite easily :)
08:06	vincenz	(defn get-methods [x] (. (. x (getClass)) (getMethods))))
08:07	rhickey	(.. x getClass getMethods)
08:08	vincenz	Thanks
08:08	vincenz	But the fundamentals as explained in your videos made a lot of sense.
08:08	rhickey	good
08:08	vincenz	Then again, as you said it "All the haskellers in the audience should be nodding"
08:09	rhickey	:)
08:09	vincenz	you mean (.. x (getClass) (getMethods)) ?
08:10	rhickey	They all work but what I showed you is possible with the latest syntax
08:10	vincenz	Ah, I guess the version I have doesn't have that yet.
08:10	vincenz	I tend to get 'No matching field found' exceptions when I don't use the parens.
08:10	rhickey	ok, not the latest
08:11	jteo	the perks of being the creator.
08:11	rhickey	No, it's up on SVN
08:11	rhickey	I share :)
08:11	jteo	technically, you always get first dibs rhickey.
08:11	jteo	;)
08:11	rhickey	true
08:12	vincenz	How hard was it ot make the backend for clojjure? Say that continuations won't work on JVM, how hard would it be to to make an interpreter on some other language to test out the combination of transcations with continuations?
08:17	rhickey	I'm not sure I believe in continuations ;)
08:18	vincenz	They can be quite powerful when properly used.
08:18	vincenz	But perhaps you're thinking of traditional continuations, not direct-style ones.
08:18	vincenz	?
08:20	rhickey	the future-of-the-computation notion is bound to a notion of programs as calculations that doesn't resonate with me, and the implementation difficulties don't speak well for their practicality.
08:20	vincenz	Right, you're thinking of traitional ones
08:21	vincenz	There are other types of continuations, namely direct-style/functional ones, which are far more interesting imho
08:21	rhickey	paper?
08:21	vincenz	They do not only have a starting point, but an ending point as well, and you only reify the path between those two points
08:21	vincenz	sec
08:24	vincenz	http://www.cs.rutgers.edu/~ccshan/recur/recur.pdf
08:24	vincenz	This is my favourite paper on the subject
08:25	vincenz	and it seems I abused dterminolgy, the proper terminology is "delimited continuations"
08:26	rhickey	will read, thanks
08:34	vincenz	I know that you can't run 'alter' outside of a sync body, but I made an alter statement escape by putting it inside a function. I get a very odd error when I run this
08:34	vincenz	(let ([dum (dosync (fn [](alter foobar (fn [x] (+ x 2)))))]) (dum))
08:34	vincenz	foobar is simply (def foobar (ref 1))
08:35	vincenz	http://rafb.net/p/p8ehT552.html
08:36	vincenz	nm
08:37	vincenz	Wrong let syntax, disregard, please.
09:55	vincenz	rhickey: Let me know when you care to discuss the paper :)
09:56	rhickey	I won't get to read it until later on
09:57	vincenz	rhickey: No rush :)
12:09	Modius	Just wondering - and I don't know if Clojure would have to "inherit" this from the JRM or from how it interacts with it - does Clojure tend to make a reference eligible for garbage collection the last time it is referenced, or when it leaves scope? I have seen GCed languages do it both ways (C# does, Lispworks does, SBCL doesn't).
12:51	Modius	I fiddled around enough to answer my question - but now have one - if (make-huge) makes a new data structure of size such that only one fits in RAM, (let [x (make-huge)] (let [y (make-huge)] (+ 1 2))) <-- This runs out of RAM - would static analysis in Closure allow this sort of code to work, or would it be some JVM limitation? What would be the pros and cons of getting something like this working?
12:54	Chouser	what was the answer to your first question?
12:54	Chouser	I imagine it might depend on which JVM you use.
12:55	jteo	i presume that since closure compiles to bytecode, it would follow the rules of the GC in the jvm you use.
12:56	Modius	The answer was, the language is retaining the reference for the full scope.
12:57	Modius	The behavior may be JVM related, or may be the language - both would be required to release x and/or y inside-scope.
12:57	Modius	It would take code analysis to know that x and y won't be used again.
12:58	Modius	If a desirable behavior, this could be taken care of via static analysis. What would be harder would be dealing with the case of calls - i.e. having the caller keep the reference alive. If this is inherent to Java calling, this early release may not be possible.
13:00	Chouser	I'm no language designer, but it may be impossible. Any function name can be made to point to a new definition at any time.
13:01	Chouser	well... I guess if you have a lexical variable (like x above), you can see at compile time that it's not used or passed to any other funcs, so it could be freed.
13:01	Chouser	But if it's passed to function f, even if f currently doesn't use it a later re-definition of f might.
13:03	Modius	It's possible as I've used environments that do it. Has nothing to do with function definition but how parameters are evaluated stored and handled at point of call.
13:04	Modius	(some-function (make-huge)) <-- "huge" will be made at point of call, and some-functoin will take it. To have it be eligible for collection, nothing about the caller can keep a reference to the "huge" after the call is made (or during).
13:55	jgracin	rhickey: thanks for elaborate explanation on my post regarding thread starvation.
13:56	vincenz	jgracin: link?
13:57	jgracin	http://groups.google.com/group/clojure/browse_thread/thread/960b717c4d5b95ad
13:58	rhickey	jgracin: sure, it'll take a while before everyone gets comfortable with the Clojure way of looking at those issues
14:00	rhickey	Modius: Clojure does do releasing of locals on tail-calls, but no mid-function analysis
14:03	Modius	rhickey: Thanks! I'm interested to know if the caller releases its reference to a function, and if the behavior is inherently dictated by JVM to preclude if not.
14:03	Modius	(Or by Java calling conventions)
14:04	rhickey	no help from Java/JVM, there is no TCO, so the references are still on the stack. I manually set them to null before the tail-call.
14:13	Modius	I meant on "normal" calls - function to function
14:13	Modius	(somefunction (make-huge)) <-- Does my call maintain a reference as well, or does somefunction have access to all of the references? (Which it could, theoretically, null out)?
14:15	rhickey	all references are live until you make a tail call, no mid-function releasing
14:16	Modius	I'm talking about non-recursive cases now.
14:16	Modius	The normal call to a normal function.
14:16	Modius	(Thanks for the answers btw)
14:17	rhickey	tail has to do with the position of the call, not its recursiveness. a tail call is a call whose return value is the return value of the function that called it
14:18	rhickey	(defn foo [] (bar) (baz)) - baz is a tail call
14:24	Modius	Thanks for helping me out with terminology. (defn foo [] (bar (make-huge)) (baz)) <-- when is the "huge" eligible for collection here - and to what degree is THAT dictated by Java calling?
14:26	rhickey	the stack for the call to bar should be unwound on return from bar, and thus huge eligible for collection then, no help needed from me.
14:27	rhickey	But in this case, I release bar before calling baz:
14:27	rhickey	(defn foo [] (let [bar (make-huge)] (baz)))
15:27	MarkJP	rhickey: I swapped a netbeans static java framework class with a clojure generated java wrapper that forwards calls to a clojure proxy....
15:28	MarkJP	object gets created on startup as it did before and the methods are being forwarded to the clojure proxy at runtime...
15:29	MarkJP	I am getting an assertion in one of the proxied methods about not being in an AWT thread
15:29	MarkJP	Assertion failed. WindowsAPI is required to be called from AWT thread only
15:30	MarkJP	I should still be on the AWT thread as the wrapper is being triggered just the same as it was before
15:55	ericthorsen	rich: If a clojure proxy is created on 1 thread and later called from another thread there _should_ not be a problem there yes?
16:20	rhickey	eric: should be no problem
16:21	rhickey	MarkJP: how do you ensure it gets called on the AWT thread?
16:33	MarkJP	rhickey: If I replace the forwarding call with the original java code, it works
17:31	ericthor	rich: The threading issue was related to a override of a protected method needed at the java class level...guess we will have to look at that as well.
17:32	rhickey	you can override protected methods in proxies
17:32	MarkJP	how do I represent the backslash character in clojure
17:32	vincenz	rhickey: thanks for reply on list (I'm Christophe Poucet)
17:33	Chouser	MarkJP: \\
17:33	ericthor	I does bring up a question: What happens if you have a protected method in your proxy call? Does the clojure compiler cough on that?
17:33	rhickey	you can't access protected members in a proxy
17:34	ericthor	i know...but you can add additional clojure functions not overridden yes?
17:34	Chouser	MarkJP: that's either in a string literal, or a literal directly in the code: (print (str "hi" \\ "there\\Mark"))
17:34	MarkJP	cool thx
17:34	rhickey	eric: no, you can only implement/override methods from superclass/interfaces
17:35	MarkJP	and forwardslash is \/
17:35	MarkJP	?
17:35	ericthor	remember, I'm generating the java class source from the clojure code...I was wondering if I could take advantage of that to have the static java class override the protected member and forward the call to a clojure call.
17:38	rhickey	if you are generating a Java shim class you can define a public method that exposes the protected member, then you can use that in Clojure
17:39	ericthor	ok...just wondering what options were open to me there.
17:40	rhickey	protected member exposing could be a good feature for he stub generator
17:41	Chouser	MarkJP: \/ or just "/"
17:44	ericthor	agreed
17:45	MarkJP	ericthor: there where 2 problems, the protected method, and overriding a method I shouldn't have
17:51	ericthor	mark: ok
17:51	ericthor	...and oops
18:04	vincenz	rhickey: how hard was it to make STM?
18:22	rhickey	vincenz: hard