#clojure – 2008-07-24

2008-07-24

05:03	meredydd	Hey - lib.clj question. I seem to be Doing It Wrong wrt unqualified lib names
05:03	meredydd	I'm writing a lib in 'com.foo.bar, in com/foo/bar/bar.clj, and (use com.foo.bar) works just fine
05:03	meredydd	but if, from within that file, I try to (use baz),
05:04	meredydd	although it successfully loads com/foo/bar/baz.clj, I then get an Exception saying "namespace 'baz' not found after loading file:/.../com/foo/bar/baz.clj"
08:22	cemerick	rhickey_: I don't know if you can do anything about the error message for this, but it could really trip someone up if they don't notice the destructure mention in the stack trace (or don't know what destructuring is, for that matter): (let [a 5 c] a)
08:23	rhickey_	agreed, I'll look into providing a better message
09:52	cemerick	how can I emit type hints in a macro, assuming I have a seq of classes, for example?
09:52	cemerick	all of the variations I've tried so far result in reader errors...
09:53	StartsWithK	i know there is answer on google group.. but i can't find it..
09:54	rhickey_	cemerick: you can use with-meta to attach metadata to symbols you emit in the macro, :tag is the type-hint key
09:54	cemerick	rhickey_: I don't want :tag, I want to type-hint args
09:54	cemerick	sorry for not specifying
09:55	rhickey_	could you give an example?
09:55	cemerick	:arglist seems to swallow type hints I put in there
09:56	cemerick	rhickey_: well, I've got a proxy-based defbean working pretty decently; I'd like to type-hint the args to the function it generates so that you can't pass in values that aren't aligned with the return types of the methods defined on the interface being implemented by the proxy.
09:56	StartsWithK	http://is.gd/12vv
09:57	rhickey_	StartsWithK: right, that's what I was talking about
09:57	cemerick	StartsWithK: ah-ha, thanks
09:57	cemerick	it was not at all clear to me that :tag could be applied to args
09:58	rhickey_	cemerick: it can get confusing, the trick is to distinguish metadata on the forms/symbols presented to the compiler and metadata on the data/vars a macro produces
09:59	rhickey_	type-hints are code-as-data just like everything else
09:59	cemerick	rhickey_: yeah, that's totally fine -- I scrubbed the docs pretty well (I think), but :tag isn't mentioned as being applicable to anything other than a fn or a var
10:01	rhickey_	cemerick: that's what I'm trying to clarify. if you read in http://clojure.org/reader under Macro Characters/dispatch (#)
10:01	rhickey_	A shorthand version allows the metadata to be a simple symbol or keyword, in which case it is treated as a single entry map with a key of :tag and a value of the symbol provided, e.g.:
10:01	rhickey_	#^String x is the same as #^{:tag String} x
10:02	rhickey_	so, outside of any interpretive context, #^String x is read as a symbol with the metadata {:tag String}
10:03	rhickey_	macros can emit symbols similarly tagged
10:04	rhickey_	the trick is to look at #^String x as one thing
10:05	rhickey_	and not try to independently emit the metadata part
10:07	rhickey_	I've improved the docs for validators - anyone try them yet?
10:20	rhickey_	Chouser: how so?
10:21	lisppaste8	cemerick pasted "defbean -- arg :tag metadata issues" at http://paste.lisp.org/display/64146
10:22	cemerick	About 2/3 of the way down is where I'm dropping in the arg :tag metadata (`(fn [~@(map #(with-meta % {:tag (% prop-types)}) properties)])
10:22	cemerick	Some quick println debugging shows that the metadata is set on the symbols, but the fn yielded by the macro doesn't complain when provided with non-conforming args.
10:24	Chouser	rhickey_: I've started to use lib.clj a little, and my understanding is that the new code layout might break it.
10:24	rhickey_	type-hints are hints, not prohibitions. They are only used to optimize calls via the hinted objects, so if there are no calls there are no typed usages to fail
10:25	rhickey_	Chouser: ah, I hope Steve is working on that, didn't hear back from him on the group
10:26	cemerick	rhickey_: well, I have an interface (Bounded) that has a "bounds" method defined that returns a Rectangle class. The defbean macro should be returning an fn that will only accept a Rectangle instance as an argument, but it currently accepts anything.
10:26	Chouser	but I only barely use lib, so I should just press ahead...
10:28	rhickey_	cemerick: I think you have expectations of type hints that exceed their current behavior:
10:28	rhickey_	(defn #^String test-hint [#^String s] s)
10:28	rhickey_	(test-hint 42)
10:28	Chouser	rhickey_: typo in boot.clj: whwich
10:31	rhickey_	Chouser: fixed, thanks
10:31	cemerick	rhickey_: So I do. I suppose that particular bit of effort is for naught anyway, since the resulting proxied instance can have different-typed values assoc'ed in anyway.
10:31	StartsWithK	cemerick, i think you'll have to use something (if (instance? MyType a) (do ..) (throw (new Exeception)))
10:32	cemerick	StartsWithK: Yeah. I don't think I'll bother, given the backdoor that assoc provides, anyway.
10:32	meredydd	Hey, is anyone currently working on easier XML-munging for Clojure?
10:32	rhickey_	cemerick: yeah, the normal channels are all Object-based
10:33	cemerick	rhickey_: If I had slowed down to remember that IFn's invoke signatures all use Object, I would have been better off, for sure :-)
10:33	meredydd	I've just run into an XMLish problem, and am currently considering doing something XPath-based
10:34	meredydd	but if there's something Clojure-idiomatic that'll do the same thing already, I'd love to hear it.
10:34	Chouser	meredydd: I have a thing I wrote that I'm using in places where I would otherwise use xpath
10:34	rhickey_	cemerick: well, I could do pre-emptive checkcasts in hinted functions, but usually the checkcasts occur during the body anyway
10:34	meredydd	Chouser: Have you posted it anywhere?
10:34	Chouser	clojure-contrib zip-filter
10:35	Chouser	it builds on top of rhickey's zip.clj
10:35	rhickey_	Chouser: have you sent a CA? I'd like to start picking through contrib for things to add to Clojure proper
10:35	cemerick	rhickey_: I think that would be a good idea in general, simply to speed up failures as much as possible.
10:36	Chouser	I dunno if it'll be right for your use case, but it's pure Clojure and lazy, so it integrates more cleanly than an xpath string would.
10:36	rhickey_	cemerick: I'm trying to remember why I didn't, I think there can currently be cases where the hint is only used on one code path and the other doesn't care
10:37	Chouser	meredydd: probably not sufficiently documented though -- mostly just the examples at the end. please bug me if you have any questions.
10:37	meredydd	okies.
10:37	meredydd	That's looking very much like what I was looking for.
10:39	meredydd	I was going to head up the route of using XPaths as inputs to things working like (map) or (reduce)
10:39	cemerick	rhickey_: Yeah, there's plenty of scenarios where the hint is unused; most of the ones I'm thinking of involve the arg being closed over for a thunk of some kind, and then the type mismatch is delayed indefinitely.
10:39	meredydd	but explicit navigation of the tree with Clojure predicates sounds cool too
10:39	cemerick	I can imagine that there might be a performance hit for universally checkcasting everything that's hinted, though.
10:40	rhickey_	cemerick: that too
10:40	Chouser	meredydd: I haven't done any speed comparisons with xpath -- I suspect zip-filter is significantly slower right now. It hasn't mattered yet for my use cases.
10:41	cemerick	rhickey_: someone might make an argument that correctness is preferable to high-performing failure conditions, though ;-)
10:41	rhickey_	cemerick: type hints are just that, hints, and can be ignored. So they are not intended for the same purpose as type declarations in Java
10:42	meredydd	Chouser: Well, I strongly suspect the available DOM implementations wouldn't take kindly to being thrown around as disposable, immutable bits, so that approach would probably have involved either a nasty mismatch with the DOM bits or writing my own XPath parser (har har)
10:44	rhickey_	cemerick: you can write checks with cast
10:44	Chouser	yeah, that's part of why I headed this direction. Also because when using xpath I've often wanted some function or other that doesn't exist. With zip-filter I can just drop in a #() predicate or whatever.
10:45	cemerick	rhickey_: Yeah. I haven't decided whether I'm going to bother in the case of defbean.
10:51	rhickey_	cemerick: well, the validators are a step in the direction I described yesterday. You can strap a validator fn on any var/ref/agent and it won't allow it to take on a state that fails validation. But validation can do so much more then types - it can check types, values, ranges, the presence/absence of keys, relationships between fields etc.
10:51	rhickey_	than
10:51	Chouser	but at runtime
10:52	Chouser	don't get me wrong -- I think they're cool, and I'm looking forward to slapping one on every ref I make
10:53	Chouser	but much of the allure of types is you get whatever leverl of checking they provide without needing any unit tests
10:55	cemerick	rhickey_: sounds like a good step; is this in svn? Didn't see a post on the group...
10:55	Chouser	he posted here instead.
10:56	cemerick	ah, I missed that
10:56	Chouser	I don't think sourceforge provides svn log feeds, which is unfortunate.
10:56	cemerick	I agree with Chouser; validators are good, but compile-time checking is also good.
10:57	Chouser	I've been ambivalent about static types for while now, and still am.
10:57	cemerick	I've held at r937 -- we're using lib more than a little, so I'm waiting for those changes to shake out.
10:58	Chouser	I think Steve Yegge's comments on static types are good.
10:59	cemerick	I really enjoy working with a properly type-inferred static environment, but it can get very tiring in many circumstances. Optional type declarations (especially when interfacing with Java itself) would be a great middle ground.
11:00	Chouser	http://steve-yegge.blogspot.com/2008/06/rhinos-and-tigers.html and search for Switzerland
11:03	cemerick	Yeah, that's a fair assessment of scala. I can't take Yegge very seriously since most of his arguments appear to be based on hearsay (as this one is), but he's quoting the right hearsay.
11:05	cemerick	The type system isn't really the bad part (although yes, it's extraordinarily complex) -- the bad part are implicits and similar features. They're great, until you take the wrong step, and then nothing makes sense because values are being converted (in a chained fashion, even!) out from under you, and it's not always easy to know why.
11:07	Chouser	maybe I meant more "funny" than "good"
11:10	cemerick	I guess I'm feeling spunky today.
11:11	Chouser	cemerick: I'm not disagreeing with you either. :-)
11:11	cemerick	Chouser: I didn't think so. :-)
11:28	StartsWithK	implicits can be managed in very explicit way.. for now no problems there..
13:28	cemerick	rhickey: regarding the CA: should I just put down all of my identifying nicks, google group logins, and email addresses I might use to submit patches, etc., in the "username" fields of that form?
13:28	rhickey	cemerick: sure
13:29	Chouser	I just put my sourceforge username. I thought that's what the instructions were implying.
13:29	rhickey	cemerick: if you don't have a sourceforge name, get one first
13:29	rhickey	sourceforge and google group are most important, I won't take contributions on IRC or pastes
13:30	cemerick	I hadn't even thought of sf. I might have to make an account.
13:31	rhickey	cemerick: yes, please, I definitely see you in contrib at some point
13:32	rhickey	Chouser: no worries
13:32	cemerick	Q: why sourceforge, anyway?
13:33	rhickey	why did I choose sf for Clojure?
13:34	cemerick	yeah. Not that there are any other obvious options out there.
13:35	rhickey	exactly, I had other projects there, seemed fairly large/stable, generally works - I wish the forums/trackers were useable
13:40	Chouser	nobody was ever fired for going with sourceforge
13:41	cemerick	that's a good way of framing it
13:43	Chouser	I'm joking of course. ...something people used to say about IBM, I believe. Comparing IBM to SourceForge is funny, right?
13:44	cemerick	funny, but accurate insofar as sf is functional, but loved by no one.
13:44	cemerick	it's actually still better than google code, which I really find irritating in various ways.
13:50	rhickey	cemerick: what's irritating about google code (currently not an option for Clojure due to license restrictions)?
13:54	StartsWithK	will my code break in any way if i update to trunk?
13:55	StartsWithK	i mean, is ns/var now ns.var? is my.package now my(ns).package(sub-ns) or are they still just one name?
13:56	StartsWithK	and are java static member now called as Class/member or Class.member?
13:58	rhickey	there is a new convention for namespaces that is used by boot.clj et al but not otherwise enforced, those files have moved, and nothing is different about Java statics. If you use lib there may be some issues
14:00	StartsWithK	will there be in the future a way to refer to namespace inside only one function and not on source file level?
14:02	rhickey	StartsWithK: some sort of (with-refer ...) ?
14:02	StartsWithK	maybe even inside (do) or (loop) blocks
14:02	StartsWithK	something like that
14:03	StartsWithK	i was thinking that inside function that could go like metadata, so it wouldn't increase nesting
14:03	cemerick	rhickey: The bug tracker is not much better than sf's, and the wiki is completely substandard. Given their resources, it's a really poor showing, I think. All of that is secondary to my general feeling that being in the loving embrace of the google ecosystem comes with a culture tax. The fact that there isn't a default spot for a project homepage outside of google code is telling, I think.
14:03	StartsWithK	and macro could expand it later to something like (with-refer ...)
14:03	rhickey	StartsWithK: I don't think so, depends on demand. With aliases, you can give a ns a short nickname
14:03	cemerick	It's exactly the kind of service and experience I'd expect from MS, except with google seasoning instead.
14:04	rhickey	cemerick: interesting
14:05	StartsWithK	in python i import thing only at call site if only one function is using that package, it helps with moving to other files
14:05	cemerick	I'm something of a fogbugz fanboy anyway, so I'll readily complain about almost any bug tracker and wiki. :-P
14:06	rhickey	StartsWithK: import at call site looks like what in Python?
14:07	StartsWithK	def myfunc(): from other.package import some_func; some_func(x, y)
14:08	rhickey	that's easier than (other.package/somefunc x y)?
14:08	StartsWithK	well in general i import whole package inside that one function
14:09	StartsWithK	or more than one function at the time, so i don't need to repeat package prefix on them
14:09	StartsWithK	this could be just my (bad) habbit, but i do similar thing in c++ with import
14:30	blackdog	any examples of gen-and-load-class anywhere?
14:40	cemerick	blackdog: oodles of them on the google group
14:40	blackdog	ah ok
14:40	cemerick	gen-and-save-class has essentially the same usage, so search for that too
14:40	blackdog	ok
16:00	cemerick	It seems that the list of symbols I need to exclude from my (clojure/refer 'clojure) call at the top of a .clj file that is loaded by a saved gen-class varies significantly from class to class. I end up having to discover that list by trial and error -- is there a better way?
16:03	cemerick	or, probably more accurately, what am I doing wrong?
16:05	StartsWithK	when i was genereting class for applet i found same problem
16:05	StartsWithK	so what i did was..
16:06	StartsWithK	in my MyApplet.clj i first did (clojure/in-ns 'myapplet) (clojure/refer 'clojure)
16:06	StartsWithK	(defn init[] ..) (defn paint[] ...) .. and all other methods
16:06	StartsWithK	then in same file (clojure/in-ns 'my.package.MyApplet) (clojure/defn init[] (myapplet/init)
16:07	StartsWithK	so you have in same file namespace of your class.. and before that another namespace that is clean
16:07	StartsWithK	just map all calls from class namespace to that new namespace and you don't need to exclude anything inside it
16:08	StartsWithK	worked for me
16:08	cemerick	ugh. I'm sure that's not how it's supposed to work. FYI, you can just do (clojure/def init myapplet/init)
16:09	cemerick	although, genclass.clj has an example at the bottom with this line: (clojure/refer 'clojure :exclude '(assoc seq count cons empty)) -- so it looks like this is known behaviour
16:11	cemerick	my current exclude list is up to '(seq empty assoc count cons compare); a prior gen-class namespace only needed '(empty cons)
16:12	StartsWithK	http://pastebin.com/d1dd2f7c0
16:12	StartsWithK	this way you don't have to exclude anything
16:15	cemerick	StartsWithK: Yeah, I see that that routes around the problem, thanks.
16:16	StartsWithK	but you are right, i should use def, no need for that extra function call in there
16:17	Chouser	cemerick: you're doing it the best that you can right now. It's a known problem, and rhickey has proposed a solution.
16:18	StartsWithK	also i think genclass will not work if you try to generate class that is not inside any package..
16:18	cemerick	Chouser: Thanks. Good to know. This was the first gen-class that has resulted in a different exclude list, which is what really surprised me.
16:19	cemerick	well, default-package classes indicate failure, anyway :-P
16:21	cemerick	that workaround doesn't work when the classname is based on the current namespace
16:34	meredydd	(Chouser)++
16:34	Chouser	hm, does that mean zip-filter is actually working?
16:34	meredydd	I'm just investigating and desgining atm. It looks beautiful in the examples, and I'm just putting together what I want to do with it.
16:35	Chouser	great. well, keep me updated. It's pretty immature -- I've only used it in a couple apps.
16:37	meredydd	Hmm. Have you tracked rhickey's new code layout?
16:37	meredydd	cause you appear to be importing 'zip rather than 'clojure.zip
16:38	Chouser	the most complicated thing about it, and the design feature I'm least confident about, is how it chooses to automatically go "down" after some filter expressions
16:38	Chouser	ah, no I haven't. let me fix that.
16:39	meredydd	(in fact, you're using it explicitly as zip/symname, which will get very cumbersome very quickly)
16:39	meredydd	(unfortunately, it would appear there's a symbol conflict for "replace" between 'clojure and 'clojure.zip, so you can't (refer) it without explicitly excluding one or the other
16:40	Chouser	yeah.
16:40	Chouser	neither of those are under my control of course.
16:41	meredydd	uh-huh. That was more of a prod at rhickey, although I'm trying to leave him alone at the moment :0
16:41	meredydd	*:)
16:44	rhickey	meredydd: zip was designed to be used with the namespace prefix, the names are pretty generic and likely to conflict with something
16:44	Chouser	meredydd: ok, aliases added so it should work again.
16:45	meredydd	Aha! (alias) to the rescue.
16:45	Chouser	and I just accidentally fiddled with duck-streams.clj
16:45	meredydd	rhickey: I thought you'd just condemned everyone to typing "clojure.zip/foo" forever more
16:49	Chouser	rhickey: have you given more thought to name mangling for gen-and-*-class namespaces? Is a leading . your favorite?
16:52	rhickey	Chouser: it's on my todo, I'm a bit afraid of leading '.' being too clever, as it will break:
16:52	rhickey	user=> (macroexpand '(.foo bar))
16:52	rhickey	(. bar foo)
16:52	Chouser	yep
16:54	rhickey	not limited to classes
16:54	Chouser	that's about two notches over my head.
16:55	Chouser	I haven't needed multimethods yet, as far as I know.
16:55	rhickey	Chouser: the test for dispatch will no longer be =, but isa?, so you could match any collection with a dispatch value of Collection
16:56	rhickey	but it would be an arbitrary function of the arguments, as it is now, and...
16:57	rhickey	you can define new relationships for symbols/keywords: (derive :square :rectangle)
16:57	Chouser	so right now the dispatch function takes all args and returns a value that is compared using = to the available methods, right? So you're chaning the = to "isa?"
16:57	Chouser	heh. wow.
16:58	rhickey	and then we all are free of the fixed type system
16:58	rhickey	invent as many taxonomies as you want, place them in any attributes you want
16:58	rhickey	hierarchy a la carte
16:59	Chouser	"derive" modifies the same single global state that "isa?" queries?
16:59	rhickey	yes
17:00	rhickey	I imagine more use of symbols here, since the namespaces partition the 'world'
17:01	Chouser	would you integrate Java's view of the type heirarchy automatically?
17:01	rhickey	yes
17:01	Chouser	fascinating.
17:20	Chouser	well, when I need a multimethod I'll be sure to invent my own new taxonomy, just because I can.
17:20	rhickey	Chouser: consider print, currently a nasty, inextensible multi-way conditional
17:21	Chouser	ah! execllent.
17:21	Chouser	and too general to be solved with functions in meta-data like clojure.zip
17:21	rhickey	right
17:21	rhickey	an open world
17:48	meredydd	Chouser: Okay, I have what is either a suggestion for zip-filter-xml, or a request for help because I don't understand it.
17:48	Chouser	ok
17:49	meredydd	Would it be useful to add a filter expression which would evaluate a predicate, then return that node's children iff that predicate returned true?
17:51	meredydd	because currently, as I understand it, unless you explicitly do that with the zip/ functions, xml-> will carry on looking at everyone
17:51	Chouser	the problem is that both descending to the children and not descending to the children are common, depending.
17:51	meredydd	(Looks like tag= does this)
17:51	meredydd	yeah, which is why I'm thinking of offering some way of choosing, while reusing a predicate
17:52	Chouser	I want :entry :author :name to descend for each tag, but if I toss in a [] or (attr=) I probably don't want to descend.
17:53	meredydd	something like: (xml-> my-root-loc (children-when <some-pred>) <some-other-pred>) etc
17:53	meredydd	with the default handling for :tag-names being (children-when (tag= %))
17:53	Chouser	so I've got this "auto" stuff sprinkled through, which allows each predicate to indicate if wants to descend or not.
17:54	meredydd	hmm...is that what allows the descent? I thought you explicitly needed to return the result of (children-auto loc) to descend...
17:55	meredydd	(or am I talking about the same thing as you?)
17:55	Chouser	I may have to go away in a minute here, but I admit the existing solution is a bit complex.
17:56	Chouser	if you look at the examples, generally what you expect to happen (at least coming from xpath-land) will happen by default.
17:56	meredydd	that's fine. I'm only starting to properly understand it, so take what I say with a pinch of salt.
17:56	Chouser	if you want to descend what the default is not to, I think all you need to do is add a call to children after it.
17:57	Chouser	I'm very much open to a better solution, but I want to be competitive in brevity.
17:58	Chouser	did you write your own predicate that's not behaving like you want?
17:59	meredydd	Chouser: Mostly messing around so far. I'm going to go use this in anger for a couple of days before I say anything solid
17:59	meredydd	Thanks for the help.
18:00	Chouser	"use this in anger"?
18:00	slava	rhickey: is your multiple dispatch amneable to static analysis at all?
18:01	meredydd	Chouser: I have some XML documents I want to grovel through for this project. So I'll go see what I actually need, and what's solving imaginary problems.
18:02	Chouser	very good.
18:04	rhickey	slava: not really, there's no basis, being dynamically typed, and having the dispatch function be an arbitrary function of the arguments, but the whole point of it is runtime polymorphism
18:05	slava	rhickey: well self and hotspot did pretty well given that they're essentially dynamically typed :)
18:06	rhickey	right, I just have the self and PCL papers right in front of me. It's more like the PCL case, though, but same solution - caches
18:07	slava	rhickey: i'm a subscriber to the theory that most real-world dynamic code doesn't use the full generality available all of the time -- so I'm always interested in research where the goal is to extract static info from dynamic code
18:08	slava	PCL has some interesting tricks
18:08	slava	for example, if you have a generic word with a method on (shape shape), the first time you call it with, say, (rectangle circle) it will compile a specialized version of that method, which may result in other methods being inlined
18:10	rhickey	that would be possible for me, but I'm going with simpler cache maps in the first go-round
18:10	slava	anyway, I'll back to work...
19:08	meredydd	Okay, I'm sure this must be documented somewhere, but I can't find it. How does one access the "state" returned by the :init function as specified in gen-and-*-class?
19:09	meredydd	The method fns have a "this" parameter, but I would assume this to be an object reference to the class itself. Is there some magic (get-clojure-state-from-object-ref) fn or method I'm missing?
19:20	Chouser	you can specify a name using :state
19:21	Chouser	and then refer to it by that name