2009-07-27
| 00:01 | mebaran151 | are the hashCodes for clojure collections consistent between JVM runs? |
| 04:14 | rdsr | Hi all, I have a problem with which I'm struggling since morning. First off here's the code |
| 04:14 | rdsr | (ns maxtemperature |
| 04:14 | rdsr | (:gen-class) |
| 04:14 | rdsr | (:import [org.apache.hadoop.io IntWritable Text] |
| 04:14 | rdsr | [org.apache.hadoop.mapred |
| 04:14 | rdsr | JobConf JobClient |
| 04:14 | rdsr | FileInputFormat FileOutputFormat |
| 04:14 | rdsr | Mapper Reducer MapReduceBase]) |
| 04:14 | rdsr | (:use [clojure.contrib.str-utils :only (re-split)])) |
| 04:14 | rdsr | |
| 04:14 | rdsr | (gen-class |
| 04:14 | rdsr | :name mapper |
| 04:14 | rdsr | :extends org.apache.hadoop.mapred.MapReduceBase |
| 04:14 | rdsr | :implements [org.apache.hadoop.mapred.Mapper] |
| 04:14 | rdsr | :prefix "m-") |
| 04:14 | rdsr | |
| 04:15 | rdsr | (defn m-map [this key value output-collector reporter] |
| 04:15 | rdsr | (let [line (str value) |
| 04:15 | rdsr | year (subs line 15 19) |
| 04:15 | rdsr | air-temperature (if (= (nth line 87) \+) |
| 04:15 | rdsr | (Integer/parseInt (subs line 88 92)) |
| 04:15 | rdsr | (Integer/parseInt (subs line 87 92))) |
| 04:15 | rdsr | quality (subs line 92 93)] |
| 04:15 | rdsr | (if (and (not (= air-temperature 9999)) |
| 04:15 | rdsr | (re-matches quality "[01459]")) |
| 04:15 | rdsr | (.collect output-collector (Text. year) (IntWritable. air-temperature))))) |
| 04:15 | jdz | lisppaste: url |
| 04:15 | lisppaste8 | To use the lisppaste bot, visit http://paste.lisp.org/new/clojure and enter your paste. |
| 04:15 | rdsr | |
| 04:15 | rdsr | (gen-class |
| 04:15 | rdsr | :name reducer |
| 04:15 | rdsr | :extends org.apache.hadoop.mapred.MapReduceBase |
| 04:15 | rdsr | :implements [org.apache.hadoop.mapred.Reducer] |
| 04:15 | rdsr | :prefix "r-") |
| 04:15 | jdz | rdsr: please stop pasting code here |
| 04:15 | rdsr | |
| 04:15 | rdsr | (defn r-reduce [this key values output-collector] |
| 04:15 | rdsr | (.collect output-collector key (apply max values))) |
| 04:15 | rdsr | |
| 04:16 | rdsr | (defn -main [s] |
| 04:16 | rdsr | (let [conf (JobConf. maxtemperature) |
| 04:16 | rdsr | args (re-split " +" s 2)] |
| 04:16 | rdsr | (doto conf |
| 04:16 | rdsr | (.setJobName "Max temperature") |
| 04:16 | rdsr | (.setMapperClass mapper) |
| 04:16 | rdsr | (.setReducerClass reducer) |
| 04:16 | jdz | rdsr: you're not helping yourself here |
| 04:16 | rdsr | (.setOutputKeyClass Text) |
| 04:16 | rdsr | (.setOutputValueClass IntWritable)) |
| 04:16 | rdsr | (FileInputFormat/addInputPath conf (first args)) |
| 04:16 | rdsr | (FileOutputFormat/setOutputPath conf (second args)))) |
| 04:16 | rdsr | |
| 04:16 | rdsr | (compile 'maxtemperature) |
| 04:16 | rdsr | The problem is a direct translation of a hadoop job from the Book Hadoop The definite guide |
| 04:16 | rdsr | The problem I'm facing is whenever i try to evaluate the function main |
| 04:16 | rdsr | it cribs saying the its unable to resolve maxtemperature |
| 04:16 | rdsr | oh sorry |
| 04:17 | rdsr | I paste it there |
| 04:17 | rdsr | thks for the info |
| 04:17 | rdsr | sorry I'll paste the code on paste.lisp.org |
| 04:17 | lisppaste8 | rdsr pasted "hadoop example not working" at http://paste.lisp.org/display/84264 |
| 04:18 | rdsr | Hi all, sorry about the previous paste |
| 04:18 | rdsr | What i wanted to know was what is it i'm doing wrong when I try to evaluate the main function bu it cribs saying that "its |
| 04:19 | rdsr | unable to resolve the symbol maxtemperature" |
| 04:19 | rdsr | Without the main function the code compiles fine |
| 04:20 | rdsr | I guess my classpath is set correctly since I'm able to compile my code without the main function |
| 04:20 | Fossi | you need to compile the gen-classes first |
| 04:20 | jdz | well |
| 04:21 | Fossi | but (as i just learned on the weekend) most likely you want to use proxz a whole lot more |
| 04:21 | jdz | what is the variable maxtemperature in the main function supposed to contain |
| 04:21 | jdz | ? |
| 04:21 | rdsr | its a class name which i define in the beginning of the code |
| 04:21 | rdsr | I hope my syntax is correct |
| 04:22 | rdsr | The JobConf expects a parameter of type Class |
| 04:23 | rdsr | Fossi: I tried compiling the class before the main function but I guess you can only write the compile function at the end of your file |
| 04:24 | rdsr | Fossi: Also I think proxies won't help in that they just give me an on the fly obj what I need is a class |
| 04:29 | Fossi | you need a class because of what? |
| 04:29 | Fossi | do you actually pass the class object to the hadoop stuff? |
| 04:30 | rdsr | Fossi yes |
| 04:30 | Fossi | ok, then i'd put my gen-class things in a different file and compile them aot |
| 04:31 | rdsr | yes JobConf expects a class object |
| 04:31 | rdsr | hmmm |
| 04:38 | rdsr | Fossi: I think the problem is something else cos, if you see the compilation example at http://clojure.org/compilation |
| 04:38 | AWizzArd | ~seen kotarak |
| 04:38 | clojurebot | kotarak was last seen quiting IRC, 4945 minutes ago |
| 04:38 | rdsr | the main function does access the constructor of the generated class |
| 04:40 | rdsr | even if i modify my code, like that of the example, the compiler again gives the same error |
| 05:48 | lbj | Top of the morning gents |
| 05:57 | cgrand | lbj: what happened to your old nick? |
| 06:13 | rottcodd | is it possible to access the metadata in this proxy? (proxy [java.util.Map] [] (metadata [] {:thing 1})) |
| 06:16 | Chousuke | Map does not have a metadata method, does it? :/ |
| 06:16 | Chousuke | what are you trying to accomplish? |
| 06:20 | rottcodd | I want to attach some data to a proxy for dispatching on a multimethod |
| 06:21 | Chousuke | hm, well, you could always implement IMeta with it. |
| 06:22 | Chousuke | or you could use a map as a wrapper for the actual proxy object and have the metadata on the map. |
| 06:39 | rottcodd | (let [obj (proxy [java.util.Map] [] (metadata [] {:thing 1}))] ((get (.__getClojureFnMappings obj) "metadata") obj)) |
| 06:43 | Chousuke | rottcodd: I'm still fairly certain java.util.Map has no metadata interface :/ |
| 06:43 | Chousuke | rottcodd: if you want to implement Clojure-compatible metadata, you need clojure.lang.IMeta I think. |
| 06:44 | rottcodd | this has nothing to do with java.util.Map, it should work with any proxy object |
| 06:47 | Chousuke | Well, a proxy can implement multiple interfaces. |
| 06:49 | rottcodd | implementing clojure.lang.IMeta would be the right way to do it, but I want to avoid aot compilation |
| 06:53 | Chousuke | rottcodd: why would you need AOT compilation? you're using proxy, aren't you? |
| 06:54 | Chousuke | (proxy [your.class.Here clojure.lang.IMeta] ...) |
| 06:54 | rottcodd | ah, I wasn't thinking straight |
| 06:55 | rottcodd | thanks Chousuke |
| 07:07 | AWizzArd | Hello rhickey. I programmed a very simple lexer in Clojure. I did that manually because it is simple (natural language: whitespace, words, numbers and punctuation). Because I need this very often it must run extremly fast. |
| 07:07 | AWizzArd | It is just a very small part of a whole system. To let it run as fast as possible I made a pure Java version. It's 25 LOC vs 34 in Java, so, no big deal. Now I see that Java is about 5x faster for many small texts. |
| 07:07 | AWizzArd | But, this is strange, for very big texts the pure Clojure version is faster. Do you have an idea how this can be? Maybe the GC? |
| 07:09 | Chousuke | can you show the code? |
| 07:11 | AWizzArd | No, I can't show it (copyright issue). But what it does: tokenizing a string into words, numbers, whitespace and specials. A token is represented as a hashmap, for example {:token "Hello", :category :word, :start 0, :end 5}. In Java this is a java.util.HashMap. |
| 07:11 | AWizzArd | Specials will be a hashmap where :token is a string of length 1. |
| 07:12 | AWizzArd | The other groups (words, whitespace and digits) group together. |
| 07:12 | Chousuke | is the hashmap a structmap? |
| 07:12 | AWizzArd | So, partition-by is doing nearly the right job, with the exception of specials. |
| 07:12 | Chousuke | hmm |
| 07:12 | AWizzArd | Yes, the hashmap is a struct. |
| 07:13 | AWizzArd | if you have a function get-category [char] which uses Character/isLetter(), Character/isWhitespace() and isDigit then partition-by nearly does the right job. So that's the code basically, and in Java a simple for loop. |
| 07:14 | AWizzArd | But for some reason Java is slow for very big inputs. |
| 07:14 | maacl | Is there a good email library for Clojure? or does one use JavaMail? |
| 07:17 | Chousuke | AWizzArd: are you using substrings in java? |
| 07:17 | Chousuke | really hard to think of any reasons without seeing the code :P |
| 07:18 | Chousuke | are you sure the clojure version is fast for big inputs and not just lazy? :) |
| 07:18 | AWizzArd | Chousuke: yes, I use substring in Java and subs in Clojure. First I tried collecting the single chars and then applied str to that. But subs is a good bit faster, so I used the same strategy in Java as well. |
| 07:19 | AWizzArd | Chousuke: right, it is not lazy, I count the number of tokens so that each is touched. |
| 07:22 | maacl | ah just found Postal http://github.com/drewr/postal/tree/master |
| 07:22 | Chousuke | I suppose it might be the laziness. you never need to hold on to the head, do you? |
| 07:23 | AWizzArd | I collect all tokens of my input text in a vector. And I count the number of elements. |
| 07:23 | Chousuke | okay, so you do hold on to the whole data :P |
| 07:23 | Chousuke | hm |
| 07:23 | AWizzArd | Timing example for one MB text: Clojure 1800 msecs, Java 2400 msecs |
| 07:24 | AWizzArd | Timing example for 30 texts of 2kb - 25kb each: Clojure 100 msecs, Java 17 msecs |
| 07:25 | AWizzArd | When I let my make-token function just return a string and not an instance of a struct map then Clojure performs in 1000 msecs in the first example. |
| 07:25 | AWizzArd | When I remove the makeToken function in Java it reduces the above timing to 400 msecs. |
| 07:25 | Chousuke | how about java? |
| 07:25 | Chousuke | hm |
| 07:26 | AWizzArd | So, the real pain sits in the part where the 745k java.util.HashMap's are created. |
| 07:26 | Chousuke | you represent the tokens as HashMaps in java? |
| 07:26 | AWizzArd | yes |
| 07:26 | Chousuke | that would be it, I guess. |
| 07:26 | AWizzArd | I thought they would be so very fast? Not? |
| 07:26 | AWizzArd | Maybe I should use Clojure structs directly... |
| 07:26 | Chousuke | StructMaps can share their base keys, leading to some speed and space efficiency. |
| 07:26 | Chousuke | HashMaps can't. |
| 07:27 | Chousuke | if you had a Token class with fields in Java, I bet it would be even faster |
| 07:27 | AWizzArd | I see. So using Clojures StructMaps in Java may speed things up and also make it more comfortable for me. |
| 07:27 | AWizzArd | Chousuke: I will try both. I guess what you said about having a Token class will be fastest. |
| 07:28 | AWizzArd | Does the wikibook say how one can use Clojures data structures in Java? :) |
| 07:28 | Chousuke | well, clojure maps are java.util.Maps |
| 07:28 | Chousuke | so as long as you don't need to modify them, they should work just fine with any function expecting those. |
| 07:29 | AWizzArd | For printing the direct Clojure class is nicer. |
| 07:29 | AWizzArd | But right, I don't need to modify the tokens. |
| 07:29 | AWizzArd | So, I could use a mutable datastructure. |
| 07:30 | Chousuke | or declare final fields :p |
| 07:31 | AWizzArd | How? |
| 07:32 | Chousuke | um, just class Token { public final String token; ...}? |
| 07:32 | Chousuke | I wonder if it works like that. |
| 07:32 | Chousuke | you obviously need to have a constructor that initialises them. |
| 07:34 | AWizzArd | Chousuke: do you know how I can use a clojure.lang.PersistentStructMap in Java? |
| 07:34 | AWizzArd | I guess having the clojure.jar in the CP is the first step. |
| 07:34 | Chousuke | that's all you need as far as I know. |
| 07:36 | AWizzArd | But how would I then say the equivalent of (struct +token+ :token :category :start :end)? |
| 07:36 | AWizzArd | uhm, defstruct of course |
| 07:36 | Chousuke | for that I suppose you need to do some magic to get to the clojure evaluattor |
| 07:36 | Chousuke | -r |
| 07:36 | Chousuke | er |
| 07:36 | Chousuke | -t :P |
| 07:37 | Chousuke | would probably be easier to :genclass something that offers java factory functions for whatever you need. |
| 07:39 | AWizzArd | in principle the PersistentStructMap will offer some constructors and methods. I will have a look at the sources, it will be there I guess. |
| 08:29 | cemerick | rhickey: I was about to reply on the list, but I thought (if you're available) that I'd confine my embarrassment to the channel :-) |
| 08:31 | Chouser | ~max |
| 08:31 | clojurebot | max people is 164 |
| 08:31 | AWizzArd | not yet.. but soon again, probably in autum |
| 08:32 | AWizzArd | Btw Chouser, do you know how I can create a clojure.lang.PersistentVector after I imported it into Java? It seems to offer two constructors... |
| 08:33 | AWizzArd | Or is there a way how I can say: PersistentVector x = SomeClass/staticMethod("(vector 1 2 3)"); ? |
| 08:37 | Chouser | AWizzArd: the constructors that take a java.util.List and Object[] seem like good choices. |
| 08:37 | Chouser | oh, sorry |
| 08:37 | Chouser | factory methods. |
| 08:38 | AWizzArd | Chouser: ah, good |
| 08:38 | Chouser | clojure.lang.PersistentVector.create(Object[]{1,2,3}) or something, right? |
| 08:39 | AWizzArd | yes, the static methods look fine |
| 08:39 | AWizzArd | Chousuke: I now made a Token class and use that in my Lexer instead of a HashMap. Blindingly fast stuff now. |
| 08:40 | Chouser | AWizzArd: those factory methods use the mutable vector state in the chunks branch. |
| 08:44 | AWizzArd | I don't know the chunk branch yet. I saw master and par. |
| 08:53 | rhickey | cemerick: there's nothing to be embarrassed about, it's not a bad thing to want (dynamic named classes), just not something Java was designed to deliver well |
| 08:54 | rhickey | I had a big philosophical struggle with newnew over the weekend |
| 08:54 | rhickey | on the surface, it looks like an interop thing, i.e. given its similarity to proxy |
| 08:55 | Fossi | [insert clever greek philosopher joke] |
| 08:55 | rhickey | but it really shouldn't be. It has to be looked at as a core Clojure construct |
| 08:56 | rhickey | since Clojure is defined in terms of these interfaces, and the objective is to let Clojure define itself, Clojure needs to be able to define and implement interfaces |
| 08:56 | rhickey | but it should do so minimally, not incidentally adopt the semantics of Java |
| 08:58 | rhickey | concrete derivation in particular complicates things significantly |
| 08:58 | rhickey | I'm seriously thinking about limiting newnew to implementing interfaces only |
| 08:59 | rhickey | using closure semantics only for named values, no explicit notion of fields or ctors |
| 09:00 | rhickey | I came up with a hybrid strategy for mutability - close over a value and declare it mutable/volatile in the newnew construct |
| 09:01 | cemerick | rhickey: the 'embarrassment' bit was definitely tongue-in-cheek :-) |
| 09:02 | rhickey | cemerick: actually I think public discussions of the named class issues are important, as few people understand the limitations - better understanding would engender more realistic expectations, also the value proposition of working well with modular systems |
| 09:02 | cemerick | I'm clearly not aware of a lot of the issues, especially any kind of design aesthetic w.r.t. low-level impl. My primary concern was: if there are no statically-named classes, perhaps especially w.r.t. core collections, then it seems that they would not be usable in any scenario where Serialization was a requirement: ORMs, JMS queues, etc. |
| 09:03 | Chouser | I like the idea of the volatile thing being named (again) in the newnew form, to clarify the syntactic scope. |
| 09:03 | rhickey | Chouser: the key thing is that it is not introduced there, just qualified |
| 09:04 | Chouser | yep, so it's still 'let' creating the things to be closed-over. |
| 09:04 | rhickey | cemerick: It is a good point, and I have to think about that |
| 09:04 | cemerick | OK, glad I'm not totally off the reservation. |
| 09:05 | MarkVolkmann | Hey Rich, hate to bother you and know your time is occupied by many other things, but if you get some time, can you take a look at my email question about LockingTransaction and a wait call? Hopefully it's an easy question for you. |
| 09:05 | rhickey | but it is not an argument for names in newnew, it is an argument against using newnew to define the core collections |
| 09:05 | cemerick | ah-ha. |
| 09:06 | Chouser | but surely if newnew is incapable of defining Clojure's own core collections, that indicates a critical weakness in newnew? |
| 09:06 | rhickey | MarkVolkmann: the reason for the wait call is to prevent optimistic spinning, i.e. the transaction is likely to get right to the same failure if it repeats immediately, by blocking on the current owner you prevent that waste |
| 09:06 | cemerick | rhickey: for a moment, assume no newnew usage would occur at runtime, and the entire body of clojure code is AOT compiled, then being able to define static names would be entirely harmless and only beneficial for interop, yes? |
| 09:07 | MarkVolkmann | Thanks Rich! |
| 09:07 | Chouser | or perhaps you are considering a second new construct for Java interop scenarios? |
| 09:07 | rhickey | MarkVolkmann: however, the use of wait specifically is an implementation detail, and likely to change in the near future, possibly to a countdown latch |
| 09:08 | rhickey | cemerick: the problem with that assumption is the assumption |
| 09:08 | cemerick | yeah, I'm just firming up my conception of things. |
| 09:09 | rhickey | MarkVolkmann: I'm partial to the arguments made in this paper: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.105.3507&rep=rep1&type=url&i=0 |
| 09:09 | rhickey | aargh: http://berkeley.intel-research.net/rennals/pubs/052RobEnnals.pdf |
| 09:11 | MarkVolkmann | Thank you very much! Looks like a good read. |
| 09:12 | rhickey | cemerick: also, even if intended for AOT, having to restart on bugfix would stink during development, but true too of genclass |
| 09:12 | cemerick | rhickey: I guess I'd say that, if my last supposition is correct, then newnew could be allowed to define static names at AOT-compile-time, to redefine classes associated with those names at runtime, but not be able to introduce new static names at runtime? |
| 09:12 | rhickey | above pdf is: Software Transactional Memory Should Not Be Obstruction-Free |
| 09:13 | cemerick | it's a helluva wrinkle, but it's an out for the interop issues, perhaps |
| 09:13 | AWizzArd | I would like to call the static public PersistentStructMap construct(Def def, ISeq valseq) method, to create a Struct, from within Java. How can I create a Def object. Def seems to be something like an inner class? |
| 09:13 | rhickey | cemerick: one can't: redefine classes associated with those names at runtime |
| 09:14 | cemerick | oh, righty, can't unload classes without classloader majick. |
| 09:16 | cemerick | Actually, the class wouldn't have to be redefined, only new methods added. |
| 09:16 | AWizzArd | Calling PersistentStructMap.Def.createSlotMap(somePersistentVector) does not work. |
| 09:16 | rhickey | Chouser: not a second construct, as we have genclass, what's missing from it is indirection-free method invocation, but unlike proxy gen-class invocation is direct, not hashmap lookup |
| 09:16 | Chouser | AWizzArd: if you follow the code path from the clojure fns you know, you should be able to answer these questions yourself. That's all I'm doing for you anyway. |
| 09:17 | AWizzArd | Ah ok, I started in the PersistentStructMap.java file. |
| 09:17 | rhickey | cemerick: any kind of class reloading/redefining is too far off the beaten path for Clojure |
| 09:18 | cemerick | you mean, in order to work properly within osgi, et al.? |
| 09:18 | Chouser | AWizzArd: defstruct expands to create-struct; create-struct call PersistentStructMap.createSlotMap() |
| 09:18 | rhickey | cemerick: right, and everything else (tools etc) |
| 09:18 | rhickey | cemerick: I don't want to take on 'fixing' Java |
| 09:18 | cemerick | no, certainly not. |
| 09:19 | cemerick | But, it would be a pretty big deal if, to put a clojure map into a JMS queue, you had to copy it into a java.util.HashMap. :-/ |
| 09:19 | rhickey | also I think you'd run into serialization issues with modified versions of the same-named class |
| 09:19 | AWizzArd | Chouser: how do you follow that code path? Do you use some kind of stepper? |
| 09:20 | Chouser | AWizzArd: no, I open core.clj, search for defstruct, read it, search for create-struct, etc. |
| 09:20 | rhickey | cemerick: you aren't using prn for that? |
| 09:20 | Chouser | AWizzArd: vim has a handy keystroke that finds other occurances of the word under the cursor. I use that a lot. |
| 09:21 | AWizzArd | Chouser: oki, I will see what I can find there. Thanks for the suggestion. I was hoping that I know enough Java by now to find it out by looking at the direct .java file. But Rich used some more complex constructs that I don't know the syntax of. |
| 09:21 | cemerick | rhickey: Nope. Just a PersistentVector, directly. Serialization works just fine for transient stuff. |
| 09:22 | Chouser | AWizzArd: but that path just leads to createSlotMap -- why did you say that doesn't work? |
| 09:22 | cemerick | (I only mentioned map above off-hand) |
| 09:22 | cemerick | Actually, we don't use maps for that purpose because of the interning issues with keywords |
| 09:22 | rhickey | cemerick: I haven't audited Clojure's data structures for serializability at all |
| 09:22 | rhickey | a todo item |
| 09:23 | cemerick | well, it works well enough at the moment :-) |
| 09:23 | AWizzArd | Chouser: I saw in PersistentStructMap.java an inner class Def. Only for that class I found the createSlotMap method. |
| 09:24 | AWizzArd | So I thought it could be good to somehow mention Def in the call. Either Def.createSlotMap() or PersistantStructMap.Def.createSlotMap() sounded good. |
| 09:24 | cemerick | rhickey: and honestly, I never thought twice about it -- in my head, core datastructures should *always* be serializable. I would have thought it a bug if they weren't, actually. There's just too many APIs out there that depend on it. |
| 09:24 | rhickey | http://groups.google.com/group/clojure/browse_frm/thread/9b7ef55a719f4b0b/80897665ffed6f9f?lnk=gst&q=serializeable+keywords#80897665ffed6f9f |
| 09:25 | rhickey | cemerick: you want to submit a patch for keywords?, the poster doesn't have a CA |
| 09:26 | rhickey | cemerick: but AOT-compiled newnew classes *will* have names, so if you've got the same jar on both ends you should still be able to (de)serialize |
| 09:26 | cemerick | that's interesting, we've successfully serialized keywords plenty. It doesn't quite work, though, as the deserialized keyword instances aren't properly interned. |
| 09:27 | cemerick | Maybe our framework uses reflection to back into a serialization.....? |
| 09:27 | rhickey | cemerick: reflection should work too, the only problem is code that explicitly uses the names |
| 09:28 | cemerick | rhickey: OK, but you shut down your JMS server (with messages enqueued), deploy a new version of clojure, and *boom* |
| 09:29 | rhickey | cemerick: I understand, but depending upon what changed, it could go boom anyway |
| 09:30 | rhickey | serialized objects as messages being extremely fragile |
| 09:30 | cemerick | yeah, just illustrating |
| 09:30 | rhickey | prn/read!! |
| 09:30 | AWizzArd | Chouser: but now it works. I didn't know that Java allows to call static methods of inner static classes by not mentioning the name of those inner classes. |
| 09:31 | Chouser | AWizzArd: hm! I still don't know that. :-) |
| 09:32 | rhickey | cemerick: so it's not merely names, but serialVersionUIDs etc |
| 09:33 | AWizzArd | Chouser: yeah, but you just teamed up with me, and we found that out. We are becoming Java experts. Maybe. |
| 09:33 | rhickey | cemerick: perhaps the collections should define read/writeObject |
| 09:33 | cemerick | rhickey: I'm completely sympathetic, but prn/read can't be the answer to interop. |
| 09:34 | cemerick | yeah, that's what I was offering as a potential solution in my first email, but still, there needs to be a static name, or the ObjectInputStream will never be able to find a corresponding readObject method. |
| 09:34 | rhickey | cemerick: ah, nevermind |
| 09:37 | cemerick | I think everyone would agree. However, definitions of 'ephemeral' differ (e.g. my example of halting a JMS server). |
| 09:38 | rhickey | neat: http://clojuratica.weebly.com/ |
| 09:38 | cemerick | heh, that's pretty crazy |
| 09:40 | rhickey | cemerick: I'm just playing devil's advocate, the serialization issue you've raised is an important one that I haven't given enough consideration, being personally afraid of serialization-based solutions |
| 09:42 | cemerick | the really crazy thing being, serialization is baked into so much of what larger jvm apps/frameworks do... |
| 09:43 | rhickey | cemerick: that is the key argument, even if you don't choose it, it may be forced upon you, thus it is an interop issue |
| 09:44 | AWizzArd | that clojuratica seems to be a nice helper.. wow :) |
| 09:47 | cemerick | rhickey: I think serialization could be really great, especially if it were straightforward to get print-dup-driven read/writeObject impls on clojure objects. You'd then have the benefits of both, I think: future-proofed serialization output sitting on top of the well-tested Serialization backend. The only hurdle is those static names. |
| 09:49 | rhickey | cemerick: there's really no hurdle with gen-class style AOT |
| 09:51 | rhickey | perhaps newnew could be made to fail/warn if named and re-evaluated, i.e. no new code will be associated with the class |
| 09:51 | rhickey | until restart |
| 09:51 | Chouser | rhickey: so at the moment you're advocating that cinc uses gen-class to implement the core data structures? |
| 09:52 | cemerick | yes, you can certainly write your own read/writeObject impls, but if your object graph is entirely clojure-based, then you should be able to generate it based on classname/fields/etc |
| 09:52 | cemerick | ("you" being the royal "you", not necessarily rhickey :-) ) |
| 09:53 | cemerick | rhickey: yes, that's what I was suggesting earlier: newnew produces stable names at AOT compile |
| 09:53 | rhickey | Chouser: Not at all, I think the data structures should be defined with something with significantly simpler semantics than gen-class |
| 09:54 | rhickey | cemerick: that's not what I'm saying, I'm saying if named, will produce with that name, and if named and reloaded, can't reload so complains |
| 09:54 | rhickey | produce class with that name |
| 09:54 | Chouser | gen-interface did that once upon a time |
| 09:54 | rhickey | so not limited to AOT |
| 09:55 | rhickey | Chouser: right, which brings up the second set of problems, if name is used explicitly at runtime the dynamically-loaded nature of the bytecode will be a problem, since not visible from all classloaders |
| 09:56 | rhickey | if named would have to use the compile-to-file-in-classpath-dir I mentioned in my clojure-dev message |
| 09:57 | Chouser | like AOT does now while compiling. |
| 09:57 | rhickey | that would let it be loaded by the same classloader that loaded clojure itself, so fine for most interactive/scripty things |
| 09:58 | rhickey | but I still dread the - this isn't working dynamically under Tomcat scenarios |
| 09:59 | rhickey | I don't know if people would understand the 'now that you've named this it is static' aspect |
| 09:59 | rhickey | not having explicit names is much cleaner |
| 10:02 | rhickey | Chouser: so maybe still two worlds, newnew is really generate an instance of an anonymous implementation of some interfaces, and some static AOT-only generate a named implementation of some interfaces |
| 10:02 | AWizzArd | When I have a static method FOO in a Java class which uses new to create some immutable objects, then I can run pmap over FOO without problems yes? I don't need to synchronize that method somehow right? |
| 10:03 | rhickey | in both cases substantially reduced semantics from proxy/gen-class |
| 10:03 | AWizzArd | One and the same static methods can run in 20 threads at the same time, and each thread works on its own new'ed objects? |
| 10:03 | rhickey | AWizzArd: yes |
| 10:03 | AWizzArd | good |
| 10:04 | rhickey | when you pmap inc it is calling new Integer at some point |
| 10:07 | cemerick | rhickey: as long as proxy (or some corollary) remains, I don't think you'll hear too much |
| 10:08 | rhickey | I guess another option is that, if named, newnew is a no-op when not AOT |
| 10:09 | cemerick | (however, the newnew wiki page saying that it will replace proxy is definitely confusing if that's the case) |
| 10:09 | rhickey | cemerick: that is an ideas page, it will contain conflicting ideas until I've worked it out, then another documentation page will descibe what won |
| 10:10 | cemerick | rhickey: you mean ideas don't pop out of your head fully-formed and with three potential implementations?!? ;-) |
| 10:10 | rhickey | until this weekend I thought it would replace proxy, now I feel making some very simple subset semantics is important |
| 10:12 | rhickey | cemerick: the wiki page is an experiment in exposing the true nature of how this stuff comes about, and an effort to get others involved, and input, but scarily revelatory |
| 10:12 | cemerick | maybe newnew never takes a name, but another form links a static name to a newnew impl, copying its bytecode? |
| 10:13 | rhickey | cemerick: all paths to such things involve ctors in a way I'd like to avoid. I'm currently thinking about these things as closure-objects |
| 10:14 | rhickey | that too may become an interop issue |
| 10:14 | Chouser | if newnew doesn't help with core data structures, is it needed before cinc? Or do we just need the reduced gen-class? |
| 10:15 | rhickey | but I've been thinking about auto-generating static factory methods |
| 10:15 | Chouser | leaving newnew as a high-performance introp thing for later. |
| 10:16 | rhickey | Chouser: newnew is not about interop, it is about cinc - what's missing from Clojure to let it define itself? |
| 10:16 | rhickey | right now part of Clojure is defined with Java semantics - those interfaces at least |
| 10:18 | rhickey | a simple approach would be to say, well, use gen-class to implement those interfaces, and worry about making gen-class faster later |
| 10:19 | Chouser | interfaces and (what are they called...) abstract implementations |
| 10:19 | rhickey | but gen-class adopts many complex semantics of Java, necessary for interop perhaps but not the cinc job. I'd like newnew to be something whose semantics are defined by Clojure, not be reference to Java, and implementable easily in more places than the JVM |
| 10:20 | rhickey | Chouser: right, and avoiding fields, access specifiers, ctors, implementation inheritance etc etc |
| 10:20 | rhickey | but with an easily explained mapping to the host |
| 10:21 | Chouser | ok, so because all this is for cinc, ignore java interop for now and find the clean minimal requirements for named and (perhaps separately) anonymous things that can be mapped to high-performance host constructs. |
| 10:22 | rhickey | exactly |
| 10:22 | Chouser | is there anything in clojure's core .java stuff now that could be anonymous? |
| 10:22 | rhickey | its for cinc and its the story for writing high-perf constructs like the core data structures in (portable?) Clojure |
| 10:23 | rhickey | it's |
| 10:23 | rhickey | I think i could define all of Clojure this way, all the data structures, everything |
| 10:24 | rhickey | the only hitch is the abstract base classes |
| 10:24 | Chouser | "this way" being a sort of simplified gen-class? or anonymous newnew thing? |
| 10:25 | rhickey | mine I could get rid of, but the java.util.AbstractMap et al are really useful |
| 10:26 | rhickey | Chouser: I don't need the names but cemerick does in order to serialize them, but let's consider it one thing, a way to implement interfaces, given gen-interface as a way to define them. The runtime/AOT issues are orthogonal |
| 10:27 | rhickey | I was working this weekend on trying to treat abstract classes as mixins |
| 10:27 | Chouser | yeah, I found that concept quite intruiging |
| 10:27 | rhickey | I took a look at Scala's Traits and was horrified |
| 10:31 | rhickey | I decided I don't want or need a way to define abstract classes in Clojure, but on the fence about newnew being able to consume them (as if an interface), subject to a no-arg-ctor rule, which is true for the java.util AbstractBases |
| 10:32 | Chouser | huh. You'd rather repeat chunks of code for clojure core classes? What about Traits drove that home? |
| 10:32 | rhickey | Chouser: no, but there are other options for CLojure, using macro-like things |
| 10:32 | Chouser | ah, ok. |
| 10:33 | rhickey | the traits thought was separate, just unsettlingly complex and nasty |
| 10:33 | Chouser | sure, I've got no objection there. |
| 10:37 | Chouser | so if cemerick didn't need names, all cinc would really need is a simple newnew (implementing only pure interfaces) and a handful of macros. |
| 10:38 | rhickey | Chouser: I think so |
| 10:39 | rhickey | what about an :aot-compile-as fixed.class.Name flag on newnew? |
| 10:39 | Chouser | such a pity. #=(hash-map :a 1, :b 2) looks good to me. :-P |
| 10:41 | Chouser | so that would use a gen'ed name at runtime if that class name was not found? |
| 10:41 | rhickey | it would always use a gen-ed name when not AOT compiling |
| 10:42 | rhickey | or yes, if not found |
| 11:04 | AWizzArd | I compile my .java file while still editing it every two minutes. How can I reload the new class file (sits in my CP) in Clojure without restarting it? |
| 11:05 | stuartsierra | AWizzArd: you can't, not without using custom ClassLoaders or an IDE like Eclipse. |
| 11:05 | AWizzArd | can I add new classes that I compiled after Clojure started, but which I didn't use yet? |
| 11:06 | stuartsierra | I don't know; probably yes. |
| 11:07 | AWizzArd | maybe via add-classpath |
| 11:07 | AWizzArd | I just put the compiled file into new paths and then load those classes ;-) |
| 11:07 | stuartsierra | No, that adds a JAR or directory containing compiled files. |
| 11:07 | stuartsierra | maybe |
| 11:11 | AWizzArd | Btw stuartsierra, what stepper do you use for your Clojure programming? |
| 11:11 | Chouser | if the class has the same name, I don't think it matter what directory it's getting loaded from -- you can still only do it the first time. |
| 11:11 | AWizzArd | Chouser: yes, guess so. This means happy restarting of Clojure every few moments. The staticism of Java transports to CLJ. |
| 11:12 | Chouser | AWizzArd: when writing .java, yes. proxy and gen-class both do work for you to allow loading new implementations into a running jvm. |
| 11:12 | stuartsierra | This is what OSGi frameworks and Java IDEs were made to solve, but they're big, complicated mechanisms. |
| 11:13 | mebaran151 | is there a simple functional way to turn an integer into an array of bytes or do I have to do it imperatively? |
| 11:14 | mebaran151 | I want to cast Integer into a bigendian representation and put that in a byte array |
| 11:14 | stuartsierra | Look at the java.lang.Integer class; there should be a method that does that. |
| 11:15 | mebaran151 | a couple methods will reverse the bytes, which I don't think I want, but none of them seem to return an array of bytes |
| 11:16 | mebaran151 | I'm trying to make an efficient byte order sortable representation of integer |
| 11:18 | mebaran151 | big endian integer encoding should work |
| 11:23 | achim` | ,(-> 12345 bigint .toByteArray) |
| 11:23 | clojurebot | #<byte[] [B@18119d7> |
| 11:26 | hiredman | busy morning |
| 11:27 | Chouser | no way to alias a classname yet, right? |
| 11:30 | Jomyoot | what is yield :javascript ? |
| 11:30 | Jomyoot | <%= yield :javascipt %> |
| 11:30 | Jomyoot | what does it do? |
| 11:31 | Chouser | Jomyoot: that looks like ruby to me. where are you seeing that? |
| 11:36 | Jomyoot | wrong channel |
| 11:36 | Jomyoot | i am seriously sorry |
| 11:36 | Chouser | :-) np |
| 11:44 | rhickey | Chouser: not yet |
| 11:51 | Chouser | It'd be handy if proxy methods had fns named like the methods they implement |
| 11:51 | hiredman | eh? |
| 11:52 | hiredman | aren't proxy methods fns? |
| 11:52 | Chouser | yes, anonymous fns |
| 11:52 | Chouser | Wrong number of args passed to: Foo$array-input-stream--1231$fn |
| 11:53 | Chouser | that error could have the method name in there. |
| 11:53 | hiredman | ah |
| 11:56 | Chouser | hm, well that was easy. |
| 12:07 | Chouser | how can I get the size of a Java array from Clojure? |
| 12:07 | hiredman | count? |
| 12:07 | hiredman | ,(count (into-array [1 2 3])) |
| 12:07 | clojurebot | 3 |
| 12:09 | gulagong | hi |
| 12:09 | gulagong | i want to do fast matrix multiplikation |
| 12:09 | gulagong | but i don't get it fast ;) |
| 12:09 | gulagong | maybe on of you wizzards could have a look at the code? |
| 12:10 | stuartsierra | matrix multiplication is kind of specialized, you probably want a dedicated matrix library |
| 12:10 | Chouser | hiredman: ha! thanks. |
| 12:10 | Chouser | hm, 'counted?' says false. But that's not right, is it? |
| 12:10 | gulagong | i want to do it myself... |
| 12:10 | stuartsierra | counted checks inheritance |
| 12:11 | Chouser | stuartsierra: right, but I wonder if it should have a special case for Java arrays |
| 12:11 | hiredman | ,(doc counted) |
| 12:11 | clojurebot | "/;nil; " |
| 12:11 | hiredman | ,(doc counted?) |
| 12:11 | clojurebot | "([coll]); Returns true if coll implements count in constant time" |
| 12:12 | gulagong | my concrete broblem is: (get (long-array '(1 2 3 4 5)) 2) is not guaranteed to be a long-value |
| 12:12 | gulagong | so this slows me down |
| 12:14 | gulagong | and i don't see a reason in having long-arrays at all if they don't return longs |
| 12:14 | Chouser | gulagong: you've looked at the type hint and java primitive info at http://clojure.org/java_interop ? |
| 12:15 | gulagong | chouser: yes, but i'll do again |
| 12:15 | gulagong | chouser: hm, aget seems promising |
| 12:16 | gulagong | chouser: next time i'll read better |
| 12:16 | Chouser | gulagong: np. |
| 12:19 | rhickey | anyone explicitly deriving from RestFn? |
| 12:20 | hiredman | ~def c.l.RestFn |
| 12:23 | gulagong | speedup 5 :) |
| 12:23 | Chouser | gulagong: 5x ?? nice |
| 12:26 | Chouser | ,(expression-info '(let [a (atom 5)] (+ (int @a) @a))) |
| 12:26 | clojurebot | {:class java.lang.Number, :primitive? false} |
| 12:26 | Chouser | ,(expression-info '(let [a (atom 5)] (+ (int @a) (int @a)))) |
| 12:26 | clojurebot | {:class int, :primitive? true} |
| 12:34 | rzoom | where should i post a request for comment on a contrib patch? here? a mailing list? |
| 12:35 | Chouser | rzoom: either one. You've read http://clojure.org/patches ? |
| 12:36 | rzoom | yes, i have it in a github repo |
| 12:36 | rzoom | but it maybe easier to just attach the patch. |
| 12:37 | Chouser | sure, git-formatted patches are nice |
| 12:38 | Chouser | as described there. I asked because that page answers your original question (bring it up on the google group) and also has a reminder about the CA |
| 12:38 | rzoom | yeah, sent CA last week |
| 12:39 | rzoom | wasnt sure about emailing to google group since I am not a member |
| 12:39 | Chouser | hm. It probably won't let you if you're not a member. It might be sufficient to bring it up here, depending on the patch. |
| 12:40 | rzoom | okie, will bring it up in here tomorrow, when I am off, hopefully get some feedback. |
| 12:42 | lbj | rhickey: Rour reference to 'STM should not be obstruction free', leaves me wondering: Are you planning to optimise the STM per their advice, or have you already done that? |
| 12:42 | rhickey | lbj: Clojure's STM is (already) not a spinning design |
| 12:45 | lbj | rhickey: Alright, and your meta-data handling matches what they described? |
| 12:46 | rhickey | lbj: not at all |
| 12:46 | rhickey | completely different implementation |
| 12:46 | lbj | Ok |
| 12:46 | lbj | To me, meta-data is meta-data, but I haven't got a clue about the low level handling of it |
| 12:58 | cemerick | Goodness, the innards of java serialization(+externalizable) are just *horrible*. |
| 12:58 | cemerick | That said, the breadth and depth of APIs that depend on it is staggering. |
| 13:35 | Fossi | the {var :keyword} destructuring always gets me. why is it that way around? |
| 13:36 | Chouser | to allow for options like :or, :as, :keys, etc. |
| 13:36 | Fossi | hmmm. makes sense |
| 13:37 | Chouser | ,(let [{a :alpha, :keys [b c d], :as all} {:alpha 1, :b 2, :x 3}] [a b c d all]) |
| 13:37 | clojurebot | [1 2 nil nil {:alpha 1, :b 2, :x 3}] |
| 13:43 | Chouser | http://clojure.org/special_forms#let |
| 13:45 | Fossi | ah, ok. i was looking for the defn variant, but i guess that's just a macro using let |
| 13:47 | Chouser | we don't have anything that encapsulates a blocking queue yet, right? seque notwithstanding |
| 13:47 | Fossi | ah, it works with anything which supports nth. that's neat |
| 13:49 | cemerick | Chouser: not that I'm aware of, no |
| 13:51 | cemerick | interesting that APM.keySet() returns an anonymous AbstractSet. |
| 14:09 | Fossi | is there a nice way to mock a class that's normally generated by gen-class for testing? |
| 14:10 | Chouser | you can use 'binding' to replace (locally in a thread) the definitions of any of the method |
| 14:11 | Fossi | @(.state activity) i'm tring to mock that |
| 14:11 | Fossi | the activity |
| 14:12 | Chouser | (binding [full.namespace.name.Activity/-state (fn mock-method-def [] ...)] @(.state activity)) ... |
| 14:13 | Chouser | of course by now you should have asked yourself if gen-class really is the right solution... |
| 14:13 | Fossi | ah. the /- part does the magic :) |
| 14:13 | Chouser | well, that's the name of the Var that's implementing your class, right? That's what you defn'ed in the .clj file |
| 14:13 | Fossi | well, in this case this is my interface to the world, so i have to genclass that :) |
| 14:14 | Chouser | Fossi: I'd still highly recommend using gen-interface and proxy whenever possible. |
| 14:14 | Fossi | well, this is handed to android, so not possible :\ |
| 14:14 | Chouser | Then gen-class only for static methods |
| 14:14 | Chouser | oh. :-P |
| 14:14 | Chouser | wait |
| 14:14 | Chouser | why not? |
| 14:15 | Chouser | which of those don't work with android? |
| 14:15 | Fossi | well, what would bootstrap the proxy? |
| 14:16 | Chouser | proxy macros AOT compile to regular named classes |
| 14:17 | Fossi | that i didn't understand |
| 14:17 | Chouser | have you tried using proxy with android? |
| 14:17 | Fossi | yes, it works for the inner things |
| 14:17 | clojurebot | for is not a loop |
| 14:18 | rhickey | gen-interface and proxy should work fine with Android when AOT compiled |
| 14:18 | Chouser | use gen-class to define a class with only static methods |
| 14:19 | Fossi | ah, ok. that might work |
| 14:19 | Chouser | those static method(s) can then act as factories -- using proxy to create new instances of objects that implement your gen-interface'd interfaces. |
| 14:20 | Chouser | I pretty recently and for the first time created a clojure lib meant for consumption by java. Doing it this way is beautiful and clean. |
| 14:20 | Fossi | still, i need to hold my state somewhere inbetween calls from the android api |
| 14:21 | Chouser | I ended up with a single my/Foo.clj that had a gen-class and several gen-interfaces, but no actual implementation at all. |
| 14:21 | Chouser | the gen-class names my.foo as the implementing namespace |
| 14:21 | Chouser | sorry, my.foo.clojure |
| 14:22 | Chouser | so then I had a my/foo/clojure.clj that had all the helper fns, macros, and the gen-class method implementations |
| 14:23 | Fossi | so, if you get called multiple times, where do you keep your state? |
| 14:24 | Chouser | proxies can close over state |
| 14:25 | Chouser | (defn make-foo [a b c] (let [mutable-state-thing (atom d)] (proxy ...))) |
| 14:25 | Fossi | hmmm. i guess i'll look at that when i have something working with one dynamic gen-class :) |
| 14:25 | Chouser | so in there your proxy fns can use the a b c "constructor" values and do things with d |
| 14:40 | Chouser | how do I specify a byte[] parameter to a gen-class method? |
| 14:41 | stuartsierra | try "[B" |
| 14:41 | Chouser | not: "[B" "byte[]" "B[]" #=(eval '(class (make-array Byte/TYPE 0))) |
| 14:41 | gulagong | (let [la (long-array '(1 2 3 4))] (do (aset la 1 10.0) (aget la 1))) |
| 14:41 | gulagong | ,(let [la (long-array '(1 2 3 4))] (do (aset la 1 10.0) (aget la 1))) |
| 14:41 | clojurebot | java.lang.IllegalArgumentException: argument type mismatch |
| 14:42 | gulagong | d'oh |
| 14:42 | gulagong | ,(reduce #(conj %1 (vec (repeat 5 val))) [] (range 3)) |
| 14:42 | clojurebot | java.lang.IllegalArgumentException: Wrong number of args passed to: sandbox$eval--5137$fn |
| 14:42 | gulagong | i don't understand this .... |
| 14:43 | gulagong | maybe a bug? |
| 14:43 | stuartsierra | reduce always takes a 2-argument function |
| 14:43 | gulagong | aye |
| 14:43 | gulagong | i see and its anonymous |
| 14:44 | Chouser | maybe you wanted %2 instead of val ? |
| 14:44 | mebaran151 | what's the best way of communicating something akin to an interface |
| 14:45 | mebaran151 | I'd like to make it clear that if you implement a certain set of functions, you can plugin a different backend to my hashdb library and it will work |
| 14:46 | Chouser | mebaran151: using a few defmethod calls? |
| 14:46 | stuartsierra | Or just describe it in documentation. |
| 14:46 | mebaran151 | yeah I have a nice big comment at the top |
| 14:46 | AWizzArd | Is it possible to create a new (Java) class at runtime? One would give it a name, say what attributes it has (all are private and final) and add a method to it? |
| 14:47 | Chouser | AWizzArd: no |
| 14:47 | AWizzArd | ah, too sad |
| 14:47 | Chouser | AWizzArd: http://groups.google.com/group/clojure-dev/msg/04ef3280f7678e54?hl=en |
| 14:48 | Chouser | "So, no more pretending. If you want a class name to be visible you |
| 14:48 | Chouser | must compile a .class file and put it in the classpath. " |
| 14:48 | Chouser | gah. sorry. |
| 14:48 | AWizzArd | It would have been nice as an alternative to defstruct to have a defclass. That could give an ideal candidate for throw away objects. |
| 14:49 | AWizzArd | Today I found how extremly fast it is to create hundreds of thousands of instances of a class, and gc it. |
| 14:50 | AWizzArd | Pure Java often still has the edge in performance. |
| 14:52 | mebaran151 | also I have some common support functions between database backends |
| 14:52 | rhickey | Chouser: ? |
| 14:52 | mebaran151 | however, these support macros expect certain functions to be defined in the package that imports |
| 14:52 | mebaran151 | *in the package that requires them |
| 14:53 | rhickey | AWizzArd: you create classes in Clojure all the time with fn |
| 14:53 | angerman | how would I make items in a database user-positional? e.g. the user gets a list of items [a b c d ...] and can sort them, e.g. [a d b c ...]. Does anyone know a nice approach to that problem? |
| 14:54 | Chouser | rhickey: There's probably another solution, but I've got this jni lib auto-loading when a class is loaded now. This is nice except when compiling Clojure code that depends on it, it expects the whole runtime environment to be available. |
| 14:54 | AWizzArd | rhickey: you mean creating closures? |
| 14:54 | stuartsierra | AWizzArd: fn actually creates a class at run-time |
| 14:55 | AWizzArd | And how can I define the fields? |
| 14:55 | rhickey | Chouser: that's really an issue with Clojure compiler using reflection for type info, on the todo list for cinc compiler to get rid of that |
| 14:55 | rhickey | AWizzArd: let |
| 14:56 | Chouser | rhickey: how else would it know what to compile? |
| 14:56 | rhickey | Chouser: jar files, like javac |
| 14:56 | stuartsierra | Huh, never thought of it that way. So (let [x 1, y 2] (fn [] (prn x y))) creates a class with public member fields x and y? |
| 14:57 | cemerick | rhickey: earlier, you mentioned something about gen-class not having the performance issues of proxy...did I misunderstand you? I thought gen-class' bytecode was 2/3 levels of indirection away from a fn (genclass method -> Var -> fn), similar to proxy? |
| 14:57 | Chouser | rhickey: oh. my. |
| 14:57 | rhickey | cemerick: proxy does a hashmap lookup |
| 14:57 | Chouser | stuartsierra: private member fields, populated in the constructor |
| 14:57 | AWizzArd | rhickey: will that also use type hints i gave in the let? And will it make the fields final? |
| 14:58 | cemerick | rhickey: right, forgot about that. |
| 14:58 | rhickey | cemerick: gen-class does no lookup, mere indirections aren't necessarily impediments to hotspot inlining |
| 14:58 | Chouser | oh. public final |
| 14:59 | Chouser | :-) |
| 14:59 | rhickey | IntelliJ refactoring is awesome |
| 14:59 | AWizzArd | hmm |
| 14:59 | AWizzArd | why? |
| 15:00 | AWizzArd | Will it change all affected files for you automatically? |
| 15:00 | Chouser | whoa! public non-final!? |
| 15:00 | Chouser | ,(let [foo 1, obj (fn [] (prn foo))] (set! (.foo obj) 10) (obj)) |
| 15:00 | clojurebot | java.lang.NullPointerException |
| 15:00 | Chouser | ,(let [foo 1, obj (fn [] (prn foo))] (.foo obj)) |
| 15:00 | clojurebot | 1 |
| 15:01 | Chouser | anyway, I still can't figure out how to tell gen-class this method wants a byte array arg |
| 15:01 | clojurebot | this is not a bug |
| 15:02 | AWizzArd | What I did today was writing a very short Java class, where PF stands for private final class Token { PF String token; PF int start; PF int end; } plus getters for these three fields. |
| 15:02 | rhickey | "[B" |
| 15:03 | AWizzArd | My Lexer will create instances of Token. That was dramatically speeding up my code. Before I used java.util.HashMap instead of Token. |
| 15:03 | Chouser | java.lang.ClassNotFoundException: java/lang/[B |
| 15:03 | clojurebot | ☕ |
| 15:04 | AWizzArd | I guess this is still more performant than using a structmap, but of course those can't grow. |
| 15:04 | AWizzArd | rhickey: will those fns make use of the type hints I gave in the let? |
| 15:04 | Chouser | hm... maybe it works with (gen-class ...) but not (ns ... (:gen-clas ...)) |
| 15:05 | rhickey | AWizzArd: yes |
| 15:05 | AWizzArd | oh that's good then |
| 15:05 | mebaran151 | how can one namespace export symbols into another? |
| 15:07 | rhickey | mebaran151: can't |
| 15:08 | Chouser | (gen-class :name my.Foo :methods [#^{:static true} [newArrayInputStream [int "[B"] java.io.InputStream]]) |
| 15:08 | mebaran151 | well then what would be the best way to setup a set of support macros that another namespace can require or use? |
| 15:08 | AWizzArd | mebaran151: you can use things that you def'ed or defn'ed in one namespace from others. |
| 15:09 | Chouser | mebaran151: if namespace a.A has (defmacro aaa ...), then in namespace b.B you can (require 'a.A) and use (a.A/aaa ...) |
| 15:10 | rhickey | ,(Class/forName "[B") |
| 15:10 | clojurebot | [B |
| 15:10 | mebaran151 | I know that, but the problem is the macros wrap functions in the namespace to which they are imported |
| 15:11 | Fossi | wow. that is one ugly expression :D |
| 15:11 | hiredman | eh |
| 15:11 | Chouser | mebaran151: ah.... if the functions aren't marked private, it should Just Work. |
| 15:11 | hiredman | Chouser: he might not be using ` |
| 15:12 | Chouser | hm. true. |
| 15:12 | mebaran151 | let me set up the problem a little more clearly: I have a macro that wraps up transaction handling: it requires a definition of open-transaction commit and abort |
| 15:12 | hiredman | mebaran151: so use ` so your macros emit namespace qualified symbols |
| 15:12 | rhickey | so, (new [interfaces] :as this ...) or (obj this [interfaces] ...) |
| 15:12 | hiredman | the second one |
| 15:12 | mebaran151 | I do use ` |
| 15:13 | mebaran151 | let me just do a pastie: it'll be easier to explain |
| 15:13 | cemerick | rhickey: first one |
| 15:13 | cemerick | (with 'this' as default?) |
| 15:14 | Chouser | I prefer 'new' over 'obj' |
| 15:14 | rhickey | cemerick: no defaults since these will often be wrapped by macros (think fn) and thus injected names won't be apparent |
| 15:14 | rhickey | anyone remember thisfn? |
| 15:14 | cemerick | indeed |
| 15:14 | rhickey | didn't work |
| 15:15 | hiredman | I'd prefer a minimum of :as :with :doing :etc |
| 15:15 | rhickey | obj could be object or instance |
| 15:16 | lisppaste8 | mebaran151 pasted "Macro Injection" at http://paste.lisp.org/display/84280 |
| 15:16 | cemerick | presuming this is going to be used often (still a proxy replacement...?), shorter is better |
| 15:17 | rhickey | a lot of uses don't need 'this' |
| 15:17 | Chouser | It seems plausible that a few common use cases could be implemented as macros on top of 'new', just like fn will be |
| 15:17 | rhickey | yes |
| 15:17 | cemerick | it's hard to get better than new. could be (new [interfaces] ...), skipping 'this' |
| 15:18 | mebaran151 | see, those macros rely on a parent namespace supplying implementations of cursor, transaction, commit etc |
| 15:18 | rhickey | cemerick: this optional in either case |
| 15:18 | mebaran151 | how would I get these functions to use the functions in the namespace into which they were imported |
| 15:18 | cemerick | then I'd go with option #2, but with 'new' |
| 15:18 | hiredman | oh |
| 15:18 | rhickey | cemerick: that's not being offered, ambiguous with current new |
| 15:19 | hiredman | mebaran151: so you *want* unsafe macros? |
| 15:19 | cemerick | Oh, I figured the current new was being retired! |
| 15:19 | cemerick | nevermind then :-) |
| 15:19 | mebaran151 | I think I may |
| 15:19 | rhickey | current new is not being retired |
| 15:20 | mebaran151 | this code is useful for more than my bdb library: I thought it would be nice to somehow abstract it, though I haven't found a good way |
| 15:20 | rhickey | ,`~'foo |
| 15:20 | clojurebot | foo |
| 15:21 | rhickey | a name emitted like that will capture the name in the scope in which it is expanded |
| 15:22 | sgtarr_ | ,`~'bar |
| 15:22 | clojurebot | bar |
| 15:22 | rhickey | ,`rest |
| 15:22 | clojurebot | clojure.core/rest |
| 15:22 | sgtarr_ | oh. |
| 15:22 | rhickey | `~'rest |
| 15:22 | rhickey | ,~~'rest |
| 15:22 | clojurebot | java.lang.IllegalStateException: Var clojure.core/unquote is unbound. |
| 15:23 | rhickey | `~'rest |
| 15:23 | rhickey | ,`~'rest |
| 15:23 | clojurebot | rest |
| 15:23 | rhickey | there |
| 15:23 | rhickey | :) |
| 15:23 | ataggart | lol |
| 15:23 | sgtarr_ | what if someone gives an infinite loop to the bot? |
| 15:23 | Chousuke | I'd still like the args to newnew be just a map literal :/ |
| 15:23 | slashus2 | sgtarr_: It times out. |
| 15:23 | sgtarr_ | I see :) |
| 15:24 | hiredman | Chousuke: makes sense to me |
| 15:25 | hiredman | ,(new [Thing] {methodname ([args] stuff doing)}) |
| 15:25 | clojurebot | java.lang.IllegalArgumentException: Unable to resolve classname: [Thing] |
| 15:26 | Chousuke | hiredman: hmm, I meant the keyword args though. |
| 15:26 | Chousuke | hiredman: but that could work too :P |
| 15:26 | rhickey | and overloaded methods? |
| 15:27 | Chouser | #=(java.lang.Class/forName "[B") works |
| 15:28 | Chousuke | I mean, if newnew potentially could have parameters other than :as and :volatile, it would be easier to have them passed around in a real map, so macros using newnew can just look up the args they're interested in, or use merge to allow the user to override "defaults" easily. |
| 15:28 | rhickey | Chouser: seems like a bug in genclass' the-class |
| 15:28 | hiredman | (new [Thing] {methodname (([args] stuff doing) ([arg1 arg2] wild and crazy stuff))}) |
| 15:29 | rhickey | (some #{\.} strx) => (some #{\. \[} strx) |
| 15:32 | Chouser | gah! I didn't even want a byte array. sheesh. |
| 15:32 | rhickey | Chousuke: so, not about the methods, (new [interfaces] {:as this ...} ...) |
| 15:33 | Chousuke | yeah, like that. |
| 15:35 | Chouser | rhickey: I'm curious if you'd entertain syntax that avoids the sub-form (foo ...) when defining a newnew method foo |
| 15:35 | Chouser | proxy is currently one of the few places that (foo ...) means anything other than invoking foo. |
| 15:35 | rhickey | Chousuke: I don't care much, but those complaining about :as might |
| 15:35 | rhickey | Chouser: what about fn? |
| 15:35 | mebaran151 | , (fn inf [] (inf)) |
| 15:36 | clojurebot | #<sandbox$eval__5175$inf__5177 sandbox$eval__5175$inf__5177@300522> |
| 15:36 | mebaran151 | , ((fn inf [] (inf))) |
| 15:36 | clojurebot | java.lang.StackOverflowError |
| 15:36 | ataggart | don't poke clojurebot |
| 15:36 | Chousuke | mebaran151: you can't send it to an infinite loop that easily :) |
| 15:36 | Chouser | rhickey: as in (new [Foo] (fn fooMethod [] ...)) ? looks good to me. |
| 15:36 | rhickey | no, as in isn't fn an example? |
| 15:37 | ataggart | can someon paste the link to the newnew stuff pls |
| 15:37 | ataggart | looked but couldnt find it |
| 15:37 | Chousuke | http://www.assembla.com/wiki/show/clojure/New_new |
| 15:37 | ataggart | thx |
| 15:38 | rhickey | that's just an idea scratchpad, btw |
| 15:38 | ataggart | yup, I just wanted to be a bit less ignorant while following along with your conversation |
| 15:38 | rhickey | Chouser: (fn foo ([bar] ...) ...) |
| 15:39 | Chousuke | I think that's exempt because you don't usually call a literal vector :) |
| 15:39 | Chouser | oh, yes, multi-body fn's are also a (regrettable ;-) example |
| 15:40 | Chouser | but at least they start with a vector rather than a plain word. Still potentially confusing, but perhaps slightly less so. |
| 15:40 | rhickey | :( |
| 15:40 | Chousuke | ([foo bar] 1) is going to make anyone scratch their head, even if it is valid :P |
| 15:41 | rhickey | Chouser: what's your alternative? |
| 15:41 | ataggart | thankfully the vector is usally a name, so it's not so jarring, e.g., (myvec 1) |
| 15:41 | cemerick | the evils of macros, staring us straight in the face... :-D |
| 15:42 | Chousuke | though ([foo bar] bar) is potentially meaningful as a function call... |
| 15:42 | cemerick | Chouser: Quick! Pull out the :do-what-i-want option! :-P |
| 15:42 | Chouser | rhickey: I hadn't considered alternatives for mutli-body fn. But for 'proxy' and similar, I thought using (defm ...) or (method ...) might be nice |
| 15:42 | rhickey | Chouser: so for a multiple method interface you want to say method over and over? |
| 15:43 | Chouser | um. yes. |
| 15:43 | rhickey | not me |
| 15:43 | Chouser | I have to say def and defn over and over anyway |
| 15:43 | rhickey | Chouser: but they are not nested in anything |
| 15:44 | rhickey | here we have context |
| 15:44 | hiredman | Chouser: so you want boiler plate? |
| 15:44 | Fossi | what's a nice way of converting [1 2] to {:foo 1 :bar 2}? |
| 15:45 | Chousuke | ,(zipmap [:foo :bar] [1 2]) |
| 15:45 | clojurebot | {:bar 2, :foo 1} |
| 15:45 | hiredman | ,`zipmap |
| 15:45 | clojurebot | clojure.core/zipmap |
| 15:45 | Chouser | Clojure has this beautiful, possibly undervalued, and unique-among-lisps feature of (foo ...) in almost every context meaning 'use the function or macro named "foo"' |
| 15:45 | Fossi | knew it had to be one of those obscure functional things :) |
| 15:46 | rhickey | Chouser: I'm not disagreeing |
| 15:46 | Fossi | to back up Chouser: it occasionally stumps me with proxy as well |
| 15:46 | rhickey | but defm or method wouldn't be that either - you have to lose the parens |
| 15:47 | Chouser | perhaps better syntax hightlighting would be sufficient -- if the method names in 'proxy' and newnew stood out differently from fn calls. |
| 15:47 | Fossi | multiple ( below each other instinctively shouts 'do' to me |
| 15:47 | hiredman | ,((pl λx x) 1) |
| 15:47 | clojurebot | 1 |
| 15:48 | hiredman | or use a map for maping names to functionality |
| 15:48 | Chouser | heh. funky |
| 15:48 | angerman | would clojure be a vaiable choice for a json-rpc-client? |
| 15:48 | Fossi | a map wpuld be great |
| 15:49 | hiredman | angerman: sure |
| 15:49 | angerman | hiredman: I'm thinking about designing the interface with NetBeans Mantissa (or what it's called now) and use clojure to drive the gui |
| 15:50 | Chouser | (fn foo {[] (foo 1 2)} {[a b] (+ a b)}) |
| 15:50 | rhickey | Chouser: right now I'm building on my 'failure', each method clause will mimic named fn without the fn (methfoo [x] ...) (methbar ([x] ...) ([x y] ...)), but in no case are they doing anything the way calls or macros are |
| 15:50 | Chouser | hm |
| 15:50 | hiredman | Chouser: woa, that is really funky |
| 15:51 | Chouser | hiredman: oh, isn't that what you meant? |
| 15:51 | rhickey | Chouser: that will fail to support same-arity type overloading when types in metadata, as non-unique map keys |
| 15:51 | gulagong | i'm still having trouble with type-hints, i don't understand why won't compile this: |
| 15:51 | gulagong | (defn tester [#^longs x] (loop [a (long 0)] (recur (aget x 1 1)))) |
| 15:51 | hiredman | I mean something like (proxy [Thing] [] {someMethodname ([a] do stuff to a)}) |
| 15:52 | Chouser | rhickey: no, it's not a macro or fn call, any more than 'catch' or 'finally' is, but still it's a word at the front of a list that controls the meaning of that list. |
| 15:54 | Chouser | rhickey: oh, I see. you still want an (optional) level of nesting for same-named methods. |
| 15:54 | Chouser | instead of repeating the method name for each method arity or arg type difference |
| 15:54 | rhickey | Chouser: the repetition kills it for me, like having to say public over and over again in Java |
| 15:55 | Chouser | ok |
| 15:56 | rhickey | I'd consider a paren-free structural alternative |
| 15:56 | rhickey | but IMO fn and new are already saying 'what's in here is special' |
| 15:56 | rhickey | i.e. not a block-o-code |
| 15:57 | Chouser | ok |
| 15:57 | hiredman | and any macro can change things |
| 15:57 | Chouser | yeah, but that's not the point |
| 15:57 | rhickey | I take your point but the alternative would have to be really good, I personally like this use of parens for defining callable things, in addition to calling them |
| 15:57 | hiredman | so it's not like fn or whatever else is unique |
| 15:58 | Chouser | the problem here is reading other people's proxy block, not writing my own. |
| 15:58 | Chouser | rhickey: well, it's common in other lisps, right? (define (foo ...)) |
| 15:59 | rhickey | Chouser: parens are used for everything there though |
| 15:59 | Chouser | so true |
| 16:00 | rhickey | I'm looking at future-call in core.clj, I don't get confused, but I could see if it was badly formatted... |
| 16:01 | rhickey | I agree highlighting the method names would be awesome |
| 16:02 | Fossi | at a glance, to me it looks like a big do blob |
| 16:02 | rhickey | content-handler in xml.clj is bigger, but... |
| 16:03 | Fossi | proxy being almost as short as do and being highlighted, then parantheses and methodnames |
| 16:04 | Chouser | this is more of an issue at first glance, I guess. It doesn't take a whole lot of studying to figure out that (foo [a b c] ...) is not actually building a vector and passing it to foo |
| 16:04 | rhickey | if the method names were highlighted it would be a complete non-issue, and possible I guess? |
| 16:05 | Fossi | i'd say yes |
| 16:05 | Chouser | it's more a matter of the initial thought "calling foo!? But what *is* foo" -- a feeling that generally overwhelmed me when trying to read any chunk of CL code. |
| 16:05 | Fossi | changing the methodnames to :foo (highlighting them) makes it a whole lot more recognizable |
| 16:06 | Chouser | I haven't looked at the vimclojure highlighting stuff at all. I assume it would be possible. |
| 16:06 | rhickey | Chouser: as I said, I agree and that's why Clojure is the way it (mostly) is |
| 16:06 | rhickey | the exceptions are just in contexts where you will be defining things you call with parens |
| 16:07 | Fossi | and sorry for interrupting, but clojure stacktraces *suck*. |
| 16:08 | Chouser | Fossi: huh. Well, nobody's ever complained before, so... |
| 16:08 | Fossi | well, it's halfways the jvm's fault |
| 16:08 | Fossi | "java.lang.ClassCastException: clojure.lang.PersistentArrayMap"... so what. |
| 16:08 | Chouser | Fossi: I'm kidding, BTW. People make such statements quite frequently. |
| 16:08 | rhickey | Fossi: are you using Slime? |
| 16:09 | duncanm | i think the stacktraces is recent slime has gotten better |
| 16:09 | Fossi | unfortunately not when i develop for android :) |
| 16:10 | angerman | does clojure-mode honor some a classpath setting? |
| 16:10 | rhickey | Slime (used to/still does?) have the problem of not putting the root cause first, like the repl does |
| 16:10 | technomancy | rhickey: that's fixed as of about a month ago |
| 16:10 | technomancy | but yeah, that was a big problem |
| 16:11 | rhickey | technomancy: great! |
| 16:11 | technomancy | also it colorizes clojure internals differently, which helps a lot. |
| 16:11 | duncanm | rhickey: i'm still interested in adding some type-coercion bits to the compiler so that a single Fn can be used in place of a one-method iface, and a Map<keyword, Fn> be used as a general interface |
| 16:12 | technomancy | angerman: you may be looking for the swank-clojure-extra-classpaths variable |
| 16:12 | duncanm | i've been writing some Swing code, and having to write out the Listeners explicitly in proxy form is getting tiresome |
| 16:12 | Fossi | also , the parser throws some real bad things at you. i mean "ArrayIndexOutOfBoundEexception"? come on :) |
| 16:14 | Chouser | duncanm: have you tried to write a macro to do that? |
| 16:15 | angerman | technomancy: thanks. yes that looks pretty much like what I want :) |
| 16:16 | duncanm | Chouser: but that's still one more set of parens, right? |
| 16:16 | clojurebot | Who?? |
| 16:16 | arohner | I'm using a cond inside a let to determine the value of a variable. the variable is a boolean, and the default case is "true false". that looks really weird |
| 16:16 | arohner | i.e. (let [foo (cond (bar) true true false)]) |
| 16:16 | arohner | is there any way to improve that? |
| 16:17 | Chouser | duncanm: not sure exactly what you're aiming for, but maybe (new-listener TheType [e] (do-something-with e)) ? |
| 16:17 | trb | is there a downloadable doc bundle or a way to generate one so i can have some sort of api reference offline? |
| 16:17 | duncanm | Chouser: but avoiding having to specify the listener type is part of the goal here |
| 16:18 | Chouser | huh |
| 16:18 | duncanm | if i need to type out (new-listener TheType ...), then I could just write out the proxy form |
| 16:18 | duncanm | which is okay (except that clojure-mode in emacs kinda indents it a little funny) |
| 16:19 | Chouser | duncanm: what do you expect the compiler to do? use the type expected by the outer form to control the class generated by the inner form? |
| 16:19 | duncanm | right |
| 16:19 | StartsWithK | i have something like (on widget :actionPerformed (fn [e] ...)) |
| 16:19 | rhickey | arohner: ? |
| 16:19 | cemerick | yeah, I can see that working. (.addActionListener button #(blah)) ends up being a reflective call anyway, so it seems possible to try to build a reasonable proxy in that situation |
| 16:19 | duncanm | i was looking at the compiler source |
| 16:19 | liron | re multi-body fns, i think it's wrong that this doesn't work: (fn ([] (recur 1))([x] (println x))) |
| 16:19 | cemerick | not sure if rhickey would appreciate whatever additional overhead that causes in Reflector, though. |
| 16:20 | Chousuke | arohner: (boolean (bar))? |
| 16:20 | duncanm | if you find a Fn param, and the matching arg is an interface (of some type T), then modify the arg list by replace the Fn with an interface instance |
| 16:20 | ataggart | arohner (let [foo (bar)] ...) |
| 16:20 | duncanm | and check arity |
| 16:20 | StartsWithK | as no two listeners on swing share the same name, with reflection you can find where :actionPerformed is defined and construct a proxy for that listener |
| 16:20 | Chousuke | arohner: usually, you use a keyword for the "true" case of cond though |
| 16:20 | duncanm | StartsWithK: that's a cute idea! |
| 16:21 | Chousuke | arohner: eg. (cond foo bar :else zonk) |
| 16:21 | arohner | Chousuke: ah! thanks |
| 16:21 | arohner | right, keywords are true |
| 16:21 | StartsWithK | duncanm: i don't have a time today, but if you'll be here in a day or two i can rip it out and add some docs |
| 16:22 | duncanm | StartsWithK: i'm always in this channel |
| 16:22 | rhickey | duncanm: I don't want to add that kind of gimmick to the language semantics. but a macro could look at the name addActionListener and figure out a lot |
| 16:22 | StartsWithK | duncanm: ok, i'll find you then |
| 16:23 | StartsWithK | rhickey: not all listener handle just one event |
| 16:23 | StartsWithK | *s |
| 16:23 | StartsWithK | so reflection is needed in the end |
| 16:23 | rhickey | reflection can happen in a macro |
| 16:25 | duncanm | ha |
| 16:25 | StartsWithK | i used, my own multimethod, with :default finding a right listener and adding a new .addMethod so i can cache the results |
| 16:25 | StartsWithK | never tested is there any improvement in speed |
| 16:25 | rhickey | a macro could know what classes are imported in the namespace, look for addActionListener methods... |
| 16:41 | angerman | technomancy: I see clojure-http-client is in your github repo, does it use httpclient from apache? |
| 16:43 | technomancy | angerman: no, it uses JDK functionality only |
| 16:44 | angerman | technomancy: was there a certain reasoning behind that? just asking, so I understand :) |
| 16:44 | technomancy | angerman: there's currently no good way to handle dependencies in Clojure |
| 16:45 | angerman | technomancy: you mean to handle the apache dep? |
| 16:45 | technomancy | I don't think putting jars in a lib/ directory is a good idea; as soon as you get more than a couple of them it becomes a lot of manual labor |
| 16:45 | technomancy | yes. |
| 16:45 | angerman | technomancy: does it support ssl? |
| 16:46 | technomancy | angerman: heh; I'm not sure. It does if net.java.URL does. =) |
| 16:49 | angerman | technomancy: hmm seems to if one sets some properties |
| 16:51 | gulagong | i don't get it.... can somebody please tell me how to typehint arrays of more than one dimension? |
| 16:51 | gulagong | i tried: |
| 16:51 | gulagong | ,(defn test [#^"[[Ljava.lang.Long;" x] (loop [a (long 0)] (recur (aget x 1 1)))) |
| 16:51 | clojurebot | DENIED |
| 16:56 | Chouser | 'aget' is only inlined by Clojure for one-dimentional arrays |
| 16:56 | Knekk | Chouser: why's that? |
| 16:57 | gulagong | ok, so wrap in a fn |
| 16:57 | gulagong | ? |
| 16:57 | gulagong | and it works |
| 16:57 | gulagong | nope, thats silly |
| 16:57 | angerman | technomancy: so as far as I understand it, all jars in ~/.clojure/ will be added to the classpath right? |
| 16:57 | Chouser | Knekk: I couldn't say. possibly just hasn't been done yet. |
| 16:57 | Chouser | gulagong: try another let level |
| 16:58 | Chouser | ,(expression-info '(let [#^"[[J" x 5] (aget (let [#^"[J" y (aget x 1)] y) 1))) |
| 16:58 | clojurebot | {:class long, :primitive? true} |
| 16:58 | Knekk | not that it's hard to manage a one-dimensional array to support more-dimensions, if performance is an issue |
| 16:58 | technomancy | angerman: totally depends on how you invoke clojure |
| 16:59 | gulagong | you have to calculate the positions ;) |
| 16:59 | gulagong | hui, chousers code is very compact |
| 16:59 | Knekk | sure, but it's not hard |
| 17:00 | Fossi | n8 |
| 17:00 | Knekk | and probably only marginally slower than letting the compiler do it for you |
| 17:00 | angerman | technomancy: with slime |
| 17:01 | Chouser | gulagong: you really want a 2-dimentional array of Object like you have? Or of primitive long? |
| 17:01 | gulagong | chouser: no, i want primitive long |
| 17:02 | Chouser | (defn foo [#^"[[J" x] (loop [a (long 0)] (recur (aget (let [#^"[J" y (aget x 1)] y) 1)))) |
| 17:02 | Chouser | so something like that |
| 17:03 | technomancy | angerman: anything more than one command to put all the dependencies where they need to be is suboptimal in my book |
| 17:03 | Chouser | not sure why it needs another 'let' instead of just using #^"[J" on the inner (aget ...), but it apparently does. |
| 17:03 | Chousuke | Chouser: That's just horrible :P |
| 17:04 | gulagong | ok, i see that it works |
| 17:04 | Chousuke | but then again, dealing with arrays in Clojure never seems to lead to anything good. |
| 17:04 | gulagong | but isn't it overhead - that nested let |
| 17:05 | angerman | technomancy: No, I ment when I start slime with M-x slime. I thought it would be wrapped into the swak-clojure-extra-classpath args. so that every .jar in ~/.clojure would be added to the classpath |
| 17:05 | Knekk | isn't there a better way to represent #^"[J]" ? |
| 17:05 | Chousuke | gulagong: nah. |
| 17:05 | Chouser | gulagong: no, in this case it should run faster |
| 17:06 | gulagong | thanks guys |
| 17:06 | Chousuke | gulagong: let doesn't really "do" anything. it just establishes a binding. |
| 17:06 | gulagong | you are great :) |
| 17:06 | Chouser | Knekk: yes! |
| 17:06 | Chouser | ,(defn foo [#^"[[J" x] (loop [a (long 0)] (recur (aget (let [#^longs y (aget x 1)] y) 1)))) |
| 17:06 | clojurebot | DENIED |
| 17:06 | Chouser | Knekk: thanks for the push. :-) |
| 17:07 | Knekk | I mean, I like JVM type definition strings as much as the next guy... XNI rules my world. |
| 17:07 | Knekk | err, JNI |
| 17:07 | technomancy | angerman: that's the default value of the extra-classpaths variable, but I never use the default value |
| 17:07 | Chouser | so #^longs is the same as #^"[J", but I don't think there's anything for #^"[[J" |
| 17:07 | Chousuke | it should be "longss" :P |
| 17:08 | Chousuke | but of course it isn't. |
| 17:08 | Chouser | Chousuke: my first thought too. |
| 17:08 | gulagong | isn't #^"[[J" short for #^"[[Ljava.lang.Long" ? |
| 17:09 | Chouser | no |
| 17:09 | Chouser | because long is not the same as Long |
| 17:09 | angerman | technomancy: it somehow fails to set up the classpath in slime for me :( |
| 17:10 | gulagong | ok |
| 17:10 | gulagong | i have to read a lot of code to learn all this ;) |
| 17:11 | Knekk | angerman: I ended up using a shell script to set up the path. Slime starts up the lisp using the script |
| 17:11 | angerman | technomancy: hmm ahh now |
| 17:11 | angerman | args |
| 17:11 | angerman | no |
| 17:12 | Chouser | gulagong: you and me both. 95% of the Java I know has been learned using Clojure. |
| 17:13 | gulagong | hehe |
| 17:13 | gulagong | that gives hope :) |
| 17:27 | chopmo | This may be a stupid question, but what is the easiest way to read binary data from a file? |
| 17:28 | Chouser | the same way you would in Java |
| 17:28 | chopmo | I'm trying to store a PDF file as an attachment to a couchdb document. |
| 17:28 | chopmo | Chouser: Thanks. I was trying to use slurp. |
| 17:28 | drewr | that only works for text |
| 17:28 | drewr | ..but I guess that's why you asked your q :-) |
| 17:29 | chopmo | drewr: Right :-) |
| 17:29 | drewr | <-- captain obvious |
| 17:39 | angerman | technomancy: I'm having issued to figure out why the swank variable is not passed to slime :( |
| 17:41 | technomancy | angerman: what is your swank-clojure-extra-classpaths variable set to? |
| 17:43 | angerman | technomancy: it's set to ~/.clojure/ |
| 17:43 | angerman | what happes though is, that slime-lisp-implementations ends up with two items |
| 17:43 | angerman | for cojure |
| 17:44 | technomancy | angerman: that's probably not what you want. you probably want it set to a value of all the jars *in* ~/.clojure. |
| 17:44 | technomancy | which is what it should be out of the box |
| 17:45 | angerman | technomancy: that is what I expected to happen too. |
| 17:46 | technomancy | angerman: my suggestion is to unpack all your dependencies inside your project directory. See what I wrote at http://technomancy.us/126#projects |
| 17:46 | technomancy | that way your classpath will remain simple |
| 17:48 | angerman | technomancy: I think we have a communication issue :) |
| 17:49 | dysinger | that's a Led Zepplin song right ? |
| 17:49 | technomancy | angerman: I think I know what you're trying to do... I'm saying that's more complicated than my way. |
| 17:49 | angerman | Here's what I expected to happen: "When I do M-x slime" I expected slime to start up with the -cp set to swank-clojure-extra-classpaths |
| 17:50 | technomancy | angerman: is swank-clojure-extra-classpaths a list or a string? |
| 17:51 | angerman | now the issue seems to be that swank-clojure-extra-classpaths is for some reason not evaluated to what it's set (when (file-directory-p ....) ( ... ) ) |
| 17:52 | angerman | later during load time (eval-after-load "slime" is invoked and clojure is added to the slime-lisp-implementations |
| 17:53 | angerman | at that point it binds the -cp to what it was at load time. :( |
| 17:53 | angerman | Therefore if I evaluate the (when (file...) (...)) and set swank-clojure-extra-classpaths |
| 17:54 | angerman | then run the (eval-after-load section from the swank-clojure-autoload.el |
| 17:54 | angerman | and finally run (setq slime-lisp-implementations (cdr slime-lisp-implementations)) the first clojure item with the wrong -cp is removed |
| 17:54 | technomancy | angerman: maybe if you pasted some code for what you're doing it would help. |
| 17:55 | angerman | and the second one is retained |
| 17:55 | technomancy | angerman: are you using slime with CL and clojure? |
| 17:55 | angerman | no. just clojure |
| 17:56 | angerman | technomancy: I'll try something |
| 17:56 | technomancy | angerman: I think you're making this more complicated than it needs to be. Did you read the article at http://technomancy.us/126? |
| 18:00 | dysinger | I hope I don't have that much trouble with I add CL to the slime/swank mix |
| 18:00 | dysinger | Setting up for clojure only was easy |
| 18:00 | dysinger | oh - angerman - you said you were just using clojure ? |
| 18:01 | dysinger | Yeah setup a project elisp function like technomancies and you'll be good |
| 18:01 | dysinger | lol s/technomancies/technomancy\'s |
| 18:19 | AWizzArd | rhickey: when closures are compiled into pojos, then Drools could maybe work on those? |
| 18:22 | AWizzArd | ~seen kotarak |
| 18:22 | clojurebot | kotarak was last seen quiting IRC, 5769 minutes ago |
| 18:22 | hiredman | ,(/ 5769 60 24) |
| 18:22 | clojurebot | 641/160 |
| 18:23 | hiredman | ,(int (/ 5769 60 24)) |
| 18:23 | clojurebot | 4 |
| 18:32 | AWizzArd | Btw, how long can a CP become? :) |
| 18:33 | AWizzArd | Is there some size limit such as 1024 chars or something like that? |
| 18:38 | arohner | AWizzArd: what is a CP? |
| 18:38 | arohner | classpath? |
| 18:39 | AWizzArd | yes |
| 18:40 | arohner | well, your shell has a size limit |
| 18:40 | arohner | wouldn't surprise me if you could hit that limit before the classpath limit |
| 18:41 | dysinger | hmm - I think you are doing it wrong if you need to worry about CP length |
| 18:42 | AWizzArd | I just downloaded Drools. It comes with 19 .jars and 43 more dependencies. |
| 18:43 | AWizzArd | So, that would mean adding funny 62 more .jar files to the classpath. |
| 18:43 | AWizzArd | or maybe some wildcard syntax can help |
| 18:43 | arohner | JDK 6 supports * in cp |
| 18:44 | AWizzArd | Especially because I don't want to explicitly write out those damn names, or write a script which does so. |
| 19:30 | duncanm | so if i have a list of coords, Xs and Ys |
| 19:30 | duncanm | (map #(Point. %1 %2) xs ys) will only get me the diagonal |
| 19:31 | duncanm | is there a concise way to write it so i get all the points (x, y) |
| 19:31 | duncanm | right now, i have (flatten (map (fn [x] (map (fn [y] (Point. x y)) ys)) xs)), which is ugly |
| 19:32 | arohner | (for [x (range ...) y (range ...)] #(Point. %1 %2)) |
| 19:32 | duncanm | ahh |
| 19:46 | rhickey | very nice story, thanks: http://infolace.blogspot.com/2009/07/coming-to-clojure.html |
| 21:19 | ataggart | what's the trick to getting new code from an assembla ticket into github? |
| 21:20 | ataggart | e.g., http://www.assembla.com/spaces/clojure-contrib/tickets/7-Logging-functions |
| 21:22 | arohner | http://msdn.microsoft.com/en-us/devlabs/ee334183.aspx |
| 21:29 | Chouser | ataggart: you just need a contrib committer to adopt your project |
| 21:30 | ataggart | and how does one foment that? |
| 21:37 | Chouser | good question. no committer asked for it on the google group? |
| 21:38 | ataggart | asked for it to be written? no, I needed it for myself and thought others might find it useful |
| 21:40 | ataggart | at least 3 others did think it was useful, judging by the response on the groups thread |
| 21:41 | Chouser | I just don't know how this should all work. |
| 21:42 | ataggart | likewise |
| 21:42 | Chouser | I think your best bet is for someone with commit privs to suddenly need logging features and to discover your patch. :-) |
| 21:43 | Chouser | I dunno if any committer should commit your code without evaluating it. Might be an okay policy, or maybe not. |
| 21:44 | ataggart | that certainly is one way to keep stuff out of contrib |
| 21:44 | ataggart | hmm |
| 21:44 | ataggart | is there a list of contrib commmitters beyond http://clojure.org/contributing |
| 21:44 | ataggart | because I see Tom Faulhauber had last commit in github |
| 21:45 | Chouser | I don't think so, unfortunately. Used to be on google code, but I don't see anywhere on github to get a list of who has commit privs. |
| 21:45 | ataggart | and he was one who responded positively t the thread, but he's not on that list |
| 21:45 | ataggart | hmph |
| 21:46 | ataggart | even if he had github privs would that mean he also has some more authority in assembla? |
| 21:46 | ataggart | this is all very confusing |
| 21:47 | Chouser | ok, sure enough. he has a contrib commit with no "sign-off", so he must have privs. |
| 21:47 | Chouser | you should ping him, either directly or via the google group, and se if he'll put it in for you. |
| 21:49 | Chouser | I think anyone with a CA in can ask for membership on clojure and contrib assembla groups |
| 21:50 | Chouser | that's all the privs anyone there gets, other than rich himself. I think. |
| 21:50 | ataggart | well I am in assemabla |
| 21:51 | ataggart | and reading the istructions, it says all code needs to be submitted via a ticket |
| 21:51 | ataggart | kinda of dead-ends there |
| 21:51 | ataggart | who knows, maybe I have commit privs in github |
| 21:52 | ataggart | I'm really not incluned to bug eople about doing stuff for me though |
| 21:56 | Chouser | ataggart: well, that's how it is for now. If he wants it, as he seems to, he'd probably be happy to check it in for you. |
| 21:58 | Chouser | I guess another option would be to write rich and ask for contrib commit privs. I guess that's how that works, not sure. |
| 23:54 | tomoj | what's fn*? |
| 23:56 | hiredman | tomoj: the real fn primitive |
| 23:57 | hiredman | fn is listed as "primitive" but it really is a macro that emits a call to fn* |
| 23:57 | tomoj | ah, I see |
| 23:57 | tomoj | trying to figure out the lazy-seq source |
| 23:57 | tomoj | thanks |