#clojure logs

2010-10-08

00:00tomojno
00:01tomoj,(supers (class :foo))
00:01clojurebot#{java.util.concurrent.Callable java.lang.Runnable java.io.Serializable java.lang.Object clojure.lang.Named java.lang.Comparable clojure.lang.IFn}
00:01tomojkeywords implement IFn
00:01tomojthe reader also optimizes out some keyword access I believe
00:01tomojthe compiler I mean
00:01nollidjalright. thanks.
00:02tomoj,(supers (class #{}))
00:02clojurebot#{clojure.lang.IPersistentSet java.util.concurrent.Callable clojure.lang.IMeta clojure.lang.Counted clojure.lang.Seqable java.lang.Runnable java.io.Serializable java.util.Collection java.lang.Object clojure.lang.IEditableCollection java.util.Set clojure.lang.IObj clojure.lang.APersistentSet clojure.lang.IPersistentCollection java.lang.Iterable clojure.lang.AFn clojure.lang.IFn}
00:23nollidjis there any notion of providing a function to pre-fill fields in a record when it is created with (record-thing. foo bar baz) ?
00:34chousernollidj: nothing standard yet. Either roll your own or use one of the macros that are floating around.
00:52duncanmdum de dum
00:58coldheadyo ChanServ can i get some ops too?
00:59nollidji've seen some mentions of pmap on the web. is there anything else to make running stuff in parallel easy?
01:02nollidje.g., a parallel reduce that partitions the set being reduced over, reduces the partitions, and then combines their values could be nice
01:05chousernollidj: there's been some work on that, but there's nothing really application-ready as far as I know
01:06chousernollidj: clojure's data structures (hash-maps, sorted-maps, vectors) are trees internally and thus really well suited to parallel processing, like a parallel-reduce
01:07chouserthere's a 'par' branch on the main clojure repo on github that uses the forkjoin framework to do that sort of thing, but I think it's based on a rather older version of clojure.
01:08nollidji know it's deprecated
01:08chouserno, that's something else I think
01:08nollidji was hoping someone had whipped something up in contrib. oh well. a parallel reduce should be pretty easy to implement, except for the obstacle of not executing too many futures at once
01:08nollidjmaybe it's a separate par library
01:08chouserthe clojure.par namespace or something is even older and deprecated
01:09KirinDaveThat's like pre-1.1, right?
01:09chouserhm, it has chunks, so that's circa 1.1
01:11chouserwow, July 2009.
01:12chouserI wonder how hard it would be to merge
01:12chousergo go gadget git
01:15KirinDaveI'm reading Joy of Clojure for the first time in a few months.
01:15KirinDaveSome of these macros fogus has in here are pretty epic. :)
01:15chouser:-)
01:16chouser"epic macro" sounds really impressive
01:16KirinDaveI'm looking at as-futures
01:16KirinDaveWhich is a pretty gnarly way to go about a sublanguage.
01:16chouseroh, yeah. that one messes with your head, doesn't it?
01:16KirinDaveIt's straightforward
01:17KirinDaveBut
01:17KirinDaveThe decision to partition via => is pretty amusing.
01:18KirinDaveI thought that was frowned upon in clojure's culture.
01:19KirinDaveMaybe things are changing? :)
01:19chouserthis is chapter 8, right?
01:20chouseroh, 11
01:20KirinDave11.
01:20KirinDaveListing 11.8
01:21KirinDaveI'd be a hypocrite of the worst order if I said I disapproved, but I've gotta confess it unsettles me a little walking through that de-structuring. :)
01:21KirinDaveIt's a clever and useful macro tho.
01:22chouserhmmm
01:22chouser(doc condp)
01:22clojurebot"([pred expr & clauses]); Takes a binary predicate, an expression, and a set of clauses. Each clause can take the form of either: test-expr result-expr test-expr :>> result-fn Note :>> is an ordinary keyword. For each clause, (pred test-expr expr) is evaluated. If it returns logical true, the clause is a match. If a binary clause matches, the result-expr is returned, if a ternary clause matches, its result-fn, which must b
01:24chouserI think the more narrow an embedded DSL's usage, the more warranted (and useful) syntaxy features are likely to be.
01:24hiredmanhttp://gist.github.com/616395
01:24chouserThat's not to say Rich has approved every line of code in the book. :-)
01:24chouserhiredman: that's yours? looks pretty tidy.
01:25vacho(apply f args* argseq)
01:25hiredman:)
01:25vachowhy do I get error when I run that?
01:25vacho#<CompilerException java.lang.Exception: Unable to resolve symbol: f in this context
01:25chouserhttp://gist.github.com/560765#file_abomination_desolation.clj
01:26hiredmanwith-db transforms that into thr traditional nested functions and binds for a monad
01:27jcromartiechouser: how could you
01:27KirinDavehiredman: Is that purely functional then?
01:28hiredmanit's a state monad under the covers
01:28jcromartieI was demonstrating the power of macros to a friend. I wrote: (defmacro dont)
01:28KirinDavechouser: Haha @ your abomination macro
01:28KirinDavechouser: Next up, (loop ...)
01:29chouserheh. can't use ; in imperative 'for' loop
01:30KirinDaveOkay.
01:30KirinDaveas-promises is actually starting to freak me out.
01:31KirinDaveerr, with-promises
01:31hiredmanmmm promises
01:40chouserI must really not want to work on these slides. I'm finding the most ridiculous other tasks so interesting...
01:48vachocan someone please explain this from the book: (get a-map key not-found-val?)
01:48vachoget is the function name, is "a-map" a parameter?
01:53tomojyes
01:54tomojyou're callin gthe 'get' function and passing it a-map, key, and not-found-val
01:55tomoj,(get {:foo 1 :bar 2} :foo :not-found)
01:55clojurebot1
01:55tomoj,(get {:foo 1 :bar 2} :baz :not-found)
01:55clojurebot:not-found
01:56tomoj,({:foo 1} :foo 3)
01:56clojurebot1
01:56tomoj,(:bar {:foo 1} 3)
01:56clojurebot3
02:15lenwmorning all
02:46nollidj= is the same as java's .equals(). how can i do identity comparison between two data structures?
02:47lenw(identical? x x)
02:48nollidjthank you. that seems to do it.
02:48lenwnp
02:48hiredmannollidj: = is not the same as .equals
02:49nollidji'm using it on clojure hashmaps, so in this case it is
02:50nollidjmodulo working on nil
03:14yayitsweihello folks
03:15yayitsweianyone used clojure on app engine, and experienced very slow deployment times?
03:15yayitsweilike 10 min. to upload a simple "hello world" WAR
03:18kryftyayitswei: That's a great nick. :)
03:19yayitsweithanks :) it helps some people pronounce my name
03:19kryftyayitswei: (I know almost nothing about clojure, so I'm just here to make irrelevant comments.)
03:19kryftHehe
03:19yayitsweihaha. there's also #clojure-casual
03:19kryftYou'd think wei wasn't hard to pronounce. :P
03:20kryftyayitswei: Well, to be fair, I guess I do *mostly* ask relevant questions. ;)
03:21yayitsweinice. I'm relatively new too but i could try to tackle some of your questions
03:25kryftyayitswei: I don't have any specific questions right now, but thanks. :) I joined the channel a few days ago to find out what book would best suit my needs (guess what chouser recommended?)
03:26kryftyayitswei: I've since bugged people with some general questions such as "what kind of numeric performance can you expect".
03:26tomojkryft: you're the data mining dude, right?
03:27tomojmy memory is terrible...
03:27kryfttomoj: Machine learning, yes. :)
03:28tomojplease stick around
03:28yayitsweikryft: nice. i just finished stuarthalloway's book
03:28kryfttomoj: I don't really do what most people would call data mining, but of course all machine learning can be seen as mining data for useful knowledge.
03:28yayitsweikryft: hence the performance queries?
03:30kryftyayitswei: Yes. :) So far I've used C++ for things that need to be fast, and while I actually like C++ more than most people who are drawn to more elegant languages, it would be nice if I could get 'close enough' performance from clojure.
03:30kryftyayitswei: (Which I should be able to, based on what people here have told me and what I've read on the web. So yay!)
03:31kryfttomoj: Don't worry, I will. :)
03:33yayitsweikryft: if you ever end up using clojure's STM's for concurrency let me know, I'd be interested to hear your experience with them
03:35tomojkryft: hadoop?
03:37kryfttomoj: Come again?
03:37tomojdo you use it?
03:38tomojI use it and clojure and am wondering about how other people do so
03:38tomojbecause my workflow sucks
03:39tomojlein jar and java -cp .......... clojure_hadoop.job -job foo.bar/baz-job -input baz/input -output baz/output
03:39tomojthen .../bin/hadoop -text baz/output to see the output
03:40tomojand any exceptions have terrible uninformative stacktraces
03:40tomojand I _know_ there has got to be a better way
03:59ordnungswidriganybody using clojure-mode from technomancy repository?
04:01hiredmaneveryone does
04:02ordnungswidrignavigation in clojures-1.2.0/test.clj with CTRL-. fails for me
04:03ordnungswidrig"No known definition for: do-report (in ^{:author)" I suppose it tails parsing the ns declaration correctly
04:06kryfttomoj: Ah, no, I have no idea what hadoop is. :D
04:09tomojorly
04:09tomojit's a mapreduce framework
04:09tomojand some other stuff too I guess
04:10esjmorning folks
04:11kryftMorning esj!
04:11kryftDid you dream of clojure?
04:11kryftI only dreamt of not being able to sing. ;/
04:15esji dreamt I could program clojure.... then I woke up and it was as before :(
04:16kryft:D
04:20tomojany alephers around?
04:20kryftesj: Actually I'm pretty sure I know what my first clojure project is going to be. I'm going to reimplement a dimensionality reduction algorithm that I originally wrote in C++.
04:21esjkryft: Super, if you'd like some help, I'm happy to try.
04:22kryftesj: Thanks, I probably will need some pointers (or at least they will be helpful) on the general design. :)
04:22esjno pointers - that's C++ remember !
04:23kryftesj: As in how to go from an object-oriented/generic framework to clojure-optimal.
04:23kryft:D
04:24ChousukeYour biggest problem will probably be figuring out how to express the algorithm without (too much) mutation.
04:24kryftHmm, I finally feel the desire to upgrade my core2duo E6600 from 2006.
04:24Chousuke(and whether it'll still be efficient if you do that.)
04:24tomojkryft: what kind of dimensionality reduction?
04:24clojurebotreduction is http://groups.google.com/group/clojure/msg/4f7e3ca1af3b8b18
04:24tomojI'm intensely interestion in that topic recently
04:24kryftChousuke: Right.
04:24kryfttomoj: It's an algorithm specifically designed for visualization.
04:25tomojPCA?
04:25Chousukeone possibility is inventing a completely new algorithm that fits the functional model better :P
04:25kryfttomoj: No, a new algorithm. :) http://www.jmlr.org/papers/volume11/venna10a/venna10a.pdf <- here's our paper
04:26kryftChousuke: It's basically just minimizing a cost function using conjugate gradient descent, so it should fit the paradigm quite easily
04:27tomojwaay over my head
04:28kryfttomoj: What did you do again? My memory is as terrible as yours, I'm afraid. Were you the guy working for a startup?
04:28tomojyeah
04:28Chousukethe more mathy terms you can use to describe it, the better fit it likely is for a functional implementation
04:28kryfttomoj: Ok, so I did remember.
04:28esjoff the top of my head I would agree that conj grad is gonna be suitable for recursion
04:29tomojI am wondering whether random projections could be used to provide feature vectors for training "contentful image" classifiers
04:29kryfttomoj: How is that paper way over your head, btw? I don't think it should be more complicated than the stuff you're doing. :)
04:29tomojtoo much unfamiliar jargon
04:29kryftAh
04:30tomojI still haven't finished undergrad and I am a philosophy major
04:30kryfttomoj: No wonder you ended up programming clojure. ;)
04:30esjtomoj: I don't know about 'random projections' specifically, but any dimensionality reduction is probably a good idea when feeding a classifier
04:30tomojI basically just read papers over and over and over until I get some little shred of intuition about the solution
04:30tomojthen try to implement it
04:31tomojthe question is, what is the high-dimensional space for contentful images?
04:31esjwhether the nature of the random porjection preserves the information most salient for your classification is probably going to be a matter of 'suck it and see'.
04:31tomojcan it involve edge-detection, etc?
04:31kryfttomoj: Define contentful?
04:31tomojexactly
04:31tomojI don't want to have to define it
04:31tomojI just want to provide a training set with some images marked as contentful and some not
04:32tomojI saw an old paper about this which used hand-devised features
04:32kryfttomoj: Fair enough, I meant "define" as in "what do you mean by this word". :) "This training set" is an acceptable answer.
04:33tomojyou seen the feature in facebook where when sharing a link, you can page through images in the page you're linking to?
04:33tomojcontentful images are the ones you want to show up in that list
04:33kryfttomoj: One thing you could do is try various feature extraction methods (eg. dimensionality reduction methods) for your training data and then see which ones seem to give you good separation.
04:33esjOK, so they're the images labelled 1 as far as your classifier is concerned ?
04:33tomojyeah
04:34tomojI am doing similar things with text
04:34tomojI hope there is a unified approach..
04:35tomojcan't quite understand the Johnson–Lindenstrauss lemma yet
04:36esjtomoj: kryft is correct, you just have to suck it and see.
04:36tomojwoohoo
04:36esjas I said yesterday, the general field of ML is quite unstructured.
04:36tomojit's unfortunate
04:37tomojsupposedly compling is taking a turn for better understood theoretical models
04:37tomojI guess if it works, though..
04:37esjif you cannot define a model that is able to describe the presence or absence of your features, then you have nothing to do but shotgun try everything
04:37kryfttomoj: Have you tried kernel pca?
04:37esjif do have a model, then all sorts of nice things open up
04:38tomojkryft: I don't know exactly how that goes, but I don't think its an option
04:38tomojI can't keep the original high-dimensional data around forever
04:38tomojI need to be able to incrementally add to the reduced space
04:38kryfttomoj: Do you need to learn the model incrementally as well?
04:39tomojyes
04:39kryfttomoj: Ok, so you need some kind of online algorithm then.
04:39tomojrandom indexing works for text
04:39kryfttomoj: I don't know much about that. (Well, actually I don't know much about anything. :)
04:39esjTomoj: Thats hard. For linear type projections you will be able to project new data down. However , the optimal projection found for the initial data set is is not going to remain correct.
04:39tomojinstead of starting in a high dimensional space, you just use a random projection and generate random low-dimensional vectors
04:40jcromartie*whoosh*
04:40esjalthough if your data is sufficiently nice, the optimum wont move too much and the projection will remain useful. But its a prayer.
04:40tomoje.g. for a wordbag LSA approach you just add the random low-d vectors for each word in the context window to the accumlated vector for that word
04:40kryftI remember reading somewhere that by the time you get your MSc degree, you realize that you don't really know anything about anything. If you go on to get a PhD, you'll come to realize that neither does anyone else.
04:41esjtomoj: I don't know anything about random projections. Can you shoot me a link to an intro reference please ?
04:42tomojesj: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.96.2230&amp;rep=rep1&amp;type=pdf
04:42kryfttomoj: I assume you know that your features are going to change over time, that is, the definition of relevant content will shift?
04:42tomojyes
04:42tomojideally, a feedback loop :)
04:42kryftRight.
04:42esjtomoj: merci.
04:43tomojthat's about "random indexing"
04:43kryftAnd here I was feeling guilty for sitting around on IRC instead of dragging my ass to work.
04:43tomojwhich is as I take it a particular approach to using random projections for dimensionality reduction
04:43tomoj(where you don't actually have to do the reduction)
04:43kryfttomoj: "where you don't actually have to do the reduction"?
04:44tomojyou just generate random low dimensional vectors
04:44tomojbut their dimensionality is high enough that with high probability any two of them are nearly orthogonal
04:45tomojso the projection preserves pairwise distances
04:45tomojI may be off my rocker
04:46tomojpiecing together scraps from many papers I only sort of understand, and maybe I got it pieced together all wrong :)
04:47kryfttomoj: Ah, "don't have to do the reduction" in that sense.
04:47kryftHmm, now I really have to drag my ass to work, bbl. :)
04:47kryftEven though this is technically work, I have a lunch appointment in 40 minutes.
04:49esjaaaah..... So you put your text into the matrix F, but this thing is mondo HD. So you want to reduce the dimension. But the salient trait is the pairwise distance between rows, F_w, as that is the context you're after. So you want a pairwise distance preserving projection ?
04:55esjso then. The random index is just a way of producing an orthogonal basis. Then you effectively scan the documents and incrementally project each word onto that basis. Very smart !
04:55esjtomoj: great paper, thanks :)
05:08tomojesj: yeah
05:08tomojI just wonder how widely this can be applied
05:08esjits nice when authors bother to explain the intuition behind what's going on
05:08tomojit seems strange to use it for classification
05:09esji think the idea of the random projection is pretty generally valid
05:09tomojbecause you end up taking the cosine similarity between a sum of a bunch of context vectors compared to just one smaller sum
05:09esjthe problem is that these vectors are not particularly low dimensional themselves
05:09esjor they wouldn't be orthoganol ;(
05:09tomojlike comparing the sum of the entire "spam" corpus to a single questionable document
05:09tomojesj: right
05:10tomojI'm using 1800
05:10esjhave mercy !
05:10tomojit performs reasonably well with a totally naive slow implementation
05:10tomojso I expect to be OK when I switch to a fast implementation from mahout or something
05:11esji have no intuition, but that seems reasonable.
05:12tomojthe similarity values end up skewed
05:12tomojususally (-0.02, 0.3) or so
05:12esjin what sense ?
05:12tomojfor cosine similarity
05:12tomojwhich could be [-1,1]
05:13esjup to 0.3 is pretty noisy.... I think.
05:13esj15% noise ?
05:13esjbut maybe not too bad for the application.
05:13tomojwhat is this noise you speak of
05:14esjsorry, I misunderstood you. I thought you meant that the random projection introduced a skew in the reduction, when you meant that distribution was skewed. Sorry.
05:14esjthere is no noise. Doh.
05:15tomojI build up one vector by summing the context vectors observed in any document in the positive set
05:15tomojthen compare that vector to individual documents to classify
05:15tomojdoesn't seem right
05:17esjindeed, it seems you are comparing different things
05:18tomojhttp://www.kyb.mpg.de/bs/people/weston/papers/unified_nlp.pdf
05:18tomojthat paper has got me thinking
05:18esji thought the projection allowed you to compare words
05:18esjnot words to documents
05:18tomojyes, that is the original application
05:18esjbut then I only scanned it at rapid pace, so might easily have missed the point.
05:18tomojthere was some work on using that output for SVMs
05:18tomojto do classification
05:19tomojI get decent results this way, though
05:19tomojthe similarity range is just skewed
05:19esjdo you require centrality for the the SVM ?
05:19esji'm lost
05:21tomojI didn't read the SVM stuff closely, I just know it exists
05:21esjso why is the skew upsetting you ?
05:22tomojhttp://acl.ldc.upenn.edu/coling2004/MAIN/pdf/70-640.pdf
05:22tomojwell, I just feel like this is not the right model
05:22tomojI never saw anything like this in the research
05:23tomoj(that last link is the SVM stuff)
05:24tomojwish I could pay sahlgren to talk to me for a few hours
05:24esjspeaking off the top of my head, if you want to compare documents you want to put them into this context space.
05:24temp01coldhead:
05:25esjyou have a method for doing it stepwise on words
05:25esjso if you have the collection of contexts for each word in a document if you can find a sensible way to combine you have a representation of the contextual content of the documetn
05:26esjso you do this for each document in the corpus, and feed that representation to the classifier of your choice
05:26esjthe slippy bit is aggregating the context-by-word to context-by-document
05:27esjits tempting to say just add the context matrix produced for each word
05:27esjthe space is nearly orthogonal, so why not ?
05:28esjbut I'm way beyond myself by now, so I'm likely to be talking pure bollocks
05:30tomojwell, that's what I did
05:31esjawesome !
05:31LauJensenGood morning all
05:31tomojthe weighting scheme becomes very important I think
05:31esjhello Lau.
05:31tomojbecause both spam and non-spam contain many instances of common words
05:31esjthat should be OK though
05:31tomojfiltering stopwords helps, but doesn't solve the problem
05:31esjbecause the classifier shouldn't care
05:32tomojoh, hmm
05:32tomojI'm not actually really running a classifier
05:32tomojI just directly compare (cosine similarity) the summed contexts by document to a single context for a questionable document
05:32LauJensenAre you rebuilding SpamAssassin ?
05:32esjtomoj: You should really throw it at a classifier
05:32tomojno, a menagerie of classifiers
05:32tomojyes
05:33esjtomoj: indeed, try a full menagarie.
05:33tomojwell, I meant I have a bunch of different classification problems
05:33tomojbut, yeah, that too :(
05:33tomojmaybe mahout will save my day
05:34LauJensentomoj: What are you doing?
05:34esjthere is a tried and tested method of PhD students in Machine Learning which is to get http://rii.ricoh.com/~stork/DHS.html the computer manual part of that
05:34esjwhich is a uniform interface to all the classifiers in the book
05:34esjso you can rapidly iterate through all your options
05:35esjits intellecutally bankrupt, but I've seen it done.
05:36tomojhehe
05:36temp01cais2002: You Head!
05:36temp01coldhead: sry
05:37cais2002temp01: did u start from letter c? how come some are skipped?
05:38temp01you're the first nick with letter c
05:40tomojLauJensen: uhh, content classification
05:45notsonerdysunnyHello everybody, I was just exploring cake .. but haven't been successfull using it .. I got http://gist.github.com/616564 when I did "cake deps" can anybody help?
05:57_ato`notsonerdysunny: /var/lib/gems/1.9.1/bin/cake:9:in equire': no such file to load -- rubygems (LoadError)
05:57_ato`sounds like you might need to install rubygems http://rubygems.org/
05:58fliebelmorning
06:08LauJensennotsonerdysunny: how did you install it ?
06:14notsonerdysunnyLauJensen: I installed it via gem install
06:15notsonerdysunnyand then just added the directory containing the cake to path
06:17LauJensennotsonerdysunny: On which OS?
06:18notsonerdysunnylinux ubuntu 10.04
06:18LauJensenactually I think it might be your path thats messing things up. When you install via gems you shouldn't touch the path: http://github.com/ninjudd/cake <-- See installation
06:23notsonerdysunnyLauJensen: If I don't add the gems directory to path then .. how would it know where the executable is?
06:23raekjust saw this very neat summary of the Clojure web ecosystem: http://groups.google.com/group/clojure/msg/64e8ed91791d81d2
06:24LauJensennotsonerdysunny: As I recall, once you've installed Ruby and the gem system, the cake gem is installed somewhere local to that application
06:24LauJensenI havent done any permanent path fiddling on Arch IIRC
06:26notsonerdysunnyLauJensen: I am new to the gem thing .. is it ok if reinstall using gem install?
06:26LauJensenProbably, but also clear cake from your PATH or reboot if you're not sure
06:29LauJensenAnybody got a link to that hilarious post about lisp syntax that fogus did?
06:30LauJensengot it
06:31esjshare with the class ?
06:31notsonerdysunnyLauJensen: http://gist.github.com/616606 is what I got when I reinstalled it via "gem install -V cake" now cake is not in path .. what do you think I should do?
06:32LauJensenesj: http://blog.fogus.me/2010/08/30/community-standards/
06:32esjmerci
06:32LauJensennotsonerdysunny: What does lein deps get you now?
06:34notsonerdysunny"lein deps" seems to work fine
06:34notsonerdysunnyI just created "lein new cakeexp" and then ran "lein deps" .. it seems fine
06:35notsonerdysunnybut I want to try cake ...
06:36bobo_notsonerdysunny: link /bin/cake to /var/lib/gems/1.9.1/gems/cake
06:36bobo_except use the correct path
06:40notsonerdysunnybobo_: I got http://gist.github.com/616608
06:40notsonerdysunnysame as before I redid the installation ..
06:41mrBlissLauJensen: do you use rcirc or erc?
06:41LauJensenerc
06:41mrBlissme too, but I just read rcirc is the standard irc client for emacs. Have you got any experience with it?
06:42LauJensenyea, I didnt like it. Couldnt get logging and notifications right with it
06:42mrBlisswere there any advantages over erc?
06:43LauJensenIts more lightweight I guess
06:44mrBlissok, I'll just stick with ERC.
07:07AWizzArdLauJensen: please give me the timing: (do (import 'java.io.ByteArrayOutputStream) (def baos (ByteArrayOutputStream. 100000000)) (time (dotimes [i 80000000] (.write ^ByteArrayOutputStream baos 0))) (.reset baos))
07:07LauJensenAWizzArd: 796.92 msecs
07:09AWizzArdfast, what cpu?
07:09LauJensenyou mean, 'which cpu' ? :)
07:09AWizzArdyes
07:09LauJensenI actually dont know
07:09AWizzArd<--- not a native speaker
07:10LauJensenIntel(R) Core(TM)2 Duo CPU T6600@2.20 GHz x 2
07:10AWizzArdThough it is faster than my cpu or the one of lpetit: why is it so slow? The timing you gave boils down to about 100 mb/sec. This is 1% of what should be possible.
07:11AWizzArdYes, this is byte by byte, but still...
07:13AWizzArdLauJensen: and what is shocking me even more: lpetit translated this into a very simple pura Java program. And on my machine the pure Java code runs much slower. How is that possible?
07:17LauJensenI'll think about it once I get back, I have an appointment I have to run to, sorry :)(
07:18AWizzArdkk
07:26_atoAWizzArd: I get the same times for Clojure and Java code, around 410 ms (after JIT warmup)
07:26_atoI guess maybe ByteArrayOutputStream is just slow, setting an array directly is much faster
07:27_atoah.. one thing is it's thread-safe
07:28AWizzArd_ato: a BAOS is thread safe?
07:28AWizzArd_ato: can you please post your clj and java which gave you 410 msecs?
07:32_atoAWizzArd: http://gist.github.com/616651
07:34AWizzArd_ato: ah okay, so you also notice that Java at first is way slower than Clojure in this case.
07:34_atoAWizzArd: and yep, BAOS locks with "synchronize" keywords on the methods: http://www.java2s.com/Open-Source/Java-Document/6.0-JDK-Core/io-nio/java/io/ByteArrayOutputStream.java.htm
07:35AWizzArdThe interesting this is that I warmed up the cache, but not so deeply as you did, and I will now try this too.
07:35_atoAWizzArd: it's probably only slower cause my Clojure process had already warmed the JIT since I was doing other stuff in it earlier
07:35AWizzArd_ato: btw, what CPU do you have?
07:35_atomodel name : Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz
07:36AWizzArdAAh, Core i7 are real beasts.
07:36AWizzArdOkay, so the synchronized thing has impacts on the performance.
07:37AWizzArdAlthough only one thread is doing the work here.
07:40_atohttp://gist.github.com/616655
07:40AWizzArdwhat the..
07:41_atolooks like Clojure's "locking" is much slower than synchronize even.
07:42AWizzArdamazing
07:42_atothey might be implemented differently, but it's the same sort of thing. Locks can be pretty expensive
07:43AWizzArdso, the synchronized is causing that speeddown, great, you solved this mystery
07:43AWizzArdSo a C program should also be slowed down drastically when locking each access
07:44_atoin theory at least
07:46AWizzArd_ato: what Clojure version do you run? Your 50 msecs translate into 2250 on my machine, with a recent clojure.jar
07:46_atoerm.. some old version 1.1
07:47_atosome git snapshot of 1.1
07:47AWizzArdI see.
07:47_atomight need type hinting or something, I dunno
07:50AWizzArdbut not by a factor of 45 times, and I ran it also several times now
07:51AWizzArdstrange
07:51AWizzArd_ato: 64 bit os?
07:53_atoyeah
07:54_atovery similar results with C: http://gist.github.com/616670
07:54_atoI just timed with the shell "time" command. Too lazy to figure out the C API for getting current milliseconds
07:56AWizzArd_ato: cool, thank you very much, I am interested in those results.
07:57AWizzArd_ato: So, the C program is faster, but it also shows the same effects, that locks are very expensive.
08:00_atoyeah
08:02_atofairly similar times to the JVM actually.. my crappy C being maybe 20% faster. (There's probably much faster ways to implement that in C, but in the interest of making the microbenchmarks similar kind of code...)
08:07AWizzArd_ato: this benchmark is important for me, because it gives me a base to compare my Clojure serializer lib with.
08:18lpetithello
08:18AWizzArdHi lpetit
08:18AWizzArdI wasn't away yesterday.
08:19AWizzArdAnd I got your Java program and tested it, just moments after you left.
08:19lpetitif some people meet problems with a recent release of ccw, please talk. I was not playing with it that much the last weeks, so don't count on me finding the bugs by myself :)
08:19AWizzArd_ato has just tried out also some interesting things
08:19lpetitAWizArd: hi
08:19lpetitwhich ones ?
08:19AWizzArdlpetit: http://gist.github.com/616651 and http://gist.github.com/616655 and http://gist.github.com/616670
08:20AWizzArdThe slowness of the BAOS comes from it being synchronized.
08:21AWizzArdAnd clojures locking is even much more inefficient that Javas 'sychronized'.
08:21lpetitas expected, I would say, no ?
08:22lpetitand direct C on arrays is equivalent to direct clojure/java on arrays
08:22lpetitdid I read the figures correctly ?
08:22_atoyep
08:24lpetitgreat
08:24_atoand pthread's mutex seems to add a similarish amount of overhead as Java BAOS's synchronize
08:27chouserdid you know you can put multiple files in a single gist?
08:28_atochouser: yep, but I use an emacs command for gisting which doesn't support that
08:28chouserAWizzArd: why do you say clojure's locking is more inefficient than Java's 'synchronized'?
08:29AWizzArdchouser: _ato found that out
08:29_atochouser: AWizzArd is going off this http://gist.github.com/616655 as compared to ByteArrayOutputStream's synchronize write() method. I may well be doing something dumb there ;-)
08:29AWizzArdSee http://gist.github.com/616655
08:30AWizzArdHere _ato simulated the synchronized that we find in the BAOS class.
08:30AWizzArdWriting to baos takes 450 msecs
08:30AWizzArdWith _atos locking that became 9000 msecs
08:31AWizzArdWhile writing directly into a byte arrat, without locking, got it down from 450 to 50
08:32AWizzArdI am currently facing a totally different problem. When I compare my my serializer (time (with-store-writer baos (dotimes [i 80000000] (store-state (byte 0))))) with direct baos writing (time (dotimes [i 80000000] (.write ^ByteArrayOutputStream baos 0))) then I see that my serializer takes 8x more time.
08:33AWizzArdThe store-state fn is part of the PStorable protocoll. My hope/idea now is that I need to AOT the code and measure again.
08:34AWizzArdDid anyone of you experience a speedup in calling protocol fns after AOTing them?
08:34chouserI see that locking compared to not locking is much slower. Was there also a test against Java's "synchronized"?
08:34chouserbecause I'm pretty sure the same bytecode is generated.
08:34AWizzArdchouser: yes, as I said: clojure locking: 9000 msecs. java synchronized: 450 msecs
08:34AWizzArdhttp://gist.github.com/616651 <--- here _ato measured javas synchronized
08:34chouserAnd there are no other differences in the code?
08:35chouserah, that's the gist I was missing, thanks.
08:35AWizzArdyes
08:35_atohere's a more direct translation: http://gist.github.com/616708
08:35AWizzArd(/ 9000 450.0)
08:35AWizzArd,(/ 9000 450.0)
08:35clojurebot20.0
08:35AWizzArdSo it seems that clojures 'locking' takes about 20x longer than Javas synchronized
08:36chouserthis is Clojure 1.2?
08:37_atoheh nah, some old 1.1 snapshot I had lying around. ;-) I should retry on 1.2, just a sec
08:37AWizzArdrhickey: wb
08:37@rhickeyhi
08:37AWizzArd_ato: can you download a fresh clojure.jar from http://build.clojure.org/ and time it with that?
08:38@rhickey_ato: we're not doing naive benchmarking again, are we? :)
08:39_atowoah... 1.2 is much worse "Elapsed time: 10472.37607 msecs"
08:39chouserrhickey: it's our favorite kind! :-P
08:39_atorhickey: don't look at me! it's AWizzArd's benchmark :p
08:39chousertoday, Java's synchronized vs. Clojure's locking. Yeah, I know.
08:40@rhickeyfor instance, the Java compiler might know every entry in a new byte array is already 0
08:40@rhickeywhere's the Clojure code?
08:40chouserhttp://gist.github.com/616655
08:40chousermoving the cast to primitive outside the loop helps some
08:40_atocompilers are sneaky like that
08:41chouserrhickey: did you see I merged master into par? Would you like that committed onto your par branch? It seems to run.
08:41@rhickeychouser: I've seen little - been in Denmark all week
08:42LauJensen rhickey: Hey, thanks for your talk in Aarhus, was very interesting, especially the final upshot about Pods :)
08:42chouseroh, I can't keep track of your crazy globe-trotting schedule. :-) I just sent email last night.
08:42AWizzArdrhickey: compare http://gist.github.com/616651 and http://gist.github.com/616655 and http://gist.github.com/616670
08:42@rhickeychouser: I think David Liebke has already got the par bits working on master (other direction)
08:42chouseroh, ok
08:42@rhickeyAWizzArd: I'm not going to spend my time on that right now
08:43@rhickeychouser: so hopefully we will move forward with with par bits in master so par can move forward there
08:44chouserrhickey: sure. I'd think the direction doesn't really matter. I mean, the source tree I have here is all of par + all of master -- could be committed either place.
08:44chouserbut I won't move til I hear what Liebke's got
08:44AWizzArdrhickey: not required, just if you are interested. One thing that is way more interesting for me: will AOTed Protocoll functions be called much faster than non-AOTed ones?
08:44@rhickeychouser: yeah, I'm not sure if he tweaked a few bits - he's looking to move par forward
08:45@rhickeyAWizzArd: shouldn't be faster at all
08:46chouserhe's a better person for it.
08:46@rhickeyheh
08:46chouserhm, maybe that's not what I meant
08:51chouserThe creator of http://clojure.org/cheatsheet doesn't want to update it for 1.2, recommends we link to http://clojuredocs.org/quickref/varsonly/Clojure%20Core instead
09:32abedrarhickey: was there ever a thought about enabling a second arg to promise to enable the await call to have a timeout?
09:32chouser(doc await-for)
09:32clojurebot"([timeout-ms & agents]); Blocks the current thread until all actions dispatched thus far (from this thread or agent) to the agents have occurred, or the timeout (in milliseconds) has elapsed. Returns nil if returning due to timeout, non-nil otherwise."
09:32chouseroh, on promise.
09:33abedrayeah
09:37AWizzArdAah, I found my bottleneck! I just noticed that accessing a binding is much less efficient than accessing a "normal var".
09:46AWizzArdrhickey: I have a Protocol PStorable that offers a fn store-state. And I did (extend Byte PStorable {:store-state foo}). Now I call store-state 80 mio times in a loop vs 80 mio calls of foo directly. For me store-state times 5400 msecs vs 3800 msecs when using foo directly. Do you have an idea why it is so?
09:49@rhickeyAWizzArd: why wouldn't it be?
09:49AWizzArdI would think that calling a protocol fn is at least as efficient as calling an ordinary function.
09:50@rhickeyAWizzArd: why?
09:51AWizzArdProbably I misunderstood the intention of Protocols. Now that I see those results it seems they are more a replacement for defmulti and are more efficient than that. But ordinary fns are still more efficient as they don't have to dispatch a type.
09:51@rhickeyAWizzArd: why spend time on this?
09:52@rhickeypremature optimization ____
09:52AWizzArdrhickey: not really. I want my serializer to be fast. When storing gigabytes of data there is a difference between 12 minutes and 2 mins.
09:53@rhickeyum, 5400/3800 < 2, not 6
09:54AWizzArdYes yes, that is not the main problem. I noticed that writing directly into a ByteArrayOutputStream is 8x faster than using my serialize function. And I want to understand why.
09:54AWizzArdI noticed that the main cause of this is that (serialize obj) accesses the var *stream*, which is a binding.
09:54@rhickeyAWizzArd: ok, but I don't want to spend more time on optimization right now
09:54stuartsierrarhickey: abedra & I had a question: is there a reason promise/deliver doesn't offer a "read with timeout" or "check if ready" function?
09:55jcromartieI love Docco http://jashkenas.github.com/docco/
09:55@rhickeystuartsierra: It's been requested a few times, as well as some generic is-ready. What's needed is a decent design that fits into the ref framework
09:55stuartsierraah ok
09:56jcromartieIs there a Clojure code coloring lib yet? It seems like a pretty simple thing to do for Clojure, considering the hard work of parsing is already done.
09:56stuartsierraSo what we really want is "deref with timeout" and "will deref block?" functions
09:56jcromartieWow. Moment of zen here.
09:56@rhickeystuartsierra: cemerick had some requirements in this area and we've discussed it on irc a few times
09:56abedraah
09:56abedrai see
09:57@rhickeystuartsierra: you need to think about to what abstractions those correspond, and have we got the hierarchy aligned, do we need a new interface etc. I don't want meaningless methods because we hang this on the wrong thing.
09:58stuartsierrasure
09:58abedraagreed
09:58stuartsierraWe basically wanted to make sure it wasn't left out for technical reasons
09:58@rhickeynope
09:59@rhickeyI just haven't had time to work on it and haven't seen anything other than 'features' proposed
09:59stuartsierraok
09:59abedrarhickey: stuartsierra and I are working on building out some stuff with zeromq and this is where it came up
10:00abedrarhickey: we can write up a proposal this afternoon
10:01@rhickeya good start would be to make a little spreadsheet of the reference types and see where this fits, is it it's own abstraction (e.g. MaybeDeref?). Don't forget delays and futures
10:01stuartsierraright.
10:01stuartsierrapromises, delays, futures, agents, refs, atoms, vars, ... anything else?
10:02@rhickeystuartsierra: that's probably it
10:02stuartsierraok
10:02@rhickeyIMightDeref
10:03dnolenheh
10:03abedra:)
10:04stuartsierraIWishIMayIWishIMight
10:04@rhickeydnolen: what? I like it!
10:08AWizzArdWhen I want to implement my own OutputStream, a combo of BufferedOutputStream and DataOutputStream, to get rid of this Java synchronized, then deftype is what I should use, yes?
10:09stuartsierrayes
10:09AWizzArdgood
10:10chouser,(:doc (meta #'loop))
10:10clojurebot"Evaluates the exprs in a lexical context in which the symbols in\n the binding-forms are bound to their respective init-exprs or parts\n therein. Acts as a recur target."
10:10chouserthat's kinda nice, compared to (doc loop)
10:13@rhickeychouser: I give in on that. If someone wants to submit a patch with doc support for the special ops, I'll take it, as long as they contain links to the full docs and aren't too long themselves
10:22chouserok. I'll probably take a swing at it if nobody else wants to: http://www.assembla.com/spaces/clojure/tickets/454-docstrings-for-special-ops
10:23@rhickeychouser: great, thanks! - just assign it to yourself then
10:30shooverahhh, open source in action
10:52sthiagarI am using clojure.contrib.lazy-xml to parse-trim a XML string (returned from a java API) and filter to an element which contains another XML document. I assign this XML document into a map and then lazy-xml parse it to get to some text elements. When number of strings parsed in the first time is less, this works fine, when its more I hit a Caused by: java.util.concurrent.ExecutionException: org.xml.sax.SAXParseExceptio n: <Line
10:53jcromartieouch LauJensen you're getting slammed on HN
10:53sthiagarI think its because of lazy sequences and tried putting doall around the lazy-xml parse-trim and around the map where the inner xml is extracted, to no avail
10:53jcromartieI appreciated the Uncle Bob post but wow
10:53LauJensenjcromartie: Yea, people love to cry out loud. We're really in the age of emotions
10:54LauJensenI think its amazing how sensitive 'hackers' are. I make a comment stating that its hard for me to think about correct robust programming and Ruby in the same sentence, and then the whole world breaks down, I mean come
10:54LauJensenon..
10:54jcromartiejust yesterday
10:54jcromartieI was using Ruby
10:54jcromartieand this drove me nuts...
10:54LauJensenthis?
10:54clojurebotThis is a test.
10:54jcromartie"asdf"[0..3] returns "asdf"
10:54jcromartiethat's OK
10:55jcromartie"asdf"[4..5], out of bounds, returns ""
10:55jcromartie"asdf"[5..6], also out of bounds, returns nil
10:56LauJensenO_o ? :)
10:56LauJensenjcromartie: I really shouldn't comment, or else I'll get The Internet (tm) on my back
10:56fogus_jcromartie: Is "asdf"[4..5] out of bounds? Maybe it grabs the \0 ?
10:56jcromartie:)
10:56abedraLauJensen: you're safe here :)
10:57jcromartiefogus_: I asked on #ruby and they said it had nothing to do with null-terminated strings :P
10:57jcromartiebut I never got a good answer
10:57LauJensenabedra: thanks :)
10:58fogus_jcromartie: Hmm. Oh well, I'm out of ideas then
10:58abedraLauJensen: I say that as a recovering rubyist
10:58abedraeven though I still maintain some ruby libraries like RCov
10:59chousersthiagar: your first post got cut off. Can you paste the exception and/or code at a paste site?
10:59LauJensenabedra: Thats nice to know. Im not actually sure if its Ruby guys who are doing the whining. Apparantly people nowadays explode if you say anything other than "yea.. thats cool". The moment you disagree with something, you have to be very very careful unless the globe melts down. I think its pathetic. I understand it in teenage woman but not in adult developers
11:00abedraLauJensen: But.. But.. Somebody on the internet is WRONG!!!!
11:00LauJensenAnyway, Ive started 1 flamewar today, time to kick back and relax, nice weekend everybody
11:00abedraLauJensen: See you at the conj!
11:00LauJensenabedra: Yea I wish :| I'm sending my best man Christophe :)
11:01abedraLauJensen: I thought you were coming as well
11:01fliebelHey, if I need to use a jar without docs or source, is there a way to tell which methods it exposes?
11:02sthiagarchouser: this is the exception http://pastebin.com/MqEpupUP
11:02stuartsierrafliebel: try Mycroft
11:03abedra^^
11:03abedrait's a lifesaver
11:04fliebelstuartsierra: That's a search engine, right? I was hoping there was a way to inspect the class files or something in the fashion.
11:04stuartsierraNo. Mycroft is a Clojure/Java runtime inspector
11:05stuartsierrahttp://github.com/relevance/mycroft
11:05fliebelstuartsierra: Ah, that's something different from what google shows me :)
11:05stuartsierraIt's a much-used name :(
11:06chouserI looked at mycroft, but I didn't understand it.
11:08stuartsierraIt's not well-documented yet
11:08stuartsierraAdd it as a dependency, then do ...
11:08stuartsierra(use 'mycroft.main) (start 8080)
11:08stuartsierraThen (inspect foo) will pop up a browser showing everything about foo
11:09fliebelnice
11:09stuartsierraI think that should be (run 808)
11:09stuartsierra(run 8080)
11:09stuartsierrawhatever
12:42jkkramerQuestion: in Lau's latest post (http://bestinclass.dk/index.clj/2010/10/taking-uncle-bob-to-school.html) he says towards the end, "some of the functions in vector are generating anonymous functions everytime they're called" -- re a function that has a letfn. is that really true? i thought fns in letfn only get compiled once and don't incur any particular performance hit
12:43hiredmanlau is an ignorant jerk, so I recomend ignoring him
12:44technomancyjkkramer: it's not true; you are right. runtime cost is just an instantiation of a precompiled class.
12:45jkkramertechnomancy: k thanks, i'm not going crazy then
12:53dysingerI'm leaving for Clojure Conj by way of Vancouver on Monday - it's coming quick!
12:53dysinger:)
12:55amalloywow, if lambdas took a compilation every time you ran them, idiomatic clojure code would be terrible
13:05ohpauleezCan someone explain static functions in clojure 1.3 quickly
13:05ohpauleezor point me to a document
13:08raekohpauleez: http://www.assembla.com/wiki/show/clojure/Enhanced_Primitive_Support
13:08ohpauleezraek: thanks!
13:09raekbasically, calls to a static function is hard-comiled to that function instance
13:09raeki.e. redefining the var will not alter the behaviour of functions using the static function
13:10raekthis enables the compiler to do some additional optimizations, and true primitive support is possible
13:33woobythe conj is coming!
13:39chouserI know exactly how you feel.
13:44technomancyI love conjferences.
13:47amalloytechnomancy: conjurances?
13:48RaynesYay, 14 days. :D
13:50freakazoidwhere is the conj?
13:50woobyfreakazoid: durham, nc: http://clojure-conj.org/
13:51freakazoidAh, too far.
13:51freakazoidIt would have to be in the bay area for me to go
13:55amalloyfreakazoid: where in the bay area are you?
13:56KirinDaveSo, if I said, "I am going to go to a place in SF in an hour or so, does anyone want to pair program clojure?"
13:56KirinDaveWould I have any takers?
13:56amalloyKirinDave: if you said it on a weekend...probably
13:56chouserKirinDave: if by SF you mean Fort Wayne, sure!
13:57moogatronicchouser: you are in Indiana?
13:57KirinDaveamalloy: This _is_ my job.
13:57KirinDaveBut the stuff is gonna be open source soon, so I don't mind other people seeing it.
13:57moogatronic<- Bloomington
13:57amalloyKirinDave: i'm aware. but it ain't mine
13:57chousermoogatronic: yep
13:57KirinDaveamalloy: ;)
13:57KirinDaveamalloy: Might be though if clothesline ends up working out.
13:57moogatronicI grew up in Ft.W though.
13:57chousermoogatronic: ah, cool!
13:57KirinDaveWebmachien is pretty damn awesome.
13:58moogatronicmy parents still live there, maybe I'll take you up on that Pair offer someday. =)
13:58chousermoogatronic: sure, it'd be fun
14:02freakazoidamalloy: mountain view
14:02amalloyfreakazoid: i worked in mountain view for a couple years. in SF now, though
14:02freakazoidI've thought about moving there a few times, but it's unlikely to ever happen. The years that I should have lived in SF have passed.
14:03KirinDaveThere is always portland, freakazoid.
14:03amalloyheh. well, i didn't do it on purpose, but here's where the jobs were
14:03moogatroniceveryone gives portland love
14:03KirinDaveI dunno what kind of magic people expect from a city tho.
14:03KirinDaveYou gotta work it like any location.
14:03freakazoidI was just in portland for oscon. It was like a ghost town, but it was very easy to get around on public transportation and to get cabs.
14:04freakazoidCabs are essentially a joke in most of the bay area.
14:04bartjI have seen my colleague do this in emacs - change some clojure code, save it and then automatically execute the changed function on the Repl
14:04bartjis this possible in vimclojure?
14:04chouserhere too
14:04KirinDaveThey're trying to grow fast and not doing very well.
14:04moogatronici've been in smallish college towns for most of my life now.
14:04moogatronicbike to everywhere
14:04freakazoidI walk to work, and bike to shoreline lake and back (by moffett field) for exercise
14:05freakazoidessentially, I bike the distance from the train station to google and then back
14:05moogatronicin indiana people still sometimes throw stuff at you if you're on a bike on the road... even college towns.
14:05freakazoid(because I don't work for google)
14:05moogatronicthough that happened to me in Livermore, CA once too.. Maybe there's a bullseye on my back.
14:06freakazoidLivermore has a lot of rednecks
14:06amalloymoogatronic: tjat
14:06amalloyer
14:06moogatronicis it like LLNL + rednecks?
14:06amalloythat's crazy. i've never been anywhere where people would throw stuff at bikers
14:06freakazoidPretty much.
14:06moogatronici pretty mcuh rode to LLNL, and to the bart station
14:06freakazoidEasy solution to that problem. Live somewhere with open carry and carry a pistol on your hip.
14:07moogatronichaha, IN is concealed carry
14:07moogatronici have a permit, but never carry
14:07freakazoidPeople don't throw shit at you if they think you might return the favor with bullets.
14:07freakazoidHeh, big lump in your spandex shorts. "But it is concealed!"
14:07mrBliss`I'm glad I live in Europe
14:08jcromartieso it might seem kind of silly, but is there a reader that *keeps* comments?
14:08moogatronicmrBliss: SUVs running redlights have almost killed me too.
14:08freakazoidjcromartie: I've thought about that too.
14:08amalloyjcromartie: ; comments, or (comment s)?
14:08jcromartieboth
14:08jcromartieoh
14:08jcromartieI have it
14:08jcromartieconvert ; ... into (comment ...)
14:08freakazoidjcromartie: Genesis's bytecode format was completely decompilable to commented, formatted source.
14:09freakazoid; is not a reader macro?
14:09amalloy(comment) is a macro; you could probably just override it with your own macro that does something else
14:09chouserjcromartie: insufficient. Would also like one that keeps all whitespace.
14:09mrBlissmoogatronic: that's true, but here at least idiots don't get to carry guns
14:09freakazoidmrBliss: neither do non-idiots ;-)
14:09jcromartieI want to implement Docco or Pycco for Clojure
14:09amalloychouser: try using cat instead of a repl
14:09jcromartiemrBliss: yeah, only police or psychopaths
14:09chouseramalloy: oh!
14:09chouseramalloy: of course!
14:09jcromartieand the difference is sometimes hard to tell
14:09freakazoidjcromartie: you say that lik ethere's a difference… yeah, that
14:10freakazoidpeople's fear of guns is amusing
14:10mrBlissjcromartie: never encountered any violence in my life yet, never seen a real fight
14:10moogatronicnow if we can only tie this into a discussion on emacs and vim, which is better, and include religion somehow.
14:10freakazoidit's amazing how little people trust one another but how much they're willing to trust government at the same time
14:10jcromartieJESUS EDITS MODALLY
14:10amalloymoogatronic: only right-wing nuts would suggest that
14:11freakazoidnot right wing-nuts?
14:11moogatronici think the parse is valid either way
14:12chouserjcromartie: heh. so do emacs users. shhhhh
14:12mrBlissfreakazoid: In Europe you can trust your government more than in the US (don't get me started on health care and republicans). But now I'll switch back to emacs and clojure :-)
14:12jcromartieyes please
14:12freakazoidheh, I'd say people DO trust their governments more than in the US.
14:13freakazoidCAN is a different matter.
14:13jcromartielet's talk about using the reader and preserving comments
14:13jcromartie:P
14:13mrBlissjcromartie: why do you want to preserve them?
14:13jcromartieI'm sure I'd figure it out if I actually *tried* it
14:13jcromartiemrBliss: I want to associate comments with forms
14:13freakazoidjcromartie: it seems to me you'd have to do something to the compiler as well
14:13jcromartieso that I can line them up in HTML output, a la Docco
14:13jcromartiehttp://jashkenas.github.com/docco/
14:14jcromartieor Pycco http://fitzgen.github.com/pycco/ or Rocco http://rtomayko.github.com/rocco/
14:14freakazoidor maybe stick the comments in metadata via a macro, when metadata is available to you?
14:15jcromartiesince 'read' reads the next form from a stream, I could try to read comments from the stream first
14:15jcromartieand alternate, read-comments, read
14:15jcromartiebuilding a list of pairs of comments and forms
14:15jcromartienested
14:16freakazoidhmm
14:17freakazoidnaah
14:17freakazoidjust build a datastructure you can easily strip comments from
14:17sandGorgonis there any web framework in clojure with something like DB migrations - or any other way of handling schema changes ?
14:17freakazoidsort of the same thing
14:17freakazoidbut the comments would just be tagged as such
14:18jcromartieeither way
14:18freakazoidyou can do it with macros, though
14:18freakazoidjust have a macro that returns your pair
14:23chouserstuff the whitespace and comments into metadata of nearby forms
14:25freakazoidIt would be cool to have a datastructure that could be unparsed into commented source as well
14:25freakazoiddeparsed?
14:25freakazoidI guess that's "serialized"
14:25freakazoidor formatted
14:25chouserright
14:25chouserprinted
15:03djwonkquestion about clojure-conj... when does it wrap up on Saturday. figuring out travel plans.
15:33pjstadigon the registration page it says 5:30pm Saturday http://clojureconj.eventbrite.com/
15:34pjstadigbut i'm not sure if that's intentional or eventbrite's addition
15:36fogus_It could work! http://craplogo.me/logos/Clojure_logo.png
15:38chouserehe
15:38chouser+h
15:51chouserrhickey: what does "Approval: vetted" mean? is this workflow documented somewhere I keep forgetting about?
16:01shanmuI am trying to setup emacs+slime+clojure. The latest version of swank-clojure and slime seem to have problems when going to source code from breakpoints
16:02shanmuI am getting "error in process filter: cond: Elisp destructure-case failed: nil"
16:03shanmuany help please?
16:04shanmuI am getting "error in process filter: cond: Elisp destructure-case failed: nil" when pressing 'v' on any backtrace
16:04raekshanmu: do you use the snapshot version of slime? I have heard that it frequently breaks compability with swank-clojure...
16:04shanmuraek: yes its the snapshot version
16:04hiredmanclojurebot: swank
16:04clojurebotswank is try the readme. seriously.
16:05shanmuraek: is there a stable repo for slime (for clojure) somewhere>
16:05pjstadigclojurebot: suddenly
16:05clojurebotCLABANGO!
16:05raekwhen I install the stuff I need on a new computer, I use the ELPA version of slime and slime-repl
16:05raekshanmu: I think the ELPA one is the recommended
16:06Dawgmatixas someone who fought slime issues yesterday - i just want to say that the whole process is way more complicated than it needs to be
16:06shanmuraek: but I am using other lisps as well (CCL)
16:06raekwhatever swank-clojure recommends is probably the best http://github.com/technomancy/swank-clojure
16:06hiredmanDawgmatix: complicated how? did you readme the readme and do what it says?
16:07Dawgmatixyes i did.
16:07raekwhere "best" means "what I got to work"
16:07hiredmanthe readme has 2 steps
16:07hiredmanwell, maybe 3
16:08Dawgmatixthe readme broke for me because a) i already had slime installed
16:08hiredman13:12 < Dawgmatix> yes i did.
16:08ordnungswidrigwhy does slime reports version mismatch against swank-clojure?nil (slime) 20100404 (swank)?
16:08AWizzArddeftype is not for extending other classes right? Either genclass or proxy?
16:08hiredmanwell obviously you didn't do what the readme said then
16:09Dawgmatixb) even after nuking everything for some reason now i am in a state where if i use slime-connect, then clojure-mode doesnt launch
16:09Dawgmatixautomatically for clj files
16:09shanmuDawgmatix: I also have the same issue.. I have slime installed for other lisps
16:09hiredmanordnungswidrig: http://github.com/technomancy/durendal has an example of how to stop that warning
16:09raekAWizzArd: yes, I think so
16:10hiredmanordnungswidrig: you can also just use durendal
16:11scottjmaking a few small formatting/info display changes to clojars-web, anyone have any they really want?
16:11ordnungswidrighiredman: reading the lisp…, autocompile sounds nice
16:12Dawgmatixi would suggest renaming slime / swank at this point. at least this will allow parallel installs of slime for lisp / clojure
16:12ordnungswidrighiredman: but, "is it safe?"
16:12shanmuhiredman: I understand the a clean new install from ELPA would work, but is there someone here who can help with plugging in clojure/swank into an existing slime?
16:13zaphar_psshanmu: I created my own set of customizations for clojure swank: http://bitbucket.org/zaphar/dotfiles/src/tip/.emacs.d/conf.d/004-my-clojure.el
16:13raekshanmu: are you using slime-connect to connect to the swank server?
16:14hiredmanordnungswidrig: well, technomancy is also the maintainer (reluctantly) of swank-clojure
16:14raekthe swank server is most often started by leiningen or cake nowadays
16:14raekiirc, swank-clojure.el is deprecated
16:14ordnungswidrighiredman: I know :-)
16:16raekshanmu: can you successfully connect to a swank-clojure server with your current setup?
16:19shanmuraek: I tried both ways...slime-connect and M-MxSlime
16:20shanmuraek: yes I can connect fine to clojure swank
16:20raekwhen does it blow up?
16:20shanmuraek: when there is an exception and I try to go to source from the back trace (by pressing 'v') it blows up
16:24raekman. what a nice feature.
16:24shanmuzaphar_ps: your script looks promising
16:25shanmuraek: exaclty... its a blessing!
16:25shanmuif/when it works :)
16:28raekI know very little about slime except for some basic usage, so I'm afraid I'm out of advice...
16:33zaphar_psshanmu: yeah It just launches the lein swank command and connects slime to it.
16:34shanmuzaphar_ps: but my current setup can connect to it... after an exception occurs, thats where the problem starts
16:35zaphar_psexception?
16:35clojurebothttp://www.mindview.net/Etc/Discussions/CheckedExceptions
16:36shanmuzaphar_ps: when there is an exception, I try to go to source code from backtrace
16:36shanmuzaphar_ps, then I get an exception
16:37shanmuzaphar_ps: error in process filter: cond: Elisp destructure-case failed: nil
16:39zaphar_psshanmu: I'm afraid that one I can't help with :-(
16:40shanmuzaphar_ps, raek: thanks for the tips, anyways!
16:45TeXnomancyshanmu: if you have some ideas for improving compatibility with Clojure and CL at the same time that would be great. personally I don't use CL, so I am not in a position to know what's broken myself.
16:47shanmuTeXnomancy: I will try.. but I am not an expert in elisp/swank :(
16:47nollidjanyone here use jackson? i am using it now, and i want to deserialize into an abstract generic type. said type is from a third-party library, so i can't annotate it directly to get it deserialized properly, and i can't seem to specify a deserializer on a field of the type because it is generic
16:58scottjhave an opinion on what clojars search results should look like? Tell me which of these three you prefer http://img84.imageshack.us/img84/2412/clojars.png
17:03nollidjoh, i typed into the wrong channel.
17:04jcromartiescottj: I like the leftmost
17:04jcromartiethe bullet is distracting though
17:05jcromartieI'd even out the spacing between the lib name/version, description, and author/date lines
17:05jcromartieand then add a little more space between each lib's listing
17:05jcromartieand de-emphasize the description and author and date a touch (with shading)
17:06scottjshading?
17:07raekscottj: I like the first one
17:08raekthen the third
17:08scottjgood those are both my designs :)
17:08scottjnot that there's that much different about them
17:08jcromartiescottj: like a de-emphasized color
17:10jcromartieor actually a horizontal divider might be nice
17:10jcromartiewhen in doubt, copy RubyGems
17:13amalloyhas anyone used SS's clojure-hadoop?
17:14amalloyi'm picking a client to use atm, and i'd love to be able to justify a choice of clojure
17:14ohpauleezamalloy: I haven't, but I'm planning on it
17:15ohpauleezwith hadoop+lazy seqs+delays+ persistent ds, it makes a lot of sense
17:17pjstadigscottj: number 1
17:18ohpauleezscottj: #1 for sure
17:19ohpauleezMore often than not, I search clojars, then look on github for the project, check the commit dates and look at the project.clj file
17:20ohpauleezIf clojars could pull out :url from a project.clj (when present), that would be AWESOME
17:20ohpauleez(totally icing on the cake though)
17:21ohpauleezI also often when to sort the results based on date or version (no idea how you're generating these results), but that's about the only other thing I've wished for from clojars
17:25scottjohpauleez: I've already implemented the url thing, but right now only showing it on the actual project page not search results
17:25ohpauleezahh, awesome! (scottj)
17:25ohpauleez"you're the man now dog"
17:26scottjif we added that and put brackets around the name/version there'd be no need for the project page and our clicks (and ad revenue) would go down :)
17:31scottjactually it was all pretty easy because ato had actually implemented it all he just didn't display it :)
17:31rlbI forget, does clojure have something like and-let?
17:32scottjrlb, what language is and-let from?
17:33rlbhttp://srfi.schemers.org/srfi-2/srfi-2.html
17:33rlbIt's like an and, but it lets you name each intermediate value, for use in subsequent forms without recomputation.
17:34scottjnot in clojure
17:41KirinDaveI wonder if I could entice people in SF to coffee bar on sunday afternoon for a hackmoot
17:55ohpauleezKirinDave: Being in OR is rough sometimes, stuck between Bay Area Clojure people and Seajure
17:55KirinDaveohpauleez: I'm in SF
17:55KirinDaveI thinkt here are many people here
17:55KirinDaveI've got permission to start open sourcing this clothesline stuff
17:55ohpauleezyeah, a good size for sure
17:55KirinDaveI wonder if people would be interesting in helping me get it ready for launch.
17:55hiredmantechnomancy gave a preview of his conj talk at seajure last night
17:55ohpauleezhiredman: How was it?
17:56ohpauleezKirinDave: I'm more than happy to dig in
17:56hiredmanit was good, I work with him so I get a lot of lein updates daily
17:56ohpauleezahh, awesome
17:57hiredmanhe found some elisp that takes org-mode and turns into a slide show in a special purpose buffer
17:57hiredmanso the whole presentation is in emacs
17:58ohpauleezwhoa, totally rad
17:59KirinDaveUgh
17:59KirinDave“Let's make this ugly, awkward, and nearly intractable.”
18:00slyrus_KirinDave: what's a hackmoot?
18:05KirinDaveYes. And it involves hacking ant talking about it.
18:05slyrus_I assume you mean "hacking and" not "hacking on ant?
18:06KirinDavecorrect. Sorry, typo
18:06KirinDaveA little hectic getting drink 2 here.
18:06slyrus_i'm not around sunday, but i'd be up for a clojurian lunch in SF one of these days
18:07lancepantztechnomancy: do you have a bar stool or anything you use for stand up programming/
18:08technomancylancepantz: I have a recliner next to my standing desk I use a couple hours a day
18:08lancepantzthat's a good idea
18:08technomancygotta work your way up to it
18:09lancepantzninjudd and i spent the morning raising our desks and stacking milk crates on top
18:09RaynesWhy would you want to stand up while programming?
18:10lancepantzsupposedly it helps combat add
18:10KirinDaveRaynes: It can be better for your back.
18:10KirinDavelancepantz: Wait what?
18:10RaynesI don't think standing at a computer for hours on end would be very helpful to me.
18:11ninjuddADD
18:11technomancyI think it helps combat lethargy
18:11technomancy</anecdote>
18:12lancepantzsupposedly, i don't have a source, i guess some conference talked about it
18:12lancepantzi like it though
18:12lancepantzalthough my feet do hurt
18:14amalloyuntil recently there was a guy at my office whose desk had automatic raising/lowering functionality. he did about half and half, said it was because sitting so much is bad for your back
18:14lancepantzyeah, i did find a bunch of links that it leads to heart disease somehow
18:14KirinDavelancepantz: Consider jellin'.
18:15KirinDavelancepantz: That NYT article?
18:15ohpauleezthe artists and programmers here at PushButton and Playdom are 30% standing 70% sitting
18:15KirinDaveThey were sorta playing fast-and-loose with the study.
18:15ohpauleezwith a lot of people converting to standing
18:15lancepantzKirinDave: that's the one
18:15ninjuddmy mom is a school counselor, and she was to a conference yesterday on strategies for dealing with ADD. one of the main ones they talked about was using a standing desk. which is funny, because i just reread the technomancy's take interview before i talked to her...
18:16ohpauleezninjudd: link?
18:16ohpauleezfound it
18:16lancepantzi think ideally i would like to raise and lower
18:16lancepantzohpauleez: really? i want to see it
18:17ohpauleezhttp://blog.fogus.me/2010/06/28/take-8-phil-hagelberg/
18:17lancepantzoh, i thought you meant to ADD thing
18:17KirinDaveI wonder if I will ever get on take 8 now that I am a full time clojure dev for a hot startup.
18:17KirinDaveProbably not.
18:17KirinDaveIf they ask, the jig will be up regarding xkcde. :\
18:18lancepantzhow many full time clojure developers do you think there are in the world?
18:18lancepantz100 would be my guess
18:18KirinDaveProbably a few hundred.
18:19ninjuddthey didn't mention keyboard pants at the conference though...
18:19lpetitRaynes: it probably also depends a lot on personal issues, and I come to think more and more while time passes on, the age you are. I certainly wouldn't have understood this at, say 20, while now, at 37, it does have some appeal to me (especially after the lunch :-) )
18:21lancepantzi can kinda jump and sway while coding, i dig that
18:22Deranderkeyboard pants!
18:23Deranderthat's brilliant!
18:34lpetitDerander: yeah. People in my office already start to try to escape from me when I start talking about clojure for too long. I'm pretty sure that by wearing keyboard pants I'll definitely be placed in my own office :-p
18:34lpetitBut really I like the concept ! :)
18:34DeranderI bet you could actually make them
18:38Rayneslpetit: Fair enough. I guess I just don't understand enough about human anatomy and they're life styles to totally understand why standing rather than sitting would be beneficial in any way.
18:39RaynesIn good don't-knock-it-until-you've-tried-it style, I'll save my complaints until I've actually seen it in action.
18:43carkhhuu lau are you there ?
18:44lpetitRaynes: while by no means being an expert in this field myself, I've read things here and there : staying in the same position for years is not good for the back. So having several hours in a standing position, and several hours in your seat is a good compromise. And also, there are some particular problems that can arise when sitting too long a day for too many years : hemorroids, to name them (too much pressure on your back, and some
18:46lpetitof course, not a thing a young man thinks about every day he wakes up, and that's totally normal and understandable ! (Oh man, is it me saying this ?)
18:52jarpiainrlb: (domonad maybe-m ...) is like and-let except it doesn't short-circuit on false (just nil)
18:53KirinDaveOh god bless ring.
20:34ssideris_how do you guys handle the situation when you are hacking on a project and at the same time at it's dependencies
20:34ssideris_is there a source dependency feature in leiningen?
20:34hiredmanyes
20:34hiredmancheckouts
20:35ssiderisah nice
20:36ssideristhanks, I think this will help me organise things much more cleanly
20:37hiredmanI am not sure where it's documented, but I believe if you create a directory named checkouts and put checkouts of the dependencies in there, the src dir of the dependency will end up on the classpath
20:38ssiderisyeah it seems that this is was'ts implied here: http://github.com/technomancy/leiningen/blob/master/README.md
20:38ssiderisI'll give it a go
21:59bhenryclojurebot: where is everyone?
21:59clojurebotExcuse me?
21:59bhenryclojurebot: you heard me
21:59clojurebotbhenry: I'm just giving you a second chance
22:00Raynessexpbot: whatis UGT
22:00sexpbotUGT = Universal Greeting Time: http://www.total-knowledge.com/~ilya/mips/ugt.html
22:09dnolenObjected Oriented Programming in JavaScript and Clojure, http://dosync.posterous.com/30036733
22:10dnolenfeedback appreciated
22:18jeff__good stuff. i'll look forward to the next post.
22:19dnolenjeff__: thx for the feedback
22:21jeff__i'm fairly new to the clojure community. i'll add your blog to my "watch list". btw, how do you syntax highlight clojure source in html?
22:22Raynesdnolen: I enjoyed it.
22:22dnolenRaynes: thx!
22:22Raynesdnolen: I'll be watching for the next installment.
22:22Raynesjeff__: I don't know what you're looking at, but it doesn't look like he highlighted it at all.
22:23jeff__sorry, i was looking at other entries on his blog: http://dosync.posterous.com/
22:24Raynesjeff__: Looks like he is embedding gists.
22:24Raynes$google embedding gists
22:24sexpbotFirst out of 128000 results is: Embedded Gists - GitHub
22:24sexpbothttp://github.com/blog/122-embedded-gists
22:24RaynesHeadshot!
22:25dnolenjeff__: yeah just embedded gists, you can actually just past gist links and posterous embeds them intelligently for you. pretty sweet.
22:25jeff__nice. that may come in handy.
22:25dnolens/past/paste
22:26Raynesdnolen: Really? I should have considered posterous a bit more, I suppose.
22:26dnolenRaynes: it's not totally ideal but it's much quicker than me waiting for me to write my Clojure blog engine ;)
22:27RaynesI'm running wordpress at the moment.
22:32jeff__i have a question about multimethods. is it possible to have a multi-arg multimethod? seems like all examples i find are single arg.
22:34jeff__for example, this doesn't work:
22:34jeff__(defmulti testfun (fn [a b] (keyword (str a))))
22:34jeff__(defmethod testfun :test [a b] (println a b))
22:35hiredmanyes it does
22:35jeff__then i'm using it wrong.
22:35jeff__(testfun 'a 'b)
22:35jeff__yields
22:35jeff__java.lang.IllegalArgumentException: No method in multimethod 'testfun' for dispatch value: :a (repl-1:11)
22:36jeff__oops
22:36hiredman
22:36jeff__sorry
22:36jeff__:)
22:37jeff__i obviously need another coffee.
22:40scottjI think your multi and (defmulti testfun keyword) are equivalent
22:41scottjI mean (comp keyword str)
22:41hiredmanno
22:42scottjhow so? to me it looks like the dispatch doesn't have to have take the same number of args as the method
22:43hiredmanit doesn't
22:43hiredmanbut his dispatch function takes two args and returns the keyword of the first arg
22:44hiredman(defmulti foo keyword) will pass the two args to keyword
22:44RaynesAnd (comp keyword str) would do the same thing.
22:44RaynesMaking for a spectacular explosion.
22:44scottj,(do (defmulti testfun (comp keyword str)) (defmethod testfun :test [a b] (println a b)) (testfun "test" "bar"))
22:44clojurebotDENIED
22:45RaynesYou'd think people would stop trying to def things after the 10 thousandth try.
22:45Raynes:p
22:45hiredmanstr will take the two args and turn them into a single string and keyword will turn it into a keyword
22:46hiredmanwhich is not the same thing as taking two args, return the keyword of the str of the first arg
22:46scottjok I see now, I would have thought that too but my repl was actually giving me wrong results there must be some basic practice with redefining multimethods I'm not aware of
22:47scottjwhat I pasted was actually working because I'd first defined the multi with the valid dispatch and redefining it apparently isn't picking up the change
22:47RaynesI was implying that it would do the same thing (as (defmulti foo keyword)) of which I was also wrong. Still, that could cause a less spectacular but more sinister explosion if you weren't expecting that.
22:47_atoah yeah... that changed in 1.2
22:47jeff__i wonder if that's what tripped me up. i was doing this in the repl last night and kept getting errors like this: Wrong number of args (1) passed to: user$eval83$fn (repl-1:12)
22:48scottjah man I need to stop looking at IRC I missed a touchdown!
22:48jeff__i just started a new repl, typed up that simple example and didn't even notice that it was working this. i think the evolution of my multimethod in the repl yesterday was the issue. it works in a fresh repl.
22:49jeff__ Wrong number of args (1) passed to: user$eval83$fn (repl-1:12)
22:49jeff__sorry for the duplicate paste of the error
23:40gerryxiaohello
23:40gerryxiaowhat's fn?
23:41gerryxiaoFn is empty interface
23:41gerryxiao,(fn? :meta)
23:41clojurebotfalse
23:42gerryxiao,(ifn? :key)
23:42clojurebottrue
23:42gerryxiao,(fn? +)
23:42clojurebottrue
23:43dnolen(doc fn?)
23:43clojurebot"([x]); Returns true if x implements Fn, i.e. is an object created via fn."
23:43dnolengerryxiao: ^
23:44gerryxiaodnolen: i know, but what is the meaning of Fn interface?
23:45dnolengerryxiao: IFn, the interface require for something to act as a Clojure function
23:45dnolens/require/required
23:46dnolenin any case you would use fn? normally to check if something is a function
23:46gerryxiaoyes, iFn is for function,but Fn is for What?
23:47amalloygerryxiao: Fn is functions defined with (fn). IFn is anything that is callable as a function, eg (:foo my-map)
23:47gerryxiaohmm
23:48gerryxiao,(fn? (fn []))
23:48clojurebottrue
23:48dnolen,(fn? #())
23:48clojurebottrue
23:48dnolen,(fn? {})
23:48clojurebotfalse
23:49gerryxiao,(fn? :a)
23:49clojurebotfalse
23:49dnolen(:a {:a 'foo})
23:49amalloy,(ifn? a)
23:49clojurebotjava.lang.Exception: Unable to resolve symbol: a in this context
23:49dnolen,(:a {:a 'foo})
23:49clojurebotfoo
23:49amalloy,(ifn? :a)
23:49clojurebottrue
23:50gerryxiaoi'm looking at clojure src, find that Fn.java just define one empty interface
23:51gerryxiaoa tag ?
23:58scottjlooks like it