2012-06-04
| 00:00 | muhoo | heh "I threw up im my mouth a little...ok a lot...while writing this." |
| 00:00 | technomancy | muhoo: as long as you don't look at the implementation you'll do fine! =) |
| 00:01 | technomancy | we re-use this eclipse s3 publishing module that idiotically hard-codes in the access-control to public. |
| 00:01 | technomancy | I wrote an implementation that wall-hacks it by calling private methods using reflection, and then a co-worker rewrote it "properly" in java |
| 00:03 | muhoo | fun times, sounds like. |
| 00:04 | muhoo | is there a way to push the repo with acl public? |
| 00:04 | technomancy | oh sure; just use s3:// in the repository URL rather than s3p:// |
| 00:04 | muhoo | perfect, thanks |
| 00:04 | technomancy | you don't need s3-wagon-private for that fwiw |
| 00:04 | technomancy | but it's probably better documented |
| 00:04 | muhoo | that's kind of what i thought |
| 00:04 | muhoo | maven repo just looks like a tree |
| 00:05 | muhoo | but then the sha1's need to be generated, all that. better to have a library do it for me |
| 00:05 | technomancy | it's just a pile of static files |
| 00:05 | technomancy | you can even install to a file:///tmp/whatever repository and copy that to an nginx dir or whatever if you like |
| 00:07 | muhoo | using this tool? or manually? |
| 00:08 | muhoo | oh manually will be fine. it's just jar's, pom's, and sha1s |
| 00:09 | technomancy | if you've already got a server and only have one or two jars that never change, that's easiest. if you've got something more complicated look into s3 or nexus |
| 00:09 | muhoo | fair enough, thanks for demystifying! |
| 00:09 | technomancy | or start with static files and look into something more sophisticated when you get tired of updating the static files =) |
| 00:43 | tomoj | is there already an easy way to hook the compilation events under cljsbuild auto? |
| 00:44 | craftybones | Hello all. Newbie to clojure, at #60something on 4clojure |
| 00:46 | tomoj | it does an eval-in-project which launches the compiler watching |
| 00:46 | tomoj | oh wait |
| 00:47 | craftybones | Loving it so far. What's the recommended book on Clojure |
| 00:48 | tomoj | cljsbuild already has rudimentary support for notification ;) |
| 00:51 | technomancy | craftybones: I like the oreilly one best |
| 00:51 | noidi_ | craftybones, I've only skimmed through O'Reilly's Clojure Programming, but it seems very good http://amzn.com/1449394701 |
| 00:51 | craftybones | technomancy: thanks |
| 00:51 | craftybones | noidi_: thanks |
| 00:52 | craftybones | where all does clojure get used in production? |
| 00:52 | craftybones | Clojure is the first time I've understood lisp in general. |
| 00:53 | technomancy | lots of places: http://dev.clojure.org/display/community/Clojure+Success+Stories |
| 00:55 | PeregrinePDX | I also like the Pragmatic Programmer book. |
| 00:55 | PeregrinePDX | Although I have the O'Reilly book as well. |
| 01:03 | Hodapp | The only reason I use Lisp is because I find it hilarious the sort of mental gymnastics people perform when they think they're superior coders but they can't wrap their mind around why something like Lisp would be useful. |
| 01:03 | Hodapp | er, did I say that out loud? |
| 01:04 | craftybones | Hodapp: :) Doesn't matter, I actually get Lisp now as opposed to before when I didn't get Twist |
| 01:05 | craftybones | Hodapp: I meant Lisp. Twist is the project I work on :) |
| 01:07 | Hodapp | I'm mostly learning Scala right now though. |
| 01:19 | muhoo | technomancy: the jars for these things depend on a foo-parent , which is in a top-level pom.xml . any idea where i'd put that in the tree? |
| 01:19 | muhoo | to make this more concrete: http://code.google.com/p/step2/ |
| 01:20 | technomancy | hm; sorry, I don't know anything about bare poms |
| 01:25 | muhoo | haha, it worked. i created a dir called step2-parent, put the bare pom in there, and at least pomegranate is happy :-) |
| 01:26 | muhoo | https://www.refheap.com/paste/2988 , FYI |
| 01:28 | muhoo | i am amazed at how much bureucracy is involved in this maven stuff. makes me especially thankful for leiningen. |
| 01:34 | muhoo | gotta love java too. a class, that has one constructor that takes a string as an arg, and one public method, that returns the string given in the constructor |
| 01:41 | Hodapp | muhoo: It's.... object-oriented! Right? RIGHT? |
| 01:43 | ivan | maybe it is not a string but something else with a string payload |
| 01:44 | bbloom | muhoo: facepalm indeed. any class that has 1 method should just be a damn function. that sort of thing drives me nuts |
| 01:49 | Hodapp | bbloom: Java doesn't like verbs. |
| 01:50 | Hodapp | bbloom: I was also quite annoyed at how many constructs I saw in the C++ mess at work that were like... ITriangleIntersector, implemented by TriangleIntersector, with functions like setTriangle |
| 01:51 | Hodapp | makes optimization and concurrency a bit harder when something normally done by one function is now done by a stateful class. |
| 01:52 | bbloom | Hodapp: for sure. but at least in C++ it makes some sense if you've got a constructor/method/destructor trifecta for RIAA. however, even then, you probably want a generalized container like a smart pointer or context object of some kind |
| 03:06 | lypanov | currently one part of my app uses agents to "fire it off in background" and another uses "wait for everything to finish" to delay until its finished. |
| 03:06 | lypanov | agents work perfectly for this but id like to also be able to cancel. |
| 03:06 | lypanov | is there a way to coordinate futures to get the same behavior? |
| 03:16 | gfredericks | ,(doc future-cancel) |
| 03:17 | gfredericks | &(apropos "future") |
| 03:17 | lazybot | java.lang.RuntimeException: Unable to resolve symbol: apropos in this context |
| 03:18 | gfredericks | &(clojure.repl/apropos "future") |
| 03:18 | lazybot | ⇒ (future-done? future future-call future? future-cancel future-cancelled?) |
| 03:18 | muhoo | and this is what i have spent the night doing. https://www.refheap.com/paste/2989 |
| 03:18 | muhoo | or rather, HAVING to do :-/ |
| 03:18 | gfredericks | lypanov: would a combination of future-done? and future-cancel get you that? |
| 03:19 | gfredericks | muhoo: I like (fn [k r o n] ...) |
| 03:23 | gfredericks | it's crazy how far I got through that macro before realizing it was emitting cljs |
| 03:26 | amalloy | lypanov: future-cancel isn't really a 100% reliable mechanism anyway: once a thread is running, it's hard to interrupt it, if it's not planning/expecting to be stopped |
| 03:27 | amalloy | you'd be better off putting some code in your worker threads whereby they can check whether someone has requested that they stop |
| 03:29 | lypanov | its blocking on an external process. |
| 03:29 | lypanov | amalloy: would ^ be interrupted or not? |
| 03:30 | amalloy | *shrug* |
| 03:30 | lypanov | ah no, in fact i take that back. |
| 03:30 | amalloy | i don't really know the details of the thread-interruption mechanism |
| 03:30 | lypanov | ok |
| 03:31 | amalloy | i do know it's changed a few times since JDK 1, because a lot of totally-broken algorithms were used |
| 03:33 | lypanov | http://stackoverflow.com/questions/671049/how-do-you-kill-a-thread-in-java |
| 03:33 | lypanov | "See this thread by Sun on why they deprecated Thread.stop()." |
| 03:34 | lypanov | as the page referenced says, .interrupt() will work for the above external process example. |
| 03:34 | lypanov | but .stop() will almost never work. |
| 03:34 | lypanov | so "putting some code in your worker threads whereby they can check whether someone has requested that they stop" is the suggested path. :) |
| 03:35 | lypanov | thx amalloy! |
| 03:35 | amalloy | yes, i imagine future-cancel uses interrupt, somewhere |
| 03:37 | lypanov | amalloy: yup. it uses interrupt. |
| 03:39 | calvados | is there a tool to get test coverage reports in jenkins for clojure ? |
| 03:45 | gfredericks | there's a cljs lib for canvas-y graphics already, isn't there? |
| 03:46 | lypanov | several. |
| 03:47 | gfredericks | any pointers? :) |
| 03:47 | lypanov | amalloy: sorry to bug but another question, whats the advantage to futures over agents if cancel doesn't work? |
| 03:48 | lypanov | amalloy: can't i just as well store a "id" in an atom and in my agent task check the id every once in a while to verify if i shouldn't cancel myself? |
| 03:49 | amalloy | agents are supposed to model a value, changing over time. if you never inspect or change that value because the action never returns, what is the point? |
| 03:49 | amalloy | agents also use a different threadpool, and probably have a bit more overhead to keep an action queue and ensure it's updated in a threadsafe way |
| 03:53 | gfredericks | I thought I could use lein-cljsbuild's multi-builds feature to have src code and test code, but doing that in the naive way the two JS files couldn't be combined since they had a lot of overlap; is there a normal way to allow building src and test code separately? |
| 03:56 | tomoj | why do you want to combine them? |
| 03:57 | gfredericks | the test code tests the app code |
| 03:57 | gfredericks | so the test code by itself is not very useful |
| 03:57 | tomoj | if you compile the test code, the compiled js will include whatever app code is tested |
| 03:57 | tomoj | maybe I don't know what you mean |
| 03:57 | gfredericks | oh -- maybe my mistake was including both main.js _and_ test.js |
| 03:57 | gfredericks | maybe test.js would've sufficed |
| 03:58 | tomoj | just curious, how are you testing? assert? |
| 03:58 | gfredericks | yeah; I wrote an is and are macro, but yes |
| 03:59 | tomoj | cool |
| 03:59 | gfredericks | the are macro was surprisingly easy :) |
| 03:59 | tomoj | I'm testing from tests written in js so far, since I would be too tempted to get sucked into building a test framework (and I want the library to be usable from js..) |
| 04:00 | tomoj | excited to see testing libraries start popping up |
| 04:12 | sergey | hey |
| 04:15 | lypanov | amalloy: i don't understand what you mean by "because the action never returns" |
| 04:15 | lypanov | amalloy: that would only be the case if it was cancelled. i'd only cancel because a new request arrived and the existing request will use up time if it continues. |
| 04:15 | lypanov | ah but then i have another bug. |
| 04:16 | gfredericks | lypanov: you're ignoring the agent's state |
| 04:16 | lypanov | sod it. this is good enough and 3 lines of code. |
| 04:18 | lypanov | gfredericks: not true. the state is the result of the agents work and is what my web action returns. |
| 04:19 | gfredericks | oh okay. Then you're only using it once :) |
| 04:19 | lypanov | nope. |
| 04:19 | lypanov | gfredericks: file watcher -> send-off. request -> await and reply with new state. |
| 04:19 | lypanov | its a compile cache. |
| 04:20 | gfredericks | lypanov: I don't know what anybody's talking about then |
| 04:20 | lypanov | gfredericks: hehe. |
| 04:20 | lypanov | amalloy's comment helped anyway. it make me realize that its just not worth the extra effort to use futures/add cancel logic. |
| 04:21 | lypanov | as agents work by unblocking awaits when the list of pending at the time of the call to await is let go. |
| 04:22 | lypanov | this means if i press save on a file twice, the first one would be cancelled and get the old state whereas what i really want is for it to wait until the second completes. |
| 04:35 | Miko2 | Is there a "deep" variant of transient? |
| 04:43 | gfredericks | transients of transients? |
| 04:47 | tomoj | I see no need for a deep variant |
| 04:51 | tomoj | could you make clojure.zip work with transients? |
| 04:55 | ro_st | so i think i asked this before but i can't find the answer in any of the code i have written already. i have a seq of {modelA_id: … modelB_id: … } and i want to gather that into {(modelA_id's value): [(modelB_id value) (modelB_id value) …]} |
| 04:56 | ro_st | so use A's value as a key for a sequence of all of B's values that map to A in the source map |
| 04:58 | ro_st | i'm guessing that a (for) form will be used here but not sure how to construct it |
| 04:59 | gfredericks | you want [{:a 2 :b 3} {:a 2 :b 4} {:a 3 :b 10}] => {2 [3 4] 3 [10]}? |
| 04:59 | ro_st | that's right |
| 04:59 | gfredericks | well group-by can get you halfway there |
| 04:59 | borkdude | group-by |
| 04:59 | tomoj | group-by always bothers me |
| 05:00 | gfredericks | (into {} (for [[k v] (group-by :a your-maps)] [k (map :b v)])) |
| 05:00 | tomoj | if it had a [kf vf coll] version, it would be great |
| 05:00 | gfredericks | tomoj: so that we could (group-by :a :b coll) here? |
| 05:00 | tomoj | yeah |
| 05:01 | gfredericks | agrd |
| 05:01 | tomoj | really easy to do.. |
| 05:01 | tomoj | would be somewhat surprised if it hadn't been proposed |
| 05:03 | tomoj | maybe it can be fixed in reducers |
| 05:03 | tomoj | s/fixed/added/ |
| 05:03 | ro_st | or perhaps it could be in clojure 1.5 :) |
| 05:05 | borkdude | propose it! |
| 05:06 | tomoj | so if you (r/group-by identity {:a :b}), should you get {:a :b}, an error,... what? |
| 05:07 | tomoj | (the argument for {:a :b} is that (r/group-by identity {:a :b}) is short for (r/group-by identity identity {:a :b}) and that group-by should cooperate with reduce-kv) |
| 05:09 | tomoj | hmm |
| 05:09 | tomoj | that's not right |
| 05:11 | tomoj | (r/reduce (r/group-by (fn [k v] k) (fn [k v] v)) {:a :b}) => {:a :b} ? |
| 05:12 | tomoj | er, {:a [:b]} of course |
| 05:14 | tomoj | and (into {} (r/group-by ..)) |
| 05:14 | tomoj | ? |
| 05:16 | gfredericks | what was the name of fogus's cljs app? |
| 05:17 | ejackson | himera ? |
| 05:17 | gfredericks | ah jes |
| 05:17 | gfredericks | yes |
| 05:18 | gfredericks | totally doesn't start with "mo" |
| 05:19 | SuperfastJellyfi | Fetching dependencies with Leiningen seems to be not working from here? Does anyone else have this problem? |
| 05:20 | ordnungswidrig | SuperfastJellyfi: please be more specific |
| 05:22 | SuperfastJellyfi | We created a list with dependencies in our project.clj file, but at a sudden we got a OverConstrainedVersionException with fetching the dependencies from the remote repositories: central (http://repo1.maven.org/maven2) and clojars (http://clojars.org/repo/) |
| 05:23 | SuperfastJellyfi | and before it was working without a problem |
| 05:23 | ordnungswidrig | what did you change? |
| 05:23 | ordnungswidrig | du you use ranges? |
| 05:24 | SuperfastJellyfi | Nothing, even if we create a new project, we get this error. |
| 05:24 | SuperfastJellyfi | On two different computers. |
| 05:24 | SuperfastJellyfi | We don't use ranges |
| 05:25 | borkdude | SuperfastJellyfi hi |
| 05:25 | borkdude | SuperfastJellyfi what version of leiningen do you use? |
| 05:26 | borkdude | en please post the project.clj on gist.github.com |
| 05:26 | SuperfastJellyfi | borkdude: 1.7.1 |
| 05:28 | SuperfastJellyfi | borkdude: https://gist.github.com/2867410 |
| 05:31 | tomoj | why don't sorted maps of persistentqueues make good priority queues? |
| 05:35 | borkdude | SuperfastJellyfi it seems that noir 1.2.1 and korma are causing trouble |
| 05:35 | borkdude | SuperfastJellyfi when I exclude them it works fine |
| 05:37 | borkdude | SuperfastJellyfi maybe noir 1.21 uses clojure 1.2? |
| 05:38 | borkdude | 1.2.1 |
| 05:38 | SuperfastJellyfi | borkdude: is 1.2.1 deprecated now? |
| 05:39 | borkdude | SuperfastJellyfi I guess not |
| 05:39 | borkdude | SuperfastJellyfi maybe try #leiningen for this |
| 05:40 | SuperfastJellyfi | borkdude: ok thanks for your effort |
| 05:41 | borkdude | SuperfastJellyfi something seems to depend on clojure 1.2.0 |
| 05:42 | borkdude | SuperfastJellyfi ask this in #leiningen for further info |
| 05:53 | SuperfastJellyfi | borkdude: k thx |
| 06:13 | almostobsolete | I'm using Clojure in emacs with slime. When reading a source file most of the built functions I can jump to with slide-edit-definition but most libraries (required via project.clj) I can't. I'm guess this is because I don't have the source for those libraries installed. Can I instruct lein to install sources so I can jump to them? would that work? |
| 06:19 | ggg | hi |
| 06:22 | amalloy | almostobsolete: normally that works just fine, as long as you have the libraries in your classpath (normally because you've connected with lein-swank and it's in your project.clj), and you've loaded the code via C-c C-k or something |
| 06:42 | PKHG | Hello, who knows about 'round' used in http://www.gettingclojure.com/cookbook:graphics#ascii-bar-chart but clojure gives: java.lang.Exception: Unable to resolve symbol: round in this context |
| 06:45 | ejackson | PKHG: try Math/round |
| 06:45 | PKHG | thanks will use that one ... |
| 06:46 | PKHG | (replacing) ;-) |
| 07:02 | magnars | clojure-mode: How do I go about getting emacs into clojure-test-mode automatically after doing clojure-jack-in? It's a bit of a hassle to manually do M-x clojure-test-mode. |
| 07:08 | bsteuber | does anyone know how to get rid of the "Failed to collect dependencies for clojure.lang.LazySeq@3eab22f5" in recent lein? |
| 07:08 | bsteuber | happens with both preview2 and preview6 |
| 07:10 | bb_oz | #leave |
| 07:12 | wink | bsteuber: someone in #leiningen posted something like that earlier, care to post your dependencies there as well? |
| 07:20 | almostobsolete | amalloy_: Thanks! It doesn't seem to work for me though. I'm not loading code with C-c C-k, I just run clojure-jack-in then browse the source files |
| 07:33 | antares_ | bsteuber: can you post the entire stack trace? |
| 07:33 | antares_ | bsteuber: what OS and JDK do you use? |
| 07:35 | bsteuber | antares: https://gist.github.com/2867852, OSX, Oracle |
| 07:36 | bsteuber | it might be the ssl problem here, though exclusions helped make deps work |
| 07:36 | bsteuber | I just get a compile error now ^^ |
| 07:36 | antares_ | No versions available for org.clojure:clojure:jar:[1.2.1],[1.3.0] within specified range |
| 07:37 | antares_ | so yes, something somehow drags the 1.2.1 requirement |
| 07:37 | antares_ | bsteuber: for the Oracle JDK issue (they do not bundle some SSL certificates), see http://architecturalatrocities.com/post/19073788679/fixing-the-trustanchors-problem-when-running-openjdk-7 |
| 07:38 | bsteuber | ok I'll try, though I never saw the actual SSL message |
| 08:22 | ro_st | this might be a daft question, but how many concurrent users can a heroku dyno support with, say, a mongodb-backed json rest api? |
| 08:22 | ro_st | trying to get a handle on when i'd need to add dynos |
| 08:23 | ro_st | also i'm guessing it's simple to have a ring response gzip its contents |
| 08:24 | weavejester | ro_st: There's some middleware to do that, I believe |
| 08:44 | Hodapp | bbloom: problem is, as soon as you start having to introduce features like that, things rapidly get to be an overcomplicated mess... |
| 09:24 | ro_st | anyone running multi-dyno apps on heroku? |
| 09:24 | ro_st | trying to get a sense of scaling requirements |
| 09:52 | gfredericks | Is there any reason core.logic couldn't have a condr that tries the clauses in a random order? |
| 09:54 | dnolen | gfredericks: probably not, and for what purpose? |
| 09:55 | gfredericks | dnolen: if I'm interested in getting a random match, it would be faster than (rand-nth (run 10000 [q] ...)) |
| 09:58 | dnolen | gfredericks: you could write a goal that takes a vector of goals and tries a random one. |
| 10:01 | gfredericks | dnolen: that's a good idea; thanks |
| 10:13 | jweiss | any emacs users know how to get a different repl buffer for each project you're working on? I use clojure-jack-in, it asks if i want to kill the existing process, if i say no, it just switches the existing slime repl to the new project. |
| 10:17 | tmciver | jweiss: I just use separate emacs instances. |
| 10:18 | jweiss | tmciver: doing 'lein swank' from cmdline and then slime-connect does give me 2 separate buffers |
| 10:18 | jweiss | i am not sure why clojure-jack-in doesn't just do the right thing |
| 10:32 | si14 | hey, guys. anyone using ClojureScript with emacs here? Is there anything better than "inferior lisp" for browser REPL? |
| 10:50 | ro_st | any one running clojure web apps in production on heroku? what sort of reqs/sec are you getting per dyno? |
| 10:51 | dnolen | si14: not yet |
| 10:52 | si14 | dnolen: are there any plans for it? |
| 10:52 | dnolen | si14: even then to get full good-ness someone needs to design and write some kind of reflection API |
| 10:52 | si14 | dnolen: ah, you're right |
| 10:52 | dnolen | si14: plans ... as in we'd like for someone to do it. |
| 10:52 | si14 | dnolen: yeah, I see :) |
| 10:53 | si14 | dnolen: btw, I should say you "thank you" for your work at ClojureScript :) |
| 10:54 | si14 | it's awesome. has some issues here and there, but still awesome |
| 10:54 | dnolen | si14: thx! there's tons to do as far as CLJS goes, so these are just a matter of time. personally focusing on source maps and data struture perf. |
| 10:54 | dnolen | at the moment |
| 10:54 | ro_st | source maps: woohoo |
| 10:54 | ro_st | for use with the new chrome dev tools? |
| 10:55 | si14 | dnolen: nice to know. |
| 10:56 | dnolen | ro_st: yes |
| 10:56 | ro_st | awesome! |
| 11:30 | solussd | is it possible to add a 'class method' to a record via a protocol ? |
| 11:31 | ezyang | I'm getting null pointer exceptions from PersistentTreeMap, and I'm wondering if there are any invariants that I should know about but don't. |
| 11:31 | TimMc | solussd: Pretty sure there's no mechanism for class methods except via gen-class, and that might not count. |
| 11:32 | TimMc | ezyang: Maybe it doesn't accept null keys or values? |
| 11:32 | TimMc | &(assoc (clojure.lang.PersistentTreeMap.) :a nil) ;; no, that's fine |
| 11:32 | lazybot | ⇒ {:a nil} |
| 11:33 | TimMc | Can you make a reduced testcase? |
| 12:52 | ezyang | TimMc: Working on it. |
| 12:55 | ezyang | TimMc: Here we go: http://hpaste.org/69503 |
| 13:03 | ezyang | Somewhat relatedly, if I want to work on Clojure's JVM classes, do I load it into Eclipse? |
| 13:05 | dnolen | ezyang: you can just add the clojure.jar to your project if you're using a Java IDE |
| 13:06 | ezyang | Hmm, but that is a bit annoying because Eclipse doesn't understand how to get out sources. |
| 13:06 | dnolen | ezyang: you want the sources for reference? |
| 13:07 | ezyang | Mostly for debugging actually (see the error I pasted above) |
| 13:08 | ezyang | I'm sure I'm trampling over some Clojure invariant (because obviously tree maps work in real Clojure apps) but I don't know what. |
| 13:08 | ezyang | My current conjecture is that Clojure assumes that static initializers are loaded in a specific frder. |
| 13:12 | dnolen | ezyang: should probably ask on the ML |
| 13:12 | ezyang | OK. The general one or the dev one? |
| 13:12 | dnolen | ezyang: general one |
| 13:14 | TimMc | ezyang: That's pretty weird. I don't see what could be wrong. |
| 13:25 | ezyang | posted |
| 13:33 | ezyang | Well, I found a workaround. |
| 13:34 | ezyang | Also, the Clojure list hates my emails :-( |
| 13:34 | ezyang | It took four days for my last message to show up. |
| 13:35 | bsteuber | the cljs compiler used in cljsbuild 0.2.1 seems to rename throw() in advanced compilation |
| 13:35 | twhume | I'm doing something evil and unusual, and I could do with some help. I want to create, briefly use, and then dispose of a large number of Classes (literally millions). I create the class in the "let" of a function, and it is never returned; so I'd expect it to be garbage-collected, but it isn't being. After about 57,000 classes are created, I get an OutOfMemoryError. Any ideas? Code at http://pastebin.com/Z3Kk0Yu0 |
| 13:37 | bsteuber | oh nevermind, thought the rename is part of my problem, but it isn't.. |
| 13:38 | borkdude | twhume have you tried dorun? |
| 13:38 | twhume | borkdude: do you mean instead of doall in the final line? |
| 13:39 | borkdude | twhume doall retains the head, so all the classes will be in memory the same time |
| 13:39 | twhume | aha… will give that a go now. |
| 13:39 | twhume | Nope, falls over at the same point. |
| 13:40 | raek | huh, did github just start using https instead of ssh by default? |
| 13:41 | twhume | borkdude: and would they all be held in memory anyway? They're created in that test-class function and never used outside it - so I would expect them to be GCd (or available for GCing) when it's done... |
| 13:41 | borkdude | twhume good point |
| 13:42 | borkdude | twhume what does get-class do? |
| 13:43 | twhume | it uses DynamicClassLoader to load a class from a byte array of Java byte code (using ClassLoader.defineClass) |
| 13:43 | dnolen | ezyang: some bugs around ML approval, should work after the first approval. |
| 13:43 | twhume | …and returns the class thus loaded. |
| 13:43 | borkdude | twhume maybe references to the returned class are retained elsewhere? else I don't know |
| 13:44 | twhume | Yep, I'm guessing that something somewhere is keeping them, otherwise the JVM would GC - but I can't tell what that would be, and how I tell it not to... |
| 13:46 | twhume | I've tried forcing a GC after every thousand loads (so there are 57 GCs being requested before it falls over), but that doesn't help… I'm a bit stumped, and not sure where to look. |
| 13:46 | borkdude | twhume I would look in the get-class function |
| 13:46 | dnolen | twhume: have you tried that code in pure Java? |
| 13:46 | TimMc | I *think* OOM is guaranteed to only happen if GC doesn't help. |
| 13:46 | twhume | dnolen: no, that sounds like a good next step, doesn't it? |
| 13:46 | dnolen | twhume: yep |
| 13:47 | twhume | I'll give it a go now. |
| 13:47 | S11001001 | doesn't necessarily mean that gc has to run to find an OOM |
| 13:49 | TimMc | Wait, I'm thinking of SoftReference or something. |
| 13:49 | TimMc | twhume: Is this that thing where you are generating all possible classes? |
| 13:50 | twhume | TimMc: yep, that's it. |
| 13:55 | twhume | Hmm, in pure Java I can go to 100,000 classes loaded without trouble. So this looks Clojure-specific. |
| 13:55 | twhume | s/go to/generate/ |
| 13:57 | dnolen | twhume: what does you code look like w/o the doall? did you try dotimes? |
| 13:58 | patrkris | hi folks. viewing the cheatsheet on clojure.org, it seems that you're supposed to call the method .indexOf to find the index of an element in a vector. can this really be true? |
| 13:58 | twhume | dnolen: I've not tried dotimes, no. W/o the doall, it's as in http://pastebin.com/Z3Kk0Yu0 but with doall replaced by dorun. |
| 13:59 | technomancy | patrkris: I think it's a subtle hint that a vector may not be the data structure you're looking for if you want that |
| 13:59 | patrkris | ah |
| 14:00 | patrkris | technomancy: i'm using a vector to define a sort order of a set of items. do you have any suggestion to do something else? |
| 14:00 | dnolen | twhume: try dotimes, your code is just side effects, range and dorun/all not necessary |
| 14:01 | nDuff | patrkris: a map tracking object->index would make more sense in that case, being O(1) lookup. |
| 14:01 | nDuff | patrkris: ...a vector being O(n) to search through makes it very much not the right tool for the job. |
| 14:02 | patrkris | nDuff: yes, I thought about that, but had the idea that a vector is a little more succinct, and since there aren't many items, it shouldn't take long |
| 14:02 | patrkris | but I think I'll walk the map road |
| 14:02 | technomancy | if you know it's never going to get big, .indexOf is fine |
| 14:02 | twhume | Sorry, was bounced out. |
| 14:02 | twhume | dotimes displays the same behaviour - falling over after 57,000 or so classes loaded. |
| 14:02 | mebaran151 | patrkris: by the way for very tiny maps, Clojure does an optimization, using an ArrayMap instead |
| 14:03 | patrkris | coolio |
| 14:03 | mebaran151 | which probably works very similarly in terms of algorithm to indexOf |
| 14:03 | nDuff | twhume: is this PermGen space or heap that you're exhausting? |
| 14:03 | twhume | PermGen |
| 14:03 | nDuff | ahh. |
| 14:04 | nDuff | ...yeah, then; permgen space isn't typically garbage collected. Which arguments are you starting your JVM with? |
| 14:04 | nDuff | (and which JVM release is it?) |
| 14:05 | nDuff | twhume: ...also, have you set -XX:MaxPermGenSize to give yourself more breathing room? |
| 14:05 | twhume | nDuff: I think it's java version "1.6.0_31" (on mac), I'm working in Eclipse and CounterClockwise is launching the REPL for me so I'm not 100%. |
| 14:06 | twhume | nDuff: I can do that, but in future I'd like to be loading literally millions of classes… upping MaxPermGenSize might help me do more, but there seems to be an underlying issue with Clojure here (given that I can do what I want in straight Java, but not in Clojure)? |
| 14:06 | dnolen | twhume: did you try loading a million classes in Java? |
| 14:07 | twhume | dnolen: I tried loading 100,000 and it worked fine. Clojure repeatedly falls over at 57,000. I'll do a test with a million in Java now... |
| 14:07 | nDuff | twhume: I don't accept that premise, actually -- loading a million classes without enough permgen shouldn't be possible in Java either. |
| 14:07 | nDuff | twhume: ...that said, Clojure _does_ use more permgen space, to be sure, as it generates lots of little classes as a matter of course. |
| 14:07 | dnolen | twhume: one possible difference is that Clojure itself loads up a class per fn |
| 14:08 | twhume | …OK, that might explain a slightly higher limit for Java. Doing a test with 1m now. |
| 14:08 | nDuff | twhume: I'd suggest using jmap + jhat to look for situations that might be preventing GC, by the way. |
| 14:09 | nDuff | twhume: see http://frankkieviet.blogspot.ca/2006/10/how-to-fix-dreaded-permgen-space.html |
| 14:09 | twhume | thanks! |
| 14:11 | nDuff | (by the way, one place where I _do_ get permgen leaks, and am Very Not Happy about it, is the Clojure runtime not being fully unloadable -- tends to lead remnants in thread-local space, such that unloading and reloading plugins written in Clojure can result in duplicate copies of clojure.* classes accumulating, each tied to a new, plugin-specific classloader instance) |
| 14:11 | twhume | OK, I've just managed to generate and load 1m classes from a pure Java version on the same machine; so I don't think the issue is overhead of Clojure, I think something is keeping those generated classes hanging around... |
| 14:12 | nDuff | ...well, jmap+jhat are your friends. |
| 14:12 | gtrak | twhume: how big are those classes relative to your permgen and whatever's loaded up by clojure startup? |
| 14:12 | twhume | I also notice that in Java, I can repeatedly regenerate and reload the same class name. On Clojure I get a ClassLoader error unless I rename the classes each time I generate them. |
| 14:13 | twhume | gtrak: the classes are minimal, containing a 2-opcode identity method and nothing else. |
| 14:14 | nDuff | twhume: ...eh? Sounds like you're doing something interesting, then. |
| 14:14 | gtrak | I wonder if the extra runtime classes would make a big hit, then... if you increase permgen and the gap gets smaller you would know |
| 14:15 | twhume | nduff: exhaustive search of possible programs, to find non-intuitive optimal ones (it's called a superoptimiser - not my term - http://en.wikipedia.org/wiki/Superoptimization ) |
| 14:15 | nDuff | twhume: meant interesting in terms of your implementation, to trigger those behaviors. |
| 14:15 | twhume | gtrak: possibly, I'll give that a go and see |
| 14:15 | hiredman | twhume: are you loading the classes in both cases (the java and the clojure case)? |
| 14:16 | dnolen | twhume: so in the Java code are those classes getting GCed right away? |
| 14:16 | hiredman | twhume: how are you loading it in the java case? still using dynamicclassloader? |
| 14:16 | twhume | hiredman: hmm. Am definitely loading them in the Java case (and invoking a method). In the Clojure case I'm creating and loading them but not using them (in my test code exercising this problem: in the real thing I load and invoke them). |
| 14:16 | twhume | dnolen: yes, looks that way. |
| 14:16 | hiredman | twhume: what is the parent classloader in both cases? |
| 14:16 | mebaran151 | hey all, what's the best way in clojure to fetch all the elements from an xml document by tag; the output from clojure.xml/parse seems a little raw for that |
| 14:17 | srid | other than one and pinot, are there any new clojurescript frameworks of late? |
| 14:17 | hiredman | twhume: how are you loading them in the java case? |
| 14:17 | twhume | hiredman: in Java I've written a class which extends ClassLoader and calls defineClass(). In Clojure I'm using the DynamicClassLoader. |
| 14:17 | hiredman | twhume: have you looked at the source of dynamicclassloder? |
| 14:18 | twhume | hiredman: had a quick look, it seemed straightforward. I think that's next (after playing w/permgen settings). |
| 14:18 | twhume | Folks, thanks for all your help. I have to head off now but will look into your suggestions (and those links, nDuff). Will probably be back here tomorrow :-$ |
| 14:19 | hiredman | I find it hard to believe someone could look at the source and still ask about dynamicclassloader keeping a reference |
| 14:22 | rlb | If you wanted to dump a large number of random integers (as text) to a file as quickly as possible, is a tail recursion and .write probably the best way? |
| 14:23 | rlb | (just toying with a conversion of someone's trival perf test) |
| 14:25 | nDuff | rlb: ...for microbenchmarks, details matter (so what kind of structure those random numbers are stored in, for instance, matters)... but generally, yes-ish. Might want a buffered writer. |
| 14:27 | rlb | they're generated via (rand-int MAX) |
| 14:27 | jweiss | is there any way to refer to "this" in a record? eg, (defrecord myrec [a b]) (myrec. {:foo (:bar this)} :baz) |
| 14:28 | jweiss | that's a bad example, but the point is i want to pass in a literal as one of the args to the constructor, and inside there be able to refer to the overall object |
| 14:29 | dnolen | jweiss: you could do it via an atom or a mutable field - neither recommended. |
| 14:29 | magnars | emacs clojure-mode: any way to jump into clojure-test-mode after a clojure-jack-in more conveniently than M-: (require 'clojure-test-mode) M-x clojure-test-mode ? |
| 14:31 | technomancy | magnars: if you install clojure-test-mode the right way it will be activated automatically when you jack in. |
| 14:31 | technomancy | using package.el will set up the autoloads for you |
| 14:31 | technomancy | otherwise put the require in your init.el |
| 14:32 | magnars | technomancy: oh, that is excellent. putting the require in init.el didn't work out for me, but I'll give package-install a go. Many thanks. |
| 14:34 | bbloom | dnolen: i noticed that code blocks inside expressions are implemented via (function(){…})() instead of the comma operator. is that something gclosure reliably optimizes? |
| 14:34 | dnolen | bbloom: yes |
| 14:35 | bbloom | dnolen: cool. that makes things eaiser |
| 14:35 | dnolen | bbloom: though you do have to careful in some cases - one rule of thumb, if a fn doesn't get unwrapped at :simple - it won't get unwrapped. |
| 14:35 | bbloom | dnolen: I was mostly looking at emit-block |
| 14:36 | magnars | technomancy: as for just popping (require 'clojure-test-mode) into init.el, it gives me this: (file-error "Cannot open load file" "slime") unless I have an open slime-repl |
| 14:36 | bbloom | dnolen: in particular, emit :do |
| 14:37 | technomancy | magnars: oh I guess that makes sense. you can put it in (eval-after-load 'slime '(require 'clojure-mode.el)) |
| 14:37 | technomancy | but handling all this stuff for you is what package.el is good at |
| 14:38 | dnolen | bbloom: it's pretty much always done in expression contexts because of JS rules about where expressions can go. |
| 14:40 | bbloom | dnolen: yeah, a simple search for ")()" shows that. I was just wondering if there would be any benefit to using parens and the comma operator, but if gclosure optimizes it, then it's all good |
| 14:42 | dnolen | bbloom: parens is not enough in many cases far as I know. |
| 14:43 | bbloom | dnolen: i'd believe that, but i can't think of any off the top of my head. which cases? |
| 14:43 | dnolen | if(... complex test ) { ... } |
| 14:44 | bbloom | if ((console.log("x"), true)) console.log("y"); |
| 14:44 | bbloom | that seems to work in node |
| 14:44 | dnolen | bbloom: it's not enough, you need to consider how complex the test might be |
| 14:44 | dnolen | (if (let [x (or ...)] ...) ...) |
| 14:46 | magnars | technomancy: deleted my clojure-mode in site-lisp/ and installed with package.el - it's required fine, but still have to do M-x clojure-test-mode to activate after jacking in. |
| 14:47 | dnolen | bbloom: (if (some-macro ...)) is another one, satisfies? is a pretty complex one that actually appears in core.cljs |
| 14:48 | technomancy | magnars: it should loop over open clojure-mode buffers and apply it if it detects that they're tests |
| 14:48 | rlb | a loop calling (rand-int MAX) appears to be much faster than one calling (.nextInt rng MAX) |
| 14:49 | rlb | (or I'm doing something wrong...) |
| 14:49 | technomancy | meaning if the namespace includes the string "test" |
| 14:49 | neotyk | rlb: consider using channels, you will spend most time in I/O |
| 14:50 | neotyk | rlb: http://docs.oracle.com/javase/6/docs/api/java/io/FileOutputStream.html#getChannel() |
| 14:51 | magnars | technomancy: yes, you're right. I assumed it would put all clojure-mode buffers into clojure-test-mode. Sorry for the confusion. It works nicely installed from package. Thanks again for the help. |
| 14:51 | bbloom | dnolen: just experimenting w/ the compiler & a js repl. i realize why let needs the wrapper function, but i'm not convinced about all blocks |
| 14:51 | technomancy | cool |
| 14:52 | bbloom | dnolen: basically, anywhere we *emit* a javascript *statement* we need the )() wrapper -- but you wind up with double wrappers in a bunch of cases |
| 14:52 | technomancy | has anyone gotten postgres hstore working with jdbc? |
| 14:53 | neotyk | dnolen: have I seen somewhere ticket for multiple browser repls running from one process? |
| 14:56 | neotyk | dnolen: can't find it in jira, though vaguely remember it been mentioned somewhere |
| 15:00 | dnolen | bbloom: blocks might appear in as test expressions |
| 15:01 | bbloom | dnolen: ok, well i'm going to experiment & i'll let you know if i think there are any gains to be had by thinking about this more deeply |
| 15:01 | Miko2 | Hmm-m. |
| 15:02 | Miko2 | I'm getting quite weird behaviour with transient maps. |
| 15:02 | Miko2 | https://gist.github.com/2870176 |
| 15:02 | dnolen | bbloom: it's not an avenue I think that is worth pursuing - GClosure gets rid of it. Best to look at what happens during :simple to see what I mean. |
| 15:03 | dnolen | neotyk: I recall a ticket - definitely a branch on GitHub |
| 15:03 | bbloom | dnolen: yeah, i don't expect to find any potential gains. i'm just trying to understand gclosure's behavior better |
| 15:04 | Miko2 | For the first 8 conj!'s it uses TransientArrayMap, but for the nineth it conjs to TransientHashMap |
| 15:04 | dnolen | bbloom: from what I can tell - unwrapping happens during :simple, inlining during :advanced |
| 15:05 | bbloom | dnolen: yeah, simple seems to do constant folding |
| 15:06 | bbloom | gclosure really is an impressive little optimizer :-P |
| 15:06 | gtrak | Miko2: if you look on the docs, you'll notice it says they're not supposed to be bashed in place |
| 15:06 | TimMc | Miko2: THat's expected. |
| 15:07 | rlb | neotyk: thanks, though that wouldn't explain the large difference between rand-int and .nextInt. With rand-int, clojure's about half the speed of the java program (which is better than all the competitors except c++). |
| 15:07 | dnolen | bbloom: it is, in the early days I didn't observe a big perf difference between :advanced and other optimization modes. Now that we produce much more optimal JS, the perf difference is quite large. |
| 15:07 | rlb | anyway, no big deal -- I was just toying with it. |
| 15:07 | Miko2 | TimMc: I didn't expect that... |
| 15:08 | gtrak | Miko2: "You must capture and use the return value in the next call. In this way, they support the same code structure as the functional persistent code they replace." |
| 15:09 | bbloom | dnolen: yeah, but when your metaprogramming happens on startup (instead of during compilation) like ruby or raw javascript does, the restrictions advanced mode places on you can be a drag |
| 15:09 | Miko2 | Oh, hmmm. Well, that explains it. |
| 15:09 | Miko2 | Thanks. Have to change the code. :) |
| 15:09 | gtrak | np |
| 15:10 | gfredericks | what's the goodest canvas lib for cljs? |
| 15:10 | dnolen | bbloom: not IME since the kind of metaprogramming you can do in CLJS is not even possible in those languages. |
| 15:12 | bbloom | dnolen: of course, cljs definitely is a better trade off. you generate more code, compress it better w/ gclosure. rather than spend an extra 2 ms stitching strings, you spend an extra 1.5ms downloading bits. im just saying that if you're writing wrap js, like this large coffeescript project i have, then i opted to ignore advanced compilation mode b/c i wanted to be able to compose method names from strings and look them up at runtime |
| 15:12 | bbloom | wrap -> raw |
| 15:13 | PhearTheCeal | All right, I'm downloading Clojure. I've never used a functional language before. And advice for me right now? |
| 15:13 | gfredericks | don't download clojure, download lein |
| 15:13 | PhearTheCeal | too late |
| 15:13 | bbloom | better yet, don't download anything at all and try your hand at http://www.4clojure.com/ |
| 15:13 | gfredericks | abort abort |
| 15:14 | PhearTheCeal | bbloom: Awesome! Thanks |
| 15:14 | Raynes | You really do need to download lein though. |
| 15:14 | gfredericks | no way. 4clojure.com is turing complete. |
| 15:15 | Raynes | Downloading Clojure is trying to drive an engine. |
| 15:15 | cgag | yeah definitely get lein |
| 15:16 | cgag | i remember trying to get into clojure before lein was out, or atleast before i knew about it and it was hugely discouraging |
| 15:16 | dnolen | bbloom: JS devs do that because that's all the JS metaprogramming you can do & GClosure doesn't make JS programming any nicer - in fact it just makes it more tedious. |
| 15:19 | bbloom | dnolen: that's exactly what i'm saying :-) |
| 15:20 | bbloom | PhearTheCeal: make sure you follow a couple of the top ranked people, so you can learn alternative approaches! |
| 15:20 | PhearTheCeal | bbloom: Like who? |
| 15:21 | philr | /quit |
| 15:21 | dnolen | bbloom: heh, yes |
| 15:22 | PhearTheCeal | bbloom: How do I see their solutions? |
| 15:22 | bbloom | PhearTheCeal: you need to solve the puzzles on your own first ;-) |
| 15:23 | PhearTheCeal | bbloom: makes sense :P |
| 15:28 | PhearTheCeal | i'm on problem 4 |
| 15:28 | PhearTheCeal | (= (list __) '(:a :b :c)) |
| 15:28 | PhearTheCeal | I think I need to learn about syntax |
| 15:30 | AimHere | PhearTheCeal, syntax is mostly straightforward; all your code is in the form of a list (command arg1 arg2 arg3 ...). Most things that don't look like that are just syntactic sugar for something that does |
| 15:30 | AimHere | '(1 2 3) is sugar for (quote 1 2 3) for instance |
| 15:30 | gfredericks | mmm, tasty |
| 15:30 | AimHere | ,(doc list) |
| 15:31 | AimHere | &(doc list) |
| 15:31 | lazybot | ⇒ "([& items]); Creates a new list containing the items." |
| 15:31 | gfredericks | the & there means that list takes multiple arguments |
| 15:32 | AimHere | &(list 'item1 'item2 'item3 'item4) |
| 15:32 | lazybot | ⇒ (item1 item2 item3 item4) |
| 15:33 | gfredericks | &(list :item1 "item2" #{'item3} [::item4]) |
| 15:33 | lazybot | ⇒ (:item1 "item2" #{item3} [:clojure.core/item4]) |
| 15:33 | PhearTheCeal | I can't put an & |
| 15:33 | gfredericks | you don't need to |
| 15:33 | PhearTheCeal | so the answer is ":a" ":b" ":c" ? |
| 15:34 | gfredericks | shouldn't be |
| 15:34 | groovemonkey | almost. They're not strings. |
| 15:34 | AimHere | &(list ":a" ":b" ":c") |
| 15:34 | lazybot | ⇒ (":a" ":b" ":c") |
| 15:34 | gfredericks | :foo is a keyword |
| 15:34 | AimHere | keywords are a special kind of data in clojure |
| 15:35 | PhearTheCeal | oh... |
| 15:35 | PhearTheCeal | Like variables? |
| 15:35 | gfredericks | closer to strings and numbers |
| 15:35 | AimHere | Sortof, but they don't hold a value except themselves |
| 15:35 | bbloom | strings, symbols, and keywords are different things! :-) |
| 15:36 | bbloom | var s = "foo"; // var-keyword s-symbol =-keyword foo-string |
| 15:36 | amalloy | keywords are a lot like "strings that are only ever used to compare for equality" |
| 15:36 | gfredericks | that's the only thing I've ever said that amalloy liked |
| 15:36 | amalloy | gfredericks: and there's a second one! |
| 15:37 | gfredericks | killing it! |
| 15:37 | AimHere | I thought of keywords more as those symbols in SICP that you didn't need to bind any actual value to |
| 15:38 | gfredericks | keywords are like enums except you don't get typo-checking |
| 15:39 | gfredericks | that's why _real_ clojure programmers always (def foo :foo) at the top of their namespaces |
| 15:39 | PhearTheCeal | so var and = are keywords, but you can define your own keywords. And keywords are like strings? |
| 15:39 | gfredericks | oh man |
| 15:39 | AimHere | No, var and = are symbols |
| 15:40 | AimHere | They are bound to values, which in those cases are functions |
| 15:40 | AimHere | Unless var is a macro that is |
| 15:40 | gfredericks | I think he read too much into bloom's last statement |
| 15:40 | AimHere | keywords are always denoted by the preceding colon |
| 15:42 | amalloy | i think bbloom's last statement was either confusing or nonsense, so that's fair, gfredericks |
| 15:42 | bbloom | gfredericks: sorry, was just trying to point out that strings, symbols, and keywords are different things. |
| 15:42 | gfredericks | invoking javascript was an odd way to do that |
| 15:44 | gfredericks | actually it might make sense using the javascript definition of "keyword" which would be quite misleading |
| 15:44 | bbloom | gfredericks: *shrug* makes sense in my head :-P |
| 15:45 | nDuff | I'm trying to AOT-compile some code which involves setting namespace-global variables with the results of calls which will be available at runtime but don't work yet at compile-time. What's the appropriate way to make this work? |
| 15:45 | amalloy | hide everything behind delays |
| 15:46 | borkdude | what is the easiest way to open a file from the console in osx in Emacs.app (when it is not running it should start automaticall and it should work with files that don't exist, i.e. "emacs foo.txt" ) |
| 15:46 | gfredericks | you can hide the delay behind a function so that you only have to remember it's a delay in one place |
| 15:46 | borkdude | I've tried several approaches now but they all suck |
| 15:46 | gfredericks | (let [foo' (delay (make-foo))] (defn foo [] @foo')) |
| 15:46 | nDuff | The thing that bugs me here is that some of these are called somewhat frequently, so adding extra indirection is kinda' unfortunate. |
| 15:47 | gfredericks | nDuff: you could make a macro that does that ^ if you hate doing that |
| 15:47 | gfredericks | unless you just hate calling it as a function |
| 15:48 | gfredericks | in which case there's no way around it unless you want to utilize pure evil |
| 15:48 | nDuff | Hmm. At least the functions could definitely be type-hinted; I'm not sure if the delays can... |
| 15:48 | PhearTheCeal | dang it.... 4clojure is down |
| 15:48 | S11001001 | nDuff: switch on whatever the compiling var is in clojure.core |
| 15:49 | amalloy | whoever runs that website needs to get their shit together |
| 15:49 | amalloy | (it's back up now, PhearTheCeal) |
| 15:49 | S11001001 | nDuff: runtime code will thus be compiled but not run at compile time. Compile-only path will be compiled in, but that's no big deal. Then just make sure you don't use compilation to load when runtime is real |
| 15:52 | S11001001 | if you compile and what you're compiling recursively loads something that's already compiled, the compiling flag is still on |
| 15:52 | S11001001 | so you're overloading the compiling flag to mean "I don't have that function I need" |
| 15:53 | S11001001 | accordingly, if you use compile to load when you do have that function, then you're telling your code that you don't have the function |
| 15:54 | S11001001 | this is the sort of thing you have to keep in mind if you want to make module loading do different things depending on environment |
| 16:00 | amalloy | nDuff: you're causing yourself a lot of pain to avoid just using some delays. the cost of deref'ing a delay is extremely small; it's smaller than the cost of calling a function |
| 16:01 | amalloy | or i guess it's exactly the cost of calling a function, since you have to call clojure.core/deref |
| 16:01 | Sgeo | Besides no tail-call optimization, what concessions to the JVM and to interoperability with Java code does Clojure make? |
| 16:01 | S11001001 | amalloy: where does the lock go? |
| 16:03 | amalloy | S11001001: the synchronize block doesn't go away. but again that's small |
| 16:03 | ivan | Sgeo: perhaps having both nil and false |
| 16:03 | S11001001 | Sgeo: + is weird |
| 16:04 | Sgeo | S11001001, how so? |
| 16:04 | dnolen | Sgeo: JVM error handling, no continuations |
| 16:06 | S11001001 | Sgeo: won't autopromote bytes, shorts, ints, longs beyond long; same for - and *, and I think inc/dec |
| 16:06 | borkdude | is defstruct a bit deprecated or how was it again? |
| 16:06 | dnolen | borkdude: deprecated |
| 16:09 | nDuff | (def ^some.java.Class (delay ...)) doesn't seem to result in the compiler being aware of what type dereffing the delay will result in. |
| 16:09 | technomancy | Sgeo: the biggest concession to the JVM I've noticed beyond needing primitives for speed is probably the inability to treat arbitrary data types as functions |
| 16:10 | llasram | And you could hide *that* behind a symbol-macrolet! |
| 16:11 | mebaran151 | is there any easy stdlib function to convert a Java XML Element to a nice clojure hashmap? |
| 16:13 | dnolen | mebaran151: https://github.com/clojure/data.xml ? |
| 16:14 | borkdude | dnolen is this the same as clojure.xml? |
| 16:14 | borkdude | dnolen or an addition |
| 16:15 | mebaran151 | dnolen: I already have some nodes returned unfortunately (would be great if I had a handle on a string representation) |
| 16:15 | dnolen | borkdude: I don't know what clojure.xml is |
| 16:16 | borkdude | dnolen https://github.com/clojure/clojure/blob/master/src/clj/clojure/xml.clj |
| 16:16 | mebaran151 | borkdude: I think clojure.data.xml is the new clojure.contrib.lazy-xml |
| 16:16 | amalloy | dnolen: the awful/broken namespace in clojure |
| 16:16 | borkdude | it's obviously not the same then |
| 16:16 | mebaran151 | like wise if anybody had a nice clean solution for getting all the tags by tag name in pure clojure, I'd also be much obliged (the internet is rife with examples that rely on clojure.contrib.zip-filter, which seems deprecated) |
| 16:16 | hiredman | clojurebot: ping |
| 16:16 | clojurebot | PONG! |
| 16:17 | hiredman | ,*clojure-version* |
| 16:17 | clojurebot | {:interim true, :major 1, :minor 4, :incremental 0, :qualifier "master"} |
| 16:18 | borkdude | mebaran151 I think the parse function in clojure.xml turns xml into nested maps |
| 16:18 | mebaran151 | it'll take an xml string, but the problem is I have a bunch of xml nodes; it looks like they wrapped that function into the parser unfortunately |
| 16:19 | solussd | anyone here using the swiss-arows library? just discovered that the diamond one doesn't work inside a macro (backtick) |
| 16:23 | raek | what testing frameworks should one checkout these days? (I'm currently using Midje.) |
| 16:23 | mebaran151 | borkdude: I probably could get clojure.walk to do what I want actually |
| 16:24 | mebaran151 | though I don't know if there's a way to teach it how to make a node a seq |
| 16:25 | borkdude | hmm, there was a discussion here on group-by being sucky if you want to select things based on a function |
| 16:25 | borkdude | there's an example of what was suggested in that discussion on page 119 of Clojurebook |
| 16:25 | borkdude | :) |
| 16:25 | borkdude | (called reduce-by) |
| 16:29 | mprokos | I am trying to get a repl to run in my lein project. I just run clojure.main/repl from (defn -main) but it just prints a prompt and returns. |
| 16:29 | mprokos | I am using clojure 1.4.0 |
| 16:30 | technomancy | you need trampoline for that |
| 16:31 | technomancy | lein trampoline run -m clojure.main/repl |
| 16:35 | mprokos | Awww.. I see |
| 16:37 | bbloom | dnolen: weird. can't use extend-type on RegExp because gclosure will disable optimizations on regexs related to functions with known side effects |
| 16:38 | dnolen | bbloom: yep, that's mentioned on the RegExp tickets |
| 16:39 | bbloom | dnolen: i implemented IPrintable, but it doesnt work in advanced mode :-( |
| 16:39 | bbloom | ah, i see your note in CLJS-68 |
| 16:40 | dnolen | bbloom: if RegExp extension is going to work it must be done by adding a new native type to the list - those types which should never be modified directly via prototype |
| 16:41 | bbloom | dnolen: i assume that list exists now, for objects? where is that list? |
| 16:41 | dnolen | bbloom: in the macros file |
| 16:42 | bbloom | dnolen: ah, core/base-type, got it |
| 16:43 | dnolen | bbloom: and actually it won't work typeof on RegExp -> 'object' |
| 16:43 | bbloom | dnolen: ok, i haven't ventured into this code path yet. give me a moment to make sense of it |
| 16:43 | bbloom | dnolen: heh, oh, actually i did. sorta, for the apply/call/arity stuff -- amazing how much logic is packed into so little space! |
| 16:51 | borkdude | has anyone got a good example of how to use reduce-kv? I want to see if it is similar to what we were looking for in the discussion (and reduce-by in clojurebook), so that it in fact already existed all along |
| 17:00 | gfredericks | &(doc reduce-kv) |
| 17:00 | lazybot | ⇒ "([f init coll]); Reduces an associative collection. f should be a function of 3 arguments. Returns the result of applying f to init, the first key and the first value in coll, then applying f to that result and the 2nd key and value, etc. If coll contains no en... https://www.refheap.com/paste/2994 |
| 17:02 | gfredericks | ,(reduce + 0 [1 2 3 4]) |
| 17:02 | clojurebot | 10 |
| 17:02 | muhoo | having brain-lock. what's the mor idiomatic version of this: ##(reduce #(or %1 %2) [true false false]) |
| 17:02 | lazybot | ⇒ true |
| 17:02 | gfredericks | there couldn't be a more natural example than that |
| 17:02 | technomancy | muhoo: some identity maybe? |
| 17:02 | gfredericks | technomancy: you can pass a monad to some? |
| 17:03 | technomancy | you can pass anything to some! |
| 17:03 | cmajor7 | most of my experience is backend (some years). now I'd really like to get into front end (noir + clojurescript + twitter bootstrap). what is the best/fastest way to start? screencasts/posts/docs, etc.. looking at noir docs, twitter bootstrap examples, some hiccup docs. looked at some posts (e.g. http://bit.ly/Lt0Qwt ), played with noir-bootstrap (http://bit.ly/Lt10Uo), but still feel a little detached from where to start |
| 17:03 | muhoo | that'll work |
| 17:10 | borkdude | I guess I can't use reduce-kv for this |
| 17:11 | borkdude | (for the same group-by using a fn selector) |
| 17:11 | borkdude | same as |
| 17:31 | TimMc | muhoo: Is the input collection the result of mapping a predicate over something? |
| 17:33 | technomancy | seancorf`: do you know what it would take to add hstore support to c.j.jdbc? |
| 17:33 | technomancy | or even if that's the appropriate place for it? |
| 17:37 | bbloom | dnolen: ok, i think i have a good understanding of protocols in cljs now |
| 17:37 | dnolen | bbloom: cool |
| 17:38 | bbloom | dnolen: looking at the implementation of goog/typeOf -- that's a bit of a scary function to call on every protocol dispatch against native objects |
| 17:38 | dnolen | bbloom: I don't see how we could do any better. |
| 17:39 | bbloom | dnolen: yeah, i'm not sure yet… still thinking about it. |
| 17:40 | dnolen | bbloom: and even that is faster than messing w/ native prototypes. V8 punishes you for adding things to String & Array etc. directly. |
| 17:40 | dnolen | bbloom: and my impression so far has been so does SM & JSC |
| 17:40 | bbloom | dnolen: makes sense to me -- don't want to change those thing's prototypes anyway, especially not object, so you at least need a solution for that, might as well reuse it for strings, arrays, etc |
| 17:40 | dnolen | bbloom: so what we have performs well enough for native types. |
| 17:44 | bbloom | dnolen: what about obj.constructor.toString() as keys instead of goog.typeOf ? |
| 17:45 | dnolen | bbloom: you'd have 1) show that it's not slower, 2) that it works in every JS environment we care about going back to IE6 |
| 17:46 | bbloom | dnolen: IE6? seriously? *cry* ok |
| 17:49 | dnolen | bbloom: it's not worth spending too much time on IMO. what we have works - the current perf is acceptable for natives - and the Clojure stuff just zips along. |
| 17:51 | dnolen | bbloom: the one place where we do directly extend a native - String - is a large source of pain. |
| 17:51 | bbloom | dnolen: i'm exploring this so i can better understand why things are done the way they are done -- that's just how i work |
| 17:51 | bbloom | dnolen: pain == slowness? |
| 17:52 | dnolen | bbloom: yes |
| 17:52 | bbloom | dnolen: via protocols or something else? |
| 17:52 | dnolen | bbloom: no directly, we extend String so that keywords can be functions. it's very slow. I put in a temporary fix for the most common case (:foo x) |
| 17:53 | bbloom | dnolen: slow b/c it disables some of V8's optimizations? |
| 17:53 | dnolen | bbloom: disables everyones optimizations |
| 17:53 | bbloom | dnolen: heh, fair enough -- dopey javascript. such an almost beautiful little language :-) |
| 17:53 | quizme | is it possible to compile clojurescript in clojurescript ? |
| 17:54 | bbloom | quizme: no, the clojurescript compiler is written in clojure |
| 17:54 | dnolen | bbloom: it's actually faster to to construct an object (Keyword) at the call site and invoke on that. |
| 17:55 | bbloom | dnolen: hm, interesting. and if you replace all keywords with objects with a single string field, what gets slow then? |
| 17:56 | bbloom | dnolen: alternatively, every new keyword could effectively be a "leak" like symbols in matz's ruby. basically store keywords as integers instead of strings and have an array of every keyword ever seen |
| 18:01 | dnolen | bbloom: allocations are not free, I assume strings are optimized by JS engines. |
| 18:02 | dnolen | bbloom: CLJS programs have the advantage of being whole program optimized. We could preallocate all the known keywords ahead of time under advanced compilation. And fall back on the slower thing for unknown runtime keywords (from read-string say). |
| 18:03 | bbloom | dnolen: that's an interesting idea! because known keywords can be compiled to an integer lookup, which is mega fast -- no allocations |
| 18:04 | dnolen | bbloom: integer lookup could be interesting, but we already have string hash code caching in master. |
| 18:05 | dnolen | bbloom: pre allocated keywords could be constructed with their hash codes, at that point I don't think property access will be the bottle neck in any program. |
| 18:06 | bbloom | dnolen: why bother with hash codes? why not just compile :foo to known_keywords["?:foo"] where ? is that special char |
| 18:07 | dnolen | bbloom: because we have complex keys. |
| 18:07 | bbloom | dnolen: not in the set of known keywords! |
| 18:07 | dnolen | bbloom: we still need to call hash on the number. how do we know that's not a real number? |
| 18:08 | dnolen | we can't extend prototype on Number meaning -hash on Number will be slow, etc. |
| 18:08 | bbloom | dnolen: what number? let's back up a step |
| 18:09 | dnolen | bbloom: when looking up a key we generally always call hash on the key, since the key can be anything. |
| 18:11 | bbloom | dnolen: i think this all makes sense in my head, let me write up some notes & link you too them. irc is hard to get a broader idea across |
| 18:13 | dnolen | bbloom: sure though, I also suggest you take a closer look at how maps and hash currently work. |
| 18:14 | bbloom | dnolen: yeah, i looked at those a bunch. the thing is i don't think it matters. we don't need to use a map or hash, just a js object or array. |
| 18:15 | dnolen | bbloom: js object or array for what? |
| 18:15 | bbloom | dnolen: typing up notes :-) |
| 18:22 | gtrak | is instance? on a record supposed to work? |
| 18:22 | gtrak | doesn't seem to work |
| 18:27 | gtrak | might be freaking AOT |
| 18:27 | rcg | https://github.com/ruedigergad/cljNetPcap |
| 18:27 | rcg | https://github.com/ruedigergad/cljAcmeUtils |
| 18:28 | rcg | just in case you are interested or this is helpful |
| 18:28 | rcg | the first one is a wrapper/facade/whatever for using jNetPcap from withing clojure |
| 18:29 | rcg | the latter is just a bunch of helper functions i came up with over time and found quite useful |
| 18:29 | rcg | i just finally had the time and took the chance to put this stuff online |
| 18:29 | dnolen | gtrak: it does, but you have to watch out for redefinition |
| 18:30 | gtrak | dnolen: I think that's the culprit |
| 18:35 | bbloom | dnolen: sorry, got distracted by phone call. look at: https://www.refheap.com/paste/2995 |
| 19:08 | gtrak | hmm, i would have expected slurp and spit to use IOFactory on the arg, seems strange |
| 19:09 | gtrak | they kinda do the same thing |
| 19:09 | clojurebot | two things are more than one thing |
| 19:11 | gtrak | I suppose IOFactory came after protocols, after slurp/spit |
| 19:12 | TimMc | slurp and spit are *ancient* |
| 19:14 | gtrak | wait, they call jio/reader |
| 19:14 | gtrak | so I can use make-reader |
| 19:14 | gtrak | duh |
| 19:19 | gtrak | freaking great :D |
| 19:24 | srid | how do i access `this` in clojurescript from event handlers? this question was asked before <http://osdir.com/ml/clojure/2011-08/msg00558.html> but without a response |
| 19:25 | pbostrom | srid: (this-as x (do-something-with x)) |
| 19:26 | velvia | Hi guys, newbie here, really like Clojure so far |
| 19:27 | velvia | Leiningen question. If I have a project with mixed Java and Clj, and I do "lein compile" will it detect and recompile changed Java files too? |
| 19:27 | velvia | I'm using 1.6.x and it doesn't seem to detect Java file changes |
| 19:27 | srid | pbostrom: thanks! looks like cljs is not fully documented yet. |
| 19:28 | technomancy | velvia: you have to tell it where the java files are kept; set :java-source-path in project.clj IIRC |
| 19:28 | pbostrom | srid: it is also a difficult question to Google |
| 19:28 | pbostrom | 'this' returns too many irrelevant links |
| 19:28 | velvia | @technomancy: thx |
| 19:29 | srid | after your response, i googled for {{clojurescript "this-as"}} just to see if there are any docs talking about this-as. |
| 19:30 | clizzin | hey, i'm compiling a file in my test/ directory, and getting the following error: Unknown location: |
| 19:30 | clizzin | error: java.io.FileNotFoundException: Could not locate clojure/contrib/ns_utils__init.class or clojure/contrib/ns_utils.clj on classpath: |
| 19:32 | clizzin | does anyone have any ideas? i'm using clojure 1.3.0 in my project.clj, and the only additional dependency that this test file's namespace has is midje, which i'm pretty sure works with 1.3 |
| 19:33 | hiredman | clizzin: have you run 'lein deps' since adding midje? |
| 19:33 | clizzin | hiredman: yes |
| 19:33 | hiredman | is midje and ns-utils in lib or lib/dev ? |
| 19:33 | clizzin | i just removed the require and use statements, so that all the file contains is an (ns … ) declaration at the top. i think the problem is that ns-utils is somehow not on the load path. |
| 19:34 | clizzin | hiredman: ns-utils is not explicitly in lib or lib/dev. i thought ns came with clojure. is that not the case? |
| 19:34 | hiredman | do you have the full stack trace? |
| 19:34 | hiredman | ns comes with clojure |
| 19:34 | hiredman | ns-utils is some library |
| 19:34 | velvia | @technomancy: how can I tell if its actually compiling the Java files? |
| 19:34 | technomancy | I'm not sure; I've never used javac |
| 19:34 | technomancy | you can always check the classes dir if there's nothing else showing |
| 19:35 | clizzin | hiredman: never mind, the problem is precisely with using midje.sweet. here is the stack trace: https://gist.github.com/2871479 |
| 19:36 | clizzin | hiredman: midge is in dev-dependencies but not in dependencies |
| 19:36 | clizzin | midje* |
| 19:46 | dnolen | bbloom: ping |
| 19:46 | bbloom | dnolen: here |
| 19:47 | dnolen | bbloom: we should try to do something simpler for the RegExp patch, currently doing too much. |
| 19:47 | bbloom | dnolen: or two much for 1 patch? or period? |
| 19:48 | bbloom | blah fucked that sentence up. you get the idea |
| 19:48 | dnolen | bbloom: period. so GClosure complains if you refer to RegExp period right? |
| 19:48 | bbloom | except in a few key cases |
| 19:48 | bbloom | new, obviously |
| 19:49 | bbloom | and instanceof |
| 19:49 | bbloom | and maybe one or two more |
| 19:49 | dnolen | bbloom: k this is the only thing we should do then ... add a new predicate called regexp? |
| 19:50 | dnolen | (defn regexp? [x] (js* "~{o} instanceof RegExp")) |
| 19:50 | dnolen | and use that in pr-seq |
| 19:50 | dnolen | sans my typos |
| 19:50 | clizzin | hiredman: fwiw i just "solved" the problem by including [lein-midje "1.0.10"] in dev-dependencies instead of [midje "0.4.0"] |
| 19:51 | bbloom | dnolen: i'm ok with adding that predicate either way, but why not support inlining instanceof checks? we already support inlining of identical, satisfies, etc |
| 19:54 | dnolen | bbloom: we can though I tried that and benchmarked, make no difference in hot code. |
| 19:54 | dnolen | makes |
| 19:56 | dnolen | bbloom: though it would be nice to lose another js* usage from core.cljs |
| 19:56 | bbloom | dnolen: it'd be trading a js* for a js/ |
| 19:57 | bbloom | dnolen: besides, regexp are going to be special on every new backend |
| 19:58 | dnolen | bbloom: in anycase inlining instance? is a seperate thing :) regexp? for that ticket preferred. |
| 19:59 | bbloom | dnolen: agreed. i'll take a look at it a bit later. experimenting with the optimizations on keywords now. i think this approach i linked you to might work |
| 19:59 | dnolen | bbloom: link? |
| 19:59 | dnolen | bbloom: might have signed off before I saw it earlier. |
| 19:59 | bbloom | https://www.refheap.com/paste/2995 |
| 20:00 | bbloom | dnolen: also worth noting (i'm sure you know this already) symbols & keywords in cljs have extra powers that they don't in clj because they are technically strings :-P |
| 20:00 | dnolen | bbloom: yep |
| 20:00 | aperiodic | }/go dor |
| 20:00 | dnolen | bbloom: though anyone relying on that behavior is asking for it. |
| 20:00 | aperiodic | oops |
| 20:00 | dnolen | bbloom: we may change reps at anytime. |
| 20:01 | dnolen | bbloom: ok, you did hear what I said about the fact that CLJS has string hash code caching right? |
| 20:01 | bbloom | dnolen: yeah, but i wasn't sure how that was relevant |
| 20:02 | bbloom | i figured that Keyword's IHash would just call hash on k |
| 20:02 | dnolen | bbloom: when we see a symbol, string, keyword we hash it (producing an integer) and put into cache. the next time the keyword is used, it's likely to be in the cache, so we don't rehash and use that integer value for lookup. |
| 20:03 | dnolen | bbloom: so it does at runtime what you're doing at compile time. |
| 20:04 | bbloom | dnolen: hold on, let me go look at that |
| 20:08 | bbloom | dnolen: ok…. so it's storing a mutable hash value on each data structure for faster successive hashing. that makes perfect sense, but i'm still not sure what that has to do with keywords :-P keywords (as strings) just call goog.string/hashCode each time |
| 20:08 | dnolen | bbloom: where? |
| 20:09 | bbloom | dnolen: grep "extend-type string" |
| 20:09 | dnolen | bbloom: it'll never get there, we never call -hash directly |
| 20:10 | bbloom | dnolen: heh. |
| 20:10 | bbloom | dnolen: might want to delete that than :-) |
| 20:10 | bbloom | then* |
| 20:10 | dnolen | bbloom: no because someone might, it's there for completeness. |
| 20:10 | bbloom | k |
| 20:11 | dnolen | bbloom: one general CLJS strategy - all types implement the right behavior at the protocol level - -first, -hash etc. |
| 20:11 | bbloom | dnolen: makes sense |
| 20:12 | dnolen | bbloom: (CLJS perf strategy I mean). then fns like first, hash might make perf decisions. |
| 20:12 | dnolen | bbloom: because dispatching to protocol impls of native types is slow. |
| 20:12 | dnolen | we don't want to pay to dispatch to nil or string. |
| 20:13 | dnolen | etc |
| 20:28 | bbloom | dnolen: did you look at http://dev.clojure.org/jira/browse/CLJS-297 ? I ask because keyword optimization interacts with objmap construction, which i change in that patch :-P |
| 20:30 | dnolen | bbloom: yes, refactoring patches like that are low priority. |
| 20:31 | bbloom | dnolen: the goal of that patch is to reduce work that the emit phase needs to do. the macros are much more reusable across backends |
| 20:32 | dnolen | bbloom: core.clj is filled with non-portable JS specific things. |
| 20:32 | bbloom | dnolen: basically, anywhere you can replace string concatenation with working with clojure forms seems like a win for modularity |
| 20:32 | dnolen | bbloom: you're just moving non-portable stuff into a file w/ non-portable stuff. |
| 20:33 | groovemonkey | Pardon, I have a list containing an integer, and I'm trying to return a keyword from it. ex: (keyword (0)) -- that returns a "long cannot be cast to IFn" error. what simple solution am I missing? |
| 20:34 | seancorf` | ,(0) ;; that's a function invocation of zero |
| 20:34 | clojurebot | #<ClassCastException java.lang.ClassCastException: java.lang.Long cannot be cast to clojure.lang.IFn> |
| 20:34 | seancorf` | a list containing zero is '(0) |
| 20:34 | seancorf` | or use a vector [0] |
| 20:34 | seancorf` | but you can't make a keyword out of a list... |
| 20:34 | bbloom | dnolen: sure, it's surrounded by non-portable stuff, but the forms are portable once the non-portable stuff gets segregated. if we're trying to push towards pluggable backends, how can i get there without moving stuff around in "refactoring" patches? |
| 20:35 | nDuff | ...or out of a long, without making it a string first, either... |
| 20:35 | groovemonkey | seancorfield: sorry, I just tested the wrong thing. It's actually this: '(0) and trying to get a keyword from it returns nil |
| 20:35 | seancorfield | what do you mean "trying to get a keyword from it"? |
| 20:36 | nDuff | groovemonkey: maybe you want (keyword (pr-str '(0)))? |
| 20:36 | groovemonkey | nDuff: let me try... |
| 20:36 | nDuff | groovemonkey: ...but it's very unclear what your _real_ goal is for this |
| 20:36 | nDuff | groovemonkey: ...why are you actually trying to make a keyword from a list containing a long? |
| 20:37 | dnolen | bbloom: well actually, one big advantage as macros is we get Clojure :line info. |
| 20:37 | dnolen | bbloom: and hopefully later column info |
| 20:38 | groovemonkey | nDuff: I'm working with something returned from (rest) |
| 20:38 | groovemonkey | and I want to use that returned value to look up something in a map (as a part of (get-in [:x :y :z]) |
| 20:39 | nDuff | ...is there a good reason that map isn't just using plain strings as keys, then? |
| 20:39 | amalloy | don't use keywords *or* strings. just use the list itself as a map key |
| 20:39 | bbloom | dnolen: right. like i said, it's probably a maintenance and extensibility win anytime we can avoid stitching together js strings |
| 20:40 | groovemonkey | nDuff: Nope! Thank you for unbrainwashing me. |
| 20:40 | groovemonkey | amalloy: cool, I didn't think of that either. Thank you both! |
| 20:40 | bbloom | dnolen: in this case, the lua backend won't need to treat vector, set, meta, or hash-map specially. they can be treated as simple invocations |
| 20:41 | bbloom | dnolen: which, if applicable to the backend, can be optimized with macros |
| 20:42 | dnolen | bbloom: no effect on compile time? |
| 20:43 | bbloom | dnolen: benchmark you mean? i didn't run it. |
| 20:47 | bbloom | dnolen: script/benchmark for runtime perf, where is compile time perf measured? |
| 20:47 | dnolen | bbloom: I generally just do, "time ./script/test" |
| 20:48 | bbloom | my machine probably isn't the most reliable benchmark target…. lots of stuff running & it's a puney macbook air w/ the older cpu... |
| 20:48 | bbloom | but running it anyway |
| 20:52 | dnolen | bbloom: I'm not seeing any real diff far as compile time. I'll ponder the patch some more. |
| 20:53 | bbloom | dnolen: i ran it twice on master and twice with that patch. bounced back and forth around a minute plus or minus 5 seconds |
| 20:54 | bbloom | dnolen: ok, ponder the whole scope of similar refactorings. i cansee a whole bunch of opportunities to beef up code reuse between backends, while minimizing code per emit backend |
| 20:55 | dnolen | bbloom: I'm all for refactoring for backends. But the macros file is something that will need to be different for different backends. |
| 20:55 | bbloom | dnolen: also, this particular change probably has an opportunity to speed up compilation quite a bit if we move a bit of this logic into the reader. this way the reader would never need to produce a map or vector or whatever |
| 20:56 | dnolen | bbloom: the reader is the Clojure JVM reader. |
| 20:56 | bbloom | dnolen: i realize that, but it doesn't have to stay that way :-) |
| 20:57 | dnolen | bbloom: true, but expect patches to CLJ to move much slower than CLJS :) |
| 20:57 | bbloom | dnolen: right now when you read {:x 1} you get a map, but if you instead got (list 'hash-map :x 1) then you could cut out a whole bunch of logic from analyze phase |
| 20:58 | bbloom | dnolen: but beyond that, you wouldn't need to build the whole red black tree, etc. would probably speed up compilation quite a bit |
| 20:58 | hiredman | I imagine most read maps end up being array maps |
| 20:58 | bbloom | dnolen: actually, would be (list 'hash-map (keyword "x") 1) |
| 20:58 | bbloom | er 'keyword |
| 20:58 | bbloom | you get the idea |
| 20:59 | bbloom | or 'core/keyword or whatever |
| 20:59 | hiredman | bbloom: how do macros work with that? |
| 20:59 | dOxxx | Hi... I'm having some trouble with flatten and pmap... |
| 20:59 | johnmnmn | so weird... lein uberjar is not returning errors, but still no jars show up in my project dir |
| 21:00 | dOxxx | when I do something like: (flatten (pmap my-func some-list)) it seems to undo the parallelization of pmap and process it sequentially |
| 21:00 | bbloom | hiredman: i'm talking about deferring construction of the map during read |
| 21:00 | johnmnmn | "Compilation succeeded." |
| 21:01 | hiredman | bbloom: but the output of the reader is what is passed to macros |
| 21:03 | bbloom | hiredman: call eval on it? :-) |
| 21:03 | aperiodic | johnmnmn: which version of lein? |
| 21:03 | dnolen | bbloom: heh, yeah that's not going to go anywhere. |
| 21:04 | bbloom | dnolen: call very-simplistic-eval-which-only-understands-basic-calls-to-native-data-structures on it? :-) |
| 21:05 | johnmnmn | aperiodic: lein1 |
| 21:06 | aperiodic | johnmnmn: hmm, that is odd. you didn't set :jar-name or :uberjar-name in your project.clj, did you? |
| 21:06 | johnmnmn | aperiodic: I did not |
| 21:08 | aperiodic | johnmnmn: is lein jar showing the same symptoms? |
| 21:08 | johnmnmn | aperiodic: https://www.refheap.com/paste/2998 |
| 21:09 | johnmnmn | yes, it is. |
| 21:09 | aperiodic | oh, well, the uberjar is failing because you have snapshot dependencies in a release version |
| 21:10 | dOxxx | any insight on why flatten would negate the parallel-ness of pmap? |
| 21:11 | johnmnmn | dOxxx: I don't think it would |
| 21:12 | dOxxx | johnmnmn: it is. :P |
| 21:12 | johnmnmn | I've recently used reducers -- r/flatten (r/map -- to good effect |
| 21:13 | dOxxx | that's in the latest clojure alpha, right? how unstable is the alpha in general? |
| 21:14 | dOxxx | btw, this the code in question: https://www.refheap.com/paste/2999 |
| 21:15 | dOxxx | that code for a particular dataset runs in about 16 seconds |
| 21:15 | dOxxx | if I use this instead: https://www.refheap.com/paste/3000 then it runs in about 5 seconds |
| 21:16 | dOxxx | if I take a thread dump while the first version is running, all the calculation is being done in one thread and there's no sign of pmap in the stacktrace |
| 21:16 | dOxxx | wherease the second version shows multiple threads doing the calculation and another thread with pmap in the stacktrace waiting on the other threads to finish |
| 21:17 | johnmnmn | dOxxx: how heavy-weight is parse-line? |
| 21:17 | dOxxx | it's a few regex and constructing some records |
| 21:17 | johnmnmn | so it kinda depends on how heavy weight the payload is. |
| 21:18 | dOxxx | it's fairly lightweight as I understand it which is why I wanted to try batching it up to reduce context switching overheqad |
| 21:19 | dOxxx | up to 4 regex and one record |
| 21:19 | dOxxx | regex are operating on a single line of text |
| 21:20 | johnmnmn | generally, I think pmap and friends do the appropriate level of chunking for you |
| 21:21 | amalloy | dOxxx: parse-batch takes zero time |
| 21:21 | dOxxx | oh? interesting... |
| 21:21 | amalloy | so parallelizing it is a waste of resources |
| 21:21 | amalloy | because it instantly returns a lazy sequence of lines |
| 21:21 | johnmnmn | not sure if it dynamically detects the weight of a given chunk, and expands as necessary, but it does chunk in the background |
| 21:22 | dOxxx | so (pmap parse-line lines) is just doing the right thing automatically? |
| 21:22 | dOxxx | cool |
| 21:23 | dOxxx | I also discovered that concat does not work well with large lists :P |
| 21:24 | dOxxx | I was trying to write a replacement for flatten as (reduce concat list-of-lists) but it died with a stack overflow error |
| 21:24 | mthvedt | reducing lazy sees can cause stack overflows |
| 21:25 | mthvedt | for reasons i don't remember the details |
| 21:25 | dOxxx | any workaround? |
| 21:25 | mthvedt | eager seqs |
| 21:25 | dOxxx | is that function? |
| 21:25 | mthvedt | maybe wrap a doall |
| 21:25 | dOxxx | ah ok |
| 21:26 | TimMc | clojurebot: prime overflow? |
| 21:26 | clojurebot | It's greek to me. |
| 21:27 | TimMc | clojurebot: sieve |
| 21:27 | clojurebot | see the genuine sieve of eratosthenes |
| 21:27 | TimMc | bah |
| 21:27 | amalloy | $google stackoverflow dbyrne sieve clojure |
| 21:27 | lazybot | [recursion - Recursive function causing a stack overflow - Stack ...] http://stackoverflow.com/questions/2946764/recursive-function-causing-a-stack-overflow |
| 21:34 | dOxxx | got the batching working by throwing in a bunch of doalls to de-lazy the various sequences and it doesn't perform any better than the non-batch form. |
| 21:34 | dOxxx | so yay pmap for being smart |
| 21:36 | dOxxx | thanks for the help :) |
| 21:42 | johnmnmn | just used lein2 and still no juju :/ |
| 21:43 | dOxxx | johnmnmn: post your project.clj? |
| 21:43 | dOxxx | who knows, maybe the newb will spot something :) |
| 21:47 | bbloom | dnolen: updated http://dev.clojure.org/jira/browse/CLJS-300 |
| 21:49 | johnmnmn | dOxxx: https://www.refheap.com/paste/3001 |
| 21:50 | johnmnmn | I didn't notice 'lein uberjar' always saying this before: Release versions may not depend upon snapshots. |
| 21:50 | dOxxx | johnmnmn: have you tried: :main "SMCQ.core" ? |
| 21:51 | dOxxx | johnmnmn: [21:04:45] <aperiodic> oh, well, the uberjar is failing because you have snapshot dependencies in a release version |
| 21:52 | johnmnmn | huh? |
| 21:52 | muhoo | ordinary jars (lein jar) seem to have no problem with snapshot dependencies. uberjars apparently do. |
| 21:54 | johnmnmn | hmm |
| 21:55 | johnmnmn | lein jar has been doing the same thing... trying again now |
| 21:55 | johnmnmn | yea, no bueno |
| 21:56 | johnmnmn | trying with a different clojure version |
| 21:57 | johnmnmn | deleting my .m2/repository too |
| 22:03 | muhoo | johnmnmn: if you're using lein2, try "lein2 deps :tree" |
| 22:04 | muhoo | see if there are any SNAPSHOTs in there. if so, whack them. |
| 22:04 | johnmnmn | mmm, k |
| 22:07 | johnmnmn | org.clojure:clojure:jar:1.5.0-alpha2 |
| 22:08 | johnmnmn | is this correct: [org.clojure/clojure "1.5.0-alpha2"] ? |
| 22:08 | johnmnmn | maven can't find it. |
| 22:08 | dOxxx | alpha1 is latest |
| 22:08 | johnmnmn | alpha2 is out there |
| 22:10 | johnmnmn | alpha.2? |
| 22:10 | dOxxx | I don't see alpha2 in the list of tags on github |
| 22:11 | johnmnmn | hmm |
| 22:11 | dOxxx | nor do I see it here: http://mvnrepository.com/artifact/org.clojure/clojure |
| 22:12 | johnmnmn | I guess your right |
| 22:12 | johnmnmn | wonder why I thought that |
| 22:12 | dOxxx | you're living in the future |
| 22:12 | muhoo | TimMc: yes, and i found a cleaner way to do it, (some #(boolean (re-find % uri)) [#"^/css" #"^/js" #"^/img"]) |
| 22:12 | johnmnmn | I blew my cover |
| 22:13 | dOxxx | are you going to have to kill me now? :( |
| 22:13 | johnmnmn | No, I just have to kill this identity now |
| 22:14 | dOxxx | style question: should I use (->SomeRecord stuff) or (SomeRecord. stuff) in code which is in the same namespace as the SomeRecord definition? |
| 22:16 | johnmnmn | getting a lot of this: [WARNING] Overriding profile: 'null' (source: pom) with new instance from source : pom |
| 22:16 | johnmnmn | and: [INFO] Unable to find resource 'org.clojure:clojure:pom:1.+' in repository cloja |
| 22:17 | johnmnmn | oh wait: Release versions may not depend upon snapshots. |
| 22:18 | johnmnmn | is that saying that it is interpreting my project as a "release version" because I don't have the word "SNAPSHOT" in it? |
| 22:20 | johnmnmn | omfg, that was it |
| 22:21 | dOxxx | johnmnmn: yay? :) |
| 22:21 | johnmnmn | yay is right |
| 22:21 | johnmnmn | I blame myself for not understanding the error language that was in front of me the whole time. |
| 22:25 | muhoo | that's kind of weird. i've build jars of release versions which depend on snapshots before. i don't get any complaints. |
| 22:25 | muhoo | maybe, again, uberjars are different. |
| 22:31 | johnmnmn | so have I |
| 22:35 | TimMc | muhoo: The call to boolean is unecessary. |
| 22:36 | muhoo | TimMc: i wondered about that. thanks. |
| 22:48 | johnmnmn | I love writing little clojure tools to get stuff done at work... Its just so easy to use the many libraries out there to get things done. |
| 22:50 | johnmnmn | people at work are always so surpised when I drop a 50 meg jar file on their desktop that can parse PDFs, excels, out put graphs, more excels, scrape intranet sites... all in a couple days. |
| 22:50 | johnmnmn | and their surprised |
| 22:53 | mheld | what do y'all use for routing? clout? moustache? |
| 22:54 | brehaut | you probably mean compojure rather than clout |
| 22:54 | brehaut | clout is used internally by compojure |
| 22:54 | mheld | oh, didn't know that |
| 22:55 | brehaut | and compojure is the most common choice these days, it has the largest community and is probably easiest to get going with |
| 22:56 | mheld | yeah, I'm getting into webdev with clojure and there doesn't seem to be central defacto guide anywhere |
| 22:56 | mheld | I mean, I'm piecing things together |
| 22:56 | brehaut | start here http://brehaut.net/blog/2011/ring_introduction |
| 22:58 | brehaut | mheld: it might be a bit lower level than you were expecting, but ring is the foundation of web stuff in clojure. understanding it will make everything that follows easier |
| 23:01 | mheld | brehaut: cool, thanks |
| 23:02 | mheld | brehaut: great article :-) |
| 23:02 | brehaut | your welcome |
| 23:07 | readme | mheld: =o |
| 23:08 | mheld | readme: god damn it, are you in *all* the rooms I'm in? |
| 23:08 | readme | just 2 others, i think |
| 23:08 | mheld | haha |
| 23:08 | mheld | and you're in boston, right? |
| 23:08 | mheld | (ish) |
| 23:08 | readme | yep |
| 23:08 | mheld | are you an alter ego of mine? |
| 23:08 | nsxt | readme: were you at the clojure meetup a few weeks ago? |
| 23:09 | readme | i wasn't. |
| 23:35 | mea | howdy |
| 23:35 | mea | is a Keyword just like a special variable? |
| 23:36 | mea | I don't really understand why it is used |
| 23:39 | mthvedt | keywords are interned literal |
| 23:39 | mthvedt | like strings, except a little more convenient for some things |
| 23:42 | tomoj | potemkin is impossible in cljs, right? |
| 23:50 | Raynes | https://www.refheap.com/paste/3000 3000 pastes. |
| 23:50 | Raynes | Woot. |