#clojure logs

2010-08-23

00:01qbgYou could do (merge-with + m {keyword-var 1})
00:01hiredmanamalloy: nothing silly about it
00:01qbgI don't think that would be idiomatic though
00:03amalloyhired-man: it just feels like there ought to be an update function for symmetry with assoc(-in)
00:05hiredmanamalloy: uh, thats what update-in is
00:06mefestoi think he means a non-nested version like how there is get/get-in assoc/assoc-in
00:06mefestoupdate/update-in
00:06wwmorganI've run across that actually. I expected an update that expects a single associative layer
00:07amalloymefesto: yes, exactly
00:10rhudsonamalloy: from your example form, I wonder if 'frequencies might be suitable for what you're doing
00:10amalloyhm. that's an interesting thought. as soon as you said it i started explianing why that doesn't make sense for me, but maybe it does
00:12amalloyi'm tracking something that changes over time, though, and i need up-to-date counts at an arbitrary time, so frequencies would involve a lot of unnecessary recounts unless it can do some remarkable caching/optimizing
00:13technomancypolypus: lein-clojars is ... well, if it's not deprecated it's at least un-recommended
00:16technomancyso... this post about Leiningen for Windows ... I have no idea what it means.
00:17technomancy"Wow! I touched it and it reminded of REBOL interpreter. I downloaded leningen-1.3.0, and lein batch. I put leningen-1.3.0.jar on by path. Hmmm, I got electrified. Now, I can make boast by saying this 'is the real one'."
00:17technomancyis that good?
00:17defnheh
00:18defnI think it's good.
00:18defn"I can make boast by saying this 'is the real one.'"
00:18defnthat sounds good.
00:18technomancyisn't rebol that forthy language with the terrible license?
00:20defni wouldn't know
00:20defnnaming your language with a /bol/ at the end fell out of favor a long time ago though
00:20defnso it wouldn't surprise me
00:55amalloyhow robust is clojurebot? for example, if we give him a computation that never completes, is he smart enough to give up?
00:58technomancyamalloy: short answer: yes. longer answer: maybe...
01:08danlarkingranted it's only two characters longer
01:08technomancytechnicalities...
01:11danlarkina world of depth lies in those two characters
01:15amalloythe ellipsis makes it a healthy five longer
01:19hiredmanclojurebot is semi-robust, Arkham has the possibility of making it much more robust but I am on the fence about using it
01:24tomojclojurebot seems to have bad holes to me, but I guess the people here are generally nice to bots
01:27hiredmantomoj: the jvm sandbox disallows io
01:28tomojright, so the box is safe, but is the bot?
01:29hiredmansemi-robust
01:30hiredmanfixing those holes either ends up with lots of regexs or instrumenting an interpreter
01:33hiredmanhttp://github.com/hiredman/Arkham
01:34hiredmanif I do use arkham as clojurebot's evaluator I think I would like to run it out of process and have clojurebot communicate with it over a message bus
01:34hiredman(more and more reasons to just leave things as they are)
01:36slyrus:require :only is supposed to make it so I can use unqualified names, right?
01:36hiredman:require doesn't have :only
01:36slyrusd'oh... :use :only
01:37slyrusgot it. thanks.
02:53LauJensenGood morning gentlemen
03:20sunkencityrylehHow can I convert a sequence of chars to a string? (str '(\F \O \O)) doesn't work and (reduce #(.concat %1 (.toString %2)) "" '(\F \O \O)) seems a bit unweildy
03:21wwmorgan,(apply str [\F \O \O])
03:21clojurebot"FOO"
03:21sunkencityrylehok tnx
03:30slyrusarggh... error: java.lang.IllegalArgumentException: No single method: make_atom of interface: chemiclj.core.PAtomGenerator found for function: make-atom of protocol: PAtomGenerator
03:34slyrusI know I've been through this before, but no & args with protocol methods right?
04:10amalloyi think that's correct. they map directly to java methods, right?
04:28LauJensenAWizzArd: Man Im suffering today, thanks to you (neo)
04:56NikelandjeloWeb application runs on jboss server. One class contains static field, which must be initialized with value, which I put in context.xml in tomcat folder.
04:58NikelandjeloNow field is set in index.jsp. But I don't think it's good way. How can I set it on initializing time?
04:58NikelandjeloOr sorry, it's for #java channel :(
06:10fliebelhey
06:21fliebelWould Clojure be a nice language to write an easy and fast network server, like Twisted and Node.js? Not necessarily evented or threading. Just thinking about the idea…
06:22raekI've heard that Aleph is a Node.js-like server
06:22raekto make a simple network server is certrainly easy
06:23raekcheck out http://clojure.github.com/clojure-contrib/server-socket-api.html
06:24raekaleph: http://github.com/ztellman/aleph
06:24raekthis could be relevant too: http://dosync.posterous.com/22397098
06:25fliebelraek: Reading...
06:28lenwhi all, trying to implement some swing in clojure and just stuck with implementing an interface - e.g. for a button need to add an instance of an ActionListener - how do i do that in clojure please ?
06:29fliebelraek: beautiful!
06:33fliebel*torn between EMCA and lisp* *wants Clojure in JavaScrip*
06:34raeklenw: proxy
06:34lenwraek: thanks
06:34LauJensen(defmacro onClick
06:34LauJensen [obj & body]
06:34LauJensen `(.addActionListener ~obj
06:34LauJensen (proxy [ActionListener] []
06:34LauJensen (~'actionPerformed [evt#]
06:34LauJensen ~@body))))
06:34LauJensen
06:35LauJensen(onclick (JButton. "OK") (println "clicked"))
06:36lenwLauJensen: thats pretty nest thanks - it was the proxy bit that i am seeinf for the 1st time
06:37raekI was just going to point you to slide 31 of http://www.slideshare.net/skillsmatter/clojure-and-swing
06:38raekbut LauJensen's code makes that redundant
06:38lenwraek: thanks looks like i need the banna slide show anyways :)
06:39Chousukedoesn't that create a new proxy for every ActionListener? Or did proxy do "caching" of classes?
06:43raekeach (onclick ...) form will result in the creation of a new instance (and a anonymous subclass) of ActionListener
06:43ChousukeIt might be useful to do something like (defn action-listener [f] (proxy [ActionListener] [] (actionPerformed [evt] (f evt)))) and then (defmacro on-click [obj & body] `(.addActionListener ~obj (action-listener (fn [evt#] ~@body))
06:43raekah, I see
06:44raekthis allowes one to reuse the same action listener for multiple buttons
06:44lenwyes thats neat
06:47Chousukein general, you don't want to pack too much functionality into one macro or function :)
06:48Chousuke(or even if you do, do it for convenience and by composing the smaller functions)
08:10AWizzArdLauJensen: oh oh oh, yes, the first days are the worst.
08:11AWizzArdLauJensen: I sent myself emails were I reported to myself my typing speed. I think after just a few days 250 kpm are possible.
08:13LauJensen:(
08:14AWizzArdLauJensen: about the swing example code above: I see it uses proxy. Can reify be used instead?
08:14LauJensenYes
08:14LauJensenIt was an almost 3 year old code snippet I had lying around :)
08:19AWizzArdrhickey: http://clojure.org/jvm_hosted could maybe also be updated to use reify vs proxy.
08:21@chouserproxy reuses classes based on what they extend
08:21leafw,(def a (reify java.awt.event.ActionListener (actionPerformed [evt] (println "buh!"))))
08:21clojurebotDENIED
08:22leafwthat code fails. I have lots to catch up with 1.2
08:22fliebelWhat is the difference between reify and proxy?
08:23AWizzArdreify works for interfaces only, but produces very efficient/fast code.
08:23leafwAWizzArd: mind to correct my code above? What is the proper usage of reify for, say, ActionListener ?
08:23AWizzArdproxy is more general and can subclass, but it needs more lookups and is thus a bit slower.
08:23fliebelAh, so it's good for implementing interfaces, not for inheritance.
08:24fliebelThat is one not-so-good thing about Clojure in my opinion: there are many ways to do something, unlike in Python(import this).
08:24gerryxiaohow does pmap work?
08:25@chouserthus proxy is meant for interop. reify encourages Clojure's ideas of good design
08:25gerryxiao,(pmap inc (range 1 100))
08:25clojurebot(2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100)
08:25gerryxiaoit means pmap fork 100 threads to do inc ?
08:26gerryxiaoor something?
08:26@chousergerryxiao: no
08:26LauJensengerryxiao: pmap chunks the collection and passes it to n+2 threads, where n is the core count
08:26fliebelgerryxiao: I think it;s a pool
08:26leafwgerryxiao: it's a pool of Ncores + 2 or so
08:26gerryxiaook
08:27fliebelI thought pmap wasn't in-order, but the above example seems sorted.
08:27leafwfliebel: pmap is like map--so ordered.
08:27LauJensenit works on a sliding window, so its sorted so to speak
08:27LauJensenWhich is also something you need to consider against the performance profile of the task, in that threads will wait for each other
08:27leafwpmap is a drop-in replacement for map, when the fn to apply to each element is heavier than the threading overheads.
08:28gerryxiaobut pmap works weird sometimes
08:28@chouserpmap returns a lazy seq. as the consumer of the seq approaches the end of what has been generated so far, pmap kicks off another cores+2 simulateous tasks and collects the results in order at the end of the lazy seq
08:29AWizzArdleafw: you need to list 'this' explicitly in your reify example above.
08:29leafwAWizzArd: thanks!
08:29leafwback to python "self" then
08:30@chouserthe overhead of tasking threads from the pool and collecting the results is substantial -- the fn you pass to pmap needs to take a fair about of processor time in order to end up being faster than plain map
08:30fliebelleafw: But I assume also he advantages of self ;)
08:30leafwmakes sense though; otherwise there isn't a handle for the object fields, and a magic keyword "this" could lead to errors.
08:30gerryxiaoi used pmap to update drawing pictures in swing application, it work fine in linux, but werid in windows
08:30LauJensenchouser: or if the op doesnt take time, partition the input accordingly
08:31AWizzArdTry (time (count (map (fn [_] (Thread/sleep 1)) (range 1000)))) vs pmap
08:32fliebelAWizzArd: Ugh, my Clojure is rusting… What is the underscore for? It's basically a blank hole argument, right?
08:32@chouserLauJensen: sure, though usually that partitioning and recombinding adds even more overhead
08:33fogusfliebel: "I don't plan to use this"
08:33@chouserI wonder if it'd be worth making pmap checked-seq-aware.
08:34AWizzArdleafw: btw, your "this" can be named as you wish. You don't need to call it 'this'.
08:34fliebelAWizzArd: Just like self, but it's confusing to name it different.
08:34AWizzArdfliebel: the underscore is just the name for a variable. In Clojure it is idiomatic to use that name to signal to the reader of the sources that he can ignore that parameter.
08:36LauJensenchouser: are calls to partition that expensive?
08:38fliebel*sigh* Clojure is such a nice language, but PHP, Python and Javascript are hard to avoid, and sometimes easier for a certain job.
08:39gerryxiao(doall (pmap draw-pics-fn data-lists)) works in linux, but in windows, some pictures werenot drawed when updating paint
08:39gerryxiaoi have to change pmap to map
08:40_fogus_fliebel: Use the best language for the job. You can't go wrong that way.
08:40gerryxiaomap works both in linux and windows
08:40Raynes Clojure isn't trying to replace other languages. Just trying to provide a new tool for your box.
08:41AWizzArdThe problem is that it infects your mind and makes you feel bad when you must read/type code in other langs (:
08:41fliebelall very true :)
08:41AWizzArdfliebel: best way is to convince your boss to rewrite the complete code base in Clojure, whatever you have.
08:42fliebelI don't have a boss, so that's easy :)
08:42RaynesI'll be your boss.
08:42RaynesFetch me some tea.
08:42fliebelRaynes: Try sudo...
08:43Raynessudo fetch me some tea.
08:43fliebel*gives tea to Raynes*
08:44fliebelThere are currently 2 projects I have to do in non-clojure. One is because it is in the browser, the other because Twisted preovides imap and smtp while aleph does not.
08:45fliebel*waits for Clojure in Clojure, and the for Clojure in JS and everywhere else*
08:47bobo_i work in php, beat that
08:48fliebelbobo: Hey, you could run quercus and bring your PHP to Clojure :)
08:49bobo_i wouldnt be allowed to use it anyway
08:50AWizzArdfliebel: depending on your customers need *maybe* a swing applet would be okay, as the client. Don't know.
08:54fliebelAWizzArd: Nope
08:54BjeringI have finished my chat-server, I am sure it is full of very suboptimal clojure. As it is it works very nicely with ~5k clients (using my old C++ stressbot to simulate them). However it lags and eventually fails with 20k clients and my C++ server used like 2% CPU at 50k clients. Any good suggestions on how to profile a clojure chat server?
08:56RaynesOh, wait.
08:57Raynes$huggle Raynes
08:57dnolenBjering: VisualVM ? https://visualvm.dev.java.net/
08:57Raynes!
08:58Bjeringgoogling also let me know about TPTP for Eclipse, as I use counterclockwise perhaps I should try this one? Anyone succeeded profling clojure code with it?
08:59dnolenBjering: It's pretty popular. YourKit also seems to be used by many.
08:59BjeringAlright, Ill test TPTP first, and then visualvm
09:06LauJensenBjering: Is the source code online somewhere?
09:07BjeringNo, but I dont mind putting it somewhere
09:08LauJensenTry gisting it
09:08Rayneshttp://gist.github.com
09:08BjeringIt is messy though, I thought I found a good way to organize my message passing, but I think I overengineered it and created a little mess. I want to sort that out if I proceed ofc. But the primary reason for this excercise was to stresstest it. But perhaps I should start with just making it clean first...
09:08LauJensenRaynes: you mean M-x gist-region
09:08Bjeringit is 7 files, make 7 gist posts?
09:09RaynesLauJensen: Assuming he has gist-mode.
09:09LauJensendone
09:14Bjeringalright
09:14Bjeringmain.clj https://gist.github.com/fd1b5b2b1bbe08ff8c30
09:15fliebelBjering: You can attache mutiple files to a gist
09:15Bjeringserver.clj https://gist.github.com/5b4f5a7d299068c7b978
09:15Bjeringsession.clj https://gist.github.com/bd04ff4594a3f5413bf2
09:15Bjeringfliebel: Ah, too late ;)
09:15Bjeringuser.clj https://gist.github.com/e81bb48793d311df1eea
09:15Bjeringchannel.clj: https://gist.github.com/fd6b3bf4d2426b95aed0
09:15Bjeringcommon.clj: https://gist.github.com/9af1d5dc17539d94cc77
09:16fliebelBjering: You use Netty? Have you had a look at Aleph?
09:16BjeringIf anyone looks at it, be ruthless, I suspect it is bad in many ways and my purpose is to learn!
09:16LauJensenFirst off, using Netty for something this I/O heavy is likely a mistake
09:17fliebelLauJensen: What would you use?
09:17LauJensenSomething synchronous
09:17fliebelLauJensen: With threads and stuff?
09:17LauJensenYes
09:17@chouserBjering: did you know you can put multiple files in one gist?
09:17LauJensenWhen a requests exceeds a certain threshhold, the async performance will crater
09:17fliebelchouser: I told him.. to late
09:18LauJensenCompare with something like Jetty you dont get a fraction of the scaling ability
09:18BjeringLauJensen, it is a "port" of a C++ boost::asio C++ server that is async and does 50k klients without sweat.
09:18@chouserfliebel: ah, sorry, missed that among the other posts.
09:18Bjeringchouser: I know now, will do next time :)
09:18LauJensenBjering: Likely its spending less time on each request then
09:19fliebelLauJensen: What dimension of a request determines if async or sync is better? I assume it has something to do with the duration and the amount of waiting that is done?
09:20BjeringLauJensen: I am curios, all my research and all my own attempts in C++ lead me to think asynch io was _the_ way to reach anything near 50k clients. But I take it its because in C++ that would mean sync would mean 50k native threads and that is NOT a good idea I am sure.
09:20LauJensenfliebel: Duration. You probably saw the comparison where Netty blew Node.js out of the water. Thats the sweet spot for async apps. Once you add a little I/O, you die on performance
09:21fliebelLauJensen: I don't think I did...
09:21fliebelLauJensen: But aren't Jetty and Node both async?
09:21fliebel*Netty
09:22BjeringMy stressbots are sending a "Hello I am here" message every 1-10 (uniformly random) seconds to its channel, I have 10 clients in each channel, each mesage is then sent to all members of that channel.
09:22LauJensenNot Jetty
09:22LauJensenhttp://groups.google.com/group/clojure/browse_thread/thread/f67b156b39f8e2ed/fad2f3ddc4c96ec7?lnk=gst&q=aleph+timeslice#fad2f3ddc4c96ec7
09:22LauJensenI'll let Peter Schuller explain
09:23AWizzArdNetty with 100k clients: http://www.jboss.org/netty/performance/20090607-asalihefendic.html
09:25BjeringLauJensen: I am happy to rewrite using another IO-lib if it is the reason. Now I really dont think it is. I think it must be something stupid I do in my clojure code (beeing clojure (and basically any jvm-language) newbie).
09:25LauJensenBjering: the stressbot is just sending messages right? So its not logging in and out of channels?
09:26BjeringRight, all channel-membership is hard-coded (in main.clj)
09:26LauJensen(Bjering, before you rewrite, check out Peter Schullers explanation, its sound)
09:28LauJensenBjering: I think my first step would be to run a profiler and find the bottleneck (jvisualvm will do fine), and then perhaps check back here with the details
09:31BjeringWell, I have very high concurrency (I think, ~25k IO actions per second) and each call is very fast (no DB-access just a little doseq over 10 items, queuing up async-sends). And those are the cases where Peter Shuller sais an asyncrounous model ges the most advantage. If I read that post correctly.
09:31Bjeringofc I am probalbly getting the worst of both worlds as I mix in agents in it. Paying for threads anyway.
09:32BjeringLearning to profile something non-trivial like this will be well worth learning anyway, I'll do that.
09:33LauJensenI would really be disappointed if you cant get that app to handle 50k connections
09:34BjeringLauJensen: When you did your chat-bot on the course you held, did you ever stresstest it?
09:34LauJensenNo - It was mostly an interop exercise
09:37BjeringLauJensen: Any comment on my use of partial to return a function I can use an object and pass messages to it? I feel huge guilt of overenginnering that one, but it did give me very nice message passing. But I feel guilty trying to force my OO thinking on this problem.
09:39LauJensenBjering: Where is dispatcher defined?
09:39Bjeringsame file/namespace earlier
09:40LauJensenBjering: Every call to partial creates a new fn, so I think that might be hogging some resources
09:41BjeringA function is expensive?
09:42LauJensenIf you're creating thousands of functions every second, instead of just calling them, then yes I think so
09:43BjeringI am so used to template metaprogramming in C++, creating a function there isnt quite the same thing, the cost is deducted at compile time.
09:45LauJensen,(time (dotimes [i 1e6] (partial + 1 2)))
09:45clojurebot"Elapsed time: 273.291 msecs"
09:45LauJensen,(time (dotimes [i 1e6] (+ 1 2)))
09:45clojurebot"Elapsed time: 130.757 msecs"
09:45LauJensen,(time (dotimes [i 1e6] (+ 1 2)))
09:45clojurebot"Elapsed time: 186.13 msecs"
09:45LauJensenShould be about 80% - 90% difference I think
09:46Bjeringok, thats ok, I am still at the stage as I need to learn how to avoid the x100 slowdowns, the x2 slowdowns I'll fix later :)
09:47BjeringI would pick clojure for this project if it was withing a x5 performance of my C++ without hesitation.
09:48LauJensenThen I think you should grab the profiler
09:48Bjeringyep, dipping my toes into visualvm now. I'll let you know what I learn.
09:52Bjeringwow, all tools in the Java worls are so nice :) I get good stuff from visuaVm without even doing a tutorial first, it found my running process!
09:52LauJensenYea its almost too easy
09:52Bjeringclojure.lang.Reflector.invokeMatchingMethod 82%
09:52LauJensenThen what you want to do is hook it into the process. Get th clients connected, then start profiling, stop after a couple of minutes
09:53LauJensenThat will narrow it down for you quite a bit
09:53Bjeringits what I did
09:53LauJensenBjering: Ok, compile your code using (set! *warn-on-reflection* true), see where its reflecting
09:53LauJensenThen sort that out with type-hints. Any reflection will grab 500x performance or so. (ball park)
09:54Bjeringsecond place goes to org.jboss.netty.socket.nio.SelectorUtil.select, but thos 7.7% is probably ahrd to avoid.
09:54Bjeringwill do
09:54bobo_Bjering: are you doing this for work?
09:54Bjeringbobo: Yes
09:55bobo_it just sounds like so much fun. hard to belive its work =)
09:55BjeringIts my own company ;)
09:55bobo_ah!
09:57bartjBjering, company link ?
09:59Bjeringbartj: None, yet, I have link to my old company, but thats not really relevant. Setting up a new one now, todo a _fun_ web-mmo-rpg/strategy game with real syncronous interaction (not that async bs that 99% of webstrategy games today use, especially the facebook ones).
09:59_fogus_dnolen: Are you planning on a Chambers/Chen impl?
10:00Bjeringhmm, attaching visualvm to my process made it unresponsive to ctrl-c ?
10:00bartjBjering, may the force be with you
10:00Bjeringthanks
10:01Bjeringalright plenty of warnings on reflection
10:02LauJensenGreat - If any of those are in repetitively called code, you'll get a huge boost
10:02Bjeringsuch as getBytes called on every string every time I send :)
10:02LauJensenhaha
10:02LauJensenYea, something like that
10:03Bjeringok, so this is where I use type hints?
10:03Bjeringline 24 in this one: https://gist.github.com/bd04ff4594a3f5413bf2
10:04LauJensenIf its guessing what str is, then prefix it with ^String str
10:06Bjeringok, then next one is wrappedBuffer on the same line, that is a static method, didnt thought that needed type hint?
10:07Bjeringeh not same line, but almost, line 22
10:07Bjeringah its the overload on arguement causing trouble I guess
10:11BjeringHow should I type hint that?
10:18Bjeringthis is the java signature of the method I reflection warns me about: public static ChannelBuffer wrappedBuffer(byte[]... arrays)
10:19Bjeringcan I? if so how do I? type hint so that the warning goes away.
10:21Bjeringfixed it
10:22Bjering#^byte[][]
10:26@chouserreally? that worked?
10:26Bjeringyep
10:26@chouserhuh, cool.
10:27Bjeringwell, warning went away, if it crash I dont knwo yet ;)
10:32LauJensenBjering: Hows performance now?
10:33BjeringStill working my way through the warnings, I have an odd case now where the type I "know" it should be doesn seem to HAVE the method I am calling on it... I wonder how that works...
10:35@chouserBjering: perhaps the args of the method are of the wrong type(s)?
10:36BjeringYes, I have used a confusing argument name and tricked myself...
10:36LauJensenwell played
10:40npoektophi! i've just set up lein with clojure 1.2. When i do lein swank, it just starts server. How to make it run the code and start server? I just did (swank/start-repl) in my main.clj before. Is there a new way?
10:46Bjering#^byte[][] crash my program :(
10:46Bjeringgives me #<ClassCastException java.lang.ClassCastException: clojure.lang.PersistentVector
10:46Bjering cannot be cast to [B>
10:47@chousera vector is not a byte array
10:47Bjeringbut it DID resolve the static method though
10:47Bjeringor I dont know, atleast the warning went away
10:47Bjeringso now I am back
10:48@chouserthat suggests that your hint is right and that the method you're trying to call is expecting that kind of array
10:48@chouserbut that the actual object you're passing in is a vector not an array.
10:48BjeringThe problem is on line 22 here: https://gist.github.com/bd04ff4594a3f5413bf2
10:49BjeringIf (into-array) dont give me the byte[][], how do I get it?
10:50Bjering,(into-array [(byte-array (byte 0)) (byte-array (byte 0))])
10:50clojurebot#<byte[][] [[B@41785a>
10:50Bjeringits the right type...
10:51LauJensenRaynes: Christophe is working on it IIRC
10:52RaynesOh. Of course he is.
10:53mefestohttp://docs.jboss.org/netty/3.2/api/org/jboss/netty/buffer/ChannelBuffers.html#wrappedBuffer(byte[])
10:53mefestoBjering: does the method expend just an array of bytes and not an array-of-array?
10:53mefesto*expect
10:54mefestolooks like you are passing it a byte[][] when it wants byte[] only
10:54BjeringThis is how it looks: public static ChannelBuffer wrappedBuffer(byte[]... arrays)
10:55Bjeringthere are many of it
10:55mefestoBjering: that is equiv to: wrappedBuffer(byte[], byte[], ...)
10:55BjeringI want the one with var-args
10:55Bjeringok
10:56Bjeringmefesto: So how do I type hint that?
10:57mefestoBjering: this is just for your 'encode fn right?
10:57Bjeringyes
10:57mefestoBjering: do you need to type hint on it?
10:58BjeringAh I thinkg I got it working, getting rid of the into array
10:58mefestohttp://gist.github.com/545640
10:59Bjeringwell, your version is missing the ending 0 that flash XMLSocket need as delimiter
10:59npoektopi do not get it. How to use lein swank? If i do lein swank, it just starts a server. What's next? I should do M-x slime-eval-buffer in my core.clj?
10:59mefestoBjering: sorry, http://gist.github.com/545640
11:01Bjeringthat doesnt resolöve, but this does: https://gist.github.com/4ea661b655ee113bb086
11:01mefestoBjering: ah i see
11:01Bjeringbut dont work...
11:05mefestoBjering: hmmm i guess it's having probs with variable arg java methods
11:05Bjeringyes, its all failing all the time :(
11:06mefestoBjering: would it be good enough to append the byte 0 at the end of the (.getBytes str ...) call and pass just the single byte[]?
11:06Bjeringperhaps as last resot make a little own static Java wrapper of my own?
11:06Bjeringsure, it is probably good enough, and prettyer thatn getting Java in there.
11:08dnolen__fogus_: I saw you messaged me on the backlog.
11:11_fogus_dnolen: I saw that you were chatting about predicate dispatch and was curious if you plan to go down that path?
11:12dnolen__fogus_: it is interesting to me, also seems related to your Trammel work.
11:12mefestoBjering: maybe this would work? http://gist.github.com/545675
11:12_fogus_dnolen_: I think so too. The reason that I ask is because I've started down that road and if you plan to do so also perhaps we could collaborate
11:13mefestoBjering: might need to add your type hint again ^byte[]
11:13Bjeringmefesto: This also worked: https://gist.github.com/b5422d6b599e91835eed
11:13dnolen__fogus_: I'd be up for that. do you have anything in a repo yet?
11:13mefestoBjering: cool
11:14_fogus_dnolen_: I do not, but I hope to have a unification lib pushed out soon
11:14mefestoBjering: i like your version much better :)
11:14Bjeringnew test run, 15k connections ok response time but still 75% CPU, time to profile again
11:15Bjering20k and it still works
11:15Bjeringmuch beter than before
11:15BjeringI'll let it run and see if it get all the way to 50k this time
11:15LauJensenLook forward to hearing the results
11:18BjeringNow it dies at 35k connections
11:19durka42is anyone using leiningen on OS X?
11:20dnolen__fogus_: nice! is there some good reading you followed for your unification design?
11:21npoektopdurka42: i do
11:21durka42npoektop: did you upgrade it recently? i just did and now when i run "lein help" my CPU usage spikes over 100% and nothing happens
11:22npoektopi've just downloaded it from it's site, and it works fine
11:23_fogus_dnolen_: I started with the description in *Problem-solving Methods in Artificial Intelligence* and then looked at the impl in PAIP. Then Clojure-ified it
11:29durka42technomancy: have you had any other reports of this issue?
11:36qbgYes! One of the textbooks for a university class that I'm taking is Programming Clojure
11:36tomojawesome
11:36qbgI've accomplished some of my goals
11:37mrBlissqbg: I envy you
11:37pdkwowzers
11:37qbgI could probably make by without the book, but I bought it anyways
11:37qbgFinancial aid can help cover it :)
11:40dnolen__fogus_: thx, I'll look into those.
11:41_fogus_dnolen_: I'll ping you when I finally finish up this lib
11:41BjeringI think the profiler interefere alot though, with it it can start failing at only 1k users.
11:41Bjeringbut
11:41BjeringBefore that it tells me 86% of CPU is in in org.jboss.netty.socket.nio.SelectorUtil.select
11:42Bjeringtime to read that source code
11:42BahmanHi all!
11:42qbgHi Bahman
11:44BahmanHi qbg!
11:44npoektopis it normal that lein-run needs clojure and clojure-contrib 1.1 jars while i'm using clojure-1.2?
11:46Vinzentnpoektop, v1.1 is in deps of this plugin
11:47npoektopVinzent: so it won't crush 1.2?
11:52durka42ninjudd: i'm trying to install cake on OS X. it fails because it can't find bake-0.4.4.jar in cake/lib/dev - do you know how to fix this?
11:57BjeringAnyone know if Java Hotspot Client VM for Windows 7 actually use iocp for its NIO.Selectors, all I read is that it use select, which on windows is basically broken (well for >64 sockets atleast). Perhaps this is what explains why by boost::asio solution works so much better on my windows machine.
12:04technomancydurka42: if you run something like "lein help" in ~ where the src/ dir is huge, there's nothing leiningen can do; the JVM will just freak out when you try to load anything
12:04technomancythat's just a bug in Java
12:05durka42technomancy: same with "lein version" in an empty directory
12:06technomancymust be something else then; maybe a mac-specific quirk
12:20slyrusdurka42: wfm
12:31LauJensenBjering: 1) Yes the profiler inters enormously, so everything looks weird in the numbers, 2) Time to try something other than Netty, if you have to dig into the source of it?
12:32BjeringThe source was a dead end anyway, only contained one line and that was into a Java Nio class
12:36dnolen_Netty seems pretty well suited to this task. Bjering have you done any memory profiling?
12:36npoektophi! I've just ran (println (clojure-version)) using lein run. It says 1.2. I ran lein-1.3 with lein-run plugin that needs 1.1. Won't those 1.1 jars needed for plugin interfere with 1.2 jars for code?
12:36Bjeringjust rerun my C++ server... 50k connections, 2% cpu, its like a x100 difference :(
12:37LauJensenBjering: Well, you're not done optimizing yet :)
12:37Bjeringdnolen_: Nope, I am new to trying to make high-perf jvm programs. So there might be another story then the 85% CPU in org.jboss.netty.socket.nio.SelectorUtil.select the profiler told me about?
12:38ninjudddurka42: about to leave for work, but I can help you when I get in. for now, try removing lib and rerunning
12:38dnolen_Bjering: you probably need to startup with JVM process with a lot of RAM
12:39LauJensendnolen_: So you think its just dying on memory ?
12:39LauJensendnolen_: Earlier he was at 75% CPU with 20k connections
12:40BjeringI wonder if the -server switch to java does me any good
12:40dnolen_Bjering: I would start with a gig and go from there.
12:40dnolen_Bjering: you should always user -server
12:41dnolen_LauJensen: there could be a lot of things at play, the I doubt Netty is the problem.
12:41LauJensenBjering: How much mem does your system have?
12:41Bjering6 GB
12:42Bjeringbut I run 32 bit java
12:42LauJensenjava -server -Xms3G -Xmx3G then.
12:42LauJensen(at least I think G is the notation for Gigabyte)
12:42dnolen_Bjering: at the same time you need some realistic expectations. For a such a simple app, I think the CPU and memory profile of the C++ version will always trump Java even if they can serve the same # of connections.
12:43Bjeringdnolen: I am hoping for 10% CPU at 50k, that is x5, then Clojure with STM will be my pick for game logic rathern then spending the next 6 months fiddling with locking strategies in C++ for my game logic.
12:44LauJensenBjering: I didn't see you using the STM anywhere in your code, did I miss something?
12:45BjeringLauJensen, no not in this app, but this is just a benchmark, I can't write my entire game twice ;)
12:46Bjeringno locks to speak off in the C++ version either...
12:46LauJensenBjering: I understand, but the different ref types have different performance profiles. The STM is much slower than something like atoms for instance - different use-cases entirely
12:46Bjeringyes, I'll try to pick wisely
12:46BjeringI dont expect many collisions though
12:47Bjeringgame objects will be far apart, rarely interfering
12:48LauJensenIf you're picking between atoms and refs, the question is simple "Do you need to coordinate change?", if you do, refs are your only option
12:54Bjeringit lags less, my ping is now 0-1 ms just like the C++ (I am at 15k uses now), but the CPU is maxing and making it hard to type in irc window even!
12:54Bjeringping starts rising at 20k connections though.
12:56AWizzArdBjering: I did not follow exactly what you are talking about. I saw “netty” before. Do you work on some kind of web server?
12:56BjeringAWizzArd, no a chat server
12:56Bjeringconnections are not closed
12:57AWizzArdirc or your own protocol?
12:57dnolen_Bjering: heh. what other calls were taking up yr CPU time besides select?
12:57LauJensenAWizzArd: read the log from 15:00 up to now, its mostly been about this client :)
12:57BjeringWell, its a benchmark/learning experiment. If it works well it will become a gamesever for a flash-strategy-rpg-mmo....
12:58slyrusok, so if I split my (library) code up into a bunch of files and a bunch of namespaces, how do I go about providing a simple interface to the library? A core file with a bunch of defaliases?
12:58slyrusmake people remember which ns things belong to?
12:59@chouseryou can split into multiple files without splitting to multiple namespaces.
13:00slyrustrue... I was under the impression the "idiomatic" way to do things in clojure was one ns per file, however.
13:01slyrusand how is the order of loading of multiple files with the same ns controlled?
13:02slyrusand, in general, how does clojure know which files I want loaded when I :use or :require an ns?
13:02slyrus(:use and :require operate on ns'es, not files, right?)
13:03slyrusonce again, I'm used to the CL way of telling asdf which files are supposed to get loaded and having the package thing somewhat decoupled from the loading thing
13:03Bjering_hmm... does java -server means "I give you permission to totally hang my user interface if you think you need to" ?
13:04Chousukeno :P
13:04LauJensenit means, use a compiler which optimizes more heavily
13:04Bjering_had to hard reset my win 7 machine...
13:04Chousukeif your user interface hangs, the OS is not doing the scheduling right.
13:04LauJensenBjering_: Yea, developing on a Windows system.. perhaps not the best idea :)
13:04Chousukeor you're heavily oversubscribing your RAM capacity.
13:06slyrusso to spill my ns across multiple files on file gets a (ns ...) and the others get (in-ns ...)? is there a particular order in which those are loaded? does require load all of the files with the in-ns form?
13:06Bjering_I am somewhat wondering if perhaps its windos that scews my performance measurements also. As I know boost::asio is very well built on top of IOCP, windows preferred high performance network api. Perhaps Java.NIO/netty is less well tuned on Windows systems but very competetive if I ahd a system with epoll or such. ?
13:07qbgrequire loads a file at the expected path; that file would have to load the others
13:07dnolen_Bjering: it would be interesting to test on Linux yes. But looking around briefly Java NIO on Windows seems to use IOCP as well.
13:07@chouserslyrus: yeah, I think that's frequently recommended. you have to load those other files explicitly at the end of your main (ns ...) file, so you get to define the order.
13:07slyrusah, ok! thanks chouser. I was missing the explicit load step.
13:08@chouserclojure/core.clj does this -- looke for "helper files" near the end.
13:09slyrus(I think) it would be nice if ns forms had some way of re-exporting symbols
13:11slyrusand I'm I correct in surmising that defalias yields a new var with the current value of the old var s.t. if I, say, reevaluate a defn for original val, the alias points to the old function?
13:12Chousuke(doc alias)
13:12clojurebot"([alias namespace-sym]); Add an alias in the current namespace to another namespace. Arguments are two symbols: the alias to be used, and the symbolic name of the target namespace. Use :as in the ns macro in preference to calling this directly."
13:12ChousukeI think defalias uses that.
13:13slyrusnot the way I read it
13:13slyrusICBW
13:13Chousukehmmh
13:13slyrusgrumbles at having to :use/require 6 namespaces for 1 "logical" set of interfaces
13:14slyrusand at technomancy :)
13:14qbgIf your namespace is as big as clojure.core, you have done something wrong
13:15technomancyright; clojure.core is not generally a good example of how to structure user code
13:16technomancyanyway, if it takes six files to implement what you want, you should still be able to gather all the external interface functions into one or two nses
13:16technomancyjust don't use immigrate.
13:16LauJensentechnomancy: Im trying to learn this and thought of you :) http://neo-layout.org/
13:16slyrustechnomancy: yes, that's what i'm trying to. trying to figure out the best way to do that
13:16slyrusand i've never heard of immigrate
13:18slyrusthe other wrinkle is that I've got protocols, which can-get/need-to-be loaded early, I've got the implementations which I'd like to keep as discrete, disconnected units, and then I've got the external interface functions.
13:19BjeringPerhaps I should rewrite the server in Java, just to really isolate the issue and have a good way to approach people like those on Nettys mailing list.
13:20dnolen_Bjering: not a bad idea, it would be cool if you wrote up your experience somewhere. I'm interested in how this all turns out for you.
13:21dnolen_Bjering: also if you put your code up, I'd be interested in checking it out / running it.
13:22slyrustechnomancy: the problem is that i'd like both the protocol methods (is that the write term?) and the other external functions in the same ns, but I need to define the concrete classes in between those two steps. Easy enough if everything's in one file, harder if you split it up.
13:27slyrusdoes putting all of the protocols in their own ns and expecting folks to :use/:require that make sense?
13:27slyrusoh, wait, I can do the load trick here. nvm...
13:28Bjeringdnolen_: https://gist.github.com/25aa623f0df970b7282d
13:28BjeringI included main.cpp for my stressbot too
13:30Bjeringand a little readme for the protocol
13:31dnolen_Bjering: cool, will give a looksee later when I got a bit more time.
13:32Bjeringvery cool, Iäll do that java rewrite. If nothing else I should write 500 LOC java per year to keep Java on my resume without lying too much....
13:36LauJensenBjering: Wouldnt it save you a ton of time, just to test it with Jetty or some already written server first?
13:36BjeringLauJensen, sure, how do I do that?
13:36Bjeringwebserver isnt what I need
13:37LauJensenI haven't looked into your options - I just occured to me, that you could save some time by taking that route first
13:37LauJensenThough, that route may start on Google :)
13:37BjeringI need long-living connections, and I do not want to do Comet style hacks...
13:38ninjudddurka42: should be fixed now. looks like i uploaded a bad jar for cake when i was playing with the release task this weekend
13:38LauJensenBjering: Hang on, memory chip activated
13:39ninjudddurka42: you may have to clean out lib. let me know if it works. and thanks for catching the problem!
13:39LauJensenBjering: check out James Reeves reply: http://groups.google.com/group/compojure/browse_thread/thread/1f0df9d6656fa069/56fba3d74b15c38a?lnk=gst&amp;q=long+lived#56fba3d74b15c38a
13:39LauJensenThat was actually for a chat I did as well
13:40durka42ninjudd: where should i download from? the version from git clone does the same thing
13:41BjeringLauJensen: i have written a long-poll solution myself in C#, in my opinion it was a brittle thing, and wastful of bandwith etc. Why not use proper TCP-sockets when that is what you really want?
13:41LauJensenIn this case because I just need a simple web based cht
13:41LauJensenchat - took almost no time to implement
13:43BjeringBut did you stresstest it? The C# thing I wrote died horribly early arguable becuuse the way I hooked it into IIS, but I have a very time thinking that all that extra communication that comes with a http/long poll is going to help performance.
13:43Bjeringvery hard time
13:47LauJensenI didn't no
13:47LauJensenI think I saw a socket interface in contrib earlier today though
13:47LauJensenOr maybe my memory chip is failing
13:48BjeringWell well, the way this is going before it is over I shall have made a Jetty version, and a node.js version and a Python Twisted version before I can even start thinking about delivering business value ;)
13:50ninjudddurka42: forgot to bump the version... pull and try again
13:52durka42ninjudd: now it just says "unable to find cake version on clojars" :(
13:55ninjuddhmm. 0.4.6 is there. I'll fix it when I get to the office. that's what I get for trying to fix code fro
13:55ninjudd m my phone
13:55durka42heh, no problem
13:57durka42ninjudd: FWIW, using the gem-installed version, it tries to fetch deps clojure and clojure-contrib, and can't find them, even though they are there
13:59slyrushmm... it feels a little dirty, but some explicit (use ...) sprinkled throughout my chemiclj/core.clj allows me to define the protocols and helper functions in the core ns, and the defrecord forms and what not in more specific ns'es (bond, atom, etc...)
13:59LauJensenhehe, sounds a little dirty as well
14:00slyrusI'm open to alternatives, but this is the best I've come up with so far.
14:04slyrusit's in here: http://github.com/slyrus/chemiclj/tree/refactor2 if anyone's curious
14:05LauJensencurious, but a little pressed for time tonight
14:11ninjudddurka42: seems to work from a fresh git checkout for me. if you want i can help you debug in #cake.clj
14:21slyrusok LauJensen, let me know if you get a chance to take a look and have any questions
14:22LauJensenI will be sure to do that
14:22slyrusawesome
15:06arkhclojurebot: ping
15:06clojurebotPONG!
15:15tomojgrr.. clojure.core.Vec isn't print-dup'able
15:24hamzaguys, I need to read a file containing clojure code but some variables defined in the file are defined in the namespace I am reading it in something like the following (let [content "asdasd"] (eval (read-string "(println content)"))) file contains print content but contant is defined in the file reading the script is it posssible to make this work?
15:25mebaran151any recommendations for a good mail client in emacs? I realized that instead of cutting and pasting buffers into my mail client, it would probably be more efficient for me to integrate mail handling directly into emacs
15:26LauJensenmebaran151: I think currently wanderlust is your best bet - I demoed it in my last screencast
15:27bobo_is there anywhere i can find a clj-cassandra that actually have al its dependencies? tried 0.1.1-0.1.3 and master. all filing on deps
15:27bobo_*failing
15:29bmhAny Penumbra users here?
15:29technomancymebaran151: gnus is great, but it's very different from traditional MUAs.
15:29mebaran151can I set it up to work with imap properly?
15:29RaynesAnd Evolution. I'm too frightened of gnus.
15:30mebaran151I'm on Evolution right now
15:30LauJensenmebaran151: If by properly you mean get notified every time a mail rolls in no, thats an outstanding bug in Wanderlust, so I wouldnt use it without some kind of tray icon notification handy
15:30hamzamebaran151: gnus works fine with imap but it is dog slow.
15:30mebaran151I actually use my phone for that
15:30technomancymebaran151: no, since Emacs is single-threaded all imap calls are blocking
15:30technomancyso you have to use offlineimap or fetchmail or something like that.
15:30LauJensentechnomancy: It has an async option
15:31bmhoh, technomancy, perhaps I should hassle you. When I run `lein native-deps` I get an IllegalAccessError: "default-repos does not exist (native_deps.clj:1)"
15:38ninjuddbmh: i think dnolen_ wrote the lein native-deps plugin
15:43Rayneshttp://gist.github.com/546158 Why is the output different in these situations?
15:44RaynesI need the former output in the latter example.
15:45@chouserRaynes: exceptions get wrapped as they bubble up through exception handlers
15:46ninjudd*e is a clojure.lang.Compiler$CompilerException in that case
15:46@chouseryour try/catch is catching it at a lower level before the REPL's eval wraps it
15:46RaynesIt appears that print-throwable seems to print it properly.
15:47_fogus_Our own Brian Carper gets #4 geekiest HN post! http://www.swimwithoutgettingwet.com/blog/most_technical_hn/
15:49RaynesGeeky posts on HN? No way.
15:53dnolen_bmh: feel free to open an issue.
16:14LauJensenI have this annoying little NS issue. when I compile my namespace I get a few of these WARNING: something already refers to something else. Which is fine except for 2 things: 1) It doesnt say which import is triggering the warning, so debugging is tedious, and 2) I cannot reeval that ns declaration without killing the repl first. whats up ?
16:17mebaran151I sort of wish emacs package management were more like Lein and Maven, where I could just supply and edit a text file of the elisps I want to try, call build, and have them all magically available. Then if I didn't want them, I could just remove them and rerun the build function or whatnot.
16:26chillitomanyone having success with leiningen and cygwin? (work laptop.. i know i know)
16:27LauJensenhaha
16:28LauJensenchillitom: You've been grilled a little in the past I see :) I've only heard of people have problems with lein on Windows, but that may be because people arent good enough at coming in here and sharing their success stories :)
16:28chillitomLauJensen: i sense a little Nelson in your 'haha'
16:28chillitomwell i'll give it a try
16:28chillitomfailing that i guess virtualbox will be the next step
16:28LauJensenNo no, I just thought your comment was funny :)
16:29LauJensenchillitom: Do you know that you can boot a physical partition with virtual box?
16:29chillitomLauJensen: i did not, interesting indeed
16:29LauJensenYea - I can boot Windows 7 from /dev/sda2 right within Arch
16:29chillitom(but for another occasion, can't touch this MBR)
16:29LauJensenA real timesaver in case you need to debug
16:30ninjuddchillitom: you could also give cake a try. i know several people who use it successfully on windows
16:30LauJensenYea that would be my first stop
16:30chillitomninjudd: thanks for the suggestion, i'll take a look at that
16:33technomancymebaran151: you can basically do that with package..el
16:33technomancyjust not many lib authors make their stuff available through that
16:34technomancyyou can pester them about it though. =)
16:40chillitomninjudd: Cake looks nice, does it have a package repo side to it like lein/Maven?
16:42ninjuddyeah, it uses maven like lein
16:50raekhrm, why is my slime evaluation broken? it always eval everything in the repl's current namespace
16:50raekinstead of the namespace of the file
16:51raekpreviously, everything was fine if I just evaled the (ns) form first
17:02Scriptorhi, I've got a question about clojure's implementation of multimethods, is the specific function that's needed determined at run-time?
17:03Scriptoror does it try to figure out as much as possible at compile-time?
17:03arohnerScriptor: the dispatch function is called each time the multimethod is called. the matching multimethod is picked based on the return value of the dispatch function
17:03arohnerso it all happens at run time
17:03Scriptorarohner: ah, that makes sense, thanks
17:05@chouserbut some of that is cached the first time
17:09duncanmis it okay if i just place some jars in the lib directory of a lein project without listing them in project.clj?
17:09arohnerduncanm: as a quick hack, yes
17:09duncanmarohner: cool, that's all i'm looking for
17:10fliebelduncanm: How do you run Clojure with all those on the classpath? or do you just compile a super jar?
17:12fliebelI remember having a special clj set up somewhere to use that dir, but maybe I missed some magic that way.
17:19duncanmfliebel: i dunno
17:19duncanmi have my own maven server, but i was hoping not have to write a pom for each of these jars
17:25BjeringAlright, dont know if anyone is awake from when I did my experiments earlier. But if you are, I have now rewritten server in Java and the performance is exactly the same as for the Clojure one.
17:31LauJensenBjering: Thats nice :)
17:31BjeringWell, Nice for Clojure, bad for jvm.
17:32LauJensenExactly
17:32fliebelBjering: Didn't catch the whole story, but parts of it.
17:32LauJensenBjering: In most of the benchmarks Ive seen, the JVM can keep up with C++ pretty well, not worse than 2x slower in the worst cases, and faster in the best
17:33Bjeringjvm/netty/windows 7, perhaps more accuratly. Now I should post to the netty mail list and hear if this is as good as it gets or if I am doing something very wrong.
17:35BjeringThat, and/or make so I can run Linux on this machine, to see if that is it.
17:35blackrainHi guys, which book should I choose to learn clojure? Which is the most up to date?
17:37fliebelblackrain: The one that;s not yet released… forgot the name… joy of clojrue, or something like that? ask… *digs memory* fogus and chouser?
17:37bmhblackrain: Are you sure you need a book? The online resources are wonderful. (That said, I did almost buy Practical Clojure yesterday)
17:37LauJensenfliebel: http://joyofclojure.com/buy I think
17:38blackrainfliebel: yeah, I've seen that
17:39blackrainbmh: I believe I do not, but in book everything is in one place, no need to browse net
17:39bmhblackrain: totally reasonable.
17:51dnolen_Bjering: Interesting. That is good for Clojure :) On a side note, I've been using AWS EC2 to run quick Clojure perf tests on fast servers. They're easy to setup and tear down quickly. It does take some time to get a proper image setup, but once you have that it's pennies to keep an image around ready to boot.
17:53slyrusgood for clojure not running on the JVM anyway...
17:54BjeringHow fast is such a system? I mean, my workstation is a Dell T7500, that is a 3.2ghz quad-core Xeon W5580. Does it blow that out of the water?
17:55dsantiagoBjering, sorry I missed the start of the conversation; what did you write?
17:56Bjeringdsantiago, a very basic chat server, to test/learn clojure and netty.
17:56dnolen_Bjering: I've run tests on 8-core Xeon servers with 23gbs of RAM. Mostly to compare the performance of Aleph with various DBs as well as benching comparisons w/ Node.js
17:56dsantiagoOh, cool.
17:57dsantiagoHow does that relate to clojure not on the JVM?
17:57slyrusdsantiago: this sounds like a JVM-specific problem
17:58dnolen_slyrus: possibly only on Windows
17:58slyrusah, ok, nvm
17:58slyrusyes, that is good then :)
18:00Bjeringdnolen: Now the "interesting" part for me is what I see in my taskmanager right now, the C++ server, now running only one single thread, actually for some reason takes alittle more time with 1 thread than with 4, proably causes some queuing at times and that costs. It now takes about 5% CPU on average with 50k clients. If I cannot use Clojure with its nice concurrency mechanisms best contender is actually single threaded C++, then the locking issues I fea
18:00Bjeringr goes away...
18:02dnolen_Bjering: that might be the case up until the point your start having significant computations on the server right?
18:02BjeringYes, and I will keep pretty much all to O(1) operations to stuff in RAM so server load should be light.
18:03Bjeringlight per connection atleast.
18:06BjeringWell, I'll test on Linux, I should get this machine dual-boot anyway and then we will see. And perhaps ask around some more. And then finally before I give up I will try a silly solution with a connection-multiplexer/demultiplexer-only C++ server infront of a clojure server and have them speak only over small number of sockets.
18:07Bjeringnow its time to go home.
18:19AWizzArddnolen_: how did that 8 core perform with Aleph?
18:22msfhello everyone
18:23msfjust started playing with clojure 1.2.0, when I load clojure.contrib.duck-streams into my namespace I get warnings printed to the console, is there anything I can do to suppress this ?
18:25tomoj(:use [clojure.contrib.duck-streams :exclude (spit)]) :/
18:27raekmuch of duck-streams has been moved to clojure.java.io
18:30slyrusit would be nice if the warning would tell you who the offending ns was
18:30dnolen_AWizaArd: on par with Node.js for the hello world benchmarks, faster of course for talking to a DB, and way better if clients are accessing shared memory
18:33bmhdnolen_: Can you link me to the tracker for lein native dependencies? I want to submit the `native deps explodes` issue. I don't know if it's a penumbra issue or a native deps issue.
18:36dnolen_bmh: http://github.com/swannodette/native-deps/issues
18:36duncanmboo, i have this jar that i want to use, but some of the classes don't live in a package
18:36duncanmi dunno how to import them
18:37woobytechnomancy: i noticed lein uberjar succeeds even if compilation fails, is that desired for some reason + would a patch be welcome?
18:38mebaran151I'm thinking of wrapping MessagePack for Clojure RPC: are there any better clojure RPC centric solutiosn out there these days?
18:40technomancywooby: that's definitely a bug; patch would be great.
18:40gfrlogdoes lazy seq => stackoverflowexception blatantly suggest a particular kind of error?
18:41woobytechnomancy: cool i'm on the case, thanks
18:44ihodesquestion: i forked a git repo (lein's) and use it to patch stuff etc. how can i pull upstream changes? should i add a remote of the main branch and fetch from there? or is there a better/different way to do that?
18:46technomancyihodes: yes, you want to add a remote, but if you've committed to your master branch it gets a bit confusing since pulls result in unnecessary merges. that's why I recommend using a topic branch for contributions
18:47ihodesah alright
18:47technomancyihodes: details here: http://technomancy.us/135
18:47ihodesif i fetch and merge it should be ok for now, though… i'll deal
18:47ihodesah thanks
18:47ihodesi'm going to look at issue #98
18:48technomancyihodes: actually hold off on that one; I have half a fix ready
18:48technomancythanks though. =)
18:48ihodeshaha alright
18:49ihodesyou work too fast :P i definitely want a sticker, but i've got to do at least a few more fixed/features before i ask :P
18:49technomancynot at all; every little bit helps and is sticker-worthy. =)
18:53ihodesI need a *bit* more than, what, 50 characters? i refuse a sticker, for now ;) any particular issues/low-hanging fruit i might be able to hack on?
18:57mebaran151gfrlog: could the function that produces the lazy-seq have an infinite loop?
18:57mebaran151it sounds like one of the steps (maybe the first one?) is resulting in a computation that will never end
18:57gfrlogno, I don't think so. It's produced with iterate, using an iterate function that's pretty simple I think
18:58technomancyihodes: maybe a :min-leiningen-version in project.clj where it would warn if your current version was too old?
18:58technomancyeither that or issue 92
18:59gfrlogmebaran151: regardless, the stack trace clearly involves unwinding a lazy seq
19:00gfrlog,(first [])
19:00clojurebotnil
19:02ihodesi'll check both out!
19:07chillitomleiningen isn't putting itself on my path, is that normal?
19:07chillitomif i install the script myself where should I put it and with what perms?
19:11ihodeschillitom: are you on a *nix?
19:12ihodesif so, 755 /usr/bin/lein is mine. i'd use the self-install if possible.
19:13chillitomihodes: yeah linux, i made a ~/bin and added it to path for now
19:13chillitomdoes the clojure repl make use of jline if it's available?
19:14ihodesthe one you get from runing java -cp pathwhatever clojure.main?
19:15gfrlogihodes: I don't know what chillitom meant, but I'm curious about the answer to that version of the question
19:16ihodesgfrlog: chilliton: if you want to use jline, it needs to be on your classpath, and you need to run it like this java -cp jlinpath:cljpath jline.ConsoleRunner clojure.main
19:16ihodesclojure's getting started page covers this i think, if you're having trouble setting it up
19:16ihodeshttp://clojure.org/getting_started
19:16chillitomihodes: cool
19:16ihodesbut i'd recommend using lein repl or cljr repl
19:17ihodesso you don't have to worry about your classpath
19:17chillitomso can i just add it to a lein project to get it on the lein repl then?
19:17gfrlogI think lein repl includes it automatically
19:17duncanmis reify the new proxy? shouldn't i use reify instead proxy if i'm on 1.2?
19:18chillitomgfrlog: will investigate, i'm getting escape codes when pressing arrow keys
19:18duncanm(reify java.io.FilenameFilter (accept [^java.io.File dir ^String name] (-> name (.startsWith "foo"))))
19:18gfrlogmaybe the version of lein is old?
19:18ihodeschilliton: gfrlog is right. well, it used rlwrap first, then jline if rlwrap isn't around
19:18ihodesuses*
19:20chillitomthanks guys, i'll investigate further tomorrow after another delightful day in the procedural day job
19:21ihodeschillitom: enjoy :)
19:21duncanmoh, the reify docs are out-of-date - it doesn't say that you need to explicit list the 'this' argument to a method
19:23ihodesduncanm: I think it is there: "Note that the first parameter must be supplied to
19:23ihodes correspond to the target object ('this' in Java parlance)."
19:23duncanmihodes: ah, but it's not in the example (the toString)
19:24ihodesit is in the one i'm looking at! haha here's what i see: http://cl.ly/25pG
19:25duncanmoh, i'm looking at http://richhickey.github.com/clojure/clojure.core-api.html#clojure.core/reify
19:25duncanmmaybe that's not the right site?
19:25ihodesah no, that's old
19:26ihodeshttp://clojure.github.com/clojure/clojure.core
19:26ihodesclojure is all under the clojure organization now
19:31ihodesduncanm: also, while i'm not sure reify is totally a replacement for proxy, it looks like it is. there's some different in syntax, reify seems to be able to do all that proxy can and more.
19:31ihodesi'm curious why it's still there (proxy)
19:31ihodesi guess it's nice it is, otherwise i'd have to update some of my code… but no big deal…
19:32hiredmanyou cannnot reify classes
19:33tomojisn't it just the opposite? proxy can do all that reify can and more? but reify is faster?
19:34hiredmanyou cannnot reify classes
19:34tomoj...thus proxy can do all that reify can and more?
19:35ihodestomoj: ah yeah, that might be it. though can you proxy protocols? i think you can only reify those.
19:51duncanmhmm, doseq isn't quite like for-each in Scheme
19:53ihodeswhy not use map?
19:56duncanmihodes: i could, but i'm only using it for side-effect
19:56duncanmi like the symmetry between MAP and FOR-EACH in Scheme
19:58ihodes(doseq (map (fn sideeffects-and-wahetever coll)) should do you then :)
19:58ihodesdorun*
20:00gfrlog(println ob) => StackOverflowError
20:00gfrlog(println ob) ; again, immediately after. Works fine
20:01pdkneeds related code for context gfr
20:02gfrlogyeah
20:02gfrlogit's probably not worth explaining till I've done more testing
20:02gfrlogfor now I'm just crying and cursing lazy seqs
20:11duncanmhmm, how do i load clojure.java.io in slime/swank?
20:11duncanm(require 'clojure.java.io) doesn't seem to work
20:17duncanmweird
20:17gfrlogokay
20:17gfrlogI must be crazy now.
20:17gfrloghttp://gist.github.com/546627
20:18gfrlogwith that in my repl, I call (def ob (process-from-ob)), and when it returns, I call (println ob) twice. The first is stackoverflow, the second suceeds
20:29duncanmhmm, do i get destructuring in doseq?
20:31gfrlog,(doseq [[a b] [[1 2] [3 4]]] (println b))
20:31clojurebot2 4
20:31gfrloglooks like it
21:03duncanmsomehow i can't require clojure.java.io in my project
21:03jstirrell`So I'm attempting to learn about neural networks without much success. By any chance can anyone explain to me what's going on in the "apply-activation-function" function on this page:
21:03jstirrell`http://github.com/fffej/ClojureProjects/blob/master/neural-networks/src/uk/co/fatvat/mlperceptron.clj
21:03duncanmif i write 'clojure.java.io/file', it works, but 'file' alone doesn't
21:04gfrloghow could I concat eagerly?
21:05wwmorgan(doc doall) ; <- gfrlog
21:05clojurebot"([coll] [n coll]); When lazy sequences are produced via functions that have side effects, any effects other than those needed to produce the first element in the seq do not occur until the seq is consumed. doall can be used to force any effects. Walks through the successive nexts of the seq, retains the head and returns it, thus causing the entire seq to reside in memory at one time."
21:06gfrlogdoes that have an affect on the original object? like is it easier to compute the second time?
21:07msfwhat is an easy way to coerce a vector into a set ?
21:07gfrlogI'm feeling a need to use this and I don't even have side affects
21:07slyrushmm... so much rope. this is perhaps the best I've come up with so far... a separate protocol ns used by the implementation ns'es, with each method defaliased in the core file s.t. folks don't need to (and shouldn't) pull in protocol, but only core and any other ns'es as needed.
21:07gfrlog,(into #{} [:this :is :my :vector :my :my]) ; <- msf
21:07clojurebot#{:my :is :vector :this}
21:07slyrusstill, would be nice to get rid of the defaliases
21:07msfthanks
21:08duncanmihodes: ping?
21:08wwmorgan,(set [1 2 3])
21:08clojurebot#{1 2 3}
21:08gfrlogaw, pwned
21:10ihodesduncanm: i'm here :0
21:10duncanmihodes: have you used clojure.java.io?
21:10ihodesa bit yes
21:10gfrlog,(loop [n 100000 v []] (if (zero? n) v (recur (dec n) (concat v []))))
21:10clojurebotEval-in-box threw an exception:java.lang.StackOverflowError
21:10gfrlogdoes anybody know what's happening there?
21:11duncanmihodes: i can't seem to 'require' it correctly
21:11ihodes,(require 'clojure.java.io)
21:11clojurebotnil
21:11ihodesseems to work
21:12ihodes,(require '[clojure.java.io :as jio])
21:12clojurebotnil
21:12ihodes,jio.resource
21:12clojurebotjava.lang.ClassNotFoundException: jio.resource
21:12ihodes,jio/resource
21:12clojurebot#<io$resource clojure.java.io$resource@857163>
21:12ihodesjust do that and you should be fine
21:13wwmorgangfrlog: my guess is that since concat is lazy, it stacks up all the calls to find the first element. You could solve this by calling seq on v inside the loop
21:13gfrlogI figured it was something like that, but it seems like a bug...shouldn't you be able to use lazy seqs without worrying about the call stack?
21:14ihodesgfrlog: something like what your doing should probably be a lazy-seq
21:14gfrlog,(loop [n 100000 v []] (if (zero? n) v (recur (dec n) (concat (seq v) []))))
21:14clojurebot()
21:15gfrlogI guess I don't understand the difference between a lazy seq and the result of concat
21:15gfrlogthe docs say concat "Returns a lazy seq"
21:16ihodesconcat does, but the seq you're building is a seq (non-lazy) OF lazy seqs
21:16wwmorgangfrlog: this may shed some light on the subject: (if (concat []) 1 2) returns 1, but (if (seq (concat [])) 1 2) returns 2
21:17gfrlog,(concat [])
21:17clojurebot()
21:17gfrlog,(seq (concat []))
21:17clojurebotnil
21:17gfrlogwwmorgan: I think that just made it weirder
21:17gfrlog,(seq '())
21:17clojurebotnil
21:18gfrlog,(seq [])
21:18clojurebotnil
21:18gfrlognow I don't know what seq does
21:18gfrlog(doc seq)
21:18clojurebot"([coll]); Returns a seq on the collection. If the collection is empty, returns nil. (seq nil) returns nil. seq also works on Strings, native Java arrays (of reference types) and any objects that implement Iterable."
21:18gfrloghuh.
21:18ihodesgfrlog look at this - but try it in your own REPl please ;)
21:18gfrlogman I been using clojure for over a year now and apparently I don't understand seqs
21:18ihodeshttp://gist.github.com/546704
21:19wwmorganseq doesn't force lazy collections, but it will force the first element, because it needs to know whether there is "something" (a sequence) or "nothing" (nil)
21:19gfrlogihodes: did you intend the recur error?
21:20ihodeshaha no i didn't - i just whipped it together in textedit. one second
21:21ihodesah dumb me, can't use recur there. not tail position. is that the error? instead, use a function
21:21ihodesand call the function instead of recur
21:21gfrlogyeah, it was bad recur position
21:22gfrlogso we just wanna cons a bunch?
21:22ihodeshttp://gist.github.com/546704
21:22gfrloggee wiz, the (seq) thing totally fixed the error I'd been staring at all day
21:23ihodesgfrlog: maybe look at the top two responses here: http://stackoverflow.com/questions/3247045/how-are-lazy-sequences-implemented-in-clojure/3247417#3247417
21:24gfrlogihodes: thanks, I will
21:25gfrlogor am rather
21:27duncanmihodes: i can only get it to work if i use refer instead of require
21:29ihodescan you send me a gist or something of the errors you're getting/what you're doing?
21:38duncanmihodes: http://gist.github.com/546728
21:44ihodesduncanm: right, because you're not qualifying file. try doing (:require [clojure.java.io :as jio]) instead, and then use jio/file in lieu of file
22:02bmhclojurebot: ping
22:02clojurebotPONG!
22:18msfis there an alternative for read-lines from duck-streams in core ?
22:18defnyou could slurp and then split-with \n
22:19defn(i think)
22:19markgunnelsWe are playing with this concept at work called Dynamic Object Model where you basically read in an Ontology and dynamically generate your object model. It's easy to do in Ruby and I'm trying to pursue it in Clojure with defrecord using a macro.
22:19defnmarkgunnels: cool
22:19markgunnelsWhen I run my macro -
22:19markgunnels(defmacro make-record
22:19markgunnels [name attr1 attr2]
22:19markgunnels '(defrecord ~name [~attr1 ~attr2]))
22:20markgunnels(This is just a play macro but you get the idea)
22:20markgunnelsI run into an error.
22:20markgunnelsDuplicate key: null
22:20markgunnels [Thrown class java.lang.IllegalArgumentException]
22:21markgunnelsAm I doing something spectacularly stupid or are records not meant to be used in this fashion?
22:22markgunnels(dom.core/make-record "FullName" "first-name" "last-name") is how I am calling the above prototype macro.
22:24phaerlisp
22:24phaerah, wrong window
22:24tomojmarkgunnels: try (macroexpand-1 '(make-record ...
22:24defnmarkgunnels: i couldnt answer you with any degree of certainty, im trying to find out why that's the case
22:25msfdefn: is slurp lazy ?
22:25msfI'm working with a relatively large file
22:25defnmsf: i believe so yes
22:25tomojmarkgunnels: using ` instead of ' in your macro will help a bit, but you've still got another problem
22:25ihodesmsf: it is not lazy. use line-seq instead
22:26defnihodes: thanks for that
22:26defnsorry msf
22:26markgunnelstomoj: Thanks. I switched it to a `.
22:26markgunnelsIt gives a new error which I think you were alluding too.
22:26ihodesdefn: no problem, was recently writing stuff around line-seq and stuff
22:26tomojok, now if you macroexpand-1 you should see the remaining problem clearly
22:26defnis the other problem the vector?
22:27tomojthough.. your macro with ` could work just fine, you're just calling it wrong :)
22:27defntomoj: would you mind pasting macroexpand-1 of his macro
22:27markgunnelstomoj: It works if I don't pass in strings.
22:27markgunnels:-p
22:27tomojyep
22:28markgunnelstomoj: Thanks.
22:28tomojdefn: with ` you get (clojure.core/defrecord "FullName" ["first-name" "last-name"])
22:28tomojmarkgunnels: which dev env do you use for clojure?
22:28markgunnelstomoj: Emacs, swank, and leiningen.
22:29markgunnelstomoj: I feel pretty stupid for not spotting the backtick.
22:29markgunnelstomoj: What dev env do you use?
22:29tomojcool, then try putting point right on the first paren in (make-record FullName first-name last-name) and hit C-c RET or C-c C-m
22:29defnthe same :)
22:29tomojsame
22:30defnemacs brotherhood
22:30markgunnelsHeh.
22:30tomojeasier than using macroexpand-1, and it tidies things up too
22:31defntomoj: you're my new hero
22:31defngreat trick
22:31defnthanks
22:31markgunnelstomoj: mine too.
22:31markgunnelsI'll dedicate tomorrow's RDF and Clojure work to you.
22:31markgunnels:-)
22:31defnmarkgunnels: another fun one with SLIME is C-.
22:32defnerrr no
22:32defnM-.
22:32defnif | if your point, (defn|
22:32defnis your point*
22:32defnM-, takes you back to your file
22:32defnIt's a handy way to view core source
22:33markgunnelsVery nice.
22:33markgunnelsWow. That is VERY NICE.
22:33defn:)
22:34defnmarkgunnels: best part? It even works with third party libs
22:34defnpoorly documented libs become much more manageable with that little trick
22:34tomojand it's a stack, so you can dive n levels deep with M-. and then find your way back to whatever you were working on with M-,
22:35defnI didn't even consider that! Even cooler!
22:36markgunnelsdefn: I'm bouncing through clojure.contrib now. :-)
22:36tomojsomeone around here figured out how to get it to jump into clojure's java source too
22:36defntomoj: that I'd be interested to know
22:36defnFor exceptions there's nothing better
22:37tomojI think it was lau
22:37tomoj(well.. lau demoed it in that video, someone else did the figuring out I believe)
22:41defnC-c I
22:41defnbrings up the inspector
22:42tomojafter all this time I still use repl-utils/show for some reason..
22:42defnnoob...
22:42defnkidding of course heh
22:43defni wonder if fuzzy completions work better now
22:43tomojand I still switch to the repl for (doc ..) instead of just C-c C-d C-d'ing
22:43defni disabled it because it only seemed to work for core
22:44defnyeah i use (doc ...) a lot too
22:44defnwhen you're writing code just doing (foo ... starts to become a habit
22:44defni tried to switch from clojure to Ruby the other day for awhile, and it was like.. (def foo^H^H^H^H^H^H^H^Hdef foo
22:45tomoj:)
22:45defnActually I even did it while writing some regular plain text.. "(hey how's it going ..."
22:45defnheh
22:46tomojwait.. what??!!
22:46tomojlooks like slime-who-calls works now!
22:47defnwhat's that
22:47tomojC-c C-w C-c map RET
22:47tomojthen scroll through the list and it shows you each call site... beautiful
22:47defni get evaluation aborted
22:47defnmaybe im doing it wrong
22:48defnhrm
22:48tomojmaybe it's only in swank-clojure 1.3.0-SNAPSHOT
22:48defntomoj: are you connected to a swank session, or are you running slime
22:48defnah yes, i bet that's why
22:49defni was just M-x sliming
22:49defnyep it works
22:49defnthat's awesome
22:51tomojlooks like it was tcrayford's doing
22:54msfhrm
22:54msfwhy would spit be throwing NPE's all of a sudden
23:08arohnerwhen I try C-c C-w C-c map RET, I get Invalid token: ::xml/dtd [Thrown class java.lang.Exception]
23:10arohnerlooks like it also doesn't work on names with dashes in them
23:13ihodestechnomancy: you should have a pull request in your inbox re: :min-lein-version. I'll check out more issues/features tomorrow!
23:43technomancytomoj: yeah, tcrayford is th eman
23:48tomojarohner: hmm, works fine with names with dashes for me
23:49tomojswank-clojure 1.3.0-SNAPSHOT?
23:49tomoj(or at least, it works fine as far as I can tell...)
23:49arohnertomoj: yeah. 1.3.0-snapshot.
23:50arohnertomoj: huh, this time it worked. Last time I tried, it didn't
23:50tomojuh oh
23:52redalastorI'm learning D and it's pretty neat. It got immutability, pure functions, lazyness and lambdas. I wonder why I never heard people wishing for D as a target of Clojure-in-Clojure.
23:53rickmodeI would think JVM alternatives for Clojure-in-Clojure would be other VM's as opposed to other languages
23:54rickmodefor example CLR (getting Clojure to work well on .Net again)
23:54technomancyredalastor: D isn't really a runtime, is it?
23:54redalastorNope
23:56redalastorThey are thinking about AST macros. It's not likely going to be anywhere near lisp macros but it definitely look like one of the most interesting C descendent language that came out since a long while.
23:56technomancysure, but that's not saying much. =)
23:59redalastorI believe the contrary. Java was meant to bring people "half-way to lisp" but the language itself wasn't a big success in that regard.
23:59slyrustomoj: that is nice. thanks for pointing that out!