#clojure logs

2015-08-11

00:23weebzHi, I'm having trouble figuring out how to interop with this Java maven project
00:24weebzThe dependency in my project.clj is [org.seleniumhq.selenium/selenium-java "2.47.0"]
00:24weebzthe import in the example java is import org.openqa.selenium.By;
00:26weebzShoot, I feel silly. I forgot you have to use import for java things, not use/require
01:35ttt_fffhow hard is datomic to implement?
01:35ttt_fffi'm surprised no one has written an open source clone
01:37oddcullyisnt datascript going that route?
02:01Empperittt_fff: how hard is postgres to implement?
02:01Empperihard
02:02Empperidoes that answer your question about datomic?
02:03Empperiand datascript is far far away from what datomic is
02:04Empperiit is very interesting though
02:24wasamasawell, production-quality databases generally are loads of code :D
03:17Empperiindeed
05:03borkdudeI have a dependency problem in this project: https://github.com/borkdude/ring-problem
05:03borkdudeWhen I start the server (see README) I get java.lang.ClassNotFoundException: ring.middleware.not-modified, compiling:(matchmaker/api.clj:52:7)
05:09justin_smithborkdude: I'm confused as to why you would use ring.middleware.not-modified/wrap-not-modified but never require the namespace?
05:09justin_smithhttps://github.com/borkdude/ring-problem/blob/master/src/matchmaker/api.clj#L52
05:10justin_smithisn't the solution to require those namespaces?
05:11borkdudejustin_smith I'll try it
05:11justin_smiththere's another namespace right next to it that is referred to in full, but never required
05:13borkdudejustin_smith sure, that's because I stripped down the code
05:13borkdudeThat's it. Doh. I didn't even think about it, because it worked in a project where I borrowed this code from.
05:13borkdudeThanks!
05:14justin_smith Iguess they were taking advantage of the fact that some other code had required the namespace?
05:23borkdudeI guess so yeah (it was my own code btw)
06:20axski need to call (execute/run/eval?) some forms with side-effects over a vec of values but also want to construct a seq from their results
06:20axskall i could find so far is a combination of doall for do, is there a more idiomatic alternative? should i use loop?
06:22axsk,(def a (doall (for [n (range 2)] (do (println n) n)))))
06:22clojurebot0\n1\n#'sandbox/a
06:22axsk,a
06:22clojurebot(0 1)
06:29opqdonutaxsk: yeah that's about it
06:35spaceplukwhere can I see some complex examples of clojure.pprint?
07:21hellofunkare there any obvious examples I can try at a REPL that show pmap actually performing better than map? i keep trying and cannot find one, using a 2-core machine
07:27oddcullyhellofunk: you are doing something cpu intensive in your fn? otherwise the overhead for pmap eats up the gain in absolute time
07:30oddcully,(time (dorun (pmap (fn [_] (Thread/sleep 100)) (range 42))))
07:30clojurebot#error {\n :cause "no threads please"\n :via\n [{:type java.lang.SecurityException\n :message "no threads please"\n :at [clojurebot.sandbox$enable_security_manager$fn__857 invoke "sandbox.clj" 94]}]\n :trace\n [[clojurebot.sandbox$enable_security_manager$fn__857 invoke "sandbox.clj" 94]\n [clojurebot.sandbox.proxy$java.lang.SecurityManager$Door$f500ea40 checkAccess nil -1]\n [java.lang.Threa...
07:31opqdonutuser=> (time (let [fact (fn [n] (apply *' (rest (range (inc (bigint n))))))] (doall (pmap fact (range 1000 1010))) :done)
07:31opqdonut)
07:31opqdonutwith normal map I get
07:31opqdonut"Elapsed time: 12.591812 msecs"
07:31opqdonut"Elapsed time: 6.443782 msecs"
07:31opqdonut:done
07:32opqdonuthad to jump through too many hoops not to get overflow :P
08:27noncomif i have a fully qualified var like #'some.namespace/some-var - then how do i get its name? "some-var" in this example..
08:27noncom?
08:29opqdonutI don't think there's a nice way to do that
08:29opqdonutyou could always go via (str #'some.namespace/some-var) of course
08:57noncomis it feasible to create a swing gui with seesaw that is reconstructed on each application frame, as per the functional paradigm approach?
08:58noncomi want to make the changes to the gui instantly visible
08:58noncomor is swing too heavy for that? or maybe there'll be some issues with threading..?
08:59tsdhnoncom: Concerning your first question: (:name (meta #'my-ns/my-var)) => my-var
08:59tdammersmy gut feeling says you'd need to do some transparent diffing somehow
08:59tdammersdon't think swing components are built for rapid setup and teardown
09:01noncomthanks, guys!
09:37crocketWhen java 9 comes with Jigsaw, will clojure support jigsaw?
10:09noncomcan anyone give an example, how do i subscribe a checkbox to conj or disj a values to a (atom #{}) ?
10:09noncomi cannot make seesaw accept my binding... it does not fit the protocol..
10:10noncomlike No implementation of method: :subscribe of protocol: #'seesaw.bind/Bindable found for class: seesaw.core.proxy$javax.swing.JCheckBox$Tag$fd407141
10:14talvdavhi, anyone using eclipse with german keyboard layout and can tell me how to type a frickn >@< ??!!!!
10:14oddcullyisn't that on alt-gr-q?
10:15crocketHow do people load databases into maps and vectors without ORM in clojure?
10:15crocketThere's table join
10:16noncomi've managed with my query, no probelm
10:17talvdavoddcully: yeah it is, but it dose not work
10:19oddcullyderef then ;)
10:19talvdavoddcully: :/ or alt+64 on numblock....
10:19talvdavit also dose not work on mac...
10:20oddcullyat very first glance there is no compose-key-version
10:20oddcullywell on osx the "programmer"-keys are even more wierd on de-layout
10:20talvdavyeah i know
10:20oddcullywent US a longlong time ago and never looked back
10:21talvdavi got used to it really quick
10:21talvdavwell i can't change my keyboard every time a key combo dose not work...
10:21talvdavi can't find any solution on the web
10:21talvdavit semse im the only one with this damn problem
10:21oddcullywell i dont know how it's nowadays. but at G4 times you just "had to know". not even on third party keyboards
10:25kavkazWhattup clojure fam
10:42tmtwdI'm not sure why this example from joy of clojure works : http://pastebin.com/utLrPcR1
10:42tmtwdanyone can explain?
10:43scriptortmtwd: line 5 in particular, I'm guessing?
10:43tmtwdscriptor, exactly it doesn't make sense !
10:43oddcully~destructuring
10:43clojurebotdestructuring is http://clojure.org/special_forms#binding-forms
10:44scriptortmtwd: look into clojure destructuring, it lets you extract data from more complex structures
10:45tmtwdwell --- the example is from a chapter about destructuring .... :)
10:46scriptorsure, but joy of clojure isn't really meant for beginners
10:46scriptorso if you aren't already familiar with destructuring, those chapters aren't very helpful, honestly
10:48tmtwdfair enough
10:49socksytmtwd: map destructuring looks like {symbol-to-refer-to-keyword :keyword-in-map}, and if you put `:as blah` at the end, then you can refer to the map in general as blah
10:50tmtwdoh so in this example it just ignored the 'symbol-to-refer-to-keyword' part?
10:51socksyyes
10:52socksyalso, since most people just do {keyword1 :keyword1 keyword2 :keyword2 :as map-name}, this is usually shortened to {:keys [keyword1 keyword2] :as map-name}, but I'm sure that comes up too
10:53socksythis is a good blog post explaining it: http://blog.jayfields.com/2010/07/clojure-destructuring.html
10:53tmtwdsocksy, but in my example we still need a 'symbol-to-refer-to-keyword' even if we ignore it?
10:54tmtwdsocksy, thanks I'll check it out
10:54socksytmtwd: nope, don't need it if you don't refer to it
11:22timvisherall of our services are configured to run nrepl/http listeners directly from main. as such, we don't use lein ring or anything and just use `lein run`. our vms right now are getting crippled with all the memory baggage. do i loose anything important by going to `lein trampoline run`?
11:41binjuredhow can i get result# in this macro to reference the correct gensym in the let, instead of creating a new one?
11:42binjured(defmacro t [first-only]
11:42binjured `(fn [stuff]
11:42binjured (let [result# (some-func stuff)]
11:42binjured ~(if first-only (first `result#)))))
11:48broma0anyone familiar with the library Korma?
11:48puredangerbinjured: get rid of the second `
11:48binjuredpuredanger: yeah, tried that, "Unable to resolve symbol result#"
11:49binjuredso far the only way i've managed is to write a function that just takes result# and ~first-only and does the `if` in there.
11:51chouserbinjured: You'll probably need to use gensym instead of a symbol#
11:51binjuredchouser: for res?
11:51justin_smithbinjured: also you need it to be (fn [stuff#] ...)
11:52justin_smith,(defmacro t [f] `(fn [c#] (let [result# (range c#)] (when ~f (last result#)))))
11:52clojurebot#'sandbox/t
11:52chouseractually, there's a deeper problem
11:52justin_smith,((t true) 8)
11:52clojurebot7
11:52justin_smith,((t fals) 22)
11:52clojurebot#error {\n :cause "Unable to resolve symbol: fals in this context"\n :via\n [{:type clojure.lang.Compiler$CompilerException\n :message "java.lang.RuntimeException: Unable to resolve symbol: fals in this context, compiling:(NO_SOURCE_PATH:0:0)"\n :at [clojure.lang.Compiler analyze "Compiler.java" 6704]}\n {:type java.lang.RuntimeException\n :message "Unable to resolve symbol: fals in this co...
11:52justin_smith,((t false) 22)
11:52clojurebotnil
11:52binjuredyeah, my bad, the actual version has stuff#
11:53justin_smiththat's essentially the macro you want
11:53justin_smithbinjured: the thing is you only want to apply ~ to first-only
11:53chouserbinjured: You can't, at compile time, call 'first' on the value of result, which will only *have* a value later at runtime
11:53justin_smiththe rest should not be unquoted, it is generated inside the ` form
11:54binjuredjustin_smith: yeah, now i have (if ~first-only (first res#) res#) so it ends up looking like (if false ...) or (if true ...) rather than evaluating that at macro time, which seems less ideal but has the advantage of working.
11:54justin_smithchouser: I had assumed some-func was to be called at macro compile time
11:54justin_smithchouser: it would make sense that way
11:54justin_smithbinjured: first-only is evaluated at macro time
11:55binjuredyeah, not the `if` though
11:55chouserjustin_smith: maybe I'm confused. Your example is calling range at runtime
11:55justin_smithbinjured: sure it is, all the stuff in that if body is known at macro time
11:55justin_smithchouser: oh, wait...
11:55binjuredthe macro expands to (if false (clojure.core/first res__67032__auto__) res__67032__auto__)
11:56chouser...and some-func's parameter isn't known until runtime either, so I'm quite confused.
11:56chouserah
11:56chouserok, first is to be run at runtime.
11:56binjuredyes sorry to confuse
11:56chouserso, yeah, gensym.
11:57binjuredif first-only is passed to the macro, i want the macro to emit (in part) (first <gensym var>) otherwise emit <gensym var>
11:58chouserright
11:59chouser(defmacro ... (let [result-sym (gensym)] `(let [~result-sym ...] ~(if ... `(first ~result-sym)))))
12:00chouserfoo# is nice, but unfortunately can't always be used.
12:02binjuredperfect, thanks! is it just because # is ambiguous given the scoping and evaluation?
12:02binjuredi guess i couldn't move `result-sym` up to the macro let without using gensym
12:19chouserusages of foo# in different syntax-quote blocks generate different symbols. I suppose this is to keep them from capturing each other accidentally.
12:20chouser...but it means that if you want the same generated symbol in more than one syntax-quote, you can't use foo# and have to reach for good ol' gensym instead.
12:53{blake}Wow, Java 8 is all about the lambdas.
12:54arrdemLambdas, invokeDynamic and other cool invokeDynamic enabled things :D
12:55{blake}I feel like FP is winning the argument. =P
12:56{blake}It's ridiculous how much Java code gets less boiler-plate-y.
12:56arrdemhttps://i.imgur.com/Sex1E8m.jpg
12:57LicenserIs updating clojurescript prjects incredible daangerous or am I doing things horriby wrong?
12:57{blake}arrdem: lol
12:57{blake}Licenser: Yes?
12:58Licenserevery time I try to do any kind of update it ends up with some impossible to treck errors like 'Can't dynamically bind non-dynamic var: clojure.tools.reader/resolve-symbol' (or others)
12:58{blake}What is "any kind of update"?
12:59LicenserI'm slowly creeping up but I think it's mostly related to nrepl/piggyback things like that but the error doens't give much hint
13:02LicenserI did the big mistake to run a ancient upgrade w/o picking a single thing to update so I'm going through it one by one now trying to find out what exactly caused it but boy is that no fun :P
13:03{blake}Licenser: I concur! In my case, the error only turns up in deployment under Wildfly.
13:03{blake}Track THAT down.
13:03Licenseroh boy
13:04{blake}But normally, I find updates pretty painless. The Clojure 1.7 upgrade was quite nice.
13:04Licenserthey are great if they work
13:04Licenserbut with the whole cljs stuff there are so much things that depend on eachother and don't strictly enforce versions that argh it just keeps killing me
13:04Licenserwell partially perhaps because I have no idea what I'm doing ;)
13:05{blake}I hate it when that's the problem.
13:05Licenseryup
13:05{blake}Although, at least there's a solution. You're experiencing that solution right now. =P
13:05dnolenLicenser: fwiw trying to use cider/piggieback is a common source of frustration, but also easily avoided
13:06Licenseroh that is good to know
13:06dnolensome people have succeeded in getting that working in but in 8 months, it's not many people
13:07Licenseroh wow that doens't sound incredibly great
13:07{blake}dnolen is sort of the Batman of #clojure.
13:07dnolenI don't use Emacs for serious Clojure work anymore (all in with Cursive).
13:07Licenser:D
13:07dnolenI spent a good amount of time making sure that inf-clojure coul work with Emacs
13:07LicenserI love my emacs so but I'm happy without a reply for cljs stuff, reload is good enough for me really
13:07dnolener inf-clojure could work with ClojureScript
13:07{blake}dnolen: Word to cfleming!
13:08dnolenso if you want a decent ClojureScript + Emacs setup, save yourself some time and use that
13:09Licenserright now all I want is my code to compile ;) the rest doesn't matter I've given up on REPLs already
13:10LicenserI got started with chestnut but it turns out that was a bit of a mistake, too much magic to easy not to undertand what happens
13:10{blake}Licenser: Yeah! I had the same experience with it.
13:10dnolenLicenser: you're taking all the wrong turns :)
13:10dnolenRead the Quick Start, figure out how ClojureScript actually works
13:11Licenserdnolen that is one thing I am really really really really good at that
13:11dnolenthen decide what other indirection you want to add to your setup
13:11Licenseryea as it so often goes the "need something wokring now" got the better of me, not proud of it
13:12Licenserchestnut was convinient I could easiley add a proxy middleware and just have all my API calls go through the same webserver w/o worrying about reloading the page, or adding nginx or something
13:12dnolenyeah I don't know about any of these things really
13:12dnolenI don't use them
13:13LicenserI don't know about any of those things either that is a big problem I think
13:28hellofunkwhat is the best swap! strategy to push an item on a vector in something like (atom {:a []}) .. using assoc with conj feels awkward but perhaps best way?
13:29opqdonutassoc-in
13:29justin_smith hellofunk (swap! a update-in [:a] conj x)
13:29opqdonutis meant precisely for that
13:29hellofunkopqdonut: assoc-in would require knowledge of the index
13:29justin_smithopqdonut: assoc-in would just whipe out the vector
13:29hellofunkno?
13:29clojurebotno is tufflax: there was a question somewhere in there, the answer
13:29opqdonutok right, update-in
13:30hellofunkcool thanks fellas
13:30justin_smithI like the way that swap! plus update-in leads to something like backward forth code
13:31bjawould ((apply juxt fns)) or (map #(%) fns) be preferable?
13:31clojurebotGabh mo leithscéal?
13:31opqdonutbja: I'd prefer the latter
13:31justin_smithbja: depends if you want it to be lazy I guess
13:32bjait's about to be fed into (apply merge), so laziness doesn't matter
13:34puredangerin 1.7+ you can use update instead of update-in here
13:35puredanger(swap! a update :a conj x)
13:37Licenserdnolen {blake} the issue was, funny enough leiningen, 2.5.2 seems to have issue with org.clojure/tools.render
13:38{blake}huh
13:38Licenseryup
13:38Licenserthe moment I set leiningen to 2.5.2 it causes problems
13:39depIs there any consensus on the best way to do password resets for a website?
13:39depI'm currently eyeing encrypted tokens with Buddy-auth.
13:45justin_smithLicenser: do you mean org.clojure/tools.reader?
13:46devnWhoa, it's Licenser
13:46devnLong time no see
13:46Licenserdevn indeed :)
13:46Licenserjustin_smith yea sorry
13:46Licenserdevn I've spend most of my time in erlang land
13:46justin_smithLicenser: for a moment I wondered if there was an official clojure raytracer or something
13:47justin_smithLicenser: care to bring up your tools.reader issue on #leiningen?
13:47dnolenLicenser: more complications, why I just as often as not use Maven directly
13:47Licenserhaha
13:48Licenserjustin_smith sure but I'm not exactly sure what it is so I might not be incredibly much help
13:48Licenserbut let me get things actually working first before breaking them agian :P
13:49justin_smithI've got 99 problems and most of them are tooling
13:49sverijustin_smith: +100^1000000000000
13:51devnshortest way to generate an OOME in clojure?
13:52devnlike a legitimate one, not just throwing
13:52Licenserdevn limit the memory of your JVM to 129k? (crazy idea)
13:52opqdonutyeah e.g. with ulimit
13:52justin_smith(iterate #(repeat 1000 %) [])
13:53devnjustin_smith: good golfin' bud
13:53justin_smiththat should run out of memory pretty fast :)
13:53justin_smithoh, and put a doall on it if you aren't in the repl
13:53opqdonutprobably not that fast: it has sharing
13:53justin_smith(the repl will trigger via printing)
13:53opqdonutright?
13:53justin_smithahh
13:53justin_smithyeah, the sharing
13:54opqdonut#(doall (concat (repeat 1000 %))) or something might work better
13:54justin_smithopqdonut: but that doesn't recursively get bigger
13:54opqdonutI think it'll still get big enough
13:55opqdonutthe size grows 1000-fold every step after all
13:55justin_smithoh yeah, but put the doall on the iterate, now I see what you mean
13:57devnthanks
13:58opqdonutnow that I'm testing it, I can't run out of memory
13:58opqdonutprobably gets gc'd too fast
13:59justin_smithopqdonut: I'm getting a really tight loop, like not even responding to control-c
13:59opqdonutno, the mem usage does grow but pretty slowly
13:59opqdonutjustin_smith: same here
13:59devnmaybe ill just make `dd` a 2GB file and read it
13:59devnor something like that
13:59justin_smithdevn: I was being to clever
13:59opqdonutthis seems easiest:
14:00opqdonutuser=> (double-array 1000000000)
14:00opqdonutOutOfMemoryError Java heap space clojure.lang.Numbers.double_array (Numbers.java:1154)
14:00justin_smithyeah, that's the way to do it
14:00justin_smithmuch simpler, less "clever", actually works
14:00devnjustin we've known eachother on irc long enough for me to recognize you're almost always being clever :D
14:00justin_smithhahaha
14:00devnopqdonut: nice
14:00justin_smithdevn: it's a vice of mine
14:01devni have another question for y'all
14:01devnKafkaStream can be turned into an iterator, .hasNext will block while waiting for new elements, and .next, well, does what you'd expect
14:02bjasends it to the closest NeXT workstation?
14:03devnnow, what i need is something that works the same way (can be cast to an iterator, has the same behavior on hasNext and next)
14:03devnbja: hahahaha
14:03devn(inc bja)
14:03lazybot⇒ 3
14:03justin_smithdevn: huh? I thought .hasNext was a predicate, and .next blocked
14:04devnyou might be right, i guess i didn't check that, if im being honest
14:04devnbtw bja, we're a borland shop
14:05chouserdevn: Maybe java.util.concurrent.ArrayBlockingQueue ?
14:05amalloyjustin_smith: what if the sequence doesn't know whether it has a next?
14:05amalloy,(empty? (filter neg? (range)))
14:05clojureboteval service is offline
14:05devnheh
14:05justin_smithamalloy: hasNext iirc tells you whether something is ready right now, not whether something will ever be ready
14:05amalloyno
14:06justin_smithno? OK
14:06amalloythat's .available() for streams
14:06devnchouser: hmm, does that really have the properties i described? I could swar I tried that out last night.
14:06justin_smithahh
14:06chouserdevn: Depends on how precise you're being, I think.
14:06kwladykawhy clojure.pprint/pprint works only in REPL?
14:06amalloyhasNext is java's equivalent to (complement empty?) in any useful way i can think of
14:06justin_smithkwladyka: it works wherever, maybe you forgot to require it/
14:07chouserABQ can't be cast to an iterator, but it has an .iterator method that returns one.
14:07devnchouser: maybe i could...reify iterator?
14:07devnam i crazy?
14:07kwladykajustin_smith, hmm
14:07devnwait don't answer that.
14:07chouseryou could totally reify iterator
14:07kwladykajustin_smith, ech my mistake
14:08chouserThis is to mock kafka for testing?
14:08devnyeah, there's a "transceiver" that takes messages in, processes them in a go block, fires them off to the next topic
14:09kwladykajustin_smith, is too hot :)
14:09amalloyyou are talking about mocking an Iterable. that's just a seq. here is an iterable mock: (list 1 2 3). or, (cons 1 (lazy-seq (Thread/sleep 5000 (cons 2 nil)))), if you want one that takes time for whatever reason
14:13devnamalloy: we were talking about this yesterday, and I tried to make (doseq [x (seq the-KafkaStream)] ...) work, but it didn't
14:14devnim willing to give it another go with loop/recur. I understand what you're saying, just have been unable to get it to work. Probably an error on my part.
14:16devnjustin_smith: also, as far as i can tell it is hasNext that does the blocking
14:16devnFWIW
14:16justin_smithdevn: yeah, I was wrong
14:17justin_smithdevn: github is being amazingly and frustratingly slow right now, but I am pulling up an open source part of my current project that might be applicable to what you are doing
14:18justin_smithdevn: some of this (which has worked great for a while) might be helpful? https://github.com/littlebird/conduit/blob/master/src/conduit/adapter/kafka.clj#L142
14:19justin_smithdevn: in particular that I need to use the .iterator method on the stream to get an iterator from it
14:19devnisn't conduit the old jim duey lib?
14:20devnoh nevermind
14:20justin_smithdevn: I might have unknowingly reused a name
14:20devnjustin_smith: yeah, this might be interesting
14:20devnright now i have a stuartsierra component that creates a consumer and a stream, with a go-loop inside of it that handles messages
14:21justin_smithdevn: conduit is an abstraction I use in my current product that can wrap either a websocket stream or a kafka topic, with plans to also wrap vanilla core.async and http
14:21justin_smithbut the meat of the kafka part might be useful for you I think
14:21devnjustin_smith: so, this is basically identical to what im doing
14:21justin_smithhaha, nice!
14:21justin_smithwell mine is open source ;P
14:21devn:D
14:22devnso here's the thing though
14:22justin_smithit's still young though, but it's been working great for a while
14:22justin_smithyes?
14:22devnwrite a test that your kafka wrapper works :D
14:22justin_smithoh yeah I need to make tests
14:22devnbecause that's the thing i was finding some pain in doing
14:22devni don't want to test with a running ZK and Kafk
14:22devnKafka*
14:22justin_smithahh
14:23justin_smithyeah, then it's a question of what level of abstraction to test at etc.
14:23devnand it's kind of a pain to mock it out in a way that feels close to useful
14:23justin_smithdevn: first you need a lib that replicates the kafka api in process, with integration tests that prove (against a running kafka and zk) that its behavior is identical
14:24justin_smiththen you can write unit tests by swapping in that lib :)
14:24devnmaybe the sad answer is that i should just set up CI with its own kafka/zk and then if people don't want to run those locally, they can just see how they fare in their PR
14:24devnor VM it
14:24justin_smithdevn: our team is just going to bite the bullet and we are all going to have kafka running on our dev machines, because it is going to be so vital to the architecture
14:25devnactually, we *have* a VM
14:25devnbut configuration matters!
14:25devnwhich is why i was talking to you yesterday about forcing kafka to flush batched messages and such
14:26justin_smithyup, I am putting the config file into the repo, gonna have people symlink it
14:26devnbecause when you test 100 go in and 100 come out, and you're using test.check so your payloads are of variable sizes
14:26justin_smithright
14:26devnand there's batching...
14:26devnit's just a colossal pain in the ass to test TBQH
14:27devnjustin_smith: but, i should back up a bit and say im still really new to kafka
14:27devnand there is a *lot* to understand
14:28devnthere are also a lot of bugs, and that doesn't help
14:28devnlike, we ran into a bug that's been around for something like 2 years that deals with auto topic creation and topic.offset "largest"
14:29devni must have spent an entire day trying to figure that out, kind of inexcusable. but, again, arguing against myself: I loaded it up with 500,000 messages and wow, this thing screams.
15:07amalloydevn: what i'm saying is, if coll is Iterable but not Seqable (as most java/scala collections are), then (seq coll) just dispatches to (iterator-seq (.iterator coll)) internally anyway. see clojure.lang.RT/seqFrom, and compare to clojure.core/iterator-seq
15:08amalloyif you think the two are behaving differently then something *else* is going wrong and waving iterator-seq over it is just a distraction from whatever the real problem is
15:11arrdemTIL about #^"" hints
15:11chouserarrdem: sorry you had to do that. :-(
15:11amalloyarrdem: well it's no different from ^"" hints
15:12arrdemamalloy: didn't know that was a thing either... manually specifying the internal name of an object array has sapped some serious will for the day
15:12arrdemclearly we need a type inference engine that knows about into-array :P
15:12amalloyarrdem: quick, hack the compiler to add support for ^(array (array int))
15:13arrdemamalloy: probably a more useful undertaking than what I'm on right now
15:13arrdemrelated: http://dev.clojure.org/jira/browse/CLJ-440
15:14justin_smitharrdem: the issue of course is that [Lmy.whatever.YOLO; isn't something the reader is going to handle elegantly
15:14justin_smithfor at least two glaring reasons
15:16amalloyjustin_smith: of course it's clear why some solution is needed. it's just too bad that specifying the internal name yourself in a string is the one that was chosen, instead of eg what i recommended, or even (defn foo [^[[long]] xs] ...)
15:16amalloywell s/recommended/made up just now/
15:16justin_smithhaha
15:16justin_smithyeah, a psuedo-syntax using vectors would be a nice way to do it
15:17arrdemmy primary cow is that neither ^"" nor #^"" is mentioned here.. http://clojure.org/java_interop I found an old dnolen ml post from '09 where someone used #^"" and went "hope this still works".
15:17amalloyhaha
15:20arrdemI was otherwise about to hack the compiler to add support for ^ tag fns so that the tag of into-array could be either Object[] or the array of the first parameter :P
15:20arrdemwhich I was not pleased by
15:27expezfipp seems to be able to pprint clojure code now, in addition to EDN, anyone have any thoughts on it compares with say pprint+cljfmt?
15:27Licenserdnolen {blake} thanks for your help mates, I ended up ripping out all the reply stuff and it makes it a lot easyer
15:27SeyleriusAnyone taken a look at clojure-scheme? It seems to be a potentially useful path to binary.
15:28dnolenLicenser: there's also a #clojurescript specific channel these days :)
15:28Licenseroh I dind't knew that, I feel so outdated
15:29expezdnolen: is this still the gold standard for testing if macro expansion is done in cljs context? https://github.com/Prismatic/schema/blob/40f4d623299ada0b99fc381b2bc724d6902602ae/src/clj/schema/macros.clj#L10-L19
15:29arrdemexpez: I'd go with cljfmt just becuase it is actively maintained for the purpose of printing code whereas fipp just can.
15:29arrdemSeylerius: it's outdated by a fair bit last I checked.
15:30arrdemSeylerius: what do you want/need a binary for? I claim that you probably don't actually have a performance justification therefor, and if you want startup time there's Skummet
15:30dnolenexpez: you can do it however you like, I don't recommend any particular way
15:32expezdnolen: Ok, let me rephrase a little, does that solution rely on what you'd consider implementation details?
15:32dnolenSeylerius: at this point Planck probably has the most realistic future for good C integration
15:33dnolenexpez: most definitely does, it that breaks we won't care
15:33expezdnolen: any chance of adding something for this purpose to core?
15:33dnolenexpez: nothing specific to ClojureScript
15:34dnolenif anything happens it will be like conditional reading and will apply at least to the three main Clojure dialects
15:35expezdnolen: sorry, I meant clojurescript core :)
15:35dnolenexpez: and I said "no" + details
15:41dnolenhttp://facebook.github.io/react/blog/2015/08/11/relay-technical-preview.html
15:45devnamalloy: ahhhh, i see what you're saying now. miscommunication. i didn't realize you were saying that (seq coll) would wind up being (iterator-seq (.iterator coll)) anyway
15:45devnso yes, i cannot use (seq coll) because of the chunking
15:50justin_smitharrdem: I think Seylerius was talking about wanting to do systems programming in a lispy
15:51Seyleriusarrdem: Outdated could be irritating. And no, it's not a performance concern. It's more of a desire to be able to use Clojure's style and data structures on a system level. Scheme can operate at that level, and you could write stuff to run on bare metal in it, but I don't /like/ scheme as much.
15:51Seyleriusjustin_smith is right.
15:52SeyleriusIt's true: once you really start learning a lisp, it starts to spoil you for other languages somewhat...
15:53scriptoreh, I liked using C in my OS class, in a masochistic sort of way
15:53Seyleriusscriptor: Exactly.
15:53SeyleriusTo say nothing of C++.
15:53SeyleriusMasochism indeed.
15:53scriptorthe masochism bit was mostly tongue-in-cheek
15:54scriptorbut just like fp can change how you think about programming
15:54scriptorI feel that C and low-level programming in general has a similar effect
15:55justin_smithSeylerius: lush has an "interesting" take on doing low level programming in a lispy syntax. It lets you compile functions directly to assembly within the repl
15:55justin_smithSeylerius: but it is no scheme to be sure, not to mention being no clj
15:55tmtwdhow can I change the directory of cider so I can load local files?
15:55SeyleriusThanks.
15:55justin_smithSeylerius: the main crux I think is gc
15:56justin_smithSeylerius: you'll want to be able to program at a level where the gc is not available yet / is not a reasonable choice (eg. interrupts, bootstrap) and that leads to a hella ugly programming style in a lisp
15:57SeyleriusThat makes sense.
16:04justin_smithSeylerius: lush is a semi-reasonable old-school lisp with the design goal of being able to do that kind of non-gc code
16:06arrdemalso inactive... last release was in '09
16:08justin_smithyeah, sorry, didn't mean to represent it as current, just apropos to the whole low level systems programming in lisp thing, if I knew something as well developed but more recent I would cite that instead
16:08arrdemamalloy: https://www.refheap.com/108106
16:09arrdemhalfbaked but should support ^[[[[t]]]]
16:09arrdembecause reasons
16:09justin_smitharrdem: oh, nested? nice
16:10amalloynewInstance.getClass seems like a weird way to do it, but sure
16:11arrdemwell so you need to reflectively make a new aray of an arbitrary class and get whatever the name for that class is
16:11arrdemor you can string format the "[" weirdness
16:12amalloyi'm a little surprised that there's no method like: Class arrayOf(Class c)
16:12arrdemI didn't see one but that doesn't mean it doesn't exist.
16:12amalloyyeah, i don't see one either
16:13amalloyinterestingly the first promising google hit i found was by our own puredanger: https://dzone.com/articles/obtaining-array-class-java-ref (not at all helpful for this situation, as it happens)
16:14puredangersomething here perhaps: http://docs.oracle.com/javase/7/docs/api/java/lang/reflect/Array.html
16:15arrdempuredanger: that's exactly what I used :P
16:15amalloyyeah, it looks like that is the best you can do
16:15amalloyit's just really weird to me that in order to get from a Class object to another related Class object you have to instantiate an instance of one of them
16:15arrdemI mean there's nothing stopping you lifting that into static class arrayOf(class)
16:15amalloysure
16:17arrdemThat patch should also probably have a length check so that ^[t] is OK but ^[t ...] isn't
16:18sdegutis,(nth #{:a :b :c} 1)
16:18clojurebot#error {\n :cause "nth not supported on this type: PersistentHashSet"\n :via\n [{:type java.lang.UnsupportedOperationException\n :message "nth not supported on this type: PersistentHashSet"\n :at [clojure.lang.RT nthFrom "RT.java" 881]}]\n :trace\n [[clojure.lang.RT nthFrom "RT.java" 881]\n [clojure.lang.RT nth "RT.java" 847]\n [sandbox$eval25 invokeStatic "NO_SOURCE_FILE" 0]\n [sandbox$eva...
16:18sdegutisThat should work right?
16:18sdegutis,(rand-nth #{:a :b :c})
16:18clojurebot#error {\n :cause "nth not supported on this type: PersistentHashSet"\n :via\n [{:type java.lang.UnsupportedOperationException\n :message "nth not supported on this type: PersistentHashSet"\n :at [clojure.lang.RT nthFrom "RT.java" 881]}]\n :trace\n [[clojure.lang.RT nthFrom "RT.java" 881]\n [clojure.lang.RT nth "RT.java" 847]\n [clojure.core$rand_nth invokeStatic "core.clj" 6933]\n [clojure...
16:18sdegutis,(first (shuffle #{:a :b :c}))
16:18clojurebot:a
16:19sdegutisStupid. rand-nth should work here.
16:20gfredericks,(rand-nth (seq #{:a :b :c]))
16:20clojurebot#<RuntimeException java.lang.RuntimeException: Unmatched delimiter: ]>
16:20gfredericks,(rand-nth (seq #{:a :b :c}))
16:20clojurebot:c
16:20sdegutis,(seq #{:a :b :c})
16:20clojurebot(:c :b :a)
16:20sdegutisI spose.
16:20sdegutisStill stupid.
16:20sdegutis,(seq #{:a :b :c})
16:20clojurebot(:c :b :a)
16:20sdegutisHaha seq isn't so random after all.
16:20puredangerrand-nth uses *nth* which required indexed collection, which set is not
16:20gfredericksI don't think it's meant to be nondeterministic
16:21puredangerit is definitely not random
16:21gfrederickswithin a single process
16:21amalloygfredericks: incidentally it's kinda a bummer that rand-nth does two traversals over collections which are neither counted nor indexed
16:21amalloywhen there is an algorithm available that avoids that
16:21gfredericksamalloy: an algorithm that uses a lot more randomness?
16:22amalloywell, i think in theory it's the same amount of entropy, but in practice since you only get entropy out 64 bits at a time or whatever, it will use more
16:22hiredmanhaving a uniform prob of picking an element without knowing the count seems like it would be tricky
16:23amalloyhiredman: it turns out it's actually not
16:23sdegutisamalloy: So first/shuffle is efficienter?
16:23gfredericksI was thinking amalloy was thinking of (->> coll (map (juxt identity rand)) (apply max-key second) (first))
16:23arrdemamalloy: how so? that doesn't seem trivially possible to me.
16:25hiredmanyou could take one of those streaming reservoir sampling things and use it with a reservoir of size one
16:26hiredmanwhich would be pretty similar to that I guess
16:26sdegutisHi amalloy.
16:26amalloyone sec while i double-check that i've implemented the algorithm right
16:26justin_smith&(reductions (fn [t [i c]] (+ (* t (/ (dec c) c)) (* i (/ 1.0 c)))) 0 (map list [10 9 8 7 6 5] (map inc (range)))) ; running average
16:26lazybot⇒ (0 10.0 9.5 9.0 8.5 8.0 7.499999999999999)
16:27hiredmanwell, I was thinking of weight random sampling, which doesn't make sense in this context (the weighted bit) and without the weighted bit it is almost exactly what gfredericks said
16:29amalloyhttps://www.refheap.com/9b79f6941abe4011529157842
16:30kwladykahiredman, i am not in the topic but maybe http://clojuredocs.org/clojure.core/random-sample ? you said something about random sampling
16:30amalloyyou walk through the collection, keeping track of the count (N) so far and a candidate return value. at each step, you choose whether to replace your candidate with the current item, with probability 1/N
16:30amalloyand at the end you return whatever candidate
16:30arrdemamalloy: nice!
16:31amalloythis avoids the multiple traversal, and thus also holding the head of the seq. so you can (random-choose (range 1e7)) if you really want
16:32justin_smithamalloy: I wish I had the mathematical / algorithmic vocabulary to describe what my running average example above and this random-choose have in common
16:33justin_smithrandom-choose is actually much more interesting
16:33amalloyjustin_smith: that's a running average impl, right, not anything related to rand-nth?
16:33arrdema proof of correctness for the one pass select would be really fun.
16:33amalloyarrdem: i'm sure it's been done. it's not like i invented the algorith
16:33justin_smithamalloy: right, but the common thing is that they both take an unknown number of inputs and can give you an answer at any point
16:33amalloyyou can apply the same thing to a weighted choice, btw, by tracking the sum of all weights seen, instead of the count
16:33amalloyjustin_smith: online algorithm?
16:34justin_smithaha! I think that is it
16:34justin_smith(inc amalloy)
16:34lazybot⇒ 291
16:34arrdem(inc amalloy)
16:34lazybot⇒ 292
16:35kwladykammm i should do something like (+ 100 justin_smith) ;)
16:35amalloyi admit i don't totally get why my implementation is the one that works - i really expected it to start with (next coll), not (seq coll)
16:39arrdemamalloy: because you need to with certainty select the 0th element.
16:40arrdemOkay I have the proof of this :D
16:41arrdemif you consider the probability P that any element E is the _final_ selected value, that probability is the product of the telescoping series of the probabilities of not selecting any subsequent element which reduces to 1 over the final cardinality of the seq
16:41sdegutisamalloy: hi
16:42sdegutisYou know, I go back and forth on whether nerd-sniping is unethical.
16:42sdegutisJust a single little obscure snippet of Clojure code, and you guys are often gone digging into Clojure's internals and experimenting for like 20 minutes.
16:43sdegutisI suppose it would come in handy whenever I need a diversion. But is it ethical? Probably not.
16:43justin_smitharrdem: aha, so it contains a similar hidden taylor series to the running average (though more artfully hidden)
16:43gfrederickssdegutis: that sounds pretty condescending
16:43sdegutisThe statistics don't lie my friend.
16:44gfredericksokay
16:44puredangerI just to joke that the best way to kill an emacs user's productivity was to suggest that there was something that couldn't be done with emacs
16:44puredangers/just/used/
16:45bjaI promised my manager that I wouldn't adopt spacemacs until after our next product ships.
16:47amalloythe version supporting weighted choice: https://www.refheap.com/ca84e5cc573053b38d23a439c
16:47sdegutispuredanger: oddly enough, sniping doesn't work in #emacs
16:47sdegutispuredanger: they're like "meh it's possible, go figure it out"
16:48puredangerthat's a good first approx answer to most things :)
16:49hiredmanISample
16:49arrdemhttps://twitter.com/arrdem/status/631205747334885376 yep telescoping series proof
16:57arrdemyou could also use this same technique to maintain a buffer of N randomly selected elements...
16:57justin_smitharrdem: so for example maintaining a representative sample as a form of profiling?
16:58arrdemjustin_smith: that was the idea
16:58justin_smithreminds me of that notorious mongodb code that used Math.random to decide whether to log or not
16:58scriptorjustin_smith: you're kidding, right
16:58amalloyscriptor: one time in 10
16:58justin_smithscriptor: not at all, this is a thing
16:59amalloybecause otherwise there is too much logging
16:59justin_smithI mean sampling of state is a useful thing for profiling, but mixing that up with logging was misguided
16:59justin_smithand that is not a thing they do in the current code
17:01arrdemhum... so I'm actually not sure how you can do the selection buffer thing without endangering the telescoping series property...
17:05amalloyi'm not sure the selection buffer problem is well-defined enough yet
17:08arrdemhttp://arthurdejong.org/rl/ does exactly this it seams...
17:10amalloyit's easier if the items don't have weights, as in that example
17:10justin_smitharrdem: haha, I started reading and looking at the examples and I was like "oh, I could use this where I use shuf now", then I read closer and saw the part where the author was like "lol, abandoned, use shuf instead"
17:11arrdemyeah shuf also appears to do exactly this
17:11arrdemeh I'm not curious enough to clone down and read coreutils today
17:11justin_smitharrdem: fun game 'shuf -n 100 /usr/share/dict words'
17:12justin_smithor -n 10 even
17:12justin_smiththen look up the weird ones, of course
17:12justin_smithshuf -n 10 /usr/share/dict/words # typoville
17:13amalloyjustin_smith: the hidden gem in my dictionary appears to be turbinatostipitate
17:14amalloywhich i can't even pronounce
17:14justin_smithand google for it is very unhelpful too
17:15amalloyyeah i think it may not be a real word
17:15justin_smithwtf words? I trusted that file!
17:16justin_smithamalloy: I bet all the usages of that word on the web are from spambots who used the same dict file
17:16arrdemat some point I need to put my aspell dictionary under git control with the rest of my dotfiles because I expect new machines not to try and correct "clojure" or "conj" :P
17:17amalloyit looks that way, yes
17:18amalloyjustin_smith: you got my hopes up that i was going to learn a cool new word, not get turned into a spambot
17:18justin_smiththen again, if it was eg. the most rare word of all time, and it was in /usr/share/dict/words and bots were picking words at random from that file, you would expect to see millions of spambot usages of the word for every actual non-spambot usage
17:18justin_smithamalloy: so sorry
17:18justin_smithamalloy: that shuf command has gotten me gems though, I swear, and in no time this exact chat log will be the top google hit for turbinatostipitate
17:19justin_smithamalloy: like schizogenic, just now, that's a word straight out of a ztellman talk
17:20arrdemdon't say that too loudly he'll get ideas
17:21justin_smithoh cool, the "schizogenic double bind" is a concept from bateson, systems ftw
17:43seangrove,(for [[i n] (map-indexed vector [:a :b :c])] [i n])
17:43clojurebot([0 :a] [1 :b] [2 :c])
17:43seangroveAny better way to do that?
17:44justin_smith,(map list (range) [:a :b :c])
17:44clojurebot((0 :a) (1 :b) (2 :c))
17:44seangrovejustin_smith: Constrained to the `for` form ;)
17:45TEttinger,(for [c (map-indexed vector [:a :b :c])] c)
17:45clojurebot([0 :a] [1 :b] [2 :c])
17:45justin_smith,(for [[i n] (map list (range) [:a :b :c])] [i n])
17:45clojurebot([0 :a] [1 :b] [2 :c])
17:45TEttinger,(for [c (map vector (range) [:a :b :c])] c)
17:45clojurebot([0 :a] [1 :b] [2 :c])
17:45seangroveJust trying to get an index in my for form
17:54justin_smith(apply > (map count "map-indexed vector" "map list range"))
17:54justin_smith,(apply > (map count "map-indexed vector" "map list range"))
17:54mistaBIZis there a way to do that with a str?
17:54clojurebot#error {\n :cause "Wrong number of args (2) passed to: core/count"\n :via\n [{:type clojure.lang.ArityException\n :message "Wrong number of args (2) passed to: core/count"\n :at [clojure.lang.AFn throwArity "AFn.java" 429]}]\n :trace\n [[clojure.lang.AFn throwArity "AFn.java" 429]\n [clojure.lang.AFn invoke "AFn.java" 36]\n [clojure.core$map$fn__4545 invoke "core.clj" 2632]\n [clojure.lang....
17:54justin_smithergh
17:54justin_smith,(apply > (map count ["map-indexed vector" "map list range"]))
17:54clojurebottrue
17:57mistaBIZso range just counts elements int hat arg?
17:58justin_smithmistaBIZ: yeah, it's just a way to attach an incrementing number to each item
17:58justin_smith,(map list (range) [:a :b :c :d])
17:58clojurebot((0 :a) (1 :b) (2 :c) (3 :d))
17:58mistaBIZyou damn ninja
17:58justin_smithmistaBIZ: each call to list gets one item from each arg
17:59mistaBIZis there a way to sew a str to inc like that
17:59mistaBIZ*get
18:00justin_smithit's trickier, are you thinking like [a b c d e] ?
18:00mistaBIZmore like "hi' to [[0 1]
18:00justin_smithso each letter is numbered?
18:00mistaBIZor map into keys either way
18:01mistaBIZor keyed
18:01justin_smith,(map list (range) "hello")
18:01clojurebot((0 \h) (1 \e) (2 \l) (3 \l) (4 \o))
18:01johnnyrichardhow to create an typed java hashtable from clojure hash?
18:01justin_smithjohnnyrichard: if it's a generic, don't even bother with the types, it's a compiler fiction
18:01justin_smithjohnnyrichard: generics don't actually exist when you leave javac
18:02johnnyrichardjustin_smith: thanks man
18:02justin_smithnp
18:04johnnyrichardabout hateoas, do you know some library? (cannonical library)
18:04justin_smithoh man I almost did a hateoas project once...
18:06johnnyrichardjustin_smith: how do you make hypemedia for your apis today?
18:06justin_smithjohnnyrichard: I'm not sure if there's anything really useful, but liberator might be a good starting place
18:06justin_smithjohnnyrichard: I don't, not working on a publically accessible api atm
18:07amalloyi would support mmore cannonical libraries, in general. cannons are fun
18:07justin_smithand we have a small team / small app and the internal stuff is at a really ad-hoc stage
18:07justin_smithamalloy: trebuchet is taken, but those are super fun too
18:08justin_smithor wait, was I thinking of slingshot?
18:13hiredmanmy libraries are all ex cathedra
18:20TEttingerhiredman: all my libs need a catheter
18:23mistaBIZto help them relieve themselves into a vector?
18:23mistaBIZ=n
18:27dbaschI just learned that if you are using the cookie store in ring’s session middleware and don’t provide an encryption key (which is the default), you might as well be storing the sessions in memory because the encryption key disappears when the server restarts
18:31justin_smithdbasch: better than a default key of "hunter2" I guess
18:33dbaschjustin_smith: but worse than refusing to function if you don’t provide a key
18:34justin_smithdbasch: yeah, I guess that's where optimizing for "just run lein new ring my-app and it works" ends up hurting other priorities
18:35dbaschif it were a default that would be fine, but this is the wrap-session middleware that you had to include, and you had to tell it to use the cookie store
18:36dbaschit would be perfectly ok for it to throw a “key missing” exception
18:36justin_smithoh so this isn't an it-just-works issue
18:36dbaschhttps://ring-clojure.github.io/ring/ring.middleware.session.cookie.html
18:37amalloydbasch: well, it's not exactly the same as just storing the keys in memory. if you have large numbers of keys and a small server footprint, it could be a substantial improvement to make clients store their own cookies
18:37dbaschif you have a large number of keys I’m sure it’s important that they survive a server restart
18:46TEttingermistaBIZ: no, because at the very least half of them doesn't work at any given time
20:00devnIs it possible to sneak around the fact that you can't pass `loop` as a value (since it's a macro) in this situation: `(with-redefs [my-loop loop] ...)`
20:08amalloydevn: in your ideal universe, where this sneak-around exists, what would the above snippet do? it's not at all clear to me what redefining something to be "loop" should mean
20:28devnamalloy: instead of running with a core.async/go-loop i was going to swap in loop for it
20:30amalloyso that is even more fundamentally unworking because go-loop is a macro, and you can't exactly redefine a macro after the code that's using it has already been compiled
20:31devnderp
20:47justin_smithnew feature idea: time travel macros
20:48TEttingerI need your clothes, your boots, and your motorcycle
20:48devnHeh
21:59binjuredis anybody aware of a way with prismatic/schema to construct nested records? if i define a record where one of the fields is another record type, map->Foo doesn't create an instance of the nested record.
22:00justin_smithbinjured: map->Foo isn't part of prismatic/schema, that's a normal clojure record constructor, and clojure doesn't enforce the types of record fields at all
22:01binjuredjustin_smith: ok, well, strict-map->Foo then (AFAIK schema replaces the default map->Foo)
22:01justin_smithso you are using an alternate defrecord defined by prismatic/schema ?
22:02justin_smithbecause the definition of map->Foo is a side effect of defrecord
22:02binjuredyes
22:02binjured(schema.core/defrecord MyRecord [bar :- BarRecord])
22:06justin_smithbinjured: it looks like strict-map->Foo doesn't have any feature to auto-initialize fields https://github.com/Prismatic/schema/blob/master/src/clj/schema/macros.clj#L329
22:10binjuredjustin_smith: bummer.
22:12binjurednever could figure out how to implement Schema in a record in a way that would be any use there, either. i'd basically need to walk a record after it's created which is dumb.
22:12justin_smithisn't schema more about verifying the structure of things at runtime?
22:13binjuredyeah
23:45emperorcezarDoes #clojure-beginners get much traffic?