From freeakk@REDACTED Sat Oct 1 10:48:55 2011 From: freeakk@REDACTED (Michael Uvarov) Date: Sat, 1 Oct 2011 12:48:55 +0400 Subject: [erlang-questions] Testing of nifs Message-ID: Hello, How to test NIFs? I need to run different vms for each test case. Is there a test framework which allows this? -- Best regards, Uvarov Michael From norton@REDACTED Sat Oct 1 16:57:56 2011 From: norton@REDACTED (Joseph Norton) Date: Sat, 1 Oct 2011 23:57:56 +0900 Subject: [erlang-questions] Testing of nifs In-Reply-To: References: Message-ID: <1A9FDBB9-0C31-4EBB-B813-D9B1B8084C04@lovely.email.ne.jp> I'm not aware of any test frameworks for doing this sort of thing automatically. The stdlib's slave application is one (easy) way to implement your own framework. Here is a recipe for starting, restarting, and stopping slave nodes. https://github.com/norton/qc/blob/dev/src/qc_slave.erl Your test case(s) can then use the kernel's rpc application to invoke tests and/or commands on slave node(s). regards, Joe N. Joseph Norton norton@REDACTED On Oct 1, 2011, at 5:48 PM, Michael Uvarov wrote: > Hello, > > How to test NIFs? > I need to run different vms for each test case. Is there a test > framework which allows this? > > -- > Best regards, > Uvarov Michael > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From james@REDACTED Sat Oct 1 19:00:43 2011 From: james@REDACTED (James Aimonetti) Date: Sat, 1 Oct 2011 10:00:43 -0700 Subject: [erlang-questions] dialyzer -Wunderspecs and the binary() type In-Reply-To: <4E8564B2.8090304@cs.ntua.gr> References: <4E84B876.8040207@2600hz.com> <4E8564B2.8090304@cs.ntua.gr> Message-ID: <4E87473B.2090801@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Thanks for the clarification Kostis. Always nice to deepen my understanding of how to use Dialyzer more effectively. James On 09/29/2011 11:41 PM, Kostis Sagonas wrote: > On 09/29/11 21:27, James Aimonetti wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> We use -Wunderspecs a lot in our Dialyzing and the only minor complaint >> I have is when I get warnings that the success typing is more specfic >> with the binary() type. >> >> We see a lot of: >> >> foo.erl:123: Type specfication foo:a_fun(BinVal) -> 'ok' when >> is_subtype(BinVal, binary()) is a supertype of the success typing: >> foo:a_fun(<<_:8,_:_*8>>) -> 'ok'. >> >> In foo.erl, a_fun/1 is defined as: >> >> - -spec a_fun(BinVal) -> 'ok' when >> BinVal :: binary(). >> a_fun(BinVal) -> >> %% do something with BinVal >> ok. >> >> This is obviously a simplified, trivial example. > > It's not just a simplified example. It's an example that is incomplete > because it does not show the problem. Here is one that does: > > -module(foo). > -export([a_fun/1]). > > -spec a_fun(binary()) -> 'ok'. > a_fun(B) -> > do(B), ok. > > do(<<_>>) -> ok; > do(<<_,R/binary>>) -> do(R). > > Note that the first clause of do/1 has a non-empty binary there (there > is an underscore). From this clause dialyzer will infer that the do/1 > function only returns if its argument is a non-empty binary of at least > one byte. > >> Just curious if its >> possible to have Dialyzer *not* output the -Wunderspecs warnings for >> binaries? > > Why on earth would we want to special case this? > >> Or if there's a better way to specify binary inputs/outputs >> (since not all of our functions using binary() in the spec have this >> issue). > > As I explained, the warning is related to the fact that these functions > only accept non-empty binaries. (Whether this is intentional or not in > your code I will let you decide.) To suppress these warnings I > recommend the following: define a non-empty binary type > > -type ne_binary() :: <<_:8,_:_*8>>. > > and change the spec of your a_fun above to: > > -spec a_fun(ne_binary()) -> 'ok'. > > (Aside: I do not see the need to use 'when' for type variables that only > appear once in a spec. The non when form is shorter and nicer.) > > Kostis > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOh0c7AAoJENc77s1OYoGgrUcH/26BfA8xBLQ95H4541KazZ7M i5mHR4qFItD0+L1P6x/SfoOr0KFkAhoggyawUmh6pHki7l8Jq1Y5VvRDSOrJR2uU 4Ob18pENuQonvgwv7jxZLKANl67Q58oglAfJVh6FGq+4C5ksIXsqKtNl5PGOSyCE E1AAGkMzmsJqGEBSBiujPM0IZF/9JCSfLI6FRu/LP2gAY8pijJRaJLYX0Ubv3sbU 0cRB+UIR4pteWiytAdsGoSlaOpmm4yrBNfIyBFMqaR7B+ki3fv+VP5RmiFcqb16q ujVWz0xID/iU92nZm3nT+DGmPzUkTiNaZ11N6p76yzLc9mtka4JNfunfIicw/Js= =jXxw -----END PGP SIGNATURE----- From michal.niec@REDACTED Sat Oct 1 19:39:55 2011 From: michal.niec@REDACTED (Michal Niec) Date: Sat, 1 Oct 2011 19:39:55 +0200 Subject: [erlang-questions] wx on lion with erlang-solutions version? In-Reply-To: References: Message-ID: <7D610427-E2ED-4A49-892A-FC0D6A8FCF49@erlang-solutions.com> Hi, the package contains 64bit version of Erlang for Mac OS X. wx module is not supported in this configuration. From my experience WxWidgets are a bit troublesome on Lion. Compilation can be done by downloading the sources and executing configure script like this arch_flags="-arch i386" ../configure CFLAGS="$arch_flags" CXXFLAGS="$arch_flags" CPPFLAGS="$arch_flags" LDFLAGS="$arch_flags" OBJCFLAGS="$arch_flags" OBJCXXFLAGS="$arch_flags" --enable-unicode --disable-shared --with-opengl --with-macosx-version-min=10.6 --with-macosx-sdk=/Developer/SDKs/MacOSX10.6.sdk You can try to install 32bit R14B03 package for Snow Leopard and then install wx widgets using command above. Regs, Michal Niec On 30 wrz 2011, at 20:45, Wes James wrote: > On this site: > > http://www.erlang-solutions.com/section/132/erlang-otp-packages > > it has different erlang distros available. It mentions wx not being > available on windows, but it doesn't say that for OS X Lion. I > installed the Lion version and tried wx:demo(), but it failed with a > no driver error. I didn't think there was a wx for Lion, yet. Does > someone else have wx working on Lion? If so, how did you get it > installed? > > Thanks, > > -wes > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From lambdadmitry@REDACTED Sat Oct 1 21:41:12 2011 From: lambdadmitry@REDACTED (Dmitry Groshev) Date: Sat, 1 Oct 2011 12:41:12 -0700 (PDT) Subject: [erlang-questions] Erlounge or informal EUG meeting in Moscow In-Reply-To: References: Message-ID: <23847274.44.1317498072859.JavaMail.geo-discussion-forums@yqmw31> Thanks to all participants of this event! This was really awesome :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From lambdadmitry@REDACTED Sat Oct 1 21:45:17 2011 From: lambdadmitry@REDACTED (Dmitry Groshev) Date: Sat, 1 Oct 2011 12:45:17 -0700 (PDT) Subject: [erlang-questions] Testing of nifs In-Reply-To: References: Message-ID: <7501178.53.1317498317476.JavaMail.geo-discussion-forums@yqmw31> I had pretty awesome experience with PropEr on testing NIFs, so I can recommend it to you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sat Oct 1 22:23:26 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 2 Oct 2011 00:23:26 +0400 Subject: [erlang-questions] Erlounge or informal EUG meeting in Moscow In-Reply-To: <23847274.44.1317498072859.JavaMail.geo-discussion-forums@yqmw31> References: <23847274.44.1317498072859.JavaMail.geo-discussion-forums@yqmw31> Message-ID: Yes! Really cool. Meeting next on Highload++ conference on monday, 3th October From robert.virding@REDACTED Sat Oct 1 23:34:08 2011 From: robert.virding@REDACTED (Robert Virding) Date: Sun, 02 Oct 2011 01:34:08 +0400 Subject: [erlang-questions] Erlounge or informal EUG meeting in Moscow Message-ID: Thank you for a great time this afternoon and evening. It was really appreciated. Robert From my Nexus Max Lapshin wrote: >Yes! >Really cool. Meeting next on Highload++ conference on monday, 3th October >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions From lemenkov@REDACTED Sun Oct 2 08:49:41 2011 From: lemenkov@REDACTED (Peter Lemenkov) Date: Sun, 2 Oct 2011 10:49:41 +0400 Subject: [erlang-questions] Erlounge or informal EUG meeting in Moscow In-Reply-To: References: Message-ID: 2011/10/2 Robert Virding : > Thank you for a great time this afternoon and evening. It was really appreciated. Yeah, that was great! Thanks to all participants for making this event so special. -- With best regards, Peter Lemenkov. From tony@REDACTED Sun Oct 2 12:07:25 2011 From: tony@REDACTED (Tony Rogvall) Date: Sun, 2 Oct 2011 12:07:25 +0200 Subject: [erlang-questions] Erlang ThumbEE Message-ID: Hi list! Anybody knows if there is work being done using ThumbEE with Erlang ? /Tony "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Sun Oct 2 12:18:26 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Sun, 2 Oct 2011 11:18:26 +0100 Subject: [erlang-questions] Testing of nifs In-Reply-To: <7501178.53.1317498317476.JavaMail.geo-discussion-forums@yqmw31> References: <7501178.53.1317498317476.JavaMail.geo-discussion-forums@yqmw31> Message-ID: On 1 October 2011 20:45, Dmitry Groshev wrote: > I had pretty awesome experience with PropEr on testing NIFs, so I can > recommend it to you. > > Would you be willing to share some of your experiences, as I'm certain there are a great many of us that would like to know more about good ways of testing NIFs and drivers! What in particular was good about PropEr for example? I've been thinking that PropEr's state machine testing support should be good for testing linked-in drivers, especially when the various calls to it are tucked away in a gen_server. Cheers, Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: From emailregaccount@REDACTED Sun Oct 2 23:10:31 2011 From: emailregaccount@REDACTED (John Smith) Date: Sun, 2 Oct 2011 14:10:31 -0700 (PDT) Subject: [erlang-questions] Keeping massive concurrency when interfacing with C Message-ID: <579cb1b4-538e-4ad0-864e-cda3086d20aa@t11g2000yqk.googlegroups.com> Hi everyone, >From my understanding, there are four main ways to interface Erlang with C: * C Node * Port * Linked-In Driver * NIF (Native Implemented Function) My problem is if I have, for example, spawned 20,000 Erlang processes and I want them all to execute concurrently but they need to call C code, how can I have that C code run concurrently without having to spawn 20,000 threads in C (which would probably crash the OS) or using obscene amounts of memory? I've been reading over the examples of a C node, and it seems if 20,000 processes all send a message to the C node, the node will process them one-by-one and not concurrently, so it becomes a serialized bottleneck. Spawning 20,000 C nodes on a single machine isn't feasible, because of the amount of memory that would require. A port suffers from the same problem, since the Erlang processes would be communicating with a single external program, and again, I can't create 20,000 instances of that program. Reading the documentation for a linked-in driver, it says: http://www.erlang.org/doc/tutorial/c_portdriver.html "Just as with a port program, the port communicates with a Erlang process. All communication goes through one Erlang process that is the connected process of the port driver. Terminating this process closes the port driver." But on the driver documentation page: http://www.erlang.org/doc/man/erl_driver.html "A driver is a library with a set of function that the emulator calls, in response to Erlang functions and message sending. There may be multiple instances of a driver, each instance is connected to an Erlang port. Every port has a port owner process. Communication with the port is normally done through the port owner process." So this also seems to have the same problem as C nodes and ports, since in order to maintain concurrency I would need 20,000 instances of the same driver. Finally, we have NIFs. These have potential, but when I read the documentation: http://www.erlang.org/doc/man/erl_nif.html "Avoid doing lengthy work in NIF calls as that may degrade the responsiveness of the VM. NIFs are called directly by the same scheduler thread that executed the calling Erlang code. The calling scheduler will thus be blocked from doing any other work until the NIF returns." So if one Erlang process calls a NIF, does this mean the other 19,999 processes are blocked until the NIF returns (or the subset of processes a scheduler manages)? If so, this won't work either. Does anyone have a solution to this that still allows you to use C (I'm using C for the parts that are intensive number crunching)? Or will I have to implement everything in Erlang? Thanks! From lambdadmitry@REDACTED Sun Oct 2 23:27:10 2011 From: lambdadmitry@REDACTED (Dmitry Groshev) Date: Sun, 2 Oct 2011 14:27:10 -0700 (PDT) Subject: [erlang-questions] Testing of nifs In-Reply-To: References: <7501178.53.1317498317476.JavaMail.geo-discussion-forums@yqmw31> Message-ID: <10774339.2082.1317590830225.JavaMail.geo-discussion-forums@yqgn17> Well, it's about hidden details. Erlang is pretty straightforward language; C is a lot more tricky to make right. So there is a lot of non obvious bugs that I can't foresee when I write unit tests; PropEr's *random* tests are way better in this sense, because you just describe the domain of your task, not particular cases. You can also test your NIF incrementally, writing property after property (or command after command if there is some state involved). Speaking about my experience ? I'm using PropEr to test this library https://github.com/band115/ecirca , that implements mutable circular array in Erlang (all calls to this array are constrained to it's creator, so I hope this mutability can't be a problem). There are no docs right now other than code; I will definitely fix it in a week or two when I'll have more time. I've tried to test it with EUnit when it was just plain array; but then I've added different types of array and realized that I should add like 5 times more code to cover all cases, and there was a thought about adding different types of values (int16/32/64), so I realized than EUnit is definitely not the best tool, just because the space of inputs is so multidimensional (and I really wanted to cover as much of it as I can with tests ? Erlang's VM is very fragile to NIFs after all). So I started to dig into PropEr's docs, wrote first commands/checks for statem, compiled a debug version of BEAM, made a habit to attach GDB to the process running tests (this way in case of segfault, and I had a lot of them, I can quickly find where I broke the code) ? and right now I can be sure to some degree (which is proportional to amount of tests that you've asked PropEr to do) that this NIFs won't explode in my face tomorrow. -------------- next part -------------- An HTML attachment was scrubbed... URL: From krab@REDACTED Sun Oct 2 23:28:00 2011 From: krab@REDACTED (Kresten Krab Thorup) Date: Sun, 2 Oct 2011 23:28:00 +0200 Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: <579cb1b4-538e-4ad0-864e-cda3086d20aa@t11g2000yqk.googlegroups.com> References: <579cb1b4-538e-4ad0-864e-cda3086d20aa@t11g2000yqk.googlegroups.com> Message-ID: Hi John. You'd most likely want to implement anything that blocks in Erlang. Typically, Erlang runs as many threads as there are cores (can be controlled with the +S erl argument). The question is, what is it that you want to do in that C code? Because the hardware is not able to do more things concurrently than it has cores.. If you want to do I/O in the c code, then you're probably better off writing it in Erlang, or use one of the existing linked in drivers designed for doing just that. NIFs are best suited to do stuff that requires heavy computation, or something that needs access to a shared data structure, or leveraging some existing code. ...but NIFs should not run for too long since they'll hold up all the other Erlang processes on that scheduler, and NIFs should generally not do I/O. So if you have a large body of existing c code which does I/O, then there's no easy way to combine that with the benefits of erlang's inexpensive processes. Sent from my iPad On Oct 2, 2011, at 23:10, "John Smith" wrote: > Hi everyone, > > From my understanding, there are four main ways to interface Erlang > with C: > > * C Node > * Port > * Linked-In Driver > * NIF (Native Implemented Function) > > My problem is if I have, for example, spawned 20,000 Erlang processes > and I want them all to execute concurrently but they need to call C > code, how can I have that C code run concurrently without having to > spawn 20,000 threads in C (which would probably crash the OS) or using > obscene amounts of memory? > > I've been reading over the examples of a C node, and it seems if > 20,000 processes all send a message to the C node, the node will > process them one-by-one and not concurrently, so it becomes a > serialized bottleneck. Spawning 20,000 C nodes on a single machine > isn't feasible, because of the amount of memory that would require. > > A port suffers from the same problem, since the Erlang processes would > be communicating with a single external program, and again, I can't > create 20,000 instances of that program. > > Reading the documentation for a linked-in driver, it says: > http://www.erlang.org/doc/tutorial/c_portdriver.html > > "Just as with a port program, the port communicates with a Erlang > process. All communication goes through one Erlang process that is the > connected process of the port driver. Terminating this process closes > the port driver." > > But on the driver documentation page: > http://www.erlang.org/doc/man/erl_driver.html > > "A driver is a library with a set of function that the emulator calls, > in response to Erlang functions and message sending. There may be > multiple instances of a driver, each instance is connected to an > Erlang port. Every port has a port owner process. Communication with > the port is normally done through the port owner process." > > So this also seems to have the same problem as C nodes and ports, > since in order to maintain concurrency I would need 20,000 instances > of the same driver. > > Finally, we have NIFs. These have potential, but when I read the > documentation: > http://www.erlang.org/doc/man/erl_nif.html > > "Avoid doing lengthy work in NIF calls as that may degrade the > responsiveness of the VM. NIFs are called directly by the same > scheduler thread that executed the calling Erlang code. The calling > scheduler will thus be blocked from doing any other work until the NIF > returns." > > So if one Erlang process calls a NIF, does this mean the other 19,999 > processes are blocked until the NIF returns (or the subset of > processes a scheduler manages)? If so, this won't work either. > > Does anyone have a solution to this that still allows you to use C > (I'm using C for the parts that are intensive number crunching)? Or > will I have to implement everything in Erlang? > > Thanks! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From emailregaccount@REDACTED Mon Oct 3 03:44:03 2011 From: emailregaccount@REDACTED (John Smith) Date: Sun, 2 Oct 2011 18:44:03 -0700 (PDT) Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: References: <579cb1b4-538e-4ad0-864e-cda3086d20aa@t11g2000yqk.googlegroups.com> Message-ID: <024a4da7-1d79-4c9c-8230-43d3cd38417c@i9g2000yqe.googlegroups.com> Thanks for the reply, Kresten! I definitely would not be doing any disk I/O in the C code. It would be intense number crunching, so it would be CPU (and perhaps memory) bound. Everything I've read states Erlang is not good at number brunching (Cesarini mentions this in his "Erlang Programming" book) so I'm considering writing the code to do that in C. If I call a NIF, only the particular scheduler that manages that Erlang process would be blocked and no other scheduler, right? So for example, if I have a CPU with eight cores, and an Erlang scheduler thread is running on each core, and say the third scheduler is executing an Erlang process that calls a NIF (and so blocks), only that scheduler would be blocked until the NIF finishes executing, correct? I'm debating which solution would be better. Erlang would be slower at number crunching, but is extremely efficient at managing concurrent executing processes, meaning each would gradually make progress every X units of time since they'll all get a turn to execute. But I wonder if having a single process execute NIF code until it finishes (and so all the processes managed by a single scheduler execute serially) would be faster than implementing it all in Erlang and having processes execute concurrently within a single scheduler (albeit the code would be slower to execute). There would be less overhead of Erlang process context switching (although admittedly that isn't much to begin with) and the C code would be faster at number crunching. I suppose there's only one way to find out! :) I was also thinking about writing the number crunching code in some other language than C, such as OCaml. OCaml has a reputation for being as fast as C, yet not nearly as low-level. Maybe that would be a good fit with Erlang. Example benchmarks: http://shootout.alioth.debian.org/u32q/benchmark.php?test=all&lang=hipe&lang2=gpp http://shootout.alioth.debian.org/u32q/benchmark.php?test=all&lang=ocaml The Erlang benchmark was using HiPE as well. Thanks for the suggestion! From kkcmpathi@REDACTED Mon Oct 3 04:35:17 2011 From: kkcmpathi@REDACTED (chamin madura) Date: Mon, 3 Oct 2011 08:05:17 +0530 Subject: [erlang-questions] OSERL ,send short message problem ... Message-ID: Hi all, I can bind test server(it can work smpp ),I used trx mode and my pc(esme),but can't send message to the server or server . I used oserl 3.2.3 and it's doc/example/sample_esme.erl sample_esme:send({sender},{reciever},"my_text"[]). ok how is work? pls,any one can help me? Regards, chamin. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgsmcmlxxv@REDACTED Mon Oct 3 08:32:01 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Mon, 03 Oct 2011 08:32:01 +0200 Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: <024a4da7-1d79-4c9c-8230-43d3cd38417c@i9g2000yqe.googlegroups.com> References: <579cb1b4-538e-4ad0-864e-cda3086d20aa@t11g2000yqk.googlegroups.com> <024a4da7-1d79-4c9c-8230-43d3cd38417c@i9g2000yqe.googlegroups.com> Message-ID: <4E8956E1.6070304@gmail.com> Hi John, There are few things here I may add. As Kresten said, the number of real parallel threads you can run on a computing element depends on the number of cores. Nevertheless, making your code to run in threads may bring an advantage in certain cases. Now, what I missed from your e-mails is how would you like the information from Erlang to be processed. Meaning, is the information processed by an Erlang thread linked to another running thread? Or each thread has its own distinct information? These questions need to be answered before you proceed further in designing your application. In case of linked information, than serializing the data seems not so bad idea (depending of the level of relation in between threads data). Otherwise, in the case of independent data per thread, you don't need to worry about creating 20k threads in C (using NIF's), but just create a dynamic library .so (shared object) which you have to load it before starting your Erlang threads. Linux will take care of the rest (it will create as many data instances within your library as required). You just need to take care that your library to be thread safe (mainly, no memory leaking and not to try to use more memory in the buffer than you have physically). If you wonder if to use Erlang or simple C (or any programming language for that matter), think firstly about what you need in the end. All of us would like to have super-speedy applications by squeezing the maximum the computational power from our hardware, but we all need to make some compromises. What I perceived from using Erlang is that this is not suitable for regular desktop applications, but, instead, it's a very handy tool when developing applications such as non-blocking complex data processing and fast network applications (and may be more, but I used Erlang only for that for the time being). It's not that you cannot obtain all those by writing your applications in C, but why reinventing the wheel when you can just have it? Erlang is robust enough to give you a nice environment for these kinds of applications. Concluding, using Erlang is just a matter of taste and how comfortable you feel yourself with such a programming language. Searching for benchmarks of a programming language doesn't help you too much because they are usually made for certain conditions which, in 90% of the cases, do not fit your needs. In this case, you need high concurrency, I suggest you to consider more cores of lower frequency better than fewer cores of higher frequency (or, if you can afford GPU instead of CPU). Keep in mind that whatever you will choose, you will always be restricted by your hardware and for the few milliseconds you may gain per process you need to work hours if not days. Good luck! Cheers, CGS On 10/03/2011 03:44 AM, John Smith wrote: > Thanks for the reply, Kresten! > > I definitely would not be doing any disk I/O in the C code. It would > be intense number crunching, so it would be CPU (and perhaps memory) > bound. Everything I've read states Erlang is not good at number > brunching (Cesarini mentions this in his "Erlang Programming" book) so > I'm considering writing the code to do that in C. > > If I call a NIF, only the particular scheduler that manages that > Erlang process would be blocked and no other scheduler, right? So for > example, if I have a CPU with eight cores, and an Erlang scheduler > thread is running on each core, and say the third scheduler is > executing an Erlang process that calls a NIF (and so blocks), only > that scheduler would be blocked until the NIF finishes executing, > correct? > > I'm debating which solution would be better. Erlang would be slower at > number crunching, but is extremely efficient at managing concurrent > executing processes, meaning each would gradually make progress every > X units of time since they'll all get a turn to execute. But I wonder > if having a single process execute NIF code until it finishes (and so > all the processes managed by a single scheduler execute serially) > would be faster than implementing it all in Erlang and having > processes execute concurrently within a single scheduler (albeit the > code would be slower to execute). There would be less overhead of > Erlang process context switching (although admittedly that isn't much > to begin with) and the C code would be faster at number crunching. I > suppose there's only one way to find out! :) > > I was also thinking about writing the number crunching code in some > other language than C, such as OCaml. OCaml has a reputation for being > as fast as C, yet not nearly as low-level. Maybe that would be a good > fit with Erlang. > > Example benchmarks: > http://shootout.alioth.debian.org/u32q/benchmark.php?test=all&lang=hipe&lang2=gpp > > http://shootout.alioth.debian.org/u32q/benchmark.php?test=all&lang=ocaml > > The Erlang benchmark was using HiPE as well. > > Thanks for the suggestion! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From barcojie@REDACTED Mon Oct 3 09:28:49 2011 From: barcojie@REDACTED (Barco You) Date: Mon, 3 Oct 2011 15:28:49 +0800 Subject: [erlang-questions] [Erlang-Q] What does Ctrl+G do for the Eshell? Message-ID: Hello Erlangers, When starting a erl shell, we will see a printout as: Eshell V5.8.4 (abort with ^G) then, I tried to press Ctrl+G and got: User switch command --> ls Unknown command --> q(). Unknown command --> I can do nothing with the prompt "-->". How to quit from it? and what's "Ctrl+G" used for? Thank you! Barco From cgsmcmlxxv@REDACTED Mon Oct 3 09:31:52 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Mon, 03 Oct 2011 09:31:52 +0200 Subject: [erlang-questions] [Erlang-Q] What does Ctrl+G do for the Eshell? In-Reply-To: References: Message-ID: <4E8964E8.8080700@gmail.com> Try q instead of q(). You may also want to try h. Cheers, CGS On 10/03/2011 09:28 AM, Barco You wrote: > Hello Erlangers, > > When starting a erl shell, we will see a printout as: > Eshell V5.8.4 (abort with ^G) > > > then, I tried to press Ctrl+G and got: > User switch command > --> ls > Unknown command > --> q(). > Unknown command > --> > > I can do nothing with the prompt "-->". How to quit from it? and > what's "Ctrl+G" used for? > > > Thank you! > Barco > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From spawn.think@REDACTED Mon Oct 3 09:51:30 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Mon, 3 Oct 2011 09:51:30 +0200 Subject: [erlang-questions] [Erlang-Q] What does Ctrl+G do for the Eshell? In-Reply-To: <4E8964E8.8080700@gmail.com> References: <4E8964E8.8080700@gmail.com> Message-ID: Hi Barco, I also recommend to check out http://learnyousomeerlang.com/starting-out On Mon, Oct 3, 2011 at 9:31 AM, CGS wrote: > Try q instead of q(). You may also want to try h. > > Cheers, > CGS > > > > On 10/03/2011 09:28 AM, Barco You wrote: > >> Hello Erlangers, >> >> When starting a erl shell, we will see a printout as: >> Eshell V5.8.4 (abort with ^G) >> >> >> then, I tried to press Ctrl+G and got: >> User switch command >> --> ls >> Unknown command >> --> q(). >> Unknown command >> --> >> >> I can do nothing with the prompt "-->". How to quit from it? and >> what's "Ctrl+G" used for? >> >> >> Thank you! >> Barco >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Mon Oct 3 09:51:54 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 3 Oct 2011 08:51:54 +0100 Subject: [erlang-questions] Testing of nifs In-Reply-To: <10774339.2082.1317590830225.JavaMail.geo-discussion-forums@yqgn17> References: <7501178.53.1317498317476.JavaMail.geo-discussion-forums@yqmw31> <10774339.2082.1317590830225.JavaMail.geo-discussion-forums@yqgn17> Message-ID: On 2 October 2011 22:27, Dmitry Groshev wrote: > Well, it's about hidden details. Erlang is pretty straightforward language; > C is a lot more tricky to make right. So there is a lot of non obvious bugs > that I can't foresee when I write unit tests; PropEr's *random* tests are > way better in this sense, because you just describe the domain of your task, > not particular cases. You can also test your NIF incrementally, writing > property after property (or command after command if there is some state > involved). > Speaking about my experience ? I'm using PropEr to test this library > https://github.com/band115/ecirca , that implements mutable circular array > in Erlang (all calls to this array are constrained to it's creator, so I > hope this mutability can't be a problem). There are no docs right now other > than code; I will definitely fix it in a week or two when I'll have more > time. I've tried to test it with EUnit when it was just plain array; but > then I've added different types of array and realized that I should add like > 5 times more code to cover all cases, and there was a thought about adding > different types of values (int16/32/64), so I realized than EUnit is > definitely not the best tool, just because the space of inputs is so > multidimensional (and I really wanted to cover as much of it as I can with > tests ? Erlang's VM is very fragile to NIFs after all). So I started to dig > into PropEr's docs, wrote first commands/checks for statem, compiled a debug > version of BEAM, made a habit to attach GDB to the process running tests > (this way in case of segfault, and I had a lot of them, I can quickly find > where I broke the code) ? and right now I can be sure to some degree (which > is proportional to amount of tests that you've asked PropEr to do) that this > NIFs won't explode in my face tomorrow. > > That makes perfect sense. I've been struggling in my mind with the best ways to test linked-in driver code, which is a little different. The main reason I reach for a driver is that I don't want to block the VM and will probably use the asyc driver APIs or a custom thread pool. Separating the driver interface code from the work it is meant to do seems useful. I can provide a stub implementation for testing that will, for example, fake running out of memory at certain points or return "unexpected" errors. But most often these drivers are calling into some C/C++ library to do something like run as XSLT transformation or talk to a database, etc. In these cases, stubbing out the implementation isn't enough to know that the API calls I'm make are going to work as invalid XML or nonsensical SQL will just puke in a real scenario. At the moment I'm relying on traditional unit testing of the C code along with static analysis and integration testing using common_test. I guess what I'd really like is something like PULSE (the randomising scheduler employed by QuickCheck) to help me identify where my driver code might be locking up the emulator or whatever, but for open source projects that's not an option. Currently I rely on load tests in a mirror of the target environment*s(, but this isn't really good enough because if I move from an 8 core to a 16 core machine, problems can mysteriously appear that just didn't show up and they're really hard to figure out. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wiener.guy@REDACTED Mon Oct 3 12:25:54 2011 From: wiener.guy@REDACTED (Guy Wiener) Date: Mon, 3 Oct 2011 12:25:54 +0200 Subject: [erlang-questions] Erlang on ARM9? In-Reply-To: <20110927170338.GA3699@corelatus.se> References: <75F7CC13-4F65-4F07-9E9E-2A2C2CF08E73@rogvall.se> <0b58b591-dac9-4b51-8835-b941f5d6229a@knuth> <20110927170338.GA3699@corelatus.se> Message-ID: I've tried running the otp_build script with an xcomp configuration file, but encountered the following problem: the configuration script tries to run the file generated by the cross-compiler and fails, although the --host flag is given. In details: I run "./otp_build configure --xcomp-conf=xcomp/erl-xcomp-arm-linux-gnu.conf" The bootstrap build system is configured properly. When configuring the cross host system, it uses the following command: ./configure --disable-dynamic-ssl-lib --build=arm-unknown-linux-gnu --host=arm-unknown-linux-gnu CC=/opt/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi-gcc CFLAGS=-mcpu=arm926ej-s CXX=/opt/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi-g++ LD=/opt/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi-ld --disable-hipe --cache-file=/dev/null But then it causes the following error: checking whether the C compiler works... configure: error: cannot run C compiled programs. If you meant to cross compile, use `--host'. But the configuration command is given a proper --host option! I attach the cross compilation configuration file. Any advice is warmly welcomed. Thanks, Guy Wiener. On Tue, Sep 27, 2011 at 8:03 PM, Matthias Lang wrote: > On Monday, September 26, Gustav Simonsson wrote: > > > The trapexit guide assumes you have a proper cross compilation toolchain. > > Yes. And, if you want to compile R14B (i.e. the current Erlang > version), it's best to ignore the trap exit guide, it's intended > for R13B03 and earlier. > > (I wrote the trap exit guide for R10B, based on things I'd picked up > since R7 or something. A bit after R13B04 came out, I changed the wiki > page to say "don't bother reading this if you're using R13B04 or > later" near the top. Maybe it's time for me to retire the whole page, > R14B is firmly established.) > > > You can generate a ARM9 toolchain using OpenEmbedded either by following > > one of the guides online for creating a OE toolchain or by configuring OE > > for a ARM9 machine and building some app or system image with it, as that > > will also generate a ARM9 toolchain. > > Another popular (free) toolchain is DENX ELDK > > http://www.denx.de/wiki/DULG/ELDK > > I haven't used OpenEmbedded, so I don't know if it's better or worse; > just mentioning it as another alternative. > > Matt > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: erl-xcomp-arm-linux-gnu.conf Type: application/octet-stream Size: 10758 bytes Desc: not available URL: From cbenac@REDACTED Mon Oct 3 13:32:12 2011 From: cbenac@REDACTED (Clara Benac Earle) Date: Mon, 03 Oct 2011 13:32:12 +0200 Subject: [erlang-questions] [ANN] Madrid Erlounge Message-ID: <4E899D3C.7010708@fi.upm.es> WHEN:*Tuesday 4th of October* from 18:30, talk at 19:00 WHERE:*Aula A10* Escuela Universitaria de Ingenier?a T?cnica Industrial. Ronda de Valencia 3 (metro/cercan?as Embajadores) (http://www.euiti.upm.es/EUITIndustrial/Escuela/Ubicacion) WHO: Samuel Rivas, CTO of Interoud (http://www.interoud.com/) (previoulsy at LambdaStream S.L). WHAT: We've been developing an Erlang based solution for embedded devices to control the interaction between web-based interfaces and video processing and streaming hardware. The result is an Erlang middleware with two interfaces, one to control low level drivers written in C and other to control an embedded browser where GUI programmers can develop apps and user interfaces using standard javascript techniques. The target of the project is a consumer electronics device for home entertainment, as a new step forward in our ongoing plan to have one Erlang VM in each living room. See you there, Clara -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave@REDACTED Mon Oct 3 13:42:05 2011 From: dave@REDACTED (David Goehrig) Date: Mon, 3 Oct 2011 07:42:05 -0400 Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: <579cb1b4-538e-4ad0-864e-cda3086d20aa@t11g2000yqk.googlegroups.com> References: <579cb1b4-538e-4ad0-864e-cda3086d20aa@t11g2000yqk.googlegroups.com> Message-ID: John, When I see a number like 20k processes my mind automatically skips to what is the load per process? Do I need a full CPU for each, or a fractional CPU, or maybe I need 20k cores at peak? If the issue is you have 20k anything that need to be scheduled, you need 20k x ( process time + switching cost) / process. Hardware architecture determines a lot about both. On Linux you can get away with hundreds of native processes on Intel but you may not be able to eek out enough processing time per core to do useful work. If you are running on multicore ARM you can eek out maybe 10 processes per core, before just the switching cost alone kills your CPU. This is why we are seeing a move towards 128 core and 256 core ARM processors. If you need 20k cores and can afford around $32-64k in hardware, there are a couple companies that will have products shipping next year. Generally my preferred solution to this problem is event driven C talking over a socket connection to Erlang. Using kqueue or epoll you can easily handle a few thousand socket connections per core on the C side, and Erlang can easily scale out as a command and control infrastructure. If you are clever using consistent hashing to manage system memory across your nodes and job scheduling can make a handful of cores (24ish) perform like a 20k node cluster. But the specifics of your project and budget will determine if it is even possible :) Dave -=-=- dave@REDACTED -=-=- On Oct 2, 2011, at 5:10 PM, John Smith wrote: > Hi everyone, > > From my understanding, there are four main ways to interface Erlang > with C: > > * C Node > * Port > * Linked-In Driver > * NIF (Native Implemented Function) > > My problem is if I have, for example, spawned 20,000 Erlang processes > and I want them all to execute concurrently but they need to call C > code, how can I have that C code run concurrently without having to > spawn 20,000 threads in C (which would probably crash the OS) or using > obscene amounts of memory? > > I've been reading over the examples of a C node, and it seems if > 20,000 processes all send a message to the C node, the node will > process them one-by-one and not concurrently, so it becomes a > serialized bottleneck. Spawning 20,000 C nodes on a single machine > isn't feasible, because of the amount of memory that would require. > > A port suffers from the same problem, since the Erlang processes would > be communicating with a single external program, and again, I can't > create 20,000 instances of that program. > > Reading the documentation for a linked-in driver, it says: > http://www.erlang.org/doc/tutorial/c_portdriver.html > > "Just as with a port program, the port communicates with a Erlang > process. All communication goes through one Erlang process that is the > connected process of the port driver. Terminating this process closes > the port driver." > > But on the driver documentation page: > http://www.erlang.org/doc/man/erl_driver.html > > "A driver is a library with a set of function that the emulator calls, > in response to Erlang functions and message sending. There may be > multiple instances of a driver, each instance is connected to an > Erlang port. Every port has a port owner process. Communication with > the port is normally done through the port owner process." > > So this also seems to have the same problem as C nodes and ports, > since in order to maintain concurrency I would need 20,000 instances > of the same driver. > > Finally, we have NIFs. These have potential, but when I read the > documentation: > http://www.erlang.org/doc/man/erl_nif.html > > "Avoid doing lengthy work in NIF calls as that may degrade the > responsiveness of the VM. NIFs are called directly by the same > scheduler thread that executed the calling Erlang code. The calling > scheduler will thus be blocked from doing any other work until the NIF > returns." > > So if one Erlang process calls a NIF, does this mean the other 19,999 > processes are blocked until the NIF returns (or the subset of > processes a scheduler manages)? If so, this won't work either. > > Does anyone have a solution to this that still allows you to use C > (I'm using C for the parts that are intensive number crunching)? Or > will I have to implement everything in Erlang? > > Thanks! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From matthias@REDACTED Mon Oct 3 14:33:03 2011 From: matthias@REDACTED (Matthias Lang) Date: Mon, 3 Oct 2011 14:33:03 +0200 Subject: [erlang-questions] Erlang on ARM9? In-Reply-To: References: <75F7CC13-4F65-4F07-9E9E-2A2C2CF08E73@rogvall.se> <0b58b591-dac9-4b51-8835-b941f5d6229a@knuth> <20110927170338.GA3699@corelatus.se> Message-ID: <20111003123303.GA16599@corelatus.se> Hi, It's pretty odd that in your case --build is an ARM. Are you typing those commands on an ARM system or on an x86? Or: I don't know what you're doing but it looks wrong. As a quick way to give you more information, I've put a log which shows exactly what I do to build my ARM version in a clean erlang source tree: http://www.corelatus.com/~matthias/cross_arm.log.gz The only non-comment difference between my ARM xconf file and the MIPS one supplied with R14B03 is: erl_xcomp_host=arm-linux Matt ---------------------------------------------------------------------- On Monday, October 03, Guy Wiener wrote: > I've tried running the otp_build script with an xcomp configuration file, > but encountered the following problem: the configuration script tries to run > the file generated by the cross-compiler and fails, although the --host flag > is given. > > In details: > I run "./otp_build configure > --xcomp-conf=xcomp/erl-xcomp-arm-linux-gnu.conf" > The bootstrap build system is configured properly. When configuring the > cross host system, it uses the following command: > > ./configure --disable-dynamic-ssl-lib --build=arm-unknown-linux-gnu > --host=arm-unknown-linux-gnu > CC=/opt/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi-gcc > CFLAGS=-mcpu=arm926ej-s > CXX=/opt/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi-g++ > LD=/opt/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi-ld --disable-hipe > --cache-file=/dev/null > > But then it causes the following error: > > checking whether the C compiler works... configure: error: cannot run C > compiled programs. > If you meant to cross compile, use `--host'. > > But the configuration command is given a proper --host option! > > I attach the cross compilation configuration file. Any advice is warmly > welcomed. > > Thanks, > Guy Wiener. From gustav.simonsson@REDACTED Mon Oct 3 14:46:30 2011 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Mon, 03 Oct 2011 13:46:30 +0100 (BST) Subject: [erlang-questions] Erlang on ARM9? In-Reply-To: <20111003123303.GA16599@corelatus.se> Message-ID: Unless you are compiling on the ARM platform for the that platform, your --build is most likely wrong, as Matt pointed out. You have erl_xcomp_build=arm-unknown-linux-gnu in your xcomp file, try setting this value to 'guess' and see if that works, that will run erts/autoconf/config.guess to grab the (most likely) correct value for --build. It is also the default value for Erlang xcomp conf files. Regards, Gustav Simonsson ----- Ursprungligt meddelande ----- Fr?n: "Matthias Lang" Till: "Guy Wiener" Kopia: "Erlang Questions" Skickat: m?ndag, 3 okt 2011 14:33:03 ?mne: Re: [erlang-questions] Erlang on ARM9? Hi, It's pretty odd that in your case --build is an ARM. Are you typing those commands on an ARM system or on an x86? Or: I don't know what you're doing but it looks wrong. As a quick way to give you more information, I've put a log which shows exactly what I do to build my ARM version in a clean erlang source tree: http://www.corelatus.com/~matthias/cross_arm.log.gz The only non-comment difference between my ARM xconf file and the MIPS one supplied with R14B03 is: erl_xcomp_host=arm-linux Matt ---------------------------------------------------------------------- On Monday, October 03, Guy Wiener wrote: > I've tried running the otp_build script with an xcomp configuration file, > but encountered the following problem: the configuration script tries to run > the file generated by the cross-compiler and fails, although the --host flag > is given. > > In details: > I run "./otp_build configure > --xcomp-conf=xcomp/erl-xcomp-arm-linux-gnu.conf" > The bootstrap build system is configured properly. When configuring the > cross host system, it uses the following command: > > ./configure --disable-dynamic-ssl-lib --build=arm-unknown-linux-gnu > --host=arm-unknown-linux-gnu > CC=/opt/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi-gcc > CFLAGS=-mcpu=arm926ej-s > CXX=/opt/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi-g++ > LD=/opt/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi-ld --disable-hipe > --cache-file=/dev/null > > But then it causes the following error: > > checking whether the C compiler works... configure: error: cannot run C > compiled programs. > If you meant to cross compile, use `--host'. > > But the configuration command is given a proper --host option! > > I attach the cross compilation configuration file. Any advice is warmly > welcomed. > > Thanks, > Guy Wiener. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From lukas@REDACTED Mon Oct 3 14:46:09 2011 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 3 Oct 2011 14:46:09 +0200 Subject: [erlang-questions] Testing of nifs In-Reply-To: <1A9FDBB9-0C31-4EBB-B813-D9B1B8084C04@lovely.email.ne.jp> References: <1A9FDBB9-0C31-4EBB-B813-D9B1B8084C04@lovely.email.ne.jp> Message-ID: <4E89AE91.5080206@erlang.org> Hi! There are also built-in functions within common_test which allows you to start erlang slaves. See http://www.erlang.org/doc/man/ct_slave.html for details. You could create tests in common_test which in the init_per_testcase function start the slave node and then kills it in the end_per_testcase function. We use this extensively to test common_test and lots of other functionality within Erlang/OTP. Lukas On 01/10/11 16:57, Joseph Norton wrote: > I'm not aware of any test frameworks for doing this sort of thing automatically. > > The stdlib's slave application is one (easy) way to implement your own framework. > > Here is a recipe for starting, restarting, and stopping slave nodes. > > https://github.com/norton/qc/blob/dev/src/qc_slave.erl > > Your test case(s) can then use the kernel's rpc application to invoke tests and/or commands on slave node(s). > > regards, > > Joe N. > > > Joseph Norton > norton@REDACTED > > > > On Oct 1, 2011, at 5:48 PM, Michael Uvarov wrote: > >> Hello, >> >> How to test NIFs? >> I need to run different vms for each test case. Is there a test >> framework which allows this? >> >> -- >> Best regards, >> Uvarov Michael >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From wiener.guy@REDACTED Mon Oct 3 14:53:00 2011 From: wiener.guy@REDACTED (Guy Wiener) Date: Mon, 3 Oct 2011 14:53:00 +0200 Subject: [erlang-questions] Erlang on ARM9? In-Reply-To: References: <20111003123303.GA16599@corelatus.se> Message-ID: Thanks a lot, Gustav and Matthias, now I see the error of my ways :) After changing the built option to default, the configuration proceeds, but get stuck on some missing redundant library. How do I configure the cross-compilation to build only the bare minimum? I don't need auxiliary libs, just basic Erlang. Thanks again, Guy. On Mon, Oct 3, 2011 at 2:46 PM, Gustav Simonsson < gustav.simonsson@REDACTED> wrote: > > Unless you are compiling on the ARM platform for the that platform, > your --build is most likely wrong, as Matt pointed out. > > You have erl_xcomp_build=arm-unknown-linux-gnu in your xcomp file, > try setting this value to 'guess' and see if that works, that will > run erts/autoconf/config.guess to grab the (most likely) correct > value for --build. It is also the default value for Erlang xcomp > conf files. > > Regards, > Gustav Simonsson > > ----- Ursprungligt meddelande ----- > Fr?n: "Matthias Lang" > Till: "Guy Wiener" > Kopia: "Erlang Questions" > Skickat: m?ndag, 3 okt 2011 14:33:03 > ?mne: Re: [erlang-questions] Erlang on ARM9? > > Hi, > > It's pretty odd that in your case --build is an ARM. Are you typing those > commands on an ARM system or on an x86? > > Or: I don't know what you're doing but it looks wrong. > > As a quick way to give you more information, I've put a log which shows > exactly what I do to build my ARM version in a clean erlang source tree: > > http://www.corelatus.com/~matthias/cross_arm.log.gz > > The only non-comment difference between my ARM xconf file and the MIPS > one supplied with R14B03 is: > > erl_xcomp_host=arm-linux > > Matt > > ---------------------------------------------------------------------- > > On Monday, October 03, Guy Wiener wrote: > > > I've tried running the otp_build script with an xcomp configuration file, > > but encountered the following problem: the configuration script tries to > run > > the file generated by the cross-compiler and fails, although the --host > flag > > is given. > > > > In details: > > I run "./otp_build configure > > --xcomp-conf=xcomp/erl-xcomp-arm-linux-gnu.conf" > > The bootstrap build system is configured properly. When configuring the > > cross host system, it uses the following command: > > > > ./configure --disable-dynamic-ssl-lib --build=arm-unknown-linux-gnu > > --host=arm-unknown-linux-gnu > > CC=/opt/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi-gcc > > CFLAGS=-mcpu=arm926ej-s > > CXX=/opt/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi-g++ > > LD=/opt/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi-ld --disable-hipe > > --cache-file=/dev/null > > > > But then it causes the following error: > > > > checking whether the C compiler works... configure: error: cannot run C > > compiled programs. > > If you meant to cross compile, use `--host'. > > > > But the configuration command is given a proper --host option! > > > > I attach the cross compilation configuration file. Any advice is warmly > > welcomed. > > > > Thanks, > > Guy Wiener. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Mon Oct 3 15:19:22 2011 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 3 Oct 2011 15:19:22 +0200 Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: <024a4da7-1d79-4c9c-8230-43d3cd38417c@i9g2000yqe.googlegroups.com> References: <579cb1b4-538e-4ad0-864e-cda3086d20aa@t11g2000yqk.googlegroups.com> <024a4da7-1d79-4c9c-8230-43d3cd38417c@i9g2000yqe.googlegroups.com> Message-ID: On Mon, Oct 3, 2011 at 3:44 AM, John Smith wrote: > Thanks for the reply, Kresten! > > I definitely would not be doing any disk I/O in the C code. It would > be intense number crunching, so it would be CPU (and perhaps memory) > bound. Everything I've read states Erlang is not good at number > brunching (Cesarini mentions this in his "Erlang Programming" book) so This may not be true - I wrote some crypto stuff in Erlang with bignums and it turned out to be faster than some C I had. I guess this was because I could write a more advance algorithm than in C - but I never investigated why. I would expect small fixed types and array type algorithms to be faster in C, but not necessarily bignum computations. Also bear in mind that the C and Erlang will not be solving the same problem In C you might have to protect the code from buffer overflow attacks but In Erlang this would not be necessary. Also Erlang is slower *by design* to allow for code-changes on-the-fly which C cannot do. So saying Erlang is "not good at number crunching" is only a first approximation to the truth ... true for most things, but not a universal truth for which you have to read the small print ... > I'm considering writing the code to do that in C. Just curious - what type of "intense number crunching?" - there are different types of number crunching - things like digital image processing involve identical computations on a grid - so could be done on a GPU - other operations may or may not be suitable to a GPU. If the CPU demands are very-variable upping the number of cores and changing to a Tilera might help. The time and memory properties of the C are also interesting - do the C tasks always take the same time/memory or are they highly variable? This can effect the scheduling strategy - you might get CPU or memory starvation. Although number crunching might be faster in C than Erlang the round-trip times become important if you do relatively little work in C. You might spend more time in communication than the time you save in being faster in C. I'd start by making a pure Erlang solution and then measuring to see where the problems are - Guessing where the time goes is notoriously difficult - even if the the pure Erlang solution is not fast enough the code can provide a useful reference implementation to start with and should be up-and-running quicker than if you start coding NIFS etc. Virtually every time I've had a program that was too slow, and I've guessed where the problem was I've been wrong - so I'd build a reference implementation first then measure - then optimize. Cheers /Joe > If I call a NIF, only the particular scheduler that manages that > Erlang process would be blocked and no other scheduler, right? So for > example, if I have a CPU with eight cores, and an Erlang scheduler > thread is running on each core, and say the third scheduler is > executing an Erlang process that calls a NIF (and so blocks), only > that scheduler would be blocked until the NIF finishes executing, > correct? > > I'm debating which solution would be better. Erlang would be slower at > number crunching, but is extremely efficient at managing concurrent > executing processes, meaning each would gradually make progress every > X units of time since they'll all get a turn to execute. But I wonder > if having a single process execute NIF code until it finishes (and so > all the processes managed by a single scheduler execute serially) > would be faster than implementing it all in Erlang and having > processes execute concurrently within a single scheduler (albeit the > code would be slower to execute). There would be less overhead of > Erlang process context switching (although admittedly that isn't much > to begin with) and the C code would be faster at number crunching. I > suppose there's only one way to find out! :) > > I was also thinking about writing the number crunching code in some > other language than C, such as OCaml. OCaml has a reputation for being > as fast as C, yet not nearly as low-level. Maybe that would be a good > fit with Erlang. > > Example benchmarks: > http://shootout.alioth.debian.org/u32q/benchmark.php?test=all&lang=hipe&lang2=gpp > > http://shootout.alioth.debian.org/u32q/benchmark.php?test=all&lang=ocaml > > The Erlang benchmark was using HiPE as well. > > Thanks for the suggestion! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From gustav@REDACTED Mon Oct 3 15:34:18 2011 From: gustav@REDACTED (Gustav Simonsson) Date: Mon, 3 Oct 2011 15:34:18 +0200 Subject: [erlang-questions] Erlang on ARM9? In-Reply-To: References: <20111003123303.GA16599@corelatus.se> Message-ID: <4E89B9DA.5090608@erlang.org> You can put a file named 'SKIP' in the application directories in the lib directory, which will tell the build to exclude them. Other things you might be interested in doing to reach the bare minimum in terms of disk size for your resulting build: * run arm-none-linux-gnueabi-strip (I think that's the right name given your xcomp conf) on the resulting erts binaries to strip them of debug symbols. * Call http://www.erlang.org/doc/man/beam_lib.html#strip_release-1 on the built beam files * Remove the following directories in the resulting erts directory, and in all resulting application directories: src, include, doc, man, examples. * Compress the Erlang applications into .ez files. * Use fancy gcc options like the -Os optimisation flag Regards, Gustav Simonsson On 03/10/11 14:53, Guy Wiener wrote: > Thanks a lot, Gustav and Matthias, now I see the error of my ways :) > After changing the built option to default, the configuration > proceeds, but get stuck on some missing redundant library. > How do I configure the cross-compilation to build only the bare > minimum? I don't need auxiliary libs, just basic Erlang. > > Thanks again, > Guy. > > On Mon, Oct 3, 2011 at 2:46 PM, Gustav Simonsson > > wrote: > > > Unless you are compiling on the ARM platform for the that platform, > your --build is most likely wrong, as Matt pointed out. > > You have erl_xcomp_build=arm-unknown-linux-gnu in your xcomp file, > try setting this value to 'guess' and see if that works, that will > run erts/autoconf/config.guess to grab the (most likely) correct > value for --build. It is also the default value for Erlang xcomp > conf files. > > Regards, > Gustav Simonsson > > ----- Ursprungligt meddelande ----- > Fr?n: "Matthias Lang" > > Till: "Guy Wiener" > > Kopia: "Erlang Questions" > > Skickat: m?ndag, 3 okt 2011 14:33:03 > ?mne: Re: [erlang-questions] Erlang on ARM9? > > Hi, > > It's pretty odd that in your case --build is an ARM. Are you > typing those > commands on an ARM system or on an x86? > > Or: I don't know what you're doing but it looks wrong. > > As a quick way to give you more information, I've put a log which > shows > exactly what I do to build my ARM version in a clean erlang source > tree: > > http://www.corelatus.com/~matthias/cross_arm.log.gz > > > The only non-comment difference between my ARM xconf file and the MIPS > one supplied with R14B03 is: > > erl_xcomp_host=arm-linux > > Matt > > ---------------------------------------------------------------------- > > On Monday, October 03, Guy Wiener wrote: > > > I've tried running the otp_build script with an xcomp > configuration file, > > but encountered the following problem: the configuration script > tries to run > > the file generated by the cross-compiler and fails, although the > --host flag > > is given. > > > > In details: > > I run "./otp_build configure > > --xcomp-conf=xcomp/erl-xcomp-arm-linux-gnu.conf" > > The bootstrap build system is configured properly. When > configuring the > > cross host system, it uses the following command: > > > > ./configure --disable-dynamic-ssl-lib --build=arm-unknown-linux-gnu > > --host=arm-unknown-linux-gnu > > CC=/opt/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi-gcc > > CFLAGS=-mcpu=arm926ej-s > > CXX=/opt/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi-g++ > > LD=/opt/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi-ld > --disable-hipe > > --cache-file=/dev/null > > > > But then it causes the following error: > > > > checking whether the C compiler works... configure: error: > cannot run C > > compiled programs. > > If you meant to cross compile, use `--host'. > > > > But the configuration command is given a proper --host option! > > > > I attach the cross compilation configuration file. Any advice > is warmly > > welcomed. > > > > Thanks, > > Guy Wiener. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From wiener.guy@REDACTED Mon Oct 3 16:50:26 2011 From: wiener.guy@REDACTED (Guy Wiener) Date: Mon, 3 Oct 2011 16:50:26 +0200 Subject: [erlang-questions] Erlang on ARM9? In-Reply-To: <4E89B9DA.5090608@erlang.org> References: <20111003123303.GA16599@corelatus.se> <4E89B9DA.5090608@erlang.org> Message-ID: My mistake: The build did not get stuck on some library, but erts is dependent on libcurses, which is not a part of the toolchain that I use (Sourcery lite). Can I compile Erlang somehow without this lib? Curses! :) Guy On Mon, Oct 3, 2011 at 3:34 PM, Gustav Simonsson wrote: > ** > > You can put a file named 'SKIP' in the application directories in the lib > directory, which will tell the build > to exclude them. > > Other things you might be interested in doing to reach the bare minimum in > terms of disk size for your > resulting build: > > * run arm-none-linux-gnueabi-strip (I think that's the right name given > your xcomp conf) > on the resulting erts binaries to strip them of debug symbols. > * Call http://www.erlang.org/doc/man/beam_lib.html#strip_release-1 on the > built beam files > * Remove the following directories in the resulting erts directory, and in > all resulting application > directories: src, include, doc, man, examples. > * Compress the Erlang applications into .ez files. > * Use fancy gcc options like the -Os optimisation flag > > Regards, > Gustav Simonsson > > > > > > On 03/10/11 14:53, Guy Wiener wrote: > > Thanks a lot, Gustav and Matthias, now I see the error of my ways :) > After changing the built option to default, the configuration proceeds, but > get stuck on some missing redundant library. > How do I configure the cross-compilation to build only the bare minimum? I > don't need auxiliary libs, just basic Erlang. > > Thanks again, > Guy. > > On Mon, Oct 3, 2011 at 2:46 PM, Gustav Simonsson < > gustav.simonsson@REDACTED> wrote: > >> >> Unless you are compiling on the ARM platform for the that platform, >> your --build is most likely wrong, as Matt pointed out. >> >> You have erl_xcomp_build=arm-unknown-linux-gnu in your xcomp file, >> try setting this value to 'guess' and see if that works, that will >> run erts/autoconf/config.guess to grab the (most likely) correct >> value for --build. It is also the default value for Erlang xcomp >> conf files. >> >> Regards, >> Gustav Simonsson >> >> ----- Ursprungligt meddelande ----- >> Fr?n: "Matthias Lang" >> Till: "Guy Wiener" >> Kopia: "Erlang Questions" >> Skickat: m?ndag, 3 okt 2011 14:33:03 >> ?mne: Re: [erlang-questions] Erlang on ARM9? >> >> Hi, >> >> It's pretty odd that in your case --build is an ARM. Are you typing those >> commands on an ARM system or on an x86? >> >> Or: I don't know what you're doing but it looks wrong. >> >> As a quick way to give you more information, I've put a log which shows >> exactly what I do to build my ARM version in a clean erlang source tree: >> >> http://www.corelatus.com/~matthias/cross_arm.log.gz >> >> The only non-comment difference between my ARM xconf file and the MIPS >> one supplied with R14B03 is: >> >> erl_xcomp_host=arm-linux >> >> Matt >> >> ---------------------------------------------------------------------- >> >> On Monday, October 03, Guy Wiener wrote: >> >> > I've tried running the otp_build script with an xcomp configuration >> file, >> > but encountered the following problem: the configuration script tries to >> run >> > the file generated by the cross-compiler and fails, although the --host >> flag >> > is given. >> > >> > In details: >> > I run "./otp_build configure >> > --xcomp-conf=xcomp/erl-xcomp-arm-linux-gnu.conf" >> > The bootstrap build system is configured properly. When configuring the >> > cross host system, it uses the following command: >> > >> > ./configure --disable-dynamic-ssl-lib --build=arm-unknown-linux-gnu >> > --host=arm-unknown-linux-gnu >> > CC=/opt/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi-gcc >> > CFLAGS=-mcpu=arm926ej-s >> > CXX=/opt/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi-g++ >> > LD=/opt/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi-ld --disable-hipe >> > --cache-file=/dev/null >> > >> > But then it causes the following error: >> > >> > checking whether the C compiler works... configure: error: cannot run C >> > compiled programs. >> > If you meant to cross compile, use `--host'. >> > >> > But the configuration command is given a proper --host option! >> > >> > I attach the cross compilation configuration file. Any advice is warmly >> > welcomed. >> > >> > Thanks, >> > Guy Wiener. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonym@REDACTED Mon Oct 3 18:04:25 2011 From: anthonym@REDACTED (Anthony Molinaro) Date: Mon, 3 Oct 2011 09:04:25 -0700 Subject: [erlang-questions] Los Angeles Erlang Users Meetup 10/5 Message-ID: <20111003160425.GA60911@alumni.caltech.edu> Once again we're having a meetup in Los Angeles. The meeting will be in Pasadena at OpenX again. Please RSVP here http://www.erlang.la/ if you might be able to attend. Cheers, -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro From matthias@REDACTED Mon Oct 3 21:03:52 2011 From: matthias@REDACTED (Matthias Lang) Date: Mon, 3 Oct 2011 21:03:52 +0200 Subject: [erlang-questions] Erlang on ARM9? In-Reply-To: References: <20111003123303.GA16599@corelatus.se> <4E89B9DA.5090608@erlang.org> Message-ID: <20111003190352.GA2709@corelatus.se> On Monday, October 03, Guy Wiener wrote: > My mistake: The build did not get stuck on some library, but erts is > dependent on libcurses, which is not a part of the toolchain that I use > (Sourcery lite). Can I compile Erlang somehow without this lib? Curses! :) Have you tried passing --without-termcap when you run configure? Matt From anders@REDACTED Tue Oct 4 00:13:09 2011 From: anders@REDACTED (Anders Ramsell) Date: Tue, 04 Oct 2011 00:13:09 +0200 Subject: [erlang-questions] [Erlang-Q] What does Ctrl+G do for the Eshell? In-Reply-To: References: Message-ID: <4E8A3375.4090906@theheartofgold.org> Barco You wrote: > Hello Erlangers, > > When starting a erl shell, we will see a printout as: > Eshell V5.8.4 (abort with ^G) > > > then, I tried to press Ctrl+G and got: > User switch command > --> ls > Unknown command > --> q(). > Unknown command > --> > > I can do nothing with the prompt "-->". How to quit from it? and > what's "Ctrl+G" used for? > > The words "abort with ^G" are in fact hiding an extremely useful function called JCL (job control mode) [1]. I overlooked this function for years. Now I use it on a near daily basis and just love it. [1] http://www.erlang.org/doc/man/shell.html Assume you have an Erlang node named 'server@REDACTED' running as a service/daemon on your system. Wouldn't it be great if you could run interactive commands on this node just like you do in a shell? JCL let's you do that with ease. Start up a new shell 'client@REDACTED' and hit ctrl-G to enter JCL. Now start a remote shell to 'server@REDACTED'. | (client@REDACTED)1> | User switch command | --> r server@REDACTED | --> j | 1* {shell,start,[init]} | 2 {server@REDACTED,shell,start,[]} You now have a local shell with id=1 and a remote shell with id=2. Connect to the remote shell: | --> c 2 | | (server@REDACTED)1> Now you have a shell on the server node where you can run any command you want. This means you can call any exported function in all modules loaded on the server. That kind of interaction with a running system can be invaluable when trying to figure out why it doesn't behave quite the way you planned. Now of course there is a backside to the story. Doing something wrong may cause your system to crash. So hey - let's be careful out there. /Anders From mailparmalat@REDACTED Tue Oct 4 00:22:53 2011 From: mailparmalat@REDACTED (Steven Yu) Date: Tue, 4 Oct 2011 00:22:53 +0200 Subject: [erlang-questions] [Erlang-Q] What does Ctrl+G do for the Eshell? In-Reply-To: <4E8A3375.4090906@theheartofgold.org> References: <4E8A3375.4090906@theheartofgold.org> Message-ID: Hi, Not sure if its fixed in R14 but why when running without specifying node in user switch command your local erlang crashes? e.g. User switch command --> r {error_logger,{{2011,10,4},{0,18,25}},supervisor_report,[{supervisor,{<0.26.0>,user_sup}},{errorContext,child_terminated},{reason,{noproc,{gen_server,call,[{global,pool_master},get_node]}}},{offender,[{pid,<0.28.0>},{mod,user_sup}]}]}...... Can this behaviour be changed? Regards, On Tue, Oct 4, 2011 at 12:13 AM, Anders Ramsell wrote: > Barco You wrote: > >> Hello Erlangers, >> >> When starting a erl shell, we will see a printout as: >> Eshell V5.8.4 (abort with ^G) >> >> >> then, I tried to press Ctrl+G and got: >> User switch command >> --> ls >> Unknown command >> --> q(). >> Unknown command >> --> >> >> I can do nothing with the prompt "-->". How to quit from it? and >> what's "Ctrl+G" used for? >> >> >> > The words "abort with ^G" are in fact hiding an extremely useful > function called JCL (job control mode) [1]. I overlooked this > function for years. Now I use it on a near daily basis and just > love it. > > [1] http://www.erlang.org/doc/man/**shell.html > > Assume you have an Erlang node named 'server@REDACTED' running as a > service/daemon on your system. Wouldn't it be great if you could > run interactive commands on this node just like you do in a > shell? JCL let's you do that with ease. > > Start up a new shell 'client@REDACTED' and hit ctrl-G to enter JCL. > Now start a remote shell to 'server@REDACTED'. > > | (client@REDACTED)1> > | User switch command > | --> r server@REDACTED > | --> j > | 1* {shell,start,[init]} > | 2 {server@REDACTED,shell,start,[]} > > You now have a local shell with id=1 and a remote shell with > id=2. Connect to the remote shell: > > | --> c 2 > | > | (server@REDACTED)1> > > Now you have a shell on the server node where you can run any > command you want. This means you can call any exported function > in all modules loaded on the server. That kind of interaction > with a running system can be invaluable when trying to figure out > why it doesn't behave quite the way you planned. > > Now of course there is a backside to the story. Doing something > wrong may cause your system to crash. So hey - let's be careful > out there. > > /Anders > > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From emailregaccount@REDACTED Tue Oct 4 05:05:26 2011 From: emailregaccount@REDACTED (John Smith) Date: Mon, 3 Oct 2011 20:05:26 -0700 (PDT) Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: References: <579cb1b4-538e-4ad0-864e-cda3086d20aa@t11g2000yqk.googlegroups.com> <024a4da7-1d79-4c9c-8230-43d3cd38417c@i9g2000yqe.googlegroups.com> Message-ID: <01c1a14b-30a9-48bc-9c53-66b3c5d350e4@n12g2000yqh.googlegroups.com> Sorry, I should've explained in more detail what we're trying to do. That would help, eh? :) In a nutshell, our goal is take a portfolio of securities (namely bonds and derivatives), and calculate a risk/return analysis for each security. For risk, interest rate shock, and for return, future cash flows. There are different kinds of analyses you could perform. Here's a more concrete example. Pretend you're an insurance company. You have to pay out benefits to your customers, so you take their money and make investments with it, hoping for a (positive) return, of course. Quite often insurance companies will buy bonds, especially if there are restrictions on what they can invest in (e.g., AAA only). You need to have an idea of what your risk and return are. What's going to happen to the value of your portfolio if yields rise or fall? Ideally you want to know what your cash flows will look like in the future, so you can have a reasonable idea of what shape you'll be in depending on the outcome. One such calculation would involve shocking the yield curve (yields plotted against maturity). If yields rise 100 basis points, what happens to your portfolio? If they fall far enough how much would yields need to fall before any of your callable bonds started being redeemed? Part of the reason why I think Erlang would work out well is the calculations for each security are independent of each other -- it's an embarrassingly parallel problem. My goal was to spawn a process for each scenario of a security. Depending on how many securities and scenarios you want to calculate, there could be tens or hundreds of thousands, hence why I would be spawning so many processes (I would distribute these across multiple machines of course, but we would have only a few servers at most to start off with). Because Erlang is so efficient at creating and executing thousands of processes, I thought it would be feasible to create that many to do real work, but the impression I get is maybe it's not such a great idea when you have only a few dozen cores available to you. CGS, could you explain how the dynamic library would work in more detail? I was thinking it could work like that, but I wasn't actually sure how it would be implemented. For example, if two Erlang processes invoke the same shared library, does the OS simply copy each function call to its own stack frame so the data is kept separate, and only one copy of the code is used? I could see in that case then how 20,000 Erlang processes could all share the same library, since it minimizes the amount of memory used. David, the solution you described is new to me. Are there any resources I can read to learn more? Joe (your book is sitting on my desk as well =]), that's rather interesting Erlang was purposely slowed down to allow for on-the-fly code changes. Could you explain why? I'm curious. We are still in the R&D phase (you could say), so I'm not quite sure yet which specific category the number crunching will fall into (I wouldn't be surprised if there are matrices, however). I think what I'll do is write the most intensive parts in both Erlang and C, and compare the two. I'd prefer to stick purely with Erlang though! We have neither purchased any equipment yet nor written the final code, so I'm pretty flexible to whatever the best solution would be using Erlang. Maybe next year I can pick up one of those 20K core machines =) From paul.joseph.davis@REDACTED Tue Oct 4 05:28:05 2011 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Mon, 3 Oct 2011 22:28:05 -0500 Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: <01c1a14b-30a9-48bc-9c53-66b3c5d350e4@n12g2000yqh.googlegroups.com> References: <579cb1b4-538e-4ad0-864e-cda3086d20aa@t11g2000yqk.googlegroups.com> <024a4da7-1d79-4c9c-8230-43d3cd38417c@i9g2000yqe.googlegroups.com> <01c1a14b-30a9-48bc-9c53-66b3c5d350e4@n12g2000yqh.googlegroups.com> Message-ID: On Mon, Oct 3, 2011 at 10:05 PM, John Smith wrote: > Sorry, I should've explained in more detail what we're trying to do. > That would help, eh? :) > > In a nutshell, our goal is take a portfolio of securities (namely > bonds and derivatives), and calculate a risk/return analysis for each > security. For risk, interest rate shock, and for return, future cash > flows. There are different kinds of analyses you could perform. > > Here's a more concrete example. Pretend you're an insurance company. > You have to pay out benefits to your customers, so you take their > money and make investments with it, hoping for a (positive) return, of > course. Quite often insurance companies will buy bonds, especially if > there are restrictions on what they can invest in (e.g., AAA only). > > You need to have an idea of what your risk and return are. What's > going to happen to the value of your portfolio if yields rise or fall? > Ideally you want to know what your cash flows will look like in the > future, so you can have a reasonable idea of what shape you'll be in > depending on the outcome. > > One such calculation would involve shocking the yield curve (yields > plotted against maturity). If yields rise 100 basis points, what > happens to your portfolio? If they fall far enough how much would > yields need to fall before any of your callable bonds started being > redeemed? > > Part of the reason why I think Erlang would work out well is the > calculations for each security are independent of each other -- it's > an embarrassingly parallel problem. My goal was to spawn a process for > each scenario of a security. Depending on how many securities and > scenarios you want to calculate, there could be tens or hundreds of > thousands, hence why I would be spawning so many processes (I would > distribute these across multiple machines of course, but we would have > only a few servers at most to start off with). > > Because Erlang is so efficient at creating and executing thousands of > processes, I thought it would be feasible to create that many to do > real work, but the impression I get is maybe it's not such a great > idea when you have only a few dozen cores available to you. > > CGS, could you explain how the dynamic library would work in more > detail? I was thinking it could work like that, but I wasn't actually > sure how it would be implemented. For example, if two Erlang processes > invoke the same shared library, does the OS simply copy each function > call to its own stack frame so the data is kept separate, and only one > copy of the code is used? I could see in that case then how 20,000 > Erlang processes could all share the same library, since it minimizes > the amount of memory used. > > David, the solution you described is new to me. Are there any > resources I can read to learn more? > > Joe (your book is sitting on my desk as well =]), that's rather > interesting Erlang was purposely slowed down to allow for on-the-fly > code changes. Could you explain why? I'm curious. > > We are still in the R&D phase (you could say), so I'm not quite sure > yet which specific category the number crunching will fall into (I > wouldn't be surprised if there are matrices, however). I think what > I'll do is write the most intensive parts in both Erlang and C, and > compare the two. I'd prefer to stick purely with Erlang though! > > We have neither purchased any equipment yet nor written the final > code, so I'm pretty flexible to whatever the best solution would be > using Erlang. Maybe next year I can pick up one of those 20K core > machines =) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > Given your description above, I'd probably just write the first version of your application in Erlang. Normally I'm all for the NIF's but your scenario doesn't strike me as a the best fit (without first measuring the native Erlang which will be easier to code and maintain initially). The reason here is that there's a noticeable cost to passing data across the Erlang/(Driver|NIF|CNode) boundary so anything you're doing on the C side should be fast enough to more than make up for this. A good example here is from Kevin Smith's talk at the last Erlang Factory SF on using CUDA cards for numerical computations (he's illustrating the CUDA memory transfer overhead, but the same basic idea applies to passing data from Erlang to C). Given that your examples (sound to my non-financially familiar brain) to be small calculations on lots of data, you might be pleasantly surprised on the performance you'll get just from using Erlang across a large number of cores. And even if you find out in the future that you can write a small NIF that does your calculation in C using a request queue, that's just as well because you'll have tested that you need it and will know exactly how much you're saving by using C and so on. Paul From barcojie@REDACTED Tue Oct 4 08:31:21 2011 From: barcojie@REDACTED (Barco You) Date: Tue, 4 Oct 2011 14:31:21 +0800 Subject: [erlang-questions] [Erlang-Q] What does Ctrl+G do for the Eshell? In-Reply-To: <4E8A3375.4090906@theheartofgold.org> References: <4E8A3375.4090906@theheartofgold.org> Message-ID: Such a fabulous facility! Thank-you for telling! On Oct 4, 2011 6:13 AM, "Anders Ramsell" wrote: > Barco You wrote: >> Hello Erlangers, >> >> When starting a erl shell, we will see a printout as: >> Eshell V5.8.4 (abort with ^G) >> >> >> then, I tried to press Ctrl+G and got: >> User switch command >> --> ls >> Unknown command >> --> q(). >> Unknown command >> --> >> >> I can do nothing with the prompt "-->". How to quit from it? and >> what's "Ctrl+G" used for? >> >> > The words "abort with ^G" are in fact hiding an extremely useful > function called JCL (job control mode) [1]. I overlooked this > function for years. Now I use it on a near daily basis and just > love it. > > [1] http://www.erlang.org/doc/man/shell.html > > Assume you have an Erlang node named 'server@REDACTED' running as a > service/daemon on your system. Wouldn't it be great if you could > run interactive commands on this node just like you do in a > shell? JCL let's you do that with ease. > > Start up a new shell 'client@REDACTED' and hit ctrl-G to enter JCL. > Now start a remote shell to 'server@REDACTED'. > > | (client@REDACTED)1> > | User switch command > | --> r server@REDACTED > | --> j > | 1* {shell,start,[init]} > | 2 {server@REDACTED,shell,start,[]} > > You now have a local shell with id=1 and a remote shell with > id=2. Connect to the remote shell: > > | --> c 2 > | > | (server@REDACTED)1> > > Now you have a shell on the server node where you can run any > command you want. This means you can call any exported function > in all modules loaded on the server. That kind of interaction > with a running system can be invaluable when trying to figure out > why it doesn't behave quite the way you planned. > > Now of course there is a backside to the story. Doing something > wrong may cause your system to crash. So hey - let's be careful > out there. > > /Anders > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From wiener.guy@REDACTED Tue Oct 4 10:38:34 2011 From: wiener.guy@REDACTED (Guy Wiener) Date: Tue, 4 Oct 2011 10:38:34 +0200 Subject: [erlang-questions] Erlang on ARM9? In-Reply-To: <20111003190352.GA2709@corelatus.se> References: <20111003123303.GA16599@corelatus.se> <4E89B9DA.5090608@erlang.org> <20111003190352.GA2709@corelatus.se> Message-ID: It finally cross-compiled - Thanks a million, Matt and Gustav! Now I need to tweak and trim it, I hope it'll be simpler. Thanks again, Guy On Mon, Oct 3, 2011 at 9:03 PM, Matthias Lang wrote: > On Monday, October 03, Guy Wiener wrote: > > > My mistake: The build did not get stuck on some library, but erts is > > dependent on libcurses, which is not a part of the toolchain that I use > > (Sourcery lite). Can I compile Erlang somehow without this lib? Curses! > :) > > Have you tried passing > > --without-termcap > > when you run configure? > > Matt > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Tue Oct 4 13:21:34 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 4 Oct 2011 13:21:34 +0200 Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: <01c1a14b-30a9-48bc-9c53-66b3c5d350e4@n12g2000yqh.googlegroups.com> References: <579cb1b4-538e-4ad0-864e-cda3086d20aa@t11g2000yqk.googlegroups.com> <024a4da7-1d79-4c9c-8230-43d3cd38417c@i9g2000yqe.googlegroups.com> <01c1a14b-30a9-48bc-9c53-66b3c5d350e4@n12g2000yqh.googlegroups.com> Message-ID: On Tue, Oct 4, 2011 at 5:05 AM, John Smith wrote: > Sorry, I should've explained in more detail what we're trying to do. > That would help, eh? :) > > In a nutshell, our goal is take a portfolio of securities (namely > bonds and derivatives), and calculate a risk/return analysis for each > security. For risk, interest rate shock, and for return, future cash > flows. There are different kinds of analyses you could perform. > > Here's a more concrete example. Pretend you're an insurance company. > You have to pay out benefits to your customers, so you take their > money and make investments with it, hoping for a (positive) return, of > course. Quite often insurance companies will buy bonds, especially if > there are restrictions on what they can invest in (e.g., AAA only). > > You need to have an idea of what your risk and return are. What's > going to happen to the value of your portfolio if yields rise or fall? > Ideally you want to know what your cash flows will look like in the > future, so you can have a reasonable idea of what shape you'll be in > depending on the outcome. > > One such calculation would involve shocking the yield curve (yields > plotted against maturity). If yields rise 100 basis points, what > happens to your portfolio? If they fall far enough how much would > yields need to fall before any of your callable bonds started being > redeemed? > > Part of the reason why I think Erlang would work out well is the > calculations for each security are independent of each other -- it's > an embarrassingly parallel problem. My goal was to spawn a process for > each scenario of a security. Depending on how many securities and > scenarios you want to calculate, there could be tens or hundreds of > thousands, hence why I would be spawning so many processes (I would > distribute these across multiple machines of course, but we would have > only a few servers at most to start off with). > > Because Erlang is so efficient at creating and executing thousands of > processes, I thought it would be feasible to create that many to do > real work, but the impression I get is maybe it's not such a great > idea when you have only a few dozen cores available to you. > > CGS, could you explain how the dynamic library would work in more > detail? I was thinking it could work like that, but I wasn't actually > sure how it would be implemented. For example, if two Erlang processes > invoke the same shared library, does the OS simply copy each function > call to its own stack frame so the data is kept separate, and only one > copy of the code is used? I could see in that case then how 20,000 > Erlang processes could all share the same library, since it minimizes > the amount of memory used. > > David, the solution you described is new to me. Are there any > resources I can read to learn more? > > Joe (your book is sitting on my desk as well =]), that's rather > interesting Erlang was purposely slowed down to allow for on-the-fly > code changes. Could you explain why? I'm curious. I said "slow by design" - perhaps an unfortunately choice of words - What I meant was that there was design decision to allow code changes on the fly and that a consequence of this design decision means that all intermodule calls have one extra level of indirection which makes them slightly slower to implement then calls to code which cannot be changed on the fly. Suppose you have some module x executing some long-lived code (typically a telephony transaction) - you discover a bug in x. So you fix the bug. Now you have two versions of x. The x that is still currently executing, and the modified x that you will use when you start new transactions. We want to allow all the old processes running the old version of x to "run to completion" - new processes will get the next version of x. This is achieved as follows: if you call x:foo/2 you always call the latest version of the code, but inlined calls call the current version of the code. Let me give an example: Imagine the following: -module(foo). fix_loop(N) -> ... fix_loop(N+1). dynamic_loop(N) -> ... foo:dynamic_loop(N+1) In the above fix_loop and dynamic_loop have *entirely different behaviors * if we compile and reload a new version of foo, then any existing processes running fix_loop/1 inside x will continue running the old code. Any old processes running dynamic_loop/1 will jump into the new version of the code when they make the (tail) call to foo:dynamic_loop/1 To implement this requires one level of indirection in making the subroutine call. We can't just jump to the address of the code for loop, we have to call the function via a pointer. The ability to change code on the fly introduces a slight overhead in all function calls where you call the function with an explicit module name - if you omit the module name then the call will be slightly fast, since the address cannot be changed later. so calling fix_loop/1 in the above is slightly faster than calling dynamic_loop/1. Why do we want to do all this anyway? We designed Erlang for telecomms applications - we deploy applications that run for years and want to upgrade the software wihout disrupting services. If a user runs some code in a transaction that takes a a few minutes and we change the code we don't want to kill ongoing transactions using the old code - nor can we wait until all transactions are over before introducing new code (this will never happen). Banks turn off their transactions systems while upgrading the software - (apart from Klarna :- ) - aircraft upgrade the software while the planes are on the ground (I hope) - but we do it as we run the system (we don't want to loose calls just because we are upgrading the software) Now suppose you discover a fault in your software that causes to you buy or sell shares at a catastrophically bad rate - what do you do - wait for everything to stop before changing the code? - or pump in new code to fix the bug in mid session. Just killing everything might leave (say) a data base in an inconsistent state and make restarting time-consuming. Dynamic code change is useful to have under your feet just in case you need it one day - in the case on online banking companies like Klarna use this for commercial advantage :-) /Joe > > We are still in the R&D phase (you could say), so I'm not quite sure > yet which specific category the number crunching will fall into (I > wouldn't be surprised if there are matrices, however). I think what > I'll do is write the most intensive parts in both Erlang and C, and > compare the two. I'd prefer to stick purely with Erlang though! > > We have neither purchased any equipment yet nor written the final > code, so I'm pretty flexible to whatever the best solution would be > using Erlang. Maybe next year I can pick up one of those 20K core > machines =) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From robert.virding@REDACTED Tue Oct 4 13:54:25 2011 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 04 Oct 2011 12:54:25 +0100 (BST) Subject: [erlang-questions] [Erlang-Q] What does Ctrl+G do for the Eshell? In-Reply-To: Message-ID: Hi, It was created back in the days when we were thinking of Erlang as more of an OS where you could run multiple "jobs" at the same time. Apart allowing you to start many concurrent jobs, not just the shell, it also multiplexes the i/o so only the connected job (the 'c' command) gets access, both read and write, to the user terminal. I/o to the other jobs is blocked. Otherwise having mixed i/o to/from many jobs at the same time becomes a right mess. The ^G works as all user terminal input passes through this layer. Now it should maybe be rewritten to use separate windows instead? If we ever get a "standard" window interface. :-) The name JCL (Job Control Language) is a joke on IBM's JCL which they used back in the old days to control their batch jobs. It was if course completely different. Robert ----- Original Message ----- > Such a fabulous facility! Thank-you for telling! > On Oct 4, 2011 6:13 AM, "Anders Ramsell" < anders@REDACTED > > wrote: > > Barco You wrote: > >> Hello Erlangers, > >> > >> When starting a erl shell, we will see a printout as: > >> Eshell V5.8.4 (abort with ^G) > >> > >> > >> then, I tried to press Ctrl+G and got: > >> User switch command > >> --> ls > >> Unknown command > >> --> q(). > >> Unknown command > >> --> > >> > >> I can do nothing with the prompt "-->". How to quit from it? and > >> what's "Ctrl+G" used for? > >> > >> > > The words "abort with ^G" are in fact hiding an extremely useful > > function called JCL (job control mode) [1]. I overlooked this > > function for years. Now I use it on a near daily basis and just > > love it. > > > > [1] http://www.erlang.org/doc/man/shell.html > > > > Assume you have an Erlang node named 'server@REDACTED' running as a > > service/daemon on your system. Wouldn't it be great if you could > > run interactive commands on this node just like you do in a > > shell? JCL let's you do that with ease. > > > > Start up a new shell 'client@REDACTED' and hit ctrl-G to enter JCL. > > Now start a remote shell to 'server@REDACTED'. > > > > | (client@REDACTED)1> > > | User switch command > > | --> r server@REDACTED > > | --> j > > | 1* {shell,start,[init]} > > | 2 {server@REDACTED,shell,start,[]} > > > > You now have a local shell with id=1 and a remote shell with > > id=2. Connect to the remote shell: > > > > | --> c 2 > > | > > | (server@REDACTED)1> > > > > Now you have a shell on the server node where you can run any > > command you want. This means you can call any exported function > > in all modules loaded on the server. That kind of interaction > > with a running system can be invaluable when trying to figure out > > why it doesn't behave quite the way you planned. > > > > Now of course there is a backside to the story. Doing something > > wrong may cause your system to crash. So hey - let's be careful > > out there. > > > > /Anders > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mail@REDACTED Tue Oct 4 14:14:45 2011 From: mail@REDACTED (=?iso-8859-1?Q?Philip_M=FCller?=) Date: Tue, 04 Oct 2011 14:14:45 +0200 Subject: [erlang-questions] use inetrc to add entries to perceived /etc/hosts Message-ID: Hi everybody, we are working on a distributed erlang project. We want to have as little dependencies on specific infrastructure as possible. This is why we assume that 1. There is no DNS server running providing a naming service. 2. Entries in /etc/hosts on the node hosts are insufficient to provide naming for all other node hosts. We need to make it possible for all hosts to reach each other and to start all nodes from a central host. We could start all nodes via ssh, identifying hosts via IP addresses, but as we understand it slave:start/1 is made for this job and only works with hostnames. Searching for a solution we stumbled across "-kernel inetrc". The idea is to scp an inetrc to all hosts and then start nodes with slave:start and get them to read their local inetrc on startup. We tried starting a node locally with the following command: > erl -rsh ssh -sname test -kernel inetrc '"./erl_inetrc"' For the contents of erl_inetrc we tried the following two solutions, but didn't get it to work. Running slave:start(host1). on host2 always gave the following error: > {error,timeout} But a simple "ssh 192.168.2.100" works. 1. everything in erl_inetrc erl_inetrc: > {host, {192,168,2,100}, ["host1"]}. > {host, {192,168,2,101}, ["host2"]}. 2. an additional hosts file erl_inetrc: > {file, hosts, "./hosts"}. hosts file: > 192.168.2.100 host1 > 192.168.2.101 host2 Any ideas what we did wrong? Best Regards Florian Grabbe Philip M?ller From barcojie@REDACTED Tue Oct 4 14:22:07 2011 From: barcojie@REDACTED (Barco You) Date: Tue, 4 Oct 2011 20:22:07 +0800 Subject: [erlang-questions] [Erlang-Q] What does Ctrl+G do for the Eshell? In-Reply-To: References: Message-ID: A further question regarding the erlang shell. Can we run an application without running the shell? As I know, to run an erlang app I have to first start the shell and then app:start(). As you said, erlang is more than an OS, so can I understand in the way that an app cannot run when the OS (Eshell) is not started? Thanks, Barco Sent from my HTC On Oct 4, 2011 7:54 PM, "Robert Virding" < robert.virding@REDACTED> wrote: > Hi, > > It was created back in the days when we were thinking of Erlang as more of an OS where you could run multiple "jobs" at the same time. Apart allowing you to start many concurrent jobs, not just the shell, it also multiplexes the i/o so only the connected job (the 'c' command) gets access, both read and write, to the user terminal. I/o to the other jobs is blocked. Otherwise having mixed i/o to/from many jobs at the same time becomes a right mess. > > The ^G works as all user terminal input passes through this layer. Now it should maybe be rewritten to use separate windows instead? If we ever get a "standard" window interface. :-) > > The name JCL (Job Control Language) is a joke on IBM's JCL which they used back in the old days to control their batch jobs. It was if course completely different. > > Robert > > ----- Original Message ----- > >> Such a fabulous facility! Thank-you for telling! >> On Oct 4, 2011 6:13 AM, "Anders Ramsell" < anders@REDACTED >> > wrote: >> > Barco You wrote: >> >> Hello Erlangers, >> >> >> >> When starting a erl shell, we will see a printout as: >> >> Eshell V5.8.4 (abort with ^G) >> >> >> >> >> >> then, I tried to press Ctrl+G and got: >> >> User switch command >> >> --> ls >> >> Unknown command >> >> --> q(). >> >> Unknown command >> >> --> >> >> >> >> I can do nothing with the prompt "-->". How to quit from it? and >> >> what's "Ctrl+G" used for? >> >> >> >> >> > The words "abort with ^G" are in fact hiding an extremely useful >> > function called JCL (job control mode) [1]. I overlooked this >> > function for years. Now I use it on a near daily basis and just >> > love it. >> > >> > [1] http://www.erlang.org/doc/man/shell.html >> > >> > Assume you have an Erlang node named 'server@REDACTED' running as a >> > service/daemon on your system. Wouldn't it be great if you could >> > run interactive commands on this node just like you do in a >> > shell? JCL let's you do that with ease. >> > >> > Start up a new shell 'client@REDACTED' and hit ctrl-G to enter JCL. >> > Now start a remote shell to 'server@REDACTED'. >> > >> > | (client@REDACTED)1> >> > | User switch command >> > | --> r server@REDACTED >> > | --> j >> > | 1* {shell,start,[init]} >> > | 2 {server@REDACTED,shell,start,[]} >> > >> > You now have a local shell with id=1 and a remote shell with >> > id=2. Connect to the remote shell: >> > >> > | --> c 2 >> > | >> > | (server@REDACTED)1> >> > >> > Now you have a shell on the server node where you can run any >> > command you want. This means you can call any exported function >> > in all modules loaded on the server. That kind of interaction >> > with a running system can be invaluable when trying to figure out >> > why it doesn't behave quite the way you planned. >> > >> > Now of course there is a backside to the story. Doing something >> > wrong may cause your system to crash. So hey - let's be careful >> > out there. >> > >> > /Anders >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From gustav@REDACTED Tue Oct 4 14:46:03 2011 From: gustav@REDACTED (Gustav Simonsson) Date: Tue, 4 Oct 2011 14:46:03 +0200 Subject: [erlang-questions] [Erlang-Q] What does Ctrl+G do for the Eshell? In-Reply-To: References: Message-ID: <1317732363.31717.0.camel@belegost> See http://www.erlang.org/doc/man/erl.html in particular the -noshell flag. Regards, Gustav Simonsson On Tue, 2011-10-04 at 20:22 +0800, Barco You wrote: > A further question regarding the erlang shell. Can we run an > application without running the shell? As I know, to run an erlang app > I have to first start the shell and then app:start(). > > As you said, erlang is more than an OS, so can I understand in the way > that an app cannot run when the OS (Eshell) is not started? > > Thanks, > Barco > Sent from my HTC > > On Oct 4, 2011 7:54 PM, "Robert Virding" > wrote: > > Hi, > > > > It was created back in the days when we were thinking of Erlang as > more of an OS where you could run multiple "jobs" at the same time. > Apart allowing you to start many concurrent jobs, not just the shell, > it also multiplexes the i/o so only the connected job (the 'c' > command) gets access, both read and write, to the user terminal. I/o > to the other jobs is blocked. Otherwise having mixed i/o to/from many > jobs at the same time becomes a right mess. > > > > The ^G works as all user terminal input passes through this layer. > Now it should maybe be rewritten to use separate windows instead? If > we ever get a "standard" window interface. :-) > > > > The name JCL (Job Control Language) is a joke on IBM's JCL which > they used back in the old days to control their batch jobs. It was if > course completely different. > > > > Robert > > > > ----- Original Message ----- > > > >> Such a fabulous facility! Thank-you for telling! > >> On Oct 4, 2011 6:13 AM, "Anders Ramsell" < > anders@REDACTED > >> > wrote: > >> > Barco You wrote: > >> >> Hello Erlangers, > >> >> > >> >> When starting a erl shell, we will see a printout as: > >> >> Eshell V5.8.4 (abort with ^G) > >> >> > >> >> > >> >> then, I tried to press Ctrl+G and got: > >> >> User switch command > >> >> --> ls > >> >> Unknown command > >> >> --> q(). > >> >> Unknown command > >> >> --> > >> >> > >> >> I can do nothing with the prompt "-->". How to quit from it? and > >> >> what's "Ctrl+G" used for? > >> >> > >> >> > >> > The words "abort with ^G" are in fact hiding an extremely useful > >> > function called JCL (job control mode) [1]. I overlooked this > >> > function for years. Now I use it on a near daily basis and just > >> > love it. > >> > > >> > [1] http://www.erlang.org/doc/man/shell.html > >> > > >> > Assume you have an Erlang node named 'server@REDACTED' running as a > >> > service/daemon on your system. Wouldn't it be great if you could > >> > run interactive commands on this node just like you do in a > >> > shell? JCL let's you do that with ease. > >> > > >> > Start up a new shell 'client@REDACTED' and hit ctrl-G to enter JCL. > >> > Now start a remote shell to 'server@REDACTED'. > >> > > >> > | (client@REDACTED)1> > >> > | User switch command > >> > | --> r server@REDACTED > >> > | --> j > >> > | 1* {shell,start,[init]} > >> > | 2 {server@REDACTED,shell,start,[]} > >> > > >> > You now have a local shell with id=1 and a remote shell with > >> > id=2. Connect to the remote shell: > >> > > >> > | --> c 2 > >> > | > >> > | (server@REDACTED)1> > >> > > >> > Now you have a shell on the server node where you can run any > >> > command you want. This means you can call any exported function > >> > in all modules loaded on the server. That kind of interaction > >> > with a running system can be invaluable when trying to figure out > >> > why it doesn't behave quite the way you planned. > >> > > >> > Now of course there is a backside to the story. Doing something > >> > wrong may cause your system to crash. So hey - let's be careful > >> > out there. > >> > > >> > /Anders > >> > > >> > _______________________________________________ > >> > erlang-questions mailing list > >> > erlang-questions@REDACTED > >> > http://erlang.org/mailman/listinfo/erlang-questions > > > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From michael.santos@REDACTED Tue Oct 4 15:28:23 2011 From: michael.santos@REDACTED (Michael Santos) Date: Tue, 4 Oct 2011 09:28:23 -0400 Subject: [erlang-questions] use inetrc to add entries to perceived /etc/hosts In-Reply-To: References: Message-ID: <20111004132823.GC5659@ecn.lan> On Tue, Oct 04, 2011 at 02:14:45PM +0200, Philip M?ller wrote: > Hi everybody, > > we are working on a distributed erlang project. We want to have as > little dependencies on specific infrastructure as possible. > > This is why we assume that > > 1. There is no DNS server running providing a naming service. > 2. Entries in /etc/hosts on the node hosts are insufficient to > provide naming for all other node hosts. > > We need to make it possible for all hosts to reach each other and to > start all nodes from a central host. We could start all nodes via > ssh, identifying hosts via IP addresses, but as we understand it > slave:start/1 is made for this job and only works with hostnames. > > Searching for a solution we stumbled across "-kernel inetrc". The > idea is to scp an inetrc to all hosts and then start nodes with > slave:start and get them to read their local inetrc on startup. > > We tried starting a node locally with the following command: > > >erl -rsh ssh -sname test -kernel inetrc '"./erl_inetrc"' > > For the contents of erl_inetrc we tried the following two solutions, > but didn't get it to work. > Running slave:start(host1). on host2 always gave the following error: > >{error,timeout} > But a simple "ssh 192.168.2.100" works. > > 1. everything in erl_inetrc > > erl_inetrc: > >{host, {192,168,2,100}, ["host1"]}. > >{host, {192,168,2,101}, ["host2"]}. > > 2. an additional hosts file > > erl_inetrc: > >{file, hosts, "./hosts"}. > > hosts file: > >192.168.2.100 host1 > >192.168.2.101 host2 > > Any ideas what we did wrong? Try enforcing the lookup order. Add this to the "erl_inetrc" file: {lookup, [file, dns]}. From mail@REDACTED Tue Oct 4 16:31:19 2011 From: mail@REDACTED (=?iso-8859-1?Q?Philip_M=FCller?=) Date: Tue, 04 Oct 2011 16:31:19 +0200 Subject: [erlang-questions] use inetrc to add entries to perceived /etc/hosts In-Reply-To: <20111004132823.GC5659@ecn.lan> References: <20111004132823.GC5659@ecn.lan> Message-ID: On Tue, 04 Oct 2011 15:28:23 +0200, Michael Santos wrote: > On Tue, Oct 04, 2011 at 02:14:45PM +0200, Philip M?ller wrote: >> Hi everybody, >> >> we are working on a distributed erlang project. We want to have as >> little dependencies on specific infrastructure as possible. [...] >> Running slave:start(host1). on host2 always gave the following error: >> >{error,timeout} >> But a simple "ssh 192.168.2.100" works. >> >> Any ideas what we did wrong? > > Try enforcing the lookup order. Add this to the "erl_inetrc" file: > > {lookup, [file, dns]}. > Thanks for the tip but alas, it does not work. Still getting a timeout. Are there ways to diagnose this further? We do not know whether the hostname-ip mappings are imported at all. Best Regards Philip M?ller From michael.santos@REDACTED Tue Oct 4 18:36:08 2011 From: michael.santos@REDACTED (Michael Santos) Date: Tue, 4 Oct 2011 12:36:08 -0400 Subject: [erlang-questions] use inetrc to add entries to perceived /etc/hosts In-Reply-To: References: <20111004132823.GC5659@ecn.lan> Message-ID: <20111004163608.GA6401@ecn.lan> On Tue, Oct 04, 2011 at 04:31:19PM +0200, Philip M?ller wrote: > On Tue, 04 Oct 2011 15:28:23 +0200, Michael Santos > wrote: > > >On Tue, Oct 04, 2011 at 02:14:45PM +0200, Philip M?ller wrote: > >>Hi everybody, > >> > >>we are working on a distributed erlang project. We want to have as > >>little dependencies on specific infrastructure as possible. > [...] > >>Running slave:start(host1). on host2 always gave the following error: > >>>{error,timeout} > >>But a simple "ssh 192.168.2.100" works. > >> > >>Any ideas what we did wrong? > > > >Try enforcing the lookup order. Add this to the "erl_inetrc" file: > > > >{lookup, [file, dns]}. > > > > Thanks for the tip but alas, it does not work. Still getting a timeout. > Are there ways to diagnose this further? We do not know whether the > hostname-ip mappings are imported at all. The timeout is using slave:start/1? Use slave:start/3 to pass cli options to the slave: fcs:~$ cat erl_inetrc {lookup, [file, dns]}. fcs:~$ erl -rsh ssh -sname test -kernel inetrc '"./erl_inetrc"' (test@REDACTED)3> slave:start('fcs', 't', "-kernel inetrc '\"./erl_inetrc\"'"). {ok,t@REDACTED} (test@REDACTED)4> rpc:call('t@REDACTED', inet_db, get_rc, []). [{resolv_conf,"/etc/resolv.conf"}, {hosts_file,"/etc/hosts"}, {lookup,[file,dns]}] From peerst@REDACTED Tue Oct 4 18:37:57 2011 From: peerst@REDACTED (Peer Stritzinger) Date: Tue, 4 Oct 2011 18:37:57 +0200 Subject: [erlang-questions] Keeping massive concurrency when interfacing with C Message-ID: On Mon, Oct 3, 2011 at 10:05 PM, John Smith <> wrote: >* We are still in the R&D phase (you could say), so I'm not quite sure*>* yet which specific category the number crunching will fall into (I*>* wouldn't be surprised if there are matrices, however). I think what*>* I'll do is write the most intensive parts in both Erlang and C, and*>* compare the two. I'd prefer to stick purely with Erlang though!* *From time to time the "Erlang is poor at number crunching" can be heard.* *Mainly this revolves around Erlang being bad at the kind of number crunching needed for linear algebra/image processing etc.* *Having a similar requirement as John for a current project I thinking a lot recently how to use Erlang maybe together with another language or system together for this (other requirements are quite in favor of Erlang for my project).* *When pondering this I noticed that if Erlang would have a flexible n-dim array type with good performing matrix/vector manipulation functions I would not need to integrate some external system with all the complexity required to make the concurrency impedances match.* *There are several systems where efficient matrix manipulation is added to languages that would not be considered for numerical calculations without them. Examples are NumPy and pdl.perl.org. * *Usually these revolve around a basic matrix "buffer" which is reference counted. These buffers are referred to by structures of metadata containing dimensionality, matrix slice info (they support the concept of having slices of matrices which share the buffers without copying).* *The buffers are quite similar to Erlangs binaries, and the no-copy slicing looks quite functional to me.* *If we would build a matrix library on top of binaries with only a few basic operations as NIF's quite powerful things could be built on top of this using pure Erlang.* *I would invest some of my time in a library like this, probably not enough to make it general purpose. But if others want to participate, maybe we won't have to repeat the Erlangs to for (all) number crunching sentiment too often.* *Cheers* *-- Peer Stritzinger* -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Oct 5 00:17:03 2011 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 5 Oct 2011 11:17:03 +1300 Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: References: Message-ID: <8BAED16D-F958-4001-83B4-64CA5AD6DB7F@cs.otago.ac.nz> On 5/10/2011, at 5:37 AM, Peer Stritzinger wrote: [how about a linear algebra library built on top of binaries, not entirely unlike NumPy] Hasn't something like this already been done? I'm sure I remember reading about it. From freza@REDACTED Wed Oct 5 00:34:06 2011 From: freza@REDACTED (Jachym Holecek) Date: Tue, 4 Oct 2011 18:34:06 -0400 Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: <8BAED16D-F958-4001-83B4-64CA5AD6DB7F@cs.otago.ac.nz> References: <8BAED16D-F958-4001-83B4-64CA5AD6DB7F@cs.otago.ac.nz> Message-ID: <20111004223406.GA7932@circlewave.net> # Richard O'Keefe 2011-10-04: > On 5/10/2011, at 5:37 AM, Peer Stritzinger wrote: > [how about a linear algebra library built on top of binaries, not entirely > unlike NumPy] > > Hasn't something like this already been done? I'm sure I remember reading > about it. Yeah, I remember reading the paper with keen interest, but not sure the code was ever published: "High-Performance Technical Computing with Erlang" http://www.erlang.org/workshop/2008/Sess23.pdf Personally I'd consider OCaml/MLton (running as port program over stdio) for that kind of task, but then I may be missing the point of this thread (sorry, didn't follow closely). BR, -- Jachym From ok@REDACTED Wed Oct 5 00:38:56 2011 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 5 Oct 2011 11:38:56 +1300 Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: References: <579cb1b4-538e-4ad0-864e-cda3086d20aa@t11g2000yqk.googlegroups.com> <024a4da7-1d79-4c9c-8230-43d3cd38417c@i9g2000yqe.googlegroups.com> <01c1a14b-30a9-48bc-9c53-66b3c5d350e4@n12g2000yqh.googlegroups.com> Message-ID: A further clarification on what Joe wrote about hot loading: Whatever the current Erlang system actually does, the overhead of remote calls need in principle be no more than the overhead of dynamic dispatch in a language like C++. That overhead is actually surprisingly high (and yet people *willingly* write Java, go figure). There is an indirect cost to the indirection, namely that dynamic calls can't be inlined. For C++ there is an answer: link-time analysis can find calls (often lots and lots of them) that don't actually need to be polymorphic (e.g., because the declared class turns out not to have any subclasses that override the method in question) and those calls can be inlined after all. In languages which allow new code to be added at run time (like Java and Erlang) it's not that easy. Some years ago I proposed that Erlang could distinguish between "detachable" and "non-detachable" parts, so that a group of modules could be bound together in such a way that they would have to be replaced _as a unit_. The idea has not been taken up because it's very far from being Erlang's most pressing problem. To John Smith, what on earth does "shocking the yield curve" mean? One thing about architecture. Joe raised an interesting question. "Now suppose you discover a fault in your software that causes to you buy or sell shares at a catastrophically bad rate - what do you do - wait for everything to stop before changing the code?" My question is, "how could you structure your system so that if it TRIES to buy or sell at a catastrophically bad rate it CAN'T?" A couple of years ago I came up with an idea for a potential PhD candidate who ended up going somewhere else. That was inspired by a true event here, where an electricity company cut off supply to a house where there was an extremely sick woman who depended on some machine to keep her alive (I forget what kind). Needless to say, she died. And of course it was one of those stories where the computer noticed the bill hadn't been paid recently and sent out a notice to a technician who dutifully went out and turned the power off without asking any awkward questions. So what can we do to stop that? (The customer had informed the company of their special needs.) The answer I came up with turns out to be quite similar in spirit to Joe's UBF. You have a GENERATOR of actions, a CRITIC of actions, and an EFFECTOR of actions. (Come to think of it, there's a link here to Dorothy L. Sayers' "The Mind of the Maker.") The generator of actions receives inputs and decides on things to do, but doesn't actually do them. It passes its proposals on to the critic, which watches out for bad stuff. Things that the critic is happy with are passed on to the effector to be carried out. In the electricity case, the critic would use rules like "If the proposal is to disconnect supply and the customer has registered a special need and there is no record of a court order REJECT" In the trading case, the critic's rules would say something about the amount of money. The generator should not rely on the critic; if everything is working well you won't be able to tell if the critic is there or not. A rejection by the critic indicates an error in the generator requiring corrective programming. This is where it gets similar to UBF: UBF contract checking isn't there to make good things happen normally, it's there to stop bad things happening and make sure they're noticed. This is one way to use multicore: spend some of the extra cores doing more checking. From emailregaccount@REDACTED Wed Oct 5 02:48:28 2011 From: emailregaccount@REDACTED (John Smith) Date: Tue, 4 Oct 2011 17:48:28 -0700 (PDT) Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: References: <579cb1b4-538e-4ad0-864e-cda3086d20aa@t11g2000yqk.googlegroups.com> <024a4da7-1d79-4c9c-8230-43d3cd38417c@i9g2000yqe.googlegroups.com> <01c1a14b-30a9-48bc-9c53-66b3c5d350e4@n12g2000yqh.googlegroups.com> Message-ID: <1a7a7dbb-d89a-44d1-a16f-94d99d87b695@m5g2000vbe.googlegroups.com> Hi Richard, Here's an example: imagine you've plotted a curve of US Treasury yields and their maturities: http://en.wikipedia.org/wiki/File:USD_yield_curve_09_02_2005.JPG You do this for 360 months (30 years) and have a yield for every month. Now obviously there aren't data points for every month (there are no 12.5-year Treasuries) so you have to come up with data points for those months (but we can ignore that detail). Now you've constructed your yield curve and you want to shock it. What that means is you either shift the curve up or down by a fixed amount of basis points for every yield point. If you shock the curve 100 basis points up (100 basis points equals 1 percent), you move every yield point up by 100 basis points, and now you have your shocked yield curve (shocking normally occurs at major intervals, e.g., 25, 50, 100). You can then evaluate how your portfolio would fare in this environment. From freeakk@REDACTED Wed Oct 5 06:12:45 2011 From: freeakk@REDACTED (Michael Uvarov) Date: Wed, 5 Oct 2011 08:12:45 +0400 Subject: [erlang-questions] NIFs Reloading Message-ID: Hello, How to load new version of c-code with NIFs? --- Best regards, Uvarov Michael From peerst@REDACTED Wed Oct 5 09:24:19 2011 From: peerst@REDACTED (Peer Stritzinger) Date: Wed, 5 Oct 2011 09:24:19 +0200 Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: <8BAED16D-F958-4001-83B4-64CA5AD6DB7F@cs.otago.ac.nz> References: <8BAED16D-F958-4001-83B4-64CA5AD6DB7F@cs.otago.ac.nz> Message-ID: On Wed, Oct 5, 2011 at 12:17 AM, Richard O'Keefe wrote: > > On 5/10/2011, at 5:37 AM, Peer Stritzinger wrote: > [how about a linear algebra library built on top of binaries, not entirely > unlike NumPy] > > Hasn't something like this already been done? ?I'm sure I remember reading > about it. I have to admit I was not aware of this. OTOH it seems not to be available, can't find anything except the paper and EEP7 which is the foreign function interface to the external number crunching libraries they invented. However I was thinking along different lines, the approach of "HPTC with Erlang" (and also NumPy) is to slap on the big chunk of proven numerical routines as some external library. Which is the way to go if you want to do serious number crunching, since its quite hard to develop trusted and efficient numerical routines. The price you have to pay for the slapped on heavyweight library is that these usually don't scale up to the number of processes Erlang can handle. Therefore the need of the impedance adaption I mentioned. Keeping a pool of numerical processes to keep the cores busy but not too many of them that the OS is upset. Having work queues that adapt these to the 20k processes. BTW @John: this would be one solution for your problem. What I was suggesting is a more integrated and lightweight way to make some number crunching available. The suggested n-dim matrix type (e.g. a record containing the metadata and a binary for the data) combined with some NIFs on these that speed up the parts where Erlang is not so fast. Keeping in mind not to do too much work in the NIFs at one time not to block the scheduler. This is for the use cases where there is some numerical stuff needed but having real time responsiveness and Erlang process counts in mind. The use case I have e.g. is some neuronal networks stuff combined with a lot of symbolic computing to prepare the input. And its embarrassingly parallel and needs only some simple vector times matrix and n-dim array slicing. For real heavy numerical stuff I think the best way is to do this in the systems are built for this and interface them somehow to erlang with ports or sockets. Or try to get the code released from the HPTC paper Jachym mentioned. For interfacing with BLAS and their ilk some more native Erlang numerical capabilities would also be nice to have. Since they also use a kind of binary buffer with some metadata approach it would not be too hard to interface efficiently. From erlang@REDACTED Wed Oct 5 10:26:10 2011 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 5 Oct 2011 10:26:10 +0200 Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: References: <579cb1b4-538e-4ad0-864e-cda3086d20aa@t11g2000yqk.googlegroups.com> <024a4da7-1d79-4c9c-8230-43d3cd38417c@i9g2000yqe.googlegroups.com> <01c1a14b-30a9-48bc-9c53-66b3c5d350e4@n12g2000yqh.googlegroups.com> Message-ID: On Wed, Oct 5, 2011 at 12:38 AM, Richard O'Keefe wrote: > A further clarification on what Joe wrote about hot loading: > > ?Whatever the current Erlang system actually does, the overhead of remote > ?calls need in principle be no more than the overhead of dynamic dispatch > ?in a language like C++. > > ?That overhead is actually surprisingly high (and yet people *willingly* > ?write Java, go figure). ?There is an indirect cost to the indirection, > ?namely that dynamic calls can't be inlined. ?For C++ there is an answer: > ?link-time analysis can find calls (often lots and lots of them) that > ?don't actually need to be polymorphic (e.g., because the declared class > ?turns out not to have any subclasses that override the method in question) > ?and those calls can be inlined after all. ?In languages which allow new > ?code to be added at run time (like Java and Erlang) it's not that easy. > > Some years ago I proposed that Erlang could distinguish between > "detachable" and "non-detachable" parts, so that a group of modules could > be bound together in such a way that they would have to be replaced _as a > unit_. ?The idea has not been taken up because it's very far from being > Erlang's most pressing problem. > > To John Smith, what on earth does "shocking the yield curve" mean? > > > One thing about architecture. ?Joe raised an interesting question. > "Now suppose you discover a fault in your software that causes to you > buy or sell shares at a catastrophically bad rate - what do you do - > wait for everything to stop before changing the code?" > > My question is, "how could you structure your system so that if it > TRIES to buy or sell at a catastrophically bad rate it CAN'T?" ?A couple > of years ago I came up with an idea for a potential PhD candidate who > ended up going somewhere else. ?That was inspired by a true event here, > where an electricity company cut off supply to a house where there was > an extremely sick woman who depended on some machine to keep her alive > (I forget what kind). ?Needless to say, she died. ?And of course it was > one of those stories where the computer noticed the bill hadn't been > paid recently and sent out a notice to a technician who dutifully went > out and turned the power off without asking any awkward questions. ?So > what can we do to stop that? ?(The customer had informed the company of > their special needs.) ?The answer I came up with turns out to be > quite similar in spirit to Joe's UBF. > > ? ? ? ?You have a GENERATOR of actions, > ? ? ? ?a CRITIC of actions, and > ? ? ? ?an EFFECTOR of actions. > > (Come to think of it, there's a link here to Dorothy L. Sayers' "The > Mind of the Maker.") ?The generator of actions receives inputs and > decides on things to do, but doesn't actually do them. ?It passes > its proposals on to the critic, which watches out for bad stuff. > Things that the critic is happy with are passed on to the effector to > be carried out. > > In the electricity case, the critic would use rules like > ? ? ? ?"If the proposal is to disconnect supply > ? ? ? ? and the customer has registered a special need > ? ? ? ? and there is no record of a court order > ? ? ? ? REJECT" > > In the trading case, the critic's rules would say something about the > amount of money. Absolutely - move compliance into the software - If traders expose a bank to too much risk the trades should be stopped automatically. This might have prevented (say) the collapse of Baring bank. There is a trade of between security - (we have to spend a few CPUs cycles checking for illegality) and speed here. If a bank collapses due to trading violations that could have been detected by software and no such software was in place I imagine legal action could be taken. So there should be a strong argument for such software. /Joe > The generator should not rely on the critic; if everything is working > well you won't be able to tell if the critic is there or not. > > A rejection by the critic indicates an error in the generator > requiring corrective programming. ?This is where it gets similar > to UBF: ?UBF contract checking isn't there to make good things > happen normally, it's there to stop bad things happening and make > sure they're noticed. > > This is one way to use multicore: spend some of the extra cores doing > more checking. > > From cgsmcmlxxv@REDACTED Wed Oct 5 10:54:49 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Wed, 5 Oct 2011 11:54:49 +0300 Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: <01c1a14b-30a9-48bc-9c53-66b3c5d350e4@n12g2000yqh.googlegroups.com> References: <579cb1b4-538e-4ad0-864e-cda3086d20aa@t11g2000yqk.googlegroups.com> <024a4da7-1d79-4c9c-8230-43d3cd38417c@i9g2000yqe.googlegroups.com> <01c1a14b-30a9-48bc-9c53-66b3c5d350e4@n12g2000yqh.googlegroups.com> Message-ID: Hi, Sorry for delay in my answer (internet problems here). To answer your question now. Yes, the main difference in between static libraries (.a) and dynamic libraries (.so) is that the dynamic library is loaded only once for all the instances requiring that library without overlapping the processes data. In some other words, what you said it's true, the heaps are kept separately for all the distinct processes calling the same shared library, but the memory usage is less because the library resides in the memory only once. Following this idea, some LINUX developers were expressing their thoughts to make all the libraries as shared ones. But that is quite a joke because shared libraries are loaded on demand basis and not when the application starts and it is discarded when no application is using it. Just to exemplify, consider you build a graphic application which knows how to handle hundreds of graphic formats (bitmap, jpeg, gif, pixel map and so on). You don't want to use static libraries because you may not have enough RAM to load them all. Then, you pay the price for your application to be a bit slower, but you use shared libraries which are loaded only when your application requests them and discarded when your application doesn't need them anymore. Another example for you to understand the concept behind shared (aka dynamic) libraries. Let's say you want to compress with GNUZip (aka gzip) different folders to make backups on different storage elements (from your computing element to archive one folder which goes on storage element A and another folder to go on storage element B). So, you need to run gzip twice, but you don't want to wait for first archive to finish itself in order for you to start the second. Then, you open two sessions (either two terminals or start the first process and put it to work in background) and start archiving your folders (for you to be able to drink a coffee until they are finished :) ). Both of them use zlib.so (or zlibc.so, but it's not important for this example) which is loaded once, but the archives do not mix their data (if you don't believe me, try it :) ). That means, as you said, the heaps do not overlap for the two processes even if your library resides in the memory only once at the time (you can check if you don't believe me). I think the second example is what you need. So, all you need is a shared library which needs to reside in the memory (e.g., called by a dummy program which does nothing but keeps the library in the memory). In this way, your Erlang threads can call it at any time (keep in mind that Erlang is not able by itself to ask OS to load any external library). I hope I brought some light in the concept of shared library (under MS Windows are called dynamic and they can be identified by the suffix dll - dynamic linked library) as I tried to give it in simple words (please, those who are experts, do not kill me for not expressing myself in the terminology required or if I skipped some technical details). If you still have questions, let me know. Now about your project. Because what you need I think it doesn't require a lot of time to be processed (I computed trendlines and filtered them on the live securities downloaded from internet with cURL and I know it goes pretty fast the calculus), you don't need to run all of them in parallel, so, you can divide them into a number of parallel threads suitable to your hardware system. In this way you make the best of your hardware. Two more things here. Firstly, yes, Erlang can help you to update the software while running it. That will help you a lot because in C/C++ you will need to device an intelligent system to update your code without stopping it (I did once something like that, but believe me, it's not worth the work when you have this already implemented). Secondly, if you already have the code written in C/C++, then it's worth making the connection, but otherwise you can work in Erlang because you won't feel big difference for what you need. But, of course, these are just suggestions. At the end of the day, it's your responsibility, so, you will decide what approach to follow. So, I can only wish you good luck! ;) Cheers, CGS On Tue, Oct 4, 2011 at 6:05 AM, John Smith wrote: > Sorry, I should've explained in more detail what we're trying to do. > That would help, eh? :) > > In a nutshell, our goal is take a portfolio of securities (namely > bonds and derivatives), and calculate a risk/return analysis for each > security. For risk, interest rate shock, and for return, future cash > flows. There are different kinds of analyses you could perform. > > Here's a more concrete example. Pretend you're an insurance company. > You have to pay out benefits to your customers, so you take their > money and make investments with it, hoping for a (positive) return, of > course. Quite often insurance companies will buy bonds, especially if > there are restrictions on what they can invest in (e.g., AAA only). > > You need to have an idea of what your risk and return are. What's > going to happen to the value of your portfolio if yields rise or fall? > Ideally you want to know what your cash flows will look like in the > future, so you can have a reasonable idea of what shape you'll be in > depending on the outcome. > > One such calculation would involve shocking the yield curve (yields > plotted against maturity). If yields rise 100 basis points, what > happens to your portfolio? If they fall far enough how much would > yields need to fall before any of your callable bonds started being > redeemed? > > Part of the reason why I think Erlang would work out well is the > calculations for each security are independent of each other -- it's > an embarrassingly parallel problem. My goal was to spawn a process for > each scenario of a security. Depending on how many securities and > scenarios you want to calculate, there could be tens or hundreds of > thousands, hence why I would be spawning so many processes (I would > distribute these across multiple machines of course, but we would have > only a few servers at most to start off with). > > Because Erlang is so efficient at creating and executing thousands of > processes, I thought it would be feasible to create that many to do > real work, but the impression I get is maybe it's not such a great > idea when you have only a few dozen cores available to you. > > CGS, could you explain how the dynamic library would work in more > detail? I was thinking it could work like that, but I wasn't actually > sure how it would be implemented. For example, if two Erlang processes > invoke the same shared library, does the OS simply copy each function > call to its own stack frame so the data is kept separate, and only one > copy of the code is used? I could see in that case then how 20,000 > Erlang processes could all share the same library, since it minimizes > the amount of memory used. > > David, the solution you described is new to me. Are there any > resources I can read to learn more? > > Joe (your book is sitting on my desk as well =]), that's rather > interesting Erlang was purposely slowed down to allow for on-the-fly > code changes. Could you explain why? I'm curious. > > We are still in the R&D phase (you could say), so I'm not quite sure > yet which specific category the number crunching will fall into (I > wouldn't be surprised if there are matrices, however). I think what > I'll do is write the most intensive parts in both Erlang and C, and > compare the two. I'd prefer to stick purely with Erlang though! > > We have neither purchased any equipment yet nor written the final > code, so I'm pretty flexible to whatever the best solution would be > using Erlang. Maybe next year I can pick up one of those 20K core > machines =) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Wed Oct 5 11:08:34 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 05 Oct 2011 12:08:34 +0300 Subject: [erlang-questions] Testing of nifs In-Reply-To: References: <7501178.53.1317498317476.JavaMail.geo-discussion-forums@yqmw31> <10774339.2082.1317590830225.JavaMail.geo-discussion-forums@yqgn17> Message-ID: <4E8C1E92.6030806@cs.ntua.gr> On 10/03/11 10:51, Tim Watson wrote: > On 2 October 2011 22:27, Dmitry Groshev > wrote: > > Well, it's about hidden details. Erlang is pretty straightforward > language; C is a lot more tricky to make right. So there is a lot of > non obvious bugs that I can't foresee when I write unit tests; > PropEr's *random* tests are way better in this sense, because you > just describe the domain of your task, not particular cases. You can > also test your NIF incrementally, writing property after property > (or command after command if there is some state involved). > Speaking about my experience ? I'm using PropEr to test this library > https://github.com/band115/ecirca , that implements mutable circular > array in Erlang (all calls to this array are constrained to it's > creator, so I hope this mutability can't be a problem). There are no > docs right now other than code; I will definitely fix it in a week > or two when I'll have more time. I've tried to test it with EUnit > when it was just plain array; but then I've added different types of > array and realized that I should add like 5 times more code to cover > all cases, and there was a thought about adding different types of > values (int16/32/64), so I realized than EUnit is definitely not the > best tool, just because the space of inputs is so multidimensional > (and I really wanted to cover as much of it as I can with tests ? > Erlang's VM is very fragile to NIFs after all). So I started to dig > into PropEr's docs, wrote first commands/checks for statem, compiled > a debug version of BEAM, made a habit to attach GDB to the process > running tests (this way in case of segfault, and I had a lot of > them, I can quickly find where I broke the code) ? and right now I > can be sure to some degree (which is proportional to amount of tests > that you've asked PropEr to do) that this NIFs won't explode in my > face tomorrow. > > > That makes perfect sense. I've been struggling in my mind with the best > ways to test linked-in driver code, which is a little different. ... > > At the moment I'm relying on traditional unit testing of the C code > along with static analysis and integration testing using common_test. IMO, you should really use PropEr instead of the above. This will take you a long way into what you want to eventually achieve. > I guess what I'd really like is something like PULSE (the randomising > scheduler employed by QuickCheck) to help me identify where my driver > code might be locking up the emulator or whatever, but for open source > projects that's not an option. Currently I rely on load tests in a > mirror of the target environment*s(, but this isn't really good enough > because if I move from an 8 core to a 16 core machine, problems can > mysteriously appear that just didn't show up and they're really hard to > figure out. There are better techniques than random-based testing for the types of bugs you want to eliminate. One such tool is Concuerror (available from https://github.com/mariachris/Concuerror). This is currently work in progress and we had not had the time to create a proper homepage for it. However, you can take a look at its (really really basic at this point -- more to come, hopefully soon) documentation and the examples that its distribution contains and also get a good idea of the concurrency errors Concuerror can detect and how to use the tool by reading the paper: Test-Driven Development of Concurrent Programs using Concuerror http://people.inf.ethz.ch/chmaria/Pubs/ERLANG-2011.pdf The paper is very comprehensive and I strongly recommend it to anyone who takes elimination of concurrency errors in Erlang seriously. Kostis PS. If your application is open-source, I'll gladly help you in using Concuerror, but I suggest we take this off this list, if interested. From watson.timothy@REDACTED Wed Oct 5 11:56:05 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 5 Oct 2011 10:56:05 +0100 Subject: [erlang-questions] Testing of nifs In-Reply-To: <4E8C1E92.6030806@cs.ntua.gr> References: <7501178.53.1317498317476.JavaMail.geo-discussion-forums@yqmw31> <10774339.2082.1317590830225.JavaMail.geo-discussion-forums@yqgn17> <4E8C1E92.6030806@cs.ntua.gr> Message-ID: On 5 October 2011 10:08, Kostis Sagonas wrote: > > IMO, you should really use PropEr instead of the above. This will take you > a long way into what you want to eventually achieve. > > Yes I thought it should be, particularly using statem to test my port controller and things like that. > I guess what I'd really like is something like PULSE (the randomising >> scheduler employed by QuickCheck) to help me identify where my driver >> code might be locking up the emulator or whatever, but for open source >> projects that's not an option. Currently I rely on load tests in a >> mirror of the target environment*s(, but this isn't really good enough >> because if I move from an 8 core to a 16 core machine, problems can >> mysteriously appear that just didn't show up and they're really hard to >> figure out. >> > > There are better techniques than random-based testing for the types of bugs > you want to eliminate. One such tool is Concuerror (available from > https://github.com/mariachris/**Concuerror). > This is currently work in progress and we had not had the time to create a > proper homepage for it. However, you can take a look at its (really really > basic at this point -- more to come, hopefully soon) documentation and the > examples that its distribution contains and also get a good idea of the > concurrency errors Concuerror can detect and how to use the tool by reading > the paper: > > Test-Driven Development of Concurrent Programs using Concuerror > http://people.inf.ethz.ch/**chmaria/Pubs/ERLANG-2011.pdf > > The paper is very comprehensive and I strongly recommend it to anyone who > takes elimination of concurrency errors in Erlang seriously. > > Kostis > > PS. If your application is open-source, I'll gladly help you in using > Concuerror, but I suggest we take this off this list, if interested. > Awesome. I'll go read the paper and spend some time looking at the tool as well. Thanks as well for the offer of help - my project is open source and I might drop you an email off list if I get stuck! Cheers, Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgsmcmlxxv@REDACTED Wed Oct 5 12:05:24 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Wed, 5 Oct 2011 13:05:24 +0300 Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: References: <8BAED16D-F958-4001-83B4-64CA5AD6DB7F@cs.otago.ac.nz> Message-ID: Sorry for interfering, but I would suggest CBLAS (BLAS rewritten in C) instead of BLAS (which is written in FORTRAN). CBLAS is coming with GSL (GNU Scientific Library) and it's thread safe (I tried it for a scientific Monte Carlo simulation). GSL has the main advantage that it uses already tested algorithms used by the scientific community (like CERN, FermiLab and so on) where speed and precision is a must. But this is for really heavy linear algebra. Cheers, CGS On Wed, Oct 5, 2011 at 10:24 AM, Peer Stritzinger wrote: > On Wed, Oct 5, 2011 at 12:17 AM, Richard O'Keefe > wrote: > > > > On 5/10/2011, at 5:37 AM, Peer Stritzinger wrote: > > [how about a linear algebra library built on top of binaries, not > entirely > > unlike NumPy] > > > > Hasn't something like this already been done? I'm sure I remember > reading > > about it. > > I have to admit I was not aware of this. OTOH it seems not to be > available, can't find anything except the paper and EEP7 which is the > foreign function interface to the external number crunching libraries > they invented. > > However I was thinking along different lines, the approach of "HPTC > with Erlang" (and also NumPy) is to slap on the big chunk of proven > numerical routines as some external library. Which is the way to go > if you want to do serious number crunching, since its quite hard to > develop trusted and efficient numerical routines. > > The price you have to pay for the slapped on heavyweight library is > that these usually don't scale up to the number of processes Erlang > can handle. Therefore the need of the impedance adaption I mentioned. > Keeping a pool of numerical processes to keep the cores busy but not > too many of them that the OS is upset. Having work queues that adapt > these to the 20k processes. BTW @John: this would be one solution for > your problem. > > What I was suggesting is a more integrated and lightweight way to make > some number crunching available. The suggested n-dim matrix type > (e.g. a record containing the metadata and a binary for the data) > combined with some NIFs on these that speed up the parts where Erlang > is not so fast. Keeping in mind not to do too much work in the NIFs > at one time not to block the scheduler. > > This is for the use cases where there is some numerical stuff needed > but having real time responsiveness and Erlang process counts in mind. > The use case I have e.g. is some neuronal networks stuff combined > with a lot of symbolic computing to prepare the input. And its > embarrassingly parallel and needs only some simple vector times matrix > and n-dim array slicing. > > For real heavy numerical stuff I think the best way is to do this in > the systems are built for this and interface them somehow to erlang > with ports or sockets. Or try to get the code released from the HPTC > paper Jachym mentioned. > > For interfacing with BLAS and their ilk some more native Erlang > numerical capabilities would also be nice to have. Since they also > use a kind of binary buffer with some metadata approach it would not > be too hard to interface efficiently. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zerthurd@REDACTED Wed Oct 5 13:41:33 2011 From: zerthurd@REDACTED (Maxim Treskin) Date: Wed, 5 Oct 2011 18:41:33 +0700 Subject: [erlang-questions] R14B04 MD5 checksum Message-ID: Hello Thank you for R14B04 but MD5 file does not contain checksum for it and kerl cannot build this release. Can you fix it? -- Maxim Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Oct 5 14:04:54 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 5 Oct 2011 14:04:54 +0200 Subject: [erlang-questions] OpenSSL 1.0 removed md2 and this affects Erlang/OTP builds. Message-ID: Just a quick FYI: A modern Ubuntu (11.10 - will be released later this month) ships with a more recent OpenSSL library. In this SSL-library it has been decided to kill md2 as a message digest algorithm, probably due to its appalling cryptographic strength. This yields a problem with building the "dev" branch of Erlang/OTP: gcc -c -o .... /home/jlouis/Projects/otp/erts/emulator/sys/unix crypto.c crypto.c:46:25: fatal error: openssl/md2.h: No such file or directory compilation terminated. specifically in and around the unix crypto.c (crypto module). My guess is that this problem will begin to crop up shortly after the release, mainly because people will like to build their software again. The md2 functions are used in rsa_verify and rsa_sign (two C NIFs), which is the reason for the include in the first place. Also note that we have no calls to md2 from the Erlang world (luckily) so this limits the impact on Erlang code: Some signatures that could be verified can not anymore. But verifying a signature against md2 is probably not too clever. On the same basis: When do we get to ditch MD5 and SHA1? Both message digests are probably going to become problematic. MD5 is already broken for its 2nd preimage resistance to the point where nobody can use it for that anymore. SHA1 will follow shortly, there are cracks all over the place. We really want an API which is message digest agnostic. Something along the lines of: -spec crypto:message_digest(digest_algorithm(), iolist()) -> digest(). where digest_algorithm() is md5 | sha1 | sha256 | ... and digest() is either integer() or binary() - probably the latter with conversion functions to hex-strings and so on, because that format is the common one. The rationale for this choice is simple: history shows that most cryptographic algorithms breaks down over time - one way or the other. -- J. From michal.niec@REDACTED Wed Oct 5 14:18:31 2011 From: michal.niec@REDACTED (Michal Niec) Date: Wed, 5 Oct 2011 14:18:31 +0200 Subject: [erlang-questions] OpenSSL 1.0 removed md2 and this affects Erlang/OTP builds. In-Reply-To: References: Message-ID: Hi, On 5 pa? 2011, at 14:04, Jesper Louis Andersen wrote: > > A modern Ubuntu (11.10 - will be released later this month) ships with > a more recent OpenSSL library. In this SSL-library it has been decided > to kill md2 as a message digest algorithm, probably due to its > appalling cryptographic strength. This yields a problem with building > the "dev" branch of Erlang/OTP: Opensuse 11.4 (which is current release) use openssl 1.0.0 and also lacks of md2. From openssl version >1.0.0 MD2 is disabled by default. However Fedora 15, Centos 6 have openssl 1.0.0 versions with md2 enabled. > > gcc -c -o .... /home/jlouis/Projects/otp/erts/emulator/sys/unix crypto.c > crypto.c:46:25: fatal error: openssl/md2.h: No such file or directory > compilation terminated. > Dev branch on github contains problematic md2 support, but in R14B04 release this feature is removed. Regards, Michal Niec > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From attila.r.nohl@REDACTED Wed Oct 5 14:24:14 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 5 Oct 2011 14:24:14 +0200 Subject: [erlang-questions] OpenSSL 1.0 removed md2 and this affects Erlang/OTP builds. In-Reply-To: References: Message-ID: 2011/10/5, Jesper Louis Andersen : [...] > On the same basis: When do we get to ditch MD5 and SHA1? They are used by SNMPv3 so I don't think it's that easy to ditch them... > Both message > digests are probably going to become problematic. MD5 is already > broken for its 2nd preimage resistance to the point where nobody can > use it for that anymore. As far as I know, it is possible to generate two sets of binary data that has the same MD5 digest. It is also possible to generate two sets of formatted binary data in a format (.doc file, etc.) that has the same MD5 digest - the key is that these formats can include binary junk that is actually not used. Again, as far as I know, the SNMPv3 packet format is not quite like this, so I'm not sure we can say that MD5 authentication is broken for SNMPv3. I definitely would like to keep MD5 in the future. Anyway, I think there are millions of devices out there with SNMPv3 support that won't be upgraded before their hardware finally breaks down or are sold to scrap. From sverker@REDACTED Wed Oct 5 14:35:09 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Wed, 5 Oct 2011 14:35:09 +0200 Subject: [erlang-questions] NIFs Reloading In-Reply-To: References: Message-ID: <4E8C4EFD.7010200@erix.ericsson.se> Michael Uvarov wrote: > Hello, > > How to load new version of c-code with NIFs? > There are two ways to do that (in a running system). * The right way: Load the new version of your module and let that new Erlang code load your new NIF library with erlang:load_nif/2. The callback 'upgrade' will then be called in your new library (instead of 'load") and that will give you opportunity to transfer state from your old library if you want to. The 'unload' callback will later be called in your old NIF library before the old version of your module is purged. The above is the recommended way to upgrade a NIF library in runtime as it harmonize with how "normal" module upgrade is done. * The quick and dirty way: Just call erlang:load_nif/2 a second time from the same Erlang module instance. This will just replace the NIF library while leaving the Erlang code in place. The callback 'reload' is called in this case. This is only recommended during development (as a fast way to load your upgraded NIF code) and not as an upgrade procedure for a running production system. I actually regret that we supported this feature as it can complicate future improvements of the code loading mechanisms in the emulator. It is possible that we will retract this reload feature in some future release (R15 maybe). /Sverker, Erlang/OTP Ericsson From dave@REDACTED Wed Oct 5 14:39:07 2011 From: dave@REDACTED (David Goehrig) Date: Wed, 5 Oct 2011 08:39:07 -0400 Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: References: <579cb1b4-538e-4ad0-864e-cda3086d20aa@t11g2000yqk.googlegroups.com> <024a4da7-1d79-4c9c-8230-43d3cd38417c@i9g2000yqe.googlegroups.com> <01c1a14b-30a9-48bc-9c53-66b3c5d350e4@n12g2000yqh.googlegroups.com> Message-ID: <77E5BD72-A47B-4B1D-9D27-DC383BE41379@nexttolast.com> These guys have a proof of concept on their products page: http://www.calxeda.com/products.php Rumor has it they have it working as a SoC. http://www.tilera.com/ Has a functional SoC that is MIPS based While no suitable for Erlang http://greenarrays.com/ Has the most impressive offering I've played with. But this is a embedded forth programmer's chip. If you want to get into parallel number crunching applications chips like: http://www.clearspeed.com/products/csx700.php Almost take the fun out of it. Even Intel demoed an 80 core chip a year ago. We are going to see an explosion of small SoC fabless integrators pumping out massively parallel architectures all with a green energy pitch. It all comes down to simple physic butting up against economics. Too much die is left idle in most modern deep pipeline superscalar designs, single thread performance is I/O bound for most real world problems, thermal effects have made low power desirable on a macro scale. Cheapest solution shove a bunch of smaller cores on a die with a on chip bus, and shift the problem to software :) -=-=- dave@REDACTED -=-=- On Oct 3, 2011, at 11:28 PM, Paul Davis wrote: > On Mon, Oct 3, 2011 at 10:05 PM, John Smith wrote: >> Sorry, I should've explained in more detail what we're trying to do. >> That would help, eh? :) >> >> In a nutshell, our goal is take a portfolio of securities (namely >> bonds and derivatives), and calculate a risk/return analysis for each >> security. For risk, interest rate shock, and for return, future cash >> flows. There are different kinds of analyses you could perform. >> >> Here's a more concrete example. Pretend you're an insurance company. >> You have to pay out benefits to your customers, so you take their >> money and make investments with it, hoping for a (positive) return, of >> course. Quite often insurance companies will buy bonds, especially if >> there are restrictions on what they can invest in (e.g., AAA only). >> >> You need to have an idea of what your risk and return are. What's >> going to happen to the value of your portfolio if yields rise or fall? >> Ideally you want to know what your cash flows will look like in the >> future, so you can have a reasonable idea of what shape you'll be in >> depending on the outcome. >> >> One such calculation would involve shocking the yield curve (yields >> plotted against maturity). If yields rise 100 basis points, what >> happens to your portfolio? If they fall far enough how much would >> yields need to fall before any of your callable bonds started being >> redeemed? >> >> Part of the reason why I think Erlang would work out well is the >> calculations for each security are independent of each other -- it's >> an embarrassingly parallel problem. My goal was to spawn a process for >> each scenario of a security. Depending on how many securities and >> scenarios you want to calculate, there could be tens or hundreds of >> thousands, hence why I would be spawning so many processes (I would >> distribute these across multiple machines of course, but we would have >> only a few servers at most to start off with). >> >> Because Erlang is so efficient at creating and executing thousands of >> processes, I thought it would be feasible to create that many to do >> real work, but the impression I get is maybe it's not such a great >> idea when you have only a few dozen cores available to you. >> >> CGS, could you explain how the dynamic library would work in more >> detail? I was thinking it could work like that, but I wasn't actually >> sure how it would be implemented. For example, if two Erlang processes >> invoke the same shared library, does the OS simply copy each function >> call to its own stack frame so the data is kept separate, and only one >> copy of the code is used? I could see in that case then how 20,000 >> Erlang processes could all share the same library, since it minimizes >> the amount of memory used. >> >> David, the solution you described is new to me. Are there any >> resources I can read to learn more? >> >> Joe (your book is sitting on my desk as well =]), that's rather >> interesting Erlang was purposely slowed down to allow for on-the-fly >> code changes. Could you explain why? I'm curious. >> >> We are still in the R&D phase (you could say), so I'm not quite sure >> yet which specific category the number crunching will fall into (I >> wouldn't be surprised if there are matrices, however). I think what >> I'll do is write the most intensive parts in both Erlang and C, and >> compare the two. I'd prefer to stick purely with Erlang though! >> >> We have neither purchased any equipment yet nor written the final >> code, so I'm pretty flexible to whatever the best solution would be >> using Erlang. Maybe next year I can pick up one of those 20K core >> machines =) >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > Given your description above, I'd probably just write the first > version of your application in Erlang. Normally I'm all for the NIF's > but your scenario doesn't strike me as a the best fit (without first > measuring the native Erlang which will be easier to code and maintain > initially). > > The reason here is that there's a noticeable cost to passing data > across the Erlang/(Driver|NIF|CNode) boundary so anything you're doing > on the C side should be fast enough to more than make up for this. A > good example here is from Kevin Smith's talk at the last Erlang > Factory SF on using CUDA cards for numerical computations (he's > illustrating the CUDA memory transfer overhead, but the same basic > idea applies to passing data from Erlang to C). > > Given that your examples (sound to my non-financially familiar brain) > to be small calculations on lots of data, you might be pleasantly > surprised on the performance you'll get just from using Erlang across > a large number of cores. And even if you find out in the future that > you can write a small NIF that does your calculation in C using a > request queue, that's just as well because you'll have tested that you > need it and will know exactly how much you're saving by using C and so > on. > > Paul > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From kenneth.lundin@REDACTED Wed Oct 5 14:40:22 2011 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 5 Oct 2011 14:40:22 +0200 Subject: [erlang-questions] Erlang/OTP R14B04 has been released Message-ID: Erlang/OTP R14B04 has been released. This release is mainly a stabilization of the R14B03 release (but as usual there are some new functionality as well). You can find the README file for the release at http://www.erlang.org/download/otp_src_R14B04.readme The source distribution and binary distribution for Windows can be downloaded from http://www.erlang.org/download/otp_src_R14B04.tar.gz http://www.erlang.org/download/otp_win32_R14B04.exe Note: To unpack the TAR archive you need a GNU TAR compatible program. For installation instructions please read the README file that is part of the distribution. You can also find the source in the official Erlang/OTP git-repository at GitHub (http://github.com/erlang/otp) The on-line documentation can be found at: http://www.erlang.org/doc/ You can also download the complete HTML documentation or the Unix manual files http://www.erlang.org/download/otp_doc_html_R14B04.tar.gz http://www.erlang.org/download/otp_doc_man_R14B04.tar.gz We also want to thank those that sent us patches, suggestions and bug reports. The Erlang/OTP Team Kenneth Lundin, Erlang/OTP, Ericsson AB -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Oct 5 14:40:41 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 5 Oct 2011 14:40:41 +0200 Subject: [erlang-questions] OpenSSL 1.0 removed md2 and this affects Erlang/OTP builds. In-Reply-To: References: Message-ID: On Wed, Oct 5, 2011 at 14:24, Attila Rajmund Nohl wrote: > 2011/10/5, Jesper Louis Andersen : > [...] >> On the same basis: When do we get to ditch MD5 and SHA1? > > They are used by SNMPv3 so I don't think it's that easy to ditch them... I was probably too harsh in my choice of words. What I'd like is to have the cryptographic algorithm have the back seat and the concept (message digests) having the drivers seat. So you are going to say "I want a message digest algorithm, specifically this one" or "I want a MAC, specifically an HMAC-SHA1". I am aware you can't get rid of MD5, even though it shouldn't be used for anything new. Good cryptographic design today makes it easy to change one algorithm for another, because it will effectively nullify many weaknesses. I'd like an API which makes this idea easy to implement. > As far as I know, it is possible to generate two sets of binary data > that has the same MD5 digest. Yep, that is 2nd preimage resistance broken. A message digest algorithm is not injective, but we would like to get as close to an injective function as possible. > Again, as far as I know, the SNMPv3 > packet format is not quite like this, so I'm not ?sure we can say that > MD5 authentication is broken for SNMPv3. I definitely would like to > keep MD5 in the future. In SNMPv3, you may be referring to HMAC-MD5, which is a Hash-based Message Authentication Code. HMAC-MD5 does not suffer from the 2nd preimage resistance weakness directly, so it is relatively safe to use in that specific case. -- J. From aschultz@REDACTED Wed Oct 5 14:43:33 2011 From: aschultz@REDACTED (Andreas Schultz) Date: Wed, 05 Oct 2011 14:43:33 +0200 (CEST) Subject: [erlang-questions] OpenSSL 1.0 removed md2 and this affects Erlang/OTP builds. In-Reply-To: Message-ID: <9d0e125a-f947-4dc9-ba1a-9ebc55d3a046@office> Hi, ----- Original Message ----- > Just a quick FYI: [...] > On the same basis: When do we get to ditch MD5 and SHA1? Both message > digests are probably going to become problematic. MD5 is already Not in the foreseeable feature, SSLv3, TLS 1.0 and TLS 1.1 all depend on MD5 and SHA1. The prf's used in TLS 1.0 and TLS 1.1 use a combination of MD5+SHA1, so this is going to have to stay for a few more years, probably. TLS 1.2 replaces the MD5+SHA1 combination with SHA256, but TLS 1.2 is not used very much at the moment, IMHO not implemented in openssl-1.0 and Erlang's ssl app is currently limited to TLS-1.0. > broken for its 2nd preimage resistance to the point where nobody can > use it for that anymore. SHA1 will follow shortly, there are cracks > all over the place. We really want an API which is message digest > agnostic. Something along the lines of: > > -spec crypto:message_digest(digest_algorithm(), iolist()) -> > digest(). > > where digest_algorithm() is md5 | sha1 | sha256 | ... and digest() is > either integer() or binary() - probably the latter with conversion > functions to hex-strings and so on, because that format is the common > one. The rationale for this choice is simple: history shows that most > cryptographic algorithms breaks down over time - one way or the > other. I very much like this. The same scheme should be applied to the HMAC function and the _init, _update_ and _final version of the hash and HMAC functions should be removed. They look like leftovers from some C implementation, are not very erlang'ish and at least hmac_final can even crash the VM. Andreas > > -- > J. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- -- Dipl. Inform. Andreas Schultz email: as@REDACTED phone: +49-391-819099-224 mobil: +49-179-7654368 ------------------ managed broadband access ------------------ Travelping GmbH phone: +49-391-8190990 Roentgenstr. 13 fax: +49-391-819099299 D-39108 Magdeburg email: info@REDACTED GERMANY web: http://www.travelping.com Company Registration: HRB21276 Handelsregistergericht Chemnitz Geschaeftsfuehrer: Holger Winkelmann | VAT ID No.: DE236673780 -------------------------------------------------------------- From joelr1@REDACTED Wed Oct 5 15:30:41 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 5 Oct 2011 14:30:41 +0100 Subject: [erlang-questions] customizing the rebar-generated .app file Message-ID: <1749100C-2462-4808-B3C7-51F127A19970@gmail.com> I would like to customize the app file created by rebar. For example, I would like to change the env section and make sure applications includes mnesia and lager. Last but not least, I would like to change the version to the output of `git rev-parse --short HEAD`. I used to have a template file that I would run sed on to change the version. I would like to let rebar keep the modules up to date, though. Can I have my cake and eat it too? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Wed Oct 5 15:31:12 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 5 Oct 2011 14:31:12 +0100 Subject: [erlang-questions] rebar and boot files Message-ID: <30D8F422-E394-4420-AA6C-5F356670D2BB@gmail.com> How do I tell rebar to generate a boot file? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From michael.weibel@REDACTED Wed Oct 5 15:36:58 2011 From: michael.weibel@REDACTED (Michael Weibel - Amiado Group) Date: Wed, 5 Oct 2011 15:36:58 +0200 Subject: [erlang-questions] Newbie Question: Module with DB Connection as a "global state" Message-ID: <12D9DB35-E8F9-42FC-9B40-CBE981860AC2@amiadogroup.com> Hi all, I'm new to erlang, new to this list and I hope here's the right place to ask this question. I'm currently implementing a module for ejabberd where I need to log certain packets to a MySQL DB and I'm using the MySQL Native Driver for it. My current implementation is on github: https://github.com/amiadogroup/mod_log_chat_mysql5/blob/master/src/mod_log_chat_mysql5.erl In this file you see that I'm opening the MySQL Connection in the function "open_mysql_connection" and assign it to a ETS Table because I didn't find another way to store the DB Reference (I tried something with records but didn't succeed). This worked pretty well but I ran into the problem that after some days/weeks the ets table didn't have the DB Reference anymore. To prevent this, I want to create functions which setup the DB Connection again, if it doesn't have it anymore. As I'm trying to do this I run into the problem that the informations about how to connect to the db is only in the init-function available and I don't really want to store this information also in the ets table. Can anyone help me or point me to a site on how to do this? Thanks, Michael From watson.timothy@REDACTED Wed Oct 5 15:39:14 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 5 Oct 2011 14:39:14 +0100 Subject: [erlang-questions] customizing the rebar-generated .app file In-Reply-To: <1749100C-2462-4808-B3C7-51F127A19970@gmail.com> References: <1749100C-2462-4808-B3C7-51F127A19970@gmail.com> Message-ID: On 5 October 2011 14:30, Joel Reymont wrote: > I would like to customize the app file created by rebar. > > For example, I would like to change the env section and make sure > applications includes mnesia and lager. > If you put the env section in your src/.app.src file then it will be copied across verbatim. Last but not least, I would like to change the version to the output of `git > rev-parse --short HEAD`. > > If your application is in a subdirectory within the project (i.e., is one of sub_dirs) then you can do this by generating the version you want in a file inside the project subdirectory named priv/vsn.git - then you can put {vsn, git} in your app.src and it will work fine. I do something similar using a rebar plugin here: https://github.com/hyperthunk/nodewatch/blob/master/build/src/git_vsn_plugin.erl Other approaches include - write a rebar plugin that hooks into post_compile/2 and overwrites the value in the generated .app file - write a command line hook that uses sed on the generated .app file, and register it as {post_hooks, [{compile, "sed ....."}]} in your rebar.config You may even be able to do this with rebar's app.vars file, although I'm not 100% sure if that'll work. HTH! -------------- next part -------------- An HTML attachment was scrubbed... URL: From gustav@REDACTED Wed Oct 5 15:54:59 2011 From: gustav@REDACTED (Gustav Simonsson) Date: Wed, 5 Oct 2011 15:54:59 +0200 Subject: [erlang-questions] Newbie Question: Module with DB Connection as a "global state" In-Reply-To: <12D9DB35-E8F9-42FC-9B40-CBE981860AC2@amiadogroup.com> References: <12D9DB35-E8F9-42FC-9B40-CBE981860AC2@amiadogroup.com> Message-ID: <1317822899.31717.7.camel@belegost> Hi Michael! On line 48 in your code you store the connection information used in open_mysql_connection/1 in the state of the server. Perhaps you can make use of that? Regards, Gustav Simonsson On Wed, 2011-10-05 at 15:36 +0200, Michael Weibel - Amiado Group wrote: > Hi all, > > I'm new to erlang, new to this list and I hope here's the right place to ask this question. > > I'm currently implementing a module for ejabberd where I need to log certain packets to a MySQL DB and I'm using the MySQL Native Driver for it. > > My current implementation is on github: > https://github.com/amiadogroup/mod_log_chat_mysql5/blob/master/src/mod_log_chat_mysql5.erl > > In this file you see that I'm opening the MySQL Connection in the function "open_mysql_connection" and assign it to a ETS Table because I didn't find another way to store the DB Reference (I tried something with records but didn't succeed). > > This worked pretty well but I ran into the problem that after some days/weeks the ets table didn't have the DB Reference anymore. > To prevent this, I want to create functions which setup the DB Connection again, if it doesn't have it anymore. > > As I'm trying to do this I run into the problem that the informations about how to connect to the db is only in the init-function available and I don't really want to store this information also in the ets table. > > Can anyone help me or point me to a site on how to do this? > > Thanks, > Michael > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From joelr1@REDACTED Wed Oct 5 15:58:50 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 5 Oct 2011 14:58:50 +0100 Subject: [erlang-questions] customizing the rebar-generated .app file In-Reply-To: References: <1749100C-2462-4808-B3C7-51F127A19970@gmail.com> Message-ID: On Oct 5, 2011, at 2:39 PM, Tim Watson wrote: > If your application is in a subdirectory within the project (i.e., is one of sub_dirs) then you can do this by generating the version you want in a file inside the project subdirectory named priv/vsn.git - then you can put {vsn, git} in your app.src and it will work fine. A subdirectory is not necessary. How does rebar come up with the final version number using vsn.git, though? To wit: cat priv/vsn.git 0eaae0a grep vsn ebin/openpoker.app {vsn,"v0.1-111-g0eaae0a"}, Where do 0.1 and 111 come from? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Wed Oct 5 16:29:12 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 5 Oct 2011 15:29:12 +0100 Subject: [erlang-questions] customizing the rebar-generated .app file In-Reply-To: References: <1749100C-2462-4808-B3C7-51F127A19970@gmail.com> Message-ID: <928330DB-23BD-4098-992F-2402DF4DB485@gmail.com> rebar is being too clever here. Suppose you are using git and are in the top level of the project when building, as you should be. rebar will default the version to "git describe --always --tags `git log -n 1 --pretty=format:%h .`". rebar will not let you override this version string. On Oct 5, 2011, at 2:58 PM, Joel Reymont wrote: > How does rebar come up with the final version number using vsn.git, though? > > To wit: > > cat priv/vsn.git > 0eaae0a > > grep vsn ebin/openpoker.app > {vsn,"v0.1-111-g0eaae0a"}, > > Where do 0.1 and 111 come from? -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From gustav@REDACTED Wed Oct 5 16:57:58 2011 From: gustav@REDACTED (Gustav Simonsson) Date: Wed, 5 Oct 2011 16:57:58 +0200 Subject: [erlang-questions] Newbie Question: Module with DB Connection as a "global state" In-Reply-To: <4D33CF3D-7302-4837-98AB-7EADA5468466@amiadogroup.com> References: <12D9DB35-E8F9-42FC-9B40-CBE981860AC2@amiadogroup.com> <1317822899.31717.7.camel@belegost> <4D33CF3D-7302-4837-98AB-7EADA5468466@amiadogroup.com> Message-ID: <1317826678.31717.18.camel@belegost> The way you could do it is to have the log_packet_send/3 function make a callback so you can handle the request in a context where you have the state defined in the init function. I haven't used gen_mod so I don't know exactly how that works, but I assume it works in a similar way to how a gen_server implementation would call Module:handle_cast/2 for example. Note, for instance, how you have access to the state when a info message is received in handle_info/2 of your module. Also the proper syntax for accessing record fields is Expr#Name.Field which would translate to DBref = State#state.dbref in your case. I hope that clarifies things a little :) Regards, Gustav Simonsson On Wed, 2011-10-05 at 16:26 +0200, Michael Weibel - Amiado Group wrote: > Hi Gustav, > > Wow, thanks for your _quick_ reply! :) > Yeah at first I thought also that I could use that but didn't really figure out how to do that (maybe I don't really understand records yet). > > Because the "log_packet_send" is called from ejabberd (it's a hook) and from this function, as you see, I call some other functions until it arrives at the "insert_row" function. > In the insert_row function I'd need the DB Reference as you see. > There, can I just use it like this: > DBref = #state.dbref > > Is this the correct way? > > thanks, > Michael > > Am 05.10.2011 um 15:54 schrieb Gustav Simonsson: > > > Hi Michael! > > > > On line 48 in your code you store the connection information used in > > open_mysql_connection/1 in the state of the server. Perhaps you can > > make use of that? > > > > Regards, > > Gustav Simonsson > > > > On Wed, 2011-10-05 at 15:36 +0200, Michael Weibel - Amiado Group wrote: > >> Hi all, > >> > >> I'm new to erlang, new to this list and I hope here's the right place to ask this question. > >> > >> I'm currently implementing a module for ejabberd where I need to log certain packets to a MySQL DB and I'm using the MySQL Native Driver for it. > >> > >> My current implementation is on github: > >> https://github.com/amiadogroup/mod_log_chat_mysql5/blob/master/src/mod_log_chat_mysql5.erl > >> > >> In this file you see that I'm opening the MySQL Connection in the function "open_mysql_connection" and assign it to a ETS Table because I didn't find another way to store the DB Reference (I tried something with records but didn't succeed). > >> > >> This worked pretty well but I ran into the problem that after some days/weeks the ets table didn't have the DB Reference anymore. > >> To prevent this, I want to create functions which setup the DB Connection again, if it doesn't have it anymore. > >> > >> As I'm trying to do this I run into the problem that the informations about how to connect to the db is only in the init-function available and I don't really want to store this information also in the ets table. > >> > >> Can anyone help me or point me to a site on how to do this? > >> > >> Thanks, > >> Michael > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > From knut.nesheim@REDACTED Wed Oct 5 17:27:21 2011 From: knut.nesheim@REDACTED (Knut Nesheim) Date: Wed, 5 Oct 2011 17:27:21 +0200 Subject: [erlang-questions] Timeout in erl_call Message-ID: Hello list, We are seeing intermittent timeouts when using erl_call. The error message we reiceve is "erl_call: unable to start node, error = -5", which I traced down to meaning ERL_TIMEOUT when waiting for a response from the node. It only happens on one of two machines, which are identical in hardware and software. We use erl_call to every 5 minutes retrieve system stats together with fetching some statistical data from a process. There is no computation, only fetching and organising the information. Whenever I run this code manually it runs so fast that there is no noticable delay. From time to time, like 5-10 times per day erl_call will however crash with a timeout. We run erl_call and the node on the same machine, but we are using fully qualified names. We cannot find any problem in general with the machine, the erlang node does not log anything and we are unable to reproduce the problem. I do not believe the code being executed inside the node by erl_call takes so long to return that it is causing the timeout. My only "measurement" of this however is running it manually and seeing that it executes without noticeable delay. Does anyone have any idea of what the problem might be? Or an idea on how to narrow it down? We could use different means of communicating with the node (http, tcp socket, etc), but I would like to understand what is going wrong here before changing anything. Knut -- Engineering http://www.wooga.com | phone +49 151 57202523 | fax +49-30-8964 9064 wooga GmbH | Saarbruecker Str. 38 | 10405 Berlin | Germany Sitz der Gesellschaft: Berlin; HRB 117846 B Registergericht Berlin-Charlottenburg Geschaeftsfuehrung: Jens Begemann, Philipp Moeser From joelr1@REDACTED Wed Oct 5 17:33:47 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 5 Oct 2011 16:33:47 +0100 Subject: [erlang-questions] Timeout in erl_call In-Reply-To: References: Message-ID: On Oct 5, 2011, at 4:27 PM, Knut Nesheim wrote: > We cannot find any problem in general with the machine, the erlang > node does not log anything and we are unable to reproduce the problem. Is there heavy traffic between the nodes? -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From g@REDACTED Wed Oct 5 17:34:41 2011 From: g@REDACTED (Garrett Smith) Date: Wed, 5 Oct 2011 10:34:41 -0500 Subject: [erlang-questions] Newbie Question: Module with DB Connection as a "global state" In-Reply-To: <12D9DB35-E8F9-42FC-9B40-CBE981860AC2@amiadogroup.com> References: <12D9DB35-E8F9-42FC-9B40-CBE981860AC2@amiadogroup.com> Message-ID: Hi Michael, On Wed, Oct 5, 2011 at 8:36 AM, Michael Weibel - Amiado Group wrote: > Hi all, > > I'm new to erlang, new to this list and I hope here's the right place to ask this question. Welcome! > > I'm currently implementing a module for ejabberd where I need to log certain packets to a MySQL DB and I'm using the MySQL Native Driver for it. > > My current implementation is on github: > https://github.com/amiadogroup/mod_log_chat_mysql5/blob/master/src/mod_log_chat_mysql5.erl > > In this file you see that I'm opening the MySQL Connection in the function "open_mysql_connection" and assign it to a ETS Table because I didn't find another way to store the DB Reference (I tried something with records but didn't succeed). > > This worked pretty well but I ran into the problem that after some days/weeks the ets table didn't have the DB Reference anymore. > To prevent this, I want to create functions which setup the DB Connection again, if it doesn't have it anymore. You typically don't want to do this. There's a much, much better way... > As I'm trying to do this I run into the problem that the informations about how to connect to the db is only in the init-function available and I don't really want to store this information also in the ets table. The "go to" pattern in Erlang for what you're trying to do is a supervised process. Using standard OTP facilities, you get this: - Registration of process IDs using atoms (analogous to you storing a connection PID in ets) - Well defined "init" logic that's automatically applied when a process needs to be restarted - Encapsulation of the messy details of handling service provisioning Fortunately, the MySQL driver (at the more recent ones - there are several lineages out there) provides an OTP compliant connection process, so you can do something as simple as this: -module(my_db). -export([start_link/0, fetch/1]). start_link() -> Host = "localhost", Port = 3306, Db = "`my-db`", User = "root", Password = "password", mysql:start_link(?MODULE, Host, Port, User, Password, Db, fun mysql_log/4). fetch(Sql) -> mysql:fetch(?MODULE, Sql). mysql_log(_Module, _Line, _Level, _FormatFun) -> ok. This is a registered process (i.e. there's one of them at runtime) that provides access to your database. The connection config in start_link/0 would typically come from an application config, which you can read using application:get_env/2. You can also create a start_link that takes the params. To use this module as a service, you need to plug it into your supervisory hierarchy. Here's what your top-level supervisor might look like: -module(my_app_sup). -behaviour(supervisor). -export([start_link/0]). -export([init/1]). -define(SERVER, ?MODULE). start_link() -> supervisor:start_link({local, ?SERVER}, ?MODULE, []). init([]) -> {ok, {{one_for_one, 5, 5}, [{my_db, {my_db, start_link, []}, permanent, 5000, worker, [my_db]}]}}. If your my_db service crashes, the supervisor will restart it. The "config" that you'd use for that restart would come from either the OTP app config. (You *could* provide it in the supervisor child spec, but I wouldn't go that way.) Erlang gives you very nice facilities to create a mini "service oriented architecture" -- you want to look for ways to create very opaque services that have drop-dead simple start/restart semantics (i.e. simple as pushing a button) and let supervisors monitor and recover failed services as needed. If for some reason you needed to dynamically configure your MySQL connection params, you could store the config in a database, external file, etc. and force a restart of your my_db process to re-read that config on init. Garrett From alceste@REDACTED Wed Oct 5 17:55:59 2011 From: alceste@REDACTED (Alceste Scalas) Date: Wed, 05 Oct 2011 17:55:59 +0200 Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: References: <8BAED16D-F958-4001-83B4-64CA5AD6DB7F@cs.otago.ac.nz> Message-ID: On Wed, 5 Oct 2011 09:24:19 +0200, Peer Stritzinger wrote: > I have to admit I was not aware of this. OTOH it seems not to > be available, can't find anything except the paper and EEP7 > which is the foreign function interface to the external number > crunching libraries they invented. Hi, I'm one of the authors of the "HPTC with Erlang" work. You're right, nothing was publicly released except for the FFI implementation described in EEP 7 --- and since the project ended last year, I believe that nothing else will be released in the future. The last bits were a (prototypal) NIF-based FFI implementation [1], together with a request to withdraw EEP 7 (since it was clearly superseded by NIFs) [2]. [1] http://muvara.org/hg/erlang-ffi/ [2] http://erlang.org/pipermail/eeps/2010-July/000292.html > The price you have to pay for the slapped on heavyweight > library is that these usually don't scale up to the number of > processes Erlang can handle. IMHO it mostly depends on: 1. the size of the operands you're working on; 2. the complexity of the foreign functions you're going to call. Our project was primarily focused on real-time numerical computing, and thus we needed a method for quickly calling "simple" numerical foreign functions (such as multiplications of relatively small (portions of) matrices). Those functions, taken alone, would usually return almost immediately: in other words, their execution time was similar to that of regular BIFs. We used BLAS because its optimized implementations are usually "fast enough", but (if necessary) we could have developed our own optimized C code. When more complicated formulas are assembled with repeated FFI calls to those simple functions, then the Erlang scheduler can kick in several times before the final result is obtained, thus guaranteeing VM responsiveness (albeit reducing the general numerical throughput). > Keeping a pool of numerical processes to keep the cores busy > but not too many of them that the OS is upset. Having work > queues that adapt these to the 20k processes. If the native calls performed by those 20k Erlang processes are not "heavy" enough, then introducing work queues may actually increase the Erlang VM load and internal lock contention, thus decreasing responsiveness (wrt plain NIF calls). I suspect that some comparative benchmarking could be useful. > The suggested n-dim matrix type (e.g. a record containing the > metadata and a binary for the data) combined with some NIFs on > these that speed up the parts where Erlang is not so fast. > Keeping in mind not to do too much work in the NIFs at one time > not to block the scheduler. This is exactly what we did for interfacing BLAS and other numerical routines (except that we used our FFI, since NIFs were not yet available). Maybe a next-generation, general-pourpose numerical computing module for Erlang could adopt different strategies depending on the size of the operands passed to its functions: 1. if the vectors/matrices are "small enough", then the native code could be called directly using NIFs; 2. otherwise, the operands could be passed to a separate worker thread, which will later send back its result to the waiting Erlang process (using enif_send()). In the second case, the future NIF extensions planned by OTP folks may be very useful --- see Rickard Green's talk at the SF Bay Area Erlang Factory 2011: http://bit.ly/eH61tX > For real heavy numerical stuff I think the best way is to do > this in the systems are built for this and interface them > somehow to erlang with ports or sockets. Sure, but the problem with this approach is that you may need to constantly (de)serialize and transfer large numerical arrays among the Erlang VM and the external number crunching systems, thus wasting processor cycles, and memory/network bandwidth. Regards, -- Alceste Scalas From knut.nesheim@REDACTED Wed Oct 5 18:00:24 2011 From: knut.nesheim@REDACTED (Knut Nesheim) Date: Wed, 5 Oct 2011 18:00:24 +0200 Subject: [erlang-questions] Timeout in erl_call In-Reply-To: References: Message-ID: On Wed, Oct 5, 2011 at 5:33 PM, Joel Reymont wrote: > > Is there heavy traffic between the nodes? > No, I don't think the traffic should be a problem. From this machine to other machines in the cluster we do around 10 messages per second that are between 2k and 10k in size. Knut -- Engineering http://www.wooga.com | phone +49 151 57202523 | fax +49-30-8964 9064 wooga GmbH | Saarbruecker Str. 38 | 10405 Berlin | Germany Sitz der Gesellschaft: Berlin; HRB 117846 B Registergericht Berlin-Charlottenburg Geschaeftsfuehrung: Jens Begemann, Philipp Moeser From joe@REDACTED Wed Oct 5 18:05:10 2011 From: joe@REDACTED (Joe Williams) Date: Wed, 5 Oct 2011 09:05:10 -0700 Subject: [erlang-questions] rebar and boot files In-Reply-To: <30D8F422-E394-4420-AA6C-5F356670D2BB@gmail.com> References: <30D8F422-E394-4420-AA6C-5F356670D2BB@gmail.com> Message-ID: <9931489AB6B24DEB8E603181BE926CB0@joetify.com> Joel, There isn't a rebar command that will specifically create a boot file but they are generated by reltool/systools during release generation. You probably want systools:make_script/1 and systools:script2boot/1, if you end up creating something that might be a good addition to rebar patches are welcome. Also join the rebar mailing list, http://lists.basho.com/mailman/listinfo/rebar_lists.basho.com Hope this helps. -Joe -- Name: Joseph A. Williams Email: joe@REDACTED Blog: http://www.joeandmotorboat.com/ Twitter: http://twitter.com/williamsjoe On Wednesday, October 5, 2011 at 6:31 AM, Joel Reymont wrote: > How do I tell rebar to generate a boot file? > > Thanks, Joel > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Wed Oct 5 19:36:41 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 5 Oct 2011 18:36:41 +0100 Subject: [erlang-questions] rebar and boot files In-Reply-To: <9931489AB6B24DEB8E603181BE926CB0@joetify.com> References: <30D8F422-E394-4420-AA6C-5F356670D2BB@gmail.com> <9931489AB6B24DEB8E603181BE926CB0@joetify.com> Message-ID: <7CE06B8E-056D-4460-B1B6-FC61DEA7039C@gmail.com> On Oct 5, 2011, at 5:05 PM, Joe Williams wrote: > Also join the rebar mailing list, http://lists.basho.com/mailman/listinfo/rebar_lists.basho.com Would it be impolite to ask rebar questions on this list instead? ;-) -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From maxtqm@REDACTED Wed Oct 5 19:55:11 2011 From: maxtqm@REDACTED (max tan) Date: Thu, 6 Oct 2011 01:55:11 +0800 Subject: [erlang-questions] Is it possible to align binary's byte array to cache line boundary? In-Reply-To: References: Message-ID: Thank you very much Jon! And you Jesper! Through some learning, I'm now sure that Haskell's Foreign Function Interface ( FFI ) provides the level of control I need. That is to say, I can declare a instance type (for example "MyType") of class "Storable", and overload the "sizeOf", "alignment", "peek" and "poke" methods. Then I can provide "MyType" as type parameter when using the facilities offerd by "Foreign.ForeignPtr", "Foreign.Marshal", "Foreign.Ptr", "Foreign.StablePtr", etc. Haskell's FFI will allocate raw memory block of "sizeOf" bytes starting from an address being multiples of "alignment" on a heap managed by Haskell RTS. tqm On Sun, Sep 25, 2011 at 1:35 AM, Jon Watte wrote: > Maybe LazyByteStrings and Erlang binaries approach the performance of > randomly aligned heap blocks in C. > > But, if I do data streaming processing (say, transforming vertices for 3D > graphcs, transcoding media in multimedia, detecting intersections in rigid > body dynamics, etc) then the specific alignment and stride of my data > matters. Neither Erlang nor Haskell allows me to specify that "the first > byte of this byte string is always aligned on a multiple of 16 bytes" (for > architecture-specific SIMD exploitation) or "the first byte of this byte > string is always aligned on a multiple of 64 bytes" (for > architecture-specific cache line exploitation) or "no word within this > binary will share a L3 cache line with any other word allocated in the same > manner" (for architecture-specific cache contention reduction for > spinlocks). > > C allows you to control these things. That's why it's a systems programming > language, and Erlang isn't (although it's a language for programming systems > :-) > > Sincerely, > > jw > > -- > Americans might object: there is no way we would sacrifice our living > standards for the benefit of people in the rest of the world. Nevertheless, > whether we get there willingly or not, we shall soon have lower consumption > rates, because our present rates are unsustainable. > > > > On Thu, Sep 22, 2011 at 1:46 PM, Jesper Louis Andersen > wrote: >> >> On Thu, Sep 22, 2011 at 17:49, max tan wrote: >> >> > >> > I just found Haskell has "Data.ByteString.Lazy", according to a paper >> > titled >> > "Rewriting Haskell Strings": >> > >> > ? A library for ByteStrings is implemented, ? providing a purely >> > functional >> > interface, which approaches the speed of low-level mutable arrays in C. >> > _______________________________________________ >> >> This is true. In many ways, a Lazy ByteString in Haskell corresponds >> to a list of binaries in Erlang, which is a subset of the iolist() >> type in Erlang. If Haskells Lazy Bytestring implementation approaches >> the speed of low-level mutable arrays, so does binaries in Erlang. >> >> My experience is they perform at about the same speed. I don't know >> what happened in my brain when I called Haskell a dynamic language >> though. It is statically typed, and very much so. >> >> >> -- >> J. > > From joe@REDACTED Wed Oct 5 20:00:15 2011 From: joe@REDACTED (Joe Williams) Date: Wed, 5 Oct 2011 11:00:15 -0700 Subject: [erlang-questions] rebar and boot files In-Reply-To: <7CE06B8E-056D-4460-B1B6-FC61DEA7039C@gmail.com> References: <30D8F422-E394-4420-AA6C-5F356670D2BB@gmail.com> <9931489AB6B24DEB8E603181BE926CB0@joetify.com> <7CE06B8E-056D-4460-B1B6-FC61DEA7039C@gmail.com> Message-ID: Not at all, you just might fine more rebar expertise on our mailing list, not to mention rebar committers pay more attention to that list. FWIW, it's very low traffic. -Joe -- Name: Joseph A. Williams Email: joe@REDACTED Blog: http://www.joeandmotorboat.com/ Twitter: http://twitter.com/williamsjoe On Wednesday, October 5, 2011 at 10:36 AM, Joel Reymont wrote: > > On Oct 5, 2011, at 5:05 PM, Joe Williams wrote: > > > Also join the rebar mailing list, http://lists.basho.com/mailman/listinfo/rebar_lists.basho.com > > Would it be impolite to ask rebar questions on this list instead? ;-) > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Wed Oct 5 22:55:43 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 5 Oct 2011 21:55:43 +0100 Subject: [erlang-questions] telling inets to start services Message-ID: How do I tell inets to start httpd when the inets app is brought up? How do I tell reltool to do so in reltool.config? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Wed Oct 5 22:58:49 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 5 Oct 2011 21:58:49 +0100 Subject: [erlang-questions] telling inets to start services In-Reply-To: References: Message-ID: inets, the app, uses the services environment variable. Problem solved! On Oct 5, 2011, at 9:55 PM, Joel Reymont wrote: > How do I tell inets to start httpd when the inets app is brought up? > > How do I tell reltool to do so in reltool.config? -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From comptekki@REDACTED Thu Oct 6 00:21:49 2011 From: comptekki@REDACTED (Wes James) Date: Wed, 5 Oct 2011 16:21:49 -0600 Subject: [erlang-questions] erlang 14B04 - OS X Lion - wxwidgets don't work Message-ID: I was checking to see if wxwidgets 2.9.2 would compile on OS X Lion and I found some information to do that: http://forums.wxwidgets.org/viewtopic.php?f=23&t=30897 ./configure --enable-unicode --enable-debug --disable-shared --with-osx_cocoa --with-macosx-sdk=/Developer/SDKs/MacOSX10.6.sdk --with-macosx-version-min=10.5 and install it. I then ran "configure" in otp_src_R14B04 and it ends up with and error: Can not combine 64bits erlang with wxWidgets on MacOSX, wx will not be useable I noticed during otp configure that it found wx in /usr/local. I also noticed that 2.8 wx was in the otp distro already (but with 2.8 it had the same error). Why can neither of these be used? It seems that 2.9.2 was compiled for x86_64-apple-darwin11.2.0. Thanks, -wes From joelr1@REDACTED Thu Oct 6 01:23:49 2011 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 6 Oct 2011 00:23:49 +0100 Subject: [erlang-questions] ct, rebar and sys.config Message-ID: I have fully rebar-ized OpenPoker and I can successfully build a release and start it. How do I make use of the embedded node built by rebar in Common Test? I resorted to manually starting the app in init_per_group/2 but that doesn't quite work. One particular issue that's biting me is that my configuration values are stashed away in release/files/sys.config. These values are not found when I application:start(openpoker) and I cannot figure out how to pass -config .../sys.config to 'rebar ct'. Any suggestions? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From systemio@REDACTED Thu Oct 6 04:53:58 2011 From: systemio@REDACTED (=?UTF-8?B?c3lzdGVtaW8gc3lzdGVtaW8=?=) Date: Thu, 06 Oct 2011 06:53:58 +0400 Subject: [erlang-questions] =?utf-8?q?Erlang_shell_crashes_when_system_tim?= =?utf-8?q?e_was_changed?= Message-ID: hi all, i've found a strange behaviour. I change OS(win xp sp3) time to some 2055 and start werl.exe (R14B04) and it crashes always. Who knows what is it? -------------- next part -------------- An HTML attachment was scrubbed... URL: From barcojie@REDACTED Thu Oct 6 05:13:48 2011 From: barcojie@REDACTED (Barco You) Date: Thu, 6 Oct 2011 11:13:48 +0800 Subject: [erlang-questions] telling inets to start services In-Reply-To: References: Message-ID: Hi, Joel, What's your solution? Could you detail it? Thanks. Regards, Barco On Oct 6, 2011 4:58 AM, "Joel Reymont" wrote: > inets, the app, uses the services environment variable. > > Problem solved! > > On Oct 5, 2011, at 9:55 PM, Joel Reymont wrote: > >> How do I tell inets to start httpd when the inets app is brought up? >> >> How do I tell reltool to do so in reltool.config? > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Thu Oct 6 07:48:50 2011 From: dangud@REDACTED (Dan Gudmundsson) Date: Thu, 6 Oct 2011 07:48:50 +0200 Subject: [erlang-questions] erlang 14B04 - OS X Lion - wxwidgets don't work In-Reply-To: References: Message-ID: wxWidgets-2.9 is their developer release of the upcoming major 3.0 release. It's is not api compatible with 2.8, so I haven't investigated how much it breaks with the current version of wx driver. But it will probably not work out of the box. If you have spare time to investigate, please do and send me an email. /Dan On Thu, Oct 6, 2011 at 12:21 AM, Wes James wrote: > > I was checking to see if wxwidgets 2.9.2 would compile on OS X Lion > and I found some information to do that: > > http://forums.wxwidgets.org/viewtopic.php?f=23&t=30897 > > ./configure --enable-unicode --enable-debug --disable-shared > --with-osx_cocoa --with-macosx-sdk=/Developer/SDKs/MacOSX10.6.sdk > --with-macosx-version-min=10.5 > > ?and install it. ?I then ran "configure" in otp_src_R14B04 and it > ends up with and error: > > Can not combine 64bits erlang with wxWidgets on MacOSX, wx will not be useable > > I noticed during otp configure that it found wx in /usr/local. ?I also > noticed that 2.8 wx was in the otp distro already ?(but with 2.8 it > had the same error). ?Why can neither of these be used? ?It seems that > 2.9.2 was compiled for x86_64-apple-darwin11.2.0. > > Thanks, > > -wes > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From cgsmcmlxxv@REDACTED Thu Oct 6 08:35:02 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Thu, 6 Oct 2011 09:35:02 +0300 Subject: [erlang-questions] Timeout in erl_call In-Reply-To: References: Message-ID: Hi Knut, Let me see if I understood correctly. The problem is not continuous for the same machine, but only from time to time? If not, check your installation because, as far as I know, node-to-node communication uses RPC which, in this case, needs to be set to accept communication without entering any user/password (key-pair authentication and the processes to belong to the user for which the key was created). Otherwise, if the problem appears and disappears for a machine, there are few things which you can check: 1. If OTP (application/monitor/gen_server), check what your monitor is reporting and see if you can match any crash/attempt to restart a generic server. 2. Check if your listener is not busy. 3. In case of sending to a spawned thread, if the thread is up. ...and so on. You can notice that the more info you give us, the better for us to understand your problem. Parts of the log would be very helpful (just don't forget to replace the sensitive data with generic ones - e.g., mydomain.com with example.com). Cheers, CGS On Wed, Oct 5, 2011 at 7:00 PM, Knut Nesheim wrote: > On Wed, Oct 5, 2011 at 5:33 PM, Joel Reymont wrote: > > > > Is there heavy traffic between the nodes? > > > > No, I don't think the traffic should be a problem. From this machine > to other machines in the cluster we do around 10 messages per second > that are between 2k and 10k in size. > > Knut > -- > Engineering > http://www.wooga.com | phone +49 151 57202523 | fax +49-30-8964 9064 > > wooga GmbH | Saarbruecker Str. 38 | 10405 Berlin | Germany > Sitz der Gesellschaft: Berlin; HRB 117846 B > Registergericht Berlin-Charlottenburg > Geschaeftsfuehrung: Jens Begemann, Philipp Moeser > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Thu Oct 6 09:02:42 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 6 Oct 2011 08:02:42 +0100 Subject: [erlang-questions] customizing the rebar-generated .app file In-Reply-To: <928330DB-23BD-4098-992F-2402DF4DB485@gmail.com> References: <1749100C-2462-4808-B3C7-51F127A19970@gmail.com> <928330DB-23BD-4098-992F-2402DF4DB485@gmail.com> Message-ID: On 5 October 2011 15:29, Joel Reymont wrote: > rebar is being too clever here. > > Suppose you are using git and are in the top level of the project when > building, as you should be. > > rebar will default the version to "git describe --always --tags `git log -n > 1 --pretty=format:%h .`". > > So at the moment if you do {vsn, git} that is what rebar will do. As an alternative, it would be nice if rebar would allow {vsn, {scm, CmdString}} so you can customise this. *BUT*... Rebar already has a few ways to customise what's happening. There are plugins written in Erlang, which can hook into any command and run either before, during or after the command is executed, or can add completely new commands. There are also shell hooks that can run before or after any known command. I suspect there is a need for the maintainers to keep the number of extensibility points/hooks to a minimum in order to remain sane. Nevertheless, I've patched rebar to support this syntax here: https://github.com/hyperthunk/rebar/tree/custom-vsn-commands - I might make a pull request of it but let's see what people think first as it might not be the best approach. You would use it like so: {application, doodah, [ {vsn, {scm, "git rev-parse --short HEAD"}}]}. -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Thu Oct 6 09:06:23 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 6 Oct 2011 08:06:23 +0100 Subject: [erlang-questions] customizing the rebar-generated .app file In-Reply-To: References: <1749100C-2462-4808-B3C7-51F127A19970@gmail.com> <928330DB-23BD-4098-992F-2402DF4DB485@gmail.com> Message-ID: Oh and while you're at it, you might find https://github.com/hyperthunk/rebar_dist_plugin useful as well, for bundling things up once you've generated your releases. On 6 October 2011 08:02, Tim Watson wrote: > On 5 October 2011 15:29, Joel Reymont wrote: > >> rebar is being too clever here. >> >> Suppose you are using git and are in the top level of the project when >> building, as you should be. >> >> rebar will default the version to "git describe --always --tags `git log >> -n 1 --pretty=format:%h .`". >> >> > So at the moment if you do {vsn, git} that is what rebar will do. As an > alternative, it would be nice if rebar would allow {vsn, {scm, CmdString}} > so you can customise this. *BUT*... > > Rebar already has a few ways to customise what's happening. There are > plugins written in Erlang, which can hook into any command and run either > before, during or after the command is executed, or can add completely new > commands. There are also shell hooks that can run before or after any known > command. I suspect there is a need for the maintainers to keep the number of > extensibility points/hooks to a minimum in order to remain sane. > > Nevertheless, I've patched rebar to support this syntax here: > https://github.com/hyperthunk/rebar/tree/custom-vsn-commands - I might > make a pull request of it but let's see what people think first as it might > not be the best approach. You would use it like so: > > {application, doodah, > [ > {vsn, {scm, "git rev-parse --short HEAD"}}]}. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.weibel@REDACTED Thu Oct 6 09:16:25 2011 From: michael.weibel@REDACTED (Michael Weibel - Amiado Group) Date: Thu, 6 Oct 2011 09:16:25 +0200 Subject: [erlang-questions] Newbie Question: Module with DB Connection as a "global state" In-Reply-To: References: <12D9DB35-E8F9-42FC-9B40-CBE981860AC2@amiadogroup.com> Message-ID: @Gustav: Thanks for your hints, very appreciated @Garett, Thanks for your help. I'm currently trying to implement it that way :) - Michael 2011/10/5 Garrett Smith > Hi Michael, > > On Wed, Oct 5, 2011 at 8:36 AM, Michael Weibel - Amiado Group > wrote: > > Hi all, > > > > I'm new to erlang, new to this list and I hope here's the right place to > ask this question. > > Welcome! > > > > > I'm currently implementing a module for ejabberd where I need to log > certain packets to a MySQL DB and I'm using the MySQL Native Driver for it. > > > > My current implementation is on github: > > > https://github.com/amiadogroup/mod_log_chat_mysql5/blob/master/src/mod_log_chat_mysql5.erl > > > > In this file you see that I'm opening the MySQL Connection in the > function "open_mysql_connection" and assign it to a ETS Table because I > didn't find another way to store the DB Reference (I tried something with > records but didn't succeed). > > > > This worked pretty well but I ran into the problem that after some > days/weeks the ets table didn't have the DB Reference anymore. > > To prevent this, I want to create functions which setup the DB Connection > again, if it doesn't have it anymore. > > You typically don't want to do this. There's a much, much better way... > > > As I'm trying to do this I run into the problem that the informations > about how to connect to the db is only in the init-function available and I > don't really want to store this information also in the ets table. > > The "go to" pattern in Erlang for what you're trying to do is a > supervised process. Using standard OTP facilities, you get this: > > - Registration of process IDs using atoms (analogous to you storing a > connection PID in ets) > - Well defined "init" logic that's automatically applied when a > process needs to be restarted > - Encapsulation of the messy details of handling service provisioning > > Fortunately, the MySQL driver (at the more recent ones - there are > several lineages out there) provides an OTP compliant connection > process, so you can do something as simple as this: > > -module(my_db). > > -export([start_link/0, fetch/1]). > > start_link() -> > Host = "localhost", > Port = 3306, > Db = "`my-db`", > User = "root", > Password = "password", > mysql:start_link(?MODULE, Host, Port, User, Password, Db, fun > mysql_log/4). > > fetch(Sql) -> > mysql:fetch(?MODULE, Sql). > > mysql_log(_Module, _Line, _Level, _FormatFun) -> > ok. > > This is a registered process (i.e. there's one of them at runtime) > that provides access to your database. > > The connection config in start_link/0 would typically come from an > application config, which you can read using application:get_env/2. > You can also create a start_link that takes the params. > > To use this module as a service, you need to plug it into your > supervisory hierarchy. Here's what your top-level supervisor might > look like: > > -module(my_app_sup). > > -behaviour(supervisor). > > -export([start_link/0]). > > -export([init/1]). > > -define(SERVER, ?MODULE). > > start_link() -> > supervisor:start_link({local, ?SERVER}, ?MODULE, []). > > init([]) -> > {ok, {{one_for_one, 5, 5}, > [{my_db, {my_db, start_link, []}, > permanent, 5000, worker, [my_db]}]}}. > > If your my_db service crashes, the supervisor will restart it. The > "config" that you'd use for that restart would come from either the > OTP app config. (You *could* provide it in the supervisor child spec, > but I wouldn't go that way.) > > Erlang gives you very nice facilities to create a mini "service > oriented architecture" -- you want to look for ways to create very > opaque services that have drop-dead simple start/restart semantics > (i.e. simple as pushing a button) and let supervisors monitor and > recover failed services as needed. > > If for some reason you needed to dynamically configure your MySQL > connection params, you could store the config in a database, external > file, etc. and force a restart of your my_db process to re-read that > config on init. > > Garrett > -- Michael Weibel Entwickler Amiado Group AG (Axel Springer Schweiz) F?rrlibuckstrasse 110 CH-8005 Z?rich Zentrale +41 (0)44 508 23 23 Direkt +41 44 508 23 18 eMail michael.weibel@REDACTED www.amiadogroup.com | www.students.ch | www.usgang.ch | www.partyguide.ch | www.playbay.ch -------------- next part -------------- An HTML attachment was scrubbed... URL: From ms@REDACTED Thu Oct 6 11:34:40 2011 From: ms@REDACTED (Martin Sandiford) Date: Thu, 6 Oct 2011 20:04:40 +1030 Subject: [erlang-questions] Erlang shell crashes when system time was changed In-Reply-To: References: Message-ID: Just a guess, but do dates after January 2038 cause a crash? http://en.wikipedia.org/wiki/Unix_epoch Martin On Thu, Oct 6, 2011 at 1:23 PM, systemio systemio wrote: > hi all, > i've found a strange behaviour. I change OS(win xp sp3) time to some 2055 > and start werl.exe (R14B04) and it crashes always. > Who knows what is it? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgsmcmlxxv@REDACTED Thu Oct 6 11:47:04 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Thu, 6 Oct 2011 12:47:04 +0300 Subject: [erlang-questions] Erlang shell crashes when system time was changed In-Reply-To: References: Message-ID: LOL! Either foreseeing the end of the world, either the version is not expected to have such a lifetime span, or simply time functions explode numerically. It may be that MS Windows will be replaced with MS Forcefield by that time. :D On Thu, Oct 6, 2011 at 12:34 PM, Martin Sandiford wrote: > Just a guess, but do dates after January 2038 cause a crash? > > http://en.wikipedia.org/wiki/Unix_epoch > > Martin > > On Thu, Oct 6, 2011 at 1:23 PM, systemio systemio wrote: > >> hi all, >> i've found a strange behaviour. I change OS(win xp sp3) time to some 2055 >> and start werl.exe (R14B04) and it crashes always. >> Who knows what is it? >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peerst@REDACTED Thu Oct 6 13:28:25 2011 From: peerst@REDACTED (Peer Stritzinger) Date: Thu, 6 Oct 2011 13:28:25 +0200 Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: References: <8BAED16D-F958-4001-83B4-64CA5AD6DB7F@cs.otago.ac.nz> Message-ID: On Wed, Oct 5, 2011 at 5:55 PM, Alceste Scalas wrote: > > Hi, I'm one of the authors of the "HPTC with Erlang" work. > You're right, nothing was publicly released except for the FFI > implementation described in EEP 7 --- and since the project ended > last year, I believe that nothing else will be released in the > future. Its a pity we have to start over with this. I fear the number of people interested in numerical (realtime) capabilities is limited, so we will invent part of the wheel again and again. >> The price you have to pay for the slapped on heavyweight >> library is that these usually don't scale up to the number of >> processes Erlang can handle. > > IMHO it mostly depends on: > > ? ?1. the size of the operands you're working on; > > ? ?2. the complexity of the foreign functions you're going to > ? ? ? call. I agree. Thats one thing that makes it hard to find a generic solution to numerics needs in Erlang. > Our project was primarily focused on real-time numerical > computing, and thus we needed a method for quickly calling > "simple" numerical foreign functions (such as multiplications of > relatively small (portions of) matrices). ?Those functions, taken > alone, would usually return almost immediately: in other words, > their execution time was similar to that of regular BIFs. ?We > used BLAS because its optimized implementations are usually > "fast enough", but (if necessary) we could have developed > our own optimized C code. I was looking at more heavyweight BLAS implementations which do internal thread management to use the cores. Should have looked at simpler BLAS implementations which are thread safe and single threaded. > When more complicated formulas are assembled with repeated FFI > calls to those simple functions, then the Erlang scheduler can > kick in several times before the final result is obtained, thus > guaranteeing VM responsiveness (albeit reducing the general > numerical throughput). The unavoidable trade off between real-time and throughput optimization. > If the native calls performed by those 20k Erlang processes are > not "heavy" enough, then introducing work queues may actually > increase the Erlang VM load and internal lock contention, thus > decreasing responsiveness (wrt plain NIF calls). ?I suspect that > some comparative benchmarking could be useful. I'm currently experimenting with a n-dim array module in Erlang that uses the metadata + binary buffer approach. Building all operations I need in pure Erlang first and find places to optimized in NIF's. Probably won't use a external library since my numerics needs are pretty specialized (e.g. lots of multiplying bit vectors with float matrices). > Maybe a next-generation, general-pourpose numerical computing > module for Erlang could adopt different strategies depending on > the size of the operands passed to its functions: > > ?1. if the vectors/matrices are "small enough", then the native > ? ? code could be called directly using NIFs; This automation would probably be machine dependent. I can imagine that the basic matrix operations can be handled like this, probably auto-split in sub-matrix operations. Probably needs some learning phase to find the characteristics of the machine. Basically the ratio NIF overhead vs. BLAS-speed has to be measured. > ?2. otherwise, the operands could be passed to a separate worker > ? ? thread, which will later send back its result to the waiting > ? ? Erlang process (using enif_send()). > > In the second case, the future NIF extensions planned by OTP > folks may be very useful --- see Rickard Green's talk at the SF > Bay Area Erlang Factory 2011: http://bit.ly/eH61tX This would be useful for an intermediate runtime of routines. >> For real heavy numerical stuff I think the best way is to do >> this in the systems are built for this and interface them >> somehow to erlang with ports or sockets. > > Sure, but the problem with this approach is that you may need to > constantly (de)serialize and transfer large numerical arrays > among the Erlang VM and the external number crunching systems, > thus wasting processor cycles, and memory/network bandwidth. For runtimes in the minutes to hours range and very complicated code this is probably still the way to go. There is always the question of Erlang VM stability for the heavy numerical stuff. Ports are very nice from the dependability standpoint. Which is probably a issue for the trading system example that initiated the thread. For my application I'll start from the Erlang side trying to define a nice API for n-dimensional fixed element size (sub byte sizes allowed) matrices with some basic operations defined for them. Then I'll look at the minimum amount of NIF support needed to make this run at least at modest speeds. I'll publish my code early hoping others might join in. Regards Peer Stritzinger > > Regards, > -- > Alceste Scalas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From spawn.think@REDACTED Thu Oct 6 13:58:34 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Thu, 6 Oct 2011 13:58:34 +0200 Subject: [erlang-questions] Timeout in erl_call In-Reply-To: References: Message-ID: Hi Knut, Are you using the option -s with erl_call? This situation could occur if : 1) the new node started by erl_call can't connect to your live node (because of cookies for example, but in that case u should find a log about connection attempt from disallowed node) 2) there's a started node for communication, and erl_call can't connect to it to do the next call It might be helpful also if you provide more details, using -d option (debug) for erl_call On Thu, Oct 6, 2011 at 8:35 AM, CGS wrote: > Hi Knut, > > Let me see if I understood correctly. The problem is not continuous for the > same machine, but only from time to time? If not, check your installation > because, as far as I know, node-to-node communication uses RPC which, in > this case, needs to be set to accept communication without entering any > user/password (key-pair authentication and the processes to belong to the > user for which the key was created). Otherwise, if the problem appears and > disappears for a machine, there are few things which you can check: > 1. If OTP (application/monitor/gen_server), check what your monitor is > reporting and see if you can match any crash/attempt to restart a generic > server. > 2. Check if your listener is not busy. > 3. In case of sending to a spawned thread, if the thread is up. > ...and so on. You can notice that the more info you give us, the better for > us to understand your problem. Parts of the log would be very helpful (just > don't forget to replace the sensitive data with generic ones - e.g., > mydomain.com with example.com). > > Cheers, > CGS > > > > > On Wed, Oct 5, 2011 at 7:00 PM, Knut Nesheim wrote: > >> On Wed, Oct 5, 2011 at 5:33 PM, Joel Reymont wrote: >> > >> > Is there heavy traffic between the nodes? >> > >> >> No, I don't think the traffic should be a problem. From this machine >> to other machines in the cluster we do around 10 messages per second >> that are between 2k and 10k in size. >> >> Knut >> -- >> Engineering >> http://www.wooga.com | phone +49 151 57202523 | fax +49-30-8964 9064 >> >> wooga GmbH | Saarbruecker Str. 38 | 10405 Berlin | Germany >> Sitz der Gesellschaft: Berlin; HRB 117846 B >> Registergericht Berlin-Charlottenburg >> Geschaeftsfuehrung: Jens Begemann, Philipp Moeser >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Thu Oct 6 14:05:24 2011 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 6 Oct 2011 13:05:24 +0100 Subject: [erlang-questions] telling inets to start services In-Reply-To: References: Message-ID: On Oct 6, 2011, at 4:13 AM, Barco You wrote: > What's your solution? Could you detail it? Thanks. Add this to your .app or .config file: {inets, [{services, [{httpd, [{modules, [mod_alias, ?]}, ?]}]}]} -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From egil@REDACTED Thu Oct 6 14:18:35 2011 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Thu, 6 Oct 2011 14:18:35 +0200 Subject: [erlang-questions] INFORMATION: New Git branch-names and strategy Message-ID: <4E8D9C9B.2080805@erlang.org> A new branching strategy is now in place for Erlang/OTP development and patch submissions. A detailed description can be found on our Github Wiki for the Erlang/OTP project. - https://github.com/erlang/otp/wiki/Branches - https://github.com/erlang/otp/wiki/Submitting-patches To summarize. We will have two development branches: one for features and one for corrections. The 'master' branch will now reflect feature development aimed for our next major release. After our R15 release we will also have a 'maint' branch which will include any corrections and minor features submitted by the community. The 'maint' branch will progress and become our next minor release (e.g. R15B01). Regards, Bj?rn-Egil and Henrik Erlang/OTP From peerst@REDACTED Thu Oct 6 14:59:53 2011 From: peerst@REDACTED (Peer Stritzinger) Date: Thu, 6 Oct 2011 14:59:53 +0200 Subject: [erlang-questions] Keeping massive concurrency when interfacing with C In-Reply-To: References: <8BAED16D-F958-4001-83B4-64CA5AD6DB7F@cs.otago.ac.nz> Message-ID: On Wed, Oct 5, 2011 at 5:55 PM, Alceste Scalas wrote: > > In the second case, the future NIF extensions planned by OTP > folks may be very useful --- see Rickard Green's talk at the SF > Bay Area Erlang Factory 2011: http://bit.ly/eH61tX Just watched the video and read the slides of this talk. This would be perfect to handle more heavyweight numerical library support. Especially the dirty schedulers look very promising for this. Also the native processes which make it easy to run untrusted large libraries in their own node. > 1. if the vectors/matrices are "small enough", then the native > code could be called directly using NIFs; > ?2. otherwise, the operands could be passed to a separate worker > ? ? thread, which will later send back its result to the waiting > ? ? Erlang process (using enif_send()). This won't be necessary if dirty schedulers are used if you look at slide 25 of http://www.erlang-factory.com/upload/presentations/377/RickardGreen-NativeInterface.pdf "Example of dirty scheduler use" you can see this two step solution working with dirty schedulers without sending of messages and other complications. This is perfect for the numerical libraries. Cheers -- Peer Stritzinger From knutin@REDACTED Thu Oct 6 16:05:44 2011 From: knutin@REDACTED (Knut Nesheim) Date: Thu, 6 Oct 2011 16:05:44 +0200 Subject: [erlang-questions] Timeout in erl_call In-Reply-To: References: Message-ID: On Thu, Oct 6, 2011 at 1:58 PM, Ahmed Omar wrote: > Are you using the option -s with erl_call? > This situation could occur if : > 1) the new node started by erl_call can't connect to your live node (because > of cookies for example, but in that case u should find a log about > connection attempt from disallowed node) > 2) there's a started node for communication, and erl_call can't connect to > it to do the next call Yes, we are using "-s". The second situation you describe turns out to be our problem. If we are executing two erl_calls at the same time the last one will crash. Most of the time the code executed inside the node was returning so fast, that this never was a problem, but at certain times there would be a 10x spike in latency which would cause the two processes to run at the same time. The second process would then time out. Not running these at the same time solved our problem. Thanks to everybody who helped out. Regards Knut > > It might be helpful also if you provide more details, using -d option > (debug) for erl_call > On Thu, Oct 6, 2011 at 8:35 AM, CGS wrote: >> >> Hi Knut, >> Let me see if I understood correctly. The problem is not continuous for >> the same machine, but only from time to time? If not, check your >> installation because, as far as I know, node-to-node communication uses RPC >> which, in this case, needs to be set to accept communication without >> entering any user/password (key-pair authentication and the processes to >> belong to the user for which the key was created). Otherwise, if the problem >> appears and disappears for a machine, there are few things which you can >> check: >> 1. If OTP (application/monitor/gen_server), check what your monitor is >> reporting and see if you can match any crash/attempt to restart a generic >> server. >> 2. Check if your listener is not busy. >> 3. In case of sending to a spawned thread, if the thread is up. >> ...and so on. You can notice that the more info you give us, the better >> for us to understand your problem. Parts of the log would be very helpful >> (just don't forget to replace the sensitive data with generic ones - e.g., >> mydomain.com with example.com). >> Cheers, >> CGS >> >> >> >> On Wed, Oct 5, 2011 at 7:00 PM, Knut Nesheim >> wrote: >>> >>> On Wed, Oct 5, 2011 at 5:33 PM, Joel Reymont wrote: >>> > >>> > Is there heavy traffic between the nodes? >>> > >>> >>> No, I don't think the traffic should be a problem. From this machine >>> to other machines in the cluster we do around 10 messages per second >>> that are between 2k and 10k in size. >>> >>> Knut >>> -- >>> Engineering >>> http://www.wooga.com | phone +49 151 57202523 | fax +49-30-8964 9064 >>> >>> wooga GmbH | Saarbruecker Str. 38 | 10405 Berlin | Germany >>> Sitz der Gesellschaft: Berlin; HRB 117846 B >>> Registergericht Berlin-Charlottenburg >>> Geschaeftsfuehrung: Jens Begemann, Philipp Moeser >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Best Regards, > - Ahmed Omar > http://nl.linkedin.com/in/adiaa > Follow me on twitter > @spawn_think > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From wiener.guy@REDACTED Thu Oct 6 16:12:03 2011 From: wiener.guy@REDACTED (Guy Wiener) Date: Thu, 6 Oct 2011 16:12:03 +0200 Subject: [erlang-questions] Erlang on ARM9? In-Reply-To: <20111003190352.GA2709@corelatus.se> References: <20111003123303.GA16599@corelatus.se> <4E89B9DA.5090608@erlang.org> <20111003190352.GA2709@corelatus.se> Message-ID: Continuing the saga: I managed to run the two first steps of the cross-compilation, i.e., configure and build, but when I run "./otp_build release -a some_dir", that dir is empty after the script runs. What's the proper way to run the cross-compilation scripts for release? I'm obviously doing something wrong. Thanks again, Guy On Mon, Oct 3, 2011 at 9:03 PM, Matthias Lang wrote: > On Monday, October 03, Guy Wiener wrote: > > > My mistake: The build did not get stuck on some library, but erts is > > dependent on libcurses, which is not a part of the toolchain that I use > > (Sourcery lite). Can I compile Erlang somehow without this lib? Curses! > :) > > Have you tried passing > > --without-termcap > > when you run configure? > > Matt > -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Thu Oct 6 16:38:11 2011 From: comptekki@REDACTED (Wes James) Date: Thu, 6 Oct 2011 08:38:11 -0600 Subject: [erlang-questions] erlang 14B04 - OS X Lion - wxwidgets don't work In-Reply-To: References: Message-ID: On Wed, Oct 5, 2011 at 11:48 PM, Dan Gudmundsson wrote: > ?wxWidgets-2.9 is their developer release of the upcoming major 3.0 release. > It's is not api compatible with 2.8, so I haven't investigated how > much it breaks > with the current version of wx driver. But it will probably not work > out of the box. > > If you have spare time to investigate, please do and send me an email. > > /Dan I went back and removed 2.9 from /usr/local and tried to make 2.8 all version and 2.8 mac version but the both fail. I guess 2.9 has been worked on so it will make on Lion, but as noted before, it won't work with otp 14B02 make. I see i386-darwin11 during the compile so maybe it's using 32bit compile and not 64bit compile. -wes > On Thu, Oct 6, 2011 at 12:21 AM, Wes James wrote: >> >> I was checking to see if wxwidgets 2.9.2 would compile on OS X Lion >> and I found some information to do that: >> >> http://forums.wxwidgets.org/viewtopic.php?f=23&t=30897 >> >> ./configure --enable-unicode --enable-debug --disable-shared >> --with-osx_cocoa --with-macosx-sdk=/Developer/SDKs/MacOSX10.6.sdk >> --with-macosx-version-min=10.5 >> >> ?and install it. ?I then ran "configure" in otp_src_R14B04 and it >> ends up with and error: >> >> Can not combine 64bits erlang with wxWidgets on MacOSX, wx will not be useable >> >> I noticed during otp configure that it found wx in /usr/local. ?I also >> noticed that 2.8 wx was in the otp distro already ?(but with 2.8 it >> had the same error). ?Why can neither of these be used? ?It seems that >> 2.9.2 was compiled for x86_64-apple-darwin11.2.0. >> >> Thanks, >> >> -wes >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From comptekki@REDACTED Thu Oct 6 17:38:20 2011 From: comptekki@REDACTED (Wes James) Date: Thu, 6 Oct 2011 09:38:20 -0600 Subject: [erlang-questions] erlang 14B04 - OS X Lion - wxwidgets don't work In-Reply-To: References: Message-ID: On Wed, Oct 5, 2011 at 11:48 PM, Dan Gudmundsson wrote: > ?wxWidgets-2.9 is their developer release of the upcoming major 3.0 release. > It's is not api compatible with 2.8, so I haven't investigated how > much it breaks > with the current version of wx driver. But it will probably not work > out of the box. When I do configure for wxWidgets 2.9, it get this: wxWidgets-2.9.2]$ ./configure checking build system type... x86_64-apple-darwin11.2.0 checking host system type... x86_64-apple-darwin11.2.0 ..... When I do configure for erlang, I git this: ./configure Ignoring the --cache-file argument since it can cause the system to be erroneously configured Disabling caching checking build system type... i386-apple-darwin11.2.0 checking host system type... i386-apple-darwin11.2.0 ..... Why is erlang not configuring to x64? The message at the end seems like erlang is compiled to 64bit when it says "64bits" in the no wx warning. I found --enable-darwin-64bit to add to ./configure, but it still shows i386.... during configuration. -wes From michal.niec@REDACTED Thu Oct 6 19:09:29 2011 From: michal.niec@REDACTED (Michal Niec) Date: Thu, 6 Oct 2011 19:09:29 +0200 Subject: [erlang-questions] Erlang R14B04 packages Message-ID: <61100B95-D092-4087-AEC8-DD3D828E61CE@erlang-solutions.com> Hi Erlangers! We've just released our Erlang R14B04 packages for Mac OS X, Debian 6, Ubuntu 11.04, Fedora 15 and Centos 6. They are available at http://bit.ly/qa0oWS . Regards, Michal Niec -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Thu Oct 6 19:11:41 2011 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 6 Oct 2011 18:11:41 +0100 Subject: [erlang-questions] Erlang R14B04 packages In-Reply-To: <61100B95-D092-4087-AEC8-DD3D828E61CE@erlang-solutions.com> References: <61100B95-D092-4087-AEC8-DD3D828E61CE@erlang-solutions.com> Message-ID: <2F50F4B0-F6F9-497C-845E-EC9083E23597@gmail.com> Do you still have to build with -O0 on the Mac? On Oct 6, 2011, at 6:09 PM, Michal Niec wrote: > Hi Erlangers! > > We've just released our Erlang R14B04 packages for Mac OS X, Debian 6, Ubuntu 11.04, Fedora 15 and Centos 6. They are available at http://bit.ly/qa0oWS . -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From qrilka@REDACTED Thu Oct 6 19:12:20 2011 From: qrilka@REDACTED (Kirill Zaborsky) Date: Thu, 6 Oct 2011 21:12:20 +0400 Subject: [erlang-questions] Erlang R14B04 packages In-Reply-To: <61100B95-D092-4087-AEC8-DD3D828E61CE@erlang-solutions.com> References: <61100B95-D092-4087-AEC8-DD3D828E61CE@erlang-solutions.com> Message-ID: How is Benjamin Nortier connected to the releases? Best regars, Kirill Zaborsky 2011/10/6 Michal Niec > Hi Erlangers! > > We've just released our Erlang R14B04 packages for Mac OS X, Debian 6, > Ubuntu 11.04, Fedora 15 and Centos 6. They are available at > http://bit.ly/qa0oWS . > > Regards, > Michal Niec > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michal.niec@REDACTED Thu Oct 6 19:12:41 2011 From: michal.niec@REDACTED (Michal Niec) Date: Thu, 6 Oct 2011 19:12:41 +0200 Subject: [erlang-questions] Erlang R14B04 packages In-Reply-To: <2F50F4B0-F6F9-497C-845E-EC9083E23597@gmail.com> References: <61100B95-D092-4087-AEC8-DD3D828E61CE@erlang-solutions.com> <2F50F4B0-F6F9-497C-845E-EC9083E23597@gmail.com> Message-ID: No, we used gcc-4.2 to build this package. On 6 pa? 2011, at 19:11, Joel Reymont wrote: > Do you still have to build with -O0 on the Mac? > > On Oct 6, 2011, at 6:09 PM, Michal Niec wrote: > >> Hi Erlangers! >> >> We've just released our Erlang R14B04 packages for Mac OS X, Debian 6, Ubuntu 11.04, Fedora 15 and Centos 6. They are available at http://bit.ly/qa0oWS . > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > From bfulgham@REDACTED Thu Oct 6 19:47:21 2011 From: bfulgham@REDACTED (Brent Fulgham) Date: Thu, 6 Oct 2011 10:47:21 -0700 Subject: [erlang-questions] Erlang R14B04 packages In-Reply-To: <2F50F4B0-F6F9-497C-845E-EC9083E23597@gmail.com> References: <61100B95-D092-4087-AEC8-DD3D828E61CE@erlang-solutions.com> <2F50F4B0-F6F9-497C-845E-EC9083E23597@gmail.com> Message-ID: On Thu, Oct 6, 2011 at 10:11 AM, Joel Reymont wrote: > Do you still have to build with -O0 on the Mac? I updated the MacPorts build of Erlang and did not have to make that change. I think this fix corrected it: OTP-9547 Erlang/OTP can now be built on MacOS X Lion. -Brent From joelr1@REDACTED Thu Oct 6 19:52:43 2011 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 6 Oct 2011 18:52:43 +0100 Subject: [erlang-questions] Erlang R14B04 packages In-Reply-To: References: <61100B95-D092-4087-AEC8-DD3D828E61CE@erlang-solutions.com> <2F50F4B0-F6F9-497C-845E-EC9083E23597@gmail.com> Message-ID: On Oct 6, 2011, at 6:47 PM, Brent Fulgham wrote: > > I updated the MacPorts build of Erlang and did not have to make that change. Who maintains the Homebrew Erlang recipe? > I think this fix corrected it: > OTP-9547 Erlang/OTP can now be built on MacOS X Lion. Do you by chance have a commit id for this or Github link? I'd love to know what changes were required. Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From garazdawi@REDACTED Thu Oct 6 19:59:55 2011 From: garazdawi@REDACTED (Lukas Larsson) Date: Thu, 6 Oct 2011 19:59:55 +0200 Subject: [erlang-questions] Erlang R14B04 packages In-Reply-To: References: <61100B95-D092-4087-AEC8-DD3D828E61CE@erlang-solutions.com> <2F50F4B0-F6F9-497C-845E-EC9083E23597@gmail.com> Message-ID: Just search the git history for that ticket number and you should find it. Also the binary packages are of course located at: http://www.erlang-solutions.com/section/132/erlang-otp-packages Lukas On Thu, Oct 6, 2011 at 7:52 PM, Joel Reymont wrote: > > On Oct 6, 2011, at 6:47 PM, Brent Fulgham wrote: > > > > I updated the MacPorts build of Erlang and did not have to make that > change. > > Who maintains the Homebrew Erlang recipe? > > > I think this fix corrected it: > > OTP-9547 Erlang/OTP can now be built on MacOS X Lion. > > > Do you by chance have a commit id for this or Github link? > > I'd love to know what changes were required. > > Thanks, Joel > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Thu Oct 6 20:30:50 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 6 Oct 2011 20:30:50 +0200 Subject: [erlang-questions] Profiling tool to find bottleneck on my Erlang code Message-ID: Hi guys, We have at work, a heavily parallel Erlang application running for years now. We would like to concentrate our efforts in the next couple of months in optimizing it bit more (if possible). Could someone please point me to a good "high level profiling tool" (not eprof/fprof which we already now, and find not very sexy) to tracks bottlenecks? Any advice will be appreciated. Regards, Zabrane From spawn.think@REDACTED Thu Oct 6 21:52:02 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Thu, 6 Oct 2011 21:52:02 +0200 Subject: [erlang-questions] Profiling tool to find bottleneck on my Erlang code In-Reply-To: References: Message-ID: Maybe you could take a look at Percept, it's a concurrency profiling tool http://www.erlang.org/doc/apps/percept/percept_ug.html On Thu, Oct 6, 2011 at 8:30 PM, Zabrane Mickael wrote: > Hi guys, > > We have at work, a heavily parallel Erlang application running for years > now. > We would like to concentrate our efforts in the next couple of months in > optimizing it bit more (if possible). > > Could someone please point me to a good "high level profiling tool" (not > eprof/fprof which we already now, > and find not very sexy) to tracks bottlenecks? > > Any advice will be appreciated. > > Regards, > Zabrane > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think -------------- next part -------------- An HTML attachment was scrubbed... URL: From nem@REDACTED Thu Oct 6 22:04:52 2011 From: nem@REDACTED (Geoff Cant) Date: Thu, 6 Oct 2011 13:04:52 -0700 Subject: [erlang-questions] Profiling tool to find bottleneck on my Erlang code In-Reply-To: References: Message-ID: On 2011-10-06, at 11:30 , Zabrane Mickael wrote: > Hi guys, > > We have at work, a heavily parallel Erlang application running for years now. > We would like to concentrate our efforts in the next couple of months in optimizing it bit more (if possible). > > Could someone please point me to a good "high level profiling tool" (not eprof/fprof which we already now, > and find not very sexy) to tracks bottlenecks? > > Any advice will be appreciated. > > Regards, > Zabrane It sounds like your system runs without crashing, which I'm guessing means you don't have a memory bottleneck. This means you probably have something that's CPU bound, or if you're not CPU bound, you probably have an inefficient messaging pattern somewhere that is hampering you. To figure out if you have a CPU problem, I'd take a look at etop/dtop/entop and see if there are processes consuming a lot of reductions. If they're consuming these reductions in proportion to receiving a lot of input messages then they may be fine, but you can look at having multiple processes handle these messages in parallel. If they're just consuming lots of reductions, then check to see if the code in those processes can be made more efficient. Finding a messaging pattern problem is harder - typically you'll need to have an idea of which processes might be involved, and then you need to trace them with dbg or something. Then expect to spend a bunch of time looking over the trace log to figure out how many messages it takes between which processes in order to complete an operation. Then you can figure out if there's a way to do it better. So tool wise - use etop/dtop/entop first to figure out if you have CPU-bound hotspots, then use dbg tracing to investigate further. Cheers, -- Geoff Cant From knutin@REDACTED Thu Oct 6 22:18:25 2011 From: knutin@REDACTED (Knut Nesheim) Date: Thu, 6 Oct 2011 22:18:25 +0200 Subject: [erlang-questions] Profiling tool to find bottleneck on my Erlang code In-Reply-To: References: Message-ID: On Thu, Oct 6, 2011 at 8:30 PM, Zabrane Mickael wrote: > Hi guys, > > We have at work, a heavily parallel Erlang application running for years now. > We would like to concentrate our efforts in the next couple of months in optimizing it bit more (if possible). > > Could someone please point me to a good "high level profiling tool" (not eprof/fprof which we already now, > and find not very sexy) to tracks bottlenecks? I have been working on a tool for our application to accomplish this. It is still very much early stages, but it's already available on github.com/knutin/kprof The idea is to trace the path of execution of a request in your system, as it travels through different tiers and different processes. The runtime[1] of each tier is aggregated per request "id" and tier. The request is identified by a function you provide, so you can for example know the performance per API call, even though they may all travel through the same tiers. For example: "list all invoices" spends 50ms in the database tier and 200ms rendering the pdf response . "project future earnings" spends 500ms in the business logic, but almost nothing in the database, etc. The trace is collected and aggregated inside your node. The timings can be stored in couchdb where you can view them using a couchapp that creates nice graphs and tells you standard deviation, percentiles, throughput, etc. Sending the trace messages to an external node is also something I have looked into. I want it to run continuously in live systems, tracing all requests through a certain entry point or sampling them(sampling is not yet implemented though). I've been working on this project for use in our game servers, which are high throughput/low latency so we care very much about the slow-down introduced by tracing. As it is restricted to only a few functions(typically one in each tier), it is not that critical. With the sampling you can fine tune. Measuring the exact impact is on my TODO list. I would love some feedback on the idea and the project. If you have any needs that is not solved by what I mentioned above please share so I can understand what it would take to make the tool fit more than just our use case. If anyone is interested in working together on the project, this would also be really cool. Thanks Knut [1]: And time spent in gc, if I get around to implementing it > > Any advice will be appreciated. > > Regards, > Zabrane > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From peerst@REDACTED Fri Oct 7 10:08:04 2011 From: peerst@REDACTED (Peer Stritzinger) Date: Fri, 7 Oct 2011 10:08:04 +0200 Subject: [erlang-questions] Profiling tool to find bottleneck on my Erlang code In-Reply-To: References: Message-ID: On Thu, Oct 6, 2011 at 10:18 PM, Knut Nesheim wrote: > On Thu, Oct 6, 2011 at 8:30 PM, Zabrane Mickael wrote: > I have been working on a tool for our application to accomplish this. > It is still very much early stages, but it's already available on > github.com/knutin/kprof > > The idea is to trace the path of execution of a request in your > system, as it travels through different tiers and different processes. How do you trace a request across processes? Do you need some id tagging of the messages etc? Normal gen_server call like requests would probably work, but what about cast like messages. If I want to use kprof for protocol layers, where messages travel up and down the layers freely and a request after travelling down the layers is e.g. answered by a response from outside the system, which makes the top layer function call finally return. Is it possible to trace requests like these without instrumenting all layers and/or the messages? Regards -- Peer Stritzinger From knutin@REDACTED Fri Oct 7 10:53:45 2011 From: knutin@REDACTED (Knut Nesheim) Date: Fri, 7 Oct 2011 10:53:45 +0200 Subject: [erlang-questions] Profiling tool to find bottleneck on my Erlang code In-Reply-To: References: Message-ID: On Fri, Oct 7, 2011 at 10:08 AM, Peer Stritzinger wrote: > On Thu, Oct 6, 2011 at 10:18 PM, Knut Nesheim wrote: > How do you trace a request across processes? ?Do you need some id > tagging of the messages etc? Yes, this is done using "sequential tracing" offered by the VM. When the entry point is executed, a trace token is explicitly set. When this process messages another process, the message contains the trace token and by receiving the message, the trace token is also set in the receiving process. This is handled transparently by the VM, so there is no need to instrument your own code other than calling a kprof function to set the trace token in the entry point. > > Normal gen_server call like requests would probably work, but what > about cast like messages. These are also tagged by the VM and the trace token is also set in the receiving process. However, as I'm interested in measuring runtime of the actual request, the runtime of the process that received the cast should not be included. The profile measures runtime in certain functions, not processes (this could be changed). > > If I want to use kprof for protocol layers, where messages travel up > and down the layers freely and a request after travelling down the > layers is e.g. answered by a response from outside the system, which > makes the top layer function call finally return. ?Is it possible to > trace requests like these without instrumenting all layers and/or the > messages? Yes, this is exactly my use case. As mentioned above, only the entry point for the request needs to be instrumented. At the moment, it is not possible to set a random trace token from within the matchspecs (at least I could not figure out how to do it). If I could do that, no instrumentation would be needed. The profiler is built around the idea to profile each layer, by specifying one or more functions that is called when the request enters this layer. These functions must return, otherwise you cannot know their runtime (gen_server for example is not implemented in this way, so we can only profile the callbacks) Here is an example of how you would configure kprof given a very simple MVC architecture: * http_handler:handle_request/1: entry point, sets the trace token, when this returns the response to the webserver, the request is complete * business_logic:do_stuff/1 * db_access_layer:execute/1 If every request goes through these tiers, you would get a nice overview of where you are spending time broken down per "tier". This is already working. The "tiers" is nothing more fancy than just a specific function to profile, so you can do more fine grained stuff like this: * http_handler:handle_request/1: entry point, sets trace token * http_handler:request_validation/1 * http_interface:handle/1: business logic may start here * state_server:handle_call/3: some process involved in the request, keeping state, but not really a part of any tier * external_system_lookup:execute/1 * stat_lib:regression/1: slow code doing lots of fancy calculation * db_access_layer:get_key/1 * db_access_layer:put_key/1 With the above setup you would get the runtime stats for each request per function you specify. For a function that is executed multiple times within a request, you would get the total runtime (this might require some work still) As you can see, there is nothing really fancy in kprof. My goal is just to put the pieces together, like sequential tracing, aggregation, sampling, statistical analysis and being able to continuously run inside your live system. Regards Knut From vladimir.kirichenko@REDACTED Fri Oct 7 10:56:38 2011 From: vladimir.kirichenko@REDACTED (Volodymyr Kyrychenko) Date: Fri, 7 Oct 2011 11:56:38 +0300 Subject: [erlang-questions] eof or not eof for zero length Message-ID: Hello, Erlangers Having this code 1> {ok, F} = file:open("build.sh",[read,raw,binary]). {ok,{file_descriptor,prim_file,{#Port<0.600>,7}}} 2> file:pread(F, 0, 0). eof 3> file:pread(F, 0, 1). {ok,<<"#">>} makes me think that result of second line is not correct, it obviously should not be EOF, it should be {ok, <<>>}. What do you think? -- Best Regards, Volodymyr Kyrychenko From carlsson.richard@REDACTED Fri Oct 7 11:18:43 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 07 Oct 2011 11:18:43 +0200 Subject: [erlang-questions] eof or not eof for zero length In-Reply-To: References: Message-ID: <4E8EC3F3.7000408@gmail.com> On 10/07/2011 10:56 AM, Volodymyr Kyrychenko wrote: > Hello, Erlangers > > Having this code > > 1> {ok, F} = file:open("build.sh",[read,raw,binary]). > {ok,{file_descriptor,prim_file,{#Port<0.600>,7}}} > 2> file:pread(F, 0, 0). > eof > 3> file:pread(F, 0, 1). > {ok,<<"#">>} > > makes me think that result of second line is not correct, it obviously > should not be EOF, it should be {ok,<<>>}. What do you think? > I reported this to erlang-bugs back in May. No reaction from OTP, though. http://erlang.org/pipermail/erlang-bugs/2011-May/002425.html /Richard From vladimir.kirichenko@REDACTED Fri Oct 7 11:34:53 2011 From: vladimir.kirichenko@REDACTED (Volodymyr Kyrychenko) Date: Fri, 7 Oct 2011 12:34:53 +0300 Subject: [erlang-questions] eof or not eof for zero length In-Reply-To: <4E8EC3F3.7000408@gmail.com> References: <4E8EC3F3.7000408@gmail.com> Message-ID: Another one: 1> {ok, B} = file:read_file("build.sh"). {ok,<<"#!/bin/sh\nrebar compile || exit -1\nmkdir -p out\n\nMODULES=\"common leveldb yaws csv eunit io\"\n\nfor m in $MODUL"...>>} 2> L = length(binary_to_list(B)). 193 3> {ok, F} = file:open("build.sh", [read]). {ok,<0.35.0>} 4> file:pread(F, L+100,10). eof 5> file:pread(F, L+100,0). {ok,[]} Line 5 definitely should be eof. -- Best Regards, Volodymyr Kyrychenko From ward@REDACTED Fri Oct 7 11:43:19 2011 From: ward@REDACTED (Ward Bekker) Date: Fri, 07 Oct 2011 11:43:19 +0200 Subject: [erlang-questions] [ann] Erlang Factory Lite Amsterdam 16 oct Message-ID: <20111007094319.2b6ce02f@groupware.tty.nl> Erlang Solutions, Maxclass and TTY Internet Solutions will be holding an Erlang Factory Lite on the 16 October. The Factory Lite will be a chance for you to learn, socialise and network with some fantastic names. The Erlang Factory Lite is open to everybody with any level of Erlang experience from newbies to experienced developers. Why not come along and meet up with like minded developers and architects or alternatively come along and find out how more firms are using Erlang/OTP to solve previously intractable problems. The event is totally free but does require registration. More info: http://www.erlang-factory.com/conference/amsterdam See you there! -------------- next part -------------- An HTML attachment was scrubbed... URL: From ward@REDACTED Fri Oct 7 11:56:11 2011 From: ward@REDACTED (Ward Bekker) Date: Fri, 07 Oct 2011 11:56:11 +0200 Subject: [erlang-questions] [tutorial] A Basic Full Text Search Server in Erlang Message-ID: <20111007095611.7286ffaa@groupware.tty.nl> I've posted a tutorial on how you could build a, very basic, full text search server with Erlang. Feedback most welcome! http://blog.tty.nl/2011/10/07/a-basic-full-text-search-server-in-erlang/ Enjoy, Ward -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Fri Oct 7 12:02:20 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 7 Oct 2011 11:02:20 +0100 Subject: [erlang-questions] Profiling tool to find bottleneck on my Erlang code In-Reply-To: References: Message-ID: > > > As you can see, there is nothing really fancy in kprof. My goal is > just to put the pieces together, like sequential tracing, aggregation, > sampling, statistical analysis and being able to continuously run > inside your live system. > This is extremely cool. I do want to be able to turn it on and off though, as I might not want it turned on all the time despite the relatively small impact of tracing. Also, it would be nice if it could take advantage of the tracing to a port/socket feature, to minimise impact on the running system. Anyway, thanks for this contribution. It sounds really good and I for one will be keeping a close eye on the project! -------------- next part -------------- An HTML attachment was scrubbed... URL: From tfreeman@REDACTED Fri Oct 7 21:25:25 2011 From: tfreeman@REDACTED (Tim Freeman) Date: Fri, 7 Oct 2011 14:25:25 -0500 Subject: [erlang-questions] failures in supervised processes Message-ID: <20111007142525.2efa7d72@usul> Every document I have located refers to an exit signal being sent to the supervisor. To me, this implies some participation of the process or the runtime on the remote node to indicate failure. My question is about distributed processes that simply die/lose contact. Let's say the remote process is on a computer whose network cable is suddenly removed. What happens then, does the supervisor do its own active checks, what is the fallback mechanism there? Thanks, Tim From joelr1@REDACTED Fri Oct 7 21:28:26 2011 From: joelr1@REDACTED (Joel Reymont) Date: Fri, 7 Oct 2011 20:28:26 +0100 Subject: [erlang-questions] rebar release broken on windows? Message-ID: <2A3E37A4-34C4-4CD3-B69D-008B7CBBF667@gmail.com> Is anyone using 'rebar release' on Windows? How do you work around the errors? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From freza@REDACTED Fri Oct 7 23:04:19 2011 From: freza@REDACTED (Jachym Holecek) Date: Fri, 7 Oct 2011 17:04:19 -0400 Subject: [erlang-questions] failures in supervised processes In-Reply-To: <20111007142525.2efa7d72@usul> References: <20111007142525.2efa7d72@usul> Message-ID: <20111007210419.GA777@circlewave.net> # Tim Freeman 2011-10-07: > Every document I have located refers to an exit signal being sent to the > supervisor. To me, this implies some participation of the process or the > runtime Yes, the runtime generates the exit signal whenever some process with nonempty link-list dies (that's either ordinary links or monitors). > on the remote node to indicate failure. My question is about > distributed processes that simply die/lose contact. > > Let's say the remote process is on a computer whose network cable is suddenly > removed. What happens then, does the supervisor do its own active checks, > what is the fallback mechanism there? Normally you would have a supervisor only take care of local processes. Supervising remote processes is probably technically possible, but isn't something I'd contemplate for a production system (but maybe that's just me). Connection loss is a possibility your code needs to deal with, the right response is too application specific to handle generically: queue incoming requests hoping for better times? revert to backup peer? schedule reconnect attempt and fail all requests meanwhile? etc. OTP is forthcoming in that it gives you all the tools to build whatever solution is right for the system at hand. HTH, -- Jachym From magnus.klaar@REDACTED Sat Oct 8 00:08:04 2011 From: magnus.klaar@REDACTED (Magnus Klaar) Date: Sat, 8 Oct 2011 00:08:04 +0200 Subject: [erlang-questions] failures in supervised processes In-Reply-To: <20111007210419.GA777@circlewave.net> References: <20111007142525.2efa7d72@usul> <20111007210419.GA777@circlewave.net> Message-ID: > > > > on the remote node to indicate failure. My question is about > > distributed processes that simply die/lose contact. > > > > Let's say the remote process is on a computer whose network cable is > suddenly > > removed. What happens then, does the supervisor do its own active > checks, > > what is the fallback mechanism there? > > Normally you would have a supervisor only take care of local processes. > Supervising remote processes is probably technically possible, but isn't > something I'd contemplate for a production system (but maybe that's just > me). > > The distribution protocol transparently support process links and process monitors. If the connection to a remote node is lost all links and monitors associated with a process on the remote node fire with the reason 'noconnection'. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tfreeman@REDACTED Sat Oct 8 09:49:02 2011 From: tfreeman@REDACTED (Tim Freeman) Date: Sat, 8 Oct 2011 02:49:02 -0500 Subject: [erlang-questions] failures in supervised processes In-Reply-To: References: <20111007142525.2efa7d72@usul> <20111007210419.GA777@circlewave.net> Message-ID: <20111008024902.0e8b4254@usul> On Sat, 8 Oct 2011 00:08:04 +0200 Magnus Klaar wrote: > > > > > > > on the remote node to indicate failure. My question is about > > > distributed processes that simply die/lose contact. > > > > > > Let's say the remote process is on a computer whose network cable is > > suddenly > > > removed. What happens then, does the supervisor do its own active > > checks, > > > what is the fallback mechanism there? > > > > Normally you would have a supervisor only take care of local processes. > > Supervising remote processes is probably technically possible, but isn't > > something I'd contemplate for a production system (but maybe that's just > > me). > > > > > The distribution protocol transparently support process links and process > monitors. If the connection to a remote node is lost all links and monitors > associated with a process on the remote node fire with the reason > 'noconnection'. Thankyou for the responses Magnus, Jachym. So it sounds like there is an active connection/session maintained. Is it kept in check with an occasional heartbeat if there is no activity? Thanks, Tim From magnus.klaar@REDACTED Sat Oct 8 17:40:54 2011 From: magnus.klaar@REDACTED (Magnus Klaar) Date: Sat, 8 Oct 2011 17:40:54 +0200 Subject: [erlang-questions] failures in supervised processes In-Reply-To: <20111008024902.0e8b4254@usul> References: <20111007142525.2efa7d72@usul> <20111007210419.GA777@circlewave.net> <20111008024902.0e8b4254@usul> Message-ID: > > > So it sounds like there is an active connection/session maintained. Is it > kept in check with an occasional heartbeat if there is no activity? > > Thanks, > Tim > Correct, a session is associated with each connection in an erlang cluster. A heartbeat/keepalive mechanism is used to detect disconnected/locked nodes. If a node shuts down cleanly the connection is torn down immediately. If you want to dig into the defaults and configuration options the documentation for the kernel application is a good start. It's worth noting that erlang supports additional mechanisms for timeouts. The client libraries for the built in behaviors all use a shorter timeout for request-response interactions by default. The erlang:send_after and erlang:start_timer function and receive ... after MS -> .. end expressions are useful primitives to accomplish the same thing. It isn't necessarily very important where an unresponsive process is located if the unresponsiveness is generating timeout errors further down in the supervision tree. MVH Magnus -------------- next part -------------- An HTML attachment was scrubbed... URL: From fritchie@REDACTED Sat Oct 8 22:54:58 2011 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Sat, 08 Oct 2011 15:54:58 -0500 Subject: [erlang-questions] DTrace plug, was Re: Profiling tool to find bottleneck on my Erlang code In-Reply-To: Message of "Thu, 06 Oct 2011 20:30:50 +0200." Message-ID: <64993.1318107298@snookles.snookles.com> Zabrane Mickael wrote: mz> Could someone please point me to a good "high level profiling tool" mz> (not eprof/fprof which we already now, and find not very sexy) to mz> tracks bottlenecks? Mickael, you didn't say if your application is perhaps CPU-bound or I/O-bound (disk I/O or network I/O) or if that's what you're trying to figure out. OS system measurement tools like "iostat" and "sar" can help answer the above question. (Just in case you haven't considered them ... they're quite useful, especially detailed stats generation by "iostat -xk" on Linux or "sar -ugd" on Solaris.) In collaboration with Michal Ptaszek (Erlang Solutions) and Dustin Sallings (CouchBase), I've been working on adding DTrace probes to the Erlang virtual machine. The latest work is available on the "dtrace-experiment+michal2" branch of the git://github.com/slfritchie/otp.git source repo. I know that this compiles cleanly on a Mac Snow Leopard machine as well as Solaris 10 + GCC 4.2.1.(*) Though there's plenty more probes to add (and testing to do :-), I'll include a copy of the current probe definition file below. If it can help with your search, great! If not, spread the word at least. We'll gladly accept patches for new probes. -Scott (*) I haven't tried yet using Linux + SystemTap's DTrace compatibility mode because SystemTap's user-space tracing is a case study in usability horror and documentation ghostworld. --- snip --- snip --- snip --- snip --- snip --- /* * %CopyrightBegin% * * Copyright Dustin Sallings, Michal Ptaszek, Scott Lystig Fritchie 2011. * All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in * compliance with the License. You should have received a copy of the * Erlang Public License along with this software. If not, it can be * retrieved online at http://www.erlang.org/. * * Software distributed under the License is distributed on an "AS IS" * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See * the License for the specific language governing rights and limitations * under the License. * * %CopyrightEnd% */ provider erlang { /** * Fired when a message is sent from one local process to another. * * @param sender the PID (string form) of the sender * @param receiver the PID (string form) of the receiver * @param size the size of the message being delivered */ probe send(char *sender, char *receiver, uint32_t size); /** * Fired when a message is delivered to a local process. * * @param receiver the PID (string form) of the receiver * @param size the size of the message being delivered * @param queue_len length of the queue of the receiving process */ probe receive(char *receiver, uint32_t size, uint32_t queue_len); /** * Fired when an Eterm structure is being copied. * * @param size the size of the structure */ probe copy_struct(uint32_t size); /** * Fired when an Eterm is being copied onto a process. * * @param proc the PID (string form) of the recipient process * @param size the size of the structure */ probe copy_object(char *proc, uint32_t size); /* PID, Module, Function, Arity */ /** * Fired whenever a user function is being called. * * @param p the PID (string form) of the process * @param mfa the m:f/a of the function * @param depth the stack depth */ probe function_entry(char *p, char *mfa, int depth); /** * Fired whenever a user function returns. * * @param p the PID (string form) of the process * @param mfa the m:f/a of the function * @param depth the stack depth */ probe function_return(char *p, char *mfa, int depth); /** * Fired whenever a Built In Function is called. * * @param p the PID (string form) of the process * @param mfa the m:f/a of the function */ probe bif_entry(char *p, char *mfa); /** * Fired whenever a Built In Function returns. * * @param p the PID (string form) of the process * @param mfa the m:f/a of the function */ probe bif_return(char *p, char *mfa); /** * Fired whenever a Native Function is called. * * @param p the PID (string form) of the process * @param mfa the m:f/a of the function */ probe nif_entry(char *p, char *mfa); /** * Fired whenever a Native Function returns. * * @param p the PID (string form) of the process * @param mfa the m:f/a of the function */ probe nif_return(char *p, char *mfa); /** * Fired when a process is spawned. * * @param p the PID (string form) of the new process. * @param mfa the m:f/a of the function */ probe spawn(char *p, char *mfa); /** * Fired when a process is exiting. * * @param p the PID (string form) of the exiting process * @param reason the reason for the exit (may be truncated) */ probe exit(char *p, char *reason); /** * Fired when exit signal is delivered to a local process. * * @param sender the PID (string form) of the exiting process * @param receiver the PID (string form) of the process receiving EXIT signal * @param reason the reason for the exit (may be truncated) */ probe exit_signal(char *sender, char *receiver, char *reason); /** * Fired when a major GC is starting. * * @param p the PID (string form) of the exiting process * @param need the number of words needed on the heap */ probe gc_major_start(char *p, int need); /** * Fired when a minor GC is starting. * * @param p the PID (string form) of the exiting process * @param need the number of words needed on the heap */ probe gc_minor_start(char *p, int need); /** * Fired when a major GC is starting. * * @param p the PID (string form) of the exiting process * @param reclaimed the amount of space reclaimed */ probe gc_major_end(char *p, int reclaimed); /** * Fired when a minor GC is starting. * * @param p the PID (string form) of the exiting process * @param reclaimed the amount of space reclaimed */ probe gc_minor_end(char *p, int reclaimed); /** * Fired when a process is scheduled. * * @param p the PID (string form) of the newly scheduled process * @param mfa the m:f/a of the function it should run next */ probe process_scheduled(char *p, char *mfa); /** * Fired when a process is unscheduled. * * @param p the PID (string form) of the process that has been * unscheduled. */ probe process_unscheduled(char *p); /** * Fired when a process goes into hibernation. * * @param p the PID (string form) of the process entering hibernation * @param mfa the m:f/a of the location to resume */ probe hibernate(char *p, char *mfa); /** * Fired when process' heap is growing. * * @param p the PID (string form) of the existing process * @param old_size the size of the old heap * @param new_size the size of the new heap */ probe process_heap_grow(char *p, int old_size, int new_size); /** * Fired when process' heap is shrinking. * * @param p the PID (string form) of the existing process * @param old_size the size of the old heap * @param new_size the size of the new heap */ probe process_heap_shrink(char *p, int old_size, int new_size); /** * Fired when port_command is issued. * * @param proces the PID (string form) of the existing process * @param port the Port (string form) of the existing port * @param port_name the string used when opening a port * @param command_type type of the issued command, one of: "close", "command" or "connect" */ probe port_command(char *process, char *port, char *port_name, char *command_type); /** * Fired when port_control is issued. * * @param proces the PID (string form) of the existing process * @param port the Port (string form) of the existing port * @param port_name the string used when opening a port * @param command_no command number that has been issued to the port */ probe port_control(char *process, char *port, char *port_name, int command_no); /* Async driver pool */ /** * Show the post-add length of the async driver thread pool member's queue. * * @param pool member number * @param new queue length */ probe async_io_pool_add(int, int); /** * Show the post-get length of the async driver thread pool member's queue. * * @param pool member number * @param new queue length */ probe async_io_pool_get(int, int); /* Probes for efile_drv.c */ /** * Entry into the efile_drv.c file I/O driver * * For a list of command numbers used by this driver, see the section * "Guide to probe arguments" in ../../../README.md. That section * also contains explanation of the various integer and string * arguments that may be present when any particular probe fires. * * @param thread-id number of the scheduler Pthread arg0 * @param tag number: {thread-id, tag} uniquely names a driver operation * @param user-tag string arg2 * @param command number arg3 * @param string argument 1 arg4 * @param string argument 2 arg5 * @param integer argument 1 arg6 * @param integer argument 2 arg7 * @param integer argument 3 arg8 * @param integer argument 4 arg9 */ probe file_drv_entry(int, int, char *, int, char *, char *, int64_t, int64_t, int64_t, int64_t); /* 0 1 2 3 */ /* thread-id, tag, work-thread-id, command */ /** * Entry into the driver's internal work function. Computation here * is performed by a async worker pool Pthread. * * @param thread-id number * @param tag number * @param worker pool thread-id number * @param command number */ probe file_drv_int_entry(int, int, int, int); /** * Return from the driver's internal work function. * * @param thread-id number * @param tag number * @param worker pool thread-id number * @param command number */ probe file_drv_int_return(int, int, int, int); /** * Return from the efile_drv.c file I/O driver * * @param thread-id number arg0 * @param tag number arg1 * @param user-tag string arg2 * @param command number arg3 * @param Success? 1 is success, 0 is failure arg4 * @param If failure, the errno of the error. arg5 * @param thread-id number of the scheduler Pthread executing now arg6 */ probe file_drv_return(int, int, char *, int, int, int, int); }; #pragma D attributes Evolving/Evolving/Common provider erlang provider #pragma D attributes Private/Private/Common provider erlang module #pragma D attributes Private/Private/Common provider erlang function #pragma D attributes Evolving/Evolving/Common provider erlang name #pragma D attributes Evolving/Evolving/Common provider erlang args From barcojie@REDACTED Sun Oct 9 07:19:45 2011 From: barcojie@REDACTED (Barco You) Date: Sun, 9 Oct 2011 13:19:45 +0800 Subject: [erlang-questions] [Zotonic] How to distribute zotonic across multiple machines? Message-ID: Thanks! From paul.james.barry@REDACTED Sun Oct 9 13:06:38 2011 From: paul.james.barry@REDACTED (Paul Barry) Date: Sun, 9 Oct 2011 12:06:38 +0100 Subject: [erlang-questions] Need an introductory talk on Erlang? Message-ID: Hi all. The good folk at O'Reilly Media have uploaded my recent webast to YouTube for all the world to see and share: http://www.youtube.com/watch?v=yRJZxV8uPhg Regards. Paul. P.S. I delivered the same talk at the PyCon Ireland 2011 conference yesterday in Dublin. I'm delighted to report that the good-sized room was "standing room only". Lots of interest in Erlang from the Python community in Ireland. :-) -- Paul Barry, w: http://paulbarry.itcarlow.ie, e: paul.barry@REDACTED Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From joshmuza@REDACTED Sun Oct 9 14:32:53 2011 From: joshmuza@REDACTED (Joshua Muzaaya) Date: Sun, 9 Oct 2011 15:32:53 +0300 Subject: [erlang-questions] MNESIA FRAGMENTATION AND REPLICATION CHALLENGES Message-ID: Hi guys, i have asked a question here: http://stackoverflow.com/questions/7703170/mnesia-fragmentation-and-replication-resultant-availability-and-reliabilityand need solutions from the forum. I could not edit and reformat sionce it would be time wasting so i deceide to paste you the link above. You can follow the link and provide your solutions. Thanks guys -- *Muzaaya Joshua Systems Engineer +256774115170* -------------- next part -------------- An HTML attachment was scrubbed... URL: From nick.sfx.1@REDACTED Mon Oct 10 01:12:44 2011 From: nick.sfx.1@REDACTED (Nick) Date: Mon, 10 Oct 2011 02:12:44 +0300 Subject: [erlang-questions] Commercial Erlang application licence question Message-ID: What kind of licence would apply to programs (for commercial use, and closed source) written in Erlang? Does code need to be shared and under what licence? No Erlang libraries/files are touched. Just need to be extra sure of licence terms of Erlang. (Essentially means distributing set of text files with .erl, that follows Erlang programming principles and key-words, but I am not legal expert myself:) -------------- next part -------------- An HTML attachment was scrubbed... URL: From dan@REDACTED Mon Oct 10 01:59:09 2011 From: dan@REDACTED (Daniel Dormont) Date: Sun, 9 Oct 2011 19:59:09 -0400 Subject: [erlang-questions] Commercial Erlang application licence question In-Reply-To: References: Message-ID: Are you bundling the Erlang run-time system or any of its components? Regardless of whether you modified them or not, the answer likely depends on whether you're distributing them. If not, I can't see how there could be any restriction. Surely Ericsson can't force you to use a certain license merely because your code is written in Erlang. dan On Sun, Oct 9, 2011 at 7:12 PM, Nick wrote: > What kind of licence would apply to programs (for commercial use, and > closed source) written in Erlang? Does code need to be shared and under what > licence? > > No Erlang libraries/files are touched. > > Just need to be extra sure of licence terms of Erlang. (Essentially means > distributing set of text files with .erl, that follows Erlang programming > principles and key-words, but I am not legal expert myself:) > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From overminddl1@REDACTED Mon Oct 10 02:23:14 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Sun, 9 Oct 2011 18:23:14 -0600 Subject: [erlang-questions] [tutorial] A Basic Full Text Search Server in Erlang In-Reply-To: <20111007095611.7286ffaa@groupware.tty.nl> References: <20111007095611.7286ffaa@groupware.tty.nl> Message-ID: On Fri, Oct 7, 2011 at 3:56 AM, Ward Bekker wrote: > I've posted a tutorial on how you could build a, very basic, full text > search server with Erlang. Feedback most welcome! > http://blog.tty.nl/2011/10/07/a-basic-full-text-search-server-in-erlang/ I like, need to set aside some time to download and follow/breakdown all of it. From simon@REDACTED Mon Oct 10 13:24:01 2011 From: simon@REDACTED (Simon MacMullen) Date: Mon, 10 Oct 2011 12:24:01 +0100 Subject: [erlang-questions] ssl:peercert/1 has gone? Message-ID: <4E92D5D1.9020509@rabbitmq.com> This merge: https://github.com/erlang/otp/commit/00202339445daae6ed931f28f932089d5c3dd455 Says "Remove old ssl implementation and deprecated function ssl:peercert/1" But it was only ssl:peercert/2 that was marked as deprecated. I thought. Indeed otp_internal.erl says: > %% Added in R14A. > obsolete_1(ssl, peercert, 2) -> > {deprecated,"deprecated (will be removed in R15A); use > ssl:peercert/1 and public_key:pkix_decode_cert/2 instead"}; So was ssl:peercert/1 meant to be removed? If so, what should I use to get the peer certificate? Cheers, Simon -- Simon MacMullen RabbitMQ, VMware From joshmuza@REDACTED Mon Oct 10 14:08:38 2011 From: joshmuza@REDACTED (Joshua Muzaaya) Date: Mon, 10 Oct 2011 15:08:38 +0300 Subject: [erlang-questions] MNESIA FRAGMENTATION AND REPLICATION Message-ID: i still have a number of challenges as regards mnesia fragmentation and replication. Consider the following scenario (The question am asking is based on what follows below): You have a data driven enterprise application which should be highly available within the enterprise. If the internal information source is down for any reason, the enterprise applications must switch to fetch data from a *recovery center* which is offsite (remote). You decide to have the database replicated onto Two Nodes within the enterprise (refered to as *DB side A* and *DB side B*). These two are running on separate Hardware but linked together with say, a *Fast Ethernet or Optical Fibre link*. Logically, you create some kind of tunnel or secure communications between these two Mnesia DBs. The two (A and B) should have the same replica of data and are in sync all the time. Now, meanwhile, the recovery center must too, have the same copy of data and in sync all the time just in case the local data access is cutoff due to an attack or hardware failure. So the same Database schema must be replicated across the 3 sites (*Side A* , *Side B* and *recovery center*). Now, within the enterprise, the application middle ware is capable of switching data requests amongst the database sites. If A is down, then without the application realizing it, the request is re-routed to Database B and so on. The middle ware layer can be configured to do load balancing (request multiplexing) or to do be flexible with fail over techniques. *Further Analysis*: At Database/Schema creation time, all involved Nodes must be up and running Mnesia. To achieve this, you create say: *'db_side_A@REDACTED'*, *'db_side_B@REDACTED'* and finally, *'db_recovery_center@REDACTED'* Now, at Table creation, you would want to have your mnesia tables fragmented. So you decide on the following parameters: *n_disc_only_copies =:= number of nodes involved in the pool =:= 3 **Reason:* You are following the documentation that this parameter regulates how many disc_only_copies replicas that each fragment should have. So you want each table to have each of its fragments on each mnesia Node. *node_pool =:= all nodes involved =:= ['db_side_A@REDACTED', 'db_side_B@REDACTED', 'db_recovery_center@REDACTED'] * All your tables are then created based on the following arrangement Nodes = [ 'db_side_A@REDACTED', 'db_side_B@REDACTED', 'db_recovery_center@REDACTED' ], No_of_fragments = 16, {atomic,ok} = mnesia:create_table(*TABLE_NAME*,[ {frag_properties,[ {node_pool,Nodes}, {n_fragments,No_of_fragments}, {n_disc_only_copies,length(Nodes)}] }, {index,[]}, {attributes,record_info(fields,*RECORD_NAME_HERE*)}] ), NOTE: In the syntax above, RECORD_NAME_HERE cannot be a variable in reality since records must be known at compile time with Erlang. From the installation, you see that for each table, every fragment, say, table_name_frag2, appears on every Node's file system. *Challenges and arising Questions*: After following what is listed down above, your first database start is okay since mnesia is running on all nodes. Several challenges start to show up as the application runs and am listing the below: *1. * Supposing, you decide that all writes are first tried on DB Side A and if side A at that instant is unavailable, the call is re-tried on DB Side Band so on to recovery center, and if the call fails to return on all the 3 database nodes, then the application network middle ware layer reports back that the database servers are all unavailable (this decision could have been influenced by the fact that if you let applications randomly write to your mnesia replicas, its very possible to have inconsistent database errors showing up in case your mnesia nodes lose a network connection with each other yet writes are being committed on each by different Erlang applications. If you decide on having master_nodes, then you could be at risk of loosing data). So by behavior, you are forcing DB Side A to be the master. This makes the other Database Nodes Idle for all the time as long as DB Side A is up and running and so as many requests as hit side A and it does not go down, No request will hit side B and recovery center at all. *2. * Mnesia on start, normally, should see all involved nodes running (mnesia must be running on all involved nodes) so that it can do its negotiations and consistency checks. It means that if mnesia goes down on all nodes, mnesia must be started on all nodes before it can fully initialize and load tables. Its even worse if the Erlang VM dies along with Mnesia on a remote site. Well, several tweaks and scripts here and there could help restart the entire VM plus the intended applications if it goes down. To cut the long story short, let me go to the questions. *Questions*: *1. * What would a Database administrator do if mnesia generates events of inconsistent_database, starting to run database behind a partitioned network, in a situation where setting a mnesia master node is not desirable (for fear of data loss) ? *2. * What is the consequence of the mnesia event inconsistent_database, starting to run database behind a partitioned network as regards my application ? What if i do not react to this event and let things continue the way they are ? am i loosing data ? *3. * In large mnesia clusters, what can one do if Mnesia goes down together with the Erlang VM on a remote site ? are there any known good methods of automatically handling this situation ? *4. * There times when one or two nodes are unreachable due to network problems or failures, and mnesia on the surviving Node reports that a given file does not exist especially in cases where you have indexes. So at run time, what would be the behavior of my application if some replicas go down. Would you advise me to have a master node within a mnesia cluster ? As you answer the questions above, you could also highlight on the layout described at the beginning, whether or not it would ensure availability. You can give your personal experiences on working with mnesia fragmented and replicated Databases in production. In reference to the linked (quoted) question at the very beginning of this text, do provide alternative settings that could offer more reliability at database creation, say in terms of the number of fragments, Operating system dependencies, node pool size, table copy types e.t.c. Thanks guys -- *Muzaaya Joshua Systems Engineer +256774115170* -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Mon Oct 10 14:30:39 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 10 Oct 2011 14:30:39 +0200 Subject: [erlang-questions] MNESIA FRAGMENTATION AND REPLICATION In-Reply-To: References: Message-ID: On 10 Oct 2011, at 14:08, Joshua Muzaaya wrote: > Questions: > 1. What would a Database administrator do if mnesia generates events of inconsistent_database, starting to run database behind a partitioned network, in a situation where setting amnesia master node is not desirable (for fear of data loss) ? Take a look at http://github.com/esl/unsplit It was designed to help resolve netsplits. > 2. What is the consequence of the mnesia event inconsistent_database, starting to run database behind a partitioned network as regards my application ? What if i do not react to this event and let things continue the way they are ? am i loosing data ? The event basically means that the returning node did not synch its tables with the running cluster. Something needs to be done at this point. The unsplit application triggers on the 'inconsistent_database' event and performs an automatic merge. Note that you have to plan your database in order to make such a merge possible. > 3. In large mnesia clusters, what can one do if Mnesia goes down together with the Erlang VM on a remote site ? are there any known good methods of automatically handling this situation ? From a mnesia/erlang point of view, this is the same as when a local node dies. Could you clarify the question, please? > 4. There times when one or two nodes are unreachable due to network problems or failures, and mnesia on the surviving Node reports that a given file does not exist especially in cases where you have indexes. So at run time, what would be the behavior of my application if some replicas go down. Would you advise me to have a master node within a mnesia cluster ? When dealing with fragmentation in mnesia, you may want to take control over the allocation of fragments to nodes. While mnesia offers to do this automatically, you can be explicit about the location of fragments, and you can also move them around. The automatic indexing is problematic with fragmented tables. It's better to create your own 'index', using a second mnesia table. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgsmcmlxxv@REDACTED Mon Oct 10 15:03:01 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Mon, 10 Oct 2011 16:03:01 +0300 Subject: [erlang-questions] MNESIA FRAGMENTATION AND REPLICATION CHALLENGES In-Reply-To: References: Message-ID: Hi Joshua, I am not an expert in Mnesia, but I can try to give you some suggestions (for sure you might have guessed them by now, but at least you can have a confirmation): Q1: "What would a Database administrator do if mnesia generates events of inconsistent_database, starting to run database behind a partitioned network, in a situation where setting amnesia master node is not desirable (for fear of data loss) ?" A1: Check if the nodes are up and try to update manually the DB from the node which was working the longest (providing the DB was correct written). Q2: "What is the consequence of the mnesia event inconsistent_database, starting to run database behind a partitioned network as regards my application ? What if i do not react to this event and let things continue the way they are ? am i loosing data ?" A2: The state is coming from all the nodes failure, that means you can no longer read/write to/from DB. That means, yes, you lose data. Q3: "In large mnesia clusters, what can one do if Mnesia goes down together with the Erlang VM on a remote site ? are there any known good methods of automatically handling this situation ?" A3: http://www.erlang.org/doc/apps/mnesia/Mnesia_chap7.html#id311335 If that doesn't work, build inside your middleware a distributed application which should monitor Erlang nodes. If you are still not satisfied, create a watchdog. Q4: "There times when one or two nodes are unreachable due to network problems or failures, and mnesia on the surviving Node reports that a given file does not exist especially in cases where you haveindexes. So at run time, what would be the behavior of my application if some replicas go down. Would you advise me to have a master node within a mnesia cluster ? " A4: If some replicas go down and your middleware knows how to handle the connection failure, your application should not receive any error. Your application should receive an error only when the BD is sliced or the working node didn't have time to make a replica of the new entry (entries). Few more considerations here which may work in all the cases when you don't trust the way a DB handles your data over the network: 1. Replace the replication system with your own data distribution. E.g., instead of using data replication, use your own system to send the data to all nodes. Further on, when a connection fails, broadcast a message to all similar nodes and, following a certain algorithm to pick up another node, switch to another node, trying to revive and update the dead node meanwhile. 2. Replace the DB error reporting with a watchdog. E.g., continuing the idea above, put a bunch of functions to check periodically the nodes and the applications inside he nodes. You can add to it a probabilistic method to decide which node is longer up and to consider that as the most stable node, so, the read/write to be directed with preference to that node. One more thing. If you don't like Mnesia replication and how it handles the data over the network, try CouchDB which doesn't have master-slave replication, but directional replication. It's written in Erlang as well, but it's no-SQL and it recovers faster from failures. I don't say Mnesia it's not good (well, it's great actually), but following your "fears", I just suggest an alternative which may suits you better. I hope this mail will help you in choosing correct setup for your network distributed DB. Cheers, CGS On Sun, Oct 9, 2011 at 3:32 PM, Joshua Muzaaya wrote: > > Hi guys, i have asked a question here: > http://stackoverflow.com/questions/7703170/mnesia-fragmentation-and-replication-resultant-availability-and-reliabilityand need solutions from the forum. I could not edit and reformat sionce it > would be time wasting so i deceide to paste you the link above. You can > follow the link and provide your solutions. Thanks guys > -- > *Muzaaya Joshua > Systems Engineer > +256774115170* > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joshmuza@REDACTED Mon Oct 10 16:24:51 2011 From: joshmuza@REDACTED (Joshua Muzaaya) Date: Mon, 10 Oct 2011 17:24:51 +0300 Subject: [erlang-questions] MNESIA BACKUP IN LARGE SYSTEMS Message-ID: Thanks guys for supporting us through this forum In our setup, we found that issuing the following backup call with mnesia:backup_checkpoint(Name, Opaque, [Mod]) is more expensive as compared to this one: (mnesia:backup(Opaque, [Mod])). The issue is to do with Memory. Whenever we specify our own checkpoint name, mnesia consumes a lot of memory than when we leave it to do its thing. By using the later function, we only specify the file name of the resultant backup and all goes well, But i have a question below: Backups work very well, however, i wonder, in a system running a replicated schema (database across 3 Nodes), if i issue a mnesia backup on one of the Nodes, does the back up only consider the local table state or the backup is transactional (ATOMIC) in nature in that mnesia may consult involved Nodes to find all the most recent changes in other replicas in order to make the back up. In other words, what is the scope of the backup in cases of say: Fragmentation, Replication, in cases where i have some Nodes down and in cases where a remote table has more recent data than the one on which the backup call is issued ? thanks in advance guys *Muzaaya Joshua Systems Engineer +256774115170* -------------- next part -------------- An HTML attachment was scrubbed... URL: From torben.lehoff@REDACTED Mon Oct 10 17:48:55 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Mon, 10 Oct 2011 17:48:55 +0200 Subject: [erlang-questions] Creation of HTTP URLs with arguments Message-ID: Hi, I want to create URLs that looks like this: http://my.coolest.site.com?arg1=val1,arg2=val2 Are there any libraries out there that makes this kind of creation easy? I have tried playing with ibrowse, but that was not a success (might be me at fault more than ibrowse ;-)) I am dreaming of something that takes a proplist ([{Key, Value}]) and does all the magic for me, but I am open to other approaches as long as my life becomes easier than fiddling with strings. Otherwise, I will just create my own library for this, but it seems like a thing that others have might have had to do before. Cheers, Torben -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela@REDACTED Mon Oct 10 17:54:26 2011 From: ingela@REDACTED (Ingela Andin) Date: Mon, 10 Oct 2011 17:54:26 +0200 Subject: [erlang-questions] ssl:peercert/1 has gone? In-Reply-To: <4E92D5D1.9020509@rabbitmq.com> References: <4E92D5D1.9020509@rabbitmq.com> Message-ID: Hi! Hoops! Only peercert/2 should have been removed, my bad. I will put it back. I think that you usually would access the peer-cert in the verify_fun during the handshake and not when the connection is already up, hence I overlooked that it still might be useful and backwards compatible to keep ssl:peercert/1 . Regards Ingela Erlang /OTP team - Ericsson AB 2011/10/10 Simon MacMullen : > This merge: > > https://github.com/erlang/otp/commit/00202339445daae6ed931f28f932089d5c3dd455 > > Says "Remove old ssl implementation and deprecated function ssl:peercert/1" > > But it was only ssl:peercert/2 that was marked as deprecated. I thought. > Indeed otp_internal.erl says: > >> %% Added in R14A. >> obsolete_1(ssl, peercert, 2) -> >> ? ? {deprecated,"deprecated (will be removed in R15A); use >> ssl:peercert/1 and public_key:pkix_decode_cert/2 instead"}; > > So was ssl:peercert/1 meant to be removed? If so, what should I use to get > the peer certificate? > > Cheers, Simon > > -- > Simon MacMullen > RabbitMQ, VMware > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From dmercer@REDACTED Mon Oct 10 18:56:11 2011 From: dmercer@REDACTED (David Mercer) Date: Mon, 10 Oct 2011 11:56:11 -0500 Subject: [erlang-questions] Erlang SMTP Client? Message-ID: <000901cc876d$847336f0$8d59a4d0$@com> What is the state of the art for sending email from Erlang? I can google up a couple of results, but (1) I was surprised there is no such module in the standard distribution (that I could find); and (2) among the results, I wasn't sure which is the current state of the art. Please advise. Thank-you. David Mercer -------------- next part -------------- An HTML attachment was scrubbed... URL: From ericbmerritt@REDACTED Mon Oct 10 19:05:40 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Mon, 10 Oct 2011 12:05:40 -0500 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: <000901cc876d$847336f0$8d59a4d0$@com> References: <000901cc876d$847336f0$8d59a4d0$@com> Message-ID: There is an smtp client here that might be useful for you https://github.com/ericbmerritt/erfc_parsers On Mon, Oct 10, 2011 at 11:56 AM, David Mercer wrote: > What is the state of the art for sending email from Erlang?? I can google up > a couple of results, but (1) I was surprised there is no such module in the > standard distribution (that I could find); and (2) among the results, I > wasn?t sure which is the current state of the art. > > > > Please advise.? Thank-you. > > > > David Mercer > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From g@REDACTED Mon Oct 10 19:30:29 2011 From: g@REDACTED (Garrett Smith) Date: Mon, 10 Oct 2011 12:30:29 -0500 Subject: [erlang-questions] Creation of HTTP URLs with arguments In-Reply-To: References: Message-ID: Hi Torben, > I want to create URLs that looks like this: > http://my.coolest.site.com?arg1=val1,arg2=val2 > Are there any libraries out there that makes this kind of creation easy? >From https://github.com/mochi/mochiweb: mochiweb_util:urlencode([{"arg1", "val1"}, {"arg2", "val2"}]) From vinoski@REDACTED Mon Oct 10 19:16:22 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Mon, 10 Oct 2011 13:16:22 -0400 Subject: [erlang-questions] Creation of HTTP URLs with arguments In-Reply-To: References: Message-ID: On Mon, Oct 10, 2011 at 11:48 AM, Torben Hoffmann wrote: > Hi, > I want to create URLs that looks like this: > http://my.coolest.site.com?arg1=val1,arg2=val2 The usual standard format is http://my.coolest.site.com?arg1=val1&arg2=val2 Any reason you want a comma instead of an ampersand there? > Are there any libraries out there that makes this kind of creation easy? I > have tried playing with ibrowse, but that was not a success (might be me at > fault more than ibrowse ;-)) > I am dreaming of something that takes a proplist ([{Key, Value}]) and does > all the magic for me, but I am open to other approaches as long as my life > becomes easier than fiddling with strings. > Otherwise, I will just create my own library for this, but it seems like a > thing that others have might have had to do before. In yaws there's yaws_api:format_url which takes a url record and can produce the above (with the ampersand rather than the comma). https://github.com/klacke/yaws/blob/master/src/yaws_api.erl#L1349 --steve From erlang@REDACTED Mon Oct 10 19:56:08 2011 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 10 Oct 2011 19:56:08 +0200 Subject: [erlang-questions] Commercial Erlang application licence question In-Reply-To: References: Message-ID: On Mon, Oct 10, 2011 at 1:12 AM, Nick wrote: > What kind of licence would apply to programs (for commercial use, and closed > source) written in Erlang? Does code need to be shared and under what > licence? You decide this - for code you write and distribute that is not derived from other licensed code you decide the licensing conditions. If you modify and distribute individual files in the Erlang distribution you will have to comply with the licence that pertains to these files. /Joe > > No Erlang libraries/files are touched. > > Just need to be extra sure of licence terms of Erlang. (Essentially means > distributing set of text files with .erl, that follows Erlang programming > principles and key-words, but I am not legal expert myself:) > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From andrew@REDACTED Mon Oct 10 20:40:15 2011 From: andrew@REDACTED (Andrew Thompson) Date: Mon, 10 Oct 2011 14:40:15 -0400 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: References: <000901cc876d$847336f0$8d59a4d0$@com> Message-ID: <20111010184015.GC20887@hijacked.us> On Mon, Oct 10, 2011 at 12:05:40PM -0500, Eric Merritt wrote: > There is an smtp client here that might be useful for you > > https://github.com/ericbmerritt/erfc_parsers > All I see in this repo is a CSV parser? I'm the author and maintainer of (in my biased view) the most complete SMTP library for erlang, gen_smtp: https://github.com/Vagabond/gen_smtp gen_smtp includes both a SMTP client, and a callback framework for writing SMTP servers. It's part of several other erlang projects now, including Zotonic and Chicago Boss. The other alternative worth considering is Geoff's esmtp: https://github.com/archaelus/esmtp I'm not aware of any other SMTP clients that are reliable or mature enough to be considered for serious usage. There's a random smtp fsm kicking around the internet, but its not maintained and the licensing on it is uncertain (or its under the GPL, its unclear). Andrew From marc@REDACTED Mon Oct 10 21:21:24 2011 From: marc@REDACTED (Marc Worrell) Date: Mon, 10 Oct 2011 21:21:24 +0200 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: <20111010184015.GC20887@hijacked.us> References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> Message-ID: <598F8113-02B2-4500-A3DA-14B6A1A15E16@worrell.nl> Don't be shy, Andrew. Your library is the best one out there! And battle tested. - Marc On 10 okt 2011, at 20:40, Andrew Thompson wrote: > 'm the author and maintainer of (in my biased view) the most complete > SMTP library for erlang, gen_smtp: > > https://github.com/Vagabond/gen_smtp -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Mon Oct 10 22:12:20 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Mon, 10 Oct 2011 22:12:20 +0200 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: <000901cc876d$847336f0$8d59a4d0$@com> References: <000901cc876d$847336f0$8d59a4d0$@com> Message-ID: <4E9351A4.8040309@gmail.com> On 10/10/2011 06:56 PM, David Mercer wrote: > What is the state of the art for sending email from Erlang? I can google > up a couple of results, but (1) I was surprised there is no such module > in the standard distribution (that I could find); and (2) among the > results, I wasn?t sure which is the current state of the art. If you just want something simple and reliable, going through sendmail can be a good idea. We've used this library at Klarna for years: https://github.com/richcarl/sendmail For local testing, e.g., on a laptop, I can recommend installing nullmailer as sendmail replacement rather than postfix or exim4 (http://undesigned.org.za/2007/11/22/nullmailer-a-developers-best-friend) if you're just interested in forwarding mail from the local machine to a smarthost. /Richard From justmike2000@REDACTED Mon Oct 10 23:06:51 2011 From: justmike2000@REDACTED (Michael Mileusnich) Date: Mon, 10 Oct 2011 16:06:51 -0500 Subject: [erlang-questions] Erlsrv Message-ID: Hello, Erlsrv looks very simple however I have tried many different combinations of commands and cannot get my server to stay active or get accurate debug information. Could anybody provide me with what the command to form a Windows service using Erlsrv with the following command: erl -pa ebin -run hqcontroller run -extra $DATA - My beam files are in the ebin folder Thanks, Michael Mileusnich -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmercer@REDACTED Mon Oct 10 23:30:19 2011 From: dmercer@REDACTED (David Mercer) Date: Mon, 10 Oct 2011 16:30:19 -0500 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: <20111010184015.GC20887@hijacked.us> References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> Message-ID: <006401cc8793$d0751ca0$715f55e0$@com> On Monday, October 10, 2011, Andrew Thompson wrote: > I'm the author and maintainer of (in my biased view) the most complete > SMTP library for erlang, gen_smtp: > > https://github.com/Vagabond/gen_smtp > > gen_smtp includes both a SMTP client, and a callback framework for > writing SMTP servers. It's part of several other erlang projects now, > including Zotonic and Chicago Boss. Thanks. That worked fine out of the box. I must admit, I am surprised there is no SMTP client in the standard distribution. You would think that would be a fairly common communication protocol... Thanks, Andrew. Cheers, DBM From ericbmerritt@REDACTED Mon Oct 10 23:34:01 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Mon, 10 Oct 2011 16:34:01 -0500 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: <20111010184015.GC20887@hijacked.us> References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> Message-ID: On Mon, Oct 10, 2011 at 1:40 PM, Andrew Thompson wrote: > On Mon, Oct 10, 2011 at 12:05:40PM -0500, Eric Merritt wrote: >> There is an smtp client here that might be useful for you >> >> https://github.com/ericbmerritt/erfc_parsers >> > All I see in this repo is a CSV parser? You are absolutely right. I am not sure where my head was. I have an smtp parser to publish there as well. I guess in my mind I had already published it. Sorry for the confusion. From carlsson.richard@REDACTED Tue Oct 11 02:01:32 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 11 Oct 2011 02:01:32 +0200 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: <006401cc8793$d0751ca0$715f55e0$@com> References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> <006401cc8793$d0751ca0$715f55e0$@com> Message-ID: <4E93875C.4030907@gmail.com> On 10/10/2011 11:30 PM, David Mercer wrote: > On Monday, October 10, 2011, Andrew Thompson wrote: > >> I'm the author and maintainer of (in my biased view) the most complete >> SMTP library for erlang, gen_smtp: >> >> https://github.com/Vagabond/gen_smtp >> >> gen_smtp includes both a SMTP client, and a callback framework for >> writing SMTP servers. It's part of several other erlang projects now, >> including Zotonic and Chicago Boss. > > Thanks. That worked fine out of the box. I must admit, I am surprised > there is no SMTP client in the standard distribution. You would think that > would be a fairly common communication protocol... Often, what you want is more than just SMTP. If there are some network issues (or your mail server is simply down) at the time your program tries to send the mail, you usually want your program to still be able to regard the mail as sent and carry on. This means that the mail needs to be persistently stored on disk in a spool area and that the MTA regularly tries to send any outgoing mail that's been spooled, so it will eventually be on its way once the problem is resolved. That's why we like to rely on sendmail instead. But it all depends on the needs of your application. /Richard From thaterlangguy@REDACTED Tue Oct 11 07:43:37 2011 From: thaterlangguy@REDACTED (Ahmed Al-Saadi) Date: Tue, 11 Oct 2011 01:43:37 -0400 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: <006401cc8793$d0751ca0$715f55e0$@com> References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> <006401cc8793$d0751ca0$715f55e0$@com> Message-ID: <4E93D789.7050207@gmail.com> On 10/10/11 5:30 PM, David Mercer wrote: > On Monday, October 10, 2011, Andrew Thompson wrote: > ... > ... I must admit, I am surprised > there is no SMTP client in the standard distribution. You would think that > would be a fairly common communication protocol... > ... You will be more surprised to find (almost) no Unicode support in the standard distribution! -signed(ahmed). From rtrlists@REDACTED Tue Oct 11 11:20:17 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 11 Oct 2011 10:20:17 +0100 Subject: [erlang-questions] Erlsrv In-Reply-To: References: Message-ID: On Mon, Oct 10, 2011 at 10:06 PM, Michael Mileusnich wrote: > Hello, > > Erlsrv looks very simple however I have tried many different combinations > of commands and cannot get my server to stay active or get accurate debug > information. Could anybody provide me with what the command to form a > Windows service using Erlsrv with the following command: > > erl -pa ebin -run hqcontroller run -extra $DATA > > - My beam files are in the ebin folder > > Thanks, > Michael Mileusnich > > Something along these lines should work: set PATH=%ERLPATH%;%PATH% erlsrv add "MyService" ^ -internalservicename "MyService" ^ -sname "MyService@REDACTED" ^ -stopaction "init:stop()." ^ -workdir "C:/MyWorkdir" ^ -args "-pa ebin -run hqcontroller run -extra %DATA%" ^ -env PATH="%PATH%" The important bit here is the -workdir option (not sure if you need -sname or -stopaction for your purposes), telling the service where it'll be running. Without that, you "-pa ebin" doesn't say much. And you better make sure your DATA env var is properly formatted/quoted, or you'll end up with lots of bizarreness. Robby -------------- next part -------------- An HTML attachment was scrubbed... URL: From rtrlists@REDACTED Tue Oct 11 11:31:29 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 11 Oct 2011 10:31:29 +0100 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: <4E93D789.7050207@gmail.com> References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> <006401cc8793$d0751ca0$715f55e0$@com> <4E93D789.7050207@gmail.com> Message-ID: On Tue, Oct 11, 2011 at 6:43 AM, Ahmed Al-Saadi wrote: > On 10/10/11 5:30 PM, David Mercer wrote: > >> On Monday, October 10, 2011, Andrew Thompson wrote: >> ... >> ... I must admit, I am surprised >> >> there is no SMTP client in the standard distribution. You would think >> that >> would be a fairly common communication protocol... >> ... >> > You will be more surprised to find (almost) no Unicode support in the > standard distribution! > > -signed(ahmed). > > ??? Dealing with Unicode is highly application dependent. I would say that the majority of code dealing with strings transports them from one place to another, and then binaries and the unicode module are excellent! And for reasonable string manipulation a list of code points (or UTF-8 binaries) are all you will need for a long time. If you require a deeper understanding of Unicode code points, combining forms, and their visual representations (e.g., when implementing an editor), then you've got a ton more problems. And those are hard problems with more corner cases that you would ever wish for. Apologies for veering off topic, and probably coming across a wee bit too harsh, Robby -------------- next part -------------- An HTML attachment was scrubbed... URL: From torben.lehoff@REDACTED Tue Oct 11 11:39:48 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Tue, 11 Oct 2011 11:39:48 +0200 Subject: [erlang-questions] Creation of HTTP URLs with arguments In-Reply-To: References: Message-ID: On Mon, Oct 10, 2011 at 19:16, Steve Vinoski wrote: > On Mon, Oct 10, 2011 at 11:48 AM, Torben Hoffmann > wrote: > > Hi, > > I want to create URLs that looks like this: > > http://my.coolest.site.com?arg1=val1,arg2=val2 > > The usual standard format is > > http://my.coolest.site.com?arg1=val1&arg2=val2 > > Any reason you want a comma instead of an ampersand there? > Typo... > > > Are there any libraries out there that makes this kind of creation easy? > I > > have tried playing with ibrowse, but that was not a success (might be me > at > > fault more than ibrowse ;-)) > > I am dreaming of something that takes a proplist ([{Key, Value}]) and > does > > all the magic for me, but I am open to other approaches as long as my > life > > becomes easier than fiddling with strings. > > Otherwise, I will just create my own library for this, but it seems like > a > > thing that others have might have had to do before. > > In yaws there's yaws_api:format_url which takes a url record and can > produce the above (with the ampersand rather than the comma). > > https://github.com/klacke/yaws/blob/master/src/yaws_api.erl#L1349 I will have a look! Thanks for the pointer Cheers, Torben > > > --steve > -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From torben.lehoff@REDACTED Tue Oct 11 11:42:02 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Tue, 11 Oct 2011 11:42:02 +0200 Subject: [erlang-questions] Creation of HTTP URLs with arguments In-Reply-To: References: Message-ID: On Mon, Oct 10, 2011 at 19:30, Garrett Smith wrote: > Hi Torben, > > > I want to create URLs that looks like this: > > http://my.coolest.site.com?arg1=val1,arg2=val2 > > Are there any libraries out there that makes this kind of creation easy? > > From https://github.com/mochi/mochiweb: > > mochiweb_util:urlencode([{"arg1", "val1"}, {"arg2", "val2"}]) > Another one to choose from. Great! I will experiment a bit and see which one fits my needs the best. Cheers, Torben -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From nick.sfx.1@REDACTED Tue Oct 11 12:05:26 2011 From: nick.sfx.1@REDACTED (Nick) Date: Tue, 11 Oct 2011 13:05:26 +0300 Subject: [erlang-questions] Commercial Erlang application licence question In-Reply-To: References: Message-ID: Thanks Joe and Daniel. Would the same apply to beam files as well? We will essentially distribute beam files, which end user will run using Erlang distribution. Nick On Mon, Oct 10, 2011 at 8:56 PM, Joe Armstrong wrote: > On Mon, Oct 10, 2011 at 1:12 AM, Nick wrote: > > What kind of licence would apply to programs (for commercial use, and > closed > > source) written in Erlang? Does code need to be shared and under what > > licence? > > You decide this - for code you write and distribute that is not derived > from other licensed code you decide the licensing conditions. > > If you modify and distribute individual files in the Erlang distribution > you will have to comply with the licence that pertains to these files. > > /Joe > > > > > No Erlang libraries/files are touched. > > > > Just need to be extra sure of licence terms of Erlang. (Essentially means > > distributing set of text files with .erl, that follows Erlang programming > > principles and key-words, but I am not legal expert myself:) > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mabrek@REDACTED Tue Oct 11 12:22:17 2011 From: mabrek@REDACTED (mabrek) Date: Tue, 11 Oct 2011 14:22:17 +0400 Subject: [erlang-questions] asynchronous api patterns for returning partial results ? Message-ID: Hello. Are there good examples of asynchronous api in Erlang for returning partial results? Sometimes I need to return more than a single result from a gen_server. gen_server:call with gen_server:reply isn't suitable for this because gen_server:reply can be called only once per gen_server:call. Desired behaviour can be emulated with: Ref = make_ref(), gen_server:cast(ServerName, {self(), Ref, Request}), receive_results(Ref). receive_results(Ref) -> receive {Ref, done} -> ok; {Ref, PartialResult} -> do_something_with(PartialResults), receive_results(Ref) end. Is it a good way to receive several results from another process? Regards, Anton Lebedevich. From erlang@REDACTED Tue Oct 11 13:40:02 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 11 Oct 2011 13:40:02 +0200 Subject: [erlang-questions] Commercial Erlang application licence question In-Reply-To: References: Message-ID: On Tue, Oct 11, 2011 at 12:05 PM, Nick wrote: > Thanks Joe and Daniel. > Would the same apply to beam files as well? We will essentially distribute > beam files, which end user will run using Erlang distribution. > Nick Yes /Joe > > On Mon, Oct 10, 2011 at 8:56 PM, Joe Armstrong wrote: >> >> On Mon, Oct 10, 2011 at 1:12 AM, Nick wrote: >> > What kind of licence would apply to programs (for commercial use, and >> > closed >> > source) written in Erlang? Does code need to be shared and under what >> > licence? >> >> You decide this - for code you write and distribute that is not derived >> from other licensed code you decide the licensing conditions. >> >> If you modify and distribute individual files in the Erlang distribution >> you will have to comply with the licence that pertains to these files. >> >> /Joe >> >> > >> > No Erlang libraries/files are touched. >> > >> > Just need to be extra sure of licence terms of Erlang. (Essentially >> > means >> > distributing set of text files with .erl, that follows Erlang >> > programming >> > principles and key-words, but I am not legal expert myself:) >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From justmike2000@REDACTED Tue Oct 11 15:36:01 2011 From: justmike2000@REDACTED (Michael Mileusnich) Date: Tue, 11 Oct 2011 08:36:01 -0500 Subject: [erlang-questions] Erlsrv In-Reply-To: References: Message-ID: Thank you very much I will try when I get home On Tue, Oct 11, 2011 at 4:20 AM, Robert Raschke wrote: > > On Mon, Oct 10, 2011 at 10:06 PM, Michael Mileusnich < > justmike2000@REDACTED> wrote: > >> Hello, >> >> Erlsrv looks very simple however I have tried many different combinations >> of commands and cannot get my server to stay active or get accurate debug >> information. Could anybody provide me with what the command to form a >> Windows service using Erlsrv with the following command: >> >> erl -pa ebin -run hqcontroller run -extra $DATA >> >> - My beam files are in the ebin folder >> >> Thanks, >> Michael Mileusnich >> >> > Something along these lines should work: > > set PATH=%ERLPATH%;%PATH% > > erlsrv add "MyService" ^ > -internalservicename "MyService" ^ > -sname "MyService@REDACTED" ^ > -stopaction "init:stop()." ^ > -workdir "C:/MyWorkdir" ^ > -args "-pa ebin -run hqcontroller run -extra %DATA%" ^ > -env PATH="%PATH%" > > The important bit here is the -workdir option (not sure if you need -sname > or -stopaction for your purposes), telling the service where it'll be > running. Without that, you "-pa ebin" doesn't say much. And you better make > sure your DATA env var is properly formatted/quoted, or you'll end up with > lots of bizarreness. > > Robby > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thaterlangguy@REDACTED Tue Oct 11 15:52:20 2011 From: thaterlangguy@REDACTED (Ahmed Al-Saadi) Date: Tue, 11 Oct 2011 09:52:20 -0400 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> <006401cc8793$d0751ca0$715f55e0$@com> <4E93D789.7050207@gmail.com> Message-ID: <4E944A14.60207@gmail.com> On 10/11/11 5:31 AM, Robert Raschke wrote: > On Tue, Oct 11, 2011 at 6:43 AM, Ahmed Al-Saadi > > wrote: > > On 10/10/11 5:30 PM, David Mercer wrote: > > On Monday, October 10, 2011, Andrew Thompson wrote: > ... > ... I must admit, I am surprised > > there is no SMTP client in the standard distribution. You > would think that > would be a fairly common communication protocol... > ... > > You will be more surprised to find (almost) no Unicode support in > the standard distribution! > > -signed(ahmed). > > > ??? > > Dealing with Unicode is highly application dependent. > > I would say that the majority of code dealing with strings transports > them from one place to another, and then binaries and the unicode > module are excellent! And for reasonable string manipulation a list of > code points (or UTF-8 binaries) are all you will need for a long time. > > If you require a deeper understanding of Unicode code points, > combining forms, and their visual representations (e.g., when > implementing an editor), then you've got a ton more problems. And > those are hard problems with more corner cases that you would ever > wish for. > > Apologies for veering off topic, and probably coming across a wee bit > too harsh, > Robby I meant to point out that Erlang does not have a Unicode string as a first-class citizen that the standard library generally uses as the standard input/output. However, I should mention that I find treating strings as a list of codepoints (or encoding bytes) to be elegant (read: more general, less magical). Coming from a Python background, I was surprised (read: confused) by how Erlang dealt with strings. On the plus side, I prefer its "closer-to-the-metal" approach to dealing with text and encoding, especially when I have extra brain-power to spend on understanding how things work ;) Cheers! Ahmed -------------- next part -------------- An HTML attachment was scrubbed... URL: From barcojie@REDACTED Tue Oct 11 16:51:03 2011 From: barcojie@REDACTED (Barco You) Date: Tue, 11 Oct 2011 22:51:03 +0800 Subject: [erlang-questions] asynchronous api patterns for returning partial results ? In-Reply-To: References: Message-ID: What do you mean by partial results? I think it depends on what your callback handle_cast do, what kinds of partial results it will rerun. Best regards, Barco On Oct 11, 2011 6:22 PM, "mabrek" wrote: > Hello. > > Are there good examples of asynchronous api in Erlang for returning > partial results? > Sometimes I need to return more than a single result from a gen_server. > gen_server:call with gen_server:reply isn't suitable for this because > gen_server:reply can be called only once per gen_server:call. > > Desired behaviour can be emulated with: > > Ref = make_ref(), > gen_server:cast(ServerName, {self(), Ref, Request}), > receive_results(Ref). > > receive_results(Ref) -> > receive > {Ref, done} -> > ok; > {Ref, PartialResult} -> > do_something_with(PartialResults), > receive_results(Ref) > end. > > Is it a good way to receive several results from another process? > > Regards, > Anton Lebedevich. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mabrek@REDACTED Tue Oct 11 17:42:12 2011 From: mabrek@REDACTED (mabrek) Date: Tue, 11 Oct 2011 19:42:12 +0400 Subject: [erlang-questions] asynchronous api patterns for returning partial results ? In-Reply-To: References: Message-ID: By partial results I mean something like rows from database query. In this case first process sends SQL "select columns from table" to database driver. driver:handle_cast({request, ...}, ...) encodes and sends the request to socket and waits results from {active, true} socket. driver:handle_info({tcp, _, Data}, ...) decodes result rows from Data and sends them back to the first process as a separate messages. On Tue, Oct 11, 2011 at 6:51 PM, Barco You wrote: > What do you mean by partial results? I think it depends on what your > callback handle_cast do, what kinds of partial results it will rerun. > > Best regards, > Barco > > On Oct 11, 2011 6:22 PM, "mabrek" wrote: >> >> Hello. >> >> Are there good examples of asynchronous api in Erlang for returning >> partial results? >> Sometimes I need to return more than a single result from a gen_server. >> gen_server:call with gen_server:reply isn't suitable for this because >> gen_server:reply can be called only once per gen_server:call. >> >> Desired behaviour can be emulated with: >> >> Ref = make_ref(), >> gen_server:cast(ServerName, {self(), Ref, Request}), >> receive_results(Ref). >> >> receive_results(Ref) -> >> ? ?receive >> ? ? ? ?{Ref, done} -> >> ? ? ? ? ? ?ok; >> ? ? ? ?{Ref, PartialResult} -> >> ? ? ? ? ? ?do_something_with(PartialResults), >> ? ? ? ? ? ?receive_results(Ref) >> ? ?end. >> >> Is it a good way to receive several results from another process? >> >> Regards, >> Anton Lebedevich. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From dmercer@REDACTED Tue Oct 11 19:21:03 2011 From: dmercer@REDACTED (David Mercer) Date: Tue, 11 Oct 2011 12:21:03 -0500 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: <4E93875C.4030907@gmail.com> References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> <006401cc8793$d0751ca0$715f55e0$@com> <4E93875C.4030907@gmail.com> Message-ID: <001401cc883a$28691dc0$793b5940$@com> On Monday, October 10, 2011, Richard Carlsson wrote: > On 10/10/2011 11:30 PM, David Mercer wrote: > > I must admit, I am > surprised > > there is no SMTP client in the standard distribution. You would > think that > > would be a fairly common communication protocol... > > Often, what you want is more than just SMTP. If there are some network > issues (or your mail server is simply down) at the time your program > tries to send the mail, you usually want your program to still be able > to regard the mail as sent and carry on. This means that the mail needs > to be persistently stored on disk in a spool area and that the MTA > regularly tries to send any outgoing mail that's been spooled, so it > will eventually be on its way once the problem is resolved. That's why > we like to rely on sendmail instead. But it all depends on the needs of > your application. Having an SMTP module would enable you to write in Erlang what you need for storing messages for later delivery, if that happened to be your requirement. What do people think of an EEP to add SMTP support? I'm thinking I might be the only person who needs it, since Erlang is, like, 100 years old and it's not in there yet. Is there a silent majority secretly hoping for SMTP in the next release of Erlang, or am I standing alone on this hill? Cheers, DBM From essen@REDACTED Tue Oct 11 19:38:55 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 11 Oct 2011 19:38:55 +0200 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: <001401cc883a$28691dc0$793b5940$@com> References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> <006401cc8793$d0751ca0$715f55e0$@com> <4E93875C.4030907@gmail.com> <001401cc883a$28691dc0$793b5940$@com> Message-ID: <4E947F2F.7090307@dev-extend.eu> On 10/11/2011 07:21 PM, David Mercer wrote: > On Monday, October 10, 2011, Richard Carlsson wrote: > >> On 10/10/2011 11:30 PM, David Mercer wrote: >>> I must admit, I am >> surprised >>> there is no SMTP client in the standard distribution. You would >> think that >>> would be a fairly common communication protocol... >> >> Often, what you want is more than just SMTP. If there are some network >> issues (or your mail server is simply down) at the time your program >> tries to send the mail, you usually want your program to still be able >> to regard the mail as sent and carry on. This means that the mail needs >> to be persistently stored on disk in a spool area and that the MTA >> regularly tries to send any outgoing mail that's been spooled, so it >> will eventually be on its way once the problem is resolved. That's why >> we like to rely on sendmail instead. But it all depends on the needs of >> your application. > > Having an SMTP module would enable you to write in Erlang what you need for > storing messages for later delivery, if that happened to be your > requirement. > > What do people think of an EEP to add SMTP support? I'm thinking I might be > the only person who needs it, since Erlang is, like, 100 years old and it's > not in there yet. Is there a silent majority secretly hoping for SMTP in > the next release of Erlang, or am I standing alone on this hill? There are SMTP modules. Why do you need them to be in the standard distribution? You can just use them as a dependency. -- Lo?c Hoguin Dev:Extend From dale@REDACTED Tue Oct 11 19:53:28 2011 From: dale@REDACTED (Dale Harvey) Date: Tue, 11 Oct 2011 18:53:28 +0100 Subject: [erlang-questions] Documentation Format Changes (erldocs) In-Reply-To: References: <327898192.289261307959408715.JavaMail.root@zimbra> Message-ID: It looks like a lot more of the documentation has moved over arity format which requires a source code lookup to find the correct function signature. Does anyone know how given a the path to a .erl file, a function and an arity I can find a human readable function signature from the type specs, looking over the current edoc / docbuilder source I am not finding much https://github.com/erlang/otp/blob/master/lib/stdlib/src/lists.erl#L1136 find_function_signature("otp_src_R14B04/lib/stdlib/src/lists.erl", "all", 2) -> "all(Pred, List) -> boolean()". If anyone could give more insight into why this information isnt embedded into the intermediary xml format it would be nice, having to do a source code lookup makes the intermediary xml format pointless, it would be hugely easier to generate documentation directly from a single source of truth(the source files) if it wasnt for the fact that half the information needed is not included in the source files, for example the @doc annotation for lists:all Return true if all elements in the list satisfyPred is not anywhere in https://github.com/erlang/otp/blob/master/lib/stdlib/src/lists.erl#L1136 I have offered this before but erldocs is open source, it was built before the current attempt at a search as you type implementation on the erlang.orgsite, and is still more complete (it works offline, it doesnt use iframes and therefore is linkable and searchable) if there is anything I can do to get it 'blessed' in a way that it doesnt get broken during every erlang release then I will be happy to do it. It is reasonably popular (around 9k visits the average month) and given the amount of people that contact me when its late to update or broken I think its beneficial for the community for it not to be so. Cheers Dale On 22 June 2011 16:45, Dale Harvey wrote: > It seems like if we need to cross reference the documentation xml with the > type annotations in the source, what is the point in having docbuilder > generate intermediary xml (with a constantly changing and non documented > schema) at all? we could just generate the output format directly from the > source. > > the only reason I dont do now is that a good chunk of the documentation > (the important modules like lists etc) are only available in the xml format > and not in the source. > > the old documentation used to take edoc formatted type annotations and > embed them into the generated xml (like shown above). the format may be > different but I cant see why that isnt far better than cross referencing > stuff. > > Cheers > Dale > > > On 13 June 2011 11:03, Lukas Larsson wrote: > >> Hi Dale! >> >> All of the documentation is right now being reworked to generate type >> information from the specs in the source instead of manually writing them. >> tells docbuilder (I think) to generate >> the type description from the specs instead of the XML file. If you dig >> around in the source code of docbuilder/edoc there should be functions >> somewhere there which you can use to generate type information for erldocs. >> >> Lukas >> ----- Original Message ----- >> From: "Dale Harvey" >> To: "erlang-questions" >> Sent: Sunday, 12 June, 2011 02:25:29 GMT +01:00 Amsterdam / Berlin / Bern >> / Rome / Stockholm / Vienna >> Subject: [erlang-questions] Documentation Format Changes (erldocs) >> >> >> Hey all >> >> I was wondering about the changes in the format documentation sources, in >> the last release I noticed a few function definitions changed from >> >> keymember(Key, N, TupleList) -> boolean() >> Test for membership of a list of tuples >> >> Key = term() >> N = 1..tuple_size(Tuple) >> TupleList = [Tuple] >>   Tuple = tuple() >> to >> >> >> Merge two key-sorted lists of tuples >> 1..tuple_size(Tuple) >> https://github.com/erlang/otp/blob/dev/lib/stdlib/doc/src/lists.xml#L299is the source for these changes, somehow the recent erlang documention ( >> http://www.erlang.org/doc/man/lists.html#keymerge-3 ) shows the full >> parameter list information and I am quite confused at how that can be >> derived from the source xml. >> >> I was also just curious about the reason for the change and what was >> planned for going forward >> >> since >> http://erldocs.com/R14B03/stdlib/lists.html?i=0&search=lists%20keymerge#keymerge/3 >> is a lot less useful than >> http://erldocs.com/R14B02/stdlib/lists.html?i=0&search=lists%20keymerge#keymerge/3 >> >> Cheers, good work on the new release >> Dale >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmercer@REDACTED Tue Oct 11 20:13:51 2011 From: dmercer@REDACTED (David Mercer) Date: Tue, 11 Oct 2011 13:13:51 -0500 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: <4E947F2F.7090307@dev-extend.eu> References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> <006401cc8793$d0751ca0$715f55e0$@com> <4E93875C.4030907@gmail.com> <001401cc883a$28691dc0$793b5940$@com> <4E947F2F.7090307@dev-extend.eu> Message-ID: <005b01cc8841$884421c0$98cc6540$@com> On Tuesday, October 11, 2011, Lo?c Hoguin wrote: > There are SMTP modules. Why do you need them to be in the standard > distribution? You can just use them as a dependency. Good question. I guess just to avoid having to ask this list which is the state of the art, but that's very unsocial of me... I'm not pushing for it. I've got a solution to my immediate needs (by asking this list), I just thought it might help those who come after me. Ah, how very altruistic of me, eh? Cheers, DBM From essen@REDACTED Tue Oct 11 20:30:06 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 11 Oct 2011 20:30:06 +0200 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: <005b01cc8841$884421c0$98cc6540$@com> References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> <006401cc8793$d0751ca0$715f55e0$@com> <4E93875C.4030907@gmail.com> <001401cc883a$28691dc0$793b5940$@com> <4E947F2F.7090307@dev-extend.eu> <005b01cc8841$884421c0$98cc6540$@com> Message-ID: <4E948B2E.1050206@dev-extend.eu> On 10/11/2011 08:13 PM, David Mercer wrote: > On Tuesday, October 11, 2011, Lo?c Hoguin wrote: > >> There are SMTP modules. Why do you need them to be in the standard >> distribution? You can just use them as a dependency. > > Good question. I guess just to avoid having to ask this list which is the > state of the art, but that's very unsocial of me... > > I'm not pushing for it. I've got a solution to my immediate needs (by > asking this list), I just thought it might help those who come after me. > Ah, how very altruistic of me, eh? What we need, in my opinion, is not a big standard distribution that can do everything including distributed coffee, but a big package repository that would be well integrated into Erlang's tools so we could just have Erlang pull them when needed. Rebar dependencies are doing half the work there, all that's missing is a community maintained repository that's supported officially by a big player (either Erlang/OTP or rebar). -- Lo?c Hoguin Dev:Extend From dmercer@REDACTED Tue Oct 11 23:26:02 2011 From: dmercer@REDACTED (David Mercer) Date: Tue, 11 Oct 2011 16:26:02 -0500 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: <4E948B2E.1050206@dev-extend.eu> References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> <006401cc8793$d0751ca0$715f55e0$@com> <4E93875C.4030907@gmail.com> <001401cc883a$28691dc0$793b5940$@com> <4E947F2F.7090307@dev-extend.eu> <005b01cc8841$884421c0$98cc6540$@com> <4E948B2E.1050206@dev-extend.eu> Message-ID: <006801cc885c$6202a110$2607e330$@com> On Tuesday, October 11, 2011, Lo?c Hoguin (which is a really cool name, by the way) wrote: > What we need, in my opinion, is not a big standard distribution that > can > do everything including distributed coffee, but a big package > repository > that would be well integrated into Erlang's tools so we could just have > Erlang pull them when needed. Rebar dependencies are doing half the > work > there, all that's missing is a community maintained repository that's > supported officially by a big player (either Erlang/OTP or rebar). Rebar, I don't think, is in Erlang, and it doesn't work on all Erlang platforms. If it can be rewritten in Erlang, it might work for this purpose... I don't disagree with you otherwise. Cheers, DBM From jack@REDACTED Tue Oct 11 23:32:16 2011 From: jack@REDACTED (Jack Moffitt) Date: Tue, 11 Oct 2011 15:32:16 -0600 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: <006801cc885c$6202a110$2607e330$@com> References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> <006401cc8793$d0751ca0$715f55e0$@com> <4E93875C.4030907@gmail.com> <001401cc883a$28691dc0$793b5940$@com> <4E947F2F.7090307@dev-extend.eu> <005b01cc8841$884421c0$98cc6540$@com> <4E948B2E.1050206@dev-extend.eu> <006801cc885c$6202a110$2607e330$@com> Message-ID: > Rebar, I don't think, is in Erlang, and it doesn't work on all Erlang > platforms. ?If it can be rewritten in Erlang, it might work for this > purpose... Um, rebar is most definitely in Erlang. However, you are correct that it's windows support is a little weaker than the other systems. Code can be browsed here: http://github.com/basho/rebar jack. From grimlog@REDACTED Tue Oct 11 23:35:44 2011 From: grimlog@REDACTED (Michael Schreckenbauer) Date: Tue, 11 Oct 2011 23:35:44 +0200 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: <006801cc885c$6202a110$2607e330$@com> References: <000901cc876d$847336f0$8d59a4d0$@com> <4E948B2E.1050206@dev-extend.eu> <006801cc885c$6202a110$2607e330$@com> Message-ID: <1499360.fojGIJzEkr@pc> On Tuesday, 11. October 2011 16:26:02 David Mercer wrote: > On Tuesday, October 11, 2011, Lo?c Hoguin (which is a really cool name, by > > the way) wrote: > > What we need, in my opinion, is not a big standard distribution that > > can > > do everything including distributed coffee, but a big package > > repository > > that would be well integrated into Erlang's tools so we could just have > > Erlang pull them when needed. Rebar dependencies are doing half the > > work > > there, all that's missing is a community maintained repository that's > > supported officially by a big player (either Erlang/OTP or rebar). > > Rebar, I don't think, is in Erlang, and it doesn't work on all Erlang > platforms. If it can be rewritten in Erlang, it might work for this > purpose... https://github.com/basho/rebar looks rather erlang-ish to me :) > I don't disagree with you otherwise. > > Cheers, > DBM Best, Michael From dmercer@REDACTED Tue Oct 11 23:43:48 2011 From: dmercer@REDACTED (David Mercer) Date: Tue, 11 Oct 2011 16:43:48 -0500 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> <006401cc8793$d0751ca0$715f55e0$@com> <4E93875C.4030907@gmail.com> <001401cc883a$28691dc0$793b5940$@com> <4E947F2F.7090307@dev-extend.eu> <005b01cc8841$884421c0$98cc6540$@com> <4E948B2E.1050206@dev-extend.eu> <006801cc885c$6202a110$2607e330$@com> Message-ID: <006c01cc885e$dd30fba0$9792f2e0$@com> On Tuesday, October 11, 2011, Jack Moffitt wrote: > > Rebar, I don't think, is in Erlang, and it doesn't work on all Erlang > > platforms. ?If it can be rewritten in Erlang, it might work for this > > purpose... > > Um, rebar is most definitely in Erlang. However, you are correct that > it's windows support is a little weaker than the other systems. Mea culpa. I was just guessing, given its lack of Windows support. Why wouldn't it work on Windows, then, if it's in Erlang. Erlang runs on Windows... DBM From jack@REDACTED Tue Oct 11 23:54:59 2011 From: jack@REDACTED (Jack Moffitt) Date: Tue, 11 Oct 2011 15:54:59 -0600 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: <006c01cc885e$dd30fba0$9792f2e0$@com> References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> <006401cc8793$d0751ca0$715f55e0$@com> <4E93875C.4030907@gmail.com> <001401cc883a$28691dc0$793b5940$@com> <4E947F2F.7090307@dev-extend.eu> <005b01cc8841$884421c0$98cc6540$@com> <4E948B2E.1050206@dev-extend.eu> <006801cc885c$6202a110$2607e330$@com> <006c01cc885e$dd30fba0$9792f2e0$@com> Message-ID: > Mea culpa. ?I was just guessing, given its lack of Windows support. ?Why > wouldn't it work on Windows, then, if it's in Erlang. ?Erlang runs on > Windows... I believe it makes a few assumptions about the shell environment. I believe there are people working to address this if list traffic on the rebar list is any indication. I imagine some parts of it work just fine, and I'm sure feedback would be appreciated to find the remaining rough edges. jack. From anthonym@REDACTED Wed Oct 12 02:17:58 2011 From: anthonym@REDACTED (Anthony Molinaro) Date: Tue, 11 Oct 2011 17:17:58 -0700 Subject: [erlang-questions] Documentation Format Changes (erldocs) In-Reply-To: References: <327898192.289261307959408715.JavaMail.root@zimbra> Message-ID: <3FEFF7A9-F840-4666-9E07-B329891A0B3B@alumni.caltech.edu> As an active user of erldocs I'd like to +1 anything which will make this work again. I have to use old docs just so I can see the right information. Maybe if enough people call out that they would like to see this fixed we can get a fix for the next release. -Anthony On Oct 11, 2011, at 10:53 AM, Dale Harvey wrote: > It looks like a lot more of the documentation has moved over arity format which requires a source code lookup to find the correct function signature. > > Does anyone know how given a the path to a .erl file, a function and an arity I can find a human readable function signature from the type specs, looking over the current edoc / docbuilder source I am not finding much > > https://github.com/erlang/otp/blob/master/lib/stdlib/src/lists.erl#L1136 > > find_function_signature("otp_src_R14B04/lib/stdlib/src/lists.erl", "all", 2) -> > "all(Pred, List) -> boolean()". > > If anyone could give more insight into why this information isnt embedded into the intermediary xml format it would be nice, having to do a source code lookup makes the intermediary xml format pointless, it would be hugely easier to generate documentation directly from a single source of truth(the source files) if it wasnt for the fact that half the information needed is not included in the source files, for example the @doc annotation for lists:all > > Return true if all elements in the list satisfyPred > > is not anywhere in > https://github.com/erlang/otp/blob/master/lib/stdlib/src/lists.erl#L1136 > > > I have offered this before but erldocs is open source, it was built before the current attempt at a search as you type implementation on the erlang.org site, and is still more complete (it works offline, it doesnt use iframes and therefore is linkable and searchable) if there is anything I can do to get it 'blessed' in a way that it doesnt get broken during every erlang release then I will be happy to do it. It is reasonably popular (around 9k visits the average month) and given the amount of people that contact me when its late to update or broken I think its beneficial for the community for it not to be so. > > > Cheers > Dale > > On 22 June 2011 16:45, Dale Harvey wrote: > It seems like if we need to cross reference the documentation xml with the type annotations in the source, what is the point in having docbuilder generate intermediary xml (with a constantly changing and non documented schema) at all? we could just generate the output format directly from the source. > > the only reason I dont do now is that a good chunk of the documentation (the important modules like lists etc) are only available in the xml format and not in the source. > > the old documentation used to take edoc formatted type annotations and embed them into the generated xml (like shown above). the format may be different but I cant see why that isnt far better than cross referencing stuff. > > Cheers > Dale > > > On 13 June 2011 11:03, Lukas Larsson wrote: > Hi Dale! > > All of the documentation is right now being reworked to generate type information from the specs in the source instead of manually writing them. tells docbuilder (I think) to generate the type description from the specs instead of the XML file. If you dig around in the source code of docbuilder/edoc there should be functions somewhere there which you can use to generate type information for erldocs. > > Lukas > ----- Original Message ----- > From: "Dale Harvey" > To: "erlang-questions" > Sent: Sunday, 12 June, 2011 02:25:29 GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna > Subject: [erlang-questions] Documentation Format Changes (erldocs) > > > Hey all > > I was wondering about the changes in the format documentation sources, in the last release I noticed a few function definitions changed from > > keymember(Key, N, TupleList) -> boolean() > Test for membership of a list of tuples > > Key = term() > N = 1..tuple_size(Tuple) > TupleList = [Tuple] >   Tuple = tuple() > to > > > Merge two key-sorted lists of tuples > 1..tuple_size(Tuple) > https://github.com/erlang/otp/blob/dev/lib/stdlib/doc/src/lists.xml#L299 is the source for these changes, somehow the recent erlang documention ( http://www.erlang.org/doc/man/lists.html#keymerge-3 ) shows the full parameter list information and I am quite confused at how that can be derived from the source xml. > > I was also just curious about the reason for the change and what was planned for going forward > > since http://erldocs.com/R14B03/stdlib/lists.html?i=0&search=lists%20keymerge#keymerge/3 > is a lot less useful than http://erldocs.com/R14B02/stdlib/lists.html?i=0&search=lists%20keymerge#keymerge/3 > > Cheers, good work on the new release > Dale > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From scott.kidder11@REDACTED Wed Oct 12 02:49:12 2011 From: scott.kidder11@REDACTED (Scott Kidder) Date: Tue, 11 Oct 2011 20:49:12 -0400 Subject: [erlang-questions] [tutorial] A Basic Full Text Search Server in Erlang In-Reply-To: References: <20111007095611.7286ffaa@groupware.tty.nl> Message-ID: I also wanted to say this looks really nice. I've just been waiting for a chunk of time to dig into to it as well. From erlang@REDACTED Wed Oct 12 09:02:22 2011 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 12 Oct 2011 09:02:22 +0200 Subject: [erlang-questions] [tutorial] A Basic Full Text Search Server in Erlang In-Reply-To: <20111007095611.7286ffaa@groupware.tty.nl> References: <20111007095611.7286ffaa@groupware.tty.nl> Message-ID: Hello, You might like to read the mail I sent earlier: http://erlang.org/pipermail/erlang-questions/2011-September/061305.html This mail has some pointers to techniques used for building an inverted index and index compression. You might find this useful. /Joe On Fri, Oct 7, 2011 at 11:56 AM, Ward Bekker wrote: > I've posted a tutorial on how you could build a, very basic, full text > search server with Erlang. Feedback most welcome! > http://blog.tty.nl/2011/10/07/a-basic-full-text-search-server-in-erlang/ > Enjoy, > Ward > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From tony@REDACTED Wed Oct 12 09:28:58 2011 From: tony@REDACTED (Tony Rogvall) Date: Wed, 12 Oct 2011 09:28:58 +0200 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: <4E948B2E.1050206@dev-extend.eu> References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> <006401cc8793$d0751ca0$715f55e0$@com> <4E93875C.4030907@gmail.com> <001401cc883a$28691dc0$793b5940$@com> <4E947F2F.7090307@dev-extend.eu> <005b01cc8841$884421c0$98cc6540$@com> <4E948B2E.1050206@dev-extend.eu> Message-ID: <2E7BEA81-A562-4785-9F4C-2DD306E7A84A@rogvall.se> On 11 okt 2011, at 20:30, Lo?c Hoguin wrote: > > What we need, in my opinion, is not a big standard distribution that can > do everything including distributed coffee, but a big package repository > that would be well integrated into Erlang's tools so we could just have > Erlang pull them when needed. Rebar dependencies are doing half the work > there, all that's missing is a community maintained repository that's > supported officially by a big player (either Erlang/OTP or rebar). > +1 /Tony > -- > Lo?c Hoguin > Dev:Extend > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From rtrlists@REDACTED Wed Oct 12 09:59:51 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 12 Oct 2011 08:59:51 +0100 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: <006c01cc885e$dd30fba0$9792f2e0$@com> References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> <006401cc8793$d0751ca0$715f55e0$@com> <4E93875C.4030907@gmail.com> <001401cc883a$28691dc0$793b5940$@com> <4E947F2F.7090307@dev-extend.eu> <005b01cc8841$884421c0$98cc6540$@com> <4E948B2E.1050206@dev-extend.eu> <006801cc885c$6202a110$2607e330$@com> <006c01cc885e$dd30fba0$9792f2e0$@com> Message-ID: On Tue, Oct 11, 2011 at 10:43 PM, David Mercer wrote: > On Tuesday, October 11, 2011, Jack Moffitt wrote: > > > > Rebar, I don't think, is in Erlang, and it doesn't work on all Erlang > > > platforms. If it can be rewritten in Erlang, it might work for this > > > purpose... > > > > Um, rebar is most definitely in Erlang. However, you are correct that > > it's windows support is a little weaker than the other systems. > > Mea culpa. I was just guessing, given its lack of Windows support. Why > wouldn't it work on Windows, then, if it's in Erlang. Erlang runs on > Windows... > > DBM > > I've never experienced any issues with running rebar under Windows. The only hurdle is that it is usually invoked from a Makefile. And that's not really a hurdle, more like a bit of uneven ground. ;-) Robby -------------- next part -------------- An HTML attachment was scrubbed... URL: From ward@REDACTED Wed Oct 12 10:03:13 2011 From: ward@REDACTED (Ward Bekker) Date: Wed, 12 Oct 2011 10:03:13 +0200 Subject: [erlang-questions] [tutorial] A Basic Full Text Search Server in Erlang Message-ID: <20111012080313.f6dab5a0@groupware.tty.nl> > Hello, > You might like to read the mail I sent earlier: > http://erlang.org/pipermail/erlang-questions/2011-September/061305.html > This mail has some pointers to techniques used for building an > inverted index and > index compression. You might find this useful. Hi Joe, Thx! I have a freshly delivered copy on my desk of the "Managing Gigabytes" book. My colleague ordered it for me when reading your usenet post ;-). Ward > /Joe > On Fri, Oct 7, 2011 at 11:56 AM, Ward Bekker wrote: > > I've posted a tutorial on how you could build a, very basic, full text > > search server with Erlang. Feedback most welcome! > > http://blog.tty.nl/2011/10/07/a-basic-full-text-search-server-in-erlang/ > > Enjoy, > > Ward > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gordon@REDACTED Wed Oct 12 10:45:31 2011 From: gordon@REDACTED (Gordon Guthrie) Date: Wed, 12 Oct 2011 09:45:31 +0100 Subject: [erlang-questions] Documentation Format Changes (erldocs) In-Reply-To: <3FEFF7A9-F840-4666-9E07-B329891A0B3B@alumni.caltech.edu> References: <327898192.289261307959408715.JavaMail.root@zimbra> <3FEFF7A9-F840-4666-9E07-B329891A0B3B@alumni.caltech.edu> Message-ID: I second that emotion On 12 October 2011 01:17, Anthony Molinaro wrote: > As an active user of erldocs I'd like to +1 anything which will make this > work again. I have to use old docs just so I can see the right information. > Maybe if enough people call out that they would like to see this fixed we > can get a fix for the next release. > > -Anthony > > > On Oct 11, 2011, at 10:53 AM, Dale Harvey wrote: > > It looks like a lot more of the documentation has moved over arity format > which requires a source code lookup to find the correct function signature. > > Does anyone know how given a the path to a .erl file, a function and an > arity I can find a human readable function signature from the type specs, > looking over the current edoc / docbuilder source I am not finding much > > > https://github.com/erlang/otp/blob/master/lib/stdlib/src/lists.erl#L1136 > > find_function_signature("otp_src_R14B04/lib/stdlib/src/lists.erl", "all", > 2) -> > "all(Pred, List) -> boolean()". > > If anyone could give more insight into why this information isnt embedded > into the intermediary xml format it would be nice, having to do a source > code lookup makes the intermediary xml format pointless, it would be hugely > easier to generate documentation directly from a single source of truth(the > source files) if it wasnt for the fact that half the information needed is > not included in the source files, for example the @doc annotation for > lists:all > > Return true if all elements in the list > satisfyPred > > is not anywhere in > > https://github.com/erlang/otp/blob/master/lib/stdlib/src/lists.erl#L1136 > > > I have offered this before but erldocs is open source, it was built before > the current attempt at a search as you type implementation on the > erlang.org site, and is still more complete (it works > offline, it doesnt use iframes and therefore is linkable and searchable) if > there is anything I can do to get it 'blessed' in a way that it doesnt get > broken during every erlang release then I will be happy to do it. It is > reasonably popular (around 9k visits the average month) and given the amount > of people that contact me when its late to update or broken I think its > beneficial for the community for it not to be so. > > > Cheers > Dale > > On 22 June 2011 16:45, Dale Harvey < > dale@REDACTED> wrote: > >> It seems like if we need to cross reference the documentation xml with the >> type annotations in the source, what is the point in having docbuilder >> generate intermediary xml (with a constantly changing and non documented >> schema) at all? we could just generate the output format directly from the >> source. >> >> the only reason I dont do now is that a good chunk of the documentation >> (the important modules like lists etc) are only available in the xml format >> and not in the source. >> >> the old documentation used to take edoc formatted type annotations and >> embed them into the generated xml (like shown above). the format may be >> different but I cant see why that isnt far better than cross referencing >> stuff. >> >> Cheers >> Dale >> >> >> On 13 June 2011 11:03, Lukas Larsson < >> lukas.larsson@REDACTED> wrote: >> >>> Hi Dale! >>> >>> All of the documentation is right now being reworked to generate type >>> information from the specs in the source instead of manually writing them. >>> tells docbuilder (I think) to generate >>> the type description from the specs instead of the XML file. If you dig >>> around in the source code of docbuilder/edoc there should be functions >>> somewhere there which you can use to generate type information for erldocs. >>> >>> Lukas >>> ----- Original Message ----- >>> From: "Dale Harvey" < dale@REDACTED> >>> To: "erlang-questions" < >>> erlang-questions@REDACTED> >>> Sent: Sunday, 12 June, 2011 02:25:29 GMT +01:00 Amsterdam / Berlin / Bern >>> / Rome / Stockholm / Vienna >>> Subject: [erlang-questions] Documentation Format Changes (erldocs) >>> >>> >>> Hey all >>> >>> I was wondering about the changes in the format documentation sources, in >>> the last release I noticed a few function definitions changed from >>> >>> keymember(Key, N, TupleList) -> boolean() >>> Test for membership of a list of tuples >>> >>> Key = term() >>> N = 1..tuple_size(Tuple) >>> TupleList = [Tuple] >>>   Tuple = tuple() >>> to >>> >>> >>> Merge two key-sorted lists of tuples >>> 1..tuple_size(Tuple) >>> >>> https://github.com/erlang/otp/blob/dev/lib/stdlib/doc/src/lists.xml#L299is the source for these changes, somehow the recent erlang documention ( >>> >>> http://www.erlang.org/doc/man/lists.html#keymerge-3 ) shows the full >>> parameter list information and I am quite confused at how that can be >>> derived from the source xml. >>> >>> I was also just curious about the reason for the change and what was >>> planned for going forward >>> >>> since >>> >>> http://erldocs.com/R14B03/stdlib/lists.html?i=0&search=lists%20keymerge#keymerge/3 >>> is a lot less useful than >>> >>> http://erldocs.com/R14B02/stdlib/lists.html?i=0&search=lists%20keymerge#keymerge/3 >>> >>> Cheers, good work on the new release >>> Dale >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.widgren@REDACTED Wed Oct 12 13:19:13 2011 From: daniel.widgren@REDACTED (Daniel Widgren) Date: Wed, 12 Oct 2011 13:19:13 +0200 Subject: [erlang-questions] non-erlang SNMP Agents Message-ID: Hi, I have some questions. I have a system where a non-erlang system sends snmp traps depending on what is going on. I have tried to set up a snmp manager on another computer to listen to these traps. When I do so I get: =INFO REPORT==== 12-Oct-2011::10:15:36 === SNMPM default user callback received handle_agent: So SNMPM uses handle_agent instead of handle_trap and is wondering why? Is it because I use snmp_default_user? If so, how should I configure a user to listen to a non-erlang snmp system. Do I need a agent and is that agent the snmp system sending traps? Is it possible to listen to all traps that comes from a IP without specify port? regards, Daniel From andy.kriger@REDACTED Wed Oct 12 15:40:53 2011 From: andy.kriger@REDACTED (Andy Kriger) Date: Wed, 12 Oct 2011 09:40:53 -0400 Subject: [erlang-questions] right tool for the job? Message-ID: (I originally posted this to the Tsung mailing list but received no response so I'm broadening my request there to see if anyone with Tsung or Common Test experience might have an opinion on the matter) I have a complex JMeter script that reads in a CSV where each row defines parameters to a test that makes an HTTP request as well as the expectations on the response. The JMeter script has logic to perform the correct assertions based on the incoming parameters. In short, it is not so much a load test as a way of doing many many permutations of HTTP requests (though as a side-effect it does put load on the server because there are so many permutations being run). I'm curious if Tsung allows for this kind of reading in data sets and performing logic based on the data to execute assertions or if it is intended for load testing only. ?Can you extend Tsung with custom Erlang to accomplish more than just load testing? One reason I'm looking at Tsung is that, being Erlang, it offers the benefits of built-in lightweight procs and easy distributed testing. The JMeter test does not run as quickly as I'd like so I'm hoping Erlang can help out there. Would Erlang Common Test be a better approach in this case? Is inets sufficient for spawning the requests and handling the responses or is there a better HTTP client that I should be using? thank youandy kriger From dmercer@REDACTED Wed Oct 12 17:19:24 2011 From: dmercer@REDACTED (David Mercer) Date: Wed, 12 Oct 2011 10:19:24 -0500 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> <006401cc8793$d0751ca0$715f55e0$@com> <4E93875C.4030907@gmail.com> <001401cc883a$28691dc0$793b5940$@com> <4E947F2F.7090307@dev-extend.eu> <005b01cc8841$884421c0$98cc6540$@com> <4E948B2E.1050206@dev-extend.eu> <006801cc885c$6202a110$2607e330$@com> <006c01cc885e$dd30fba0$9792f2e0$@com> Message-ID: <002701cc88f2$54a00c60$fde02520$@com> Aw, y?all are making me work for my opinion. I?m going to have to pull out rebar again and see what my problems with it were. Make shouldn?t have been an issue, since I use make all the time. I?ll let you know what I find out? From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Robert Raschke Sent: Wednesday, October 12, 2011 3:00 AM To: erlang-questions@REDACTED Subject: Re: [erlang-questions] Erlang SMTP Client? On Tue, Oct 11, 2011 at 10:43 PM, David Mercer wrote: On Tuesday, October 11, 2011, Jack Moffitt wrote: > > Rebar, I don't think, is in Erlang, and it doesn't work on all Erlang > > platforms. If it can be rewritten in Erlang, it might work for this > > purpose... > > Um, rebar is most definitely in Erlang. However, you are correct that > it's windows support is a little weaker than the other systems. Mea culpa. I was just guessing, given its lack of Windows support. Why wouldn't it work on Windows, then, if it's in Erlang. Erlang runs on Windows... DBM I've never experienced any issues with running rebar under Windows. The only hurdle is that it is usually invoked from a Makefile. And that's not really a hurdle, more like a bit of uneven ground. ;-) Robby -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Wed Oct 12 18:29:15 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 12 Oct 2011 17:29:15 +0100 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> <006401cc8793$d0751ca0$715f55e0$@com> <4E93875C.4030907@gmail.com> <001401cc883a$28691dc0$793b5940$@com> <4E947F2F.7090307@dev-extend.eu> <005b01cc8841$884421c0$98cc6540$@com> <4E948B2E.1050206@dev-extend.eu> <006801cc885c$6202a110$2607e330$@com> <006c01cc885e$dd30fba0$9792f2e0$@com> Message-ID: On 11 October 2011 22:54, Jack Moffitt wrote: > > Mea culpa. I was just guessing, given its lack of Windows support. Why > > wouldn't it work on Windows, then, if it's in Erlang. Erlang runs on > > Windows... > > I believe it makes a few assumptions about the shell environment. I > believe there are people working to address this if list traffic on > the rebar list is any indication. I imagine some parts of it work just > fine, and I'm sure feedback would be appreciated to find the remaining > rough edges. > > There are indeed a number of people working towards better windows support and it's getting better all the time! -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Wed Oct 12 18:31:46 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 12 Oct 2011 17:31:46 +0100 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: <2E7BEA81-A562-4785-9F4C-2DD306E7A84A@rogvall.se> References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> <006401cc8793$d0751ca0$715f55e0$@com> <4E93875C.4030907@gmail.com> <001401cc883a$28691dc0$793b5940$@com> <4E947F2F.7090307@dev-extend.eu> <005b01cc8841$884421c0$98cc6540$@com> <4E948B2E.1050206@dev-extend.eu> <2E7BEA81-A562-4785-9F4C-2DD306E7A84A@rogvall.se> Message-ID: On 12 October 2011 08:28, Tony Rogvall wrote: > > On 11 okt 2011, at 20:30, Lo?c Hoguin wrote: > > What we need, in my opinion, is not a big standard distribution that can > do everything including distributed coffee, but a big package repository > that would be well integrated into Erlang's tools so we could just have > Erlang pull them when needed. Rebar dependencies are doing half the work > there, all that's missing is a community maintained repository that's > supported officially by a big player (either Erlang/OTP or rebar). > > +1 > > /Tony > I thoroughly agree that this is a great idea! -------------- next part -------------- An HTML attachment was scrubbed... URL: From prof3ta@REDACTED Wed Oct 12 19:29:12 2011 From: prof3ta@REDACTED (Roberto Aloi) Date: Wed, 12 Oct 2011 19:29:12 +0200 Subject: [erlang-questions] Webmachine OAuth server Message-ID: <23CD0FC9-DE40-4E06-B623-8DFFC56DF1AF@gmail.com> Hi all, Do you know if there's a Webmachine OAuth server out there? Roberto Please excuse the brevity of this email. Sent from an iPhone. From ulf.wiger@REDACTED Wed Oct 12 21:18:23 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 12 Oct 2011 21:18:23 +0200 Subject: [erlang-questions] Webmachine OAuth server In-Reply-To: <23CD0FC9-DE40-4E06-B623-8DFFC56DF1AF@gmail.com> References: <23CD0FC9-DE40-4E06-B623-8DFFC56DF1AF@gmail.com> Message-ID: Justin Sheehy says there are a few, but none that are both public and good. He may be able to dig a bit more when you find him online. BR, Ulf (having a beer with Justin after the GOTO conference) On 12 Oct 2011, at 19:29, Roberto Aloi wrote: > Hi all, > > Do you know if there's a Webmachine OAuth server out there? > > Roberto > > Please excuse the brevity of this email. Sent from an iPhone. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From joelr1@REDACTED Wed Oct 12 21:33:50 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 12 Oct 2011 20:33:50 +0100 Subject: [erlang-questions] dialyzer error: apply_op_not_a_variable Message-ID: <2B829087-50C0-4A55-B753-0294DA77D5F9@gmail.com> How do I troubleshoot this? This is running on a code base without type annotations. Thanks, Joel --- make analyze Checking whether the PLT .openpoker_dialyzer.plt is up-to-date... yes Compiling some key modules to native code... done in 0m37.24s Proceeding with analysis... =ERROR REPORT==== 12-Oct-2011::20:29:26 === Error in process <0.2606.0> with exit value: {{apply_op_not_a_variable,tuple},[{dialyzer_dep,traverse,4},{dialyzer_dep,traverse_clauses,6},{dialyzer_dep,traverse,4},{dialyzer_dep,traverse_defs,4},{dialyzer_dep,traverse,4},{dialyzer_dep,analyze,1},{dialyzer_callgraph... dialyzer: Analysis failed with error: {{apply_op_not_a_variable,tuple}, [{dialyzer_dep,traverse,4}, {dialyzer_dep,traverse_clauses,6}, {dialyzer_dep,traverse,4}, {dialyzer_dep,traverse_defs,4}, {dialyzer_dep,traverse,4}, {dialyzer_dep,analyze,1}, {dialyzer_callgraph,reset_from_funs,...}, {dialyzer_analysis_callgraph,...}]} Last messages in the log cache: Reading files and computing callgraph... make: *** [analyze] Error 1 -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From kostis@REDACTED Wed Oct 12 22:44:02 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 12 Oct 2011 23:44:02 +0300 Subject: [erlang-questions] dialyzer error: apply_op_not_a_variable In-Reply-To: <2B829087-50C0-4A55-B753-0294DA77D5F9@gmail.com> References: <2B829087-50C0-4A55-B753-0294DA77D5F9@gmail.com> Message-ID: <4E95FC12.2010808@cs.ntua.gr> On 10/12/11 22:33, Joel Reymont wrote: > How do I troubleshoot this? First you try to dialyze smaller portions of your code base at a time and locate the module where dialyzer crashes. Alternatively, you make a lucky guess which module it is. > dialyzer: Analysis failed with error: {{apply_op_not_a_variable,tuple}, > [{dialyzer_dep,traverse,4}, ... Judging from the error, I am guessing that you most probably have some "old-style fun" application in your code. Something of the form: {M,F}(Arg1,...,ArgN) Change this to M:F(Arg1,...,ArgN) instead which is more kosher, or to a fun, which is more... fun :) Kostis From joelr1@REDACTED Wed Oct 12 22:45:05 2011 From: joelr1@REDACTED (Joel Reymont) Date: Wed, 12 Oct 2011 21:45:05 +0100 Subject: [erlang-questions] dialyzer error: apply_op_not_a_variable In-Reply-To: <4E95FC12.2010808@cs.ntua.gr> References: <2B829087-50C0-4A55-B753-0294DA77D5F9@gmail.com> <4E95FC12.2010808@cs.ntua.gr> Message-ID: On Oct 12, 2011, at 9:44 PM, Kostis Sagonas wrote: > Judging from the error, I am guessing that you most probably have some "old-style fun" application in your code. Parameterized modules maybe? I have one. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From wallentin.dahlberg@REDACTED Wed Oct 12 22:48:00 2011 From: wallentin.dahlberg@REDACTED (=?ISO-8859-1?Q?Bj=F6rn=2DEgil_Dahlberg?=) Date: Wed, 12 Oct 2011 22:48:00 +0200 Subject: [erlang-questions] Erlang SMTP Client? In-Reply-To: References: <000901cc876d$847336f0$8d59a4d0$@com> <20111010184015.GC20887@hijacked.us> <006401cc8793$d0751ca0$715f55e0$@com> <4E93875C.4030907@gmail.com> <001401cc883a$28691dc0$793b5940$@com> <4E947F2F.7090307@dev-extend.eu> <005b01cc8841$884421c0$98cc6540$@com> <4E948B2E.1050206@dev-extend.eu> <2E7BEA81-A562-4785-9F4C-2DD306E7A84A@rogvall.se> Message-ID: 2011/10/12 Tim Watson > On 12 October 2011 08:28, Tony Rogvall wrote: > >> >> On 11 okt 2011, at 20:30, Lo?c Hoguin wrote: >> >> What we need, in my opinion, is not a big standard distribution that can >> do everything including distributed coffee, but a big package repository >> that would be well integrated into Erlang's tools so we could just have >> Erlang pull them when needed. Rebar dependencies are doing half the work >> there, all that's missing is a community maintained repository that's >> supported officially by a big player (either Erlang/OTP or rebar). >> >> +1 >> >> /Tony >> > > I thoroughly agree that this is a great idea! > + 1 agner? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jack@REDACTED Wed Oct 12 23:23:41 2011 From: jack@REDACTED (Jack Moffitt) Date: Wed, 12 Oct 2011 15:23:41 -0600 Subject: [erlang-questions] selectively disabling warnings Message-ID: Is there a way to tell the compiler to ignore certain modules for certain warnings? I have a grammar that produces a parser module, and that parser module includes no typespecs, but I want to enable warn_missing_spec for the rest of this projects code. Also, it seems that warn_missing_spec is not actually documented among the compiler options, but I see it reference in many posts using Google. Is that a bug, or am I not supposed to be using this? jack. From nox@REDACTED Thu Oct 13 10:39:59 2011 From: nox@REDACTED (Anthony Ramine) Date: Thu, 13 Oct 2011 10:39:59 +0200 Subject: [erlang-questions] dialyzer error: apply_op_not_a_variable In-Reply-To: References: <2B829087-50C0-4A55-B753-0294DA77D5F9@gmail.com> <4E95FC12.2010808@cs.ntua.gr> Message-ID: <26FFACA6-85BF-4B92-A39F-5B855A979DA7@dev-extend.eu> Hi, Parameterized modules calls are {M, ...}:F(Args...). Old-style funs are {M, F}(Args). Regards, Le 12 oct. 2011 ? 22:45, Joel Reymont a ?crit : > > On Oct 12, 2011, at 9:44 PM, Kostis Sagonas wrote: > >> Judging from the error, I am guessing that you most probably have some "old-style fun" application in your code. > > Parameterized modules maybe? > > I have one. -- Anthony Ramine / @nokusu Dev:Extend ? http://dev-extend.eu/ So as I pray, ?Unlimited Erlang Works? From arjan@REDACTED Thu Oct 13 13:51:47 2011 From: arjan@REDACTED (Arjan Scherpenisse) Date: Thu, 13 Oct 2011 13:51:47 +0200 Subject: [erlang-questions] [Zotonic] How to distribute zotonic across multiple machines? In-Reply-To: References: Message-ID: <4E96D0D3.3090905@scherpenisse.net> Hi Barco, I don't think this is currently possible. What kind of application are you building that demands this? Last week a big Zotonic site was pulling 35MBit without breaking a sweat, from a single VPS. Zotonic's principal author, Marc Worrell, is researching Zotonic's distribution model in a project called "Elastic Zotonic", however, as of yet it is still in a research phase. kind regards, Arjan On 10/09/2011 07:19 AM, Barco You wrote: > Thanks! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From barcojie@REDACTED Thu Oct 13 14:16:50 2011 From: barcojie@REDACTED (Barco You) Date: Thu, 13 Oct 2011 20:16:50 +0800 Subject: [erlang-questions] [Zotonic] How to distribute zotonic across multiple machines? In-Reply-To: <4E96D0D3.3090905@scherpenisse.net> References: <4E96D0D3.3090905@scherpenisse.net> Message-ID: Hi Arjan, Thanks for your reply. I just wander whether zotonic is good for internet TV service, which may support millions of user's viewing. Best regards, Barco On Oct 13, 2011 7:51 PM, "Arjan Scherpenisse" wrote: > Hi Barco, > > I don't think this is currently possible. What kind of application are > you building that demands this? Last week a big Zotonic site was pulling > 35MBit without breaking a sweat, from a single VPS. > > Zotonic's principal author, Marc Worrell, is researching Zotonic's > distribution model in a project called "Elastic Zotonic", however, as of > yet it is still in a research phase. > > kind regards, > Arjan > > On 10/09/2011 07:19 AM, Barco You wrote: > > Thanks! > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From enewhuis@REDACTED Thu Oct 13 15:24:12 2011 From: enewhuis@REDACTED (Eric Newhuis (personal)) Date: Thu, 13 Oct 2011 08:24:12 -0500 Subject: [erlang-questions] Define "exists". Message-ID: <853BB2AF-672A-4C36-B00D-8918554BE2DB@gmail.com> The problem is with list_to_existing_atom/1. Really it is a great idea. But... 1. I have raw query strings coming in from the cruel outside world. 2. I convert some bits of those into atoms by way of list_to_existing_atom so that someone cannot kill me with DOS attack by bloating my atom table. 3. With me so far? Good. 4. My server, an OTP app, has a .app file that lists several other apps. 5. list_to_existing_atom fails on atoms defined in those several other apps. Suggestion: The documentation should specify what "exists" really means. Question: What should I do to force the atoms from my dependent apps to be loaded? I've been manually calling Module:module_info/0 just in time but, alas, this is starting to fail due to other module dependencies that are unknown at my call sites. For reference, here is the existing Erlang doc from the .org site: http://www.erlang.org/doc/man/erlang.html#list_to_existing_atom-1 list_to_existing_atom(String) -> atom() Types: String = string() Returns the atom whose text representation is String, but only if there already exists such atom. Failure: badarg if there does not already exist an atom whose text representation is String. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Thu Oct 13 15:28:55 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 13 Oct 2011 15:28:55 +0200 Subject: [erlang-questions] Define "exists". In-Reply-To: <853BB2AF-672A-4C36-B00D-8918554BE2DB@gmail.com> References: <853BB2AF-672A-4C36-B00D-8918554BE2DB@gmail.com> Message-ID: Hi, I think that the modules defining the atoms must be loaded when you call to_existing_atom. Can you preloaded those? Regards, vlad On Oct 13, 2011 3:24 PM, "Eric Newhuis (personal)" wrote: > The problem is with list_to_existing_atom/1. Really it is a great idea. > But... > > 1. I have raw query strings coming in from the cruel outside world. > 2. I convert some bits of those into atoms by way of list_to_existing_atom > so that someone cannot kill me with DOS attack by bloating my atom table. > 3. With me so far? Good. > 4. My server, an OTP app, has a .app file that lists several other apps. > 5. list_to_existing_atom fails on atoms defined in those several other > apps. > > Suggestion: The documentation should specify what "exists" really means. > > Question: What should I do to force the atoms from my dependent apps to be > loaded? I've been manually calling Module:module_info/0 just in time but, > alas, this is starting to fail due to other module dependencies that are > unknown at my call sites. > > For reference, here is the existing Erlang doc from the .org site: > > http://www.erlang.org/doc/man/erlang.html#list_to_existing_atom-1 > > list_to_existing_atom(String) -> atom() > > Types: > String = string() > > Returns the atom whose text representation is String, but only if there > already exists such atom. > > Failure: badarg if there does not already exist an atom whose text > representation is String. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Thu Oct 13 15:31:33 2011 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 13 Oct 2011 14:31:33 +0100 Subject: [erlang-questions] typer not finding include files Message-ID: <5181CB03-D14E-4911-B696-F9E4B3D2F446@gmail.com> What am I missing? Thanks, Joel --- ls -la release/openpoker/lib/piqi-0.5.6/include/piqirun.hrl -rw-r--r-- 1 joelr staff 2197 Aug 24 20:11 release/openpoker/lib/piqi-0.5.6/include/piqirun.hrl typer --plt openpoker_dialyzer.plt --show -r . -I include -I release/openpoker/lib -pa release/openpoker/lib typer: Analysis failed with error report: ./src/xxx_response.erl:4: can't find include lib "piqi/include/piqirun.hrl" -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From vm@REDACTED Thu Oct 13 15:36:18 2011 From: vm@REDACTED (Floris van Manen) Date: Thu, 13 Oct 2011 15:36:18 +0200 Subject: [erlang-questions] discrete event simulation In-Reply-To: <853BB2AF-672A-4C36-B00D-8918554BE2DB@gmail.com> References: <853BB2AF-672A-4C36-B00D-8918554BE2DB@gmail.com> Message-ID: <08F9D6F8-DBC6-4A79-BADC-195CB3610EC3@klankschap.nl> While googling around for (simple) erlang implementations of discrete event simulations, i come across a single paper about the subject. "Discrete Event Simulation in Erlang" by A. Ermedahl (1995) Does it mean that is a non issue, or an ultimate example of write once use for ever ? Is there more information available ? Thanks .Floris From joelr1@REDACTED Thu Oct 13 15:38:10 2011 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 13 Oct 2011 14:38:10 +0100 Subject: [erlang-questions] typer not finding include files In-Reply-To: <5181CB03-D14E-4911-B696-F9E4B3D2F446@gmail.com> References: <5181CB03-D14E-4911-B696-F9E4B3D2F446@gmail.com> Message-ID: <7017FA79-54AE-4923-B823-E3BD5D820412@gmail.com> Solved thusly typer --plt openpoker_dialyzer.plt --show -r . -I include -pa release/openpoker/lib/piqi-0.5.6/ebin On Oct 13, 2011, at 2:31 PM, Joel Reymont wrote: > What am I missing? > > Thanks, Joel > > --- > > ls -la release/openpoker/lib/piqi-0.5.6/include/piqirun.hrl > -rw-r--r-- 1 joelr staff 2197 Aug 24 20:11 release/openpoker/lib/piqi-0.5.6/include/piqirun.hrl > > typer --plt openpoker_dialyzer.plt --show -r . -I include -I release/openpoker/lib -pa release/openpoker/lib > typer: Analysis failed with error report: > ./src/xxx_response.erl:4: can't find include lib "piqi/include/piqirun.hrl" -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From fred.hebert@REDACTED Thu Oct 13 15:41:56 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Thu, 13 Oct 2011 09:41:56 -0400 Subject: [erlang-questions] Define "exists". In-Reply-To: <853BB2AF-672A-4C36-B00D-8918554BE2DB@gmail.com> References: <853BB2AF-672A-4C36-B00D-8918554BE2DB@gmail.com> Message-ID: <38B56804-0ECB-4034-8AC0-FFEF6B5AB694@erlang-solutions.com> The existing atoms is simply a question of what atoms were loaded into the VM's atom table. Atoms are loaded when you evaluate them, have them in loaded modules, etc. Until then, they have not been declared and are not counted as existing. As for loading dependent atoms, have you tried to load applications? By doing application:load(AppName), Erlang should take care of loading all the app files in these applications, creating the required atoms when you need them, if they're in there. It won't load the object code for you, but you could do it yourself. In any case, it worries me that you're getting errors on dependencies and whatnot. To me, it sounds like you're trying to protect yourself against the public, but the public is allowed to call atoms matching modules and function calls -- are you building some kind of RPC module matching strings to modules? If so, your approach might be dangerous; the only safe option there is usually a white list. Am I wrong in assuming that's what you're trying to do? -- Fred H?bert http://www.erlang-solutions.com On 2011-10-13, at 09:24 AM, Eric Newhuis (personal) wrote: > The problem is with list_to_existing_atom/1. Really it is a great idea. But... > > 1. I have raw query strings coming in from the cruel outside world. > 2. I convert some bits of those into atoms by way of list_to_existing_atom so that someone cannot kill me with DOS attack by bloating my atom table. > 3. With me so far? Good. > 4. My server, an OTP app, has a .app file that lists several other apps. > 5. list_to_existing_atom fails on atoms defined in those several other apps. > > Suggestion: The documentation should specify what "exists" really means. > > Question: What should I do to force the atoms from my dependent apps to be loaded? I've been manually calling Module:module_info/0 just in time but, alas, this is starting to fail due to other module dependencies that are unknown at my call sites. > > For reference, here is the existing Erlang doc from the .org site: > > http://www.erlang.org/doc/man/erlang.html#list_to_existing_atom-1 > list_to_existing_atom(String) -> atom() > > Types: > > String = string() > > Returns the atom whose text representation is String, but only if there already exists such atom. > > Failure: badarg if there does not already exist an atom whose text representation is String. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Thu Oct 13 15:52:48 2011 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 13 Oct 2011 14:52:48 +0100 Subject: [erlang-questions] dialyzer error: apply_op_not_a_variable In-Reply-To: <26FFACA6-85BF-4B92-A39F-5B855A979DA7@dev-extend.eu> References: <2B829087-50C0-4A55-B753-0294DA77D5F9@gmail.com> <4E95FC12.2010808@cs.ntua.gr> <26FFACA6-85BF-4B92-A39F-5B855A979DA7@dev-extend.eu> Message-ID: <68A9F424-FD23-4FC4-8DE2-CA505FB753E5@gmail.com> This is the problem ({?MODULE, State})(Event, Bot) On Oct 13, 2011, at 9:39 AM, Anthony Ramine wrote: > Hi, > > Parameterized modules calls are {M, ...}:F(Args...). > Old-style funs are {M, F}(Args). -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From kostis@REDACTED Thu Oct 13 16:33:53 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 13 Oct 2011 17:33:53 +0300 Subject: [erlang-questions] dialyzer error: apply_op_not_a_variable In-Reply-To: <68A9F424-FD23-4FC4-8DE2-CA505FB753E5@gmail.com> References: <2B829087-50C0-4A55-B753-0294DA77D5F9@gmail.com> <4E95FC12.2010808@cs.ntua.gr> <26FFACA6-85BF-4B92-A39F-5B855A979DA7@dev-extend.eu> <68A9F424-FD23-4FC4-8DE2-CA505FB753E5@gmail.com> Message-ID: <4E96F6D1.5050101@cs.ntua.gr> On 10/13/11 16:52, Joel Reymont wrote: > This is the problem > > ({?MODULE, State})(Event, Bot) Exactly what I wrote: >> Judging from the error, I am guessing that you most probably have >> some "old-style fun" application in your code. Something of the >> form: >> >> {M,F}(Arg1,...,ArgN) >> >> Change this to M:F(Arg1,...,ArgN) instead which is more kosher Kostis From ulf.wiger@REDACTED Thu Oct 13 17:59:35 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 13 Oct 2011 17:59:35 +0200 Subject: [erlang-questions] Define "exists". In-Reply-To: <853BB2AF-672A-4C36-B00D-8918554BE2DB@gmail.com> References: <853BB2AF-672A-4C36-B00D-8918554BE2DB@gmail.com> Message-ID: <13E309D1-6D9E-4B92-A947-04CA1E881033@erlang-solutions.com> On 13 Oct 2011, at 15:24, Eric Newhuis (personal) wrote: > Question: What should I do to force the atoms from my dependent apps to be loaded? I've been manually calling Module:module_info/0 just in time but, alas, this is starting to fail due to other module dependencies that are unknown at my call sites. Eric, I presume you have a proper boot script? If you start the system with the command-line option 'mode -embedded', all modules will be loaded at boot time. This also turns off dynamic code loading - no free lunch here. You can still explicitly load modules, but unfortunately (?), it's not possible to tell the code_server to go from embedded to 'interactive' mode. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From carlsson.richard@REDACTED Thu Oct 13 18:33:34 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 13 Oct 2011 18:33:34 +0200 Subject: [erlang-questions] dialyzer error: apply_op_not_a_variable In-Reply-To: <68A9F424-FD23-4FC4-8DE2-CA505FB753E5@gmail.com> References: <2B829087-50C0-4A55-B753-0294DA77D5F9@gmail.com> <4E95FC12.2010808@cs.ntua.gr> <26FFACA6-85BF-4B92-A39F-5B855A979DA7@dev-extend.eu> <68A9F424-FD23-4FC4-8DE2-CA505FB753E5@gmail.com> Message-ID: <4E9712DE.4090007@gmail.com> On 10/13/2011 03:52 PM, Joel Reymont wrote: > This is the problem > > ({?MODULE, State})(Event, Bot) This depends on knowing that the current implementation of parameterized modules uses the representation {Mod, X1, ...Xn}. Don't create such tuples directly: call Mod:new(State) instead. If your snippet above is literally what's in your code, then you're doing something even stranger: having a module create an instance of itself (the tuple) and then call a function on that instance. Don't abuse the current underlying mechanism - if you want to use parameterized modules, then declare them as such: -module(mymodule, [X]). and use the automatic mymodule:new(X) function to instantiate them. In any case, parameterizing modules on a "state" variable is a pretty fishy thing to do. You might want to rethink your approach. /Richard From joelr1@REDACTED Thu Oct 13 18:36:29 2011 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 13 Oct 2011 17:36:29 +0100 Subject: [erlang-questions] dialyzer error: apply_op_not_a_variable In-Reply-To: <4E9712DE.4090007@gmail.com> References: <2B829087-50C0-4A55-B753-0294DA77D5F9@gmail.com> <4E95FC12.2010808@cs.ntua.gr> <26FFACA6-85BF-4B92-A39F-5B855A979DA7@dev-extend.eu> <68A9F424-FD23-4FC4-8DE2-CA505FB753E5@gmail.com> <4E9712DE.4090007@gmail.com> Message-ID: On Oct 13, 2011, at 5:33 PM, Richard Carlsson wrote: > On 10/13/2011 03:52 PM, Joel Reymont wrote: >> This is the problem >> >> ({?MODULE, State})(Event, Bot) > > If your snippet above is literally what's in your code, then you're doing something even stranger: having a module create an instance of itself (the tuple) and then call a function on that instance. Don't abuse the current underlying mechanism - if you want to use parameterized modules, then declare them as such: All I'm doing is ?MODULE:State(Event, Bot) -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Thu Oct 13 18:42:51 2011 From: joelr1@REDACTED (Joel Reymont) Date: Thu, 13 Oct 2011 17:42:51 +0100 Subject: [erlang-questions] return type spec Message-ID: Is there a way to say that the return of this function is that of Mod:Fun, e.g. supervisor:start_child? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From carlsson.richard@REDACTED Thu Oct 13 18:54:59 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 13 Oct 2011 18:54:59 +0200 Subject: [erlang-questions] dialyzer error: apply_op_not_a_variable In-Reply-To: References: <2B829087-50C0-4A55-B753-0294DA77D5F9@gmail.com> <4E95FC12.2010808@cs.ntua.gr> <26FFACA6-85BF-4B92-A39F-5B855A979DA7@dev-extend.eu> <68A9F424-FD23-4FC4-8DE2-CA505FB753E5@gmail.com> <4E9712DE.4090007@gmail.com> Message-ID: <4E9717E3.4010702@gmail.com> On 10/13/2011 06:36 PM, Joel Reymont wrote: > > On Oct 13, 2011, at 5:33 PM, Richard Carlsson wrote: > >> On 10/13/2011 03:52 PM, Joel Reymont wrote: >>> This is the problem >>> >>> ({?MODULE, State})(Event, Bot) >> >> If your snippet above is literally what's in your code, then you're doing something even stranger: having a module create an instance of itself (the tuple) and then call a function on that instance. Don't abuse the current underlying mechanism - if you want to use parameterized modules, then declare them as such: > > > All I'm doing is ?MODULE:State(Event, Bot) Ugh, sorry for the noise, I misread. Yes, you're using an "old-style fun", not a parameterized module. (They have the same kind of representation). If the format of your function call is indeed ?MODULE:State(Event, Bot) then there should be no problem (assuming State is an atom). If you need to pass around the {?MODULE, State} as in your first example, then use Fun=erlang:make_fun(ModuleName, FunctionName, 2). This lets you call Fun(Event, Bot) at the place that gets Fun. (It would be nice if you could just say Fun=fun M:F/A, where M, F, and A are variables, but the language doesn't support that yet, so you can only use that syntax if M, F, and A are constant literals.) /Richard From kostis@REDACTED Thu Oct 13 19:36:41 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 13 Oct 2011 20:36:41 +0300 Subject: [erlang-questions] return type spec In-Reply-To: References: Message-ID: <4E9721A9.4010100@cs.ntua.gr> On 10/13/11 19:42, Joel Reymont wrote: > Is there a way to say that the return of this function is that of Mod:Fun, e.g. supervisor:start_child? The only way is if the 'supervisor' module exports a type start_child_ret() which is the return type of start_child and then you can use it as a remote type in your module with the notation: supervisor:start_child_ret() Kostis From zabrane3@REDACTED Thu Oct 13 20:07:46 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 13 Oct 2011 20:07:46 +0200 Subject: [erlang-questions] Dennis Ritchie, 1941-2011: Computer scientist, Unix co-creator, C programming language co-inventor Message-ID: io:format("RIP Dennis~n"). https://plus.google.com/u/0/101960720994009339267/posts/ENuEDDYfvKP?hl=en http://moviecitynews.com/2011/10/rip-dennis-ritchie-70-author-of-c-co-author-of-unix-computer-language-219/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Thu Oct 13 21:48:43 2011 From: jwatte@REDACTED (Jon Watte) Date: Thu, 13 Oct 2011 12:48:43 -0700 Subject: [erlang-questions] Define "exists". In-Reply-To: <853BB2AF-672A-4C36-B00D-8918554BE2DB@gmail.com> References: <853BB2AF-672A-4C36-B00D-8918554BE2DB@gmail.com> Message-ID: Why do they need to be atoms? We have a very similar case, where we take HTTP headers from calls to our own servers and turn them into proplists, with atoms for the header kinds ("content-type," "cache-control" etc). However, we'd probably be just as well off, if not better, to use binaries as the keys -- we get those straight out of the HTTP request anyway. Then we don't have the problem of growing atom store, and/or having to pre-declare all possible headers we might care about. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Thu, Oct 13, 2011 at 6:24 AM, Eric Newhuis (personal) wrote: > The problem is with list_to_existing_atom/1. Really it is a great idea. > But... > > 1. I have raw query strings coming in from the cruel outside world. > 2. I convert some bits of those into atoms by way of list_to_existing_atom > so that someone cannot kill me with DOS attack by bloating my atom table. > 3. With me so far? Good. > 4. My server, an OTP app, has a .app file that lists several other apps. > 5. list_to_existing_atom fails on atoms defined in those several other > apps. > > Suggestion: The documentation should specify what "exists" really means. > > Question: What should I do to force the atoms from my dependent apps to be > loaded? I've been manually calling Module:module_info/0 just in time but, > alas, this is starting to fail due to other module dependencies that are > unknown at my call sites. > > For reference, here is the existing Erlang doc from the .org site: > > http://www.erlang.org/doc/man/erlang.html#list_to_existing_atom-1 > > list_to_existing_atom(String) -> atom() > > Types: > String = string() > > Returns the atom whose text representation is String, but only if there > already exists such atom. > > Failure: badarg if there does not already exist an atom whose text > representation is String. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Thu Oct 13 21:52:00 2011 From: jwatte@REDACTED (Jon Watte) Date: Thu, 13 Oct 2011 12:52:00 -0700 Subject: [erlang-questions] dialyzer error: apply_op_not_a_variable In-Reply-To: <4E9717E3.4010702@gmail.com> References: <2B829087-50C0-4A55-B753-0294DA77D5F9@gmail.com> <4E95FC12.2010808@cs.ntua.gr> <26FFACA6-85BF-4B92-A39F-5B855A979DA7@dev-extend.eu> <68A9F424-FD23-4FC4-8DE2-CA505FB753E5@gmail.com> <4E9712DE.4090007@gmail.com> <4E9717E3.4010702@gmail.com> Message-ID: In this context: what is the validity of funs in terms of code reloading? Specifically, if I define a fun and hang on to it "for a long time," I generally find that code reloading to the point where I purge the code that initially defined the fun, will cause an exception the next time that fun is invoked. If this is actual and intended behavior, then I think I also understand why it should be so. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Thu, Oct 13, 2011 at 9:54 AM, Richard Carlsson < carlsson.richard@REDACTED> wrote: > On 10/13/2011 06:36 PM, Joel Reymont wrote: > >> >> On Oct 13, 2011, at 5:33 PM, Richard Carlsson wrote: >> >> On 10/13/2011 03:52 PM, Joel Reymont wrote: >>> >>>> This is the problem >>>> >>>> ({?MODULE, State})(Event, Bot) >>>> >>> >>> If your snippet above is literally what's in your code, then you're doing >>> something even stranger: having a module create an instance of itself (the >>> tuple) and then call a function on that instance. Don't abuse the current >>> underlying mechanism - if you want to use parameterized modules, then >>> declare them as such: >>> >> >> >> All I'm doing is ?MODULE:State(Event, Bot) >> > > Ugh, sorry for the noise, I misread. Yes, you're using an "old-style fun", > not a parameterized module. (They have the same kind of representation). > > If the format of your function call is indeed ?MODULE:State(Event, Bot) > then there should be no problem (assuming State is an atom). > > If you need to pass around the {?MODULE, State} as in your first example, > then use Fun=erlang:make_fun(**ModuleName, FunctionName, 2). This lets you > call Fun(Event, Bot) at the place that gets Fun. > > (It would be nice if you could just say Fun=fun M:F/A, where M, F, and A > are variables, but the language doesn't support that yet, so you can only > use that syntax if M, F, and A are constant literals.) > > /Richard > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jack@REDACTED Thu Oct 13 22:37:51 2011 From: jack@REDACTED (Jack Moffitt) Date: Thu, 13 Oct 2011 14:37:51 -0600 Subject: [erlang-questions] Define "exists". In-Reply-To: References: <853BB2AF-672A-4C36-B00D-8918554BE2DB@gmail.com> Message-ID: > However, we'd probably be just as well off, if not better, to use binaries > as the keys -- we get those straight out of the HTTP request anyway. Then we > don't have the problem of growing atom store, and/or having to pre-declare > all possible headers we might care about. I often do this and make the getters/setters convert atoms to binary so that I can still avoid typing the <<" and ">> as much as possible. Of course, this won't help if you are pattern matching. jack. From nicolas@REDACTED Fri Oct 14 00:11:07 2011 From: nicolas@REDACTED (Nicolas Niclausse) Date: Fri, 14 Oct 2011 00:11:07 +0200 Subject: [erlang-questions] right tool for the job? In-Reply-To: References: Message-ID: <4E9761FB.5030504@niclux.org> Andy Kriger ecrivait le 12/10/2011 15:40: > (I originally posted this to the Tsung mailing list but received no > response so I'm broadening my request there to see if anyone with > Tsung or Common Test experience might have an opinion on the matter) I can't find your mail in the archive. > I have a complex JMeter script that reads in a CSV where each row > defines parameters to a test that makes an HTTP request as well as the > expectations on the response. The JMeter script has logic to perform > the correct assertions based on the incoming parameters. In short, it > is not so much a load test as a way of doing many many permutations of > HTTP requests (though as a side-effect it does put load on the server > because there are so many permutations being run). > I'm curious if Tsung allows for this kind of reading in data sets and > performing logic based on the data to execute assertions or if it is > intended for load testing only. Can you extend Tsung with custom > Erlang to accomplish more than just load testing? I think you can do this in tsung. For example, if you have a csv with 2 columns, url + expected response, you can: ... %%_response%% you can also include erlang code inside the tsung configuration file, see for example: http://lists.process-one.net/pipermail/tsung-users/2011-October/001797.html -- Nicolas From erlang@REDACTED Fri Oct 14 10:30:15 2011 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 14 Oct 2011 10:30:15 +0200 Subject: [erlang-questions] servers for mountable remote file systems Message-ID: Hello, I want to make a remote virtual device that I can mount on mac-os X/linux etc. It seems like there are three alternatives a) webdav b) samba/smb c) NFS I need an Erlang server that implements a virtual file system as a back-end to the protocol in question. I can imagine emulating a large number of small files - the mounted file system would appear to have a large number of small files - but in fact these would not exist but be emulated in the server. Has anybody implemented servers for the above protocols in Erlang? or do I have to start from scratch? Is any of these protocols to be preferred over the others, given that I want to emulate a large number of small files. Is my list a) - c) complete are there other protocols I should consider? /Joe From rtrlists@REDACTED Fri Oct 14 10:42:10 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Fri, 14 Oct 2011 09:42:10 +0100 Subject: [erlang-questions] servers for mountable remote file systems In-Reply-To: References: Message-ID: On Fri, Oct 14, 2011 at 9:30 AM, Joe Armstrong wrote: > Hello, > > I want to make a remote virtual device that I can mount on mac-os X/linux > etc. > > It seems like there are three alternatives > > a) webdav > b) samba/smb > c) NFS > > I need an Erlang server that implements a virtual file system as a > back-end to the protocol > in question. > > I can imagine emulating a large number of small files - the mounted > file system would appear to have a > large number of small files - but in fact these would not exist but be > emulated in the server. > > Has anybody implemented servers for the above protocols in Erlang? or > do I have to start from > scratch? Is any of these protocols to be preferred over the others, > given that I want to emulate > a large number of small files. Is my list a) - c) complete are there > other protocols I should consider? > > /Joe > > If it's OK to also supply some kind of client to your users, then I'd recommend targetting 9P (http://en.wikipedia.org/wiki/9P) as it is at least one order of magnitude simpler than NFS or samba (not sure about webdav). There's a Linux implementation for it here: http://sourceforge.net/apps/mediawiki/v9fs And the original documentation is here: http://cm.bell-labs.com/sys/man/5/INDEX.html The canonical example of a 9P server is ramfs. In C: http://swtch.com/plan9port/man/man4/ramfs.html In Go: http://code.google.com/p/go9p/source/browse/p/srv/examples/ramfs.go Robby -------------- next part -------------- An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Fri Oct 14 10:45:12 2011 From: gleber.p@REDACTED (Gleb Peregud) Date: Fri, 14 Oct 2011 10:45:12 +0200 Subject: [erlang-questions] servers for mountable remote file systems In-Reply-To: References: Message-ID: On Fri, Oct 14, 2011 at 10:42, Robert Raschke wrote: >> I want to make a remote virtual device that I can mount on mac-os X/linux >> etc. >> >> It seems like there are three alternatives Don't forget about sshfs too. Since Erlang has somehow-complete ssh server implementation, sshfs may be not that hard to implement. From connorsml@REDACTED Fri Oct 14 11:48:09 2011 From: connorsml@REDACTED (Michael Connors) Date: Fri, 14 Oct 2011 11:48:09 +0200 Subject: [erlang-questions] [Zotonic] How to distribute zotonic across multiple machines? In-Reply-To: References: Message-ID: Hi Barco, I have spent sometime thinking about this, but I have not actually needed to do it yet. If I was attempting to do it, I would probably be using the following tools. You could put Zotonic behind nginx and use nginx as a load balancer with sticky sessions, so that the user gets sent to the site that has their session. You could replicate Postgresql using Slony so you have your database distributed, and use pgpool to ensue that all writes go to the master database and all reads come from the local database. You could replicate the images and other files using rsync. http://nginx.org/ http://slony.info/ http://pgpool.projects.postgresql.org/ Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From jodie.burch@REDACTED Fri Oct 14 11:56:20 2011 From: jodie.burch@REDACTED (Jodie Burch) Date: Fri, 14 Oct 2011 10:56:20 +0100 Subject: [erlang-questions] Erlang User Conference 2011 - Early Bird Rate Ends Tomorrow Message-ID: Hi Just to let you all know that the Early Bird Rate for this year?s Erlang User Conference ends tomorrow. This year there will be 2 additional tracks, offering your more choice and variety of talks and speakers. You can view the full programme here: http://www.erlang-factory.com/conference/ErlangUserConference2011/programme To book you place, use the below link https://www.erlang-factory.com/conference/ErlangUserConference2011/register We can?t wait to see you there! Thanks Jodie -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Fri Oct 14 14:07:00 2011 From: tony@REDACTED (Tony Rogvall) Date: Fri, 14 Oct 2011 14:07:00 +0200 Subject: [erlang-questions] servers for mountable remote file systems In-Reply-To: References: Message-ID: <4967D497-8424-4763-A08E-26B01299BB6A@rogvall.se> Hi joe! Luke did hack a cool erlang NFS server 10 years ago :-) http://jungerl.cvs.sourceforge.net/viewvc/jungerl/jungerl/lib/enfs/ Nothing new under the sun ! /Tony On 14 okt 2011, at 10:30, Joe Armstrong wrote: > Hello, > > I want to make a remote virtual device that I can mount on mac-os X/linux etc. > > It seems like there are three alternatives > > a) webdav > b) samba/smb > c) NFS > > I need an Erlang server that implements a virtual file system as a > back-end to the protocol > in question. > > I can imagine emulating a large number of small files - the mounted > file system would appear to have a > large number of small files - but in fact these would not exist but be > emulated in the server. > > Has anybody implemented servers for the above protocols in Erlang? or > do I have to start from > scratch? Is any of these protocols to be preferred over the others, > given that I want to emulate > a large number of small files. Is my list a) - c) complete are there > other protocols I should consider? > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From barcojie@REDACTED Fri Oct 14 16:40:49 2011 From: barcojie@REDACTED (Barco You) Date: Fri, 14 Oct 2011 22:40:49 +0800 Subject: [erlang-questions] [Zotonic] How to distribute zotonic across multiple machines? In-Reply-To: References: Message-ID: Hi Michael, Thank you very much for your great suggestion. I've been actually also thinking of using nginx, but dont know slony and pgpool. I will try them. Thank you very much! Best regards, Barco On Oct 14, 2011 5:48 PM, "Michael Connors" wrote: > Hi Barco, > > I have spent sometime thinking about this, but I have not actually needed > to do it yet. > > If I was attempting to do it, I would probably be using the following > tools. > > You could put Zotonic behind nginx and use nginx as a load balancer with > sticky sessions, so that the user gets sent to the site that has their > session. > > You could replicate Postgresql using Slony so you have your database > distributed, and use pgpool to ensue that all writes go to the master > database and all reads come from the local database. > > You could replicate the images and other files using rsync. > > http://nginx.org/ > http://slony.info/ > http://pgpool.projects.postgresql.org/ > > Michael > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sunwood360@REDACTED Fri Oct 14 17:26:59 2011 From: sunwood360@REDACTED (envelopes envelopes) Date: Fri, 14 Oct 2011 08:26:59 -0700 Subject: [erlang-questions] [Zotonic] How to distribute zotonic across multiple machines? In-Reply-To: References: Message-ID: Will the write fail if the master node is down? On Oct 14, 2011 2:48 AM, "Michael Connors" wrote: > Hi Barco, > > I have spent sometime thinking about this, but I have not actually needed > to do it yet. > > If I was attempting to do it, I would probably be using the following > tools. > > You could put Zotonic behind nginx and use nginx as a load balancer with > sticky sessions, so that the user gets sent to the site that has their > session. > > You could replicate Postgresql using Slony so you have your database > distributed, and use pgpool to ensue that all writes go to the master > database and all reads come from the local database. > > You could replicate the images and other files using rsync. > > http://nginx.org/ > http://slony.info/ > http://pgpool.projects.postgresql.org/ > > Michael > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From juanjo@REDACTED Fri Oct 14 17:28:49 2011 From: juanjo@REDACTED (Juan Jose Comellas) Date: Fri, 14 Oct 2011 12:28:49 -0300 Subject: [erlang-questions] [Zotonic] How to distribute zotonic across multiple machines? In-Reply-To: References: Message-ID: Slony-I works very well. I've used it for pre-9.0 versions of Postgres with somewhat big databases (> 60 GB, several tables with over 60 million rows) and with lots of concurrent read and writes and it could handle them without problems. Slony modifies your schema and adds triggers to all the tables that are to be replicated. It is a bit write-heavy, as it accumulates all the changes in tables in its own schema and then ships them to the slave servers. To take advantage of the fact that you multiple database servers, Zotonic would need to be able to write to the master server and read from all the servers (master and slaves). I don't think it can do that yet. I probably shouldn't even mention it, but make sure that all the tables have primary indexes. Slony will not be able to replicate a table without that. Regrettably, I've seen several cases of people not using primary keys as an "optimization" to avoid the cost of the primary key index (when dealing with very large tables every byte counts). Slony can compensate that by adding its own primary key, but once you do that your schema and Slony become tied at the hip. Replication is one of the features that are made much easier (even trivial) by several NoSQL databases (CouchDB and Riak are prime examples). Hope this helps. Juanjo On Fri, Oct 14, 2011 at 11:40 AM, Barco You wrote: > Hi Michael, > > Thank you very much for your great suggestion. I've been actually also > thinking of using nginx, but dont know slony and pgpool. I will try them. > Thank you very much! > > Best regards, > Barco > On Oct 14, 2011 5:48 PM, "Michael Connors" wrote: > >> Hi Barco, >> >> I have spent sometime thinking about this, but I have not actually needed >> to do it yet. >> >> If I was attempting to do it, I would probably be using the following >> tools. >> >> You could put Zotonic behind nginx and use nginx as a load balancer with >> sticky sessions, so that the user gets sent to the site that has their >> session. >> >> You could replicate Postgresql using Slony so you have your database >> distributed, and use pgpool to ensue that all writes go to the master >> database and all reads come from the local database. >> >> You could replicate the images and other files using rsync. >> >> http://nginx.org/ >> http://slony.info/ >> http://pgpool.projects.postgresql.org/ >> >> Michael >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From connorsml@REDACTED Fri Oct 14 17:29:25 2011 From: connorsml@REDACTED (Michael Connors) Date: Fri, 14 Oct 2011 17:29:25 +0200 Subject: [erlang-questions] [Zotonic] How to distribute zotonic across multiple machines? In-Reply-To: References: Message-ID: On 14 October 2011 17:26, envelopes envelopes wrote: > Will the write fail if the master node is down? > > It will, but at least your users will have read access during that time. -------------- next part -------------- An HTML attachment was scrubbed... URL: From connorsml@REDACTED Fri Oct 14 17:31:05 2011 From: connorsml@REDACTED (Michael Connors) Date: Fri, 14 Oct 2011 17:31:05 +0200 Subject: [erlang-questions] [Zotonic] How to distribute zotonic across multiple machines? In-Reply-To: References: Message-ID: > > To take advantage of the fact that you multiple database servers, Zotonic > would need to be able to write to the master server and read from all the > servers (master and slaves). I don't think it can do that yet. > > That is where pgpool comes in. -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc@REDACTED Fri Oct 14 18:30:12 2011 From: marc@REDACTED (Marc Worrell) Date: Fri, 14 Oct 2011 18:30:12 +0200 Subject: [erlang-questions] [Zotonic] How to distribute zotonic across multiple machines? In-Reply-To: References: Message-ID: A short note. We are working on the distributed version of zotonic. One of the problems we are tackling is the cache synchronization between nodes. Objective is to be fault tolerant (read partition tolerant). Later this month(s) more information while we define our puzzle pieces. - Marc Sent from my iPhone On 14 okt. 2011, at 17:29, Michael Connors wrote: > > > On 14 October 2011 17:26, envelopes envelopes wrote: > Will the write fail if the master node is down? > > > > It will, but at least your users will have read access during that time. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From freeakk@REDACTED Fri Oct 14 19:49:15 2011 From: freeakk@REDACTED (Michael Uvarov) Date: Fri, 14 Oct 2011 21:49:15 +0400 Subject: [erlang-questions] ErlNifTid to integer Message-ID: Hello, Is there a standard way to cast ErlNifTid to integer? I have a resource which stores copies of data for each thread. One thread - one copy. Now I search the copy for the current thread in the cycle. It is also possible to do this task using hashing and direct access to the copy. How can I do this in effective manner? -- Best regards, Uvarov Michael From ngreco@REDACTED Fri Oct 14 20:32:02 2011 From: ngreco@REDACTED (Nahuel Greco) Date: Fri, 14 Oct 2011 15:32:02 -0300 Subject: [erlang-questions] servers for mountable remote file systems In-Reply-To: <4967D497-8424-4763-A08E-26B01299BB6A@rogvall.se> References: <4967D497-8424-4763-A08E-26B01299BB6A@rogvall.se> Message-ID: You can also mount a filesystem implemented in erlang using FUSE ( http://code.google.com/p/fuserl/ ) and then use regular Samba to share it to the network. Saludos, Nahuel Greco. On Fri, Oct 14, 2011 at 9:07 AM, Tony Rogvall wrote: > Hi joe! > Luke did hack a cool erlang NFS server 10 years ago :-) > http://jungerl.cvs.sourceforge.net/viewvc/jungerl/jungerl/lib/enfs/ > Nothing new under the sun ! > /Tony > On 14 okt 2011, at 10:30, Joe Armstrong wrote: > > Hello, > > I want to make a remote virtual device that I can mount on mac-os X/linux > etc. > > It seems like there are three alternatives > > ????a) webdav > ????b) samba/smb > ????c) NFS > > I need an Erlang server that implements a virtual file system as a > back-end to the protocol > in question. > > I can imagine emulating a large number of small files - the mounted > file system would appear to have a > large number of small files - but in fact these would not exist but be > emulated in the server. > > Has anybody implemented servers for the above protocols in Erlang? or > do I have to start from > scratch? Is any of these protocols to be preferred over the others, > given that I want to emulate > a large number of small files. Is my list a) - c) complete are there > other protocols I should consider? > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > "Installing applications can lead to corruption over time.?Applications > gradually write over each other's libraries, partial upgrades occur, user > and system errors happen, and minute changes may be unnoticeable and > difficult to fix" > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From thomas.burdick@REDACTED Fri Oct 14 23:53:14 2011 From: thomas.burdick@REDACTED (Tom Burdick) Date: Fri, 14 Oct 2011 16:53:14 -0500 Subject: [erlang-questions] Building a C port Message-ID: I'm looking to wrap libsamplerate and am looking for some advice on how to go about it. I can't use NIF or port driver as libsamplerate is GPL. NIF or a port driver don't seem to be the right solution anyways as libsamplerate may take awhile to do some work and report back. So I'm left thinking either a C port or a C node would be best, I'm leaning towards a C port implementation. The C port example however as it shows running gcc with -I and -L flags pointing to a particular erlang install. I'd like to make my port portable so that it can compile on different machines with possible different versions of erlang without having to always specify where erlang is installed, or even worse where erl_interface is located. Do people use autotools for this? Is there a project template that uses autotools for this? Could I use rebar instead? Thanks! Tom Burdick From max.lapshin@REDACTED Sat Oct 15 00:01:10 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 15 Oct 2011 02:01:10 +0400 Subject: [erlang-questions] Building a C port In-Reply-To: References: Message-ID: What is the problem with GPL? On Sat, Oct 15, 2011 at 1:53 AM, Tom Burdick wrote: > I'm looking to wrap libsamplerate and am looking for some advice on > how to go about it. > > I can't use NIF or port driver as libsamplerate is GPL. NIF or a port > driver don't seem to be the right solution anyways as libsamplerate > may take awhile to do some work and report back. > > So I'm left thinking either a C port or a C node would be best, I'm > leaning towards a C port implementation. > > The C port example however as it shows running gcc with -I and -L > flags pointing to a particular erlang install. I'd like to make my > port portable so that it can compile on different machines with > possible different versions of erlang without having to always specify > where erlang is installed, or even worse where erl_interface is > located. > > Do people use autotools for this? Is there a project template that > uses autotools for this? > > Could I use rebar instead? > > Thanks! > > Tom Burdick > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From icarus.alive@REDACTED Sat Oct 15 02:31:27 2011 From: icarus.alive@REDACTED (Icarus Alive) Date: Sat, 15 Oct 2011 06:01:27 +0530 Subject: [erlang-questions] Building a C port In-Reply-To: References: Message-ID: On Sat, Oct 15, 2011 at 3:31 AM, Max Lapshin wrote: > What is the problem with GPL? > Probably the missing, "linking exception", thus potentially forcing a proprietary project to go open. That's one possibility, I'd say. > On Sat, Oct 15, 2011 at 1:53 AM, Tom Burdick wrote: >> I'm looking to wrap libsamplerate and am looking for some advice on >> how to go about it. >> >> I can't use NIF or port driver as libsamplerate is GPL. NIF or a port >> driver don't seem to be the right solution anyways as libsamplerate >> may take awhile to do some work and report back. >> >> So I'm left thinking either a C port or a C node would be best, I'm >> leaning towards a C port implementation. >> >> The C port example however as it shows running gcc with -I and -L >> flags pointing to a particular erlang install. I'd like to make my >> port portable so that it can compile on different machines with >> possible different versions of erlang without having to always specify >> where erlang is installed, or even worse where erl_interface is >> located. >> >> Do people use autotools for this? Is there a project template that >> uses autotools for this? >> >> Could I use rebar instead? >> >> Thanks! >> >> Tom Burdick >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mjtruog@REDACTED Sat Oct 15 03:05:54 2011 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 14 Oct 2011 18:05:54 -0700 Subject: [erlang-questions] Building a C port In-Reply-To: References: Message-ID: <4E98DC72.6030501@gmail.com> Answers below: On 10/14/2011 02:53 PM, Tom Burdick wrote: > I'm looking to wrap libsamplerate and am looking for some advice on > how to go about it. > > I can't use NIF or port driver as libsamplerate is GPL. NIF or a port > driver don't seem to be the right solution anyways as libsamplerate > may take awhile to do some work and report back. > > So I'm left thinking either a C port or a C node would be best, I'm > leaning towards a C port implementation. > > The C port example however as it shows running gcc with -I and -L > flags pointing to a particular erlang install. I'd like to make my > port portable so that it can compile on different machines with > possible different versions of erlang without having to always specify > where erlang is installed, or even worse where erl_interface is > located. > > Do people use autotools for this? Is there a project template that > uses autotools for this? Some people use autotools for this. I have within CloudI, so there is an example there: https://github.com/okeuday/CloudI/blob/master/src/configure.ac You will notice that I am using "AC_ERLANG_CHECK_LIB([erl_interface])". That configuration is probably more complex than what you want, but it does integrate with rebar in a way where rebar is built during the configuration. That way you don't need a random rebar binary in your source tree (from an unknown Erlang version!). I also have GEPD (https://github.com/okeuday/generic-erlang-port--driver-) which could allow you to switch between port and port_driver based on "whether you had permission to from the software author", or something like that, so you don't violate the GPL. Please keep in mind I am not a lawyer, nor am I giving you legal advice. > Could I use rebar instead? Yes, you might be able to use rebar for doing this, but rebar integration with C/C++ port/port_driver/nif is both minimal and ad-hoc. So, you won't have something flexible like autotools where you can use macros to add extra checking, if you need it. Rebar integration should work for a simple case where you don't case as much about where the various executables are coming from and where the output files are going. Good luck, Michael From mattevans123@REDACTED Sat Oct 15 03:29:04 2011 From: mattevans123@REDACTED (Matthew Evans) Date: Fri, 14 Oct 2011 21:29:04 -0400 Subject: [erlang-questions] New NIF behavior in R15??? Message-ID: Hi Within the next month or two we will be starting a project interfacing Erlang to a C driver. This requires the use of a thread pool to support asynchronous behaviors. Ordinarily I would implement this with a linked in port driver. However, Rickard Green at his 2011 SF Erlang conference talk indicated that new support in the NIF framework might be forthcoming in R15 to support linked in driver style threading behavior in NIFS. I would rather hold off implementing a linked in driver if the new NIF features are around the corner. Can anyone give any hints if this will be the case? Thanks Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From barcojie@REDACTED Sat Oct 15 04:21:19 2011 From: barcojie@REDACTED (Barco You) Date: Sat, 15 Oct 2011 10:21:19 +0800 Subject: [erlang-questions] [Zotonic] How to distribute zotonic across multiple machines? In-Reply-To: References: Message-ID: Hi Marc, I'm very glad to hear of this, looking forward to seeing the announcement of the distributed zotonic in the near future. Best regards, Barco On Oct 15, 2011 12:30 AM, "Marc Worrell" wrote: > A short note. > > We are working on the distributed version of zotonic. > > One of the problems we are tackling is the cache synchronization between > nodes. Objective is to be fault tolerant (read partition tolerant). > > Later this month(s) more information while we define our puzzle pieces. > > - Marc > > Sent from my iPhone > > On 14 okt. 2011, at 17:29, Michael Connors wrote: > > > > On 14 October 2011 17:26, envelopes envelopes wrote: > >> Will the write fail if the master node is down? >> >> > It will, but at least your users will have read access during that time. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Sat Oct 15 04:29:22 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 15 Oct 2011 03:29:22 +0100 Subject: [erlang-questions] Building a C port In-Reply-To: <4E98DC72.6030501@gmail.com> References: <4E98DC72.6030501@gmail.com> Message-ID: > > > > Could I use rebar instead? > > Yes, you might be able to use rebar for doing this, but rebar integration > with C/C++ port/port_driver/nif is both minimal and ad-hoc. So, you won't > have something flexible like autotools where you can use macros to add extra > checking, if you need it. Rebar integration should work for a simple case > where you don't case as much about where the various executables are coming > from and where the output files are going. > > In my experience, rebar support for building ports is neither minimal nor ad-hoc. Rebar fully supports overriding where the inputs, dependencies and outputs should be, as well as allowing you to override CC and LDFLAGS plus half a dozen other variables. I'll be the first to admit that it's got a way to go in terms of Windows support, but on most other platforms, you can configure it not only with custom build parameters but also with OS, arch and type specific flags. I have cross platform rebar builds that do very nicely on several Linux distros, FreeBSD and OS-X with very minimal config. Take a look at the rebar.example.config to get a feel for it. Obviously rebar isn't going to provide everything autotools does, but for a simple port/driver you should be ok. If you need to do minor custom stuff in your builds, you can always write a rebar plugin. It'll load these from source so you don't have to package them up or included them in your production code, and they're pure Erlang so you can do whatever you like. It isn't going to replace autotools, but for simple ports/drivers rebar will often suffice. I hadn't considered compiling rebar as part of my build process before, but it's something I might start thinking about for future projects, so thanks for pointing that out Michael. Cheers -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Sat Oct 15 11:28:00 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 15 Oct 2011 10:28:00 +0100 Subject: [erlang-questions] New NIF behavior in R15??? In-Reply-To: References: Message-ID: I was under the impression that a concept like "Native Processes" was going to be introduced as well, to make the writing of linked-in driver code a lot easier. Is that still on the roadmap as well, and if so why are NIFs getting threading support? On 15 October 2011 02:29, Matthew Evans wrote: > Hi > > Within the next month or two we will be starting a project interfacing > Erlang to a C driver. This requires the use of a thread pool to support > asynchronous behaviors. Ordinarily I would implement this with a linked in > port driver. However, Rickard Green at his 2011 SF Erlang conference talk > indicated that new support in the NIF framework might be forthcoming in R15 > to support linked in driver style threading behavior in NIFS. > > I would rather hold off implementing a linked in driver if the new NIF > features are around the corner. Can anyone give any hints if this will be > the case? > > Thanks > > Matt > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rapsey@REDACTED Sat Oct 15 12:44:43 2011 From: rapsey@REDACTED (Rapsey) Date: Sat, 15 Oct 2011 12:44:43 +0200 Subject: [erlang-questions] New NIF behavior in R15??? In-Reply-To: References: Message-ID: NIFs already have threading support. Sergej On Sat, Oct 15, 2011 at 11:28 AM, Tim Watson wrote: > I was under the impression that a concept like "Native Processes" was going > to be introduced as well, to make the writing of linked-in driver code a lot > easier. Is that still on the roadmap as well, and if so why are NIFs getting > threading support? > > On 15 October 2011 02:29, Matthew Evans wrote: > >> Hi >> >> Within the next month or two we will be starting a project interfacing >> Erlang to a C driver. This requires the use of a thread pool to support >> asynchronous behaviors. Ordinarily I would implement this with a linked in >> port driver. However, Rickard Green at his 2011 SF Erlang conference talk >> indicated that new support in the NIF framework might be forthcoming in R15 >> to support linked in driver style threading behavior in NIFS. >> >> I would rather hold off implementing a linked in driver if the new NIF >> features are around the corner. Can anyone give any hints if this will be >> the case? >> >> Thanks >> >> Matt >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Sat Oct 15 13:27:10 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 15 Oct 2011 12:27:10 +0100 Subject: [erlang-questions] New NIF behavior in R15??? In-Reply-To: References: Message-ID: My mistake - I haven't used NIFs yet, as most of the native code I've had to integrate has required threading and I was under the impression that linked-in drivers are the way to go if you want to avoid blocking the emulator whilst potentially long running tasks execute. On 15 October 2011 11:44, Rapsey wrote: > NIFs already have threading support. > > > Sergej > > > On Sat, Oct 15, 2011 at 11:28 AM, Tim Watson wrote: > >> I was under the impression that a concept like "Native Processes" was >> going to be introduced as well, to make the writing of linked-in driver code >> a lot easier. Is that still on the roadmap as well, and if so why are NIFs >> getting threading support? >> >> On 15 October 2011 02:29, Matthew Evans wrote: >> >>> Hi >>> >>> Within the next month or two we will be starting a project interfacing >>> Erlang to a C driver. This requires the use of a thread pool to support >>> asynchronous behaviors. Ordinarily I would implement this with a linked in >>> port driver. However, Rickard Green at his 2011 SF Erlang conference talk >>> indicated that new support in the NIF framework might be forthcoming in R15 >>> to support linked in driver style threading behavior in NIFS. >>> >>> I would rather hold off implementing a linked in driver if the new NIF >>> features are around the corner. Can anyone give any hints if this will be >>> the case? >>> >>> Thanks >>> >>> Matt >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Sat Oct 15 13:59:11 2011 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 15 Oct 2011 12:59:11 +0100 Subject: [erlang-questions] New NIF behavior in R15??? In-Reply-To: References: Message-ID: <4F1698B0-172B-4A22-B536-E388C074F101@gmail.com> On Oct 15, 2011, at 12:27 PM, Tim Watson wrote: > I was under the impression that linked-in drivers are the way to go if you want to avoid blocking the emulator whilst potentially long running tasks execute. What is preventing you from using a queue to talk to your NIF? You can then spawn as many threads as you like in your C code to avoid blocking the emulator. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From watson.timothy@REDACTED Sat Oct 15 14:10:26 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 15 Oct 2011 13:10:26 +0100 Subject: [erlang-questions] New NIF behavior in R15??? In-Reply-To: <4F1698B0-172B-4A22-B536-E388C074F101@gmail.com> References: <4F1698B0-172B-4A22-B536-E388C074F101@gmail.com> Message-ID: Sure I get that it's possible but as I said, I've not used NIFs yet and from reading the documentation you'd expect that the *expected* thing to do with a NIFs is keep it bound to the calling process and use it for fast synchronous calls, whereas linked-in drivers provide an explicit API for executing tasks asynchronously. On 15 October 2011 12:59, Joel Reymont wrote: > > On Oct 15, 2011, at 12:27 PM, Tim Watson wrote: > > > I was under the impression that linked-in drivers are the way to go if > you want to avoid blocking the emulator whilst potentially long running > tasks execute. > > What is preventing you from using a queue to talk to your NIF? > > You can then spawn as many threads as you like in your C code to avoid > blocking the emulator. > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mattevans123@REDACTED Sat Oct 15 15:45:04 2011 From: mattevans123@REDACTED (Matthew Evans) Date: Sat, 15 Oct 2011 09:45:04 -0400 Subject: [erlang-questions] New NIF behavior in R15??? In-Reply-To: References: , , , , <4F1698B0-172B-4A22-B536-E388C074F101@gmail.com>, Message-ID: >From what I understand the new NIF features are intended to supercede the use of linked in drivers. NIFS are fast to call C code, much faster than linked in drivers, but as you know they can block the scheduler if your C code does something nasty like read from a driver or sit in a loop. I can, of course, use a NIF as an accessor to a C API that can dispatch a message to a thread in a thread pool (as you would using a linked in driver, in fact I believe the NIF threading functions are wrappers around the exisiting linked in driver functions). What the new NIFS provide is the ability to "convert" an Erlang process into native code. Effectively you will invoke native code that will enter into its own event loop on one of the threads in the thread pool and other Erlang processes can send/receive messages to it. Basically providing an "OTP" like interface to your C code. Please see: http://www.erlang-factory.com/upload/presentations/377/RickardGreen-NativeInterface.pdf Matt Date: Sat, 15 Oct 2011 13:10:26 +0100 From: watson.timothy@REDACTED To: joelr1@REDACTED CC: erlang-questions@REDACTED Subject: Re: [erlang-questions] New NIF behavior in R15??? Sure I get that it's possible but as I said, I've not used NIFs yet and from reading the documentation you'd expect that the *expected* thing to do with a NIFs is keep it bound to the calling process and use it for fast synchronous calls, whereas linked-in drivers provide an explicit API for executing tasks asynchronously. On 15 October 2011 12:59, Joel Reymont wrote: On Oct 15, 2011, at 12:27 PM, Tim Watson wrote: > I was under the impression that linked-in drivers are the way to go if you want to avoid blocking the emulator whilst potentially long running tasks execute. What is preventing you from using a queue to talk to your NIF? You can then spawn as many threads as you like in your C code to avoid blocking the emulator. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Sat Oct 15 16:16:10 2011 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 15 Oct 2011 15:16:10 +0100 Subject: [erlang-questions] return type of lists:foreach Message-ID: What is the return type of lists:foreach? I cannot find void() in the type spec reference manual and no_return() is reserved for throwing exceptions. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Sat Oct 15 16:24:56 2011 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 15 Oct 2011 15:24:56 +0100 Subject: [erlang-questions] return type of lists:foreach In-Reply-To: References: Message-ID: <74170BC1-507D-45DA-B311-86D6C7DEBE30@gmail.com> none() is a proper return type. Apologies for the noise! On Oct 15, 2011, at 3:16 PM, Joel Reymont wrote: > What is the return type of lists:foreach? > > I cannot find void() in the type spec reference manual and no_return() is reserved for throwing exceptions. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From kostis@REDACTED Sat Oct 15 16:28:20 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Sat, 15 Oct 2011 17:28:20 +0300 Subject: [erlang-questions] return type of lists:foreach In-Reply-To: References: Message-ID: <4E999884.3070405@cs.ntua.gr> On 10/15/11 17:16, Joel Reymont wrote: > What is the return type of lists:foreach? > > I cannot find void() in the type spec reference manual and no_return() is reserved for throwing exceptions. The return type of lists:foreach/2 is 'ok'. If the documentation says void() it should be fixed. There is no void(). Kostis PS. no_return() or none() have nothing to do with lists:foreach/2 type. From joelr1@REDACTED Sat Oct 15 16:33:42 2011 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 15 Oct 2011 15:33:42 +0100 Subject: [erlang-questions] return type of lists:foreach In-Reply-To: <4E999884.3070405@cs.ntua.gr> References: <4E999884.3070405@cs.ntua.gr> Message-ID: <2B1E057C-5FF8-44A0-B645-9DD3352D814B@gmail.com> On Oct 15, 2011, at 3:28 PM, Kostis Sagonas wrote: > PS. no_return() or none() have nothing to do with lists:foreach/2 type. I beg to differ! typer x.erl %% File: "x.erl" %% ------------- -spec x() -> none(). cat x.erl -module(x). -compile([export_all]). x() -> lists:foreach(fun() -> ok end, []). -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Sat Oct 15 16:39:20 2011 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 15 Oct 2011 15:39:20 +0100 Subject: [erlang-questions] return type of lists:foreach In-Reply-To: <4E999884.3070405@cs.ntua.gr> References: <4E999884.3070405@cs.ntua.gr> Message-ID: <4FE6F3AF-0BC2-4E91-A1C7-BDE9319FE39C@gmail.com> On Oct 15, 2011, at 3:28 PM, Kostis Sagonas wrote: > The return type of lists:foreach/2 is 'ok'. If the documentation says void() it should be fixed. There is no void(). You are absolutely right. none() is reserved for when the anonymous function you pass to lists:foreach/2 does not take any arguments ;-). Mea maxima culpa! -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Sat Oct 15 17:43:22 2011 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 15 Oct 2011 16:43:22 +0100 Subject: [erlang-questions] typing anonymous functions Message-ID: <9D6B94B1-5FB8-4E61-8C31-C30BBF11AE98@gmail.com> Why is this a bad type declaration? -type anon :: fun((iolist(),term()) -> iolist()). After all, it works just fine as part of the bar spec! -spec bar(fun((iolist(),term()) -> iolist()), iolist(), term()) -> any(). Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Sat Oct 15 17:45:53 2011 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 15 Oct 2011 16:45:53 +0100 Subject: [erlang-questions] typing anonymous functions In-Reply-To: <9D6B94B1-5FB8-4E61-8C31-C30BBF11AE98@gmail.com> References: <9D6B94B1-5FB8-4E61-8C31-C30BBF11AE98@gmail.com> Message-ID: This fails too. Why? -type pickler :: {fun((iolist(), term()) -> iolist()), fun((binary()) -> {term(), binary()})}. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Sat Oct 15 17:50:22 2011 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 15 Oct 2011 16:50:22 +0100 Subject: [erlang-questions] typing anonymous functions In-Reply-To: References: <9D6B94B1-5FB8-4E61-8C31-C30BBF11AE98@gmail.com> Message-ID: <8DD69364-0443-4DB4-AC25-78480E1D4603@gmail.com> Note the absence of () after the type name. Foot in mouth! On Oct 15, 2011, at 4:45 PM, Joel Reymont wrote: > This fails too. Why? > > -type pickler :: {fun((iolist(), term()) -> iolist()), fun((binary()) -> {term(), binary()})}. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Sat Oct 15 18:15:33 2011 From: joelr1@REDACTED (Joel Reymont) Date: Sat, 15 Oct 2011 17:15:33 +0100 Subject: [erlang-questions] New NIF behavior in R15??? In-Reply-To: References: <4F1698B0-172B-4A22-B536-E388C074F101@gmail.com> Message-ID: <3E87084A-9F7B-42B4-8DF0-CE9E543D3950@gmail.com> On Oct 15, 2011, at 5:05 PM, Max Lapshin wrote: > As I've found, there are affinity issues. All spawned threads are tighed to one core. Wouldn't this be an OS issue? After all, the threads you spawn from your NIF, a C function, are regular C threads. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From watson.timothy@REDACTED Sat Oct 15 20:37:44 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 15 Oct 2011 19:37:44 +0100 Subject: [erlang-questions] New NIF behavior in R15??? In-Reply-To: References: <4F1698B0-172B-4A22-B536-E388C074F101@gmail.com> Message-ID: On 15 October 2011 14:45, Matthew Evans wrote: > From what I understand the new NIF features are intended to supercede the > use of linked in drivers. > > Right, I apologize for the confusion. This is exactly what I was talking about - on slide deck he calls them "Native Processes" - I had forgotten that the enhancements were going to sit along side the existing NIF APIs as it's been quite some time since I saw the presentation online. -------------- next part -------------- An HTML attachment was scrubbed... URL: From laforge@REDACTED Sat Oct 15 22:45:42 2011 From: laforge@REDACTED (Harald Welte) Date: Sat, 15 Oct 2011 22:45:42 +0200 Subject: [erlang-questions] asn1ct/asn1rt EXTERNAL 1990/1994 conversion information loss In-Reply-To: References: <20110417175454.GI9379@prithivi.gnumonks.org> Message-ID: <20111015204542.GE28063@prithivi.gnumonks.org> Hi Kenneth, I was recently looking into current otp.git, and it seems like this bug has still not been fixed one way or the other. I would appreciate if you could look into a way of permanently resolving this issue. I'm happy to work on alternative solutions, if you think the current proposals (including your one-liner change) are note OK to be applied to mainline otp.git. Regards, Harald On Wed, Apr 27, 2011 at 01:52:50PM +0200, Kenneth Lundin wrote: > Hi Harald, > > For good or bad we have once taken the decision to not introduce > compiler options for > chosing which ASN.1 standard 1990, 1994, 1997 etc. to use. Instead we > try to support all > reasonable constructs from these standards at the same time. > > When it comes to the EXTERNAL type we allow input values both as a > 5-tuple (1990) > and as a 4-tuple (1994) and use the 5 tuple 1990 format for encoding > as the standard recommends. > When decoding an EXTERNAL type this ends up as a 5-tuple (1990) as > you describe and > we convert it "back" to a 4-tuple (1994) representation since > otherwise a value using the 1994 4-tuple > format would not yield the same value back when encoded and then > decoded back again. > > The problem with this approach is as you also explain that some > information possible present in the > 1990 5 -tuple vesion get lost when converted to the 1994 4 -tuple format. > > I suggest an easy first step which might be sufficient since the > EXTERNAL type in not recommended to be used at all > any more is to continue with the same conversion upon decode but only > convert to 1994 4-tuple format when this can be done > without information loss i.e. only when the "encoding" component is > {octet-aligned, Octets}. > This will result in that a 4-tuple (1994) or a 5-tuple (1990) value > can be returned from decode and that encode will accept > both of these formats as well. > > This will only require a one-liner change in asn1rt_check.erl (plus > the documentation of course) > You can try it if you want. > > Just change : in function transform_to_EXTERNAL1994/1 > > case Encoding of > {_,Val} -> > > to > case Encoding of > {'octet-aligned',Val} when is_list(Val);is_binary(Val) -> > > > /Regards Kenneth, Erlang/OTP, Ericsson > > On Sun, Apr 17, 2011 at 7:54 PM, Harald Welte wrote: > > Hi! > > > > I'm working on a variety of (Free Software) projects related to mobile > > communications (http://cgit.osmocom.org/). One of them is dealing with the GSM > > MAP protocol (TS 09.02, TS 29.002) which is specified in ASN.1 > > > > I'm using the Erlang asn1ct/asn1rt to parse incoming MAP messages from the > > worldwide SS7 backbone, filtering + patching some of the data contained in > > it and re-transmit the MAP messages. It's like an application-level-gateway / > > proxy that can modify some of the data. > > > > MAP itself uses TCAP (Q.773). TCAP messages have a dialogue and a component > > portion. The dialogue portion is specified as an EXTERNAL type: > > > > DialoguePortion ::= [APPLICATION 11] EXPLICIT EXTERNAL > > > > Note: The MAP sender are (normally?) using the 1990 representation of EXTERNAL, > > not the 1994 version. > > > > Erlangs asn1ct-generated dec_EXTERNAL code first internally generates along those > > lines: > > > > #'EXTERNAL'{ > > 'direct-reference' = {0,0,17,773,1,1,1}, > > 'indirect-reference' = asn1_NOVALUE, > > 'data-value-descriptor' = asn1_NOVALUE, > > encoding = > > {'single-ASN1-type', > > [96,15,128,2,7,128,161,9,6,7,4,0,0,1,0,1,3]}} > > > > However, the asn1ct generated code automatically converts this #'EXTERNAL'{} > > record into a tuple of the following structure (using > > transform_to_EXTERNAL1994/1): > > > > {'EXTERNAL', {syntax,{0,0,17,773,1,1,1}}, > > asn1_NOVALUE, > > [96,15,128,2,7,128,161,9,6,7,4,0,0,1,0,1,3]} > > > > The problem is, when you pass this tuple into the asn1ct-generated encoder, > > the 'single-ASN1-type' information has been lost, and it will be replaced with > > 'octet-aligned'. > > > > The result is that instead of {tag,'CONTEXT',0,'EXPLICIT',32} we now use > > {tag,'CONTEXT',1,'IMPLICIT',0} for encoding the data. This is not valid within > > the scope of MAP, and both the wireshark MAP dissector as well as all MAP > > implementations I've seen reject or ignore the resulting message. > > > > The problem fundamentally can be summarized as: > > * EXTERNAL can contain inner data in either 'Single-ASN1-type' or > > 'octet-aligned' > > * the OTP asn1ct (at least in ber_bin mode) looses this information and > > does not pass it back to the caller when decoding > > * if the resulting data is passed through the encoder, the encoded data > > semantically differs from the original undecoded BER data. > > > > I think this is probably not the intended behaviour. It would be weird if > > decoding and re-encoding the same unmodified data leads to different BER tags > > in the tree. > > > > What is wrong in passing the 1990 or 1994 EXTERNAL types from the decoder into > > the user program? Why should the user not know which of the two formats was > > being used? I believe at the very least, this automatic conversion should be > > something that can be configured as a compiler option. > > > > Any comments/ideas/suggestions? > > > > Regards, > > Harald > > -- > > - Harald Welte http://laforge.gnumonks.org/ > > ============================================================================ > > "Privacy in residential applications is a desirable marketing option." > > (ETSI EN 300 175-7 Ch. A6) > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From joelr1@REDACTED Sun Oct 16 17:17:45 2011 From: joelr1@REDACTED (Joel Reymont) Date: Sun, 16 Oct 2011 16:17:45 +0100 Subject: [erlang-questions] unhandled fpe on the mac Message-ID: I'm getting these a lot, both in R14B03 and R14B04: ERTS_FP_CHECK_INIT at 0x1b2e815b: detected unhandled FPE at 0x1 Any clues? -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From andrew.pennebaker@REDACTED Sun Oct 16 20:02:04 2011 From: andrew.pennebaker@REDACTED (Andrew Pennebaker) Date: Sun, 16 Oct 2011 14:02:04 -0400 Subject: [erlang-questions] erl vs escript Message-ID: Could erl please consider shebangs as comments, so that escript code can be compiled with erl? Cheers, Andrew Pennebaker www.yellosoft.us -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Sun Oct 16 21:54:04 2011 From: bob@REDACTED (Bob Cowdery) Date: Sun, 16 Oct 2011 20:54:04 +0100 Subject: [erlang-questions] Managing children Message-ID: <4E9B365C.6090905@bobcowdery.plus.com> Hi I have an OTP system with one supervisor that starts 7 processes. When all processes are running I need to do some initialisation by sending event messages from an ets database. What would be the appropriate way to achieve this. I don't see anything specifically in supervisor that helps. 1. I could make the last process to start make the call, but that's a fudge. 2. I could send a message from every process at the end of its init/1 to a monitor process and get it to make the call when everything has started, but that seems like duplicating supervisor function. 3. I could run a process inside my supervisor module to either do what (2) does or maybe call count_children/1 until I see all processes active. Is there some reason why supervisor does not have callbacks for managing children. Regards Bob From mihai@REDACTED Sun Oct 16 22:15:35 2011 From: mihai@REDACTED (Mihai Balea) Date: Sun, 16 Oct 2011 16:15:35 -0400 Subject: [erlang-questions] Managing children In-Reply-To: <4E9B365C.6090905@bobcowdery.plus.com> References: <4E9B365C.6090905@bobcowdery.plus.com> Message-ID: <792E187C-D0B2-46F7-AE9A-BDD7A32D4386@hates.ms> On Oct 16, 2011, at 3:54 PM, Bob Cowdery wrote: > Hi > > I have an OTP system with one supervisor that starts 7 processes. When > all processes are running I need to do some initialisation by sending > event messages from an ets database. > > What would be the appropriate way to achieve this. I don't see anything > specifically in supervisor that helps. > > 1. I could make the last process to start make the call, but that's a fudge. > 2. I could send a message from every process at the end of its init/1 to > a monitor process and get it to make the call when everything has > started, but that seems like duplicating supervisor function. > 3. I could run a process inside my supervisor module to either do what > (2) does or maybe call count_children/1 until I see all processes active. > > Is there some reason why supervisor does not have callbacks for managing > children. The supervisor's start_link function will only return {ok, pid()} when all children are up and running. You can start sending your event messages after that. Mihai From tristan.sloughter@REDACTED Sun Oct 16 23:29:01 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Sun, 16 Oct 2011 16:29:01 -0500 Subject: [erlang-questions] Mnesia: Error on insert object with same key in Set Message-ID: I wanted to make sure there was still no way to tell Mnesia to return an error if it is given a record to write that has the same key as an object already stored in the table. Reading through the docs it looks like there is no way through configuration but you must simply read and then write if its not there, like I found when this was asked in 2004 on this mailing list: insert(K,V) -> fun() -> case mnesia:wread(K) of [Exist] -> {abort, Exist}; _ -> mnesia:write(V) end end. Which doesn't seem safe, as Ulf points on in that thread: http://erlang.org/pipermail/erlang-questions/2004-May/012307.html As well as slow. So I was hoping maybe in 2011 there is a better way to do this? :) Thanks, Tristan -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Mon Oct 17 08:40:51 2011 From: dangud@REDACTED (Dan Gudmundsson) Date: Mon, 17 Oct 2011 08:40:51 +0200 Subject: [erlang-questions] Mnesia: Error on insert object with same key in Set In-Reply-To: References: Message-ID: This should be fixed now and thus safe to use now. /Dan On Sun, Oct 16, 2011 at 11:29 PM, Tristan Sloughter wrote: > I wanted to make sure there was still no way to tell Mnesia to return an > error if it is given a record to write that has the same key as an object > already stored in the table. > Reading through the docs it looks like there is no way through configuration > but you must simply read and then write if its not there, like I found when > this was asked in 2004 on this mailing list: > insert(K,V) -> > ? fun() -> > ? case mnesia:wread(K) of > ? ??[Exist] -> {abort, Exist}; > ? ??_ -> mnesia:write(V) > ? end > ? end. > Which doesn't seem safe, as Ulf points on in that > thread:?http://erlang.org/pipermail/erlang-questions/2004-May/012307.html > As well as slow. > So I was hoping maybe in 2011 there is a better way to do this? :) > Thanks, > Tristan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From kostis@REDACTED Mon Oct 17 08:45:14 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 17 Oct 2011 09:45:14 +0300 Subject: [erlang-questions] Mnesia: Error on insert object with same key in Set In-Reply-To: References: Message-ID: <4E9BCEFA.6040806@cs.ntua.gr> On 10/17/11 09:40, Dan Gudmundsson wrote: > This should be fixed now and thus safe to use now. But doesn't the code below have an obvious race condition? Kostis > On Sun, Oct 16, 2011 at 11:29 PM, Tristan Sloughter > wrote: >> I wanted to make sure there was still no way to tell Mnesia to return an >> error if it is given a record to write that has the same key as an object >> already stored in the table. >> Reading through the docs it looks like there is no way through configuration >> but you must simply read and then write if its not there, like I found when >> this was asked in 2004 on this mailing list: >> insert(K,V) -> >> fun() -> >> case mnesia:wread(K) of >> [Exist] -> {abort, Exist}; >> _ -> mnesia:write(V) >> end >> end. >> Which doesn't seem safe, as Ulf points on in that >> thread: http://erlang.org/pipermail/erlang-questions/2004-May/012307.html >> As well as slow. >> So I was hoping maybe in 2011 there is a better way to do this? :) >> Thanks, >> Tristan >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ward@REDACTED Mon Oct 17 08:59:08 2011 From: ward@REDACTED (Ward Bekker) Date: Mon, 17 Oct 2011 08:59:08 +0200 Subject: [erlang-questions] records and types Message-ID: <20111017085908.eaf37ff0@groupware.tty.nl> I have the following (compiling) record and type definitions: -record( field, {name :: unicode:unicode_binary()}). -record( schema,{name :: unicode:unicode_binary(), fields :: [ #field{} ]}). -type text_field() :: #field{}. -type integer_field() :: #field{}. The following function will give compile errors: demo_schema() -> #schema{ name = u("stackoverflow_post"), fields = [ #text_field{ name = u("title") }, #integer_field{ name = u("user_id") } ] }. Is it possible to create a field record using the type definition name? Or do I need to define a record for every field type? Regards, Ward From ulf.wiger@REDACTED Mon Oct 17 09:00:56 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 17 Oct 2011 09:00:56 +0200 Subject: [erlang-questions] Mnesia: Error on insert object with same key in Set In-Reply-To: <4E9BCEFA.6040806@cs.ntua.gr> References: <4E9BCEFA.6040806@cs.ntua.gr> Message-ID: <9EF26B78-FEC4-4C52-9645-CFCA0E6AEFBD@erlang-solutions.com> On 17 Oct 2011, at 08:45, Kostis Sagonas wrote: > On 10/17/11 09:40, Dan Gudmundsson wrote: >> This should be fixed now and thus safe to use now. > > But doesn't the code below have an obvious race condition? > > Kostis Kostis, Those functions in the mnesia API only work when called from within a transaction, so the race condition is managed through implicit locking. BR, Ulf > >> On Sun, Oct 16, 2011 at 11:29 PM, Tristan Sloughter >> wrote: >>> I wanted to make sure there was still no way to tell Mnesia to return an >>> error if it is given a record to write that has the same key as an object >>> already stored in the table. >>> Reading through the docs it looks like there is no way through configuration >>> but you must simply read and then write if its not there, like I found when >>> this was asked in 2004 on this mailing list: >>> insert(K,V) -> >>> fun() -> >>> case mnesia:wread(K) of >>> [Exist] -> {abort, Exist}; >>> _ -> mnesia:write(V) >>> end >>> end. >>> Which doesn't seem safe, as Ulf points on in that >>> thread: http://erlang.org/pipermail/erlang-questions/2004-May/012307.html >>> As well as slow. >>> So I was hoping maybe in 2011 there is a better way to do this? :) >>> Thanks, >>> Tristan >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From torben.lehoff@REDACTED Mon Oct 17 09:06:25 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Mon, 17 Oct 2011 09:06:25 +0200 Subject: [erlang-questions] records and types In-Reply-To: <20111017085908.eaf37ff0@groupware.tty.nl> References: <20111017085908.eaf37ff0@groupware.tty.nl> Message-ID: <4E9BD3F1.3090905@gmail.com> I am afraid you need to create a record for every field type. Then you can write something like: -type field() :: #text_field{} | #integer_filed{}. -record(schema, {fields :: [field()]}). And then you might want to look at Ulf Wiger's exprecs library - it may help with this kind of repetitive code. Cheers, Torben On Mon Oct 17 08:59:08 2011, Ward Bekker wrote: > I have the following (compiling) record and type definitions: > > -record( field, {name :: unicode:unicode_binary()}). > -record( schema,{name :: unicode:unicode_binary(), fields :: [ #field{} ]}). > -type text_field() :: #field{}. > -type integer_field() :: #field{}. > > The following function will give compile errors: > > demo_schema() -> > #schema{ name = u("stackoverflow_post"), > fields = [ > #text_field{ name = u("title") }, > #integer_field{ name = u("user_id") } > ] > }. > > Is it possible to create a field record using the type definition name? Or do I need to define a record for every field type? > > Regards, > > Ward > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bob@REDACTED Mon Oct 17 10:19:10 2011 From: bob@REDACTED (Bob Cowdery) Date: Mon, 17 Oct 2011 09:19:10 +0100 Subject: [erlang-questions] Managing children In-Reply-To: <792E187C-D0B2-46F7-AE9A-BDD7A32D4386@hates.ms> References: <4E9B365C.6090905@bobcowdery.plus.com> <792E187C-D0B2-46F7-AE9A-BDD7A32D4386@hates.ms> Message-ID: <4E9BE4FE.5090800@bobcowdery.plus.com> On 16/10/2011 21:15, Mihai Balea wrote: > On Oct 16, 2011, at 3:54 PM, Bob Cowdery wrote: > >> Hi >> >> I have an OTP system with one supervisor that starts 7 processes. When >> all processes are running I need to do some initialisation by sending >> event messages from an ets database. >> >> What would be the appropriate way to achieve this. I don't see anything >> specifically in supervisor that helps. >> >> 1. I could make the last process to start make the call, but that's a fudge. >> 2. I could send a message from every process at the end of its init/1 to >> a monitor process and get it to make the call when everything has >> started, but that seems like duplicating supervisor function. >> 3. I could run a process inside my supervisor module to either do what >> (2) does or maybe call count_children/1 until I see all processes active. >> >> Is there some reason why supervisor does not have callbacks for managing >> children. > The supervisor's start_link function will only return {ok, pid()} when all children are up and running. > You can start sending your event messages after that. Ok, thanks. That did the trick. Bob > Mihai > From joelr1@REDACTED Mon Oct 17 10:22:24 2011 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 17 Oct 2011 09:22:24 +0100 Subject: [erlang-questions] cocoa bridge Message-ID: Cocoa requires the event loop to run in the firs app thread (thread #0). Is there a way to force the Erlang VM to start a new thread for itself to free thread #0 for Cocoa? Am I stuck with hacking the VM to do so? Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From mattevans123@REDACTED Mon Oct 17 13:43:48 2011 From: mattevans123@REDACTED (Matthew Evans) Date: Mon, 17 Oct 2011 07:43:48 -0400 Subject: [erlang-questions] New NIF behavior in R15??? In-Reply-To: References: , , , , <4F1698B0-172B-4A22-B536-E388C074F101@gmail.com>, , , Message-ID: Right, In my case I have C code (drivers and an api to some hardware), Java code (some app logic and an ipc to a larger java app) and some Erlang doing ipc and running a pattern matching app. What would be nice with the new NIFS is to build the C code as a collection of NIF processes, use Jinterface for Java; effectively giving the ability to split processing into "applets" all communicating via erlang ipc. The java apps could talk to the C code via an erlang message directly, we could then move work from java to erlang or visa versa. I can do this now using NIFS or linked-in drivers and an erlang middleman process, but the ability to do it directly is compelling (I.e. Java to a C NIF directly using erlang ipc as opposed to Java to Erlang process to C). Matt Date: Sat, 15 Oct 2011 19:37:44 +0100 Subject: Re: [erlang-questions] New NIF behavior in R15??? From: watson.timothy@REDACTED To: mattevans123@REDACTED CC: joelr1@REDACTED; erlang-questions@REDACTED On 15 October 2011 14:45, Matthew Evans wrote: >From what I understand the new NIF features are intended to supercede the use of linked in drivers. Right, I apologize for the confusion. This is exactly what I was talking about - on slide deck he calls them "Native Processes" - I had forgotten that the enhancements were going to sit along side the existing NIF APIs as it's been quite some time since I saw the presentation online. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kkcmpathi@REDACTED Mon Oct 17 14:24:41 2011 From: kkcmpathi@REDACTED (chamin madura) Date: Mon, 17 Oct 2011 17:54:41 +0530 Subject: [erlang-questions] how send short message using OSERL? Message-ID: Hi all, I have already successfully connect the smpp server in trx mode(using oserl/doc/example/sample_esme.erl ) ,then try send the short message to smpp server or mobile phone, using, sample_esme:send(Src,Dest,Msg,Opt). I used data eg-: Src={192,168,13,4} Dest={Smpp server} | Mobile No Msg="hello" Opt=[ ] reply is "ok", but didn't send the data to smpp server,What is the reason for it? How to fix it? pls,any one can help to me? Regards, Chamin. -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Mon Oct 17 15:06:51 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 17 Oct 2011 14:06:51 +0100 Subject: [erlang-questions] New NIF behavior in R15??? In-Reply-To: References: <4F1698B0-172B-4A22-B536-E388C074F101@gmail.com> Message-ID: Sounds rather involved, but an interesting example of Erlang providing the infrastructure glue to various technologies - something that it seems to be very good at. I've got a couple of linked in drivers I'd like to migrate, one of which provides an interface to various XSLT processors (Xalan-C, libxslt, Sablotron, etc) and the other which I recently started, providing a binding to Oracle via the OCI call interface. It seems to me that both would benefit from the stability improvements that should come from Native Processes. There's one thing I am a little unclear on, and perhaps it's just a case of RTFM on NIFs. For linked-in drivers, you've got the outputv callback to get a direct handle to data (in an ErlIOVec) without incurring copying - I guess that NIFs already minimise the copying which is why they're fast? Also I make heavy use of the 'driver_send_term' API call, which on the SMP runtime is safe to use from an arbitrary thread. I'm hoping that this latter facility (being able to send a message to an arbitrary pid from an arbitrary worker thread) will be preserved in the new API. On 17 October 2011 12:43, Matthew Evans wrote: > Right, > > In my case I have C code (drivers and an api to some hardware), Java code > (some app logic and an ipc to a larger java app) and some Erlang doing ipc > and running a pattern matching app. > > What would be nice with the new NIFS is to build the C code as a collection > of NIF processes, use Jinterface for Java; effectively giving the ability to > split processing into "applets" all communicating via erlang ipc. The java > apps could talk to the C code via an erlang message directly, we could then > move work from java to erlang or visa versa. > > I can do this now using NIFS or linked-in drivers and an erlang middleman > process, but the ability to do it directly is compelling (I.e. Java to a C > NIF directly using erlang ipc as opposed to Java to Erlang process to C). > > Matt > > ------------------------------ > Date: Sat, 15 Oct 2011 19:37:44 +0100 > > Subject: Re: [erlang-questions] New NIF behavior in R15??? > From: watson.timothy@REDACTED > To: mattevans123@REDACTED > CC: joelr1@REDACTED; erlang-questions@REDACTED > > > On 15 October 2011 14:45, Matthew Evans wrote: > > From what I understand the new NIF features are intended to supercede the > use of linked in drivers. > > > Right, I apologize for the confusion. This is exactly what I was talking > about - on slide deck he calls them "Native Processes" - I had forgotten > that the enhancements were going to sit along side the existing NIF APIs as > it's been quite some time since I saw the presentation online. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave@REDACTED Mon Oct 17 15:18:12 2011 From: dave@REDACTED (David Goehrig) Date: Mon, 17 Oct 2011 09:18:12 -0400 Subject: [erlang-questions] cocoa bridge In-Reply-To: References: Message-ID: Is there any reason you can't swap order, have the Cocoa app spawn erlang in a new thread and treat the original process as a port? I ask because this is what I ended up doing for SDLPerl, where both Cocoa and Perl were fighting over the mainthread. -=-=- dave@REDACTED -=-=- On Oct 17, 2011, at 4:22 AM, Joel Reymont wrote: > Cocoa requires the event loop to run in the firs app thread (thread #0). > > Is there a way to force the Erlang VM to start a new thread for itself to free thread #0 for Cocoa? > > Am I stuck with hacking the VM to do so? > > Thanks, Joel > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From joelr1@REDACTED Mon Oct 17 15:57:43 2011 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 17 Oct 2011 14:57:43 +0100 Subject: [erlang-questions] cocoa bridge In-Reply-To: References: Message-ID: <9FF3B217-5C9B-4F40-B49E-96D198211526@gmail.com> On Oct 17, 2011, at 2:18 PM, David Goehrig wrote: > Is there any reason you can't swap order, have the Cocoa app spawn erlang in a new thread and treat the original process as a port? I was hoping to avoid marshaling function calls over TCP. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From sverker@REDACTED Mon Oct 17 16:32:39 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Mon, 17 Oct 2011 16:32:39 +0200 Subject: [erlang-questions] unhandled fpe on the mac In-Reply-To: References: Message-ID: <4E9C3C87.7010308@erix.ericsson.se> Joel Reymont wrote: > I'm getting these a lot, both in R14B03 and R14B04: > > ERTS_FP_CHECK_INIT at 0x1b2e815b: detected unhandled FPE at 0x1 > > Any clues This indicates that there is something wrong with the handling of floating point exceptions (FPE). This is a known issue on mac. The use of FPE's can be disabled with configure option --disable-fp-exceptions. It will make floating point calculations somewhat slower. /Sverker, Erlang/OTP Ericsson From joelr1@REDACTED Mon Oct 17 16:36:23 2011 From: joelr1@REDACTED (Joel Reymont) Date: Mon, 17 Oct 2011 15:36:23 +0100 Subject: [erlang-questions] unhandled fpe on the mac In-Reply-To: <4E9C3C87.7010308@erix.ericsson.se> References: <4E9C3C87.7010308@erix.ericsson.se> Message-ID: On Oct 17, 2011, at 3:32 PM, Sverker Eriksson wrote: > This indicates that there is something wrong with the handling of floating point exceptions (FPE). > This is a known issue on mac. Is it difficult to fix? Are there any repro cases? I may tackle this myself if you tell me where to look. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From tristan.sloughter@REDACTED Mon Oct 17 16:46:03 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Mon, 17 Oct 2011 09:46:03 -0500 Subject: [erlang-questions] Mnesia: Error on insert object with same key in Set In-Reply-To: <9EF26B78-FEC4-4C52-9645-CFCA0E6AEFBD@erlang-solutions.com> References: <4E9BCEFA.6040806@cs.ntua.gr> <9EF26B78-FEC4-4C52-9645-CFCA0E6AEFBD@erlang-solutions.com> Message-ID: Oh great, thanks! On Mon, Oct 17, 2011 at 2:00 AM, Ulf Wiger wrote: > > On 17 Oct 2011, at 08:45, Kostis Sagonas wrote: > > > On 10/17/11 09:40, Dan Gudmundsson wrote: > >> This should be fixed now and thus safe to use now. > > > > But doesn't the code below have an obvious race condition? > > > > Kostis > > Kostis, > > Those functions in the mnesia API only work when called from within a > transaction, so the race condition is managed through implicit locking. > > BR, > Ulf > > > > > >> On Sun, Oct 16, 2011 at 11:29 PM, Tristan Sloughter > >> wrote: > >>> I wanted to make sure there was still no way to tell Mnesia to return > an > >>> error if it is given a record to write that has the same key as an > object > >>> already stored in the table. > >>> Reading through the docs it looks like there is no way through > configuration > >>> but you must simply read and then write if its not there, like I found > when > >>> this was asked in 2004 on this mailing list: > >>> insert(K,V) -> > >>> fun() -> > >>> case mnesia:wread(K) of > >>> [Exist] -> {abort, Exist}; > >>> _ -> mnesia:write(V) > >>> end > >>> end. > >>> Which doesn't seem safe, as Ulf points on in that > >>> thread: > http://erlang.org/pipermail/erlang-questions/2004-May/012307.html > >>> As well as slow. > >>> So I was hoping maybe in 2011 there is a better way to do this? :) > >>> Thanks, > >>> Tristan > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >>> > >>> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From iostres@REDACTED Mon Oct 17 17:24:26 2011 From: iostres@REDACTED (Ivan Ostres) Date: Mon, 17 Oct 2011 17:24:26 +0200 Subject: [erlang-questions] unhandled fpe on the mac In-Reply-To: References: <4E9C3C87.7010308@erix.ericsson.se> Message-ID: On 10/17/11 4:36 PM, Joel Reymont wrote: > On Oct 17, 2011, at 3:32 PM, Sverker Eriksson wrote: > >> This indicates that there is something wrong with the handling of floating point exceptions (FPE). >> This is a known issue on mac. > Is it difficult to fix? > > Are there any repro cases? > > I may tackle this myself if you tell me where to look. > > Some months ago someone posted it was about truncating big floats in Erlang that gives this exception. Can't find it at this moment... Ivan From cclamb@REDACTED Mon Oct 17 20:06:30 2011 From: cclamb@REDACTED (Lamb, Christopher Charles) Date: Mon, 17 Oct 2011 18:06:30 +0000 Subject: [erlang-questions] Network Sniffer? Message-ID: Hey everybody, noob question I'm sure, but I'm trying to find some kind of a tutorial on how I can sniff network traffic off my NIC using Erlang. Does such a thing exist? I've seen quite a few network programming examples, but nothing that's grabbing traffic from my NIC. I expect I may need to interface w/ C/C++ to do this? Anyway, Thanks for any guidance! - chris -------------- next part -------------- An HTML attachment was scrubbed... URL: From kris@REDACTED Mon Oct 17 20:08:26 2011 From: kris@REDACTED (Kris Rasmussen) Date: Mon, 17 Oct 2011 11:08:26 -0700 Subject: [erlang-questions] ETS bag performance characteristics Message-ID: Hi, I am currently using the ETS bag table type to store a list of PIDs associated with a given key. I'm concerned that the bag implementation may require a linear traversal to remove (and possibly also add) elements to the bag. Does anyone know how ETS bags are implemented? Ideally, I'd like to have constant lookups by key as well as constant adds/contains/removals. If I were to implement this outside of erlang I would use a Map type data structure. Thanks, Kris -------------- next part -------------- An HTML attachment was scrubbed... URL: From reynaldomic@REDACTED Mon Oct 17 20:11:51 2011 From: reynaldomic@REDACTED (Reynaldo Baquerizo) Date: Mon, 17 Oct 2011 13:11:51 -0500 Subject: [erlang-questions] Network Sniffer? In-Reply-To: References: Message-ID: Hi Christopher, You've got this: An Erlang port interface to libpcap https://github.com/msantos/epcap On Mon, Oct 17, 2011 at 1:06 PM, Lamb, Christopher Charles wrote: > Hey everybody, noob question I'm sure, but I'm trying to find some kind of a > tutorial on how I can sniff network traffic off my NIC using Erlang. ?Does > such a thing exist? ?I've seen quite a few network programming examples, but > nothing that's grabbing traffic from my NIC. I expect I may need to > interface w/ C/C++ to do this? > Anyway, Thanks for any guidance! > - chris > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From mattevans123@REDACTED Mon Oct 17 20:24:25 2011 From: mattevans123@REDACTED (Matthew Evans) Date: Mon, 17 Oct 2011 14:24:25 -0400 Subject: [erlang-questions] Network Sniffer? In-Reply-To: References: , Message-ID: Right, Another good one this guy wrote is: https://github.com/msantos/procket ecap runs in a separate process, procket is a nif, and can pass the fd back to the VM's event loop.... > Date: Mon, 17 Oct 2011 13:11:51 -0500 > From: reynaldomic@REDACTED > To: cclamb@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Network Sniffer? > > Hi Christopher, > > You've got this: > > An Erlang port interface to libpcap > https://github.com/msantos/epcap > > On Mon, Oct 17, 2011 at 1:06 PM, Lamb, Christopher Charles > wrote: > > Hey everybody, noob question I'm sure, but I'm trying to find some kind of a > > tutorial on how I can sniff network traffic off my NIC using Erlang. Does > > such a thing exist? I've seen quite a few network programming examples, but > > nothing that's grabbing traffic from my NIC. I expect I may need to > > interface w/ C/C++ to do this? > > Anyway, Thanks for any guidance! > > - chris > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From cclamb@REDACTED Mon Oct 17 20:25:45 2011 From: cclamb@REDACTED (Lamb, Christopher Charles) Date: Mon, 17 Oct 2011 18:25:45 +0000 Subject: [erlang-questions] Network Sniffer? In-Reply-To: Message-ID: Thanks for the references! I'll check 'em out. - c. From: Matthew Evans > Date: Mon, 17 Oct 2011 14:24:25 -0400 To: >, Lamb Christopher > Cc: > Subject: RE: [erlang-questions] Network Sniffer? Right, Another good one this guy wrote is: https://github.com/msantos/procket ecap runs in a separate process, procket is a nif, and can pass the fd back to the VM's event loop.... > Date: Mon, 17 Oct 2011 13:11:51 -0500 > From: reynaldomic@REDACTED > To: cclamb@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Network Sniffer? > > Hi Christopher, > > You've got this: > > An Erlang port interface to libpcap > https://github.com/msantos/epcap > > On Mon, Oct 17, 2011 at 1:06 PM, Lamb, Christopher Charles > > wrote: > > Hey everybody, noob question I'm sure, but I'm trying to find some kind of a > > tutorial on how I can sniff network traffic off my NIC using Erlang. Does > > such a thing exist? I've seen quite a few network programming examples, but > > nothing that's grabbing traffic from my NIC. I expect I may need to > > interface w/ C/C++ to do this? > > Anyway, Thanks for any guidance! > > - chris > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Mon Oct 17 23:09:23 2011 From: comptekki@REDACTED (Wes James) Date: Mon, 17 Oct 2011 15:09:23 -0600 Subject: [erlang-questions] return list or send message back Message-ID: I'm trying to learn how message passing works so I have a function that sends an "ls" message to another node and then when the other node receives it since "ls" is one of the matches for that it then runs a routine to list the files in a folder. Should I build a list and return the results or send a message back with the results. thanks, -wes From gerryw@REDACTED Mon Oct 17 23:14:30 2011 From: gerryw@REDACTED (Gerry Weaver) Date: Mon, 17 Oct 2011 16:14:30 -0500 Subject: [erlang-questions] Erlang newbie questions Message-ID: <0be88a3c1122a5cc0098411971a1364d@compvia.com> Hello All, I am new to Erlang and I'm trying to make the case to continue learning this language. I have read all of the books available and seen several presentations on the language. I am interested in Erlang primarily for the distributed and fault tolerant features it offers. However, I have some reservations about how useful this really is. For example, if you have a network server and the listener process fails, how do you recover? It seems that the fault tolerance is only applicable in the VM context. This is, in my mind equivalent to any other application process. If I have a network server written in C, I can have a watchdog process to kill and/or restart it in the event of failure. The C based approach actually seems more robust. In this scenario, how would one restart the Erlang VM? It has been my experience that network servers most often fail due to OS or hardware limits rather than bugs in the code (at least for thoroughly tested production ready code). When you factor in the amount of baggage that comes with Erlang, or any other VM based language, it's difficult to justify. Now there is also the rapid development aspect, but I'm finding that at least for now, the time savings is being eaten up by looking for documentation. I understand that this situation will improve over time, but the various posts I've seen from folks with more experience seem to indicate that this will take quite a while. I like the language (except the , ; . thing). You can do some pretty cool things with it, but when I got over that initial gee whiz, I had some trouble seeing a real world production use case. I'm not trying to be critical, I just figure I must have missed something along the way. Any perspectives or advice on this would be greatly appreciated. Thanks, -G From silent_vendetta@REDACTED Mon Oct 17 23:15:12 2011 From: silent_vendetta@REDACTED (Chris Hicks) Date: Mon, 17 Oct 2011 14:15:12 -0700 Subject: [erlang-questions] return list or send message back In-Reply-To: References: Message-ID: Returning something from a function and passing a message are two completely different mechanisms. You can only 'return' a result from a function that has been 'called' such as: initial_function() do stuff .... .... Value = call_function() do more stuff .... call_function() -> do stuff .... .... return value. If you send a message from Process A to Process B, the only way to get anything from B to A would be to send a message back using the PID. Be careful in doing this that you don't create a situation where both processes are waiting for a message from the other in order to continue. > Date: Mon, 17 Oct 2011 15:09:23 -0600 > From: comptekki@REDACTED > To: erlang-questions@REDACTED > Subject: [erlang-questions] return list or send message back > > I'm trying to learn how message passing works so I have a function > that sends an "ls" message to another node and then when the other > node receives it since "ls" is one of the matches for that it then > runs a routine to list the files in a folder. Should I build a list > and return the results or send a message back with the results. > > thanks, > > -wes > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Oct 18 00:18:17 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 18 Oct 2011 00:18:17 +0200 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: <0be88a3c1122a5cc0098411971a1364d@compvia.com> References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> Message-ID: On Mon, Oct 17, 2011 at 23:14, Gerry Weaver wrote: [...] > For example, if you have a network server and the listener process fails, how do you recover? It seems that the fault tolerance is only applicable in the VM context. This is, in my mind equivalent to any other application process. If I have a network server written in C, I can have a watchdog process to kill and/or restart it in the event of failure. The C based approach actually seems more robust. There is a distinct, subtle difference here. C programs tend to be an all or nothing approach. If there is a mistake in the C server, then it doesn't matter where that error happen. It can be in the parsing of a configuration file when you send it a SIGHUP or something such. This will kill the C server, all its internal state, all undergoing transactions, all connections currently on the server and more. That is, either the C program runs and you are happy, or it doesn't run and you have trouble. Erlang has, due to process isolation, the ability to subdivide the server into smaller compartments. An error in one of these will in general not take down the whole server. Like you say, it is rarely the listen loop itself which has the error. It is often something else. Here, Erlang has a built-in mechanism for ditching a wrong or faulty TCP socket without affecting the other connections. So you can say that in Erlang systems, it is not an all-or-nothing approach but more fine grained. The point is that the isolation lets you segment your program such that errors can not propagate uncontrolled from any system to any system. The other subtle difference is that in C, you trade execution speed for safety. C programs are not safe. They segfault rather than raise error. Now, if you have the time and effort to build your program well, it can become robust and stable. But the point is that you will be able to get to that same level of robustness in Erlang for a fraction of the effort. And in addition, Erlang programs tend to run really fast when concurrency and distribution is involved - you need to beat the Erlang VM, a system with well over 15 years of optimization. Finally, there is the point that Erlang is a way better language than C. C is an excellent language if you need an imperative simple language. But if you need abstraction, it is notoriously bad. Erlang, with a proper lambda calculus base, fares much better there. -- J. From kaiduanx@REDACTED Tue Oct 18 00:19:28 2011 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Mon, 17 Oct 2011 18:19:28 -0400 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: <0be88a3c1122a5cc0098411971a1364d@compvia.com> References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> Message-ID: For the network server, if the listener process fails, it gets restarted by the supervisor, listens on the port again, and only THAT process is restarted. The other part of the server is not restarted. In C, you will kill the WHOLE server. Also most of the time, the listener process does little except blocking and accepting connection, then spawning another process to do the real work, the chance it gets restarted is remote. The learning curve is a little bit long, but Erlang/OTP really shines for network server. I have written a network server in Erlang, it has 1/4 LOC compared with the C version with the same feature set. On 10/17/11, Gerry Weaver wrote: > Hello All, > > I am new to Erlang and I'm trying to make the case to continue learning this > language. I have read all of the books available and seen several > presentations on the language. I am interested in Erlang primarily for the > distributed and fault tolerant features it offers. However, I have some > reservations about how useful this really is. For example, if you have a > network server and the listener process fails, how do you recover? It seems > that the fault tolerance is only applicable in the VM context. This is, in > my mind equivalent to any other application process. If I have a network > server written in C, I can have a watchdog process to kill and/or restart it > in the event of failure. The C based approach actually seems more robust. In > this scenario, how would one restart the Erlang VM? It has been my > experience that network servers most often fail due to OS or hardware limits > rather than > bugs in the code (at least for thoroughly tested production ready code). > When you factor in the amount of baggage that comes with Erlang, or any > other VM based language, it's difficult to justify. Now there is also the > rapid development aspect, but I'm finding that at least for now, the time > savings is being eaten up by looking for documentation. I understand that > this situation will improve over time, but the various posts I've seen from > folks with more experience seem to indicate that this will take quite a > while. I like the language (except the , ; . thing). You can do some pretty > cool things with it, but when I got over that initial gee whiz, I had some > trouble seeing a real world production use case. I'm not trying to be > critical, I just figure I must have missed something along the way. Any > perspectives or advice on this would be greatly appreciated. > > > Thanks, > -G > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Sent from my mobile device From g@REDACTED Tue Oct 18 00:27:14 2011 From: g@REDACTED (Garrett Smith) Date: Mon, 17 Oct 2011 17:27:14 -0500 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: <0be88a3c1122a5cc0098411971a1364d@compvia.com> References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> Message-ID: Hi Gerry, On Mon, Oct 17, 2011 at 4:14 PM, Gerry Weaver wrote: > I am new to Erlang and I'm trying to make the case to continue learning this language. I have read all of the books available and seen several presentations on the language. I am interested in Erlang primarily for the distributed and fault tolerant features it offers. However, I have some reservations about how useful this really is. For example, if you have a network server and the listener process fails, how do you recover? It seems that the fault tolerance is only applicable in the VM context. This is, in my mind equivalent to any other application process. If I have a network server written in C, I can have a watchdog process to kill and/or restart it in the event of failure. The C based approach actually seems more robust. In this scenario, how would one restart the Erlang VM? It has been my experience that network servers most often fail due to OS or hardware limits rather than > bugs in the code (at least for thoroughly tested production ready code). When you factor in the amount of baggage that comes with Erlang, or any other VM based language, it's difficult to justify. Now there is also the rapid development aspect, but I'm finding that at least for now, the time savings is being eaten up by looking for documentation. I understand that this situation will improve over time, but the various posts I've seen from folks with more experience seem to indicate that this will take quite a while. I like the language (except the , ; . thing). You can do some pretty cool things with it, but when I got over that initial gee whiz, I had some trouble seeing a real world production use case. I'm not trying to be critical, I just figure I must have missed something along the way. Any perspectives or advice on this would be greatly appreciated. You typically don't want to use Erlang's supervisory patterns across a network. As you've observed, those are suited for a single VM. Fail fast + watch dog patterns are indeed something you can implement in any language. In Erlang, however, you have a finer level of granularity of isolation. In a C program, your level of isolation is the OS process. If that application encounters a fault, you need to bring the entire OS process down and restart it. In Erlang, your program can consist of hundreds of Erlang processes (or more) each completely isolated from one another and free to crash / restart. Because your level of isolation is more granular, a fault will have relatively less impact on your system. To answer your question, your don't restart the Erlang VM. Think of the Erlang VM as an operating system for code and Erlang processes like OS processes. It's the same basic architecture, but mapped to application space. Erlang's pattern matching semantics will let you write clear "happy path" code without extra defensive or fail-fast code. In C you need to constantly check results -- tedious and error prone. In Erlang, you typically get the assertion for free in patterns like: {ok, F} = file:open(...) Because process supervision comes with Erlang, you don't need defensive code (e.g. gratuitous exception handlers or case statements) -- if something unexpected happens, your code blows up, but gets restarted in a known/clean state. This results in much, much less code *and* improved fault tolerance! Since you're new to Erlang, you're going to be more productive in any other language at the moment. That will certainly change. In my experience, Erlang is an extremely productive language -- I'll argue more so than Python or Ruby. Sounds strange, but I've found that to be the case. Good luck! Garrett From freza@REDACTED Tue Oct 18 00:41:19 2011 From: freza@REDACTED (Jachym Holecek) Date: Mon, 17 Oct 2011 18:41:19 -0400 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: <0be88a3c1122a5cc0098411971a1364d@compvia.com> References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> Message-ID: <20111017224119.GA1454@circlewave.net> # Gerry Weaver 2011-10-17: > I am new to Erlang and I'm trying to make the case to continue learning > this language. I have read all of the books available and seen several > presentations on the language. I am interested in Erlang primarily for > the distributed and fault tolerant features it offers. And you won't be let down on that -- but it takes a while to appreciate the higher-level engineering aspects of OTP, so don't give up too early. Given your good theoretical preparations, I'd suggest the next good step would be to write a non-trivial toy project in Erlang -- after all, it's a hard fact of nature that actually doing it is the only way to ever learn anything. > However, I have > some reservations about how useful this really is. For example, if you > have a network server and the listener process fails, how do you recover? The bright idea is that Erlang/OTP provides you with very natural tools to address these situations (impenetrable process isolation being the basis, but you have timeouts for free, links, monitors, etc), but the exact strategy is inherently application-specific. Popular choice for dealing with failed communication link would be to schedule reconnect attempt and revert to hot-spare meanwhile. I suppose MIT X11's mantra, "mechanism, not policy", applies nicely here (regarding base Erlang/OTP). > It seems that the fault tolerance is only applicable in the VM context. Fair observation, it's actually quite reasonable to think of one BEAM process as an operating system instance of its own -- it happens to run on top of some UN*X hypervisor typically, and you'd usually arrange for the BEAM to be restarted on critical failure (much like you'd configure your OS to automatically restart on kernel panic). Does this make sense? > This is, in my mind equivalent to any other application process. If I > have a network server written in C, I can have a watchdog process to kill > and/or restart it in the event of failure. The C based approach actually > seems more robust. In this scenario, how would one restart the Erlang VM? A number of choices -- I know Solaris SMF works, I imagine DJB's supervise would work too (as exotic as djbware is...). One thing to emphasize here is that crashing BEAM means one of two things: 1) you didn't provision for the right kind of flow-control (or rate throttling or ...) mechanisms to ensure your node fits applicable resource limits -- for this you can only blame yourself, or 2) you somehow managed to provoke a drastic error in BEAM (very very very rare, but not unheard of). Either case is really akin to an OS crash -- a catastrophic, worst-case scenario. Of course, other nodes talking to the failed one can deal with it gracefully. > It has been my experience that network servers most often fail due to OS > or hardware limits rather than bugs in the code See above, hitting OS/HW limits is just engineer's failure to exert due diligence to fit within those limits. Hardware failure or operator error however I couldn't argue with. Also, see below on completely different scales of software integration you'll meet in Erlang -- that increases the likelihood of programming error wanting to spoil the party. > (at least for thoroughly tested production ready code). [Must suppress the urge to make cynical comments.] > When you factor in the amount of baggage > that comes with Erlang, or any other VM based language, it's difficult > to justify. I can understand this prejudice, other way to see this is that, much like an OS, the BEAM is part of your TCB. Any error therein is fatal and you can't do much about it. One of the services it provides you however is blank cheque on safety one Erlang process has regardless, for one, of what other Erlang processes do, but for other regardless of what the code running in that process does (is allowed to do). The guarantess given to a UN*X process by OS kernel seem rather lame in comparison. In my mind, this is definitely worth it. > Now there is also the rapid development aspect, but I'm > finding that at least for now, the time savings is being eaten up by > looking for documentation. I understand that this situation will improve > over time, but the various posts I've seen from folks with more experience > seem to indicate that this will take quite a while. I like the language > (except the , ; . thing). Yep, there's some learning curve to deal with, but the closer it starts resembling a constant function, the more unbelievable will your productivity be -- rolling production-ready solutions once every three weeks is not complete science-fiction :-). One thing I wanted to mention, and this is rather random place to do so, is that besides the other advantages of Erlang/OTP you've already mentioned (or the books you've read surely did) there's also one people don't talk about very often: high-level compositionality of sorts -- not at the level of functions or modules, but whole applications (or application stacks). You have an existing solution and suddenly are tasked with adding Diameter interface for charging, a web-based GUI for Operations and SNMP monitoring interface all at once? Big deal, just add a few protocol stacks to your boot file, write a few glue-modules exporting existing information to them, and you're good to go. > You can do some pretty cool things with it, > but when I got over that initial gee whiz, I had some trouble seeing a > real world production use case. I'd choose Erlang/OTP for anything that requires complex control logic or gluing together of many networked systems. I'm a bit too tired to give specific examples at the moment, can fill in tomorrow and others will surely be eager to provide their own if you're interested. > I'm not trying to be critical, I just > figure I must have missed something along the way. Any perspectives or > advice on this would be greatly appreciated. That's fine, coming from embedded systems background (the land of C and asm) I can understand your skepticism... (I know I wouldn't have believed nice words until I 1) was open-minded enough to listen 2) went on writing Erlang code for a while). Hope this slightly wine-infused mail was helpful, or at least completely harmless. :-) BR, -- Jachym From gerryw@REDACTED Tue Oct 18 02:17:08 2011 From: gerryw@REDACTED (Gerry Weaver) Date: Mon, 17 Oct 2011 19:17:08 -0500 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: <20111017224119.GA1454@circlewave.net> References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> Message-ID: Hi, Wow! I really appreciate your feedback. I find your responses very logical and helpful ;-)? Another issue that I've started to think about is deployment. There seem to be several competing tools/utilities for this. Is there a preferred or recommended solution? Is the Erlang language itself still evolving? The reason I ask is that the fellow that did CouchDB posted a blog entry about the things he encountered while developing CouchDB. What was interesting to me is how, even after a short time with the language, many of the issues he mentioned immediately rang true with me. For example, what is the benefit of the "; , ." terminators as opposed to a single terminator approach? I don't mean to sound negative here. I'm just poking around a bit. I find Erlang quite compelling overall. I'm digging into the OTP at this point and there are some really interesting capabilities there.? I assume a best practice is to minimize the use of modules written in other languages. What exactly would the scenario be for a C module that segfaults? I assume it would take down the VM. Thanks, -G -----Original Message----- > From: "Jachym Holecek" > To: "Gerry Weaver" > Cc: erlang-questions@REDACTED > Date: 10/17/11 17:41 > Subject: Re: [erlang-questions] Erlang newbie questions > > # Gerry Weaver 2011-10-17: > > I am new to Erlang and I'm trying to make the case to continue learning > > this language. I have read all of the books available and seen several > > presentations on the language. I am interested in Erlang primarily for > > the distributed and fault tolerant features it offers. > > And you won't be let down on that -- but it takes a while to appreciate > the higher-level engineering aspects of OTP, so don't give up too early. > Given your good theoretical preparations, I'd suggest the next good step > would be to write a non-trivial toy project in Erlang -- after all, it's > a hard fact of nature that actually doing it is the only way to ever > learn anything. > > > However, I have > > some reservations about how useful this really is. For example, if you > > have a network server and the listener process fails, how do you recover? > > The bright idea is that Erlang/OTP provides you with very natural tools > to address these situations (impenetrable process isolation being the > basis, but you have timeouts for free, links, monitors, etc), but the > exact strategy is inherently application-specific. Popular choice for > dealing with failed communication link would be to schedule reconnect > attempt and revert to hot-spare meanwhile. I suppose MIT X11's mantra, > "mechanism, not policy", applies nicely here (regarding base Erlang/OTP). > > > It seems that the fault tolerance is only applicable in the VM context. > > Fair observation, it's actually quite reasonable to think of one BEAM > process as an operating system instance of its own -- it happens to run > on top of some UN*X hypervisor typically, and you'd usually arrange for > the BEAM to be restarted on critical failure (much like you'd configure > your OS to automatically restart on kernel panic). Does this make sense? > > > This is, in my mind equivalent to any other application process. If I > > have a network server written in C, I can have a watchdog process to kill > > and/or restart it in the event of failure. The C based approach actually > > seems more robust. In this scenario, how would one restart the Erlang VM? > > A number of choices -- I know Solaris SMF works, I imagine DJB's supervise > would work too (as exotic as djbware is...). One thing to emphasize here > is that crashing BEAM means one of two things: 1) you didn't provision for > the right kind of flow-control (or rate throttling or ...) mechanisms to > ensure your node fits applicable resource limits -- for this you can only > blame yourself, or 2) you somehow managed to provoke a drastic error in > BEAM (very very very rare, but not unheard of). Either case is really > akin to an OS crash -- a catastrophic, worst-case scenario. Of course, > other nodes talking to the failed one can deal with it gracefully. > > > It has been my experience that network servers most often fail due to OS > > or hardware limits rather than bugs in the code > > See above, hitting OS/HW limits is just engineer's failure to exert due > diligence to fit within those limits. Hardware failure or operator error > however I couldn't argue with. Also, see below on completely different > scales of software integration you'll meet in Erlang -- that increases > the likelihood of programming error wanting to spoil the party. > > > (at least for thoroughly tested production ready code). > > [Must suppress the urge to make cynical comments.] > > > When you factor in the amount of baggage > > that comes with Erlang, or any other VM based language, it's difficult > > to justify. > > I can understand this prejudice, other way to see this is that, much like > an OS, the BEAM is part of your TCB. Any error therein is fatal and you > can't do much about it. One of the services it provides you however is > blank cheque on safety one Erlang process has regardless, for one, of > what other Erlang processes do, but for other regardless of what the > code running in that process does (is allowed to do). The guarantess > given to a UN*X process by OS kernel seem rather lame in comparison. > In my mind, this is definitely worth it. > > > Now there is also the rapid development aspect, but I'm > > finding that at least for now, the time savings is being eaten up by > > looking for documentation. I understand that this situation will improve > > over time, but the various posts I've seen from folks with more experience > > seem to indicate that this will take quite a while. I like the language > > (except the , ; . thing). > > Yep, there's some learning curve to deal with, but the closer it starts > resembling a constant function, the more unbelievable will your productivity > be -- rolling production-ready solutions once every three weeks is not > complete science-fiction :-). One thing I wanted to mention, and this > is rather random place to do so, is that besides the other advantages > of Erlang/OTP you've already mentioned (or the books you've read surely > did) there's also one people don't talk about very often: high-level > compositionality of sorts -- not at the level of functions or modules, > but whole applications (or application stacks). You have an existing > solution and suddenly are tasked with adding Diameter interface for > charging, a web-based GUI for Operations and SNMP monitoring interface > all at once? Big deal, just add a few protocol stacks to your boot > file, write a few glue-modules exporting existing information to them, > and you're good to go. > > > You can do some pretty cool things with it, > > but when I got over that initial gee whiz, I had some trouble seeing a > > real world production use case. > > I'd choose Erlang/OTP for anything that requires complex control logic > or gluing together of many networked systems. I'm a bit too tired to > give specific examples at the moment, can fill in tomorrow and others > will surely be eager to provide their own if you're interested. > > > I'm not trying to be critical, I just > > figure I must have missed something along the way. Any perspectives or > > advice on this would be greatly appreciated. > > That's fine, coming from embedded systems background (the land of C and > asm) I can understand your skepticism... (I know I wouldn't have believed > nice words until I 1) was open-minded enough to listen 2) went on writing > Erlang code for a while). > > Hope this slightly wine-infused mail was helpful, or at least completely > harmless. :-) > > BR, > ?????-- Jachym > > > > From mononcqc@REDACTED Tue Oct 18 02:40:57 2011 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 17 Oct 2011 20:40:57 -0400 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> Message-ID: Going to reply inline: On Mon, Oct 17, 2011 at 8:17 PM, Gerry Weaver wrote: > > Another issue that I've started to think about is deployment. There seem > to be several competing tools/utilities for this. Is there a preferred or > recommended solution? You've got 3 levels: 1. Rolling out basic Erlang. This is often done by people who do not want or know how to deal with the OTP framework, up to supervisors and applications. You'll see basic scripts calling main, scaffoldings of escripts and whatnot. This isn't something I'd recommend (not that Escript is the problem, but OTP is a nice way to work with code with the rest of Erlang programmers). 2. OTP Applications are being used. People up to there use OTP applications, can see dependencies and whatnot, and will manually start everything using calls to 'application:start(AppName)' in sequence. Again, escripts might be used here as a way to bootstrap the application. This model is somewhat sane, everyone can understand it, but might have problems when it comes to upgrading running code. 3. OTP Releases. They're a bit annoying to figure out. You've got two ways to package releases: reltools and systools. People will pick one depending on whatever they find most convenient. In any case, most people will tend to use tools like rebar to handle them (especially since rebar also handles downloading dependencies and whatnot). Despite the complexity, releases will have nice aspects, like automatically handling application dependencies, starting things for you, safely doing application and release upgrades (relups). If you're doing it by hand (or using cross-compatible tools), release building should be cross-compatible with other platforms. This doesn't mention how to do things like setting up distributed applications (more work with OTP apps/releases, central connection nodes, etc.), how people push code on servers (some people use Erlang to send code to other nodes with the code server and rpc, rsync it, use pulls with source control, Chef, linux packages, etc.) and a few other things. Self-advertisement: OTP Releases: http://learnyousomeerlang.com/release-is-the-word Relups: http://learnyousomeerlang.com/relups > Is the Erlang language itself still evolving? The reason I ask is that the > fellow that did CouchDB posted a blog entry about the things he encountered > while developing CouchDB. What was interesting to me is how, even after a > short time with the language, many of the issues he mentioned immediately > rang true with me. For example, what is the benefit of the "; , ." > terminators as opposed to a single terminator approach? > Prolog would be the easy answer. I would however want to correct you. The full stop is a terminator of Erlang *forms* and others (;,) are *separators for expressions and forms.* I won't get into too many details. I've decided to write a blog post on this a year ago because this is such a common complaint: http://ferd.ca/on-erlang-s-syntax.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.joseph.davis@REDACTED Tue Oct 18 05:36:34 2011 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Mon, 17 Oct 2011 22:36:34 -0500 Subject: [erlang-questions] Listing differences in loaded beam files Message-ID: I hit an odd issue earlier today and one of the debugging questions I tried to answer was "Do these two nodes have the same exact BEAM files loaded, and if not, what's different?" After Googling for awhile I haven't found anything related. Ideally I'm only looking for something that basically does: Mods1 = [{Module, beam_lib:version(Module)} || Module <- all_modules_on_node(N1)], Mods2 = [{Module, beam_lib:version(Module)} || Module <- all_modules_on_node(N2)], diff(Mods1, Mods2). But with bonus points if it shows some sort of N-way diff for all nodes that are connected. Any hints or suggestions would be appreciated. Thanks, Paul J. Davis From erlang@REDACTED Tue Oct 18 09:12:43 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 18 Oct 2011 09:12:43 +0200 Subject: [erlang-questions] return list or send message back In-Reply-To: References: Message-ID: On Mon, Oct 17, 2011 at 11:09 PM, Wes James wrote: > I'm trying to learn how message passing works so I have a function > that sends an "ls" message to another node and then when the other > node receives it since "ls" is one of the matches for that it then > runs a routine to list the files in a folder. ?Should I build a list > and return the results or send a message back with the results. The first alternative is easy "build a list and return the results: yields the code loop() -> receive {From, ls} -> {ok, Dir} = file:get_cwd(), {ok, FileNames} = file:list_dir(Dir), From ! {self(), FileNames}, loop() end. (Note: This *implies* a certain directory - so I have used file:get_cwd() (the current working directory) But a better interface is to accept the message {ls, Dir} rather than ls. "send a message back with the results" - but what does "the results" mean? Does it mean this? loop() -> receive {From, ls} -> {ok, Dir} = file:get_cwd(), From ! {self(), file:list_dir(Dir)}, loop() end. Note - you could be even *more* general - the server loop might be like this: loop() -> receive {From, F} -> From ! {self(), F()}, loop() end. Then you send the function you want evaluating to the server in a message like this: Server ! {self(), fun() -> file:list_dir("/home/joe") end} As you can see there are many ways to do things. Which is the "best" way - there is no answer - it depends upon your criteria for "bestness" The one that is "easiest to debug" is probably the one that just sends back a list of files it's well typed and easy to understand. Sending funs (the last alternative) is *incredibly* powerful but can yield code that it difficult to understand and debug. Cheers /Joe > > thanks, > > -wes > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ulf.wiger@REDACTED Tue Oct 18 09:47:45 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 18 Oct 2011 09:47:45 +0200 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: <0be88a3c1122a5cc0098411971a1364d@compvia.com> References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> Message-ID: <0F49BCCC-EE36-4C21-A717-951D79BD730F@erlang-solutions.com> On 17 Oct 2011, at 23:14, Gerry Weaver wrote: > It has been my experience that network servers most often fail due to OS or hardware limits rather than > bugs in the code (at least for thoroughly tested production ready code). When you factor in the amount of baggage that comes with Erlang, or any other VM based language, it's difficult to justify. Gerry, You have been given other helpful answers from the good people of this list. I thought I'd add something from my own experience writing commercial code in Erlang. The thing I've noticed about Erlang-based systems is that they tend to mature very well. I attribute this to a number of things: == Functional programming style == The parts of your code that rely on pure functions (no side-effects) have the nice property that once you have weeded out the bugs, the code *stays correct*, until requirements change. I've seen this in industrial erlang-based systems: they have been robust from the start, but the quality just keeps going up. == Error handling + programming for the correct case == The combo of pattern-matching, purely functional style, and process supervision, further support the notion of writing functions that are correct by design. They do what they are supposed to for expected input, and simply crash for unexpected/invalid input. This style of programming actually works very well as a *default* in Erlang. For extra robustness, one usually needs to trap and handle errors especially in a few places, but that kind of hardening is fairly straightforward, and can be done as incremental improvements. == Ability to limit the state-event space == It's been mentioned that Erlang shines especially for control and coordination scenarios. An important reason is that Erlang allows you to minimise the interference between different components and events. In many design approaches, it is very hard to introduce new protocols or events, as the effects cut across the code in a very bad way. For many systems, this has the effect that they don't mature well. Changes in traffic conditions, hardware and network topologies, and requirements all have a tendency to trigger a new flurry of bugs, due to things happening in a different order than before, or events combining in unexpected ways. I have compared this to the evils of GOTO programming [1]. Given the old rule of thumb that 80% of the lifecycle cost of a system lies in evolution and maintenance, this makes of a pretty formidable advantage. In one system I worked with, our first release was 4x better than the company norm on faults/KLOC. After a few releases - with significant feature growth - we were 12x better. Code that was corrected for bugs tended to stay correct, and bug fixes normally didn't introduce any new bugs. A colleague of mine, Mats Cronqvist, once wrote in a workshop paper [2], after having analysed some 150 trouble reports from function- and system testing: "Most of the errors were not coding errors, but simply a working implementation of the wrong thing." (page 2) Another way of putting it would be that there are comparatively few "accidental errors" in Erlang-based systems - that is, errors that are a consequence of the implementation technique rather than a misunderstanding of the requirements. BR, Ulf W [1] http://www.infoq.com/presentations/Death-by-Accidental-Complexity [2] "Troubleshooting a Large Erlang System", Mats Cronqvist, 2004 ACM/SIGPLAN Erlang Workshop http://dl.acm.org/citation.cfm?id=1022474 http://www.erlang.se/workshop/2004/cronqvist.pdf Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.widgren@REDACTED Tue Oct 18 10:15:36 2011 From: daniel.widgren@REDACTED (Daniel Widgren) Date: Tue, 18 Oct 2011 10:15:36 +0200 Subject: [erlang-questions] C code coverage Message-ID: Hi, I'm testing c code with Common Test and is wondering if it is any good tools to combine Erlang and get code coverage when you run ct suites. regards, Daniel From cgsmcmlxxv@REDACTED Tue Oct 18 10:32:33 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Tue, 18 Oct 2011 10:32:33 +0200 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> Message-ID: <4E9D39A1.2050008@gmail.com> Hi, Interesting discussion here and I would like few points (as C is also my favorite programming language). The main point for entering the discussion is the fact that many who replied here were saying that if something crashes in your application doesn't kill your application like in C. I beg to differ on this point. If your Erlang application is single threaded, then your application (I do not speak here about OTP application) will crash if an error occurs there. If you construct your C application like an OTP application, then that means your C applications is multithreaded, so, if one thread dies, it doesn't crash your main application (as well as in OTP application, if one gen_server crashes the whole application doesn't crash). Now, Gerry, think Erlang as being multithread application in C where you don't need to care too much about how to construct the thread. That means, the construction of each thread is much simpler and you already have the monitor (watchdog, if you prefer) part implemented, so, you can focus on what you want to develop and less on the tools you need. There are few more aspects here you may want to consider before choosing in which language you want to write your code. Firstly, the network applications. In C, due to the lack of standards, you have to know the compiler in which you want to program a port listener (as example). In Erlang, you get the standard which is missing in C. Therefore, the productivity is higher in case of Erlang in this case. Staying in the field of networking, some may argue that Erlang is not the only one to be productive here. And that is correct if you consider JAVA (as one of the main languages capable of creating cross-platform network applications). Still, I would prefer Erlang for its VM and basic concepts, and, as C programmer, I think you would understand me why I prefer each thread to have its own memory (just for example). Secondly, as desktop application, generally, Erlang is far worse choice than C. Still, if you don't care too much of the speed of your application, Erlang offers some nice tools for increasing your productivity as programmer. Also, due to its capability of interacting with external functions written in C, you can create your own fast libraries which you can connect them to Erlang (here you will spend a little bit of time to understand the concept behind). Thirdly, concerning the application design. If in C you need to design your application from A to Z, in Erlang you have tools which limit your design, but it can increase your overall productivity. Actually, I consider OTP design as being one of the most useful for general purpose. I will stop here with the list. You will discover many other advantages using Erlang during making your own experience with Erlang. I am no expert in Erlang and many things I am still discovering while programming in this language. And, for you to see that I bumped into the same problems, I had also some headaches with the line separators (many times I got compiling errors due to them). But, in the end, those separators turned out to be more useful than pain in the... Just for example, a loop in C when you don't want to use the for statement is written like this (take it as principle and not as something to be compiled): myfunc(myvars) { exit_condition; the_loop_body; myfunc(modified_vars); } while in Erlang, it is looks like that: myfunc(myvars) -> the_loop_body, myfunct(modified_vars); myfunc(stop_vars) -> end_of_loop. As you can see it's not so different, but you have to implement explicitly the end of the loop. Quite a pain you may say. Actually, using this method, think of having two functions with the same name, but with the language choosing which one to be used depending on what arguments you pass to it. Doesn't it sound a bit familiar? Yes. And more than that, you know the advantage of this in practical applications. So, "two rabbits with a single gun shot" (as there is a saying in my country). OK! One can speak whole day long about the advantages and disadvantages of Erlang, but the most important is for you to make your own experience. So far, my experience with Erlang was positive and I found myself liking Erlang quite some. I wish your experience with Erlang to be as good as mine. Just don't forget each language has its own beauty, but you need to know from which angle you should look at it. Cheers, CGS On 10/18/2011 02:17 AM, Gerry Weaver wrote: > Hi, > > Wow! I really appreciate your feedback. I find your responses very logical and helpful ;-) > > > Another issue that I've started to think about is deployment. There seem to be several competing tools/utilities for this. Is there a preferred or recommended solution? > > > Is the Erlang language itself still evolving? The reason I ask is that the fellow that did CouchDB posted a blog entry about the things he encountered while developing CouchDB. What was interesting to me is how, even after a short time with the language, many of the issues he mentioned immediately rang true with me. For example, what is the benefit of the "; , ." terminators as opposed to a single terminator approach? > > > I don't mean to sound negative here. I'm just poking around a bit. I find Erlang quite compelling overall. I'm digging into the OTP at this point and there are some really interesting capabilities there. > > > I assume a best practice is to minimize the use of modules written in other languages. What exactly would the scenario be for a C module that segfaults? I assume it would take down the VM. > > > > > Thanks, > -G > > > > > > > > > > -----Original Message----- >> From: "Jachym Holecek" >> To: "Gerry Weaver" >> Cc: erlang-questions@REDACTED >> Date: 10/17/11 17:41 >> Subject: Re: [erlang-questions] Erlang newbie questions >> >> # Gerry Weaver 2011-10-17: >>> I am new to Erlang and I'm trying to make the case to continue learning >>> this language. I have read all of the books available and seen several >>> presentations on the language. I am interested in Erlang primarily for >>> the distributed and fault tolerant features it offers. >> And you won't be let down on that -- but it takes a while to appreciate >> the higher-level engineering aspects of OTP, so don't give up too early. >> Given your good theoretical preparations, I'd suggest the next good step >> would be to write a non-trivial toy project in Erlang -- after all, it's >> a hard fact of nature that actually doing it is the only way to ever >> learn anything. >> >>> However, I have >>> some reservations about how useful this really is. For example, if you >>> have a network server and the listener process fails, how do you recover? >> The bright idea is that Erlang/OTP provides you with very natural tools >> to address these situations (impenetrable process isolation being the >> basis, but you have timeouts for free, links, monitors, etc), but the >> exact strategy is inherently application-specific. Popular choice for >> dealing with failed communication link would be to schedule reconnect >> attempt and revert to hot-spare meanwhile. I suppose MIT X11's mantra, >> "mechanism, not policy", applies nicely here (regarding base Erlang/OTP). >> >>> It seems that the fault tolerance is only applicable in the VM context. >> Fair observation, it's actually quite reasonable to think of one BEAM >> process as an operating system instance of its own -- it happens to run >> on top of some UN*X hypervisor typically, and you'd usually arrange for >> the BEAM to be restarted on critical failure (much like you'd configure >> your OS to automatically restart on kernel panic). Does this make sense? >> >>> This is, in my mind equivalent to any other application process. If I >>> have a network server written in C, I can have a watchdog process to kill >>> and/or restart it in the event of failure. The C based approach actually >>> seems more robust. In this scenario, how would one restart the Erlang VM? >> A number of choices -- I know Solaris SMF works, I imagine DJB's supervise >> would work too (as exotic as djbware is...). One thing to emphasize here >> is that crashing BEAM means one of two things: 1) you didn't provision for >> the right kind of flow-control (or rate throttling or ...) mechanisms to >> ensure your node fits applicable resource limits -- for this you can only >> blame yourself, or 2) you somehow managed to provoke a drastic error in >> BEAM (very very very rare, but not unheard of). Either case is really >> akin to an OS crash -- a catastrophic, worst-case scenario. Of course, >> other nodes talking to the failed one can deal with it gracefully. >> >>> It has been my experience that network servers most often fail due to OS >>> or hardware limits rather than bugs in the code >> See above, hitting OS/HW limits is just engineer's failure to exert due >> diligence to fit within those limits. Hardware failure or operator error >> however I couldn't argue with. Also, see below on completely different >> scales of software integration you'll meet in Erlang -- that increases >> the likelihood of programming error wanting to spoil the party. >> >>> (at least for thoroughly tested production ready code). >> [Must suppress the urge to make cynical comments.] >> >>> When you factor in the amount of baggage >>> that comes with Erlang, or any other VM based language, it's difficult >>> to justify. >> I can understand this prejudice, other way to see this is that, much like >> an OS, the BEAM is part of your TCB. Any error therein is fatal and you >> can't do much about it. One of the services it provides you however is >> blank cheque on safety one Erlang process has regardless, for one, of >> what other Erlang processes do, but for other regardless of what the >> code running in that process does (is allowed to do). The guarantess >> given to a UN*X process by OS kernel seem rather lame in comparison. >> In my mind, this is definitely worth it. >> >>> Now there is also the rapid development aspect, but I'm >>> finding that at least for now, the time savings is being eaten up by >>> looking for documentation. I understand that this situation will improve >>> over time, but the various posts I've seen from folks with more experience >>> seem to indicate that this will take quite a while. I like the language >>> (except the , ; . thing). >> Yep, there's some learning curve to deal with, but the closer it starts >> resembling a constant function, the more unbelievable will your productivity >> be -- rolling production-ready solutions once every three weeks is not >> complete science-fiction :-). One thing I wanted to mention, and this >> is rather random place to do so, is that besides the other advantages >> of Erlang/OTP you've already mentioned (or the books you've read surely >> did) there's also one people don't talk about very often: high-level >> compositionality of sorts -- not at the level of functions or modules, >> but whole applications (or application stacks). You have an existing >> solution and suddenly are tasked with adding Diameter interface for >> charging, a web-based GUI for Operations and SNMP monitoring interface >> all at once? Big deal, just add a few protocol stacks to your boot >> file, write a few glue-modules exporting existing information to them, >> and you're good to go. >> >>> You can do some pretty cool things with it, >>> but when I got over that initial gee whiz, I had some trouble seeing a >>> real world production use case. >> I'd choose Erlang/OTP for anything that requires complex control logic >> or gluing together of many networked systems. I'm a bit too tired to >> give specific examples at the moment, can fill in tomorrow and others >> will surely be eager to provide their own if you're interested. >> >>> I'm not trying to be critical, I just >>> figure I must have missed something along the way. Any perspectives or >>> advice on this would be greatly appreciated. >> That's fine, coming from embedded systems background (the land of C and >> asm) I can understand your skepticism... (I know I wouldn't have believed >> nice words until I 1) was open-minded enough to listen 2) went on writing >> Erlang code for a while). >> >> Hope this slightly wine-infused mail was helpful, or at least completely >> harmless. :-) >> >> BR, >> -- Jachym >> >> >> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From clist@REDACTED Tue Oct 18 10:47:31 2011 From: clist@REDACTED (Angel J. Alvarez Miguel) Date: Tue, 18 Oct 2011 10:47:31 +0200 Subject: [erlang-questions] escript and archives Message-ID: <201110181047.31133.clist@uah.es> Hi, I want to packe some beams on a .ez archive and then insert the escript header to get a selfcontaines script ala rebar.. i made a script witth a zip archive the usual way.. Files = filelib:fold_files("./ebin", ".*", true, fun(Path, Acc) -> io:format("Procesando fichero ~s \n",[Path]), {ok, Bin} = file:read_file(Path), [{Path, Bin}|Acc] end, []), {ok, {"mem", ZipBin}} = zip:create("mem", Files, [memory]), Script = <<"#!/usr/bin/env escript\n%%! -noshell -noinput\n", ZipBin/binary>>, ok = file:write_file("walker", Script), os:cmd("chmod u+x walker"), i have my ebin dir with my two test files, the getopt.erl from jcomellas git and a simple walker.erl file: -module(walker). -export([main/1]). main([String]) -> io:format("Up and running!! mayormente... (args:~s )\n",[String]), halt(0); main(_) -> io:format("Uso: ~s \n",[escript:script_name()]), halt(1). but then escript doesnt find the walker file: ./walker escript: exception error: undefined function walker:main/1 in function escript:run/2 in call from escript:start/1 in call from init:start_it/1 in call from init:start_em/1 if I place the escript into the ebin files it obviouosly find the waker module but is the .beam that is there!. Do need I to include a "walker.app" to allow script to find where my main function is from the zip archive? Regards, Angel From rickard@REDACTED Tue Oct 18 10:59:34 2011 From: rickard@REDACTED (Rickard Green) Date: Tue, 18 Oct 2011 10:59:34 +0200 Subject: [erlang-questions] Fwd: New NIF behavior in R15??? In-Reply-To: References: Message-ID: <4E9D3FF6.2090301@erlang.org> Matthew Evans wrote: > > Hi > > Within the next month or two we will be starting a project interfacing > Erlang to a C driver. This requires the use of a thread pool to > support asynchronous behaviors. Ordinarily I would implement this with > a linked in port driver. However, Rickard Green at his 2011 SF Erlang > conference talk indicated that new support in the NIF framework might > be forthcoming in R15 to support linked in driver style threading > behavior in NIFS. > > I would rather hold off implementing a linked in driver if the new NIF > features are around the corner. Can anyone give any hints if this will > be the case? > > Thanks > > Matt > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Unfortunately, these features are not around the corner and wont be part of the R15 release. We have had to prioritize other things. We still plan to implement these features, but currently I cannot say anything about when to expect them to show up. The only thing I can say is that a fully supported, non-experimental version will not show up until at *earliest* R16. Regards, Rickard Green, Erlang/OTP, Ericsson AB From sverker@REDACTED Tue Oct 18 11:03:19 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Tue, 18 Oct 2011 11:03:19 +0200 Subject: [erlang-questions] ETS bag performance characteristics In-Reply-To: References: Message-ID: <4E9D40D7.30905@erix.ericsson.se> Kris Rasmussen wrote: > Hi, > > I am currently using the ETS bag table type to store a list of PIDs > associated with a given key. I'm concerned that the bag implementation may > require a linear traversal to remove (and possibly also add) elements to the > bag. Does anyone know how ETS bags are implemented? Ideally, I'd like to > have constant lookups by key as well as constant adds/contains/removals. If > I were to implement this outside of erlang I would use a Map > type data structure. > All ETS tables, except ordered_set, are hash tables with separate chaining to resolve collisions. On average constant time for insert/lookup/removal of scattered keys. A bag with lots of identical keys may give bad performance as that will result in linear searches between objects with the same key (and others that happen to hash to the same bucket). ordered_set is implented as a binary AVL tree. /Sverker, Erlang/OTP Ericsson From watson.timothy@REDACTED Tue Oct 18 11:07:48 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 18 Oct 2011 10:07:48 +0100 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> Message-ID: > > Another issue that I've started to think about is deployment. There seem to > be several competing tools/utilities for this. Is there a preferred or > recommended solution? > > I think like most platforms (Java, Python, Ruby, etc) this tends to depend on what people actually prefer to do. At work we have one rule, which is that deployment has to be fully automated. The data centre guys use puppet to manage boxes, so we have put together some service definitions that are always installed on certain "kinds" of machine. In other cases, I've seen teams deploy using Capistrano and/or Fabric to great effect. The Dukes of Erl have a solution for putting an Erlang/OTP release into a package management chain so you can install using apt/yum/etc. Erlang/OTP provides a very useful concept called a release, which is essentially a tarball containing everything need to run your application/stack. This is probably the basic unit of distribution for most deployments, and can be produced automatically by build tools such as rebar (shameless plug for my https://github.com/hyperthunk/rebar_dist_pluginthere). :) > > Is the Erlang language itself still evolving? The reason I ask is that the > fellow that did CouchDB posted a blog entry about the things he encountered > while developing CouchDB. What was interesting to me is how, even after a > short time with the language, many of the issues he mentioned immediately > rang true with me. For example, what is the benefit of the "; , ." > terminators as opposed to a single terminator approach? > > Whilst the core syntax of the language is unlikely to change in the near future, many new language features are appearing all the time! The "; , ." thing got me for a little while, as coming from Python and OCaml I was used to indentation and block termination keywords (e.g., "end") and of course the C-like languages that use only ";" everywhere. Nowadays however, not only am I fully used to the syntax, I actually find it helpful in reading the code. It really does work as punctuation, and I'm sure there are lots of ways to improve it but it doesn't need to reduce your productivity as long as you can get used to it after a while (took me over 6 months writing code every day to be comfortable without scrutinising too often). Also beam's abstract code is very trivial to generate, so writing your own front end to compile down to regular Erlang modules is very plausible. There is a Lisp Flavoured Erlang project which is super cool and runs on the beam emulator, and there is a Ruby-like language called Reia ( http://reia-lang.org/). I have no idea whether these projects are fit for use in a production system, but they certainly demonstrate just what awesome looking stuff you can build to run on beam. I don't know if these compile down to erlang modules or have some kind of runtime environment, but either approach is plausible. Personally I'd like Erlang to have a syntax like Haskell or OCaml, but I'm far too busy building functional stuff so I've learned to live with the existing syntax and now we're old friends. > I assume a best practice is to minimize the use of modules written in other > languages. What exactly would the scenario be for a C module that segfaults? > I assume it would take down the VM. > > Depends on how it's been integrated. There are four ways to integrate C/C++ code today, two of which will not bring down the VM and one of which that will. They are 1. C Nodes 2. Port Programs 3. Linked-In Drivers 4. Native Implemented Functions I won't go into all the details here, as the documentation for these is "quite good" and the good folk on this list are very helpful when you have specific questions. To boil it down to the essentials, a C node is an external program that "looks like" a regular Erlang node and communicates with it's peers using the distribution protocol. If it crashes, it needs to be restarted but nobody else is hit. A Port Program is also an external program, which the emulator talks to using pipes/stdio. If it crashes, nothing else does and it can be restarted. Linked-in drivers and NIFs are shared libraries, loaded into the beam process address space and sharing data and flow control with the emulator itself. If these segfault, they will indeed bring down the whole beam process. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lfredlund@REDACTED Tue Oct 18 11:28:16 2011 From: lfredlund@REDACTED (=?ISO-8859-1?Q?Lars-=C5ke_Fredlund?=) Date: Tue, 18 Oct 2011 11:28:16 +0200 Subject: [erlang-questions] [ANN] JavaErlang, a library for easing the interfacing of Erlang and Java using JInterface Message-ID: <4E9D46B0.6080804@fi.upm.es> This is an announcement for JavaErlang, a small library which tries to reduce the effort of interfacing Erlang with Java code using JInterface. Normally the library removes the need to write special Java code for communicating with an Erlang node. We use the library here at UPM to test Java code written by students using Erlang QuickCheck (a small example for how to do this is included as an example). The code is available at git@REDACTED:fredlund/JavaErlang.git A web page with the generated edoc documentation is at http://babel.ls.fi.upm.es/~fred/JavaErlang/ /Lars-Ake Fredlund From attila.r.nohl@REDACTED Tue Oct 18 11:39:15 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Tue, 18 Oct 2011 11:39:15 +0200 Subject: [erlang-questions] Reading from private ets tables? Message-ID: Hello! Is there a way to read data from private ets tables? I have to debug mnesia, but the interesting tables (e.g. mnesia_held_locks) are private, so I can't find whose locks are deadlocking. From erlang@REDACTED Tue Oct 18 11:47:10 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 18 Oct 2011 11:47:10 +0200 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: <4E9D39A1.2050008@gmail.com> References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> <4E9D39A1.2050008@gmail.com> Message-ID: On Tue, Oct 18, 2011 at 10:32 AM, CGS wrote: > Hi, > > Interesting discussion here and I would like few points (as C is also my > favorite programming language). > > The main point for entering the discussion is the fact that many who replied > here were saying that if something crashes in your application doesn't kill > your application like in C. I beg to differ on this point. If your Erlang > application is single threaded, then your application (I do not speak here > about OTP application) will crash if an error occurs there. If you construct > your C application like an OTP application, then that means your C > applications is multithreaded, so, if one thread dies, it doesn't crash your > main application (as well as in OTP application, if one gen_server crashes > the whole application doesn't crash). Right but in order to "construct your C application like an OTP application" means you will have to re-invent 50% (at a conservative estimate) of the Erlang run-time. The deep problem is not making your C application mutithreaded - it's making it fault-resilient. Mutithreaded C implies some kind of data exchange between different C threads - now there are many ways to do this - the cleanest alternative is by copying message passing. The dirtiest - is by shared memory and mutexes. Shared memory and mutexes is not where you want to go (willingly). The deep problem has a deeper problem hiding inside it - memory fragmentation and reallocation of pointers. This implies some kind of GC or some kind of static analysis that is beyond the brain of man or beast to comprehend. So what do you do? - you decide to not share memory and let each process have it's own stack and heap and you copy everything. Now you've invented 15% of Erlang and are approximately where we were in 1986. The tricky bits are deep under the surface - addressing memory fragmentation and failure. Now simple things don't need all this stuff - so if you want to write an embedded washing machine controller write it in C. But if you want to write a few millions of lines of complex code don't write it in C. Write the 5% of messy stuff that needs to be fast in C and write the other 95% in Erlang. (( This is exactly what Dennis Richie (RIP) said about C and assembler only with the 5% in assembler)) The productivity gain comes from the fact that it is possible to write 95% of a system in {Erlang, C} and 5% in {C, Assembler} Just as in the early days of C some systems should be written entirely in Assembler, today some systems should be written entirely in C. What Erlang does is move the goal-posts. If you're only writing small systems - you won't notice this and C is fine. But if you want to write systems that run forever and have a large number of lines of code use Erlang. Try to write it all in Erlang, if this fails write selected parts in C. This is actually what you would do in C. Write it all in C and then if this fails write selected parts in assembler. All we've done is move the goal posts. Cheers /Joe > > Now, Gerry, think Erlang as being multithread application in C where you > don't need to care too much about how to construct the thread. That means, > the construction of each thread is much simpler and you already have the > monitor (watchdog, if you prefer) part implemented, so, you can focus on > what you want to develop and less on the tools you need. > > There are few more aspects here you may want to consider before choosing in > which language you want to write your code. > > Firstly, the network applications. In C, due to the lack of standards, you > have to know the compiler in which you want to program a port listener (as > example). In Erlang, you get the standard which is missing in C. Therefore, > the productivity is higher in case of Erlang in this case. > > Staying in the field of networking, some may argue that Erlang is not the > only one to be productive here. And that is correct if you consider JAVA (as > one of the main languages capable of creating cross-platform network > applications). Still, I would prefer Erlang for its VM and basic concepts, > and, as C programmer, I think you would understand me why I prefer each > thread to have its own memory (just for example). > > Secondly, as desktop application, generally, Erlang is far worse choice than > C. Still, if you don't care too much of the speed of your application, > Erlang offers some nice tools for increasing your productivity as > programmer. Also, due to its capability of interacting with external > functions written in C, you can create your own fast libraries which you can > connect them to Erlang (here you will spend a little bit of time to > understand the concept behind). > > Thirdly, concerning the application design. If in C you need to design your > application from A to Z, in Erlang you have tools which limit your design, > but it can increase your overall productivity. Actually, I consider OTP > design as being one of the most useful for general purpose. > > I will stop here with the list. You will discover many other advantages > using Erlang during making your own experience with Erlang. I am no expert > in Erlang and many things I am still discovering while programming in this > language. And, for you to see that I bumped into the same problems, I had > also some headaches with the line separators (many times I got compiling > errors due to them). But, in the end, those separators turned out to be more > useful than pain in the... Just for example, a loop in C when you don't want > to use the for statement is written like this (take it as principle and not > as something to be compiled): > > myfunc(myvars) > { > ?exit_condition; > the_loop_body; > myfunc(modified_vars); > } > > while in Erlang, it is looks like that: > > myfunc(myvars) -> > the_loop_body, > myfunct(modified_vars); > > myfunc(stop_vars) -> > end_of_loop. > > As you can see it's not so different, but you have to implement explicitly > the end of the loop. Quite a pain you may say. Actually, using this method, > think of having two functions with the same name, but with the language > choosing which one to be used depending on what arguments you pass to it. > Doesn't it sound a bit familiar? Yes. And more than that, you know the > advantage of this in practical applications. So, "two rabbits with a single > gun shot" (as there is a saying in my country). > > OK! One can speak whole day long about the advantages and disadvantages of > Erlang, but the most important is for you to make your own experience. So > far, my experience with Erlang was positive and I found myself liking Erlang > quite some. I wish your experience with Erlang to be as good as mine. Just > don't forget each language has its own beauty, but you need to know from > which angle you should look at it. > > Cheers, > CGS > > > > > On 10/18/2011 02:17 AM, Gerry Weaver wrote: >> >> Hi, >> >> Wow! I really appreciate your feedback. I find your responses very logical >> and helpful ;-) >> >> Another issue that I've started to think about is deployment. There seem >> to be several competing tools/utilities for this. Is there a preferred or >> recommended solution? >> >> >> Is the Erlang language itself still evolving? The reason I ask is that the >> fellow that did CouchDB posted a blog entry about the things he encountered >> while developing CouchDB. What was interesting to me is how, even after a >> short time with the language, many of the issues he mentioned immediately >> rang true with me. For example, what is the benefit of the "; , ." >> terminators as opposed to a single terminator approach? >> >> >> I don't mean to sound negative here. I'm just poking around a bit. I find >> Erlang quite compelling overall. I'm digging into the OTP at this point and >> there are some really interesting capabilities there. >> >> I assume a best practice is to minimize the use of modules written in >> other languages. What exactly would the scenario be for a C module that >> segfaults? I assume it would take down the VM. >> >> >> >> >> Thanks, >> -G >> >> >> >> >> >> >> >> >> >> -----Original Message----- >>> >>> From: "Jachym Holecek" >>> To: "Gerry Weaver" >>> Cc: erlang-questions@REDACTED >>> Date: 10/17/11 17:41 >>> Subject: Re: [erlang-questions] Erlang newbie questions >>> >>> # Gerry Weaver 2011-10-17: >>>> >>>> I am new to Erlang and I'm trying to make the case to continue learning >>>> this language. I have read all of the books available and seen several >>>> presentations on the language. I am interested in Erlang primarily for >>>> the distributed and fault tolerant features it offers. >>> >>> And you won't be let down on that -- but it takes a while to appreciate >>> the higher-level engineering aspects of OTP, so don't give up too early. >>> Given your good theoretical preparations, I'd suggest the next good step >>> would be to write a non-trivial toy project in Erlang -- after all, it's >>> a hard fact of nature that actually doing it is the only way to ever >>> learn anything. >>> >>>> However, I have >>>> some reservations about how useful this really is. For example, if you >>>> have a network server and the listener process fails, how do you >>>> recover? >>> >>> The bright idea is that Erlang/OTP provides you with very natural tools >>> to address these situations (impenetrable process isolation being the >>> basis, but you have timeouts for free, links, monitors, etc), but the >>> exact strategy is inherently application-specific. Popular choice for >>> dealing with failed communication link would be to schedule reconnect >>> attempt and revert to hot-spare meanwhile. I suppose MIT X11's mantra, >>> "mechanism, not policy", applies nicely here (regarding base Erlang/OTP). >>> >>>> It seems that the fault tolerance is only applicable in the VM context. >>> >>> Fair observation, it's actually quite reasonable to think of one BEAM >>> process as an operating system instance of its own -- it happens to run >>> on top of some UN*X hypervisor typically, and you'd usually arrange for >>> the BEAM to be restarted on critical failure (much like you'd configure >>> your OS to automatically restart on kernel panic). Does this make sense? >>> >>>> This is, in my mind equivalent to any other application process. If I >>>> have a network server written in C, I can have a watchdog process to >>>> kill >>>> and/or restart it in the event of failure. The C based approach actually >>>> seems more robust. In this scenario, how would one restart the Erlang >>>> VM? >>> >>> A number of choices -- I know Solaris SMF works, I imagine DJB's >>> supervise >>> would work too (as exotic as djbware is...). One thing to emphasize here >>> is that crashing BEAM means one of two things: 1) you didn't provision >>> for >>> the right kind of flow-control (or rate throttling or ...) mechanisms to >>> ensure your node fits applicable resource limits -- for this you can only >>> blame yourself, or 2) you somehow managed to provoke a drastic error in >>> BEAM (very very very rare, but not unheard of). Either case is really >>> akin to an OS crash -- a catastrophic, worst-case scenario. Of course, >>> other nodes talking to the failed one can deal with it gracefully. >>> >>>> It has been my experience that network servers most often fail due to OS >>>> or hardware limits rather than bugs in the code >>> >>> See above, hitting OS/HW limits is just engineer's failure to exert due >>> diligence to fit within those limits. Hardware failure or operator error >>> however I couldn't argue with. Also, see below on completely different >>> scales of software integration you'll meet in Erlang -- that increases >>> the likelihood of programming error wanting to spoil the party. >>> >>>> (at least for thoroughly tested production ready code). >>> >>> [Must suppress the urge to make cynical comments.] >>> >>>> When you factor in the amount of baggage >>>> that comes with Erlang, or any other VM based language, it's difficult >>>> to justify. >>> >>> I can understand this prejudice, other way to see this is that, much like >>> an OS, the BEAM is part of your TCB. Any error therein is fatal and you >>> can't do much about it. One of the services it provides you however is >>> blank cheque on safety one Erlang process has regardless, for one, of >>> what other Erlang processes do, but for other regardless of what the >>> code running in that process does (is allowed to do). The guarantess >>> given to a UN*X process by OS kernel seem rather lame in comparison. >>> In my mind, this is definitely worth it. >>> >>>> Now there is also the rapid development aspect, but I'm >>>> finding that at least for now, the time savings is being eaten up by >>>> looking for documentation. I understand that this situation will improve >>>> over time, but the various posts I've seen from folks with more >>>> experience >>>> seem to indicate that this will take quite a while. I like the language >>>> (except the , ; . thing). >>> >>> Yep, there's some learning curve to deal with, but the closer it starts >>> resembling a constant function, the more unbelievable will your >>> productivity >>> be -- rolling production-ready solutions once every three weeks is not >>> complete science-fiction :-). One thing I wanted to mention, and this >>> is rather random place to do so, is that besides the other advantages >>> of Erlang/OTP you've already mentioned (or the books you've read surely >>> did) there's also one people don't talk about very often: high-level >>> compositionality of sorts -- not at the level of functions or modules, >>> but whole applications (or application stacks). You have an existing >>> solution and suddenly are tasked with adding Diameter interface for >>> charging, a web-based GUI for Operations and SNMP monitoring interface >>> all at once? Big deal, just add a few protocol stacks to your boot >>> file, write a few glue-modules exporting existing information to them, >>> and you're good to go. >>> >>>> You can do some pretty cool things with it, >>>> but when I got over that initial gee whiz, I had some trouble seeing a >>>> real world production use case. >>> >>> I'd choose Erlang/OTP for anything that requires complex control logic >>> or gluing together of many networked systems. I'm a bit too tired to >>> give specific examples at the moment, can fill in tomorrow and others >>> will surely be eager to provide their own if you're interested. >>> >>>> I'm not trying to be critical, I just >>>> figure I must have missed something along the way. Any perspectives or >>>> advice on this would be greatly appreciated. >>> >>> That's fine, coming from embedded systems background (the land of C and >>> asm) I can understand your skepticism... (I know I wouldn't have believed >>> nice words until I 1) was open-minded enough to listen 2) went on writing >>> Erlang code for a while). >>> >>> Hope this slightly wine-infused mail was helpful, or at least completely >>> harmless. :-) >>> >>> BR, >>> ? ? ?-- Jachym >>> >>> >>> >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ulf.wiger@REDACTED Tue Oct 18 11:52:26 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 18 Oct 2011 11:52:26 +0200 Subject: [erlang-questions] Reading from private ets tables? In-Reply-To: References: Message-ID: <9D983A94-16F2-43E7-BE95-CFAA7194C776@erlang-solutions.com> No, you can't read private ets tables (owned by some other process). However, in this particular case, mnesia does offer some functions that might be of help. mnesia:info() prints quite a lot of information, including info about held locks. Another function: mnesia_locker:get_held_locks(): 3> mnesia_locker:get_held_locks(). [] 4> mnesia:create_table(t, []). {atomic,ok} 5> spawn(fun() -> mnesia:transaction(fun() -> mnesia:write({t,1,a}), timer:sleep(30000) end) end). <0.55.0> 6> mnesia_locker:get_held_locks(). [{{t,1},write,{tid,4,<0.55.0>}}] ... 7> mnesia_locker:get_held_locks(). [] Finally, mnesia_lib:dist_coredump() is a big hammer, when all else fails. BR, Ulf W On 18 Oct 2011, at 11:39, Attila Rajmund Nohl wrote: > Hello! > > Is there a way to read data from private ets tables? I have to debug > mnesia, but the interesting tables (e.g. mnesia_held_locks) are > private, so I can't find whose locks are deadlocking. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From ulf.wiger@REDACTED Tue Oct 18 11:53:43 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 18 Oct 2011 11:53:43 +0200 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> <4E9D39A1.2050008@gmail.com> Message-ID: <34550D66-70EC-49BF-AAD8-29703C84C688@erlang-solutions.com> On 18 Oct 2011, at 11:47, Joe Armstrong wrote: > Mutithreaded C implies some kind of data exchange between > different C threads - now there are many ways to do this - the > cleanest alternative > is by copying message passing. The dirtiest - is by shared memory and mutexes. I beg to differ. The dirtiest way is by shared memory *without* mutexes. ;-) BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Tue Oct 18 12:06:32 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 18 Oct 2011 12:06:32 +0200 Subject: [erlang-questions] in-flight module transforms - request for comments Message-ID: <40EF3227-F8CA-472C-99DF-22195B9C3A01@erlang-solutions.com> From those of you who enjoy modifying existing code at run-time, I would like to ask for some input. Inspired by Joe Norton's use of Meck to introduce support for asciiedoc syntax in EDoc [1], I started playing with Meck to do similar things with e.g. tweaking epp on the fly for alternative syntax support. I found that Meck was a bit limited in this regard, and ultimately, it's a slight abuse of the tool anyway. Since I maintain the parse_trans application [2], I thought I'd continue my experimentation there (in the module-transforms branch). First, I copied meck_mod.erl (into parse_trans_mod.erl) and made a slight addition to it: parse_trans_mod:transform_module(Module, Transforms, Options) -> CompileAndLoadResult which fetches the abstract code of a module, transforms it, then compiles and loads the result. I then added two helper functions (replace_function/3 and export_function/3), for starters, to help make easy transforms. Here's an example [3]: -module(test_transform_mod). -export([ex1/0]). -include("codegen.hrl"). ex1() -> parse_trans_mod:transform_module( ex1, [fun(Fs, _Os) -> parse_trans:export_function(int, 0, Fs) end, fun transform_ex1/2], [{pt_pp_src,true}]). transform_ex1(Forms, _Opts) -> NewF = codegen:gen_function(add, fun(A, B) -> A - B end), parse_trans:replace_function(add, 2, NewF, Forms). A shell dialogue to illustrate: Eshell V5.8.4 (abort with ^G) 1> ex1:add(5,3). 8 2> test_transform_mod:ex1(). Pretty-printed in "./ex1.xfm" ok 3> ex1:add(5,3). 2 This changes the example module ex1 [4], by exporting a previously internal function, int/0, and changing the semantics of the add/2 function. The idea is hardly new. Yariv Sadan did something similar with smerl. I wanted to add a few functions that would draw on the stuff already supported by parse_trans. For those of you out there who have used similar things before, what kind of support would you like to see? BR, Ulf W [1] https://github.com/norton/asciiedoc/blob/master/src/asciiedoc_lib.erl#L39 [2] http://github.com/esl/parse_trans [3] https://github.com/esl/parse_trans/blob/module-transforms/examples/test_transform_mod.erl [4] In ex1, add(X, Y) -> X+Y. After transformation, add(A, B) -> A - B. Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From huss01@REDACTED Tue Oct 18 12:07:40 2011 From: huss01@REDACTED (=?ISO-8859-1?Q?H=E5kan_Huss?=) Date: Tue, 18 Oct 2011 12:07:40 +0200 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> Message-ID: On Tue, Oct 18, 2011 at 11:07, Tim Watson wrote: >> >> Is the Erlang language itself still evolving? The reason I ask is that the >> fellow that did CouchDB posted a blog entry about the things he encountered >> while developing CouchDB. What was interesting to me is how, even after a >> short time with the language, many of the issues he mentioned immediately >> rang true with me. For example, what is the benefit of the "; , ." >> terminators as opposed to a single terminator approach? >> > > Whilst the core syntax of the language is unlikely to change in the near > future, many new language features are appearing all the time! The "; , ." > thing got me for a little while, as coming from Python and OCaml I was used > to indentation and block termination keywords (e.g., "end") and of course > the C-like languages that use only ";" everywhere. Well, saying that C-like languages use ";" everywhere is a slight over-simplification. The C standard lists the following punctuators: [ ] ( ) { } . -> ++ -- & * + - ~ ! / % << >> < > <= >= == != ^ | && || ? : ; ... = *= /= %= += -= <<= >>= &= ^= |= , # ## <: :> <% %> %: %:%: Even if we only look at expression/statement separators, the list would need to include ";" (which is a terminator in most cases, but a separator in for loop heads), "," (which is used as a separator, but can be used as a terminator in initializations of e.g., arrays), "?" and ":" (which are used as separators in a very specific case), "\n" (which is significant as a terminator in macro definitions) and of course {} (which are used for grouping except when they are actually mandated by syntax whether they group something or not, and which may or may not negate the need for a terminating ";"). To say that C-like languages have simple punctuation is wrong. (Believe me, I have taught C to students who have no experience with C-like languages.) You may be used to that punctuation so it feels natural to you, but simple it is not. With regards to Erlang I have taught Erlang to beginners as well, so I know that it is confusing in Erlang as well. But I do not feel it is harder to learn in Erlang than in C-like languages. Regards, /H?kan From erlang@REDACTED Tue Oct 18 12:20:01 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 18 Oct 2011 12:20:01 +0200 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: <34550D66-70EC-49BF-AAD8-29703C84C688@erlang-solutions.com> References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> <4E9D39A1.2050008@gmail.com> <34550D66-70EC-49BF-AAD8-29703C84C688@erlang-solutions.com> Message-ID: On Tue, Oct 18, 2011 at 11:53 AM, Ulf Wiger wrote: > > On 18 Oct 2011, at 11:47, Joe Armstrong wrote: > > Mutithreaded C implies some kind of data exchange between > different C threads - now there are many ways to do this - the > cleanest alternative > is by copying message passing. The dirtiest - is by shared memory and > mutexes. > > I beg to differ. The dirtiest way is by shared memory *without* mutexes. You mean ... I didn't know that was possible - I though emacs detected this and you were instantly fired and your house set on fire, and your bones melted down for glue and so on ... /J > ?;-) > BR, > Ulf W > Ulf Wiger,?CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > From ulf.wiger@REDACTED Tue Oct 18 12:26:14 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 18 Oct 2011 12:26:14 +0200 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> <4E9D39A1.2050008@gmail.com> <34550D66-70EC-49BF-AAD8-29703C84C688@erlang-solutions.com> Message-ID: <2D74FD07-C2A5-491C-A102-1690AE10D4F2@erlang-solutions.com> On 18 Oct 2011, at 12:20, Joe Armstrong wrote: > On Tue, Oct 18, 2011 at 11:53 AM, Ulf Wiger > wrote: >> >> On 18 Oct 2011, at 11:47, Joe Armstrong wrote: >> >> Mutithreaded C implies some kind of data exchange between >> different C threads - now there are many ways to do this - the >> cleanest alternative >> is by copying message passing. The dirtiest - is by shared memory and >> mutexes. >> >> I beg to differ. The dirtiest way is by shared memory *without* mutexes. > > You mean ... I didn't know that was possible - I though emacs detected > this and you were instantly > fired and your house set on fire, and your bones melted down for glue > and so on ? Like I said? dirty. I recall an admonition from a seminar on programming for multicore: "You cannot use priorities for mutual exclusion!" Presumably, this technique is commonly used, since it was deemed necessary to point out especially that this is not safe on a multicore machine. http://www.chibios.org/dokuwiki/doku.php?id=chibios:guides:mutual_exclusion_guide#priority_boost BR, Ulf Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Tue Oct 18 12:47:48 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 18 Oct 2011 11:47:48 +0100 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> Message-ID: On 18 October 2011 11:07, H?kan Huss wrote: > On Tue, Oct 18, 2011 at 11:07, Tim Watson > wrote: > >> > >> Is the Erlang language itself still evolving? The reason I ask is that > the > >> fellow that did CouchDB posted a blog entry about the things he > encountered > >> while developing CouchDB. What was interesting to me is how, even after > a > >> short time with the language, many of the issues he mentioned > immediately > >> rang true with me. For example, what is the benefit of the "; , ." > >> terminators as opposed to a single terminator approach? > >> > > > > Whilst the core syntax of the language is unlikely to change in the near > > future, many new language features are appearing all the time! The "; , > ." > > thing got me for a little while, as coming from Python and OCaml I was > used > > to indentation and block termination keywords (e.g., "end") and of course > > the C-like languages that use only ";" everywhere. > > Well, saying that C-like languages use ";" everywhere is a slight > over-simplification. The C standard lists the following punctuators: > > [ ] ( ) { } . -> > ++ -- & * + - ~ ! > / % << >> < > <= >= == != ^ | && || > ? : ; ... > = *= /= %= += -= <<= >>= &= ^= |= > , # ## > <: :> <% %> %: %:%: > > Even if we only look at expression/statement separators, the list > would need to include ";" (which is a terminator in most cases, but a > separator in for loop heads), "," (which is used as a separator, but > can be used as a terminator in initializations of e.g., arrays), "?" > and ":" (which are used as separators in a very specific case), "\n" > (which is significant as a terminator in macro definitions) and of > course {} (which are used for grouping except when they are actually > mandated by syntax whether they group something or not, and which may > or may not negate the need for a terminating ";"). > > To say that C-like languages have simple punctuation is wrong. > (Believe me, I have taught C to students who have no experience with > C-like languages.) You may be used to that punctuation so it feels > natural to you, but simple it is not. > > With regards to Erlang I have taught Erlang to beginners as well, so I > know that it is confusing in Erlang as well. But I do not feel it is > harder to learn in Erlang than in C-like languages. > > I totally agree, it is not simple in C but as you say, very familiar. I wasn't trying to say that Erlang syntax is hard, just unfamiliar but once I learned it I actually now find it totally intuitive. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Tue Oct 18 13:12:40 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 18 Oct 2011 13:12:40 +0200 Subject: [erlang-questions] in-flight module transforms - request for comments In-Reply-To: <40EF3227-F8CA-472C-99DF-22195B9C3A01@erlang-solutions.com> References: <40EF3227-F8CA-472C-99DF-22195B9C3A01@erlang-solutions.com> Message-ID: <71777264-E128-4463-87EA-E0ADE1933EF1@erlang-solutions.com> On 18 Oct 2011, at 12:06, Ulf Wiger wrote: > From those of you who enjoy modifying existing code at run-time, I would like to ask for some input. > > Inspired by Joe Norton's use of Meck to introduce support for asciiedoc syntax in EDoc [1], I started playing with Meck to do similar things with e.g. tweaking epp on the fly for alternative syntax support. So, just to illustrate what tweaking epp could look like: Say I want to abbreviate the Erlang fun ? end syntax, using something shorter: -module(toker_test). -export([double/1, i2l/1]). double(L) -> lists:map(`(X) -> X*2`, L). i2l(L) -> lists:map(`integer_to_list/1, L). A sample shell dialogue: Eshell V5.8.4 (abort with ^G) 1> c(toker_test,[{outdir,"../ebin"}]). ./toker_test.erl:5: syntax error before: '`' ./toker_test.erl:8: syntax error before: '`' ./toker_test.erl:2: function double/1 undefined ./toker_test.erl:2: function i2l/1 undefined error 2> c(toker_c,[{outdir,"../ebin"}]). {ok,toker_c} 3> toker_c:c(toker_test,[{outdir,"../ebin"}]). {ok,toker_test} 4> toker_test:double([1,2,3]). [2,4,6] 5> toker_test:i2l([1,2,3]). ["1","2","3"] The actual tweak is almost exactly like the example I showed earlier, but replacing epp:parse_erl_form/1 with a version that calls my own, modified version of erl_parse. The other thing needed is to export epp:epp_request/2. Not that I'm recommending doing this for real! ;-) BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From watson.timothy@REDACTED Tue Oct 18 13:14:58 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 18 Oct 2011 12:14:58 +0100 Subject: [erlang-questions] in-flight module transforms - request for comments In-Reply-To: <40EF3227-F8CA-472C-99DF-22195B9C3A01@erlang-solutions.com> References: <40EF3227-F8CA-472C-99DF-22195B9C3A01@erlang-solutions.com> Message-ID: On 18 October 2011 11:06, Ulf Wiger wrote: > For those of you out there who have used similar things before, what kind > of support would you like to see? > I'm interesting in writing up some simple code instrumentation tools so I can do stuff like declaratively inserting performance counters and/or logging statements into my code. I've already used parse_trans and love it, but doing this without forcing users into using a parse_transform sounds really good - a post compile step provided by a rebar plugin could transform the beam code afterwards, or as you say this can even be done at runtime. One thing that would be useful would be some kind of qlc-esque selection mechanism for finding code that matches certain conditions. Whether these are match specifications or something else, they would make a very useful addition. Matching on type/spec information would be even better, although I suspect that's a little outside of the scope you're looking at right now. Cheers, Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Tue Oct 18 13:17:07 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 18 Oct 2011 12:17:07 +0100 Subject: [erlang-questions] in-flight module transforms - request for comments In-Reply-To: <71777264-E128-4463-87EA-E0ADE1933EF1@erlang-solutions.com> References: <40EF3227-F8CA-472C-99DF-22195B9C3A01@erlang-solutions.com> <71777264-E128-4463-87EA-E0ADE1933EF1@erlang-solutions.com> Message-ID: On 18 October 2011 12:12, Ulf Wiger wrote: > The actual tweak is almost exactly like the example I showed earlier, but > replacing epp:parse_erl_form/1 with a version that calls my own, modified > version of erl_parse. The other thing needed is to export epp:epp_request/2. > > Not that I'm recommending doing this for real! ;-) > > Actually I think that making epp a bit more extensible would be a really superb thing. When will we be able to do quai-quotations eh!? :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgsmcmlxxv@REDACTED Tue Oct 18 13:29:39 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Tue, 18 Oct 2011 13:29:39 +0200 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: <2D74FD07-C2A5-491C-A102-1690AE10D4F2@erlang-solutions.com> References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> <4E9D39A1.2050008@gmail.com> <34550D66-70EC-49BF-AAD8-29703C84C688@erlang-solutions.com> <2D74FD07-C2A5-491C-A102-1690AE10D4F2@erlang-solutions.com> Message-ID: <4E9D6323.8000402@gmail.com> Thanks, Joe and Ulf for those comments. Yes, I was missing that part in expressing my point. In my post, there were just two simple points I wanted to pass further: 1. one cannot compare a single thread application written in C with multithread application in Erlang (if you do want to compare, the comparison should go from multithread Erlang application to multithread C application); 2. as Joe simplified and I was also mentioning that, one doesn't need to reinvent Erlang/OTP when you have it already. Joe, I met cases when those 5% meant a lot (especially in the long time consuming physics simulations). In those cases, I would still prefer C to Erlang. But, as I said, each language has its own beauty and you need to look at it from the correct angle to make the best use of it. I would still write a game AI in C and not in Erlang or JavaScript (where there is an attempt based on jQuery). Maybe it's about me, but I still consider C as being more suitable for that task. On the other hand, as I said it before, I wouldn't start writing a network application in C when Erlang provides such simple ways to do it. But these are just examples and there is much more in each of them. So, it's a matter of what you need and where each of the languages shows more beauty in approaching the problem. I wouldn't just discard C yet, but I wouldn't deny the productivity of Erlang. It's the same case with all the languages: they came as a need to cover different sectors (and I can give a lot of examples, but you already know them). Cheers, CGS On 10/18/2011 12:26 PM, Ulf Wiger wrote: > > On 18 Oct 2011, at 12:20, Joe Armstrong wrote: > >> On Tue, Oct 18, 2011 at 11:53 AM, Ulf Wiger >> > > wrote: >>> >>> On 18 Oct 2011, at 11:47, Joe Armstrong wrote: >>> >>> Mutithreaded C implies some kind of data exchange between >>> different C threads - now there are many ways to do this - the >>> cleanest alternative >>> is by copying message passing. The dirtiest - is by shared memory and >>> mutexes. >>> >>> I beg to differ. The dirtiest way is by shared memory *without* mutexes. >> >> You mean ... I didn't know that was possible - I though emacs detected >> this and you were instantly >> fired and your house set on fire, and your bones melted down for glue >> and so on ? > > Like I said? dirty. > > I recall an admonition from a seminar on programming for multicore: > > "You cannot use priorities for mutual exclusion!" > > Presumably, this technique is commonly used, since it was deemed > necessary to point out especially that this is not safe on a multicore > machine. > > http://www.chibios.org/dokuwiki/doku.php?id=chibios:guides:mutual_exclusion_guide#priority_boost > > BR, > Ulf > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From textonpc@REDACTED Tue Oct 18 14:25:46 2011 From: textonpc@REDACTED (TexTonPC) Date: Tue, 18 Oct 2011 14:25:46 +0200 Subject: [erlang-questions] Mnesia Fragmentation, duplicated records after rehashing Message-ID: Hi, we are encountering a strange scenario using mnesia fragmentation in our production system: our cluster had around 20 tables spread over 8 mnesia nodes each running on a single server, totalling 1024 frags per table (128 frags per node). Now we added 8 new machines to the cloud, and started the rehashing process by adding other 128 frags per table on each new node. I started this process from a different host in the cluster (lot of free ram space) attached to the mnesia cluster calling mnesia:change_table_frag(Table, {add_frag, [NewNode]}) for each table in order to have 2048 frags per table spread over 16 nodes. 1. The adding_fragments process took a week to rehash all the table records while working on a single core of this "maintenance" node. I read on the mnesia docs and on this list that this kind of op locks the involved table, but I was not able to parallelize on the different tables (parallel processes each running add_frag on different table) in order to take advantage of multiple cores. I had the feeling that add_frag "locks" the entire mnesia transaction manager. Any perspectives or advice on this would be greatly appreciated. 2. At the end of the frags-creation and rehashing process I noticed some size unbalancing between old and new frags so I started a consistency scanner that simply takes each record on each fragment and ensures that the mnesia_frag hashing module actually maps that record on that specific fragment. It turns out that the unbalanced frags have some records that were moved to the new destination frag during the rehashing process, but were not removed from the old source frag! I thought mnesia:change_table_frag(Table, {add_frag, [NewNode]}) was running in atomic transaction context, has anyone ever faced with something like this? Thank you -- textonpc@REDACTED atessaro@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Tue Oct 18 16:27:34 2011 From: g@REDACTED (Garrett Smith) Date: Tue, 18 Oct 2011 09:27:34 -0500 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: <4E9D39A1.2050008@gmail.com> References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> <4E9D39A1.2050008@gmail.com> Message-ID: Hi CGS, On Tue, Oct 18, 2011 at 3:32 AM, CGS wrote: > > The main point for entering the discussion is the fact that many who replied > here were saying that if something crashes in your application doesn't kill > your application like in C. I beg to differ on this point. If your Erlang > application is single threaded, then your application (I do not speak here > about OTP application) will crash if an error occurs there. I'm inclined to argue that a "single threaded" application isn't a valid Erlang app :) I'll be slightly pedantic here -- just because I think it's worth mentioning... * When you run "erl" you start an OTP facility -- it's not an optional feature; it's fundamental to an Erlang system * A "single threaded" app would presumably be an app that's run using "erl" with -s or -eval. That'll work for running an application, but it's not the right way :) * The right way to run an Erlang app is to create an OTP application and include that app in a release, which is specified when you run "erl" with the -boot option If you run your Erlang application this way, you get all the benefits people have been describing. This is admittedly a bit of a bugaboo for me, but I think the dichotomy that people draw between "Erlang" and "OTP" is overall not helpful. The so called OTP patterns that are treated as optional/advanced are fundamental to the way the Erlang runtime is designed. So I tend to blur the two together into a single "Erlang" :) I think with some improvements in tooling and documentation, the headaches associated with using "OTP" will go away, which will be a huge boon for developers -- especially those new to Erlang. Garrett From cgsmcmlxxv@REDACTED Tue Oct 18 17:51:38 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Tue, 18 Oct 2011 17:51:38 +0200 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> <4E9D39A1.2050008@gmail.com> Message-ID: <4E9DA08A.8050804@gmail.com> Hi Garrett, I think you misunderstood me. I specified that "Erlang application" does not refer to "OTP application". It's true that I haven't specified what exactly is "Erlang application" in my context, but at least I mentioned that it's not about "OTP application". Nevertheless, your remark is correct that "single threaded" application can barely be considered as Erlang application, but, can anyone say you are not able to create a single threaded "Erlang application" by the mean of pure application and not as OTP designed it? Just to give an example, I created a simple "application" which is not OTP and I am calling it from YAWS tag. The "application" (which consists in a single module with two functions) accepts entries, formats the entries and sends messages to a server (where I used the full OTP design). It is an application by definition (it can be used as independent entity - how you launch it does not matter here - and it has a specific task) even if it is few lines only. So, that's what I called "Erlang application" (disregarding what's behind gen_tcp). It is single threaded, so, here we are at my definition of "single threaded Erlang application". I hope I was clear now and I apologize if I created any confusion. If my post still generates confusions, please, let me know. Cheers, CGS On 10/18/2011 04:27 PM, Garrett Smith wrote: > Hi CGS, > > On Tue, Oct 18, 2011 at 3:32 AM, CGS wrote: >> The main point for entering the discussion is the fact that many who replied >> here were saying that if something crashes in your application doesn't kill >> your application like in C. I beg to differ on this point. If your Erlang >> application is single threaded, then your application (I do not speak here >> about OTP application) will crash if an error occurs there. > I'm inclined to argue that a "single threaded" application isn't a > valid Erlang app :) > > I'll be slightly pedantic here -- just because I think it's worth mentioning... > > * When you run "erl" you start an OTP facility -- it's not an optional > feature; it's fundamental to an Erlang system > > * A "single threaded" app would presumably be an app that's run using > "erl" with -s or -eval. That'll work for running an application, but > it's not the right way :) > > * The right way to run an Erlang app is to create an OTP application > and include that app in a release, which is specified when you run > "erl" with the -boot option > > If you run your Erlang application this way, you get all the benefits > people have been describing. > > This is admittedly a bit of a bugaboo for me, but I think the > dichotomy that people draw between "Erlang" and "OTP" is overall not > helpful. The so called OTP patterns that are treated as > optional/advanced are fundamental to the way the Erlang runtime is > designed. So I tend to blur the two together into a single "Erlang" :) > > I think with some improvements in tooling and documentation, the > headaches associated with using "OTP" will go away, which will be a > huge boon for developers -- especially those new to Erlang. > > Garrett From cgsmcmlxxv@REDACTED Tue Oct 18 18:13:13 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Tue, 18 Oct 2011 18:13:13 +0200 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: <4E9DA08A.8050804@gmail.com> References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> <4E9D39A1.2050008@gmail.com> <4E9DA08A.8050804@gmail.com> Message-ID: <4E9DA599.6000305@gmail.com> Oh, yes, there is another confusion I might have created with my post. I used "OTP application" to refer to behavior(application). Sorry for not defining everything. Cheers, CGS On 10/18/2011 05:51 PM, CGS wrote: > Hi Garrett, > > I think you misunderstood me. I specified that "Erlang application" > does not refer to "OTP application". It's true that I haven't > specified what exactly is "Erlang application" in my context, but at > least I mentioned that it's not about "OTP application". > > Nevertheless, your remark is correct that "single threaded" > application can barely be considered as Erlang application, but, can > anyone say you are not able to create a single threaded "Erlang > application" by the mean of pure application and not as OTP designed it? > > Just to give an example, I created a simple "application" which is not > OTP and I am calling it from YAWS tag. The "application" (which > consists in a single module with two functions) accepts entries, > formats the entries and sends messages to a server (where I used the > full OTP design). It is an application by definition (it can be used > as independent entity - how you launch it does not matter here - and > it has a specific task) even if it is few lines only. So, that's what > I called "Erlang application" (disregarding what's behind gen_tcp). It > is single threaded, so, here we are at my definition of "single > threaded Erlang application". > > I hope I was clear now and I apologize if I created any confusion. If > my post still generates confusions, please, let me know. > > Cheers, > CGS > > > > > On 10/18/2011 04:27 PM, Garrett Smith wrote: >> Hi CGS, >> >> On Tue, Oct 18, 2011 at 3:32 AM, CGS wrote: >>> The main point for entering the discussion is the fact that many who >>> replied >>> here were saying that if something crashes in your application >>> doesn't kill >>> your application like in C. I beg to differ on this point. If your >>> Erlang >>> application is single threaded, then your application (I do not >>> speak here >>> about OTP application) will crash if an error occurs there. >> I'm inclined to argue that a "single threaded" application isn't a >> valid Erlang app :) >> >> I'll be slightly pedantic here -- just because I think it's worth >> mentioning... >> >> * When you run "erl" you start an OTP facility -- it's not an optional >> feature; it's fundamental to an Erlang system >> >> * A "single threaded" app would presumably be an app that's run using >> "erl" with -s or -eval. That'll work for running an application, but >> it's not the right way :) >> >> * The right way to run an Erlang app is to create an OTP application >> and include that app in a release, which is specified when you run >> "erl" with the -boot option >> >> If you run your Erlang application this way, you get all the benefits >> people have been describing. >> >> This is admittedly a bit of a bugaboo for me, but I think the >> dichotomy that people draw between "Erlang" and "OTP" is overall not >> helpful. The so called OTP patterns that are treated as >> optional/advanced are fundamental to the way the Erlang runtime is >> designed. So I tend to blur the two together into a single "Erlang" :) >> >> I think with some improvements in tooling and documentation, the >> headaches associated with using "OTP" will go away, which will be a >> huge boon for developers -- especially those new to Erlang. >> >> Garrett > From g@REDACTED Tue Oct 18 18:44:51 2011 From: g@REDACTED (Garrett Smith) Date: Tue, 18 Oct 2011 11:44:51 -0500 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: <4E9DA08A.8050804@gmail.com> References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> <4E9D39A1.2050008@gmail.com> <4E9DA08A.8050804@gmail.com> Message-ID: On Tue, Oct 18, 2011 at 10:51 AM, CGS wrote: > I think you misunderstood me. I specified that "Erlang application" does not > refer to "OTP application". It's true that I haven't specified what exactly > is "Erlang application" in my context, but at least I mentioned that it's > not about "OTP application". Indeed, "application" is an overloaded term in this thread -- I was really more hijacking your comment to make a soap box speech. Technically not trolling, but dangerously close ;) From ehudez@REDACTED Tue Oct 18 19:36:35 2011 From: ehudez@REDACTED (Ehud Ezra) Date: Tue, 18 Oct 2011 19:36:35 +0200 Subject: [erlang-questions] Erlang on TILERA Message-ID: <000301cc8dbc$7fc1ff10$7f45fd30$@com> Hello, I am trying to install the Elrang on to Tilera using cross-compile, using the file Tilera has provided i have done the following: (1) cd $ERL_TOP -- ok (2) ./otp_build configure --xcomp-conf= (3) ./otp_build boot -a -> recieve many errors: cc1: error: invalid option argument '-OPT:Olimit=0' cc1: error: unrecognized command line option "-WOPT:lpre=off:spre=off:epre=off" gmake[3]: *** [obj/tile-tilera-linux-gnu/opt/smp/beam_emu.o] Error 1 gmake[3]: Leaving directory `/home/bgu_student/e4/otp_src_R14B04/erts/emulator' gmake[2]: *** [opt] Error 2 gmake[2]: Leaving directory `/home/bgu_student/e4/otp_src_R14B04/erts/emulator' gmake[1]: *** [smp] Error 2 gmake[1]: Leaving directory `/home/bgu_student/e4/otp_src_R14B04/erts' gmake: *** [emulator] Error 2 (4) ./otp_build release -a /lhome/x/otp -> also reicive same errors. (5) ./Install -cross -minimal /lhome/x/otp -> can't reach this step, only lib folder exists. any ideas? Ehud -------------- next part -------------- An HTML attachment was scrubbed... URL: From corticalcomputer@REDACTED Tue Oct 18 19:52:26 2011 From: corticalcomputer@REDACTED (G.S.) Date: Tue, 18 Oct 2011 13:52:26 -0400 Subject: [erlang-questions] Erlang on TILERA In-Reply-To: <000301cc8dbc$7fc1ff10$7f45fd30$@com> References: <000301cc8dbc$7fc1ff10$7f45fd30$@com> Message-ID: Hello Ezra, This is slightly offtopic, but I have for a while now been trying to find a place where to buy one of the Tilera boards. It seems that you have been able to procure one, do you know where I could buy one? and which board do you recommend? I'm sure that there are others on this mailing list that also would like to know where to buy one. Best regards, -Gene On Tue, Oct 18, 2011 at 1:36 PM, Ehud Ezra wrote: > Hello, > > > I am trying to install the Elrang on to Tilera using cross-compile, using > the file Tilera has provided i have done the following: > (1) cd $ERL_TOP -- *ok* > > (2) ./otp_build configure --xcomp-conf= erl-xcomp-TileraMDE2.0-tilepro.conf> > > (3) ./otp_build boot -a -> recieve many errors: > > cc1: error: invalid option argument ?-OPT:Olimit=0? > cc1: error: unrecognized command line option > "-WOPT:lpre=off:spre=off:epre=off" > gmake[3]: *** [obj/tile-tilera-linux-gnu/opt/smp/beam_emu.o] Error 1 > gmake[3]: Leaving directory > `/home/bgu_student/e4/otp_src_R14B04/erts/emulator' > gmake[2]: *** [opt] Error 2 > gmake[2]: Leaving directory > `/home/bgu_student/e4/otp_src_R14B04/erts/emulator' > gmake[1]: *** [smp] Error 2 > gmake[1]: Leaving directory `/home/bgu_student/e4/otp_src_R14B04/erts' > gmake: *** [emulator] Error 2 > > > (4) ./otp_build release -a /lhome/x/otp -> also reicive same errors. > > > (5) ./Install -cross -minimal /lhome/x/otp -> can't reach this step, only > lib folder exists. > > any ideas? > > Ehud > > **** > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Tue Oct 18 19:57:00 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 18 Oct 2011 18:57:00 +0100 Subject: [erlang-questions] dialyzer is driving me nuts Message-ID: <8CFE6241-B86F-41AA-A187-545C1743D57F@gmail.com> Let's just say I've been going batshit crazy since last week, managing to fix a single file to Dialyzer's liking. Consider the following https://gist.github.com/5744c70b2a6f918ddb56 The first message can be summarized as barrier_start.erl:50: The specification for barrier_start:barrier_start/3 states that the function might also return {'stop',#game{},tuple()} but the inferred return is {'continue',#game{{},_} | {'skip',#game{},_} but what about this clause in my code? Does Dialyzer think it will never be triggered? Why? barrier_start(Game = #game{}, Ctx, {'EXIT', Barrier, _}) when is_pid(Barrier), Barrier =:= Game#game.barrier -> gamelib:notify_start_game(Game), {stop, Game, Ctx}; The second message is a beauty! barrier_start.erl:74: The call gamelib:join(Game::#game{},Event::#pm_event{pm_message::#pm_join{}},1) does not have a term of type #game{} (with opaque subterms) as 1st argument What in the world does it mean? Help!!! -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From mononcqc@REDACTED Tue Oct 18 20:02:04 2011 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 18 Oct 2011 14:02:04 -0400 Subject: [erlang-questions] dialyzer is driving me nuts In-Reply-To: <8CFE6241-B86F-41AA-A187-545C1743D57F@gmail.com> References: <8CFE6241-B86F-41AA-A187-545C1743D57F@gmail.com> Message-ID: On Tue, Oct 18, 2011 at 1:57 PM, Joel Reymont wrote: > Let's just say I've been going batshit crazy since last week, managing to > fix a single file to Dialyzer's liking. > > Consider the following https://gist.github.com/5744c70b2a6f918ddb56 > > The first message can be summarized as > > barrier_start.erl:50: The specification for barrier_start:barrier_start/3 > states that the function might also return {'stop',#game{},tuple()} but the > inferred return is {'continue',#game{{},_} | {'skip',#game{},_} > > but what about this clause in my code? Does Dialyzer think it will never be > triggered? Why? > > barrier_start(Game = #game{}, Ctx, {'EXIT', Barrier, _}) > when is_pid(Barrier), > Barrier =:= Game#game.barrier -> > gamelib:notify_start_game(Game), > {stop, Game, Ctx}; > I think for that one the problem is that it infers it can find 'skip' but your type doesn't have it in there. You won't easily be able to add that clause without overloading contracts though. This is a wild guess. > > The second message is a beauty! > > barrier_start.erl:74: The call > gamelib:join(Game::#game{},Event::#pm_event{pm_message::#pm_join{}},1) does > not have a term of type #game{} (with opaque subterms) as 1st argument > I believe this means you'd be breaking the contract of gamelib:join/2 if it doesn't accept #game{} as a tuple in its own type spec, or that the #game{} it accepts doesn't have similar opaque terms inside of it. I haven't tried to fix it and see though. > > What in the world does it mean? > > Help!!! > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Tue Oct 18 20:07:42 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 18 Oct 2011 19:07:42 +0100 Subject: [erlang-questions] dialyzer is driving me nuts In-Reply-To: References: <8CFE6241-B86F-41AA-A187-545C1743D57F@gmail.com> Message-ID: On Oct 18, 2011, at 7:02 PM, Fred Hebert wrote: > I think for that one the problem is that it infers it can find 'skip' but your type doesn't have it in there. You won't easily be able to add that clause without overloading contracts though. How should this be fixed then? The message I get is that overloaded contracts are not supported. The problem of the catch-all clause in gen_servers, etc. is a generic one. How is it to be typed to please Dialyzer? > I believe this means you'd be breaking the contract of gamelib:join/2 if it doesn't accept #game{} as a tuple in its own type spec, or that the #game{} it accepts doesn't have similar opaque terms inside of it. I haven't tried to fix it and see though. This is gamelib:join, it does accept #game{} in its own type spec and otherwise. -spec join(#game{}, #pm_event{pm_message :: #pm_join{}}, seat_state()) -> #game{}. join(Game = #game{}, #pm_event{ player = Player, wsop_id = WSOPID, pm_message = #pm_join{} = R }, State) -> -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From joelr1@REDACTED Tue Oct 18 20:13:59 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 18 Oct 2011 19:13:59 +0100 Subject: [erlang-questions] dialyzer is driving me nuts In-Reply-To: References: <8CFE6241-B86F-41AA-A187-545C1743D57F@gmail.com> Message-ID: I attached another header file with a bunch of types to the original gist. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From clist@REDACTED Wed Oct 19 10:16:27 2011 From: clist@REDACTED (Angel J. Alvarez Miguel) Date: Wed, 19 Oct 2011 10:16:27 +0200 Subject: [erlang-questions] mini rebar recipe, (Was Re: escript and archives ) In-Reply-To: <201110181047.31133.clist@uah.es> References: <201110181047.31133.clist@uah.es> Message-ID: <201110191016.27150.clist@uah.es> as nobody care about this question, i post this just for newbies like me. I ended using rebar... i created a .src rebar file for my setup. walker.app.src {application, walker, [ {description, ""}, {vsn, "1"}, {registered, []}, {applications, [ kernel, stdlib ]}, {mod, { walker, []}}, {env, []} ]}. and then having my walker.erl and the getopt.erl filesin my src directory i did "rebar compile" and then "rebar escriptize", rebar cares of generating a walker.app file and then packages all the beam files and the app making a fresh escript file. So you have to make a app file to allow escript manage the zip file.. This is the result app file: {application,walker, [{description,[]}, {vsn,"1"}, {registered,[]}, {applications,[kernel,stdlib]}, {mod,{walker,[]}}, {env,[]}, {modules,[getopt,walker]}]}. Just for the record the walker.erl (a rip of ex1.erl from getopt) is : -module(walker). -export([main/1]). %% as of getopt examples option_spec_list() -> CurrentUser = case os:getenv("USER") of false -> "user"; User -> User end, [ {recursion, $r, "recurse", {boolean,false}, "Recurse over directory contents"}, {directory, undefined, undefined, string, "Target directory"} ]. main([]) -> getopt:usage(option_spec_list(), escript:script_name()); main(Args) -> OptSpecList = option_spec_list(), io:format("Args (~p) ~ngetopt:parse/2 returns:~n~n", [Args]), case getopt:parse(OptSpecList, Args) of {ok, {Options, NonOptArgs}} -> io:format("Options:~n ~p~n~nNon-option arguments:~n ~p~n", [Options, NonOptArgs]), case proplists:is_defined(directory,Options) of true -> do_walk(Options); false -> getopt:usage(OptSpecList, escript:script_name()) %% No target directory? end; {error, {Reason, Data}} -> io:format("Error: ~s ~p~n~n", [Reason, Data]), getopt:usage(OptSpecList, escript:script_name()) end. do_walk(Args) -> Files = filelib:fold_files(proplists:get_value(directory,Args), ".*", proplists:get_bool(recursion,Args), fun(Path, Acc) -> io:format("Procesando fichero ~s \n",[Path]), {ok, Bin} = file:read_file(Path), [{Path, Bin}|Acc] end, []). (A simple escript exercise for un cmdline tool that im planning to do) /Angel On Martes, 18 de Octubre de 2011 10:47:31 Angel J. Alvarez Miguel escribi?: > Hi, > > I want to packe some beams on a .ez archive and then insert the escript > header to get a selfcontaines script ala rebar.. > > i made a script witth a zip archive the usual way.. > > Files = filelib:fold_files("./ebin", > ".*", > true, > fun(Path, Acc) -> > io:format("Procesando fichero ~s \n",[Path]), > {ok, Bin} = file:read_file(Path), > [{Path, Bin}|Acc] end, > []), > {ok, {"mem", ZipBin}} = zip:create("mem", Files, [memory]), > Script = <<"#!/usr/bin/env escript\n%%! -noshell -noinput\n", > ZipBin/binary>>, > > ok = file:write_file("walker", Script), > os:cmd("chmod u+x walker"), > > > i have my ebin dir with my two test files, the getopt.erl from jcomellas > git and a simple walker.erl file: > > -module(walker). > > -export([main/1]). > > main([String]) -> > io:format("Up and running!! mayormente... (args:~s )\n",[String]), > halt(0); > > main(_) -> > io:format("Uso: ~s \n",[escript:script_name()]), > halt(1). > > > but then escript doesnt find the walker file: > > ./walker > escript: exception error: undefined function walker:main/1 > in function escript:run/2 > in call from escript:start/1 > in call from init:start_it/1 > in call from init:start_em/1 > > if I place the escript into the ebin files it obviouosly find the waker > module but is the .beam that is there!. > > > Do need I to include a "walker.app" to allow script to find where my main > function is from the zip archive? > > > Regards, Angel > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From cgsmcmlxxv@REDACTED Wed Oct 19 10:52:52 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Wed, 19 Oct 2011 10:52:52 +0200 Subject: [erlang-questions] Erlang website In-Reply-To: <4E9E705A.8020007@gmail.com> References: <4E9E705A.8020007@gmail.com> Message-ID: <4E9E8FE4.9090800@gmail.com> Never mind. Everything's back to normal now. Cheers, CGS On 10/19/2011 08:38 AM, CGS wrote: > Hi everyone, > > I am trying to connect to it for the documentation but it remains > hanged (not giving any response). Does anyone have the same problem? > My internet connection works just fine because I am listening radio on > line and other websites are loaded with no problem. > > Cheers, > CGS From watson.timothy@REDACTED Wed Oct 19 11:03:20 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 19 Oct 2011 10:03:20 +0100 Subject: [erlang-questions] dialyzer is driving me nuts In-Reply-To: References: <8CFE6241-B86F-41AA-A187-545C1743D57F@gmail.com> Message-ID: > > > The problem of the catch-all clause in gen_servers, etc. is a generic one. > > Do gen_servers really need a catch-all clause? Can't they just be left to crash? I don't mean to be isolent, I'm just wondering. -------------- next part -------------- An HTML attachment was scrubbed... URL: From joelr1@REDACTED Tue Oct 18 22:00:39 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 18 Oct 2011 21:00:39 +0100 Subject: [erlang-questions] dialyzer is driving me nuts In-Reply-To: References: <8CFE6241-B86F-41AA-A187-545C1743D57F@gmail.com> <9A6A09CA-308B-4527-9E6A-DA726DA23182@gmail.com> Message-ID: <3E5AD097-C888-401B-ACD7-A0E1F0EAD9E8@gmail.com> To all of you wondering, commenting out the call to gamelib:notify_start_game/1 makes all the trouble go away. All of it, I mean it. No more dialyzer warnings for barrier_start.erl. -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From watson.timothy@REDACTED Wed Oct 19 11:07:22 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 19 Oct 2011 10:07:22 +0100 Subject: [erlang-questions] mini rebar recipe, (Was Re: escript and archives ) In-Reply-To: <201110191016.27150.clist@uah.es> References: <201110181047.31133.clist@uah.es> <201110191016.27150.clist@uah.es> Message-ID: Thanks for posting the solution. Using `rebar escriptize` you can also configure the generated archive in a number of ways: - escript_incl_apps lists applications whose beam files should also be bundled into the archive - escript_shebang adds a custom shebang to the generated escript/archive - escript_emu_args allows you to pass custom emulator arguments to the generated escript/archive Cheers On 19 October 2011 09:16, Angel J. Alvarez Miguel wrote: > as nobody care about this question, i post this just for newbies like me. > > I ended using rebar... > > i created a .src rebar file for my setup. > > walker.app.src > > {application, walker, > [ > {description, ""}, > {vsn, "1"}, > {registered, []}, > {applications, [ > kernel, > stdlib > ]}, > {mod, { walker, []}}, > {env, []} > ]}. > > and then having my walker.erl and the getopt.erl filesin my src directory > > i did "rebar compile" and then "rebar escriptize", rebar cares of > generating a walker.app file > and then packages all the beam files and the app making a fresh escript > file. > > So you have to make a app file to allow escript manage the zip file.. > > This is the result app file: > > > {application,walker, > [{description,[]}, > {vsn,"1"}, > {registered,[]}, > {applications,[kernel,stdlib]}, > {mod,{walker,[]}}, > {env,[]}, > {modules,[getopt,walker]}]}. > > Just for the record the walker.erl (a rip of ex1.erl from getopt) is : > > -module(walker). > > -export([main/1]). > > %% as of getopt examples > option_spec_list() -> > CurrentUser = case os:getenv("USER") of > false -> "user"; > User -> User > end, > [ > {recursion, $r, "recurse", {boolean,false}, > "Recurse over directory contents"}, > {directory, undefined, undefined, string, > "Target directory"} > ]. > > main([]) -> > getopt:usage(option_spec_list(), escript:script_name()); > main(Args) -> > OptSpecList = option_spec_list(), > io:format("Args (~p) ~ngetopt:parse/2 returns:~n~n", [Args]), > case getopt:parse(OptSpecList, Args) of > {ok, {Options, NonOptArgs}} -> > io:format("Options:~n ~p~n~nNon-option arguments:~n > ~p~n", [Options, NonOptArgs]), > case proplists:is_defined(directory,Options) of > true -> do_walk(Options); > false -> getopt:usage(OptSpecList, > escript:script_name()) %% No target directory? > end; > {error, {Reason, Data}} -> > io:format("Error: ~s ~p~n~n", [Reason, Data]), > getopt:usage(OptSpecList, escript:script_name()) > end. > > do_walk(Args) -> > Files = filelib:fold_files(proplists:get_value(directory,Args), > ".*", > proplists:get_bool(recursion,Args), > fun(Path, Acc) -> > io:format("Procesando fichero ~s \n",[Path]), > {ok, Bin} = file:read_file(Path), > [{Path, Bin}|Acc] end, > []). > > > (A simple escript exercise for un cmdline tool that im planning to do) > > > /Angel > > On Martes, 18 de Octubre de 2011 10:47:31 Angel J. Alvarez Miguel escribi?: > > Hi, > > > > I want to packe some beams on a .ez archive and then insert the escript > > header to get a selfcontaines script ala rebar.. > > > > i made a script witth a zip archive the usual way.. > > > > Files = filelib:fold_files("./ebin", > > ".*", > > true, > > fun(Path, Acc) -> > > io:format("Procesando fichero ~s > \n",[Path]), > > {ok, Bin} = file:read_file(Path), > > [{Path, Bin}|Acc] end, > > []), > > {ok, {"mem", ZipBin}} = zip:create("mem", Files, [memory]), > > Script = <<"#!/usr/bin/env escript\n%%! -noshell -noinput\n", > > ZipBin/binary>>, > > > > ok = file:write_file("walker", Script), > > os:cmd("chmod u+x walker"), > > > > > > i have my ebin dir with my two test files, the getopt.erl from jcomellas > > git and a simple walker.erl file: > > > > -module(walker). > > > > -export([main/1]). > > > > main([String]) -> > > io:format("Up and running!! mayormente... (args:~s )\n",[String]), > > halt(0); > > > > main(_) -> > > io:format("Uso: ~s \n",[escript:script_name()]), > > halt(1). > > > > > > but then escript doesnt find the walker file: > > > > ./walker > > escript: exception error: undefined function walker:main/1 > > in function escript:run/2 > > in call from escript:start/1 > > in call from init:start_it/1 > > in call from init:start_em/1 > > > > if I place the escript into the ebin files it obviouosly find the waker > > module but is the .beam that is there!. > > > > > > Do need I to include a "walker.app" to allow script to find where my > main > > function is from the zip archive? > > > > > > Regards, Angel > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Wed Oct 19 03:59:23 2011 From: comptekki@REDACTED (Wes James) Date: Tue, 18 Oct 2011 19:59:23 -0600 Subject: [erlang-questions] erlang.org down? Message-ID: Anyone know why erlang.org is down? It's been down a lot of the day. The list seems to be working though. thanks, -wes From joelr1@REDACTED Tue Oct 18 21:29:50 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 18 Oct 2011 20:29:50 +0100 Subject: [erlang-questions] dialyzer is driving me nuts In-Reply-To: References: <8CFE6241-B86F-41AA-A187-545C1743D57F@gmail.com> <9A6A09CA-308B-4527-9E6A-DA726DA23182@gmail.com> Message-ID: Removing all types from #game{} and fixing the barrier_start type spec as follows, gives me this erorr: barrier_start.erl:53: The specification for barrier_start:barrier_start/3 states that the function might also return {'stop',{'game',_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_},tuple()} but the inferred return is {'continue',{'game',_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_},_} | {'skip',{'game',_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_},_} -type barrier_event() :: {timeout, _, _} | #pm_event{pm_message :: #pm_join{}} | #pm_event{pm_message :: #pm_leave{}} | {'EXIT', _, _}. -type barrier_return() :: {continue | stop | skip, #game{}, game_ctx()}. -spec barrier_start(#game{}, game_ctx(), barrier_event()) -> barrier_return(). What am I missing? -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From sedrik@REDACTED Wed Oct 19 11:11:44 2011 From: sedrik@REDACTED (Fredrik Andersson) Date: Wed, 19 Oct 2011 11:11:44 +0200 Subject: [erlang-questions] Erlang website In-Reply-To: <4E9E8FE4.9090800@gmail.com> References: <4E9E705A.8020007@gmail.com> <4E9E8FE4.9090800@gmail.com> Message-ID: For the next time you can always check http://www.downforeveryoneorjustme.com/ On Wed, Oct 19, 2011 at 10:52 AM, CGS wrote: > Never mind. Everything's back to normal now. > > Cheers, > CGS > > > > On 10/19/2011 08:38 AM, CGS wrote: > >> Hi everyone, >> >> I am trying to connect to it for the documentation but it remains hanged >> (not giving any response). Does anyone have the same problem? My internet >> connection works just fine because I am listening radio on line and other >> websites are loaded with no problem. >> >> Cheers, >> CGS >> > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Wed Oct 19 11:19:03 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 19 Oct 2011 11:19:03 +0200 Subject: [erlang-questions] Reading from private ets tables? In-Reply-To: <9D983A94-16F2-43E7-BE95-CFAA7194C776@erlang-solutions.com> References: <9D983A94-16F2-43E7-BE95-CFAA7194C776@erlang-solutions.com> Message-ID: Hello! Thanks, the get_held_locks() was useful. It looks like two nodes have different ideas about which process has write lock to the schema table: (netsim_guinode__4071@REDACTED)50> mnesia_locker:get_held_locks(). [{{schema,'______WHOLETABLE_____'},write,{tid,3,<0.382.0>}}] (netsim_superserver@REDACTED)59> mnesia_locker:get_held_locks(). [{{schema,'______WHOLETABLE_____'},write,{tid,565762,<21481.17181.0>}}] It all happens when the first node tries to add itself to mnesia: {ok, _Nodes} = mnesia:change_config(extra_db_nodes, [SuperNode]). Bad luck. 2011/10/18 Ulf Wiger : > > No, you can't read private ets tables (owned by some other process). > > However, in this particular case, mnesia does offer some functions that might be of help. > > mnesia:info() prints quite a lot of information, including info about held locks. > > Another function: mnesia_locker:get_held_locks(): > > 3> mnesia_locker:get_held_locks(). > [] > 4> mnesia:create_table(t, []). > {atomic,ok} > 5> spawn(fun() -> mnesia:transaction(fun() -> mnesia:write({t,1,a}), timer:sleep(30000) end) end). > <0.55.0> > 6> mnesia_locker:get_held_locks(). > [{{t,1},write,{tid,4,<0.55.0>}}] > ... > 7> mnesia_locker:get_held_locks(). > [] > > Finally, mnesia_lib:dist_coredump() is a big hammer, when all else fails. > > BR, > Ulf W > > On 18 Oct 2011, at 11:39, Attila Rajmund Nohl wrote: > >> Hello! >> >> Is there a way to read data from private ets tables? I have to debug >> mnesia, but the interesting tables (e.g. mnesia_held_locks) are >> private, so I can't find whose locks are deadlocking. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > From cgsmcmlxxv@REDACTED Wed Oct 19 08:38:18 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Wed, 19 Oct 2011 08:38:18 +0200 Subject: [erlang-questions] Erlang website Message-ID: <4E9E705A.8020007@gmail.com> Hi everyone, I am trying to connect to it for the documentation but it remains hanged (not giving any response). Does anyone have the same problem? My internet connection works just fine because I am listening radio on line and other websites are loaded with no problem. Cheers, CGS From gerryw@REDACTED Tue Oct 18 22:11:18 2011 From: gerryw@REDACTED (Gerry Weaver) Date: Tue, 18 Oct 2011 15:11:18 -0500 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> <4E9D39A1.2050008@gmail.com> <4E9DA08A.8050804@gmail.com> Message-ID: Hello All, I want to thank y'all again for the extremely helpful information and explanations. I must say I am pleasantly surprised by the objectivity in many of your responses. I too, am much more interested in using the "right tool for the job". I have been doing quite a bit of experimentation with Erlang and OTP. I find myself at a point where I definitely want the Erlang tool in my toolbox. I can see several instances on both past and future projects where Erlang could have been, or would be, a pretty interesting choice. Having said this, I want to come up with a good project that would afford me the opportunity to dig in a lot deeper. One thing that I think is missing from the big Erlang picture, is a good development environment (IDE). I know that there are a few attempts at this, but they are not really what I'm looking for. I think the best way to describe what I want, is something like QtCreator/C++ Builder. I originally didn't think much of these tools, but I have found the productivity boost they provide very useful. When I think about the things that would need to be done to make such a tool for Erlang, I believe it could give me insight into several things I would like to know a lot more about. I think it could be a good driver to push me into learning those things. When I consider some of the Erlang specific things like C nodes, Toolbar, etc., I imagine a very interesting project. I realize this may be somewhat ambitious, but regardless of the ultimate outcome, I really don't see the downside of trying. So... to that end... What would be the best/recommended way, in Erlang specific terms, to integrate with the Erlang VM. I would think that the ability to reset the VM to a fresh state, start and stop processes, Monitor processes, etc. etc. would be important beyond the normal compile/run functions. I guess what I'm really asking here is just some general advice about what a tool like this would look like in Erlang terms. I intend to use Qt as the gui toolkit. It would allow me to cover all of the most import platforms. I am very interested in any thoughts y'all have about this. I intend to start laying the foundation for this straight away. Another big motivating factor is the fact that I would like very much to have a tool like this for my own use. The code editor, Erlang shell, Toolbar combination is usable, but it would be nice to pull all of these things together in one interface.? Thanks, -G From comptekki@REDACTED Wed Oct 19 01:02:21 2011 From: comptekki@REDACTED (Wes James) Date: Tue, 18 Oct 2011 17:02:21 -0600 Subject: [erlang-questions] string to binary Message-ID: Is there a way to get "00:11:22:33:44:55" in to this format <<16#00,16#11,16#22,16#33,16#44,16#55>>? I can do string:tokens(S) to split the string (list) up, but how do you put back in <<16#....>> format? thanks, -wes From cgsmcmlxxv@REDACTED Wed Oct 19 12:01:25 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Wed, 19 Oct 2011 12:01:25 +0200 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> <4E9D39A1.2050008@gmail.com> <4E9DA08A.8050804@gmail.com> Message-ID: <4E9E9FF5.7020308@gmail.com> Hi Gerry, I think there is a nice editor based on Eclipse product (check this link http://erlide.org/index.html). It's not visual environment in the classic sense, but it can boost your productivity pretty much. I used eclipse for JAVA (never tried for Erlang because I got used with ol' good text editors like mcedit and vim :D ) and I know it's really powerful (I liked it even more than NetBeans). So, give it a try, maybe it's what you need. Cheers, CGS On 10/18/2011 10:11 PM, Gerry Weaver wrote: > Hello All, > > I want to thank y'all again for the extremely helpful information and explanations. I must say I am pleasantly surprised by the objectivity in many of your responses. I too, am much more interested in using the "right tool for the job". > > > I have been doing quite a bit of experimentation with Erlang and OTP. I find myself at a point where I definitely want the Erlang tool in my toolbox. I can see several instances on both past and future projects where Erlang could have been, or would be, a pretty interesting choice. Having said this, I want to come up with a good project that would afford me the opportunity to dig in a lot deeper. One thing that I think is missing from the big Erlang picture, is a good development environment (IDE). I know that there are a few attempts at this, but they are not really what I'm looking for. I think the best way to describe what I want, is something like QtCreator/C++ Builder. I originally didn't think much of these tools, but I have found the productivity boost they provide very useful. When I think about the things that would need to be done to make such a tool for Erlang, I believe it > could give me insight into several things I would like to know a lot more about. I think it could be a good driver to push me into learning those things. When I consider some of the Erlang specific things like C nodes, Toolbar, etc., I imagine a very interesting project. I realize this may be somewhat ambitious, but regardless of the ultimate outcome, I really don't see the downside of trying. So... to that end... > > > What would be the best/recommended way, in Erlang specific terms, to integrate with the Erlang VM. I would think that the ability to reset the VM to a fresh state, start and stop processes, Monitor processes, etc. etc. would be important beyond the normal compile/run functions. I guess what I'm really asking here is just some general advice about what a tool like this would look like in Erlang terms. I intend to use Qt as the gui toolkit. It would allow me to cover all of the most import platforms. I am very interested in any thoughts y'all have about this. I intend to start laying the foundation for this straight away. Another big motivating factor is the fact that I would like very much to have a tool like this for my own use. The code editor, Erlang shell, Toolbar combination is usable, but it would be nice to pull all of these things together in one interface. > > Thanks, > -G > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From boris.muehmer@REDACTED Tue Oct 18 22:47:16 2011 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Tue, 18 Oct 2011 22:47:16 +0200 Subject: [erlang-questions] Is "erlang.org down? In-Reply-To: References: Message-ID: Or is there another way to inform the people running "erlang.org", that there my be problems accessing that site, please? - boris From boris.muehmer@REDACTED Tue Oct 18 22:27:35 2011 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Tue, 18 Oct 2011 22:27:35 +0200 Subject: [erlang-questions] Is "erlang.org down? Message-ID: As the subject states, is "erlang.org" down? - boris From chandrashekhar.mullaparthi@REDACTED Wed Oct 19 12:12:50 2011 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Wed, 19 Oct 2011 11:12:50 +0100 Subject: [erlang-questions] string to binary In-Reply-To: References: Message-ID: Here is one solution. list_to_binary([list_to_integer(X) || X <- string:tokens("00:11:22:33:44:55", ":")]). cheers Chandru On 19 October 2011 00:02, Wes James wrote: > Is there a way to get "00:11:22:33:44:55" in to this format > <<16#00,16#11,16#22,16#33,16#44,16#55>>? > > I can do string:tokens(S) to split the string (list) up, but how do > you put back in <<16#....>> format? > > thanks, > > -wes > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Oct 19 12:14:30 2011 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 19 Oct 2011 12:14:30 +0200 Subject: [erlang-questions] cookbook entry #1 - unicode/UTF-8 strings Message-ID: cookbook # 1 - draft 1 Cookbook Question: I have often seen the words "UTF-8 string" used in sentences like "Java has UTF-8 strings". What does this mean when applied to Erlang? ---------------------------------------------------------------------- Answer: In Erlang strings are syntactic sugar for "lists of integers" Imagine the string "10(Euro)" - (Euro) is the glyph representing the Euro currency symbol. The term "UF8-string" representing "10(euro)" in Erlang could mean one of two things: Either a) [49,48,8364] (ie its a list of three unicode integers) Or b) [49,48,226,130,172] (ie its the UTF-8 encoding of the unicode characters) The so words "UTF-8" string might mean a) or might mean b) Erlang folks have always said "unicode/UTF-8 is easy in Erlang since strings are just lists of integers" - by this we mean that Erlang programs should always manipulate strings given the type a) interpretation. *all* library functions assume type a) encoding. The type b) interpretation only has meaning when you write data to a file etc. and should be as invisible to the user as possible (but when things go wrong and you get the wrong character printed you need to understand the difference) Question 1) How can we get a unicode characters into a list item? or what does a string literal look like? > X = "10\x{20ac}" [49,48,8364] This is not described in my book since the change came after the book was published (is it in the other Erlang books yet?) Question 2) How can we convert between representations a) and b) above? Easy - though one has to dig in the documentation a bit. > B = unicode:characters_to_binary(X, unicode, utf8). <<49,48,226,130,172>> > unicode:characters_to_list(B). [49,48,8364] Question 3) Can I write "10(Euro)" in an editor which supports unicode/UTF-8 and does the erlang tool chain support this? Will "erlc foo.erl" automatically detect that foo.erl is unicode encoded and do the right thing when scanning and tokenising strings? Answer: I don't know? Question 4) Can string literals be improved on? I hope so -- In Html I can say (I hope) € I'd like to say: X = "10€" in Erlang People who know far more about this than I do can tell me if this is OK ---------------------------------------------------------------------- From raimo+erlang-questions@REDACTED Wed Oct 19 12:20:59 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 19 Oct 2011 12:20:59 +0200 Subject: [erlang-questions] erlang.org down? In-Reply-To: References: Message-ID: <20111019102056.GB3470@erix.ericsson.se> On Tue, Oct 18, 2011 at 07:59:23PM -0600, Wes James wrote: > Anyone know why erlang.org is down? It's been down a lot of the day. > The list seems to be working though. Our internal Internet provider had a switch that dropped dead at about 20:20 CET yesterday. It was replaced this morning at 10:33 CET. The list should have been equally dead, but mail delays are harder to be sure of; mail is generally not disposed until after about 5 days. > > thanks, > > -wes > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From jesper.louis.andersen@REDACTED Tue Oct 18 20:35:10 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 18 Oct 2011 20:35:10 +0200 Subject: [erlang-questions] dialyzer is driving me nuts In-Reply-To: <8CFE6241-B86F-41AA-A187-545C1743D57F@gmail.com> References: <8CFE6241-B86F-41AA-A187-545C1743D57F@gmail.com> Message-ID: On Tue, Oct 18, 2011 at 19:57, Joel Reymont wrote: > The first message can be summarized as > > barrier_start.erl:50: The specification for barrier_start:barrier_start/3 states that the function might also return {'stop',#game{},tuple()} but the inferred return is {'continue',#game{{},_} | {'skip',#game{},_} One problem is the specifiction is not general enough: -spec barrier_start(#game{}, game_ctx(), barrier_event()) -> {continue, #game{}, game_ctx()}; (#game{}, game_ctx(), {'EXIT', pid(), _}) -> {stop, #game{}, game_ctx()}. Clearly, this specification says nothing about the return value {skip, Game, Ctx}. Remember that a success type here states: "If this function, barrier_start/3, returns, then its return value is of the return type in the contract and the inputs were of the form specified. The occurrence of the following application: barrier_start(Game, Ctx, foo), would reduce to {skip, Game, Ctx}, in violation of the contract. > The second message is a beauty! > > barrier_start.erl:74: The call gamelib:join(Game::#game{},Event::#pm_event{pm_message::#pm_join{}},1) does not have a term of type #game{} (with opaque subterms) as 1st argument > > What in the world does it mean? My guess is that gamelib:join/3 outright requires the types in #game{} to be correct, whereas the barrier_start/3 function only needs it to be of the right record type and doesn't care for the internals. -- J. From raimo+erlang-questions@REDACTED Wed Oct 19 12:27:26 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 19 Oct 2011 12:27:26 +0200 Subject: [erlang-questions] Is "erlang.org down? In-Reply-To: References: Message-ID: <20111019102726.GC3470@erix.ericsson.se> On Tue, Oct 18, 2011 at 10:47:16PM +0200, Boris M?hmer wrote: > Or is there another way to inform the people running "erlang.org", > that there my be problems accessing that site, please? I'll contemplate on some kind of externally originating supervision. The mail server runs on the same machine, so this channel only works when the web server crashes, not when someone cuts a cable. > > - boris > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From spawn.think@REDACTED Wed Oct 19 12:31:39 2011 From: spawn.think@REDACTED (Ahmed Omar) Date: Wed, 19 Oct 2011 12:31:39 +0200 Subject: [erlang-questions] Listing differences in loaded beam files In-Reply-To: References: Message-ID: Maybe these two functions could be useful ? beam_lib:cmp/2 http://www.erlang.org/doc/man/beam_lib.html#cmp-2 beam_lib:cmp_dirs/2 http://www.erlang.org/doc/man/beam_lib.html#cmp_dirs-2 On Tue, Oct 18, 2011 at 5:36 AM, Paul Davis wrote: > I hit an odd issue earlier today and one of the debugging questions I > tried to answer was "Do these two nodes have the same exact BEAM files > loaded, and if not, what's different?" After Googling for awhile I > haven't found anything related. Ideally I'm only looking for something > that basically does: > > Mods1 = [{Module, beam_lib:version(Module)} || Module <- > all_modules_on_node(N1)], > Mods2 = [{Module, beam_lib:version(Module)} || Module <- > all_modules_on_node(N2)], > diff(Mods1, Mods2). > > But with bonus points if it shows some sort of N-way diff for all > nodes that are connected. > > Any hints or suggestions would be appreciated. > > Thanks, > Paul J. Davis > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Best Regards, - Ahmed Omar http://nl.linkedin.com/in/adiaa Follow me on twitter @spawn_think -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrtndimitrov@REDACTED Wed Oct 19 12:46:05 2011 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Wed, 19 Oct 2011 13:46:05 +0300 Subject: [erlang-questions] Yaws process died when executing read_file Message-ID: <4E9EAA6D.3080306@gmail.com> I do performance testing of our Erlang application with OpenSTA. The test runs with 300 virtual users. At some point the following errors start popping up: Yaws process died: {{badmatch,{error,eacces}}, [{yaws_server,ut_read,1}, {yaws_server,deliver_dyn_file,5}, {yaws_server,aloop,3}, {yaws_server,acceptor0,2}, {proc_lib,init_p_do_apply,3}]} The call file:read_file(UT#urltype.fullpath) crashes in function ut_read(UT). I recompiled the module and printed the context. The error is eacces and UT holds: {urltype,yaws, {file_info,14088,regular,read_write, {{2011,9,13},{11,51,42}}, {{2011,10,17},{17,59,44}}, {{2011,3,16},{13,18,58}}, 33206,1,3,0,0,0,0}, "/handler.yaws", "c:/Temp/harmony/script/../www/handler.yaws", "/",undefined,undefined,"text/html", "/handler.yaws",undefined} The file handler.yaws is the entry point of our app and is called on every request. When I run the test with 100 or less virtual users I don't see these errors. So how can it be "Missing permission for reading the file, or for searching one of the parent directories" as the error is described in the file:read_file documentation? Thanks in advance. Martin From watson.timothy@REDACTED Wed Oct 19 12:49:13 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 19 Oct 2011 11:49:13 +0100 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: <4E9E9FF5.7020308@gmail.com> References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> <4E9D39A1.2050008@gmail.com> <4E9DA08A.8050804@gmail.com> <4E9E9FF5.7020308@gmail.com> Message-ID: There is also a very basic Erlang plugin for IntelliJ and there used to be a Netbeans based project although I don't know if it's still alive. On 19 October 2011 11:01, CGS wrote: > Hi Gerry, > > I think there is a nice editor based on Eclipse product (check this link > http://erlide.org/index.html). It's not visual environment in the classic > sense, but it can boost your productivity pretty much. I used eclipse for > JAVA (never tried for Erlang because I got used with ol' good text editors > like mcedit and vim :D ) and I know it's really powerful (I liked it even > more than NetBeans). So, give it a try, maybe it's what you need. > > Cheers, > CGS > > > > > On 10/18/2011 10:11 PM, Gerry Weaver wrote: > >> Hello All, >> >> I want to thank y'all again for the extremely helpful information and >> explanations. I must say I am pleasantly surprised by the objectivity in >> many of your responses. I too, am much more interested in using the "right >> tool for the job". >> >> >> I have been doing quite a bit of experimentation with Erlang and OTP. I >> find myself at a point where I definitely want the Erlang tool in my >> toolbox. I can see several instances on both past and future projects where >> Erlang could have been, or would be, a pretty interesting choice. Having >> said this, I want to come up with a good project that would afford me the >> opportunity to dig in a lot deeper. One thing that I think is missing from >> the big Erlang picture, is a good development environment (IDE). I know that >> there are a few attempts at this, but they are not really what I'm looking >> for. I think the best way to describe what I want, is something like >> QtCreator/C++ Builder. I originally didn't think much of these tools, but I >> have found the productivity boost they provide very useful. When I think >> about the things that would need to be done to make such a tool for Erlang, >> I believe it >> could give me insight into several things I would like to know a lot more >> about. I think it could be a good driver to push me into learning those >> things. When I consider some of the Erlang specific things like C nodes, >> Toolbar, etc., I imagine a very interesting project. I realize this may be >> somewhat ambitious, but regardless of the ultimate outcome, I really don't >> see the downside of trying. So... to that end... >> >> >> What would be the best/recommended way, in Erlang specific terms, to >> integrate with the Erlang VM. I would think that the ability to reset the VM >> to a fresh state, start and stop processes, Monitor processes, etc. etc. >> would be important beyond the normal compile/run functions. I guess what I'm >> really asking here is just some general advice about what a tool like this >> would look like in Erlang terms. I intend to use Qt as the gui toolkit. It >> would allow me to cover all of the most import platforms. I am very >> interested in any thoughts y'all have about this. I intend to start laying >> the foundation for this straight away. Another big motivating factor is the >> fact that I would like very much to have a tool like this for my own use. >> The code editor, Erlang shell, Toolbar combination is usable, but it would >> be nice to pull all of these things together in one interface. >> Thanks, >> -G >> >> >> >> >> >> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Oct 19 12:57:56 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 19 Oct 2011 12:57:56 +0200 Subject: [erlang-questions] dialyzer is driving me nuts In-Reply-To: References: <8CFE6241-B86F-41AA-A187-545C1743D57F@gmail.com> Message-ID: On Wed, Oct 19, 2011 at 11:03, Tim Watson wrote: >> >> The problem of the catch-all clause in gen_servers, etc. is a generic one. >> > > Do gen_servers really need a catch-all clause? Can't they just be left to > crash? I don't mean to be isolent, I'm just wondering. You are right that you can skip a catch-all clause, but it will lead to crashes left and right as soon as a process gets sent something which is a little off. I tend to lace my catch-all clauses with an error_logger message so I know something bad was sent. -- J. From ulf.wiger@REDACTED Wed Oct 19 13:02:18 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 19 Oct 2011 13:02:18 +0200 Subject: [erlang-questions] dialyzer is driving me nuts In-Reply-To: References: <8CFE6241-B86F-41AA-A187-545C1743D57F@gmail.com> Message-ID: On 19 Oct 2011, at 12:57, Jesper Louis Andersen wrote: > On Wed, Oct 19, 2011 at 11:03, Tim Watson wrote: >>> >>> The problem of the catch-all clause in gen_servers, etc. is a generic one. >>> >> >> Do gen_servers really need a catch-all clause? Can't they just be left to >> crash? I don't mean to be isolent, I'm just wondering. > > You are right that you can skip a catch-all clause, but it will lead > to crashes left and right as soon as a process gets sent something > which is a little off. I tend to lace my catch-all clauses with an > error_logger message so I know something bad was sent. I tend to wrap gen_server:call() in order to raise the exception in the client: call(Server, Req) -> case gen_server:call(Server, Req) of badarg -> erlang:error(badarg); Reply -> Reply end. Then: handle_call(_Unknown, _From, S) -> {reply, badarg, S}. Obviously, the scheme can be extended to carry some more information for the client-side exception. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From joelr1@REDACTED Tue Oct 18 20:34:51 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 18 Oct 2011 19:34:51 +0100 Subject: [erlang-questions] pretty-printing dialyzer messages Message-ID: <52C7DA91-8A67-4A93-B903-587F150A5B03@gmail.com> Is there a way to pretty-print dialyzer messages? It's very hard to interpret the output otherwise. Thanks, Joel -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From bob@REDACTED Wed Oct 19 13:06:38 2011 From: bob@REDACTED (Bob Ippolito) Date: Wed, 19 Oct 2011 04:06:38 -0700 Subject: [erlang-questions] string to binary In-Reply-To: References: Message-ID: You'll want to use list_to_integer(X, 16) but otherwise that looks correct. You could even use a binary comprehension: <<<<(list_to_integer(X, 16))>> || X <- string:tokens("00:11:22:33:44:55", ":")>>. On Wed, Oct 19, 2011 at 3:12 AM, Chandru wrote: > Here is one solution. > > list_to_binary([list_to_integer(X) || X <- > string:tokens("00:11:22:33:44:55", ":")]). > > cheers > Chandru > > On 19 October 2011 00:02, Wes James wrote: >> >> Is there a way to get "00:11:22:33:44:55" in to this format >> <<16#00,16#11,16#22,16#33,16#44,16#55>>? >> >> I can do string:tokens(S) to split the string (list) up, but how do >> you put back in <<16#....>> ?format? >> >> thanks, >> >> -wes >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From watson.timothy@REDACTED Wed Oct 19 13:11:58 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 19 Oct 2011 12:11:58 +0100 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> <4E9D39A1.2050008@gmail.com> <4E9DA08A.8050804@gmail.com> Message-ID: On 18 October 2011 21:11, Gerry Weaver wrote: > I have been doing quite a bit of experimentation with Erlang and OTP. I > find myself at a point where I definitely want the Erlang tool in my > toolbox. I can see several instances on both past and future projects where > Erlang could have been, or would be, a pretty interesting choice. Having > said this, I want to come up with a good project that would afford me the > opportunity to dig in a lot deeper. One thing that I think is missing from > the big Erlang picture, is a good development environment (IDE). I know that > there are a few attempts at this, but they are not really what I'm looking > for. I think the best way to describe what I want, is something like > QtCreator/C++ Builder. I originally didn't think much of these tools, but I > have found the productivity boost they provide very useful. When I think > about the things that would need to be done to make such a tool for Erlang, > I believe it > could give me insight into several things I would like to know a lot more > about. I think it could be a good driver to push me into learning those > things. When I consider some of the Erlang specific things like C nodes, > Toolbar, etc., I imagine a very interesting project. I realize this may be > somewhat ambitious, but regardless of the ultimate outcome, I really don't > see the downside of trying. So... to that end... > > Interestingly, a lot of people on this list have decried the idea of having an IDE for Erlang. I think in part this is because both the language and platform is simple enough for an IDE not to be a necessity. You can get along fine without one, because the standard libraries and platform are small enough and so on. This is not to say however, that an IDE would not be beneficial. If you do choose an existing IDE, the eclipse based erlide is probably the most mature right now. > What would be the best/recommended way, in Erlang specific terms, to > integrate with the Erlang VM. I would think that the ability to reset the VM > to a fresh state, start and stop processes, Monitor processes, etc. etc. > would be important beyond the normal compile/run functions. I guess what I'm > really asking here is just some general advice about what a tool like this > would look like in Erlang terms. I intend to use Qt as the gui toolkit. It > would allow me to cover all of the most import platforms. I am very > interested in any thoughts y'all have about this. I intend to start laying > the foundation for this straight away. Another big motivating factor is the > fact that I would like very much to have a tool like this for my own use. > The code editor, Erlang shell, Toolbar combination is usable, but it would > be nice to pull all of these things together in one interface. > > I can see a few different ways to integrate with the emulator. I think a good rule of thumb is probably to run most things in another emulator instance (i.e., node) running in a separate OS process and to rpc task requests over to it and receive the results in your own (embedded) instance. There are several ways to do this: 1. have your native (IDE) application start a C node and use that to dispatch tasks to the external worker node(s) 2. start the emulator embedded from within your native application 3. fork an OS command to start erl in another node and talk over pipes and/or socket I/O using some binary protocol (like protocol buffers, BERT or Piqi) I would expect (2) to carry the highest risk and complexity, as you'd be embedding a complex VM in a multi-threaded GUI application. I'm not 100% sure what (1) looks like in terms of the threading model on the receiving end, but it is relatively low complexity and you have the erl_interface (ei) libraries to help with marshalling/un-marshalling data in the external term format (which is what the distribution protocol uses). Obviously (3) carries additional complexity in terms of designing a protocol, but gives a nice clean separation between your application and the VM. You could spawn all sorts of tasks in the external vm, such as - collecting meta-data on applications, modules, types, functions, etc - doing the compile steps - integrating with build tools like rebar and sinan - running external tools like dialyzer, typer, conqueror And having the mechanism in place to talk to external nodes means the user can spawn a node with some code running and you can instrument and profile that node, using the trace/dbg features to have the instrumentation stats sent as messages back to your embedded c-node or whatever. Lots of possibilities - HTH. Cheers Thanks, > -G > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor@REDACTED Wed Oct 19 13:15:29 2011 From: igor@REDACTED (Igor Goryachev) Date: Wed, 19 Oct 2011 15:15:29 +0400 Subject: [erlang-questions] string to binary In-Reply-To: (Wes James's message of "Tue, 18 Oct 2011 17:02:21 -0600") References: Message-ID: <87sjmpqlke.fsf@goryachev.org> On Wed, Oct 19, 2011 at 03:02, Wes James wrote: > Is there a way to get "00:11:22:33:44:55" in to this format > <<16#00,16#11,16#22,16#33,16#44,16#55>>? > > I can do string:tokens(S) to split the string (list) up, but how do > you put back in <<16#....>> format? 1> list_to_binary([element(2, io_lib:fread("~16u", X)) || X <- string:tokens("00:11:22:33:44:55", ":")]). <<0,17,34,51,68,85>> 2> <<0,17,34,51,68,85>> == <<16#00,16#11,16#22,16#33,16#44,16#55>>. true -- Igor Goryachev From joelr1@REDACTED Tue Oct 18 20:45:03 2011 From: joelr1@REDACTED (Joel Reymont) Date: Tue, 18 Oct 2011 19:45:03 +0100 Subject: [erlang-questions] dialyzer is driving me nuts In-Reply-To: References: <8CFE6241-B86F-41AA-A187-545C1743D57F@gmail.com> Message-ID: <9A6A09CA-308B-4527-9E6A-DA726DA23182@gmail.com> On Oct 18, 2011, at 7:35 PM, Jesper Louis Andersen wrote: > Clearly, this specification says nothing about the return value {skip, > Game, Ctx}. Remember that a success type here states: "If this > function, barrier_start/3, returns, then its return value is of the > return type in the contract and the inputs were of the form specified. Expanding the type spec to -spec barrier_start(#game{}, game_ctx(), barrier_event()) -> {continue, #game{}, game_ctx()}; (#game{}, game_ctx(), {'EXIT', pid(), _}) -> {stop, #game{}, game_ctx()}; (#game{}, game_ctx(), _) -> {skip, #game{}, game_ctx()}. results in an error barrier_start.erl:50: Overloaded contract has overlapping domains; such contracts are currently unsupported and are simply ignored > My guess is that gamelib:join/3 outright requires the types in #game{} > to be correct, whereas the barrier_start/3 function only needs it to > be of the right record type and doesn't care for the internals. I had this in my record definition xref = gb_trees:empty() :: tuple(), but there's a gb_tree() built-in type that I didn't know about (thanks Kostis!). Typing xref properly made the opaque errors go away for gamelib:join and gamelib:leave. xref = gb_trees:empty() :: gb_tree(), What are the different built-in types, though? Why are they not in the type spec section of the reference manual? I tried to dialyze a simple module, e.g. -module(z). -compile([export_all]). -record(z, { xref = gb_trees:empty() }). -spec foo(#z{}) -> tuple(). foo(Z = #z{}) -> Z#z.xref. Dialyzer did not warn me on the mismatch between tuple() and gb_tree(), though. After removing the foo spec, Typer told me that the return type is any(). -------------------------------------------------------------------------- - for hire: mac osx device driver ninja, kernel extensions and usb drivers ---------------------+------------+--------------------------------------- http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont ---------------------+------------+--------------------------------------- From mononcqc@REDACTED Wed Oct 19 13:19:19 2011 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 19 Oct 2011 07:19:19 -0400 Subject: [erlang-questions] cookbook entry #1 - unicode/UTF-8 strings In-Reply-To: References: Message-ID: On Wed, Oct 19, 2011 at 6:14 AM, Joe Armstrong wrote: > cookbook # 1 - draft 1 > > > > Cookbook Question: > > I have often seen the words "UTF-8 string" used in sentences like > "Java has UTF-8 strings". What does this mean when applied to Erlang? > > ---------------------------------------------------------------------- > > Answer: > > In Erlang strings are syntactic sugar for "lists of integers" > > Imagine the string "10(Euro)" - (Euro) is the glyph representing the > Euro currency symbol. > > The term "UF8-string" representing "10(euro)" in Erlang could > mean one of two things: > > Either a) [49,48,8364] (ie its a list of three unicode > integers) > Or b) [49,48,226,130,172] (ie its the UTF-8 encoding of the > unicode characters) > > The so words "UTF-8" string might mean a) or might mean b) > No, it won't mean b). See this (using ~ts instead of ~s to specify we want unicode handling): 4> io:format("~ts~n",[<<49,48,226,130,172>>]). 10? ok 5> io:format("~ts~n",[[49,48,226,130,172]]). 10??? ok 6> io:format("~ts~n",[[49,48,8364]]). 10? ok They are not the same thing. See http://ferd.ca/will-the-real-unicode-wrangler-please-stand-up.html for a bit of stuff I've written on it, although my terminology in that blog post is far from stellar and exact according to unicode standards. The gist of it is that binaries and strings do not share the same ways to represent unicode strings. Unicode strings will have full-length codepoints, and binaries will have the byte-based representation you show. > > Erlang folks have always said "unicode/UTF-8 is easy in Erlang > since strings are just lists of integers" - by this we mean that > Erlang programs should always manipulate strings given the type a) > interpretation. *all* library functions assume type a) encoding. > Not always. list_to_binary and binary_to_list won't work well with that. You need to use unicode:characters_to_[binary|list]/1-3 and make sure the original string has been encoded correctly. > > The type b) interpretation only has meaning when you write data to a > file etc. and should be as invisible to the user as possible (but when > things go wrong and you get the wrong character printed you need to > understand the difference) > This can work when printing to a file because you can print raw bytes using a list, but it's rather hard and I would prefer people to push the distinction above (binary for bytes, lists for codepoints and numbers larger than bytes) > > Question 1) How can we get a unicode characters into a list item? > or what does a string literal look like? > > > X = "10\x{20ac}" > [49,48,8364] > This is one way to do it, yes. You can also copy/paste the character directly in the shell if I recall. > > This is not described in my book since the change came after the > book was published (is it in the other Erlang books yet?) > Not that I know of. > > Question 2) How can we convert between representations a) and b) above? > > Easy - though one has to dig in the documentation a bit. > > > B = unicode:characters_to_binary(X, unicode, utf8). > <<49,48,226,130,172>> > > unicode:characters_to_list(B). > [49,48,8364] > Yep. > > Question 3) Can I write "10(Euro)" in an editor which supports > unicode/UTF-8 and does the erlang tool chain support this? > > Will "erlc foo.erl" automatically detect that foo.erl is unicode > encoded and do the right thing when scanning and tokenising strings? > > Answer: I don't know? > Nope. Erlang assumes all files to be in latin-1. Anything that looks like unicode support is luck at best. There is no good Unicode support from static files, although the shell does support it. > > Question 4) Can string literals be improved on? > > I hope so -- In Html I can say (I hope) € > > I'd like to say: > > X = "10€" in Erlang > > People who know far more about this than I do can tell me if this > is OK > This might be funny because people might already use that escaping for HTML content on site. If you start inlining that one and converting to unicode symbols for them, there's no telling how backwards compatible this will be. In any case, I'd well prefer the compiler to support utf-8,utf-16 or utf-32 encoded files than support for '&;' or '&#ijk;' encoding. I'm pretty sure the non-latin-1 users of this mailing list would agree there, too. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Oct 18 21:00:03 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 18 Oct 2011 21:00:03 +0200 Subject: [erlang-questions] dialyzer is driving me nuts In-Reply-To: <9A6A09CA-308B-4527-9E6A-DA726DA23182@gmail.com> References: <8CFE6241-B86F-41AA-A187-545C1743D57F@gmail.com> <9A6A09CA-308B-4527-9E6A-DA726DA23182@gmail.com> Message-ID: On Tue, Oct 18, 2011 at 20:45, Joel Reymont wrote: > Expanding the type spec to > > -spec barrier_start(#game{}, game_ctx(), barrier_event()) -> {continue, #game{}, game_ctx()}; > ? ? ? ? ? ? ? ? ? (#game{}, game_ctx(), {'EXIT', pid(), _}) -> {stop, #game{}, game_ctx()}; > ? ? ? ? ? ? ? ? ? (#game{}, game_ctx(), _) -> {skip, #game{}, game_ctx()}. > > results in an error > > barrier_start.erl:50: Overloaded contract has overlapping domains; such contracts are currently unsupported and are simply ignored Yes, because now your 2nd and 3rd input is overlapping. Note that the 2nd is a specialized variant of the 3rd (technically, a subtype). These are not yet used by Dialyzer. Basically, given an output, the inputs must be discriminable from each other. You have to coalesce the inputs and outputs. It is weaker, but few type systems actually support the above notion. > > I tried to dialyze a simple module, e.g. > > -module(z). > > -compile([export_all]). > > -record(z, { > ? ?xref = gb_trees:empty() > ?}). > > -spec foo(#z{}) -> tuple(). > > foo(Z = #z{}) -> Z#z.xref. > > Dialyzer did not warn me on the mismatch between tuple() and gb_tree(), though. > > After removing the foo spec, Typer told me that the return type is any(). Technically, the dialyzer doesn't see anything problematic here. If foo/1 terminates, the output is a tuple() and the input is of type #z{}. So the success typing is okay. If you kill the foo spec, then clearly, due to erlang semantics, it doesn't matter what is in xref, the function will terminate. So as long as the input has the structure of an #z{} record, the function terminates, irrespective of the value inside that #z{}. Note that without the spec, the notion 'Z = #z{}' is just a match against if Z has record structure, {'Z', ...}. Nothing more. #z.xref is handled at compile time and expanded, so this is rougly equivalent to: foo({'Z', Xref}) -> Xref. which clearly has a type in which Xref is any(). -- J. From bob@REDACTED Wed Oct 19 13:23:27 2011 From: bob@REDACTED (Bob Ippolito) Date: Wed, 19 Oct 2011 04:23:27 -0700 Subject: [erlang-questions] cookbook entry #1 - unicode/UTF-8 strings In-Reply-To: References: Message-ID: On Wed, Oct 19, 2011 at 3:14 AM, Joe Armstrong wrote: > cookbook # 1 - draft 1 > > Question 4) ?Can string literals be improved on? > > I hope so -- In Html I can say (I hope) € > > I'd like to say: > > ? ? ?X = "10€" in Erlang > > ? ? ?People who know far more about this than I do can tell me if this > is OK In Python you can use the syntax u"\N{EURO SIGN}" which is equivalent to u"\u20ac" (in Python 3 the string literal would not be prefixed with u). This would require that Erlang ship with a big database of all the mappings (in Python this is the unicodedata module), but it would be doable. In mochiweb there's a module mochiweb_charref that can convert any HTML5 entity such as € to code points (or lists of code points in some newer cases). I think it would be quite interesting if Erlang would natively support this much smaller table of useful and easier to remember names of characters. Perhaps someone could experiment with it in a parse transform? io:format("~ts ~ts ~ts~n", [[mochiweb_charref:charref(S)] || S <- ["euro", "NotEqual", "#x24"]]). ? ? $ -bob From cgsmcmlxxv@REDACTED Wed Oct 19 13:34:24 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Wed, 19 Oct 2011 13:34:24 +0200 Subject: [erlang-questions] Yaws process died when executing read_file In-Reply-To: <4E9EAA6D.3080306@gmail.com> References: <4E9EAA6D.3080306@gmail.com> Message-ID: <4E9EB5C0.3020603@gmail.com> Hi Martin, I suppose the access permission error is coming more from the number of threads accessing that file and less from OS. Check your yaws.conf and override some default variables, especially max_connections, keepalive_maxusers and process_options variables (or fail_on_bind_err to check if it YAWS or external sources of limitation). For more information look here: http://yaws.hyber.org/yman.yaws?page=yaws.conf. Also, it wouldn't be such a bad idea to monitor your system resources while setting up limits for connections. And the last thing, if you feel your hardware system is not used at its maximum capacity, try running multiple YAWS systems. I hope it will help. Cheers, CGS On 10/19/2011 12:46 PM, Martin Dimitrov wrote: > I do performance testing of our Erlang application with OpenSTA. The > test runs with 300 virtual users. At some point the following errors > start popping up: > > Yaws process died: {{badmatch,{error,eacces}}, > [{yaws_server,ut_read,1}, > {yaws_server,deliver_dyn_file,5}, > {yaws_server,aloop,3}, > {yaws_server,acceptor0,2}, > {proc_lib,init_p_do_apply,3}]} > > The call file:read_file(UT#urltype.fullpath) crashes in function > ut_read(UT). I recompiled the module and printed the context. The error > is eacces and UT holds: > > {urltype,yaws, > {file_info,14088,regular,read_write, > {{2011,9,13},{11,51,42}}, > {{2011,10,17},{17,59,44}}, > {{2011,3,16},{13,18,58}}, > 33206,1,3,0,0,0,0}, > "/handler.yaws", > "c:/Temp/harmony/script/../www/handler.yaws", > "/",undefined,undefined,"text/html", > "/handler.yaws",undefined} > > The file handler.yaws is the entry point of our app and is called on > every request. When I run the test with 100 or less virtual users I > don't see these errors. So how can it be "Missing permission for reading > the file, or for searching one of the parent directories" as the error > is described in the file:read_file documentation? > > Thanks in advance. > > Martin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ward@REDACTED Wed Oct 19 14:29:25 2011 From: ward@REDACTED (Ward Bekker) Date: Wed, 19 Oct 2011 14:29:25 +0200 Subject: [erlang-questions] Quickcheck/PropEr: List generator not working as expected Message-ID: <20111019142925.955a0183@groupware.tty.nl> See git@REDACTED:wardbekker/proper_test.git , src/test.erl for a runnable demo app. Start with ./shell from the projects root When I test the property `proper:quickcheck(test:prop_ends_with_double_cons_is_true()).` The following error is outputted to console: Error: Couldn't produce an instance that satisfies all strict constraints after 50 tries. Is seems that `?SUCHTHAT(Drow, list(consonant()), length(Drow) > 2)` can't generate valid values. When I output the values generated by `list(consonant())` I notice that only zero or one length lists are generated. Why is that? Thx, Ward From mrtndimitrov@REDACTED Wed Oct 19 14:36:51 2011 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Wed, 19 Oct 2011 15:36:51 +0300 Subject: [erlang-questions] Yaws process died when executing read_file In-Reply-To: <4E9EB5C0.3020603@gmail.com> References: <4E9EAA6D.3080306@gmail.com> <4E9EB5C0.3020603@gmail.com> Message-ID: <4E9EC463.4030809@gmail.com> Hello, Thanks for the reply. Looking through the config options I found max_size_cached_file which has a default value of 8 kb. The yaws file that is being requested is about 15 kb. Changing the option to 20 kb made the file cashed in memory and no errors are returned. But I wonder why erlang cannot open the file for reading. Thanks again. Martin On 10/19/2011 2:34 PM, CGS wrote: > Hi Martin, > > I suppose the access permission error is coming more from the number > of threads accessing that file and less from OS. > > Check your yaws.conf and override some default variables, especially > max_connections, keepalive_maxusers and process_options variables (or > fail_on_bind_err to check if it YAWS or external sources of > limitation). For more information look here: > http://yaws.hyber.org/yman.yaws?page=yaws.conf. > > Also, it wouldn't be such a bad idea to monitor your system resources > while setting up limits for connections. > > And the last thing, if you feel your hardware system is not used at > its maximum capacity, try running multiple YAWS systems. > > I hope it will help. > > Cheers, > CGS > > > On 10/19/2011 12:46 PM, Martin Dimitrov wrote: >> I do performance testing of our Erlang application with OpenSTA. The >> test runs with 300 virtual users. At some point the following errors >> start popping up: >> >> Yaws process died: {{badmatch,{error,eacces}}, >> [{yaws_server,ut_read,1}, >> {yaws_server,deliver_dyn_file,5}, >> {yaws_server,aloop,3}, >> {yaws_server,acceptor0,2}, >> {proc_lib,init_p_do_apply,3}]} >> >> The call file:read_file(UT#urltype.fullpath) crashes in function >> ut_read(UT). I recompiled the module and printed the context. The error >> is eacces and UT holds: >> >> {urltype,yaws, >> {file_info,14088,regular,read_write, >> {{2011,9,13},{11,51,42}}, >> {{2011,10,17},{17,59,44}}, >> {{2011,3,16},{13,18,58}}, >> 33206,1,3,0,0,0,0}, >> "/handler.yaws", >> "c:/Temp/harmony/script/../www/handler.yaws", >> "/",undefined,undefined,"text/html", >> "/handler.yaws",undefined} >> >> The file handler.yaws is the entry point of our app and is called on >> every request. When I run the test with 100 or less virtual users I >> don't see these errors. So how can it be "Missing permission for reading >> the file, or for searching one of the parent directories" as the error >> is described in the file:read_file documentation? >> >> Thanks in advance. >> >> Martin >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vinoski@REDACTED Wed Oct 19 14:45:29 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 19 Oct 2011 08:45:29 -0400 Subject: [erlang-questions] Yaws process died when executing read_file In-Reply-To: <4E9EAA6D.3080306@gmail.com> References: <4E9EAA6D.3080306@gmail.com> Message-ID: On Wed, Oct 19, 2011 at 6:46 AM, Martin Dimitrov wrote: > I do performance testing of our Erlang application with OpenSTA. The > test runs with 300 virtual users. At some point the following errors > start popping up: > > Yaws process died: {{badmatch,{error,eacces}}, > ? ? ? ? ? ? ? ?[{yaws_server,ut_read,1}, > ? ? ? ? ? ? ? ? {yaws_server,deliver_dyn_file,5}, > ? ? ? ? ? ? ? ? {yaws_server,aloop,3}, > ? ? ? ? ? ? ? ? {yaws_server,acceptor0,2}, > ? ? ? ? ? ? ? ? {proc_lib,init_p_do_apply,3}]} > > The call file:read_file(UT#urltype.fullpath) crashes in function > ut_read(UT). I recompiled the module and printed the context. The error > is eacces and UT holds: > > {urltype,yaws, > ? ? ? ? ? ? ? ? ? ? ? ? ?{file_info,14088,regular,read_write, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{{2011,9,13},{11,51,42}}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{{2011,10,17},{17,59,44}}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{{2011,3,16},{13,18,58}}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?33206,1,3,0,0,0,0}, > ? ? ? ? ? ? ? ? ? ? ? ? ?"/handler.yaws", > ? ? ? ? ? ? ? ? ? ? ? ? ?"c:/Temp/harmony/script/../www/handler.yaws", > ? ? ? ? ? ? ? ? ? ? ? ? ?"/",undefined,undefined,"text/html", > ? ? ? ? ? ? ? ? ? ? ? ? ?"/handler.yaws",undefined} > > The file handler.yaws is the entry point of our app and is called on > every request. When I run the test with 100 or less virtual users I > don't see these errors. So how can it be "Missing permission for reading > the file, or for searching one of the parent directories" as the error > is described in the file:read_file documentation? Please let us know: * what operating system and version you're using * what version of Erlang you're using * what version of Yaws you're using --steve From cgsmcmlxxv@REDACTED Wed Oct 19 14:54:38 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Wed, 19 Oct 2011 14:54:38 +0200 Subject: [erlang-questions] Yaws process died when executing read_file In-Reply-To: <4E9EC463.4030809@gmail.com> References: <4E9EAA6D.3080306@gmail.com> <4E9EB5C0.3020603@gmail.com> <4E9EC463.4030809@gmail.com> Message-ID: <4E9EC88E.4040900@gmail.com> Hi, Well, that's a question I cannot answer (I can only give you wild guesses), sorry. But if you need the file just with Erlang (not in YAWS), why don't you use the same mechanism (meaning, passing the buffer further and not accessing the harddisk all the time)? It seems reasonable and faster than accessing it over and over from the harddisk. Cheers, CGS On 10/19/2011 02:36 PM, Martin Dimitrov wrote: > Hello, > > Thanks for the reply. > > Looking through the config options I found max_size_cached_file which > has a default value of 8 kb. The yaws file that is being requested is > about 15 kb. Changing the option to 20 kb made the file cashed in memory > and no errors are returned. But I wonder why erlang cannot open the file > for reading. > > Thanks again. > > Martin > > > On 10/19/2011 2:34 PM, CGS wrote: >> Hi Martin, >> >> I suppose the access permission error is coming more from the number >> of threads accessing that file and less from OS. >> >> Check your yaws.conf and override some default variables, especially >> max_connections, keepalive_maxusers and process_options variables (or >> fail_on_bind_err to check if it YAWS or external sources of >> limitation). For more information look here: >> http://yaws.hyber.org/yman.yaws?page=yaws.conf. >> >> Also, it wouldn't be such a bad idea to monitor your system resources >> while setting up limits for connections. >> >> And the last thing, if you feel your hardware system is not used at >> its maximum capacity, try running multiple YAWS systems. >> >> I hope it will help. >> >> Cheers, >> CGS >> >> >> On 10/19/2011 12:46 PM, Martin Dimitrov wrote: >>> I do performance testing of our Erlang application with OpenSTA. The >>> test runs with 300 virtual users. At some point the following errors >>> start popping up: >>> >>> Yaws process died: {{badmatch,{error,eacces}}, >>> [{yaws_server,ut_read,1}, >>> {yaws_server,deliver_dyn_file,5}, >>> {yaws_server,aloop,3}, >>> {yaws_server,acceptor0,2}, >>> {proc_lib,init_p_do_apply,3}]} >>> >>> The call file:read_file(UT#urltype.fullpath) crashes in function >>> ut_read(UT). I recompiled the module and printed the context. The error >>> is eacces and UT holds: >>> >>> {urltype,yaws, >>> {file_info,14088,regular,read_write, >>> {{2011,9,13},{11,51,42}}, >>> {{2011,10,17},{17,59,44}}, >>> {{2011,3,16},{13,18,58}}, >>> 33206,1,3,0,0,0,0}, >>> "/handler.yaws", >>> "c:/Temp/harmony/script/../www/handler.yaws", >>> "/",undefined,undefined,"text/html", >>> "/handler.yaws",undefined} >>> >>> The file handler.yaws is the entry point of our app and is called on >>> every request. When I run the test with 100 or less virtual users I >>> don't see these errors. So how can it be "Missing permission for reading >>> the file, or for searching one of the parent directories" as the error >>> is described in the file:read_file documentation? >>> >>> Thanks in advance. >>> >>> Martin >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mrtndimitrov@REDACTED Wed Oct 19 15:38:03 2011 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Wed, 19 Oct 2011 16:38:03 +0300 Subject: [erlang-questions] Yaws process died when executing read_file In-Reply-To: References: <4E9EAA6D.3080306@gmail.com> Message-ID: <4E9ED2BB.5060002@gmail.com> On 10/19/2011 3:45 PM, Steve Vinoski wrote: > On Wed, Oct 19, 2011 at 6:46 AM, Martin Dimitrov wrote: >> I do performance testing of our Erlang application with OpenSTA. The >> test runs with 300 virtual users. At some point the following errors >> start popping up: >> >> Yaws process died: {{badmatch,{error,eacces}}, >> [{yaws_server,ut_read,1}, >> {yaws_server,deliver_dyn_file,5}, >> {yaws_server,aloop,3}, >> {yaws_server,acceptor0,2}, >> {proc_lib,init_p_do_apply,3}]} >> >> The call file:read_file(UT#urltype.fullpath) crashes in function >> ut_read(UT). I recompiled the module and printed the context. The error >> is eacces and UT holds: >> >> {urltype,yaws, >> {file_info,14088,regular,read_write, >> {{2011,9,13},{11,51,42}}, >> {{2011,10,17},{17,59,44}}, >> {{2011,3,16},{13,18,58}}, >> 33206,1,3,0,0,0,0}, >> "/handler.yaws", >> "c:/Temp/harmony/script/../www/handler.yaws", >> "/",undefined,undefined,"text/html", >> "/handler.yaws",undefined} >> >> The file handler.yaws is the entry point of our app and is called on >> every request. When I run the test with 100 or less virtual users I >> don't see these errors. So how can it be "Missing permission for reading >> the file, or for searching one of the parent directories" as the error >> is described in the file:read_file documentation? > Please let us know: > > * what operating system and version you're using > * what version of Erlang you're using > * what version of Yaws you're using > > --steve Sorry I didn't include this info: - Operating system: Windows 7 - Erlang version: R14B04 - Yaws version: 1.91 From mrtndimitrov@REDACTED Wed Oct 19 15:44:54 2011 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Wed, 19 Oct 2011 16:44:54 +0300 Subject: [erlang-questions] Yaws process died when executing read_file In-Reply-To: <4E9EC88E.4040900@gmail.com> References: <4E9EAA6D.3080306@gmail.com> <4E9EB5C0.3020603@gmail.com> <4E9EC463.4030809@gmail.com> <4E9EC88E.4040900@gmail.com> Message-ID: <4E9ED456.8020605@gmail.com> Hi, No. The file is needed only by YAWS. But since YAWS doesn't cache it in memory, it tries to open it on every request and at some point the access error is returned. Windows 7 (the operating system we are testing on) has a very high limit of opened handles. Does Erlang have some limitation on this? Martin On 10/19/2011 3:54 PM, CGS wrote: > Hi, > > Well, that's a question I cannot answer (I can only give you wild > guesses), sorry. But if you need the file just with Erlang (not in > YAWS), why don't you use the same mechanism (meaning, passing the > buffer further and not accessing the harddisk all the time)? It seems > reasonable and faster than accessing it over and over from the harddisk. > > Cheers, > CGS > > > > > On 10/19/2011 02:36 PM, Martin Dimitrov wrote: >> Hello, >> >> Thanks for the reply. >> >> Looking through the config options I found max_size_cached_file which >> has a default value of 8 kb. The yaws file that is being requested is >> about 15 kb. Changing the option to 20 kb made the file cashed in memory >> and no errors are returned. But I wonder why erlang cannot open the file >> for reading. >> >> Thanks again. >> >> Martin >> >> >> On 10/19/2011 2:34 PM, CGS wrote: >>> Hi Martin, >>> >>> I suppose the access permission error is coming more from the number >>> of threads accessing that file and less from OS. >>> >>> Check your yaws.conf and override some default variables, especially >>> max_connections, keepalive_maxusers and process_options variables (or >>> fail_on_bind_err to check if it YAWS or external sources of >>> limitation). For more information look here: >>> http://yaws.hyber.org/yman.yaws?page=yaws.conf. >>> >>> Also, it wouldn't be such a bad idea to monitor your system resources >>> while setting up limits for connections. >>> >>> And the last thing, if you feel your hardware system is not used at >>> its maximum capacity, try running multiple YAWS systems. >>> >>> I hope it will help. >>> >>> Cheers, >>> CGS >>> >>> >>> On 10/19/2011 12:46 PM, Martin Dimitrov wrote: >>>> I do performance testing of our Erlang application with OpenSTA. The >>>> test runs with 300 virtual users. At some point the following errors >>>> start popping up: >>>> >>>> Yaws process died: {{badmatch,{error,eacces}}, >>>> [{yaws_server,ut_read,1}, >>>> {yaws_server,deliver_dyn_file,5}, >>>> {yaws_server,aloop,3}, >>>> {yaws_server,acceptor0,2}, >>>> {proc_lib,init_p_do_apply,3}]} >>>> >>>> The call file:read_file(UT#urltype.fullpath) crashes in function >>>> ut_read(UT). I recompiled the module and printed the context. The >>>> error >>>> is eacces and UT holds: >>>> >>>> {urltype,yaws, >>>> {file_info,14088,regular,read_write, >>>> {{2011,9,13},{11,51,42}}, >>>> {{2011,10,17},{17,59,44}}, >>>> {{2011,3,16},{13,18,58}}, >>>> 33206,1,3,0,0,0,0}, >>>> "/handler.yaws", >>>> >>>> "c:/Temp/harmony/script/../www/handler.yaws", >>>> "/",undefined,undefined,"text/html", >>>> "/handler.yaws",undefined} >>>> >>>> The file handler.yaws is the entry point of our app and is called on >>>> every request. When I run the test with 100 or less virtual users I >>>> don't see these errors. So how can it be "Missing permission for >>>> reading >>>> the file, or for searching one of the parent directories" as the error >>>> is described in the file:read_file documentation? >>>> >>>> Thanks in advance. >>>> >>>> Martin >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From cgsmcmlxxv@REDACTED Wed Oct 19 15:57:39 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Wed, 19 Oct 2011 15:57:39 +0200 Subject: [erlang-questions] Yaws process died when executing read_file In-Reply-To: <4E9ED456.8020605@gmail.com> References: <4E9EAA6D.3080306@gmail.com> <4E9EB5C0.3020603@gmail.com> <4E9EC463.4030809@gmail.com> <4E9EC88E.4040900@gmail.com> <4E9ED456.8020605@gmail.com> Message-ID: <4E9ED753.5010701@gmail.com> As far as I know, the only specific limitations in Erlang are related to the memory (if you impose this condition) and to the fact that you open your file to write as well (if I saw correctly, you opened the file as read_write). But, I am no expert in Erlang and maybe someone more knowledgeable can confirm or correct me here. Nevertheless, it's good your problem was solved and I hope someone will satisfy your further curiosity (as I am in no position to do that due to my own limitation in knowledge on the subject). ;) :) Cheers, CGS On 10/19/2011 03:44 PM, Martin Dimitrov wrote: > Hi, > > No. The file is needed only by YAWS. But since YAWS doesn't cache it in > memory, it tries to open it on every request and at some point the > access error is returned. Windows 7 (the operating system we are testing > on) has a very high limit of opened handles. Does Erlang have some > limitation on this? > > Martin > > On 10/19/2011 3:54 PM, CGS wrote: >> Hi, >> >> Well, that's a question I cannot answer (I can only give you wild >> guesses), sorry. But if you need the file just with Erlang (not in >> YAWS), why don't you use the same mechanism (meaning, passing the >> buffer further and not accessing the harddisk all the time)? It seems >> reasonable and faster than accessing it over and over from the harddisk. >> >> Cheers, >> CGS >> >> >> >> >> On 10/19/2011 02:36 PM, Martin Dimitrov wrote: >>> Hello, >>> >>> Thanks for the reply. >>> >>> Looking through the config options I found max_size_cached_file which >>> has a default value of 8 kb. The yaws file that is being requested is >>> about 15 kb. Changing the option to 20 kb made the file cashed in memory >>> and no errors are returned. But I wonder why erlang cannot open the file >>> for reading. >>> >>> Thanks again. >>> >>> Martin >>> >>> >>> On 10/19/2011 2:34 PM, CGS wrote: >>>> Hi Martin, >>>> >>>> I suppose the access permission error is coming more from the number >>>> of threads accessing that file and less from OS. >>>> >>>> Check your yaws.conf and override some default variables, especially >>>> max_connections, keepalive_maxusers and process_options variables (or >>>> fail_on_bind_err to check if it YAWS or external sources of >>>> limitation). For more information look here: >>>> http://yaws.hyber.org/yman.yaws?page=yaws.conf. >>>> >>>> Also, it wouldn't be such a bad idea to monitor your system resources >>>> while setting up limits for connections. >>>> >>>> And the last thing, if you feel your hardware system is not used at >>>> its maximum capacity, try running multiple YAWS systems. >>>> >>>> I hope it will help. >>>> >>>> Cheers, >>>> CGS >>>> >>>> >>>> On 10/19/2011 12:46 PM, Martin Dimitrov wrote: >>>>> I do performance testing of our Erlang application with OpenSTA. The >>>>> test runs with 300 virtual users. At some point the following errors >>>>> start popping up: >>>>> >>>>> Yaws process died: {{badmatch,{error,eacces}}, >>>>> [{yaws_server,ut_read,1}, >>>>> {yaws_server,deliver_dyn_file,5}, >>>>> {yaws_server,aloop,3}, >>>>> {yaws_server,acceptor0,2}, >>>>> {proc_lib,init_p_do_apply,3}]} >>>>> >>>>> The call file:read_file(UT#urltype.fullpath) crashes in function >>>>> ut_read(UT). I recompiled the module and printed the context. The >>>>> error >>>>> is eacces and UT holds: >>>>> >>>>> {urltype,yaws, >>>>> {file_info,14088,regular,read_write, >>>>> {{2011,9,13},{11,51,42}}, >>>>> {{2011,10,17},{17,59,44}}, >>>>> {{2011,3,16},{13,18,58}}, >>>>> 33206,1,3,0,0,0,0}, >>>>> "/handler.yaws", >>>>> >>>>> "c:/Temp/harmony/script/../www/handler.yaws", >>>>> "/",undefined,undefined,"text/html", >>>>> "/handler.yaws",undefined} >>>>> >>>>> The file handler.yaws is the entry point of our app and is called on >>>>> every request. When I run the test with 100 or less virtual users I >>>>> don't see these errors. So how can it be "Missing permission for >>>>> reading >>>>> the file, or for searching one of the parent directories" as the error >>>>> is described in the file:read_file documentation? >>>>> >>>>> Thanks in advance. >>>>> >>>>> Martin >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mrtndimitrov@REDACTED Wed Oct 19 16:01:13 2011 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Wed, 19 Oct 2011 17:01:13 +0300 Subject: [erlang-questions] Yaws process died when executing read_file In-Reply-To: <4E9ED753.5010701@gmail.com> References: <4E9EAA6D.3080306@gmail.com> <4E9EB5C0.3020603@gmail.com> <4E9EC463.4030809@gmail.com> <4E9EC88E.4040900@gmail.com> <4E9ED456.8020605@gmail.com> <4E9ED753.5010701@gmail.com> Message-ID: <4E9ED829.6010607@gmail.com> YAWS opens the file with file:read_file. On 10/19/2011 4:57 PM, CGS wrote: > As far as I know, the only specific limitations in Erlang are related > to the memory (if you impose this condition) and to the fact that you > open your file to write as well (if I saw correctly, you opened the > file as read_write). But, I am no expert in Erlang and maybe someone > more knowledgeable can confirm or correct me here. > > Nevertheless, it's good your problem was solved and I hope someone > will satisfy your further curiosity (as I am in no position to do that > due to my own limitation in knowledge on the subject). ;) :) > > Cheers, > CGS > > > > > On 10/19/2011 03:44 PM, Martin Dimitrov wrote: >> Hi, >> >> No. The file is needed only by YAWS. But since YAWS doesn't cache it in >> memory, it tries to open it on every request and at some point the >> access error is returned. Windows 7 (the operating system we are testing >> on) has a very high limit of opened handles. Does Erlang have some >> limitation on this? >> >> Martin >> >> On 10/19/2011 3:54 PM, CGS wrote: >>> Hi, >>> >>> Well, that's a question I cannot answer (I can only give you wild >>> guesses), sorry. But if you need the file just with Erlang (not in >>> YAWS), why don't you use the same mechanism (meaning, passing the >>> buffer further and not accessing the harddisk all the time)? It seems >>> reasonable and faster than accessing it over and over from the >>> harddisk. >>> >>> Cheers, >>> CGS >>> >>> >>> >>> >>> On 10/19/2011 02:36 PM, Martin Dimitrov wrote: >>>> Hello, >>>> >>>> Thanks for the reply. >>>> >>>> Looking through the config options I found max_size_cached_file which >>>> has a default value of 8 kb. The yaws file that is being requested is >>>> about 15 kb. Changing the option to 20 kb made the file cashed in >>>> memory >>>> and no errors are returned. But I wonder why erlang cannot open the >>>> file >>>> for reading. >>>> >>>> Thanks again. >>>> >>>> Martin >>>> >>>> >>>> On 10/19/2011 2:34 PM, CGS wrote: >>>>> Hi Martin, >>>>> >>>>> I suppose the access permission error is coming more from the number >>>>> of threads accessing that file and less from OS. >>>>> >>>>> Check your yaws.conf and override some default variables, especially >>>>> max_connections, keepalive_maxusers and process_options variables (or >>>>> fail_on_bind_err to check if it YAWS or external sources of >>>>> limitation). For more information look here: >>>>> http://yaws.hyber.org/yman.yaws?page=yaws.conf. >>>>> >>>>> Also, it wouldn't be such a bad idea to monitor your system resources >>>>> while setting up limits for connections. >>>>> >>>>> And the last thing, if you feel your hardware system is not used at >>>>> its maximum capacity, try running multiple YAWS systems. >>>>> >>>>> I hope it will help. >>>>> >>>>> Cheers, >>>>> CGS >>>>> >>>>> >>>>> On 10/19/2011 12:46 PM, Martin Dimitrov wrote: >>>>>> I do performance testing of our Erlang application with OpenSTA. The >>>>>> test runs with 300 virtual users. At some point the following errors >>>>>> start popping up: >>>>>> >>>>>> Yaws process died: {{badmatch,{error,eacces}}, >>>>>> [{yaws_server,ut_read,1}, >>>>>> {yaws_server,deliver_dyn_file,5}, >>>>>> {yaws_server,aloop,3}, >>>>>> {yaws_server,acceptor0,2}, >>>>>> {proc_lib,init_p_do_apply,3}]} >>>>>> >>>>>> The call file:read_file(UT#urltype.fullpath) crashes in function >>>>>> ut_read(UT). I recompiled the module and printed the context. The >>>>>> error >>>>>> is eacces and UT holds: >>>>>> >>>>>> {urltype,yaws, >>>>>> {file_info,14088,regular,read_write, >>>>>> {{2011,9,13},{11,51,42}}, >>>>>> {{2011,10,17},{17,59,44}}, >>>>>> {{2011,3,16},{13,18,58}}, >>>>>> 33206,1,3,0,0,0,0}, >>>>>> "/handler.yaws", >>>>>> >>>>>> "c:/Temp/harmony/script/../www/handler.yaws", >>>>>> "/",undefined,undefined,"text/html", >>>>>> "/handler.yaws",undefined} >>>>>> >>>>>> The file handler.yaws is the entry point of our app and is called on >>>>>> every request. When I run the test with 100 or less virtual users I >>>>>> don't see these errors. So how can it be "Missing permission for >>>>>> reading >>>>>> the file, or for searching one of the parent directories" as the >>>>>> error >>>>>> is described in the file:read_file documentation? >>>>>> >>>>>> Thanks in advance. >>>>>> >>>>>> Martin >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From comptekki@REDACTED Wed Oct 19 16:04:46 2011 From: comptekki@REDACTED (Wes James) Date: Wed, 19 Oct 2011 08:04:46 -0600 Subject: [erlang-questions] erlang.org down? In-Reply-To: References: Message-ID: ok, thanks for responding. -wes On Tue, Oct 18, 2011 at 7:59 PM, Wes James wrote: > Anyone know why erlang.org is down? ?It's been down a lot of the day. > The list seems to be working though. > > thanks, > > -wes > From comptekki@REDACTED Wed Oct 19 16:09:06 2011 From: comptekki@REDACTED (Wes James) Date: Wed, 19 Oct 2011 08:09:06 -0600 Subject: [erlang-questions] string to binary In-Reply-To: <87sjmpqlke.fsf@goryachev.org> References: <87sjmpqlke.fsf@goryachev.org> Message-ID: On Wed, Oct 19, 2011 at 5:15 AM, Igor Goryachev wrote: > On Wed, Oct 19, 2011 at 03:02, Wes James wrote: > >> Is there a way to get "00:11:22:33:44:55" in to this format >> <<16#00,16#11,16#22,16#33,16#44,16#55>>? >> >> I can do string:tokens(S) to split the string (list) up, but how do >> you put back in <<16#....>> ?format? > > 1> list_to_binary([element(2, io_lib:fread("~16u", X)) || X <- string:tokens("00:11:22:33:44:55", ":")]). > <<0,17,34,51,68,85>> > 2> <<0,17,34,51,68,85>> == <<16#00,16#11,16#22,16#33,16#44,16#55>>. > true Thanks all for responding. I appreciate it. -wes From comptekki@REDACTED Wed Oct 19 16:36:16 2011 From: comptekki@REDACTED (Wes James) Date: Wed, 19 Oct 2011 08:36:16 -0600 Subject: [erlang-questions] string to binary In-Reply-To: References: Message-ID: This is interesting. I run statistics() four times and on line 2 the line is slower than line 4, but then after that it is always a little faster. What does reduction mean? Number of internal operations? Why would it be slower at first then faster the rest of the time? Erlang R14B03 (erts-5.8.4) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.4 (abort with ^G) 1> statistics(exact_reductions). {361523,361523} 2> <<<<(list_to_integer(X, 16))>> || X <- string:tokens("00:11:22:33:44:55", ":")>>. <<0,17,34,51,68,85>> 3> statistics(exact_reductions). {372349,10826} 4> list_to_binary([element(2, io_lib:fread("~16u", X)) || X <- string:tokens("00:11:22:33:44:55", ":")]). <<0,17,34,51,68,85>> 5> statistics(exact_reductions). {381557,9208} 6> 6> statistics(exact_reductions). {384225,2668} 7> <<<<(list_to_integer(X, 16))>> || X <- string:tokens("00:11:22:33:44:55", ":")>>. <<0,17,34,51,68,85>> 8> statistics(exact_reductions). {390943,6718} 9> list_to_binary([element(2, io_lib:fread("~16u", X)) || X <- string:tokens("00:11:22:33:44:55", ":")]). <<0,17,34,51,68,85>> 10> statistics(exact_reductions). {398753,7810} 11> 11> statistics(exact_reductions). {401480,2727} 12> <<<<(list_to_integer(X, 16))>> || X <- string:tokens("00:11:22:33:44:55", ":")>>. <<0,17,34,51,68,85>> 13> statistics(exact_reductions). {408436,6956} 14> list_to_binary([element(2, io_lib:fread("~16u", X)) || X <- string:tokens("00:11:22:33:44:55", ":")]). <<0,17,34,51,68,85>> 15> statistics(exact_reductions). {416078,7642} 16> 16> statistics(exact_reductions). {418788,2710} 17> <<<<(list_to_integer(X, 16))>> || X <- string:tokens("00:11:22:33:44:55", ":")>>. <<0,17,34,51,68,85>> 18> statistics(exact_reductions). {425339,6551} 19> list_to_binary([element(2, io_lib:fread("~16u", X)) || X <- string:tokens("00:11:22:33:44:55", ":")]). <<0,17,34,51,68,85>> 20> statistics(exact_reductions). {433142,7803} 21> 21> Thanks, -wes From alessandro.sivieri@REDACTED Wed Oct 19 17:47:20 2011 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Wed, 19 Oct 2011 17:47:20 +0200 Subject: [erlang-questions] Erlang R14B3 for Ubuntu 11.10 Message-ID: Hi all, as someone requested, I have published packages of the Erlang R14B3 release for Ubuntu 11.10 in my PPA [1]: they are a direct backport from Debian testing, without modifications. As I am not an official maintainer for Ubuntu, but simply a person that wants to have the lastest version on his distribution, I publish them; they work for me, they *may* work for you too, if you experience any problems you can tell me but I *cannot* promise you to (be able to) solve them (altough I can try, at least). Have fun! [1] https://launchpad.net/~scattino/+archive/ppa -- Sivieri Alessandro alessandro.sivieri@REDACTED http://sivieri.wordpress.com/ http://www.poul.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From magnus.klaar@REDACTED Wed Oct 19 18:24:04 2011 From: magnus.klaar@REDACTED (Magnus Klaar) Date: Wed, 19 Oct 2011 18:24:04 +0200 Subject: [erlang-questions] Quickcheck/PropEr: List generator not working as expected In-Reply-To: <20111019142925.955a0183@groupware.tty.nl> References: <20111019142925.955a0183@groupware.tty.nl> Message-ID: Hi! All, iirc, quickcheck-ers has a notion on the size of a value built in, this will affect the length of lists, depth of trees etc that is incremented as more tests pass to increase the complexity of the generated test cases. You are using proper, the knob in proper you need to tweak is the 'start_size' option, if you start at a size greater than 0 or 1 your length-predicate will never hold for the first value(s) in the list(constant()) sequence. MVH Magnus On Wed, Oct 19, 2011 at 2:29 PM, Ward Bekker wrote: > See git@REDACTED:wardbekker/proper_test.git , src/test.erl for a > runnable demo app. Start with ./shell from the projects root > > When I test the property > `proper:quickcheck(test:prop_ends_with_double_cons_is_true()).` The > following error is outputted to console: > > Error: Couldn't produce an instance that satisfies all strict constraints > after 50 tries. > > Is seems that `?SUCHTHAT(Drow, list(consonant()), length(Drow) > 2)` can't > generate valid values. > > When I output the values generated by `list(consonant())` I notice that > only zero or one length lists are generated. Why is that? > > Thx, > > Ward > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From magnus.klaar@REDACTED Wed Oct 19 18:26:59 2011 From: magnus.klaar@REDACTED (Magnus Klaar) Date: Wed, 19 Oct 2011 18:26:59 +0200 Subject: [erlang-questions] Quickcheck/PropEr: List generator not working as expected In-Reply-To: References: <20111019142925.955a0183@groupware.tty.nl> Message-ID: On Wed, Oct 19, 2011 at 6:24 PM, Magnus Klaar wrote: > Hi! > > All, iirc, quickcheck-ers has a notion on the size of a value built in, > this will affect the length of lists, depth of trees etc that is incremented > as more tests pass to increase the complexity of the generated test cases. > You are using proper, the knob in proper you need to tweak is the > 'start_size' option, if you start at a size greater than 0 or 1 your > length-predicate will never hold for the first value(s) in the > list(constant()) sequence. > > MVH Magnus > > Small confusing typ, "if you start at a size greater than" should have been "if you start at a size of". -------------- next part -------------- An HTML attachment was scrubbed... URL: From ward@REDACTED Wed Oct 19 21:36:38 2011 From: ward@REDACTED (Ward Bekker) Date: Wed, 19 Oct 2011 21:36:38 +0200 Subject: [erlang-questions] Quickcheck/PropEr: List generator not working as expected Message-ID: <20111019213638.9e893adb@groupware.tty.nl> > Hi! > > All, iirc, quickcheck-ers has a notion on the size of a value built in, this > will affect the length of lists, depth of trees etc that is incremented as > more tests pass to increase the complexity of the generated test cases. You > are using proper, the knob in proper you need to tweak is the 'start_size' > option, if you start at a size greater than 0 or 1 your length-predicate > will never hold for the first value(s) in the list(constant()) sequence. Thx, that did the trick! > > MVH Magnus > > On Wed, Oct 19, 2011 at 2:29 PM, Ward Bekker wrote: > > > See git@REDACTED:wardbekker/proper_test.git , src/test.erl for a > > runnable demo app. Start with ./shell from the projects root > > > > When I test the property > > `proper:quickcheck(test:prop_ends_with_double_cons_is_true()).` The > > following error is outputted to console: > > > > Error: Couldn't produce an instance that satisfies all strict constraints > > after 50 tries. > > > > Is seems that `?SUCHTHAT(Drow, list(consonant()), length(Drow) > 2)` can't > > generate valid values. > > > > When I output the values generated by `list(consonant())` I notice that > > only zero or one length lists are generated. Why is that? > > > > Thx, > > > > Ward > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > From freeakk@REDACTED Wed Oct 19 22:14:02 2011 From: freeakk@REDACTED (Michael Uvarov) Date: Thu, 20 Oct 2011 00:14:02 +0400 Subject: [erlang-questions] cookbook entry #1 - unicode/UTF-8 strings In-Reply-To: References: Message-ID: When we say "Unicode string", we do not provide full information about this string. There are many forms of Unicode. Q: Which way did Erlang developers chose? A: The Erlang way is simple. It is to represent a string as a list of code points. It helps to solve one problem: there are no encodings (UTF-8, UTF-16, UTF-32) in this form. There are no UTF-16BE or UTF-16LE (endianess on different architectures is a problem of network systems). But this way does not solve the normalization problem. Q: But how to get my strings back to UTF-8? I want to pass it to an other application. A: There is other representation of Unicode text: as a binary. This form is used for storing text and working with external programs. Q: Is it easy to work with a list of code points? A: Both yes and no. Advantages: I you have an algorithm, which is based on code-paint processing, then it will be easy to implement. If you only pass text from point A to point B, I suggest keep a string as a binary. Also you can use both UTF-8 binaries and lists together to create an iolist from them. Disadvantages: A code-paint is not a character. It is an abstract representation of graphemes or their parts. If you run `string:len(Str)', you get the count of these code-paints (not graphemes) in Str. The problem of Erlang is poor string-processing mechanisms (in string module). This problem is rare in the telecom field, but it is hot problem for Web applications. Q: What do poor string-processing mechanisms mean? A: There are Unicode standards (http://unicode.org/standard/standard.html) which declares algorithms for many operations with strings. These operations can be also locale-dependable. The most popular realization of this operations is ICU. There are few interfaces for this library. For example, I am developing the NIF interface for icu4c. The work under this library is only in the beginning, But you can see the API. The library will be ready after R15 (because some actions with this library can freeze the schedule of the VM). https://github.com/freeakk/i18n I think we need describe what 'character' means. Fix. Either a) [49,48,8364] (ie its a list of three integers. Each integer is an unicode code-paint) Or b) [49,48,226,130,172] (ie its a list of bytes (also integers, or chars from C/C++) of the UTF-8 encoding string) -- Best regards, Uvarov Michael From tristan.sloughter@REDACTED Wed Oct 19 22:27:07 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Wed, 19 Oct 2011 15:27:07 -0500 Subject: [erlang-questions] cookbook entry #1 - unicode/UTF-8 strings In-Reply-To: References: Message-ID: Do you have a process for getting comments and reviews as the book is written, besides emails like this? I really thought the way the Real World Haskell authors did their's was amazing: http://book.realworldhaskell.org/read/ You can comment on, and see all comments for each section in a chapter and get an RSS feed of changes for chapters. I could have sworn I found a repo that they host the entire site's code for all that but I can't find it once again... Tristan On Wed, Oct 19, 2011 at 3:14 PM, Michael Uvarov wrote: > When we say "Unicode string", we do not provide full information about > this string. There are many forms of Unicode. > > Q: Which way did Erlang developers chose? > A: The Erlang way is simple. It is to represent a string as a list of > code points. It helps to solve one problem: there are no encodings > (UTF-8, UTF-16, UTF-32) in this form. There are no UTF-16BE or > UTF-16LE (endianess on different architectures is a problem of network > systems). But this way does not solve the normalization problem. > > Q: But how to get my strings back to UTF-8? I want to pass it to an > other application. > A: There is other representation of Unicode text: as a binary. This > form is used for storing text and working with external programs. > > Q: Is it easy to work with a list of code points? > A: Both yes and no. > Advantages: > I you have an algorithm, which is based on code-paint processing, then > it will be easy to implement. If you only pass text from point A to > point B, I suggest keep a string as a binary. Also you can use both > UTF-8 binaries and lists together to create an iolist from them. > > Disadvantages: > A code-paint is not a character. It is an abstract representation of > graphemes or their parts. If you run `string:len(Str)', you get the > count of these code-paints (not graphemes) in Str. The problem of > Erlang is poor string-processing mechanisms (in string module). This > problem is rare in the telecom field, but it is hot problem for Web > applications. > > Q: What do poor string-processing mechanisms mean? > A: There are Unicode standards > (http://unicode.org/standard/standard.html) which declares algorithms > for many operations with strings. These operations can be also > locale-dependable. The most popular realization of this operations is > ICU. There are few interfaces for this library. For example, I am > developing the NIF interface for icu4c. > The work under this library is only in the beginning, But you can see > the API. The library will be ready after R15 (because some actions > with this library can freeze the schedule of the VM). > > https://github.com/freeakk/i18n > > > I think we need describe what 'character' means. > > Fix. > Either a) [49,48,8364] (ie its a list of three integers. > Each integer is an unicode code-paint) > Or b) [49,48,226,130,172] (ie its a list of bytes (also > integers, or chars from C/C++) of the UTF-8 encoding string) > > -- > Best regards, > Uvarov Michael > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Oct 19 23:00:42 2011 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 20 Oct 2011 10:00:42 +1300 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> Message-ID: <86AAF423-8A2C-427E-BE0A-B8F2261068CA@cs.otago.ac.nz> On 18/10/2011, at 1:17 PM, Gerry Weaver wrote: > Is the Erlang language itself still evolving? Yes. > The reason I ask is that the fellow that did CouchDB posted a blog entry about the things he encountered while developing CouchDB. What was interesting to me is how, even after a short time with the language, many of the issues he mentioned immediately rang true with me. For example, what is the benefit of the "; , ." terminators as opposed to a single terminator approach? Of those three punctuation marks, only ONE is a terminator, and that's full stop. The other two are SEPARATORS (that is, like binary operators). "," means SEQUENCE, like ";" in Algol (so in a guard, it means "AND") ";" means ALTERNATIVE (so in a guard, it means "OR"). Since you come from a C background, it may be clearest to say that "," is not entirely unlike ";" and ";" is not entirely unlike "else", and you really wouldn't think it reasonable to unify ";" and "else" in C, now would you? Nor would you think well of a proposal to unify "&&" and "||". There are alternative syntaxes for Erlang, notably LFE (Lisp-Flavoured Erlang). It's fair to say, though, that the syntax is the least important thing about Erlang (or any other programming language). The *semantics* is what matters most. > > I assume a best practice is to minimize the use of modules written in other languages. Indeed it is. It's just like Java: once you include C code all warranties are voided. If you do that you must exercise "trepidant vigilance", a phrase I got from a Rex Stout story. From kostis@REDACTED Wed Oct 19 23:17:39 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 20 Oct 2011 00:17:39 +0300 Subject: [erlang-questions] Quickcheck/PropEr: List generator not working as expected In-Reply-To: <20111019142925.955a0183@groupware.tty.nl> References: <20111019142925.955a0183@groupware.tty.nl> Message-ID: <4E9F3E73.9080505@cs.ntua.gr> On 10/19/11 15:29, Ward Bekker wrote: > See git@REDACTED:wardbekker/proper_test.git , src/test.erl for a runnable demo app. Start with ./shell from the projects root > > When I test the property `proper:quickcheck(test:prop_ends_with_double_cons_is_true()).` The following error is outputted to console: > > Error: Couldn't produce an instance that satisfies all strict constraints after 50 tries. > > Is seems that `?SUCHTHAT(Drow, list(consonant()), length(Drow)> 2)` can't generate valid values. > > When I output the values generated by `list(consonant())` I notice that only zero or one length lists are generated. Why is that? First of all, regarding PropEr, you may be getting better/faster answers if you use the e-mail address in the cc: of this mail instead of the erlang-questions. This is because only few of the PropEr developers read this list. Now, concerning your question, you already got some answers that helped you fix the problem you are facing, but I am not sure they are the proper (TM) answers, so let me also give it a try. I see that in your program you have the following: -define(VOWELS, [$a, $e, $i, $o, $u, $y]). So far so good. Then you define the following: -spec consonant() -> proper:test(). consonant() -> ?SUCHTHAT(Char, char(), lists:all(fun(C) -> Char /= C end, ?VOWELS)). Let mention that the above: 1. does not define a proper test; instead, it defines a generator 2. seems a complicated (and a bit weird) way of writing a definition for consonants... However, these remarks are besides the point here. What you should keep in mind about generators is that they are *lazy*. This is very important here. What this means is that they do not get evaluated to some concrete value (say $c) but instead they produce a closure which knows how to generate (new) values when needed. So, your use in: ?SUCHTHAT(Drow, list(consonant()), length(Drow) > 2) basically does the following: 1. It sees the consonant() and list(consonant()) generators and generates symbolic terms for them. 2. To be able to see whether the term generated for list(consonant()) satisfies the length(Drow) > 2 condition, the generator gets evaluated. Let's assume that the term that the consonant() generator gets evaluated is $c. Then a list($c) will be generated and since lists start small, these are quite often the empty list [] or the list [$c] with just one element. 3. Since for the above the ?SUCHTHAT condition is not satisfied, PropEr will try to generate a new term for the list(consonant()) generator. 4. Now, you may think that hey, the consonant() generator has already been evaluated to a value, namely $c, so a new list of $c's will be generated next, and since the empty list and the one element list have already been generated, a bigger list of $c's will be generated next. However, that's NOT what's happening here! 5. Instead, the consonant() generator is re-evaluated so a new term gets produced, let's say $d, and a new list($d) generator will be evaluated next. Now, I hope that you can see where this is heading... list($d) is a totally new generator and has similarly equal probability of generating a small list (empty or just one element) as the previous generator (list($c)) had. Anyway, the point is that the consonant() generator may look like an Erlang function, but it's not evaluated to a value once and for all. Instead, it's evaluated lazily. You should keep this in mind because it's a subtle point in an otherwise eager language like Erlang. Hope this helps, Kostis From ok@REDACTED Wed Oct 19 23:18:04 2011 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 20 Oct 2011 10:18:04 +1300 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: <4E9D39A1.2050008@gmail.com> References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> <4E9D39A1.2050008@gmail.com> Message-ID: On 18/10/2011, at 9:32 PM, CGS wrote: > > The main point for entering the discussion is the fact that many who replied here were saying that if something crashes in your application doesn't kill your application like in C. I beg to differ on this point. If your Erlang application is single threaded, then your application (I do not speak here about OTP application) will crash if an error occurs there. If you construct your C application like an OTP application, then that means your C applications is multithreaded, so, if one thread dies, it doesn't crash your main application (as well as in OTP application, if one gen_server crashes the whole application doesn't crash). In C, if one thread goes haywire, it may not *crash* the rest of the program, but it must be taken as having *corrupted* the rest of the program. I have often found that a crash in a large C program is due to a bug many source files away from where the symptom showed up and millions of instructions before. It's why I tell my software engineering students: "If you can't quickly find out what caused a problem, try to make it occur earlier. Use a tracing tool to find out which functions were called on the path to the symptom. Add assert() statements to any of them you've not annotated. Use any checking tools you can like splint (static) or valgrind (dynamic) or commercial equivalents (like Coverity and the Rational tools). Do your level best to LOCALISE faults, but expect to fail." The advantage of Erlang is that one Erlang thread *cannot* corrupt the memory of another. More errors are more localised to start with. (Of course this applies to CML and Haskell and other languages with immutable data structures.) Java shares C's problem here. OK, so you can't construct wild pointers and scribble all over memory, but Java's reliance on shared *mutable* objects means that one Java thread *can* corrupt an object that another thread relies on. And much as I love Smalltalk, Java is just crippled Smalltalk with C syntax and static typing, and the fundamental semantic problem of mutable objects shared between threads is at the heart of Smalltalk too. From mediocregopher@REDACTED Wed Oct 19 23:44:09 2011 From: mediocregopher@REDACTED (Mediocre Gopher) Date: Wed, 19 Oct 2011 17:44:09 -0400 Subject: [erlang-questions] Erlang socket doesn't receive until the second setopts {active, once} Message-ID: First I want to apologize for also posting this question to stack overflow. I'm not sure if it's bad form to post a question to multiple places, but I had already posted it to SO when it occurred to me that I should just ask the mailing list directly. Anyway (Running erlang R14B04, kernel 2.6.18-194, centos 5.5) I have a very strange problem. I have the following code to listen and process sockets: %Opts used to make listen socket -define(TCP_OPTS, [binary, {packet, raw}, {nodelay, true}, {reuseaddr, true}, {active, false},{keepalive,true}]). %Acceptor loop which spawns off sock processors when connections %come in accept_loop(Listen) -> case gen_tcp:accept(Listen) of {ok, Socket} -> Pid = spawn(fun()->?MODULE:process_sock(Socket) end), gen_tcp:controlling_process(Socket,Pid); {error,_} -> do_nothing end, ?MODULE:accept_loop(Listen). %Probably not relevant process_sock(Sock) -> case inet:peername(Sock) of {ok,{Ip,_Port}} -> case Ip of {172,16,_,_} -> Auth = true; _ -> Auth = lists:member(Ip,?PUB_IPS) end, ?MODULE:process_sock_loop(Sock,Auth); _ -> gen_tcp:close(Sock) end. process_sock_loop(Sock,Auth) -> try inet:setopts(Sock,[{active,once}]) of ok -> receive {tcp_closed,_} -> ?MODULE:prepare_for_death(Sock,[]); {tcp_error,_,etimedout} -> ?MODULE:prepare_for_death(Sock,[]); %Not getting here {tcp,Sock,Data} -> ?MODULE:do_stuff(Sock,Data); {die,From} -> From!{ok,[]}; _ -> ?MODULE:process_sock_loop(Sock,Auth) after 60000 -> ?MODULE:process_sock_loop(Sock,Auth) end; {error,_} -> ?MODULE:prepare_for_death(Sock,[]) catch _:_ -> ?MODULE:prepare_for_death(Sock,[]) end. This whole setup works wonderfully normally, and has been working for the past few months. The server operates as a message passing server with long-held tcp connections, and it holds on average about 100k connections. However now we're trying to use the server more heavily. We're making two long-held connections (in the future probably more) to the erlang server and making a few hundred commands every second per each of those connections. Each of those commands, in the common case, spawn off a new thread which will probably make some kind of read from mnesia, and send some messages based on that. The strangeness comes when we try to test those two command connections. When we turn on the stream of commands, any new connection has about 50% chance of hanging. For instance, using netcat if I were to connect and send along the string "blahblahblah" the server should immediately return back an error. In doing this it won't make any calls outside the thread (since all it's doing is trying to parse the command, which will fail because blahblahblah isn't a command). But about 50% of the time (when the two command connections are running) typing in blahblahblah results in the server just sitting there for 60 seconds before returning that error. In trying to debug this I pulled up wireshark. The tcp handshake always happens immediately, and when the first packet from the client (netcat) is sent it acks immediately, telling me that the tcp stack of the kernel isn't the bottleneck. My only guess is that the problem lies in the process_sock_loop function. It has a receive which will go back to the top of the function after 60 seconds and try again to get more from the socket. My best guess is that the following is happening: - Connection is made, thread moves on to process_sock_loop - {active,once} is set - Thread receives, but doesn't get data even though it's there - After 60 seconds thread goes back to the top of process_sock_loop - {active, once} is set again - This time the data comes through, things proceed as normal Why this would be I have no idea, and when we turn those two command connections off everything goes back to normal and the problem goes away. Please let me know if there's any other information I could give that might help! - Brian From kris@REDACTED Wed Oct 19 23:46:46 2011 From: kris@REDACTED (Kris Rasmussen) Date: Wed, 19 Oct 2011 14:46:46 -0700 Subject: [erlang-questions] Do messages casted by Process A always get received before messages called by Process A? Message-ID: Can I assume that casts will be processed before calls in erlang if sent from the same process? I would expect this to be the case given that I assume the only difference between a cast and a call is that a call waits for a response but I certainly could be wrong. Thanks, Kris -------------- next part -------------- An HTML attachment was scrubbed... URL: From nem@REDACTED Thu Oct 20 00:00:25 2011 From: nem@REDACTED (Geoff Cant) Date: Wed, 19 Oct 2011 15:00:25 -0700 Subject: [erlang-questions] Do messages casted by Process A always get received before messages called by Process A? In-Reply-To: References: Message-ID: No, messages are handled in the order they arrive to a gen_* process. So if you (1) call A, then (2) cast A, (1) will be handled before (2). Cheers, -Geoff On 2011-10-19, at 14:46 , Kris Rasmussen wrote: > Can I assume that casts will be processed before calls in erlang if sent > from the same process? I would expect this to be the case given that I > assume the only difference between a cast and a call is that a call waits > for a response but I certainly could be wrong. > > Thanks, > Kris > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From kris@REDACTED Thu Oct 20 00:09:37 2011 From: kris@REDACTED (Kris Rasmussen) Date: Wed, 19 Oct 2011 15:09:37 -0700 Subject: [erlang-questions] Do messages casted by Process A always get received before messages called by Process A? In-Reply-To: References: Message-ID: Geoff, thanks for the response. I think I may have been ambiguous in my wording. To be more specific, can you tell me if the following is true: Imagine process A does the following: call(B, 1) cast(B, 2) call(B, 3) Will B always receive the messages in the order A sent them (1, 2, 3) as opposed to something like (1, 3, 2) occasionally? On Wed, Oct 19, 2011 at 3:00 PM, Geoff Cant wrote: > No, messages are handled in the order they arrive to a gen_* process. So if > you (1) call A, then (2) cast A, (1) will be handled before (2). > > Cheers, > -Geoff > > On 2011-10-19, at 14:46 , Kris Rasmussen wrote: > > > Can I assume that casts will be processed before calls in erlang if sent > > from the same process? I would expect this to be the case given that I > > assume the only difference between a cast and a call is that a call waits > > for a response but I certainly could be wrong. > > > > Thanks, > > Kris > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nem@REDACTED Thu Oct 20 00:16:44 2011 From: nem@REDACTED (Geoff Cant) Date: Wed, 19 Oct 2011 15:16:44 -0700 Subject: [erlang-questions] Do messages casted by Process A always get received before messages called by Process A? In-Reply-To: References: Message-ID: <615BDD36-6F7A-4A04-B146-8C57C9658E44@erlang.geek.nz> B will receive the messages in the order they were sent (1,2,3 always) unless something highly creative is going on behind the scenes (i.e. someone is deliberately doing something crazy - if you can rule out crazy Erlangers, you get 1, 2, 3 always). That sequence boils down to three message sends; the call for 1, the cast for 2 and the call for 3. Erlang guarantees that between pairs of processes (A, B), messages are delivered in the order they are sent. This coupled with the fact that gen_* processes messages received in order, means that you will get handle_call (1), handle_cast (2), handle_call(3) in B's gen_server callback module. -Geoff On 2011-10-19, at 15:09 , Kris Rasmussen wrote: > Geoff, thanks for the response. I think I may have been ambiguous in my > wording. To be more specific, can you tell me if the following is true: > > Imagine process A does the following: > call(B, 1) > cast(B, 2) > call(B, 3) > > Will B always receive the messages in the order A sent them (1, 2, 3) as > opposed to something like (1, 3, 2) occasionally? > > > On Wed, Oct 19, 2011 at 3:00 PM, Geoff Cant wrote: > >> No, messages are handled in the order they arrive to a gen_* process. So if >> you (1) call A, then (2) cast A, (1) will be handled before (2). >> >> Cheers, >> -Geoff >> >> On 2011-10-19, at 14:46 , Kris Rasmussen wrote: >> >>> Can I assume that casts will be processed before calls in erlang if sent >>> from the same process? I would expect this to be the case given that I >>> assume the only difference between a cast and a call is that a call waits >>> for a response but I certainly could be wrong. >>> >>> Thanks, >>> Kris >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> >> >> -- Geoff Cant From bob@REDACTED Thu Oct 20 00:21:11 2011 From: bob@REDACTED (Bob Ippolito) Date: Wed, 19 Oct 2011 15:21:11 -0700 Subject: [erlang-questions] Do messages casted by Process A always get received before messages called by Process A? In-Reply-To: References: Message-ID: On Wed, Oct 19, 2011 at 3:09 PM, Kris Rasmussen wrote: > Geoff, thanks for the response. I think I may have been ambiguous in my > wording. To be more specific, can you tell me if the following is true: > Imagine process A does the following: > call(B, 1) > cast(B, 2) > call(B, 3) > Will?B always receive the messages in the order A sent them (1, 2, 3) as > opposed to something like (1, 3, 2)?occasionally? >From the FAQ [1]: 10.9 Is the order of message reception guaranteed? Yes, but only within one process. If there is a live process and you send it message A and then message B, it's guaranteed that if message B arrived, message A arrived before it. On the other hand, imagine processes P, Q and R. P sends message A to Q, and then message B to R. There is no guarantee that A arrives before B. (Distributed Erlang would have a pretty tough time if this was required!) [1] http://www.erlang.org/faq/academic.html#id56948 -bob From kris@REDACTED Thu Oct 20 00:29:01 2011 From: kris@REDACTED (Kris Rasmussen) Date: Wed, 19 Oct 2011 15:29:01 -0700 Subject: [erlang-questions] Do messages casted by Process A always get received before messages called by Process A? In-Reply-To: References: Message-ID: Great. This is what I expected but I wanted to double check before I made a change that depends on this behavior ;). Thanks. On Wed, Oct 19, 2011 at 3:21 PM, Bob Ippolito wrote: > On Wed, Oct 19, 2011 at 3:09 PM, Kris Rasmussen wrote: > > Geoff, thanks for the response. I think I may have been ambiguous in my > > wording. To be more specific, can you tell me if the following is true: > > Imagine process A does the following: > > call(B, 1) > > cast(B, 2) > > call(B, 3) > > Will B always receive the messages in the order A sent them (1, 2, 3) as > > opposed to something like (1, 3, 2) occasionally? > > From the FAQ [1]: > > 10.9 Is the order of message reception guaranteed? > > Yes, but only within one process. > > If there is a live process and you send it message A and then message > B, it's guaranteed that if message B arrived, message A arrived before > it. > > On the other hand, imagine processes P, Q and R. P sends message A to > Q, and then message B to R. There is no guarantee that A arrives > before B. (Distributed Erlang would have a pretty tough time if this > was required!) > > [1] http://www.erlang.org/faq/academic.html#id56948 > > -bob > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sam@REDACTED Thu Oct 20 00:34:50 2011 From: sam@REDACTED (Sam Bobroff) Date: Thu, 20 Oct 2011 09:34:50 +1100 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: <0be88a3c1122a5cc0098411971a1364d@compvia.com> References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> Message-ID: On 18 October 2011 08:14, Gerry Weaver wrote: > Hello All, > > I am new to Erlang and I'm trying to make the case to continue learning this language. I have read all of the books available and seen several presentations on the language. I am interested in Erlang primarily for the distributed and fault tolerant features it offers. However, I have some reservations about how useful this really is. For example, if you have a network server and the listener process fails, how do you recover? It seems that the fault tolerance is only applicable in the VM context. This is, in my mind equivalent to any other application process. If I have a network server written in C, I can have a watchdog process to kill and/or restart it in the event of failure. The C based approach actually seems more robust. In this scenario, how would one restart the Erlang VM? It has been my experience that network servers most often fail due to OS or hardware limits rather than > bugs in the code (at least for thoroughly tested production ready code). When you factor in the amount of baggage that comes with Erlang, or any other VM based language, it's difficult to justify. Now there is also the rapid development aspect, but I'm finding that at least for now, the time savings is being eaten up by looking for documentation. I understand that this situation will improve over time, but the various posts I've seen from folks with more experience seem to indicate that this will take quite a while. I like the language (except the , ; . thing). You can do some pretty cool things with it, but when I got over that initial gee whiz, I had some trouble seeing a real world production use case. I'm not trying to be critical, I just figure I must have missed something along the way. Any perspectives or advice on this would be greatly appreciated. > > > Thanks, > -G Hi Gerry, I agree with most of the (great) follow up posts to your initial question: Erlang, in the right situation, is a powerful and very useful language and platform. Personally I also really enjoy using it because it's a language that is relaxing to program in: things are generally sensible and it rarely bites you in the ass (contrast to C++... *CHOMP* *CHOMP* *CHOMP*). However when you start using Erlang you'll eventually need to build a package of some kind, deploy it and then manage the service or program while it runs on the OS. This is not so well handled. It's also interesting that no-one has mentioned Erlang's ability to upgrade code while the system is running ("hot code loading"). In theory this is a great feature that is very hard to get in other languages. Ericsson has used it to deliver amazing up-times but I've never seen it used anywhere else. Ever. Not in yaws, not ejabberd or rabbitMQ (someone please correct me if I'm wrong). The reason seems to be that it's both tricky to actually do, just because when you get to the details it really is a hard thing to do, but it's also tied into the way you build and release packages and upgrades which brings me to my next point. Packaging: There is no standard packaging method for Erlang (there are several good community projects like rebar but none are standard). There seems to be, or to have been, some standard system because the OTP documentation mentions it but the documentation is incomplete or missing and the functions don't really seem to hang together or do what you need them to do. See make_tar, create_release, release_handler etc. Apparently some updates are coming (at least to the documentation) soon, which is good news but at the moment this is a big pain in the ass for anyone who wants to deploy any code. Every package I've seen either uses a community project or rolls their own. Unix service integration: The VM process itself (at least on Unix) is not well behaved and causes problems for system administrators and monitoring programs. * The VM process detaches from the console immediately with a successful exit code, even if it's unable to start your Erlang applications or if it's going to crash right away. This means that when you deploy it, you have to tell your administrators to run the start script and then manually read some log file and try to see if it's running successfully or if it even began to start up. It's impossible to deliver any messages to the console during startup in detached mode so even obvious things like a missing config file have to be discovered by digging through a log file. * The VM has no support for writing a PID file and it forks when detaching so it's hard (or impossible) to externally monitor or stop. The heart watchdog (see below) makes this even worse. If multiple Erlang VMs are run on the same machine it can be hard to tell them apart, especially if they are crashing or broken somehow... and that's when it matters! * If you use the built-in heart program to restart a crashed or hung VM process you can get into the situation where you can't (easily) stop a VM that's crashing on start-up. If you kill heart then the VM restarts it, if you kill the VM then heart restarts it. This might be OK on an embedded machine but it does not make a Unix admin happy when a service needs to be stopped right away. * All these problems make package upgrades (e.g. via RPM or deb) rather unsafe: it's not easy to determine which services to stop, its hard to stop them and be sure they've stopped. When restarting you can't know if it's started! Gah! Rather than just "rpm --upgrade" you will always have a sequence of manual steps and checks in order to perform an upgrade. * The log file written by the VM has a weird rotation strategy that makes it difficult to see which file is the actual current log file. This makes the start up issues worse; I suspect that most projects replace the logging code. In short, I've found that code running IN the VM is very reliable and easy to manage (the OTP supervisor system is great) but managing the OS VM process is hard and error prone. If you're not using Erlang as a service then things aren't so bad but you're still hit by some weirdness: the program "erl" takes command line parameters in a non standard (non-gnu anyway!) way and because of this either your program will also have to do that (and so you'll end up having to explain that to every user of the program) or you'll have to write some wrapper just to tidy them up. Sam. From jesper.louis.andersen@REDACTED Thu Oct 20 00:40:40 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 20 Oct 2011 00:40:40 +0200 Subject: [erlang-questions] Erlang socket doesn't receive until the second setopts {active, once} In-Reply-To: References: Message-ID: On Wed, Oct 19, 2011 at 23:44, Mediocre Gopher wrote: > > process_sock_loop(Sock,Auth) -> > ? ?try inet:setopts(Sock,[{active,once}]) of > ? ?ok -> > ? ? ? ?receive I really don't like the try...catch block there. I am not sure process_sock_loop is tail recursive with them in because the stack requires an exception handler frame for each call. Also, I'd recommend tracing if any other call in the loop is getting hit so you are sure data is not fluked. > In trying to debug this I pulled up wireshark. The tcp handshake > always happens immediately, and when the first packet from the client > (netcat) is sent it acks immediately, telling me that the tcp stack of > the kernel isn't the bottleneck. This is good because then the kernel at least took the data and buffered it. This means that the error, if there, lies in the Erlang VM or something above that never receives. I'd probably do: enter_loop(Sock, Auth) -> ok = inet:setopts(Sock, [{active, once}]), loop(Sock, Auth). loop(Sock, Auth) -> receive ... {tcp, Sock, Data} -> ok = inet:setopts(Sock, [{active, once}]), % We do this as early as possible so data handling is interleaved handle_data(Data), % perhaps you need a continuation here as data is not framed. ok after 6000 -> ok = inet:setopts(Sock, [{active, once}]) % We probably want to warn here end, loop(Sock, Auth). Note that these are shots in the dark. I have no idea exactly what is happening, but at least this would let you start off debugging it. Make tests that acknowledges what you think happens. So test that the data is not caught and thrown away. Test that it actually hit the timer and loops again. And so on. Basically get an ACK from each thing in the system that it does things like you expect it to do. -- J. From cgsmcmlxxv@REDACTED Thu Oct 20 00:51:53 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Thu, 20 Oct 2011 00:51:53 +0200 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> <20111017224119.GA1454@circlewave.net> <4E9D39A1.2050008@gmail.com> Message-ID: <4E9F5489.3080003@gmail.com> Hmm... "Bad boy C! Let's kill it! Hurray! Down with C!" Or maybe we already forgot the small and funny poetry "let it be written in C"... On 10/19/2011 11:18 PM, Richard O'Keefe wrote: > On 18/10/2011, at 9:32 PM, CGS wrote: >> The main point for entering the discussion is the fact that many who replied here were saying that if something crashes in your application doesn't kill your application like in C. I beg to differ on this point. If your Erlang application is single threaded, then your application (I do not speak here about OTP application) will crash if an error occurs there. If you construct your C application like an OTP application, then that means your C applications is multithreaded, so, if one thread dies, it doesn't crash your main application (as well as in OTP application, if one gen_server crashes the whole application doesn't crash). > In C, if one thread goes haywire, it may not *crash* the rest of the program, > but it must be taken as having *corrupted* the rest of the program. > > I have often found that a crash in a large C program is due to a bug many source > files away from where the symptom showed up and millions of instructions before. > It's why I tell my software engineering students: "If you can't quickly find out > what caused a problem, try to make it occur earlier. Use a tracing tool to find > out which functions were called on the path to the symptom. Add assert() > statements to any of them you've not annotated. Use any checking tools you > can like splint (static) or valgrind (dynamic) or commercial equivalents (like > Coverity and the Rational tools). Do your level best to LOCALISE faults, but > expect to fail." > > The advantage of Erlang is that one Erlang thread *cannot* corrupt the memory > of another. More errors are more localised to start with. (Of course this > applies to CML and Haskell and other languages with immutable data structures.) > > Java shares C's problem here. OK, so you can't construct wild pointers and > scribble all over memory, but Java's reliance on shared *mutable* objects means > that one Java thread *can* corrupt an object that another thread relies on. > And much as I love Smalltalk, Java is just crippled Smalltalk with C syntax and > static typing, and the fundamental semantic problem of mutable objects shared > between threads is at the heart of Smalltalk too. > > > From jesper.louis.andersen@REDACTED Thu Oct 20 00:55:41 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 20 Oct 2011 00:55:41 +0200 Subject: [erlang-questions] string to binary In-Reply-To: References: Message-ID: On Wed, Oct 19, 2011 at 16:36, Wes James wrote: > This is interesting. I run statistics() four times and on line 2 the > line is slower than line 4, but then after that it is always a little > faster. ?What does reduction mean? ?Number of internal operations? > Why would it be slower at first then faster the rest of the time? Erlang/OTP contains a reduction counter for each process. This counter is normally increased by one for each function or BIF-call that process makes. After 2000 of such reductions, a context switch is made to another process see erl_vm.h CONTEXT_REDS. Your measurement has two problems. For one, these reductions is a rather bad measure of the actual time spent. It is way better to use the call timer:tc/2 for these kinds of small performance things. While "expensive" things will cost more than a single reduction and GC also costs reductions, it is a rather crude measure. For the Erlang VM it is a simple way to know when to change context, but thats about it. The second problem is more subtle: The Erlang shell is interpreted and not compiled into bytecode and then executed. Thus, it is not uncommon to see the shell being 5 times slower than a compiled module. To get accurate measurement, you should place all of your functions in a module, only export the relevant test functions and then use timer:tc/2 to test these. It gives a better view of how fast a given solution is. -- J. From dave@REDACTED Thu Oct 20 01:57:34 2011 From: dave@REDACTED (David Goehrig) Date: Wed, 19 Oct 2011 19:57:34 -0400 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: <0be88a3c1122a5cc0098411971a1364d@compvia.com> References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> Message-ID: <7DCCE42A-3AF4-40EA-B7AA-2D42C266A0AE@nexttolast.com> One thing to think about when choosing a language is the amount of effort necessary to produce a stable system. I've recently ran an experiment where I pitted Erlang vs C/C++ in a web application stack. The results were not even close: 2 months to get a C/C++ code base running and functional, but not operationally sound (due to a buggy library from a large 3rd party). 2 weeks to reimplement the entire application in Erlang in an operational capacity with some off the shelf parts. Mind you this was a project I've done 7 times before for other clients, but this was the first time using a library from the particular vendor. When the vendor came back with "this is a known issue will not fix", it didn't take much convincing for management that it was a developmental deadend. Why did Erlang take so little time to get operational? Because all of the 3rd party code pulled in was already built by engineers who grok software engineering (and were not a bunch of PhDs from Stanford with minimal real world experience). While Erlang itself is not bug free, it is generally pretty well battlehardened. Engineers who spend their time investing in the language aren't swayed by popularity metrics, and few (none) have limited experience with other languages. If you place bets, your odds of delivering a successful product are far better than the OTP marketing hype would have you believe :) Can you build a stable performant system in any language? sure. Can you do it cheaper than you can in Erlang? Short of hiring slave labor, probably not. From vinoski@REDACTED Thu Oct 20 04:39:36 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 19 Oct 2011 22:39:36 -0400 Subject: [erlang-questions] Erlang newbie questions In-Reply-To: References: <0be88a3c1122a5cc0098411971a1364d@compvia.com> Message-ID: On Wed, Oct 19, 2011 at 6:34 PM, Sam Bobroff wrote: > > * If you use the built-in heart program to restart a crashed or hung > VM process you can get into the situation where you can't (easily) > stop a VM that's crashing on start-up. If you kill heart then the VM > restarts it, if you kill the VM then heart restarts it. This might be > OK on an embedded machine but it does not make a Unix admin happy when > a service needs to be stopped right away. The techniques I described here let you limit the number of heart restarts: http://steve.vinoski.net/blog/2009/02/22/controlling-erlangs-heart/ --steve From bfulgham@REDACTED Thu Oct 20 06:11:05 2011 From: bfulgham@REDACTED (Brent Fulgham) Date: Wed, 19 Oct 2011 21:11:05 -0700 Subject: [erlang-questions] cookbook entry #1 - unicode/UTF-8 strings In-Reply-To: References: Message-ID: <183A9EC3-7797-4B9A-BEAE-A8D2329A3A76@gmail.com> Hi Joe! On Oct 19, 2011, at 3:14 AM, Joe Armstrong wrote: > cookbook # 1 - draft 1 > >