From torben.hoffmann@REDACTED Tue Jul 1 08:57:33 2014 From: torben.hoffmann@REDACTED (Torben Hoffmann) Date: Tue, 01 Jul 2014 08:57:33 +0200 Subject: [erlang-questions] Erlang for youngsters In-Reply-To: References: <539EB979.40605@meetinghouse.net> <2B7E3E06-8001-4E1A-97F2-C61B5C4B81DD@rogvall.se> <1402945075.869528733@apps.rackspace.com> <9A551891-D3DC-4140-9F8B-8A68F3A14247@cs.otago.ac.nz> Message-ID: Very interestning! Thx for sharing, Torben Angel Alvarez (GMAIL) writes: > Hi guys, > > Have any you seen this? > > http://twdkz.wordpress.com/2014/06/26/teenage-haskell/ > > I hope it is, of interest to this thread despite being haskell related.. > > Angel Alvarez > angeljalvarezmiguel at gmail.com > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Torben Hoffmann CTO Erlang Solutions Ltd. Tel: +45 25 14 05 38 http://www.erlang-solutions.com From max.lapshin@REDACTED Tue Jul 1 11:58:52 2014 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 1 Jul 2014 13:58:52 +0400 Subject: [erlang-questions] Router suitable for launching erlang on it? Message-ID: Hi. Maybe somebody knows if it is possible to launch erlang on some router with POE ports. I want to put a device that will actively take control over ip cameras plugged into it. So, problem is to find router with more than 32 MB of RAM and more than 8 MB of flash. Does anyone has such experience? -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Tue Jul 1 12:07:06 2014 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Tue, 1 Jul 2014 12:07:06 +0200 Subject: [erlang-questions] Router suitable for launching erlang on it? In-Reply-To: References: Message-ID: On Tue, Jul 1, 2014 at 11:58 AM, Max Lapshin wrote: > Hi. > > Maybe somebody knows if it is possible to launch erlang on some router with > POE ports. > > I want to put a device that will actively take control over ip cameras > plugged into it. > > So, problem is to find router with more than 32 MB of RAM and more than 8 MB > of flash. > > Does anyone has such experience? Hi, Any OpenWRT-capable router will do (I ran Erlang on TL-WR1043ND). This[1] could be a good starting point. [1]: https://www.mail-archive.com/openwrt-users@REDACTED/msg01942.html Motiejus From jon@REDACTED Tue Jul 1 12:16:25 2014 From: jon@REDACTED (Jon Schneider) Date: Tue, 1 Jul 2014 11:16:25 +0100 Subject: [erlang-questions] Router suitable for launching erlang on it? In-Reply-To: References: Message-ID: Consider turning it round and putting Erlang on something that is easier to work on (maybe a SheevaPlug or something) and use a PoE switch. You can set up VLANs for isolation. Jon > Hi. > > Maybe somebody knows if it is possible to launch erlang on some router > with > POE ports. > > I want to put a device that will actively take control over ip cameras > plugged into it. > > So, problem is to find router with more than 32 MB of RAM and more than 8 > MB of flash. > > Does anyone has such experience? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From max.lapshin@REDACTED Tue Jul 1 12:27:13 2014 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 1 Jul 2014 14:27:13 +0400 Subject: [erlang-questions] Router suitable for launching erlang on it? In-Reply-To: References: Message-ID: That Tplink doesn't have POE (power over ethernet) =((( I know that it is much easier to run linux on some modern ARM chip and glue it with a POE hub, but I'm still looking for some integrated hardware =) -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue Jul 1 15:46:10 2014 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 1 Jul 2014 17:46:10 +0400 Subject: [erlang-questions] Complexity of ets:delete_object for bag? Message-ID: What is the complexity of ets:delete_object for bag type? O(ets_size) or O(element_count_per_key)? -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Tue Jul 1 17:31:06 2014 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Tue, 1 Jul 2014 17:31:06 +0200 Subject: [erlang-questions] Complexity of ets:delete_object for bag? In-Reply-To: References: Message-ID: <53B2D43A.8010208@erix.ericsson.se> On 07/01/2014 03:46 PM, Max Lapshin wrote: > What is the complexity of ets:delete_object for bag type? > > O(ets_size) or O(element_count_per_key)? > > It's O(element_count_per_key ). (if by "element" you mean "object") Objects with the same key are stored in a linked list together with all other objects with keys that happen to hash to the same bucket. /Sverker, Erlang/OTP From cbenac@REDACTED Tue Jul 1 18:43:52 2014 From: cbenac@REDACTED (Clara Benac Earle) Date: Tue, 01 Jul 2014 18:43:52 +0200 Subject: [erlang-questions] 9th of July Madrid Erlounge Message-ID: <53B2E548.5050905@fi.upm.es> Dear all, We will hold a Madrid Erlounge the 9th of July around 7pm at the Sala de Grados, Facultad de Inform?tica, Universidad Complutense de Madrid. ?ngel Herranz, from Universidad Polit?cnica de Madrid, will give us a talk on some interesting Erlang topic. If you are interested in this event, please subscribe to the madrid-erlang-users mailing list for more details. See you there! Clara From mark.nijhof@REDACTED Tue Jul 1 20:46:00 2014 From: mark.nijhof@REDACTED (Mark Nijhof) Date: Tue, 1 Jul 2014 20:46:00 +0200 Subject: [erlang-questions] 9th of July Madrid Erlounge In-Reply-To: <53B2E548.5050905@fi.upm.es> References: <53B2E548.5050905@fi.upm.es> Message-ID: Ah that is a shame, we are now in Madrid :-) On Jul 1, 2014 6:44 PM, "Clara Benac Earle" wrote: > Dear all, > > We will hold a Madrid Erlounge the 9th of July around 7pm at the Sala de > Grados, Facultad de Inform?tica, Universidad Complutense de Madrid. ?ngel > Herranz, from Universidad Polit?cnica de Madrid, will give us a talk on > some interesting Erlang topic. > > If you are interested in this event, please subscribe to the > madrid-erlang-users mailing list for more details. > > See you there! > Clara > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue Jul 1 20:57:18 2014 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 1 Jul 2014 22:57:18 +0400 Subject: [erlang-questions] 9th of July Madrid Erlounge In-Reply-To: References: <53B2E548.5050905@fi.upm.es> Message-ID: Ah. Would be so great to visit, but I'll be in 8-11 July in Barcelona. On Tue, Jul 1, 2014 at 10:46 PM, Mark Nijhof wrote: > Ah that is a shame, we are now in Madrid :-) > On Jul 1, 2014 6:44 PM, "Clara Benac Earle" wrote: > >> Dear all, >> >> We will hold a Madrid Erlounge the 9th of July around 7pm at the Sala de >> Grados, Facultad de Inform?tica, Universidad Complutense de Madrid. ?ngel >> Herranz, from Universidad Polit?cnica de Madrid, will give us a talk on >> some interesting Erlang topic. >> >> If you are interested in this event, please subscribe to the >> madrid-erlang-users mailing list for more details. >> >> See you there! >> Clara >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From maxim@REDACTED Tue Jul 1 22:19:50 2014 From: maxim@REDACTED (Maxim Sokhatsky) Date: Tue, 1 Jul 2014 23:19:50 +0300 Subject: [erlang-questions] observer under Windows has broken statusbar Message-ID: Don't know where to put this. But under Windows 8, observer application has broken (enormous) statusbar. Here is screenshot of the problem: http://synrc.com/lj/observer-broken-statusbar.png @5HT From maxim@REDACTED Tue Jul 1 23:24:52 2014 From: maxim@REDACTED (Maxim Sokhatsky) Date: Wed, 2 Jul 2014 00:24:52 +0300 Subject: [erlang-questions] observer under Windows has broken statusbar In-Reply-To: References: , Message-ID: As you can see on screenshot it is 17.0 @5HT From dangud@REDACTED Wed Jul 2 09:31:21 2014 From: dangud@REDACTED (Dan Gudmundsson) Date: Wed, 2 Jul 2014 09:31:21 +0200 Subject: [erlang-questions] observer under Windows has broken statusbar In-Reply-To: References: Message-ID: Sigh. We don't even have a Win 8 machine here currently, so that will take some time. Also I suspect it's wxWidgets bug, since the statusbar is just created by observer and maintined and handled by wxWidgets, feels like an erlang workarond is tough to fix. Maybe it's time to switch to wxWidgets-3.0 for the windows builds, that is at least maintained. Anyway, thanks for the bug report will see what I can do. /Dan On Tue, Jul 1, 2014 at 11:24 PM, Maxim Sokhatsky wrote: > As you can see on screenshot it is 17.0 > > @5HT > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From boris.muehmer@REDACTED Wed Jul 2 15:38:31 2014 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Wed, 2 Jul 2014 15:38:31 +0200 Subject: [erlang-questions] Erlang/OTP on Cubietruck with access to GPIO (and I2S) Message-ID: Maybe this is just a question for Tony Rogcall, but I will ask all of You. I bootstraped Erlang/OTP on my Cubietruck (and my Raspberry Pi as well). The Cubietruck wasn't tortured as much as the Raspberry Pi in this process, because it runs on a SSD instead of a SD-Card. The result is much more complete (than cross-compiling), because for example wx is build as well and it is working (using ssh -X for a test), on the other hand it takes a bit longer to complete compared to a destop system with eight cores. So the next step I want to do is to access the GPIO ports/pings (and I2S) from within Erlang. Has anyone already tried this on the Cubietruck? Does Tony's gpio code for the Raspberry Pi work on the Cubietruck, too? Regards, Boris -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Wed Jul 2 17:02:06 2014 From: tony@REDACTED (Tony Rogvall) Date: Wed, 2 Jul 2014 17:02:06 +0200 Subject: [erlang-questions] Erlang/OTP on Cubietruck with access to GPIO (and I2S) In-Reply-To: References: Message-ID: <36BAE627-050B-4361-A372-4339D4BC3DFB@rogvall.se> On 2 jul 2014, at 15:38, Boris M?hmer wrote: > Maybe this is just a question for Tony Rogcall, but I will ask all of You. > > I bootstraped Erlang/OTP on my Cubietruck (and my Raspberry Pi as well). The Cubietruck wasn't tortured as much as the Raspberry Pi in this process, because it runs on a SSD instead of a SD-Card. > > The result is much more complete (than cross-compiling), because for example wx is build as well and it is working (using ssh -X for a test), on the other hand it takes a bit longer to complete compared to a destop system with eight cores. > > So the next step I want to do is to access the GPIO ports/pings (and I2S) from within Erlang. > Yes, it even has native support for Raspberry pi and some omap/2 3 platforms. The "native" support is called direct. If I remember correctly you can even use direct mode and listen for interrupts at the same time but... > Has anyone already tried this on the Cubietruck? > Does Tony's gpio code for the Raspberry Pi work on the Cubietruck, too? > The gpio code works for all gpio pins mapped on sysfs, but the native variants are of course much faster. /Tony > > Regards, > Boris > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Jul 3 03:25:36 2014 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 3 Jul 2014 13:25:36 +1200 Subject: [erlang-questions] Installing 17.1 on a T2 kills it Message-ID: I have a second-hand SPARC T2 machine running some version of SunOS 5.11 (I'll explain why no details shortly). It didn't come with SunStudio, so I've installed gcc-4.8.0 on it. The next thing to do, obviously, is install Erlang. So I downloaded otp_src_17.1, unpacked it, and bash-3.2$ ./configure --prefix=$HOME/local Ignoring the --cache-file argument since it can cause the system to be erroneously configured Disabling caching checking build system type... sparc-sun-solaris2.11 checking host system type... sparc-sun-solaris2.11 checking for gcc... gcc ... checking for posix_fadvise... no checking for closefrom... yes checking linux/falloc.h usability... no checking linux/falloc.h presence... no checking for linux/falloc.h... no checking whether fallocate() works... no checking whether posix_fallocate() works... and at that point the whole system went off the air. That's why no uname -a details; it takes a full hardware reset to get the machine back up and then a full fsck to undo the file system damage. (1) Has anyone else had this problem? (2) Is there any easier way to tell Erlang 'forget about *fallocate() than digging through the configure sources and hacking? Like a --without-fallocate switch? Oh, this is the second time it's happened, on a machine that has been running nicely for several months. From calle@REDACTED Thu Jul 3 09:04:24 2014 From: calle@REDACTED (Calle Dybedahl) Date: Thu, 3 Jul 2014 09:04:24 +0200 Subject: [erlang-questions] Cannot use io:format() in a handle_call() callback in an application? Message-ID: Hi there. I've just run into a bit of behavior that I do not understand: if I have an io:format() in a handle_call, I get a timeout instead of a return from gen_server:call(), but only if the gen_server module was started as part of an application. I ran into it while trying to do something real, but it's easy enough to reproduce in a clean environment. So. I start with an application created with "rebar create-app appid=wtf" and "rebar create template=simplesrv srvid=wtf", and the wtf module added as a child of the supervisor. If I then do "application:start(wtf)." in a shell, "gen_server:call(wtf,something)." returns "ok" as expected. If I now add the single line "io:format("In handle_call.~n")," at the start of wtf:handle_call(), and try to run the same two commands, instead of "ok" I get a pause followed by: ** exception exit: {timeout,{gen_server,call,[wtf,something]}} in function gen_server:call/2 (gen_server.erl, line 182) I totally don't understand why this happens, and only when it was started with "application:start(wtf)". If I start either the wtf_sup or wtf modules manually, the io:format() runs and prints what it should without problems. I'm using Erlang 17.1 (installed with Homebrew) under OSX 10.9. -- Calle Dybedahl -*- calle@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From eriksoe@REDACTED Thu Jul 3 09:54:20 2014 From: eriksoe@REDACTED (=?UTF-8?Q?Erik_S=C3=B8e_S=C3=B8rensen?=) Date: Thu, 3 Jul 2014 09:54:20 +0200 Subject: [erlang-questions] Cannot use io:format() in a handle_call() callback in an application? In-Reply-To: References: Message-ID: This is a long shot, but... you don't happen to have a module called 'user' or 'group' lying around in your app, do you? /Erik Den 03/07/2014 09.42 skrev "Calle Dybedahl" : > Hi there. > > I've just run into a bit of behavior that I do not understand: if I have > an io:format() in a handle_call, I get a timeout instead of a return from > gen_server:call(), but only if the gen_server module was started as part of > an application. I ran into it while trying to do something real, but it's > easy enough to reproduce in a clean environment. > > So. I start with an application created with "rebar create-app appid=wtf" > and "rebar create template=simplesrv srvid=wtf", and the wtf module added > as a child of the supervisor. If I then do "application:start(wtf)." in a > shell, "gen_server:call(wtf,something)." returns "ok" as expected. If I now > add the single line "io:format("In handle_call.~n")," at the start of > wtf:handle_call(), and try to run the same two commands, instead of "ok" I > get a pause followed by: > > ** exception exit: {timeout,{gen_server,call,[wtf,something]}} > in function gen_server:call/2 (gen_server.erl, line 182) > > I totally don't understand why this happens, and only when it was started > with "application:start(wtf)". If I start either the wtf_sup or wtf modules > manually, the io:format() runs and prints what it should without problems. > > I'm using Erlang 17.1 (installed with Homebrew) under OSX 10.9. > -- > Calle Dybedahl -*- calle@REDACTED > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Thu Jul 3 11:14:36 2014 From: lukas@REDACTED (Lukas Larsson) Date: Thu, 3 Jul 2014 11:14:36 +0200 Subject: [erlang-questions] Installing 17.1 on a T2 kills it In-Reply-To: References: Message-ID: On Thu, Jul 3, 2014 at 3:25 AM, Richard A. O'Keefe wrote: > > (1) Has anyone else had this problem? > Our sunos 5.11 (SunOS 5.11 oi_151a9 i86pc i386 i86pc) machine does not have this problem. > (2) Is there any easier way to tell Erlang 'forget about > *fallocate() than digging through the configure sources > and hacking? Like a --without-fallocate switch? > > Unfortunately there is no way to do this. Manually you can remove line 1190 to 1222 in erts/configure.in and that should make it skip the test. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Thu Jul 3 11:16:17 2014 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Thu, 3 Jul 2014 11:16:17 +0200 Subject: [erlang-questions] Cannot use io:format() in a handle_call() callback in an application? In-Reply-To: References: Message-ID: Hello! Try to trace into the io:format call to see what happens. 2014-07-03 9:04 GMT+02:00 Calle Dybedahl : > Hi there. > > I've just run into a bit of behavior that I do not understand: if I have an > io:format() in a handle_call, I get a timeout instead of a return from > gen_server:call(), but only if the gen_server module was started as part of > an application. I ran into it while trying to do something real, but it's > easy enough to reproduce in a clean environment. > > So. I start with an application created with "rebar create-app appid=wtf" > and "rebar create template=simplesrv srvid=wtf", and the wtf module added as > a child of the supervisor. If I then do "application:start(wtf)." in a > shell, "gen_server:call(wtf,something)." returns "ok" as expected. If I now > add the single line "io:format("In handle_call.~n")," at the start of > wtf:handle_call(), and try to run the same two commands, instead of "ok" I > get a pause followed by: > > ** exception exit: {timeout,{gen_server,call,[wtf,something]}} > in function gen_server:call/2 (gen_server.erl, line 182) > > I totally don't understand why this happens, and only when it was started > with "application:start(wtf)". If I start either the wtf_sup or wtf modules > manually, the io:format() runs and prints what it should without problems. > > I'm using Erlang 17.1 (installed with Homebrew) under OSX 10.9. > -- > Calle Dybedahl -*- calle@REDACTED > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ivan@REDACTED Thu Jul 3 12:08:09 2014 From: ivan@REDACTED (Ivan Uemlianin) Date: Thu, 03 Jul 2014 11:08:09 +0100 Subject: [erlang-questions] Uniqueness Typing for Resource Management in Message-Passing Concurrency Message-ID: <53B52B89.8050100@llaisdy.com> Dear All Saw this and thought of you. Uniqueness Typing for Resource Management in Message-Passing Concurrency http://arxiv.org/abs/1003.5513 (submitted 2010) Mentions erlang in passing. Here is the abstract: We view channels as the main form of resources in a message-passing programming paradigm. These channels need to be carefully managed in settings where resources are scarce. To study this problem, we extend the pi-calculus with primitives for channel allocation and deallocation and allow channels to be reused to communicate values of different types. Inevitably, the added expressiveness increases the possibilities for runtime errors. We define a substructural type system which combines uniqueness typing and affine typing to reject these ill-behaved programs. Ivan -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From mononcqc@REDACTED Thu Jul 3 14:47:39 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 3 Jul 2014 08:47:39 -0400 Subject: [erlang-questions] Uniqueness Typing for Resource Management in Message-Passing Concurrency In-Reply-To: <53B52B89.8050100@llaisdy.com> References: <53B52B89.8050100@llaisdy.com> Message-ID: <20140703124738.GA51530@ferdair.local> On 07/03, Ivan Uemlianin wrote: > Dear All > > Saw this and thought of you. > > Uniqueness Typing for Resource Management in Message-Passing Concurrency > http://arxiv.org/abs/1003.5513 > (submitted 2010) > > Mentions erlang in passing. Here is the abstract: > > We view channels as the main form of resources in a message-passing > programming paradigm. These channels need to be carefully managed in > settings where resources are scarce. To study this problem, we extend the > pi-calculus with primitives for channel allocation and deallocation and > allow channels to be reused to communicate values of different types. > Inevitably, the added expressiveness increases the possibilities for runtime > errors. We define a substructural type system which combines uniqueness > typing and affine typing to reject these ill-behaved programs. > > This sounds like it would be nicer for a language with actual channels. There are a few differences with Pi-Calculus and Erlang: - Pi-Calculus assumes synchronous message delivery, Erlang doesn't - Pi-Calculus assumes the usage of channels that are nameable and passable as if they were variables, Erlang has a per-process mailbox (you can pass pids as if they were channels and treat specific processes as channels, I guess) - Channels can be created at any time by anyone - More. Interestingly enough, there were still attempts to model a subset Erlang as pi-calculus for verification purposes (they did need to make an async pi-calc I believe): http://www.cs.usask.ca/~croy/papers/2006/Erlang06_Roy.pdf Regards, Fred. From davidnwelton@REDACTED Thu Jul 3 15:22:34 2014 From: davidnwelton@REDACTED (David Welton) Date: Thu, 3 Jul 2014 15:22:34 +0200 Subject: [erlang-questions] Huge erl_crash.dump (2 gigs) - looking for advice Message-ID: The kind people in #erlang have given me some suggestions, but I'm going to write here to appeal to a wider audience. I've got a huge erl_crash.dump, that's larger than 2 gigs, and I'm trying to figure out anything I can about the crash, which came out of the blue. Slogan: eheap_alloc: Cannot allocate 1080371408 bytes of memory (of type "heap_frag"). System version: Erlang R16B03-1 (erts-5.10.4) [source] [64-bit] [smp:4:4] [async-threads:10] [kernel-poll:true] Compiled: Sun Mar 16 05:25:57 2014 Taints: crypto Digging further, I found this: =proc:<0.6.0> State: Running Name: error_logger Spawned as: proc_lib:init_p/5 Last scheduled in for: gen_event:handle_msg/5 Spawned by: <0.2.0> Started: Wed Jul 2 11:45:41 2014 Message queue length: 1 Number of heap fragments: 0 Heap fragment data: 0 Link list: [<0.0.0>, <0.98.0>, <0.32.0>] Reductions: 19480050 Stack+heap: 137319567 OldHeap: 28690 Heap unused: 2273119 OldHeap unused: 2581 Memory: 1098789880 Program counter: 0x00007f1e51f3ed70 (gen_event:handle_msg/5 + 8) CP: 0x0000000000000000 (invalid) So that's what actually blew things up. But how did it get all that memory? =proc_dictionary:<0.6.0> H7F1E4D062F68 H7F1E4D062F80 =proc_stack:<0.6.0> 0x00007f1dccc223a0:SReturn addr 0x5204B398 (gen_server:do_cast/2 + 128) y0:H7F1DCBACA990 y1:AF:lager_crash_log y2:SCatch 0x5204D188 (gen_server:do_send/2 + 112) 0x00007f1dccc223c0:SReturn addr 0x4F2936C8 (error_logger_lager_h:log_event/2 + 10328) 0x00007f1dccc223c8:SReturn addr 0x51F422F0 (gen_event:server_update/4 + 272) y0:N y1:N y2:N y3:N y4:A8:emulator y5:H7F1DCBACA8C8 y6:H7F1DCBACA888 y7:H7F1DCBACA930 0x00007f1dccc22410:SReturn addr 0x51F41ED0 (gen_event:server_notify/4 + 136) y0:AC:error_logger y1:H7F1DCBACA8F8 y2:H7F1D8B478038 y3:H7F1D8B478068 y4:A14:error_logger_lager_h y5:SCatch 0x51F422F0 (gen_event:server_update/4 + 272) 0x00007f1dccc22448:SReturn addr 0x51F3EE68 (gen_event:handle_msg/5 + 256) y0:AC:error_logger y1:AC:handle_event y2:H7F1DCBACA8F8 y3:N 0x00007f1dccc22470:SReturn addr 0x51F359B0 (proc_lib:init_p_do_apply/3 + 56) y0:N y1:AC:error_logger y2:P<0.2.0> 0x00007f1dccc22490:SReturn addr 0x842688 () y0:SCatch 0x51F359D0 (proc_lib:init_p_do_apply/3 + 88) The strack trace seems to indicate that it's trying to log something; perhaps someone sent it a very large message? But I wonder where it came from in the first place... I tried using crashdump_viewer, but it chokes when I click on the process and it tries to load up the enormous =proc_heap section: =proc_heap:<0.6.0> 7F1DCBACA990:t2:A9:$gen_cast,H7F1DCBACA978 7F1DCBACA978:t2:A3:log,H7F1DCBACA8F8 7F1DCBACA8F8:t3:A5:error,A6:noproc,H7F1DCBACA8D8 7F1DCBACA8D8:t3:A8:emulator,H7F1DCBACA8C8,H7F1DCBACA888 7F1DCBACA888:lH7F1DCBACA878|N 7F1DCBACA878:lI39|H7F1DCBACA868 7F1DCBACA868:lI103|H7F1DCBACA858 7F1DCBACA858:lI115|H7F1DCBACA848 7F1DCBACA848:lI100|H7F1DCBACA838 7F1DCBACA838:lI95|H7F1DCBACA828 7F1DCBACA828:lI119|H7F1DCBACA818 7F1DCBACA818:lI101|H7F1DCBACA808 7F1DCBACA808:lI98|H7F1DCBACA7F8 7F1DCBACA7F8:lI64|H7F1DCBACA7E8 7F1DCBACA7E8:lI108|H7F1DCBACA7D8 7F1DCBACA7D8:lI111|H7F1DCBACA7C8 7F1DCBACA7C8:lI99|H7F1DCBACA7B8 7F1DCBACA7B8:lI97|H7F1DCBACA7A8 ... and on and on for thousands of lines ... davidw@REDACTED:~$ grep -n '^=proc_heap' erl_crash.dump 15835:=proc_heap:<0.0.0> 16133:=proc_heap:<0.3.0> 17424:=proc_heap:<0.6.0> 67540816:=proc_heap:<0.7.0> Incidentally, what *are* all those lines like 7F1DCBACA7F8:lI64|H7F1DCBACA7E8 anyway? Is there any way to hack something up that will process those 67 million lines to tell me something useful about what's going on? Other ideas about how to extract something meaningful about who plopped this massive message in the logger? Thank you -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From rtrlists@REDACTED Thu Jul 3 15:43:53 2014 From: rtrlists@REDACTED (Robert Raschke) Date: Thu, 3 Jul 2014 14:43:53 +0100 Subject: [erlang-questions] Huge erl_crash.dump (2 gigs) - looking for advice In-Reply-To: References: Message-ID: Hi David, one thing to be aware of when using sasl and error_logger is that process crashes get logged by sasl with the complete state of the process that just died, and error_logger tries to pretty print that state; this can take huge amounts of memory if the crashed process state is large. I would recommend monitoring processes with high memory usage and figuring out the usage patterns. In general it is a good idea to try and limit the amount of state a process holds. But this is obviously application dependent. Hope this helps point in the right direction, Robby On 3 July 2014 14:22, David Welton wrote: > The kind people in #erlang have given me some suggestions, but I'm > going to write here to appeal to a wider audience. I've got a huge > erl_crash.dump, that's larger than 2 gigs, and I'm trying to figure > out anything I can about the crash, which came out of the blue. > > Slogan: eheap_alloc: Cannot allocate 1080371408 bytes of memory (of > type "heap_frag"). > System version: Erlang R16B03-1 (erts-5.10.4) [source] [64-bit] > [smp:4:4] [async-threads:10] [kernel-poll:true] > Compiled: Sun Mar 16 05:25:57 2014 > Taints: crypto > > Digging further, I found this: > > =proc:<0.6.0> > State: Running > Name: error_logger > Spawned as: proc_lib:init_p/5 > Last scheduled in for: gen_event:handle_msg/5 > Spawned by: <0.2.0> > Started: Wed Jul 2 11:45:41 2014 > Message queue length: 1 > Number of heap fragments: 0 > Heap fragment data: 0 > Link list: [<0.0.0>, <0.98.0>, <0.32.0>] > Reductions: 19480050 > Stack+heap: 137319567 > OldHeap: 28690 > Heap unused: 2273119 > OldHeap unused: 2581 > Memory: 1098789880 > Program counter: 0x00007f1e51f3ed70 (gen_event:handle_msg/5 + 8) > CP: 0x0000000000000000 (invalid) > > So that's what actually blew things up. But how did it get all that > memory? > > =proc_dictionary:<0.6.0> > H7F1E4D062F68 > H7F1E4D062F80 > =proc_stack:<0.6.0> > 0x00007f1dccc223a0:SReturn addr 0x5204B398 (gen_server:do_cast/2 + 128) > y0:H7F1DCBACA990 > y1:AF:lager_crash_log > y2:SCatch 0x5204D188 (gen_server:do_send/2 + 112) > 0x00007f1dccc223c0:SReturn addr 0x4F2936C8 > (error_logger_lager_h:log_event/2 + 10328) > 0x00007f1dccc223c8:SReturn addr 0x51F422F0 (gen_event:server_update/4 + > 272) > y0:N > y1:N > y2:N > y3:N > y4:A8:emulator > y5:H7F1DCBACA8C8 > y6:H7F1DCBACA888 > y7:H7F1DCBACA930 > 0x00007f1dccc22410:SReturn addr 0x51F41ED0 (gen_event:server_notify/4 + > 136) > y0:AC:error_logger > y1:H7F1DCBACA8F8 > y2:H7F1D8B478038 > y3:H7F1D8B478068 > y4:A14:error_logger_lager_h > y5:SCatch 0x51F422F0 (gen_event:server_update/4 + 272) > 0x00007f1dccc22448:SReturn addr 0x51F3EE68 (gen_event:handle_msg/5 + 256) > y0:AC:error_logger > y1:AC:handle_event > y2:H7F1DCBACA8F8 > y3:N > 0x00007f1dccc22470:SReturn addr 0x51F359B0 (proc_lib:init_p_do_apply/3 + > 56) > y0:N > y1:AC:error_logger > y2:P<0.2.0> > 0x00007f1dccc22490:SReturn addr 0x842688 () > y0:SCatch 0x51F359D0 (proc_lib:init_p_do_apply/3 + 88) > > The strack trace seems to indicate that it's trying to log something; > perhaps someone sent it a very large message? But I wonder where it > came from in the first place... > > I tried using crashdump_viewer, but it chokes when I click on the > process and it tries to load up the enormous =proc_heap section: > > =proc_heap:<0.6.0> > 7F1DCBACA990:t2:A9:$gen_cast,H7F1DCBACA978 > 7F1DCBACA978:t2:A3:log,H7F1DCBACA8F8 > 7F1DCBACA8F8:t3:A5:error,A6:noproc,H7F1DCBACA8D8 > 7F1DCBACA8D8:t3:A8:emulator,H7F1DCBACA8C8,H7F1DCBACA888 > 7F1DCBACA888:lH7F1DCBACA878|N > 7F1DCBACA878:lI39|H7F1DCBACA868 > 7F1DCBACA868:lI103|H7F1DCBACA858 > 7F1DCBACA858:lI115|H7F1DCBACA848 > 7F1DCBACA848:lI100|H7F1DCBACA838 > 7F1DCBACA838:lI95|H7F1DCBACA828 > 7F1DCBACA828:lI119|H7F1DCBACA818 > 7F1DCBACA818:lI101|H7F1DCBACA808 > 7F1DCBACA808:lI98|H7F1DCBACA7F8 > 7F1DCBACA7F8:lI64|H7F1DCBACA7E8 > 7F1DCBACA7E8:lI108|H7F1DCBACA7D8 > 7F1DCBACA7D8:lI111|H7F1DCBACA7C8 > 7F1DCBACA7C8:lI99|H7F1DCBACA7B8 > 7F1DCBACA7B8:lI97|H7F1DCBACA7A8 > ... and on and on for thousands of lines ... > > davidw@REDACTED:~$ grep -n '^=proc_heap' erl_crash.dump > 15835:=proc_heap:<0.0.0> > 16133:=proc_heap:<0.3.0> > 17424:=proc_heap:<0.6.0> > 67540816:=proc_heap:<0.7.0> > > > Incidentally, what *are* all those lines like > > 7F1DCBACA7F8:lI64|H7F1DCBACA7E8 > > anyway? > > Is there any way to hack something up that will process those 67 > million lines to tell me something useful about what's going on? > > Other ideas about how to extract something meaningful about who > plopped this massive message in the logger? > > Thank you > -- > David N. Welton > > http://www.welton.it/davidw/ > > http://www.dedasys.com/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon@REDACTED Thu Jul 3 15:45:22 2014 From: jon@REDACTED (Jon Schneider) Date: Thu, 3 Jul 2014 14:45:22 +0100 Subject: [erlang-questions] Huge erl_crash.dump (2 gigs) - looking for advice In-Reply-To: References: Message-ID: <2626fd47d089541d46c7dc96c2d7ba15.squirrel@mail.jschneider.net> > The kind people in #erlang have given me some suggestions, but I'm > going to write here to appeal to a wider audience. I've got a huge > erl_crash.dump, that's larger than 2 gigs Assuming for a second that this doesn't contain actual dumps of data it feels like it might contain a long list of _somethings_ . Can you say (by looking at a few random points) what something is ? Jon From ivan@REDACTED Thu Jul 3 15:52:05 2014 From: ivan@REDACTED (Ivan Uemlianin) Date: Thu, 03 Jul 2014 14:52:05 +0100 Subject: [erlang-questions] Uniqueness Typing for Resource Management in Message-Passing Concurrency In-Reply-To: <20140703124738.GA51530@ferdair.local> References: <53B52B89.8050100@llaisdy.com> <20140703124738.GA51530@ferdair.local> Message-ID: <53B56005.7080407@llaisdy.com> Good points thanks. On 03/07/2014 13:47, Fred Hebert wrote: > On 07/03, Ivan Uemlianin wrote: >> Dear All >> >> Saw this and thought of you. >> >> Uniqueness Typing for Resource Management in Message-Passing Concurrency >> http://arxiv.org/abs/1003.5513 >> (submitted 2010) >> >> Mentions erlang in passing. Here is the abstract: >> >> We view channels as the main form of resources in a message-passing >> programming paradigm. These channels need to be carefully managed in >> settings where resources are scarce. To study this problem, we extend the >> pi-calculus with primitives for channel allocation and deallocation and >> allow channels to be reused to communicate values of different types. >> Inevitably, the added expressiveness increases the possibilities for runtime >> errors. We define a substructural type system which combines uniqueness >> typing and affine typing to reject these ill-behaved programs. >> >> > > This sounds like it would be nicer for a language with actual channels. > There are a few differences with Pi-Calculus and Erlang: > > - Pi-Calculus assumes synchronous message delivery, Erlang doesn't > - Pi-Calculus assumes the usage of channels that are nameable and > passable as if they were variables, Erlang has a per-process mailbox > (you can pass pids as if they were channels and treat specific > processes as channels, I guess) > - Channels can be created at any time by anyone > - More. > > Interestingly enough, there were still attempts to model a subset Erlang > as pi-calculus for verification purposes (they did need to make an async > pi-calc I believe): > http://www.cs.usask.ca/~croy/papers/2006/Erlang06_Roy.pdf > > Regards, > Fred. > -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From i.vysniauskas@REDACTED Thu Jul 3 16:01:24 2014 From: i.vysniauskas@REDACTED (=?UTF-8?Q?Ignas_Vy=C5=A1niauskas?=) Date: Thu, 3 Jul 2014 16:01:24 +0200 Subject: [erlang-questions] Uniqueness Typing for Resource Management in Message-Passing Concurrency In-Reply-To: <20140703124738.GA51530@ferdair.local> References: <53B52B89.8050100@llaisdy.com> <20140703124738.GA51530@ferdair.local> Message-ID: Hi Fred, I agree on your overall point that channel-based formalisms do not map well to Erlang. A few more points: On Thu, Jul 3, 2014 at 2:47 PM, Fred Hebert wrote: > This sounds like it would be nicer for a language with actual channels. > There are a few differences with Pi-Calculus and Erlang: > > - Pi-Calculus assumes synchronous message delivery, Erlang doesn't Just like there are many variants lambda-calculi, there are many variants of pi-calculi. Some of them synchronous, some of them not. > - Pi-Calculus assumes the usage of channels that are nameable and > passable as if they were variables, Erlang has a per-process mailbox > (you can pass pids as if they were channels and treat specific > processes as channels, I guess) You could indeed split up each process into multiple sub-processes where each of them corresponds to a particular pi-calculus channel. Then you can "kind of" do channel delegation by sending pids. However the caveat is that in Erlang, receiving a pid only means that you can send things to it. This is like knowing someone's email address. In pi-calculus receiving a channel's name means that you can send and *receive* on that channel. This is more like getting a pointer to a queue in shared memory, to which you can now both pop and push. This latter is IMHO a crucial difference and makes the usual pi-calculi unsuitable (or at least unworkable) for modeling Erlang semantics. I think a much better formalism for actor-like systems is the join-calculus. There is even a project which extends Erlang with join-patterns called JErlang, very cool stuff: http://www.doc.ic.ac.uk/~susan/jerlang/ -- Ignas From davidnwelton@REDACTED Thu Jul 3 16:08:43 2014 From: davidnwelton@REDACTED (David Welton) Date: Thu, 3 Jul 2014 16:08:43 +0200 Subject: [erlang-questions] Huge erl_crash.dump (2 gigs) - looking for advice In-Reply-To: <2626fd47d089541d46c7dc96c2d7ba15.squirrel@mail.jschneider.net> References: <2626fd47d089541d46c7dc96c2d7ba15.squirrel@mail.jschneider.net> Message-ID: >> The kind people in #erlang have given me some suggestions, but I'm >> going to write here to appeal to a wider audience. I've got a huge >> erl_crash.dump, that's larger than 2 gigs > > Assuming for a second that this doesn't contain actual dumps of data it > feels like it might contain a long list of _somethings_ . Can you say (by > looking at a few random points) what something is ? I'm not sure I follow... I don't know what the proc_heap section contains - I don't know what all the lines in it mean. Robert writes: > one thing to be aware of when using sasl and error_logger is that process crashes get logged by sasl with the complete state of the process that just died, and error_logger tries to pretty print that state; this can take huge amounts of memory if the crashed process state is large. > I would recommend monitoring processes with high memory usage and figuring out the usage patterns. In general it is a good idea to try and limit the amount of state a process holds. But this is obviously application dependent. I don't think any of the processes grew in size so much that it was anywhere near the kind of size needed to trigger an out of memory error were its stated dumped and printed. We'll keep an eye on them just in case one of them is stealthily growing, but for the moment we have not been able to reproduce the error. Thanks -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From jan.chochol@REDACTED Thu Jul 3 16:34:47 2014 From: jan.chochol@REDACTED (Jan Chochol) Date: Thu, 3 Jul 2014 16:34:47 +0200 Subject: [erlang-questions] Huge erl_crash.dump (2 gigs) - looking for advice In-Reply-To: References: <2626fd47d089541d46c7dc96c2d7ba15.squirrel@mail.jschneider.net> Message-ID: Hi David, I think this is quite common pattern - some process wants to log some huge details (but not fatally "large"), but error logger crashed, as it wants to convert all data to strings (which can consume much more memory). It happened to us several times. proc_heap section contains dump of process heap. I am not exactly sure about structure, but I think it is :. Part of your dump: 7F1DCBACA878:lI39|H7F1DCBACA868 7F1DCBACA868:lI103|H7F1DCBACA858 7F1DCBACA858:lI115|H7F1DCBACA848 7F1DCBACA848:lI100|H7F1DCBACA838 7F1DCBACA838:lI95|H7F1DCBACA828 7F1DCBACA828:lI119|H7F1DCBACA818 7F1DCBACA818:lI101|H7F1DCBACA808 7F1DCBACA808:lI98|H7F1DCBACA7F8 7F1DCBACA7F8:lI64|H7F1DCBACA7E8 7F1DCBACA7E8:lI108|H7F1DCBACA7D8 7F1DCBACA7D8:lI111|H7F1DCBACA7C8 7F1DCBACA7C8:lI99|H7F1DCBACA7B8 7F1DCBACA7B8:lI97|H7F1DCBACA7A8 Is part of some list. Type "l" is cons cell (basic construction item of lists), which contains tuple :. Value is of type "I" (small integer - representing decimal character code), and next item is of type "H" - pointer to next data. Most simple way is to expect, that data forms list in order of memory cells (which is quite often). You can e.g. use this simple Perl code to get some idea about data (what error logger want to display): $ perl -n -e 'print chr($1) if($_ =~ /:lI([0-9]*)\|/)' < part_of_crash_dump 'gsd_web@REDACTED Where file "part_of_crash_dump" contains data from proc_heap section. I am sure there are more sophisticated tools to analyze crash dumps, but this can be used for the first sight. Regards, Jan On Thu, Jul 3, 2014 at 4:08 PM, David Welton wrote: > >> The kind people in #erlang have given me some suggestions, but I'm > >> going to write here to appeal to a wider audience. I've got a huge > >> erl_crash.dump, that's larger than 2 gigs > > > > Assuming for a second that this doesn't contain actual dumps of data it > > feels like it might contain a long list of _somethings_ . Can you say (by > > looking at a few random points) what something is ? > > I'm not sure I follow... I don't know what the proc_heap section > contains - I don't know what all the lines in it mean. > > Robert writes: > > one thing to be aware of when using sasl and error_logger is that > process crashes get logged by sasl with the complete state of the process > that just died, and error_logger tries to pretty print that state; this can > take huge amounts of memory if the crashed process state is large. > > > I would recommend monitoring processes with high memory usage and > figuring out the usage patterns. In general it is a good idea to try and > limit the amount of state a process holds. But this is obviously > application dependent. > > I don't think any of the processes grew in size so much that it was > anywhere near the kind of size needed to trigger an out of memory > error were its stated dumped and printed. We'll keep an eye on them > just in case one of them is stealthily growing, but for the moment we > have not been able to reproduce the error. > > Thanks > -- > David N. Welton > > http://www.welton.it/davidw/ > > http://www.dedasys.com/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidnwelton@REDACTED Thu Jul 3 16:48:55 2014 From: davidnwelton@REDACTED (David Welton) Date: Thu, 3 Jul 2014 16:48:55 +0200 Subject: [erlang-questions] Huge erl_crash.dump (2 gigs) - looking for advice In-Reply-To: References: <2626fd47d089541d46c7dc96c2d7ba15.squirrel@mail.jschneider.net> Message-ID: > proc_heap section contains dump of process heap. I am not exactly sure about > structure, but I think it is :. > Part of your dump: > > 7F1DCBACA878:lI39|H7F1DCBACA868 > 7F1DCBACA868:lI103|H7F1DCBACA858 > 7F1DCBACA858:lI115|H7F1DCBACA848 > 7F1DCBACA848:lI100|H7F1DCBACA838 > 7F1DCBACA838:lI95|H7F1DCBACA828 > 7F1DCBACA828:lI119|H7F1DCBACA818 > 7F1DCBACA818:lI101|H7F1DCBACA808 > 7F1DCBACA808:lI98|H7F1DCBACA7F8 > 7F1DCBACA7F8:lI64|H7F1DCBACA7E8 > 7F1DCBACA7E8:lI108|H7F1DCBACA7D8 > 7F1DCBACA7D8:lI111|H7F1DCBACA7C8 > 7F1DCBACA7C8:lI99|H7F1DCBACA7B8 > 7F1DCBACA7B8:lI97|H7F1DCBACA7A8 > > Is part of some list. Type "l" is cons cell (basic construction item of > lists), which contains tuple :. Value is of type "I" > (small integer - representing decimal character code), and next item is of > type "H" - pointer to next data. > Most simple way is to expect, that data forms list in order of memory cells > (which is quite often). You can e.g. use this simple Perl code to get some > idea about data (what error logger want to display): > > $ perl -n -e 'print chr($1) if($_ =~ /:lI([0-9]*)\|/)' < part_of_crash_dump > 'gsd_web@REDACTED > > Where file "part_of_crash_dump" contains data from proc_heap section. > I am sure there are more sophisticated tools to analyze crash dumps, but > this can be used for the first sight. Your explanation and Perl code were very useful. I feel a bit like a character one of those children's detective novels where they finally crack the secret code:-) 'gsd_web@REDACTED' got a corrupted external term from 'xyz@REDACTED' on distribution channel 12877 That really narrows things down and I think ought to be enough to solve our problem. Thank you very much! -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From jon@REDACTED Thu Jul 3 16:50:59 2014 From: jon@REDACTED (Jon Schneider) Date: Thu, 3 Jul 2014 15:50:59 +0100 Subject: [erlang-questions] Huge erl_crash.dump (2 gigs) - looking for advice In-Reply-To: References: <2626fd47d089541d46c7dc96c2d7ba15.squirrel@mail.jschneider.net> Message-ID: > 7F1DCBACA7B8:lI97|H7F1DCBACA7A8 > 'gsd_web@REDACTED The 'g' is of course 97h. I'll just insert my general whinge NOT specifically about Erlang but more about link lists where the size of the overhead _dwarfs_ the data itself. Also about any code allocating from a heap something less than several times the size of a pointer. Back when pointers were sixteen bits things were bad enough but storing strings in (linked) lists where pointers are sixty four bits and the minimum overhead for allocating _something_ on any unconstrained heap is the size of one pointer is ludicrous. I could go on about how the move to 64-bit computing has got us wasting even more memory even though in practice almost nothing needs it. In short and for Erlang use binaries wherever possible (bearing in mind it precludes storing Unicode directly but not when UTF-8 encoded). Jon From felixgallo@REDACTED Thu Jul 3 19:18:44 2014 From: felixgallo@REDACTED (Felix Gallo) Date: Thu, 3 Jul 2014 10:18:44 -0700 Subject: [erlang-questions] shared records in .hrl files Message-ID: Consider a system that receives application messages in an extensibly large number of formats (e.g. json, xml, yaml, pictograms, esperanto). This system wishes to translate the messages into a unified data format, do various centralized bookkeeping tasks, and then pass those normalized messages off to workers for further processing. The system further distrusts the application messages (and the format conversion libraries) and so wishes to spawn a worker for each message and let it crash if nonsense or tomfoolery is involved. A picture: Queue of Incoming Messages -> [Pool of Specialized Translators] -> [Processing Manager] -> [Pool of Workers] The "natural" approach from a C / OO background is to define a layout for the unified format, perhaps as an object or struct, and share that across the specialized translators, the processing manager, and the worker processes. And indeed that solves the problem fairly neatly here as well, except you can't walk the streets of Los Angeles without tripping over a hobo brought to desperate circumstances who regales you with his woeful cautionary tale of attempting to perform a live upgrade with a record definition shared across multiple modules in an hrl file. After pondering it, it seems like you can choose at most two of { live upgrades, separation into multiple processes, simple and sensible records }. Needing to choose live upgrades and separation into multiple processes, one then starts thinking of bloodcurdling ways to handle the shared record issue, including: 1. Treat record formats as immutable and just keep adding them ('mystate1', 'mystate2', ... 'mystate11'...) along with all of the accompanied handling. 2. Keep a version field in every record format and start them out being as big as you might ever want them to get. 3. Clone the OTP source repository and hack in record upgrade messages. I'd be interested in hearing if anyone has any better ideas, or how others have handled this scenario in real world conditions. F. -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonym@REDACTED Thu Jul 3 20:02:56 2014 From: anthonym@REDACTED (ANTHONY MOLINARO) Date: Thu, 3 Jul 2014 11:02:56 -0700 Subject: [erlang-questions] shared records in .hrl files In-Reply-To: References: Message-ID: <02FDB237-E8AA-4F3C-9969-0E5179CB730A@alumni.caltech.edu> You could hide the record implementation inside a module, then only the module would have to know about differences between versions. All the sub-systems use the functional interface of the module and the record is just passed in as the first argument (this is similar to how one would use struct?s in C). So for instance -module (widget). -record (widget_v1, { field1 }). -record (widget_v2, { field1, field2 }). -export ([new/0, get/2, set/3]). new () -> #widget_v2 {}. get (#widget_v1 { field1 = Field1 }, field1) -> Field1; get (#widget_v2 { field1 = Field1 }, field1) -> Field1; get (#widget_v1 {}, field2) -> undefined; get (#widget_v2 { field2 = Field2 }, field2) -> Field2.. set (Widget = #widget_v1 {}, field1, Field1) -> Widget#widget_v1 { field1 = Field1 }; set (Widget = #widget_v2 {}, field1, Field1) -> Widget#widget_v2 { field1 = Field1 }; set (Widget = #widget_v1 {}, field2, Field2) -> set (convert (Widget), field2, Field2); % automatically convert v1 to v2 here set (Widget = #widget_v2 {}, field2, Field2) -> Widget#widget_v2 { field2 = Field2 }. convert (#widget_v1 { field1 = Field1 }) -> #widget_v2 { field1 = Field1 }. The different components could just do things like Widget = widget:set (widget:new(), field1, ?foo?), widget:get (Widget, field1), ? Then if you need to add fields the actual implementation is hidden. If you store a previous version of a record you can still pass it into the get/set functions and get back values or defaults. All upgrade logic could be hidden as in the above example where a record is converted when you try to set a new field in an old version. Now over time you could end up with many versions of your record, but all the logic for the different versions is hidden inside the module interface. It?s maybe a bit OO, and I?d be curious if someone has a better idea that is more functional in nature. -Anthony On Jul 3, 2014, at 10:18 AM, Felix Gallo wrote: > Consider a system that receives application messages in an extensibly large number of formats (e.g. json, xml, yaml, pictograms, esperanto). This system wishes to translate the messages into a unified data format, do various centralized bookkeeping tasks, and then pass those normalized messages off to workers for further processing. The system further distrusts the application messages (and the format conversion libraries) and so wishes to spawn a worker for each message and let it crash if nonsense or tomfoolery is involved. > > A picture: > > Queue of Incoming Messages -> [Pool of Specialized Translators] -> [Processing Manager] -> [Pool of Workers] > > The "natural" approach from a C / OO background is to define a layout for the unified format, perhaps as an object or struct, and share that across the specialized translators, the processing manager, and the worker processes. And indeed that solves the problem fairly neatly here as well, except you can't walk the streets of Los Angeles without tripping over a hobo brought to desperate circumstances who regales you with his woeful cautionary tale of attempting to perform a live upgrade with a record definition shared across multiple modules in an hrl file. > > After pondering it, it seems like you can choose at most two of { live upgrades, separation into multiple processes, simple and sensible records }. > > Needing to choose live upgrades and separation into multiple processes, one then starts thinking of bloodcurdling ways to handle the shared record issue, including: > > 1. Treat record formats as immutable and just keep adding them ('mystate1', 'mystate2', ... 'mystate11'...) along with all of the accompanied handling. > 2. Keep a version field in every record format and start them out being as big as you might ever want them to get. > 3. Clone the OTP source repository and hack in record upgrade messages. > > I'd be interested in hearing if anyone has any better ideas, or how others have handled this scenario in real world conditions. > > F. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dmkolesnikov@REDACTED Thu Jul 3 20:07:17 2014 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Thu, 3 Jul 2014 21:07:17 +0300 Subject: [erlang-questions] shared records in .hrl files In-Reply-To: References: Message-ID: <816162C8-F836-4F20-AA61-087D61654C31@gmail.com> Hello, I am afraid that massive amount of record definition will complicate the system. Have you considered the use of map or key/val lists as unified format. - Dmitry On 03 Jul 2014, at 20:18, Felix Gallo wrote: > Consider a system that receives application messages in an extensibly large number of formats (e.g. json, xml, yaml, pictograms, esperanto). This system wishes to translate the messages into a unified data format, do various centralized bookkeeping tasks, and then pass those normalized messages off to workers for further processing. The system further distrusts the application messages (and the format conversion libraries) and so wishes to spawn a worker for each message and let it crash if nonsense or tomfoolery is involved. > > A picture: > > Queue of Incoming Messages -> [Pool of Specialized Translators] -> [Processing Manager] -> [Pool of Workers] > > The "natural" approach from a C / OO background is to define a layout for the unified format, perhaps as an object or struct, and share that across the specialized translators, the processing manager, and the worker processes. And indeed that solves the problem fairly neatly here as well, except you can't walk the streets of Los Angeles without tripping over a hobo brought to desperate circumstances who regales you with his woeful cautionary tale of attempting to perform a live upgrade with a record definition shared across multiple modules in an hrl file. > > After pondering it, it seems like you can choose at most two of { live upgrades, separation into multiple processes, simple and sensible records }. > > Needing to choose live upgrades and separation into multiple processes, one then starts thinking of bloodcurdling ways to handle the shared record issue, including: > > 1. Treat record formats as immutable and just keep adding them ('mystate1', 'mystate2', ... 'mystate11'...) along with all of the accompanied handling. > 2. Keep a version field in every record format and start them out being as big as you might ever want them to get. > 3. Clone the OTP source repository and hack in record upgrade messages. > > I'd be interested in hearing if anyone has any better ideas, or how others have handled this scenario in real world conditions. > > F. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mattevans123@REDACTED Thu Jul 3 21:45:14 2014 From: mattevans123@REDACTED (Matthew Evans) Date: Thu, 3 Jul 2014 15:45:14 -0400 Subject: [erlang-questions] enif_free and free in a NIF Message-ID: Hi Team, We have a number of nifs that make use of enif_alloc and enif_free (as well as other calls such as erl_mk_tuple, erl_free_compound). The nifs also include a third party library that makes regular calls to malloc and free. We have had a number of fairly rare core files (e.g. abort in free) from the third party library. The question is is it safe to mix and match the Erlang memory allocation/deallocation NIF libraries with the standard Linux calls to free and malloc? Thanks Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Thu Jul 3 22:35:17 2014 From: felixgallo@REDACTED (Felix Gallo) Date: Thu, 3 Jul 2014 13:35:17 -0700 Subject: [erlang-questions] shared records in .hrl files In-Reply-To: <816162C8-F836-4F20-AA61-087D61654C31@gmail.com> References: <816162C8-F836-4F20-AA61-087D61654C31@gmail.com> Message-ID: I'd been waiting for maps to settle down a bit before I looked at them too hard, but that's not a bad idea. It appears the dialyzer support in eep 43 is already here in 17, which eases my concerns about contracts and documentability. If only there were a way to limit the keys to a defined list. Thanks for the idea, Dmitry! On Thu, Jul 3, 2014 at 11:07 AM, Dmitry Kolesnikov wrote: > Hello, > > I am afraid that massive amount of record definition will complicate the > system. > Have you considered the use of map or key/val lists as unified format. > > - Dmitry > > > On 03 Jul 2014, at 20:18, Felix Gallo wrote: > > > Consider a system that receives application messages in an extensibly > large number of formats (e.g. json, xml, yaml, pictograms, esperanto). > This system wishes to translate the messages into a unified data format, > do various centralized bookkeeping tasks, and then pass those normalized > messages off to workers for further processing. The system further > distrusts the application messages (and the format conversion libraries) > and so wishes to spawn a worker for each message and let it crash if > nonsense or tomfoolery is involved. > > > > A picture: > > > > Queue of Incoming Messages -> [Pool of Specialized Translators] -> > [Processing Manager] -> [Pool of Workers] > > > > The "natural" approach from a C / OO background is to define a layout > for the unified format, perhaps as an object or struct, and share that > across the specialized translators, the processing manager, and the worker > processes. And indeed that solves the problem fairly neatly here as well, > except you can't walk the streets of Los Angeles without tripping over a > hobo brought to desperate circumstances who regales you with his woeful > cautionary tale of attempting to perform a live upgrade with a record > definition shared across multiple modules in an hrl file. > > > > After pondering it, it seems like you can choose at most two of { live > upgrades, separation into multiple processes, simple and sensible records }. > > > > Needing to choose live upgrades and separation into multiple processes, > one then starts thinking of bloodcurdling ways to handle the shared record > issue, including: > > > > 1. Treat record formats as immutable and just keep adding them > ('mystate1', 'mystate2', ... 'mystate11'...) along with all of the > accompanied handling. > > 2. Keep a version field in every record format and start them out being > as big as you might ever want them to get. > > 3. Clone the OTP source repository and hack in record upgrade messages. > > > > I'd be interested in hearing if anyone has any better ideas, or how > others have handled this scenario in real world conditions. > > > > F. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Fri Jul 4 00:06:22 2014 From: lukas@REDACTED (Lukas Larsson) Date: Fri, 4 Jul 2014 00:06:22 +0200 Subject: [erlang-questions] enif_free and free in a NIF In-Reply-To: References: Message-ID: Hello, On Thu, Jul 3, 2014 at 9:45 PM, Matthew Evans wrote: > > The question is is it safe to mix and match the Erlang memory > allocation/deallocation NIF libraries with the standard Linux calls to free > and malloc? > > Depends on what you mean by mix and match. It is ok to allocate with both malloc and enif_alloc in a nif as long as you deallocate with free and enif_free respectively. It does not work to allocate with enif_alloc and then deallocate with free. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From siraaj@REDACTED Fri Jul 4 01:02:18 2014 From: siraaj@REDACTED (Siraaj Khandkar) Date: Thu, 03 Jul 2014 19:02:18 -0400 Subject: [erlang-questions] shared records in .hrl files In-Reply-To: <02FDB237-E8AA-4F3C-9969-0E5179CB730A@alumni.caltech.edu> References: <02FDB237-E8AA-4F3C-9969-0E5179CB730A@alumni.caltech.edu> Message-ID: <53B5E0FA.90108@khandkar.net> +1 This is just abstraction, and is best practice in general - nothing unfunctional or particularly OOPie about it :) On 07/03/2014 02:02 PM, ANTHONY MOLINARO wrote: > You could hide the record implementation inside a module, then only the module would have to know about differences between versions. All the sub-systems use the functional interface of the module and the record is just passed in as the first argument (this is similar to how one would use struct?s in C). > > So for instance > > -module (widget). > -record (widget_v1, { field1 }). > -record (widget_v2, { field1, field2 }). > -export ([new/0, get/2, set/3]). > > new () -> > #widget_v2 {}. > > get (#widget_v1 { field1 = Field1 }, field1) -> > Field1; > get (#widget_v2 { field1 = Field1 }, field1) -> > Field1; > get (#widget_v1 {}, field2) -> > undefined; > get (#widget_v2 { field2 = Field2 }, field2) -> > Field2.. > > set (Widget = #widget_v1 {}, field1, Field1) -> > Widget#widget_v1 { field1 = Field1 }; > set (Widget = #widget_v2 {}, field1, Field1) -> > Widget#widget_v2 { field1 = Field1 }; > set (Widget = #widget_v1 {}, field2, Field2) -> > set (convert (Widget), field2, Field2); % automatically convert v1 to v2 here > set (Widget = #widget_v2 {}, field2, Field2) -> > Widget#widget_v2 { field2 = Field2 }. > > convert (#widget_v1 { field1 = Field1 }) -> > #widget_v2 { field1 = Field1 }. > > The different components could just do things like > > Widget = widget:set (widget:new(), field1, ?foo?), > widget:get (Widget, field1), > ? > > Then if you need to add fields the actual implementation is hidden. > > If you store a previous version of a record you can still pass it into the get/set functions and get back values or defaults. All upgrade logic could be hidden as in the above example where a record is converted when you try to set a new field in an old version. Now over time you could end up with many versions of your record, but all the logic for the different versions is hidden inside the module interface. > > It?s maybe a bit OO, and I?d be curious if someone has a better idea that is more functional in nature. > > -Anthony > > > On Jul 3, 2014, at 10:18 AM, Felix Gallo wrote: > >> Consider a system that receives application messages in an extensibly large number of formats (e.g. json, xml, yaml, pictograms, esperanto). This system wishes to translate the messages into a unified data format, do various centralized bookkeeping tasks, and then pass those normalized messages off to workers for further processing. The system further distrusts the application messages (and the format conversion libraries) and so wishes to spawn a worker for each message and let it crash if nonsense or tomfoolery is involved. >> >> A picture: >> >> Queue of Incoming Messages -> [Pool of Specialized Translators] -> [Processing Manager] -> [Pool of Workers] >> >> The "natural" approach from a C / OO background is to define a layout for the unified format, perhaps as an object or struct, and share that across the specialized translators, the processing manager, and the worker processes. And indeed that solves the problem fairly neatly here as well, except you can't walk the streets of Los Angeles without tripping over a hobo brought to desperate circumstances who regales you with his woeful cautionary tale of attempting to perform a live upgrade with a record definition shared across multiple modules in an hrl file. >> >> After pondering it, it seems like you can choose at most two of { live upgrades, separation into multiple processes, simple and sensible records }. >> >> Needing to choose live upgrades and separation into multiple processes, one then starts thinking of bloodcurdling ways to handle the shared record issue, including: >> >> 1. Treat record formats as immutable and just keep adding them ('mystate1', 'mystate2', ... 'mystate11'...) along with all of the accompanied handling. >> 2. Keep a version field in every record format and start them out being as big as you might ever want them to get. >> 3. Clone the OTP source repository and hack in record upgrade messages. >> >> I'd be interested in hearing if anyone has any better ideas, or how others have handled this scenario in real world conditions. >> >> F. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mattevans123@REDACTED Fri Jul 4 02:16:12 2014 From: mattevans123@REDACTED (Matthew Evans) Date: Thu, 3 Jul 2014 20:16:12 -0400 Subject: [erlang-questions] enif_free and free in a NIF In-Reply-To: References: , Message-ID: Thanks, I actually just found the cause of the crash. We had gen_server "A" that loaded the NIF, but we also had another gen_server directly calling a nif function owned by gen_server "A" instead of doing a gen_server:call to "A". Date: Fri, 4 Jul 2014 00:06:22 +0200 Subject: Re: [erlang-questions] enif_free and free in a NIF From: lukas@REDACTED To: mattevans123@REDACTED CC: erlang-questions@REDACTED Hello, On Thu, Jul 3, 2014 at 9:45 PM, Matthew Evans wrote: The question is is it safe to mix and match the Erlang memory allocation/deallocation NIF libraries with the standard Linux calls to free and malloc? Depends on what you mean by mix and match. It is ok to allocate with both malloc and enif_alloc in a nif as long as you deallocate with free and enif_free respectively. It does not work to allocate with enif_alloc and then deallocate with free. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Jul 4 06:06:24 2014 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 4 Jul 2014 16:06:24 +1200 Subject: [erlang-questions] Installing 17.1 on a T2 kills it In-Reply-To: References: Message-ID: On 3/07/2014, at 9:14 PM, Lukas Larsson wrote: > On Thu, Jul 3, 2014 at 3:25 AM, Richard A. O'Keefe wrote: > > (1) Has anyone else had this problem? > > Our sunos 5.11 (SunOS 5.11 oi_151a9 i86pc i386 i86pc) machine does not have this problem. I was specifically asking about the T2. Has anyone else had this problem *on a T2*? (Part of the problem may be that the machine is actually a pre-production one. So there may be a hardware issue that nothing else has tickled.) I have OpenSolaris in VirtualBox on my laptop, and Solaris 11 in VirtualBox on my desktop. However, neither of those offers 64 virtual CPUS. > Unfortunately there is no way to do this. Manually you can remove line 1190 to 1222 in erts/configure.in and that should make it skip the test. I removed everything, unpacked the .tar file again, made this change, but something went wrong somehow and the thing went belly up at the same place again. Lectures resume next week so I'm not going to have much time to try to resolve this. From max.lapshin@REDACTED Fri Jul 4 06:08:42 2014 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 4 Jul 2014 08:08:42 +0400 Subject: [erlang-questions] enif_free and free in a NIF In-Reply-To: References: Message-ID: On Fri, Jul 4, 2014 at 4:16 AM, Matthew Evans wrote: > > I actually just found the cause of the crash. We had gen_server "A" that > loaded the NIF, but we also had another gen_server directly calling a nif > function owned by gen_server "A" instead of doing a gen_server:call to "A". > It is not a problem, because process doesn't own a function. Looks like your data structure doesn't survive thread migration. -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Fri Jul 4 08:21:10 2014 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Fri, 4 Jul 2014 08:21:10 +0200 Subject: [erlang-questions] shared records in .hrl files In-Reply-To: References: Message-ID: On Thu, Jul 3, 2014 at 7:18 PM, Felix Gallo wrote: > > After pondering it, it seems like you can choose at most two of { live > upgrades, separation into multiple processes, simple and sensible records }. Hi, if you don't mind the extra benefits (easy sharing, compatibility, compactness) and overhead (yeah, serialization and deserialization take CPU time) of serialization protocols like protocol buffers or Cap'n Proto, I would suggest looking at them. In our systems the "universal language" is protocol buffers (under piqi). In-application they are converted to Erlang records. Upgrades have no compatibility problems, because any data out of the application boundaries is compatibly encoded.. -- Motiejus Jak?tys From davidnwelton@REDACTED Fri Jul 4 12:05:07 2014 From: davidnwelton@REDACTED (David Welton) Date: Fri, 4 Jul 2014 12:05:07 +0200 Subject: [erlang-questions] Huge erl_crash.dump (2 gigs) - looking for advice In-Reply-To: References: <2626fd47d089541d46c7dc96c2d7ba15.squirrel@mail.jschneider.net> Message-ID: > I'll just insert my general whinge NOT specifically about Erlang but more > about link lists where the size of the overhead _dwarfs_ the data itself. Agreed. It looks like this patch might have helped, some: https://github.com/erlang/otp/pull/359 (thanks to Lo?c once again for pointing it out) > Also about any code allocating from a heap something less than several > times the size of a pointer. > > Back when pointers were sixteen bits things were bad enough but storing > strings in (linked) lists where pointers are sixty four bits and the > minimum overhead for allocating _something_ on any unconstrained heap is > the size of one pointer is ludicrous. > > I could go on about how the move to 64-bit computing has got us wasting > even more memory even though in practice almost nothing needs it. It's a bit of a tangent, but I once did some calculations about the costs of that move on things like VPS's where memory is something you pay for: http://journal.dedasys.com/2008/11/24/slicehost-vs-linode/ - it's pretty dated, being from 6 years ago, but I think it did a reasonable job of showing that having all those 64 bit pointers around costs money in certain cases. -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From m3oucat@REDACTED Fri Jul 4 16:38:00 2014 From: m3oucat@REDACTED (ami) Date: Fri, 4 Jul 2014 17:38:00 +0300 Subject: [erlang-questions] A question about erlang shell in Emacs (at startup) Message-ID: Hi Vlad, I?m trying to run erlang shell at Emacs startup. Just to get a some kind of IDE, when I have an erlang shell, an editor window, etc right after startup. And I faced with the same problem, you?d faced before. (see your question @ http://erlang.org/pipermail/erlang-questions/2004-January/011042.html ) So, I start erlang shell by (erlang-shell) in .emacs but it?s not in active buffer. I, of course, can switch to *erlang* buffer afterwards. But I don?t want to switch to *erlang* buffer manually always :( I tried (sit-for 20) right after (erlang-shell), like (erlang-shell) (sit-for 20) And I do have an erlang shell displayed in emacs right after startup. But when I press any key/mouse button, it disappears :( But I can switch to *erlang* buffer manually (it?s not the case). (switch-to-buffer "*erlang*?) doesn?t work as well.(if I place it in .emacs right after (erlang-shell) ) Has anybody got erlang shell displayed & working & active right after emacs startup? Thanks in advance, Alex From m3oucat@REDACTED Fri Jul 4 16:41:40 2014 From: m3oucat@REDACTED (ami) Date: Fri, 4 Jul 2014 17:41:40 +0300 Subject: [erlang-questions] get erlang-shell working, displayed & active right after emacs startup Message-ID: Hi all. I?m trying to run erlang shell at Emacs startup. Just to get a some kind of IDE, when I have an erlang shell, an editor window, etc right after startup. And I faced with the same problem as had been once arisen here (http://erlang.org/pipermail/erlang-questions/2004-January/011042.html ) So, I start erlang shell by (erlang-shell) in .emacs but it?s not in active buffer. I, of course, can switch to *erlang* buffer afterwards. But I don?t want to switch to *erlang* buffer manually always :( I tried (sit-for 20) right after (erlang-shell), like (erlang-shell) (sit-for 20) And I do have an erlang shell displayed in emacs right after startup. But when I press any key/mouse button, it disappears :( But I can switch to *erlang* buffer manually (it?s not the case). (switch-to-buffer "*erlang*?) doesn?t work as well.(if I place it in .emacs right after (erlang-shell) ) Has anybody got erlang shell displayed & working & active right after emacs startup? Thanks in advance, Alex From vladdu55@REDACTED Fri Jul 4 18:41:47 2014 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 4 Jul 2014 18:41:47 +0200 Subject: [erlang-questions] A question about erlang shell in Emacs (at startup) In-Reply-To: References: Message-ID: Hi Alex, Unfortunately I don't have an answer. I don't use Emacs nowadays. regards, Vlad On Fri, Jul 4, 2014 at 4:38 PM, ami wrote: > Hi Vlad, > > I?m trying to run erlang shell at Emacs startup. > Just to get a some kind of IDE, when I have an erlang shell, an editor > window, etc > right after startup. > > And I faced with the same problem, you?d faced before. > (see your question @ > http://erlang.org/pipermail/erlang-questions/2004-January/011042.html ) > > So, I start erlang shell by > > (erlang-shell) > in .emacs > > but it?s not in active buffer. I, of course, can switch to *erlang* buffer > afterwards. > But I don?t want to switch to *erlang* buffer manually always :( > > I tried (sit-for 20) right after (erlang-shell), like > > (erlang-shell) > (sit-for 20) > > And I do have an erlang shell displayed in emacs right after startup. > But when I press any key/mouse button, it disappears :( > But I can switch to *erlang* buffer manually (it?s not the case). > > (switch-to-buffer "*erlang*?) > > doesn?t work as well.(if I place it in .emacs right after (erlang-shell) ) > > Has anybody got erlang shell displayed & working & active right after > emacs startup? > > Thanks in advance, > > Alex > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Fri Jul 4 20:29:04 2014 From: vinoski@REDACTED (Steve Vinoski) Date: Fri, 4 Jul 2014 14:29:04 -0400 Subject: [erlang-questions] get erlang-shell working, displayed & active right after emacs startup In-Reply-To: References: Message-ID: On Fri, Jul 4, 2014 at 10:41 AM, ami wrote: > Hi all. > > I?m trying to run erlang shell at Emacs startup. > Just to get a some kind of IDE, when I have an erlang shell, an editor > window, etc > right after startup. > > And I faced with the same problem as had been once arisen here > > (http://erlang.org/pipermail/erlang-questions/2004-January/011042.html ) > > So, I start erlang shell by > > (erlang-shell) > in .emacs > > but it?s not in active buffer. I, of course, can switch to *erlang* buffer > afterwards. > But I don?t want to switch to *erlang* buffer manually always :( > > I tried (sit-for 20) right after (erlang-shell), like > > (erlang-shell) > (sit-for 20) > > And I do have an erlang shell displayed in emacs right after startup. > But when I press any key/mouse button, it disappears :( > But I can switch to *erlang* buffer manually (it?s not the case). > > (switch-to-buffer "*erlang*?) > > doesn?t work as well.(if I place it in .emacs right after (erlang-shell) ) > > Has anybody got erlang shell displayed & working & active right after > emacs startup? You might try something like this in your ~/.emacs file: (add-hook 'erlang-shell-mode-hook '(lambda () (delete-other-windows (display-buffer "*erlang*")))) (erlang-shell) --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From m3oucat@REDACTED Fri Jul 4 21:51:13 2014 From: m3oucat@REDACTED (ami) Date: Fri, 4 Jul 2014 22:51:13 +0300 Subject: [erlang-questions] get erlang-shell working, displayed & active right after emacs startup In-Reply-To: References: Message-ID: > You might try something like this in your ~/.emacs file: > > (add-hook 'erlang-shell-mode-hook > '(lambda () (delete-other-windows (display-buffer "*erlang*")))) > (erlang-shell) > > --steve Unfortunately, it doesn?t work for me. The same story: erlang shell is being started, but it?s not active & displayed. I have to manually (after emacs start) C-x b *erlang* (switch to *erlang* buffer). My full .emacs config file: (tool-bar-mode -1) ;(split-window-right) ;TODO it's only erlang-related (set-frame-parameter nil 'fullscreen 'fullboth) ;(set-keyboard-coding-system nil) ;Erlang related { (setq erlang-root-dir "/usr/local/lib/erlang") (setq load-path (cons (concat erlang-root-dir "/lib/tools-2.6.14/emacs") load-path)) (setq exec-path (cons (concat erlang-root-dir "/bin") exec-path)) (require 'erlang-start) (require 'erlang-flymake) ;Erlang related } ;plugins (load-file "/Users/a1/.emacs.d/etags-select.el") ;projects ;for each project -its own environment (setq project-dir "/Users/a1/dev/otp/") (setq erlang-tags "ERLANG_TAGS") (setq c-tags "C_TAGS") (setq erlang-tags-f (concat project-dir erlang-tags)) (setq c-tags-f (concat project-dir c-tags)) (defun create-tags () (interactive) (message "building ERLANG_TAGS...") (shell-command (format "find %s -type f -iname \"*.[he]rl\" | etags -l erlang -o %s -" project-dir erlang-tags-f)) (message "done") (message "building C_TAGS...") (shell-command (format "find %s -type f -iname \"*.[ch]\" | etags -l c -o %s -" project-dir c-tags-f)) (message "done") ) (defun my-find-tag-simple(tagfile) (interactive) (message tagfile) (cond ((not (file-exists-p tagfile)) create-tags) (t nil) ) (visit-tags-table tagfile) (etags-select-find-tag-at-point) ) (defun my-find-tag () (interactive) (setq my-ext (file-name-extension buffer-file-name)) (message my-ext) (cond ((string-equal my-ext "c") (my-find-tag-simple c-tags-f)) ((string-equal my-ext "h") (my-find-tag-simple c-tags-f)) ((string-equal my-ext "erl") (my-find-tag-simple erlang-tags-f)) ((string-equal my-ext "hrl") (my-find-tag-simple erlang-tags-f)) (t nil) ) ) (global-set-key (kbd "M-.") 'my-find-tag) ;(autoload 'erlang-mode "erlang.el" "Major mode for editing Python source." t) ;(autoload 'erlang-shell "ipython" "Use IPython as the Python interpreter." t) ;(setq inferior-erlang-machine-options '("-sname" "emacs")) ;(save-window-excursion ; (erlang-shell) ; (sit-for 20) ; ) ;(erlang-shell) ;(sit-for 20) (add-hook 'erlang-shell-mode-hook '(lambda () (delete-other-windows (display-buffer "*erlang*")))) (erlang-shell) On Jul 4, 2014, at 21:29, Steve Vinoski wrote: > > > > On Fri, Jul 4, 2014 at 10:41 AM, ami wrote: > Hi all. > > I?m trying to run erlang shell at Emacs startup. > Just to get a some kind of IDE, when I have an erlang shell, an editor window, etc > right after startup. > > And I faced with the same problem as had been once arisen here > > (http://erlang.org/pipermail/erlang-questions/2004-January/011042.html ) > > So, I start erlang shell by > > (erlang-shell) > in .emacs > > but it?s not in active buffer. I, of course, can switch to *erlang* buffer afterwards. > But I don?t want to switch to *erlang* buffer manually always :( > > I tried (sit-for 20) right after (erlang-shell), like > > (erlang-shell) > (sit-for 20) > > And I do have an erlang shell displayed in emacs right after startup. > But when I press any key/mouse button, it disappears :( > But I can switch to *erlang* buffer manually (it?s not the case). > > (switch-to-buffer "*erlang*?) > > doesn?t work as well.(if I place it in .emacs right after (erlang-shell) ) > > Has anybody got erlang shell displayed & working & active right after emacs startup? > > You might try something like this in your ~/.emacs file: > > (add-hook 'erlang-shell-mode-hook > '(lambda () (delete-other-windows (display-buffer "*erlang*")))) > (erlang-shell) > > --steve From vinoski@REDACTED Fri Jul 4 22:35:18 2014 From: vinoski@REDACTED (Steve Vinoski) Date: Fri, 4 Jul 2014 16:35:18 -0400 Subject: [erlang-questions] get erlang-shell working, displayed & active right after emacs startup In-Reply-To: References: Message-ID: On Fri, Jul 4, 2014 at 3:51 PM, ami wrote: > > > > > You might try something like this in your ~/.emacs file: > > > > (add-hook 'erlang-shell-mode-hook > > '(lambda () (delete-other-windows (display-buffer "*erlang*")))) > > (erlang-shell) > > > > --steve > > > Unfortunately, it doesn?t work for me. The same story: erlang shell is > being started, > but it?s not active & displayed. I have to manually (after emacs start) > C-x b *erlang* > (switch to *erlang* buffer). > My code worked for me on Ubuntu 12.04 with emacs 24.3.1. But one thing about it is that the erlang buffer was not the focus if I specified a file to visit on the emacs command line at startup. The code below doesn't have that problem, and for me it does exactly what you're requesting. (require 'erlang-start) (add-hook 'emacs-startup-hook '(lambda () (let ((f (make-frame))) (with-selected-frame f (erlang-shell) (delete-other-windows (display-buffer erlang-shell-buffer-name)))))) --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From sina.samv@REDACTED Sat Jul 5 13:46:48 2014 From: sina.samv@REDACTED (Sina Samavati) Date: Sat, 5 Jul 2014 16:16:48 +0430 Subject: [erlang-questions] A question about erlang shell in Emacs (at startup) In-Reply-To: References: Message-ID: Perhaps this is what you want: (add-hook 'emacs-startup-hook 'erlang-shell) It will start erlang shell and switch to erlang shell buffer after startup even if you are about to open a file when running emacs. On Fri, Jul 4, 2014 at 7:08 PM, ami wrote: > Hi Vlad, > > I?m trying to run erlang shell at Emacs startup. > Just to get a some kind of IDE, when I have an erlang shell, an editor > window, etc > right after startup. > > And I faced with the same problem, you?d faced before. > (see your question @ > http://erlang.org/pipermail/erlang-questions/2004-January/011042.html ) > > So, I start erlang shell by > > (erlang-shell) > in .emacs > > but it?s not in active buffer. I, of course, can switch to *erlang* buffer > afterwards. > But I don?t want to switch to *erlang* buffer manually always :( > > I tried (sit-for 20) right after (erlang-shell), like > > (erlang-shell) > (sit-for 20) > > And I do have an erlang shell displayed in emacs right after startup. > But when I press any key/mouse button, it disappears :( > But I can switch to *erlang* buffer manually (it?s not the case). > > (switch-to-buffer "*erlang*?) > > doesn?t work as well.(if I place it in .emacs right after (erlang-shell) ) > > Has anybody got erlang shell displayed & working & active right after > emacs startup? > > Thanks in advance, > > Alex > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Sina Samavati Software engineer https://github.com/s1n4 https://twitter.com/sinasamavati -------------- next part -------------- An HTML attachment was scrubbed... URL: From siraaj@REDACTED Sun Jul 6 22:55:46 2014 From: siraaj@REDACTED (Siraaj Khandkar) Date: Sun, 06 Jul 2014 16:55:46 -0400 Subject: [erlang-questions] Polymorphic opaque types and dialyzer In-Reply-To: References: Message-ID: <53B9B7D2.9080509@khandkar.net> Any update on this? Any plans for Dialyzer to ever support polymorphic opaque types? On 03/13/2012 08:38 AM, Dmitry Groshev wrote: > Dialyzer can't handle polymorphic opaque type: > > $ echo "-module(opaque)." >> opaque.erl > $ echo "-opaque test(A) :: {A, A}." >> opaque.erl > $ dialyzer --src opaque.erl > Checking whether the PLT > /home/si14/work/repos/saelmon/.saelmon_dialyzer.plt is up-to-date... > yes > Proceeding with analysis... > =ERROR REPORT==== 13-Mar-2012::16:32:50 === > Error in process <0.30.0> with exit value: > {{nocatch,{error,"Polymorphic opaque types not supported > yet"}},[{erl_types,'-t_opaque_from_records/1-anonymous-2-',3,[{file,"erl_types.erl"},{line,564}]},{dict,map_bucket,2,[{file,"dict.erl"},{line,459}]},{dict,map_bkt_list,2,... > > > dialyzer: Analysis failed with error: > {{nocatch,{error,"Polymorphic opaque types not supported yet"}}, > [{erl_types,'-t_opaque_from_records/1-anonymous-2-',3, > [{file,"erl_types.erl"},{line,564}]}, > {dict,map_bucket,2,[{file,[...]},{line,...}]}, > {dict,map_bkt_list,2,[{file,...},{...}]}, > {dict,map_bkt_list,2,[{...}|...]}, > {dict,map_seg_list,2,[...]}, > {dict,map_dict,2,...}, > {erl_types,t_opaque_from_records,...}, > {dialyzer_typesig,...}]} > Last messages in the log cache: > Reading files and computing callgraph... done in 0.06 secs > Removing edges... done in 0.01 secs > Typesig analysis for SCC: [{opaque,module_info,0}] > > Is there any solution to this? Will it be fixed in upcoming releases of Erlang? From m3oucat@REDACTED Mon Jul 7 09:36:21 2014 From: m3oucat@REDACTED (ami) Date: Mon, 7 Jul 2014 10:36:21 +0300 Subject: [erlang-questions] A question about erlang shell in Emacs (at startup) In-Reply-To: References: Message-ID: <1185016A-438B-4726-BE5D-54D0F5373C09@gmail.com> Hi Sina As I wrote before, it doesn?t work. If I add this hook, erlang shell starts. But it is not active and I have to switch to it manually. On Jul 5, 2014, at 14:46, Sina Samavati wrote: > (add-hook 'emacs-startup-hook 'erlang-shell) /Alex From m3oucat@REDACTED Mon Jul 7 14:02:35 2014 From: m3oucat@REDACTED (ami) Date: Mon, 7 Jul 2014 15:02:35 +0300 Subject: [erlang-questions] get erlang-shell working, displayed & active right after emacs startup - SOLVED In-Reply-To: References: Message-ID: Hi Steve, I?m quite a newbie in emacs lisp & emacs internals, So your code seemed to me some kind of magic. I googled a lot, read a lot. Your code ?as is? didn?t work. But it really helped me. Eventually, I came to this: (erlang-shell) (add-hook 'emacs-startup-hook '(lambda () ( (display-buffer "*erlang*") ))) And it really works! (under OSX 10.9.3, Emacs 24.3(9.0)(downloaded from http://emacsformacosx.com/) - actually, ?option key WORKS in this build as M) I tried to place (erlang-shell) directly into the hook?s lambda, like (add-hook 'emacs-startup-hook '(lambda () ( (erlang-shell) (display-buffer "*erlang*") ))) But that didn?t work. Somewhy I was constantly getting an "Invalid function: (erlang-shell)? message. How can a variable be visible in outer scope but be invisible in inner scope? Magic :) But I?m a noob in emacs lisp though. On Jul 4, 2014, at 23:35, Steve Vinoski wrote: > > > > On Fri, Jul 4, 2014 at 3:51 PM, ami wrote: > > > > > You might try something like this in your ~/.emacs file: > > > > (add-hook 'erlang-shell-mode-hook > > '(lambda () (delete-other-windows (display-buffer "*erlang*")))) > > (erlang-shell) > > > > --steve > > > Unfortunately, it doesn?t work for me. The same story: erlang shell is being started, > but it?s not active & displayed. I have to manually (after emacs start) C-x b *erlang* > (switch to *erlang* buffer). > > My code worked for me on Ubuntu 12.04 with emacs 24.3.1. But one thing about it is that the erlang buffer was not the focus if I specified a file to visit on the emacs command line at startup. The code below doesn't have that problem, and for me it does exactly what you're requesting. > > (require 'erlang-start) > (add-hook > 'emacs-startup-hook > '(lambda () > (let ((f (make-frame))) > (with-selected-frame f > (erlang-shell) > (delete-other-windows (display-buffer erlang-shell-buffer-name)))))) > > --steve From luca.favatella@REDACTED Mon Jul 7 15:18:37 2014 From: luca.favatella@REDACTED (Luca Favatella) Date: Mon, 7 Jul 2014 15:18:37 +0200 (CEST) Subject: [erlang-questions] Polymorphic opaque types and dialyzer In-Reply-To: <53B9B7D2.9080509@khandkar.net> References: <53B9B7D2.9080509@khandkar.net> Message-ID: <315697826.225305.1404739117117.JavaMail.zimbra@erlang-solutions.com> > Any update on this? Any plans for Dialyzer to ever support polymorphic > opaque types? Hi Siraaj, With OTP 17.1 your example gives me "done (passed successfully)". Have you tried retesting it? $ dialyzer --src opaque.erl --plt [...] Checking whether the PLT [...] is up-to-date... yes Proceeding with analysis... done in 0m1.69s done (passed successfully) (The exit status is 0) $ cat opaque.erl -module(opaque). -opaque test(A) :: {A, A}. Regards Luca > On 03/13/2012 08:38 AM, Dmitry Groshev wrote: > > Dialyzer can't handle polymorphic opaque type: > > > > $ echo "-module(opaque)." >> opaque.erl > > $ echo "-opaque test(A) :: {A, A}." >> opaque.erl > > $ dialyzer --src opaque.erl > > Checking whether the PLT > > /home/si14/work/repos/saelmon/.saelmon_dialyzer.plt is up-to-date... > > yes > > Proceeding with analysis... > > =ERROR REPORT==== 13-Mar-2012::16:32:50 === > > Error in process <0.30.0> with exit value: > > {{nocatch,{error,"Polymorphic opaque types not supported > > yet"}},[{erl_types,'-t_opaque_from_records/1-anonymous-2-',3,[{file,"erl_types.erl"},{line,564}]},{dict,map_bucket,2,[{file,"dict.erl"},{line,459}]},{dict,map_bkt_list,2,... > > > > > > dialyzer: Analysis failed with error: > > {{nocatch,{error,"Polymorphic opaque types not supported yet"}}, > > [{erl_types,'-t_opaque_from_records/1-anonymous-2-',3, > > [{file,"erl_types.erl"},{line,564}]}, > > {dict,map_bucket,2,[{file,[...]},{line,...}]}, > > {dict,map_bkt_list,2,[{file,...},{...}]}, > > {dict,map_bkt_list,2,[{...}|...]}, > > {dict,map_seg_list,2,[...]}, > > {dict,map_dict,2,...}, > > {erl_types,t_opaque_from_records,...}, > > {dialyzer_typesig,...}]} > > Last messages in the log cache: > > Reading files and computing callgraph... done in 0.06 secs > > Removing edges... done in 0.01 secs > > Typesig analysis for SCC: [{opaque,module_info,0}] > > > > Is there any solution to this? Will it be fixed in upcoming releases of > > Erlang? From shian5@REDACTED Mon Jul 7 17:17:44 2014 From: shian5@REDACTED (Shian Wu) Date: Mon, 7 Jul 2014 23:17:44 +0800 Subject: [erlang-questions] scheduler usage vs cpu usage Message-ID: Hi I call recon:scheduler_usage(5000) when cpu usage is low (100% ~ 200% from top, with 32 cores) It show all scheduler usage is 1.0 Is this possible? all schedulers are busy but cpu usage is low thanks --- Shian Wu -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Mon Jul 7 17:45:24 2014 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Mon, 7 Jul 2014 11:45:24 -0400 Subject: [erlang-questions] Microservices vs. Erlang processes Message-ID: <298EAE28-4128-46BC-B981-154A128760F4@writersglen.com> Hello, We're hearing much about "microservice architecture" in system design circles these days: http://martinfowler.com/articles/microservices.html Can anyone explain to me how this design practice differs from the Erlang practice of factoring a system into concurrent processes? I think I understand that they have different interfaces and that Erlang processes can be easily monitored and supervised but, beyond that... - Is this a case of another term for a functionally equivalent concept? - Is an Erlang process typically coarser or finer grained than a typical microservice? - What "itch" is being scratched by microservice design that can't be scratched by well designed Erlang? Thanks, LRP Sent from my iPad From egil@REDACTED Mon Jul 7 17:45:32 2014 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Mon, 7 Jul 2014 17:45:32 +0200 Subject: [erlang-questions] scheduler usage vs cpu usage In-Reply-To: References: Message-ID: <53BAC09C.2040706@erlang.org> On 2014-07-07 17:17, Shian Wu wrote: > Hi > > I call recon:scheduler_usage(5000) when cpu usage is low (100% ~ 200% > from top, with 32 cores) > It show all scheduler usage is 1.0 > Is this possible? all schedulers are busy but cpu usage is low Yes, it is possible. Schedulers waiting for os resources are considered utilized (they cannot handle more work). This is why scheduler utilization exists. CPU utilization is often used as a load measuring device but it is a very poor estimate load. Scheduler utilization is a better estimate. // Bj?rn-Egil > > thanks > --- > Shian Wu > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mallen@REDACTED Mon Jul 7 18:05:10 2014 From: mallen@REDACTED (Mark Allen) Date: Mon, 7 Jul 2014 11:05:10 -0500 Subject: [erlang-questions] Microservices vs. Erlang processes In-Reply-To: <298EAE28-4128-46BC-B981-154A128760F4@writersglen.com> References: <298EAE28-4128-46BC-B981-154A128760F4@writersglen.com> Message-ID: (trigger warning: top posting; Garret may want to stop here.) The biggest proponent of "microservices" I've read/listened to a lot is Adrian Cockcroft recently with Netflix, now with a VC firm. Reading the Martin Fowler article (lovely article, by the way, thanks for link) it seems like they're talking about the same ideas. If I understand the talks/writings correctly, the ideas are that you want small services that handle a single resource per service, each service is loosely coupled to the others, offer a strong quote contract unquote through the API layer and are able to scale horizontally and vertically without complexity. So I took this to mean- 1. A web service which implies some kind of HTTP server semantics for IO purposes, 2. Managing some kind of resource where the resource state is mutated by that one service. So given that understanding, I think the abstraction for Erlang is more akin to a single Erlang VM running multiple (possibly unrelated) applications, rather than Erlang processes which may compose an application and an application. If I may soapbox for a moment, I think that while microservices are a great way to design a system in some kind of greenfield environment, there has been too little discussion by the proponents of microservices on how to deal with the inherent complexities of distributed systems in that type of approach. At best they're mentioned in passing, and at worst, hand-waved away. If nothing else, a giant monolithic application helps to avoid those problems to some extent. (Not that it's necessarily the *right* design for a given problem, but doing the monkey-see-monkey-do rip'n'replace of a monolith (1:4:9) with microservices is trading the devil you know for a whole different set of concerns/problems.) Cheers, Mark On 7/7/14 10:45 AM, "Lloyd R. Prentice" wrote: >Hello, > >We're hearing much about "microservice architecture" in system design >circles these days: > >http://martinfowler.com/articles/microservices.html > >Can anyone explain to me how this design practice differs from the Erlang >practice of factoring a system into concurrent processes? > >I think I understand that they have different interfaces and that Erlang >processes can be easily monitored and supervised but, beyond that... > >- Is this a case of another term for a functionally equivalent concept? >- Is an Erlang process typically coarser or finer grained than a typical >microservice? >- What "itch" is being scratched by microservice design that can't be >scratched by well designed Erlang? > >Thanks, > >LRP > >Sent from my iPad >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions From g@REDACTED Mon Jul 7 18:05:26 2014 From: g@REDACTED (Garrett Smith) Date: Mon, 7 Jul 2014 11:05:26 -0500 Subject: [erlang-questions] Microservices vs. Erlang processes In-Reply-To: <298EAE28-4128-46BC-B981-154A128760F4@writersglen.com> References: <298EAE28-4128-46BC-B981-154A128760F4@writersglen.com> Message-ID: On Mon, Jul 7, 2014 at 10:45 AM, Lloyd R. Prentice wrote: > Hello, > > We're hearing much about "microservice architecture" in system design circles these days: > > http://martinfowler.com/articles/microservices.html This IMO is a rebranding of the Service Oriented Architecture (SOA) moniker that was hijacked by software vendors and enterprise IT to use to sell non-SOA technology. SOA is a wasteland and companies like Thoughtworks need something to rally that has some potential to work. > Can anyone explain to me how this design practice differs from the Erlang practice of factoring a system into concurrent processes? This is the point of e2: http://e2project.org It's why the primary process interface is called a "service". The idea is that SOA, or now microservices, is a good way to conceive of an application. It provides a mental model for approaching problems. > I think I understand that they have different interfaces and that Erlang processes can be easily monitored and supervised but, beyond that... > > - Is this a case of another term for a functionally equivalent concept? I think the underlying principles are the same. Microservices are language agnostic however. > - Is an Erlang process typically coarser or finer grained than a typical microservice? Finer - even much, much finer. If you compare a "microservice" implementation in another language to an Erlang "system", the later should be called "nanoservice". > - What "itch" is being scratched by microservice design that can't be scratched by well designed Erlang? It's really the only sane way to approach problems where more than one thing is going on at the same time (concurrency). The itch is more like a gasping-for-breath. "Microservices" is not language specific, so it embraces language diversity. Garrett From mfidelman@REDACTED Mon Jul 7 18:24:29 2014 From: mfidelman@REDACTED (Miles Fidelman) Date: Mon, 07 Jul 2014 12:24:29 -0400 Subject: [erlang-questions] Microservices vs. Erlang processes In-Reply-To: <298EAE28-4128-46BC-B981-154A128760F4@writersglen.com> References: <298EAE28-4128-46BC-B981-154A128760F4@writersglen.com> Message-ID: <53BAC9BD.6010003@meetinghouse.net> Lloyd R. Prentice wrote: > Hello, > > We're hearing much about "microservice architecture" in system design circles these days: > > http://martinfowler.com/articles/microservices.html > > Can anyone explain to me how this design practice differs from the Erlang practice of factoring a system into concurrent processes? > > I think I understand that they have different interfaces and that Erlang processes can be easily monitored and supervised but, beyond that... > > - Is this a case of another term for a functionally equivalent concept? > - Is an Erlang process typically coarser or finer grained than a typical microservice? > - What "itch" is being scratched by microservice design that can't be scratched by well designed Erlang? > > Well... a few knee-jerk, opinionated comments - take them for what they're worth: - first off, it depends on how one defines "microservice architecture" - the notion of building software and systems from re-usable components is not new - it generally doesn't work well, though things like CPAN are a notable exception - the notion of services as components is not new - again, the devil is in the details: --- can you say objects-oriented? --- E2EE and JavaBeans seems to have traction --- SOAs and web services seem to generate a lot of papers on "SOA governance" than real systems - IMHO because W3C web services are very complicated and high-overhead --- RESTful interfaces and mashups are another approach - again, with some traction, particularly for building things on top of Google and Amazon services --- message oriented middleware (e.g., Mule) is yet another approach --- overall, the questions always come down to: what are the services, and how does when orchestrate/choreograph them What differentiates microservices seems to be the notion of putting each service on its own server - be it hardware, a VM, or a container (can you say "Docker?"). The folks pushing the idea seem to be driven by rapid deployment and management considerations - and seem to come out of the DevOps and the cloud provisioning communities - i.e., folks who focus on tooling and platforms. (IMHO) The notion of services and composition of services is not a bad design pattern, particularly with clean interfaces; which to me, means - protocol oriented (messages or RESTful) rather than programmatic APIs. But... the microservice implementation strikes me as basically wrong-headed. The notion of a VM or container (i.e., a full o/s) carries a LOT of baggage and overhead, if all one is doing is running a single service on it. Running a micro-service on a heavyweight platform seems fundamentally broken. Granted, something like LXC offers an interim step for "containerizing" existing code bases (e.g., postfix in one container, mysql in another, apache in another) - but for new code, a much more lightweight platform seems in order - something like a very stripped down microkernal. To me, a serious "microservice platform" would look like Erlang on bare metal - with "microservices" implemented as Erlang processes (a gen-server seems like an awfully good start for a "microservice"). (Parenthetically - whatever happened to the Erlang on Bare Metal and Erlang on Xen projects?). Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From raould@REDACTED Mon Jul 7 19:08:51 2014 From: raould@REDACTED (Raoul Duke) Date: Mon, 7 Jul 2014 10:08:51 -0700 Subject: [erlang-questions] Microservices vs. Erlang processes In-Reply-To: <53BAC9BD.6010003@meetinghouse.net> References: <298EAE28-4128-46BC-B981-154A128760F4@writersglen.com> <53BAC9BD.6010003@meetinghouse.net> Message-ID: > To me, a serious "microservice platform" would look like Erlang on bare > metal - with "microservices" implemented as Erlang processes (a gen-server > seems like an awfully good start for a "microservice"). (Parenthetically - > whatever happened to the Erlang on Bare Metal and Erlang on Xen projects?). yes yes yes we need more of that fun stuff :-) http://www.xenproject.org/developers/teams/mirage-os.html From pierrefenoll@REDACTED Mon Jul 7 20:07:36 2014 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Mon, 7 Jul 2014 20:07:36 +0200 Subject: [erlang-questions] Microservices vs. Erlang processes In-Reply-To: References: <298EAE28-4128-46BC-B981-154A128760F4@writersglen.com> <53BAC9BD.6010003@meetinghouse.net> Message-ID: If we're talking about Concurix' Erlang on Bare Metal, I can't find "Erlang" on their website anymore. They seem to have completely switched to monitoring node instances. You can try and test their AMI which is still available on EC2, or you can look at part of their code here: https://github.com/Concurix/cx_runtime I hope they publish their modifications to Erlang/OTP some day as they said that they improved GC and scheduling on manycore machines! ( http://www.erlang-factory.com/conference/SFBay2013/speakers/AlexanderGounares ) Cheers, -- Pierre Fenoll On 7 July 2014 19:08, Raoul Duke wrote: > > To me, a serious "microservice platform" would look like Erlang on bare > > metal - with "microservices" implemented as Erlang processes (a > gen-server > > seems like an awfully good start for a "microservice"). (Parenthetically > - > > whatever happened to the Erlang on Bare Metal and Erlang on Xen > projects?). > > yes yes yes we need more of that fun stuff :-) > http://www.xenproject.org/developers/teams/mirage-os.html > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From siraaj@REDACTED Mon Jul 7 21:12:04 2014 From: siraaj@REDACTED (Siraaj Khandkar) Date: Mon, 07 Jul 2014 15:12:04 -0400 Subject: [erlang-questions] Polymorphic opaque types and dialyzer In-Reply-To: <315697826.225305.1404739117117.JavaMail.zimbra@erlang-solutions.com> References: <53B9B7D2.9080509@khandkar.net> <315697826.225305.1404739117117.JavaMail.zimbra@erlang-solutions.com> Message-ID: <53BAF104.2070101@khandkar.net> On 07/07/2014 09:18 AM, Luca Favatella wrote: >> Any update on this? Any plans for Dialyzer to ever support polymorphic >> opaque types? > > Hi Siraaj, > > With OTP 17.1 your example gives me "done (passed successfully)". Have you tried retesting it? > > $ dialyzer --src opaque.erl --plt [...] > Checking whether the PLT [...] is up-to-date... yes > Proceeding with analysis... done in 0m1.69s > done (passed successfully) > > (The exit status is 0) > > $ cat opaque.erl > -module(opaque). > -opaque test(A) :: {A, A}. > Hi Luca, I only tried with R16B02 before. Just tried with 17.1.1 and it does indeed work. Awesome! Thanks! My test case: $ cat foo.erl -module(foo). -export_type([t/1]). -record(t, {bar}). -opaque t(A) :: #t{bar :: A}. $ erlc +debug_info foo.erl $ dialyzer foo.beam Checking whether the PLT /home/siraaj/.dialyzer_plt is up-to-date... yes Proceeding with analysis... done in 0m6.04s done (passed successfully) > > >> On 03/13/2012 08:38 AM, Dmitry Groshev wrote: >>> Dialyzer can't handle polymorphic opaque type: >>> >>> $ echo "-module(opaque)." >> opaque.erl >>> $ echo "-opaque test(A) :: {A, A}." >> opaque.erl >>> $ dialyzer --src opaque.erl >>> Checking whether the PLT >>> /home/si14/work/repos/saelmon/.saelmon_dialyzer.plt is up-to-date... >>> yes >>> Proceeding with analysis... >>> =ERROR REPORT==== 13-Mar-2012::16:32:50 === >>> Error in process <0.30.0> with exit value: >>> {{nocatch,{error,"Polymorphic opaque types not supported >>> yet"}},[{erl_types,'-t_opaque_from_records/1-anonymous-2-',3,[{file,"erl_types.erl"},{line,564}]},{dict,map_bucket,2,[{file,"dict.erl"},{line,459}]},{dict,map_bkt_list,2,... >>> >>> >>> dialyzer: Analysis failed with error: >>> {{nocatch,{error,"Polymorphic opaque types not supported yet"}}, >>> [{erl_types,'-t_opaque_from_records/1-anonymous-2-',3, >>> [{file,"erl_types.erl"},{line,564}]}, >>> {dict,map_bucket,2,[{file,[...]},{line,...}]}, >>> {dict,map_bkt_list,2,[{file,...},{...}]}, >>> {dict,map_bkt_list,2,[{...}|...]}, >>> {dict,map_seg_list,2,[...]}, >>> {dict,map_dict,2,...}, >>> {erl_types,t_opaque_from_records,...}, >>> {dialyzer_typesig,...}]} >>> Last messages in the log cache: >>> Reading files and computing callgraph... done in 0.06 secs >>> Removing edges... done in 0.01 secs >>> Typesig analysis for SCC: [{opaque,module_info,0}] >>> >>> Is there any solution to this? Will it be fixed in upcoming releases of >>> Erlang? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mjtruog@REDACTED Mon Jul 7 22:39:57 2014 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 07 Jul 2014 13:39:57 -0700 Subject: [erlang-questions] Microservices vs. Erlang processes In-Reply-To: <53BAC9BD.6010003@meetinghouse.net> References: <298EAE28-4128-46BC-B981-154A128760F4@writersglen.com> <53BAC9BD.6010003@meetinghouse.net> Message-ID: <53BB059D.4000302@gmail.com> On 07/07/2014 09:24 AM, Miles Fidelman wrote: > Lloyd R. Prentice wrote: >> Hello, >> >> We're hearing much about "microservice architecture" in system design >> circles these days: >> >> http://martinfowler.com/articles/microservices.html >> >> Can anyone explain to me how this design practice differs from the >> Erlang practice of factoring a system into concurrent processes? >> >> I think I understand that they have different interfaces and that >> Erlang processes can be easily monitored and supervised but, beyond >> that... >> >> - Is this a case of another term for a functionally equivalent concept? >> - Is an Erlang process typically coarser or finer grained than a >> typical microservice? >> - What "itch" is being scratched by microservice design that can't be >> scratched by well designed Erlang? >> >> > > Well... a few knee-jerk, opinionated comments - take them for what > they're worth: > > - first off, it depends on how one defines "microservice architecture" > > - the notion of building software and systems from re-usable > components is not new - it generally doesn't work well, though things > like CPAN are a notable exception > - the notion of services as components is not new - again, the devil > is in the details: > --- can you say objects-oriented? > --- E2EE and JavaBeans seems to have traction > --- SOAs and web services seem to generate a lot of papers on "SOA > governance" than real systems - IMHO because W3C web services are very > complicated and high-overhead > --- RESTful interfaces and mashups are another approach - again, with > some traction, particularly for building things on top of Google and > Amazon services > --- message oriented middleware (e.g., Mule) is yet another approach > --- overall, the questions always come down to: what are the services, > and how does when orchestrate/choreograph them > > What differentiates microservices seems to be the notion of putting > each service on its own server - be it hardware, a VM, or a container > (can you say "Docker?"). The folks pushing the idea seem to be driven > by rapid deployment and management considerations - and seem to come > out of the DevOps and the cloud provisioning communities - i.e., folks > who focus on tooling and platforms. If microservices require their own server, that isn't very "micro". Older SOA could require a service to hog a server, but I hope that a microservice is smaller than that. Other people in the past have referred to Erlang processes as "nanoservices" which seems realistic. > > (IMHO) The notion of services and composition of services is not a bad > design pattern, particularly with clean interfaces; which to me, means > - protocol oriented (messages or RESTful) rather than programmatic APIs. You can have "messaging and RESTful", they are not exclusive. > > But... the microservice implementation strikes me as basically > wrong-headed. The notion of a VM or container (i.e., a full o/s) > carries a LOT of baggage and overhead, if all one is doing is running > a single service on it. Running a micro-service on a heavyweight > platform seems fundamentally broken. Granted, something like LXC > offers an interim step for "containerizing" existing code bases (e.g., > postfix in one container, mysql in another, apache in another) - but > for new code, a much more lightweight platform seems in order - > something like a very stripped down microkernal. I don't believe a "microservice" concept requires a container or virtualization. > > To me, a serious "microservice platform" would look like Erlang on > bare metal - with "microservices" implemented as Erlang processes (a > gen-server seems like an awfully good start for a "microservice"). > (Parenthetically - whatever happened to the Erlang on Bare Metal and > Erlang on Xen projects?). If erlang processes are "nanoservices", I think "microservices" would utilize more than 1 Erlang process. For an example of this, look at the cloudi_service behaviour for creating internal services with CloudI (http://cloudi.org). Some links that can help you distinguish between Erlang usage and CloudI usage are here: http://cloudi.org/faq.html#1_Actor http://cloudi.org/docs/cloudi_meetup_seattle2014.pdf I agree that Erlang can provide microservices with bare-metal. > > Miles Fidelman > > > > > > > From zxq9@REDACTED Tue Jul 8 04:47:54 2014 From: zxq9@REDACTED (zxq9) Date: Tue, 08 Jul 2014 11:47:54 +0900 Subject: [erlang-questions] Microservices vs. Erlang processes In-Reply-To: <298EAE28-4128-46BC-B981-154A128760F4@writersglen.com> References: <298EAE28-4128-46BC-B981-154A128760F4@writersglen.com> Message-ID: <10329767.WaXFFmkT2h@jalapeno> On Monday 07 July 2014 11:45:24 Lloyd R. Prentice wrote: > Hello, > > We're hearing much about "microservice architecture" in system design > circles these days: > > http://martinfowler.com/articles/microservices.html > > Can anyone explain to me how this design practice differs from the Erlang > practice of factoring a system into concurrent processes? > > I think I understand that they have different interfaces and that Erlang > processes can be easily monitored and supervised but, beyond that... > > - Is this a case of another term for a functionally equivalent concept? > - Is an Erlang process typically coarser or finer grained than a typical > microservice? - What "itch" is being scratched by microservice design that > can't be scratched by well designed Erlang? To sum up this thread: There is nothing technically new or interesting here. There is a new label, and that is interesting for reasons that have nothing to do with technology and everything to do with market psychology. Regarding "microservices" meaning bare metal: This would be a much more interesting, precise, and original meaning! Leave it to the smart guys who hang around here to read this meaning into such a term. Unfortunately, the same reasons this would be a good idea (semantically and technically) are why this is *not* what the term means, and never will be -- because buzzwords are designed to repackage a previously hyped-but-immature idea, and therefore must be vague enough to mean literally anything (consider "cloud").[1] What itch? The marketing itch. Nothing else. Its a chance for quite a few folks to clean up and republish a bunch of work on an idea that never was never really executed on properly (like so many others...). -Craig [1] I had an interesting conversation with Peer Stritzinger about bare metal Erlang just the other day. (In relation to a reactive infrastructure idea I am working on, which is a bit different than the "yet another browser accessible primate distraction device" sort of thing that gets web folks excited.) He's done a lot of work in this direction, initially using Erlang as a friendlier alternative to C in industrial applications, but clearly with an awareness that more interesting opportunities would emerge as a side effect of his project. The idea of using actual micros for "micro"-sized services is intriguing, but the implications seem to be way outside the realm of the average web-style VC's imagination. There is sometimes a conceptual fuzziness in this space between running an OTP-style constellation of distributed smart things where "thing" means "physical device" (of whatever sort -- a crane arm, a solar panel, etc.) and running Erlang directly on Xen. I've not seen this discussed anywhere. I think the groups working on both methods are targetting different problems. The Erlang-on-Xen/Erlang-as-the-OS crowd is focused on running web, business or other software-only services, and sees elimination of an increasingly shifty and messy OS layer as a simplification. The embedded- Erlang folks are focused on the problem of driving hardware, and sees Erlang as a friendlier alternative to assembler and C, and possibly one of the only reasonable new languages in an era where a "service" and a "device" may become interchangeable ideas at some level of industrial application. None of this, of course, is what "microservices architecture" is intended to mean. From lloyd@REDACTED Tue Jul 8 05:54:41 2014 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Mon, 7 Jul 2014 23:54:41 -0400 Subject: [erlang-questions] Microservices vs. Erlang processes In-Reply-To: <10329767.WaXFFmkT2h@jalapeno> References: <298EAE28-4128-46BC-B981-154A128760F4@writersglen.com> <10329767.WaXFFmkT2h@jalapeno> Message-ID: Hi Craig, >The idea of using actual micros for "micro"-sized services is > intriguing, but the implications seem to be way outside the realm of the > average web-style VC's imagination. This is an "intriguing" statement. Can you please elaborate on the implications? All the best, Lloyd Sent from my iPad > On Jul 7, 2014, at 10:47 PM, zxq9 wrote: > > The idea of using actual micros for "micro"-sized services is > intriguing, but the implications seem to be way outside the realm of the > average web-style VC's imagination. From jesper.louis.andersen@REDACTED Tue Jul 8 13:27:21 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 8 Jul 2014 13:27:21 +0200 Subject: [erlang-questions] Microservices vs. Erlang processes In-Reply-To: References: <298EAE28-4128-46BC-B981-154A128760F4@writersglen.com> Message-ID: On Mon, Jul 7, 2014 at 6:05 PM, Garrett Smith wrote: > This IMO is a rebranding of the Service Oriented Architecture (SOA) > moniker that was hijacked by software vendors and enterprise IT to use > to sell non-SOA technology. > Indeed. They took Jack Daniels and rebottled it on Ardbeg 18yo bottles. This is only going to work for so long. The whole *crux* of SOA/MS is component isolation. To manage large complex systems, you need to split them into distributed entities and make communication between services asynchronous+resilient. This is exactly the design where a system like Erlang excels. The client I am working for uses Erlang as an orchestration service in front of a large RabbitMQ cluster. Most microservices are written in Node.js, python, ocaml, whatnot. And Erlang mediates between the services. It is nice because a lot of the concurrency problems can be lifted into the Erlang-world and handled there. While still keeping a boring sequential program written in Python to do other things on the side. RabbitMQ allows horizontal scaling and a common interface for messaging with a well-defined protocol/driver interface. Decoupling is the key to manage large complex infrastructure setups. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mahesh@REDACTED Tue Jul 8 13:52:59 2014 From: mahesh@REDACTED (Mahesh Paolini-Subramanya) Date: Tue, 8 Jul 2014 07:52:59 -0400 Subject: [erlang-questions] Microservices vs. Erlang processes In-Reply-To: References: <298EAE28-4128-46BC-B981-154A128760F4@writersglen.com> Message-ID: > > Decoupling is the key to manage large complex infrastructure setups. Thats pretty much it. If anything, i'd add one tweak to the above - "Decoupling, combined with comprehensibility, is the key to manage large complex infrastructure setups." I'll take comprehensibility over efficiency any day of the week, and twice on Sundays. Combining comprehensibility with decoupling gives me the ability to not just deal with problem spaces individually, but to *understand* the impact of changes to these problem-spaces in isolation. Loosely speaking, this allows me to look at the overarching system in two orthogonal realms - the individual (decoupled) sub-systems, and the *interconnections* between these sub-systems. And yes, the 'comprehensibility' argument extends to both the sub-systems *and* the interconnections. Micro-services are all well and good, but taken to the extreme (which I pretty much guarantee you *is* going to happen) will end up as yet another rolling disaster. Divide And Conquer may have been great for the British Empire, but when combined with the natural human tendency to Chop Even Finer ends up throwing comprehensibility out the window. This will, au n?cessaire, result in millions of wee-tiny-services, each more twisty than the other, quite a few of which replicate functionality from elsewhere, all of which ends up creating more work for consultants. Which may very well be the end-game, now that I think of it... Cheers On Tue, Jul 8, 2014 at 7:27 AM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Mon, Jul 7, 2014 at 6:05 PM, Garrett Smith wrote: > >> This IMO is a rebranding of the Service Oriented Architecture (SOA) >> moniker that was hijacked by software vendors and enterprise IT to use >> to sell non-SOA technology. >> > > Indeed. They took Jack Daniels and rebottled it on Ardbeg 18yo bottles. > This is only going to work for so long. > > The whole *crux* of SOA/MS is component isolation. To manage large complex > systems, you need to split them into distributed entities and make > communication between services asynchronous+resilient. This is exactly the > design where a system like Erlang excels. > > The client I am working for uses Erlang as an orchestration service in > front of a large RabbitMQ cluster. Most microservices are written in > Node.js, python, ocaml, whatnot. And Erlang mediates between the services. > It is nice because a lot of the concurrency problems can be lifted into the > Erlang-world and handled there. While still keeping a boring sequential > program written in Python to do other things on the side. RabbitMQ allows > horizontal scaling and a common interface for messaging with a well-defined > protocol/driver interface. > > Decoupling is the key to manage large complex infrastructure setups. > > > -- > J. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- *Mahesh Paolini-Subramanya That tall bald Indian guy..* *Google+ | Blog | Twitter | LinkedIn * -------------- next part -------------- An HTML attachment was scrubbed... URL: From francesco@REDACTED Tue Jul 8 17:05:15 2014 From: francesco@REDACTED (Francesco Cesarini) Date: Tue, 08 Jul 2014 16:05:15 +0100 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly Message-ID: <53BC08AB.1030605@erlang-solutions.com> Hi all, a shameless plug. Steve Vinoski and I are working on a book focused on distributed, scalable systems with OTP. It is available from O'Reilly as an Early Release from their website. The first eight chapters we released cover (in great detail) all other behaviours. We started writing, and before we knew it, we had several hundred pages on behaviours alone. Last week, two new chapters, including the Introduction & Special processes and Implementing your own behaviours were released. We are now focusing on release handling (hgg), code upgrade and architectural patterns. What is available is an unedited draft, with new chapters and improvements published as they become available. You can find more info here: http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct If you use discount code authd, you will get 50% off the Early Release, and 40% on pre-orders of the the printed copy. Looking forward to your feedback, F -- Erlang Solutions Ltd. http://www.erlang-solutions.com From lee.sylvester@REDACTED Tue Jul 8 19:49:46 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 8 Jul 2014 18:49:46 +0100 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <53BC08AB.1030605@erlang-solutions.com> References: <53BC08AB.1030605@erlang-solutions.com> Message-ID: Well, that?s another book to add to my shelf :-) Looking forward to it. Lee On 8 Jul 2014, at 16:05, Francesco Cesarini wrote: > Hi all, > > a shameless plug. Steve Vinoski and I are working on a book focused on distributed, scalable systems with OTP. It is available from O'Reilly as an Early Release from their website. The first eight chapters we released cover (in great detail) all other behaviours. We started writing, and before we knew it, we had several hundred pages on behaviours alone. Last week, two new chapters, including the Introduction & Special processes and Implementing your own behaviours were released. We are now focusing on release handling (hgg), code upgrade and architectural patterns. > > What is available is an unedited draft, with new chapters and improvements published as they become available. You can find more info here: > > http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct > > If you use discount code authd, you will get 50% off the Early Release, and 40% on pre-orders of the the printed copy. > > Looking forward to your feedback, > F > > -- > Erlang Solutions Ltd. > http://www.erlang-solutions.com > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From lee.sylvester@REDACTED Tue Jul 8 20:04:12 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 8 Jul 2014 19:04:12 +0100 Subject: [erlang-questions] UDP Headers Message-ID: Hey guys, So, I?m using gen_udp for a server I?m building, but I really need to be able to update the headers of incoming packets. Does anyone know how I can do this with gen_udp without having to resort to using raw sockets? Thanks, Lee From mfidelman@REDACTED Tue Jul 8 20:08:35 2014 From: mfidelman@REDACTED (Miles Fidelman) Date: Tue, 08 Jul 2014 14:08:35 -0400 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: References: <53BC08AB.1030605@erlang-solutions.com> Message-ID: <53BC33A3.4000502@meetinghouse.net> On 8 Jul 2014, at 16:05, Francesco Cesarini wrote: >> Hi all, >> >> a shameless plug. Steve Vinoski and I are working on a book focused on distributed, scalable systems with OTP. It is available from O'Reilly as an Early Release from their website. The first eight chapters we released cover (in great detail) all other behaviours. We started writing, and before we knew it, we had several hundred pages on behaviours alone. Last week, two new chapters, including the Introduction & Special processes and Implementing your own behaviours were released. We are now focusing on release handling (hgg), code upgrade and architectural patterns. >> >> What is available is an unedited draft, with new chapters and improvements published as they become available. You can find more info here: >> >> http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct >> >> If you use discount code authd, you will get 50% off the Early Release, and 40% on pre-orders of the the printed copy. >> >> Looking forward to your feedback, >> Not for nothing, but $50 for the undedited work-in-progress seems a bit steep (and yes, I saw the discount code). I also note that "rough cuts" aren't included in my Safari subscription. Usual practice that I've seen is for works-in-progress to be free, with tools that support comments from early readers. Sounds interesting - but, from the TOC (all that's available without paying), it sure looks like it covers the same ground as "Erlang and OTP in Action." Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From mfidelman@REDACTED Tue Jul 8 20:10:32 2014 From: mfidelman@REDACTED (Miles Fidelman) Date: Tue, 08 Jul 2014 14:10:32 -0400 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly - slight addition In-Reply-To: <53BC33A3.4000502@meetinghouse.net> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> Message-ID: <53BC3418.9090108@meetinghouse.net> Miles Fidelman wrote: > On 8 Jul 2014, at 16:05, Francesco Cesarini > wrote: >>> Hi all, >>> >>> a shameless plug. Steve Vinoski and I are working on a book focused >>> on distributed, scalable systems with OTP. It is available from >>> O'Reilly as an Early Release from their website. The first eight >>> chapters we released cover (in great detail) all other behaviours. >>> We started writing, and before we knew it, we had several hundred >>> pages on behaviours alone. Last week, two new chapters, including >>> the Introduction & Special processes and Implementing your own >>> behaviours were released. We are now focusing on release handling >>> (hgg), code upgrade and architectural patterns. >>> >>> What is available is an unedited draft, with new chapters and >>> improvements published as they become available. You can find more >>> info here: >>> >>> http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>> >>> >>> If you use discount code authd, you will get 50% off the Early >>> Release, and 40% on pre-orders of the the printed copy. >>> >>> Looking forward to your feedback, >>> > > Not for nothing, but $50 for the undedited work-in-progress seems a > bit steep (and yes, I saw the discount code). I also note that "rough > cuts" aren't included in my Safari subscription. > > Usual practice that I've seen is for works-in-progress to be free, > with tools that support comments from early readers. > > Sounds interesting - but, from the TOC (all that's available without > paying), it sure looks like it covers the same ground as "Erlang and > OTP in Action." Now... if this were on Kickstarter, and you were avowedly asking for support, that would be a different story. The O'Reilly "rough cuts" package is for the birds. Miles -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From lee.sylvester@REDACTED Tue Jul 8 20:14:15 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 8 Jul 2014 19:14:15 +0100 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <53BC33A3.4000502@meetinghouse.net> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> Message-ID: Wow, $50? Yeah, that is steep. If this were some black arts compiler book or video encoding bible, then that would be something else. But a book on Erlang/OTP just doesn?t fall into that bracket? I may wait til it falls in the bargain bucket ;-) Lee On 8 Jul 2014, at 19:08, Miles Fidelman wrote: > On 8 Jul 2014, at 16:05, Francesco Cesarini wrote: >>> Hi all, >>> >>> a shameless plug. Steve Vinoski and I are working on a book focused on distributed, scalable systems with OTP. It is available from O'Reilly as an Early Release from their website. The first eight chapters we released cover (in great detail) all other behaviours. We started writing, and before we knew it, we had several hundred pages on behaviours alone. Last week, two new chapters, including the Introduction & Special processes and Implementing your own behaviours were released. We are now focusing on release handling (hgg), code upgrade and architectural patterns. >>> >>> What is available is an unedited draft, with new chapters and improvements published as they become available. You can find more info here: >>> >>> http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>> >>> If you use discount code authd, you will get 50% off the Early Release, and 40% on pre-orders of the the printed copy. >>> >>> Looking forward to your feedback, >>> > > Not for nothing, but $50 for the undedited work-in-progress seems a bit steep (and yes, I saw the discount code). I also note that "rough cuts" aren't included in my Safari subscription. > > Usual practice that I've seen is for works-in-progress to be free, with tools that support comments from early readers. > > Sounds interesting - but, from the TOC (all that's available without paying), it sure looks like it covers the same ground as "Erlang and OTP in Action." > > Miles Fidelman > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From francesco@REDACTED Tue Jul 8 20:36:21 2014 From: francesco@REDACTED (Francesco Cesarini) Date: Tue, 08 Jul 2014 19:36:21 +0100 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> Message-ID: <53BC3A25.8030708@erlang-solutions.com> Alas, that is how O'Reilly price their books. Having a high price and then discounting is not the approach I would pick. That is why discount codes are being handed out on public mailing lists and social media. I recommend you use them (Read, no one pays full price for an O'Reilly book). As an unedited book, the cost is for the final book which we hope will complement what is already out there. It is a different approach to OTP in action. One I've been using for 15 years when teaching OTP. /F On 08/07/2014 19:14, Lee Sylvester wrote: > Wow, $50? Yeah, that is steep. If this were some black arts compiler book or video encoding bible, then that would be something else. But a book on Erlang/OTP just doesn?t fall into that bracket? > > I may wait til it falls in the bargain bucket ;-) > > Lee > > > On 8 Jul 2014, at 19:08, Miles Fidelman wrote: > >> On 8 Jul 2014, at 16:05, Francesco Cesarini wrote: >>>> Hi all, >>>> >>>> a shameless plug. Steve Vinoski and I are working on a book focused on distributed, scalable systems with OTP. It is available from O'Reilly as an Early Release from their website. The first eight chapters we released cover (in great detail) all other behaviours. We started writing, and before we knew it, we had several hundred pages on behaviours alone. Last week, two new chapters, including the Introduction & Special processes and Implementing your own behaviours were released. We are now focusing on release handling (hgg), code upgrade and architectural patterns. >>>> >>>> What is available is an unedited draft, with new chapters and improvements published as they become available. You can find more info here: >>>> >>>> http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>>> >>>> If you use discount code authd, you will get 50% off the Early Release, and 40% on pre-orders of the the printed copy. >>>> >>>> Looking forward to your feedback, >>>> >> Not for nothing, but $50 for the undedited work-in-progress seems a bit steep (and yes, I saw the discount code). I also note that "rough cuts" aren't included in my Safari subscription. >> >> Usual practice that I've seen is for works-in-progress to be free, with tools that support comments from early readers. >> >> Sounds interesting - but, from the TOC (all that's available without paying), it sure looks like it covers the same ground as "Erlang and OTP in Action." >> >> Miles Fidelman >> >> -- >> In theory, there is no difference between theory and practice. >> In practice, there is. .... Yogi Berra >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Erlang Solutions Ltd. http://www.erlang-solutions.com From raould@REDACTED Tue Jul 8 20:39:06 2014 From: raould@REDACTED (Raoul Duke) Date: Tue, 8 Jul 2014 11:39:06 -0700 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <53BC3A25.8030708@erlang-solutions.com> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> Message-ID: i haven't looked at the books here, but having been somebody who produced things and wondered how anybody could ever make a living at it, and knowing that writers throughout history rarely made much after the publisher etc. got a cut, i didn't have the same reaction to the $50 price tag. can't say i'd *spend* the $50 since i'm broke and there's a zillion other things to spend $50 on first. i'd try to get it via inter library loan or something :-). On Tue, Jul 8, 2014 at 11:36 AM, Francesco Cesarini wrote: > Alas, that is how O'Reilly price their books. Having a high price and then > discounting is not the approach I would pick. That is why discount codes are > being handed out on public mailing lists and social media. I recommend you > use them (Read, no one pays full price for an O'Reilly book). As an unedited > book, the cost is for the final book which we hope will complement what is > already out there. It is a different approach to OTP in action. One I've > been using for 15 years when teaching OTP. > > /F > > On 08/07/2014 19:14, Lee Sylvester wrote: >> >> Wow, $50? Yeah, that is steep. If this were some black arts compiler >> book or video encoding bible, then that would be something else. But a book >> on Erlang/OTP just doesn?t fall into that bracket? >> >> I may wait til it falls in the bargain bucket ;-) >> >> Lee >> >> On 8 Jul 2014, at 19:08, Miles Fidelman >> wrote: >> >>> On 8 Jul 2014, at 16:05, Francesco Cesarini >>> wrote: >>>>> >>>>> Hi all, >>>>> >>>>> a shameless plug. Steve Vinoski and I are working on a book focused on >>>>> distributed, scalable systems with OTP. It is available from O'Reilly as an >>>>> Early Release from their website. The first eight chapters we released cover >>>>> (in great detail) all other behaviours. We started writing, and before we >>>>> knew it, we had several hundred pages on behaviours alone. Last week, two >>>>> new chapters, including the Introduction & Special processes and >>>>> Implementing your own behaviours were released. We are now focusing on >>>>> release handling (hgg), code upgrade and architectural patterns. >>>>> >>>>> What is available is an unedited draft, with new chapters and >>>>> improvements published as they become available. You can find more info >>>>> here: >>>>> >>>>> >>>>> http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>>>> >>>>> If you use discount code authd, you will get 50% off the Early Release, >>>>> and 40% on pre-orders of the the printed copy. >>>>> >>>>> Looking forward to your feedback, >>>>> >>> Not for nothing, but $50 for the undedited work-in-progress seems a bit >>> steep (and yes, I saw the discount code). I also note that "rough cuts" >>> aren't included in my Safari subscription. >>> >>> Usual practice that I've seen is for works-in-progress to be free, with >>> tools that support comments from early readers. >>> >>> Sounds interesting - but, from the TOC (all that's available without >>> paying), it sure looks like it covers the same ground as "Erlang and OTP in >>> Action." >>> >>> Miles Fidelman >>> >>> -- >>> In theory, there is no difference between theory and practice. >>> In practice, there is. .... Yogi Berra >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > -- > Erlang Solutions Ltd. > http://www.erlang-solutions.com > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From lee.sylvester@REDACTED Tue Jul 8 20:40:02 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 8 Jul 2014 19:40:02 +0100 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <53BC3A25.8030708@erlang-solutions.com> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> Message-ID: :-) That?s why I wouldn?t write a book for O?Reilly. Wrox were bad enough. I vowed my next book would be open source material. No one writes a book for money, as anyone who?s written a book will know it?s not worth it from the second book onward and if you wrote a second book, you would be of a mind where the money wasn?t important. Strange thing, books. Lee On 8 Jul 2014, at 19:36, Francesco Cesarini wrote: > Alas, that is how O'Reilly price their books. Having a high price and then discounting is not the approach I would pick. That is why discount codes are being handed out on public mailing lists and social media. I recommend you use them (Read, no one pays full price for an O'Reilly book). As an unedited book, the cost is for the final book which we hope will complement what is already out there. It is a different approach to OTP in action. One I've been using for 15 years when teaching OTP. > > /F > > On 08/07/2014 19:14, Lee Sylvester wrote: >> Wow, $50? Yeah, that is steep. If this were some black arts compiler book or video encoding bible, then that would be something else. But a book on Erlang/OTP just doesn?t fall into that bracket? >> >> I may wait til it falls in the bargain bucket ;-) >> >> Lee >> >> On 8 Jul 2014, at 19:08, Miles Fidelman wrote: >> >>> On 8 Jul 2014, at 16:05, Francesco Cesarini wrote: >>>>> Hi all, >>>>> >>>>> a shameless plug. Steve Vinoski and I are working on a book focused on distributed, scalable systems with OTP. It is available from O'Reilly as an Early Release from their website. The first eight chapters we released cover (in great detail) all other behaviours. We started writing, and before we knew it, we had several hundred pages on behaviours alone. Last week, two new chapters, including the Introduction & Special processes and Implementing your own behaviours were released. We are now focusing on release handling (hgg), code upgrade and architectural patterns. >>>>> >>>>> What is available is an unedited draft, with new chapters and improvements published as they become available. You can find more info here: >>>>> >>>>> http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>>>> >>>>> If you use discount code authd, you will get 50% off the Early Release, and 40% on pre-orders of the the printed copy. >>>>> >>>>> Looking forward to your feedback, >>>>> >>> Not for nothing, but $50 for the undedited work-in-progress seems a bit steep (and yes, I saw the discount code). I also note that "rough cuts" aren't included in my Safari subscription. >>> >>> Usual practice that I've seen is for works-in-progress to be free, with tools that support comments from early readers. >>> >>> Sounds interesting - but, from the TOC (all that's available without paying), it sure looks like it covers the same ground as "Erlang and OTP in Action." >>> >>> Miles Fidelman >>> >>> -- >>> In theory, there is no difference between theory and practice. >>> In practice, there is. .... Yogi Berra >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -- > Erlang Solutions Ltd. > http://www.erlang-solutions.com > From lee.sylvester@REDACTED Tue Jul 8 20:41:38 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 8 Jul 2014 19:41:38 +0100 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> Message-ID: <5DFC2AF5-8AEC-4520-85A3-CF06ADEF7605@gmail.com> Well that?s the thing. I?d hazard a guess (and could be wrong) that Francesco?s and Steve?s contract offers them $1 for every printed book sold and 50 cents for every ebook. Was I close? ;-) Lee On 8 Jul 2014, at 19:39, Raoul Duke wrote: > i haven't looked at the books here, but having been somebody who > produced things and wondered how anybody could ever make a living at > it, and knowing that writers throughout history rarely made much after > the publisher etc. got a cut, i didn't have the same reaction to the > $50 price tag. can't say i'd *spend* the $50 since i'm broke and > there's a zillion other things to spend $50 on first. i'd try to get > it via inter library loan or something :-). > > On Tue, Jul 8, 2014 at 11:36 AM, Francesco Cesarini > wrote: >> Alas, that is how O'Reilly price their books. Having a high price and then >> discounting is not the approach I would pick. That is why discount codes are >> being handed out on public mailing lists and social media. I recommend you >> use them (Read, no one pays full price for an O'Reilly book). As an unedited >> book, the cost is for the final book which we hope will complement what is >> already out there. It is a different approach to OTP in action. One I've >> been using for 15 years when teaching OTP. >> >> /F >> >> On 08/07/2014 19:14, Lee Sylvester wrote: >>> >>> Wow, $50? Yeah, that is steep. If this were some black arts compiler >>> book or video encoding bible, then that would be something else. But a book >>> on Erlang/OTP just doesn?t fall into that bracket? >>> >>> I may wait til it falls in the bargain bucket ;-) >>> >>> Lee >>> >>> On 8 Jul 2014, at 19:08, Miles Fidelman >>> wrote: >>> >>>> On 8 Jul 2014, at 16:05, Francesco Cesarini >>>> wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> a shameless plug. Steve Vinoski and I are working on a book focused on >>>>>> distributed, scalable systems with OTP. It is available from O'Reilly as an >>>>>> Early Release from their website. The first eight chapters we released cover >>>>>> (in great detail) all other behaviours. We started writing, and before we >>>>>> knew it, we had several hundred pages on behaviours alone. Last week, two >>>>>> new chapters, including the Introduction & Special processes and >>>>>> Implementing your own behaviours were released. We are now focusing on >>>>>> release handling (hgg), code upgrade and architectural patterns. >>>>>> >>>>>> What is available is an unedited draft, with new chapters and >>>>>> improvements published as they become available. You can find more info >>>>>> here: >>>>>> >>>>>> >>>>>> http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>>>>> >>>>>> If you use discount code authd, you will get 50% off the Early Release, >>>>>> and 40% on pre-orders of the the printed copy. >>>>>> >>>>>> Looking forward to your feedback, >>>>>> >>>> Not for nothing, but $50 for the undedited work-in-progress seems a bit >>>> steep (and yes, I saw the discount code). I also note that "rough cuts" >>>> aren't included in my Safari subscription. >>>> >>>> Usual practice that I've seen is for works-in-progress to be free, with >>>> tools that support comments from early readers. >>>> >>>> Sounds interesting - but, from the TOC (all that's available without >>>> paying), it sure looks like it covers the same ground as "Erlang and OTP in >>>> Action." >>>> >>>> Miles Fidelman >>>> >>>> -- >>>> In theory, there is no difference between theory and practice. >>>> In practice, there is. .... Yogi Berra >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> -- >> Erlang Solutions Ltd. >> http://www.erlang-solutions.com >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From anthonym@REDACTED Tue Jul 8 20:41:21 2014 From: anthonym@REDACTED (ANTHONY MOLINARO) Date: Tue, 8 Jul 2014 11:41:21 -0700 Subject: [erlang-questions] Records referring to each other Message-ID: Hi, I came across it when trying to compile some generated test code in the thrift code base. The record definition looks like this, -ifndef(_recursive_types_included). -define(_recursive_types_included, yeah). %% struct coRec -record(coRec, {other :: #coRec2{}}). %% struct coRec2 -record(coRec2, {other :: #coRec{}}). -endif. This returns the error test/recursive_types.hrl:16: record coRec2 undefined I?m wondering if there is any way to get this to work and include the types, or whether I should modify the generation to not include record types? Thanks, -Anthony From raould@REDACTED Tue Jul 8 20:44:01 2014 From: raould@REDACTED (Raoul Duke) Date: Tue, 8 Jul 2014 11:44:01 -0700 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> Message-ID: > :-) That?s why I wouldn?t write a book for O?Reilly. Wrox were bad enough. I vowed my next book would be open source material. No one writes a book for money, as anyone who?s written a book will know it?s not worth it from the second book onward and if you wrote a second book, you would be of a mind where the money wasn?t important. or start another publishing company ;-) From lee.sylvester@REDACTED Tue Jul 8 20:44:41 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 8 Jul 2014 19:44:41 +0100 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> Message-ID: <109835D8-73EC-4E8E-AE84-EDB007F46C0C@gmail.com> In fact (and yes, I realise I?m waffling), that the contract can sometimes seem good, until you reason the bit where it mentions major cuts to royalties when sold at discount, which all major retailers (Amazon etc.) tend to do. So, the author ends up with practically nothing. A book I wrote in 2006 is still selling copies (about 10k to date, I believe), yet I?ve still not earned a thing from it beyond the initial upfront lump sum, which totalled about $800. When you compare that to the 600 man hours I put into it, it really isn?t worth it :-D But then, I never wrote it for the cash. Lee On 8 Jul 2014, at 19:39, Raoul Duke wrote: > i haven't looked at the books here, but having been somebody who > produced things and wondered how anybody could ever make a living at > it, and knowing that writers throughout history rarely made much after > the publisher etc. got a cut, i didn't have the same reaction to the > $50 price tag. can't say i'd *spend* the $50 since i'm broke and > there's a zillion other things to spend $50 on first. i'd try to get > it via inter library loan or something :-). > > On Tue, Jul 8, 2014 at 11:36 AM, Francesco Cesarini > wrote: >> Alas, that is how O'Reilly price their books. Having a high price and then >> discounting is not the approach I would pick. That is why discount codes are >> being handed out on public mailing lists and social media. I recommend you >> use them (Read, no one pays full price for an O'Reilly book). As an unedited >> book, the cost is for the final book which we hope will complement what is >> already out there. It is a different approach to OTP in action. One I've >> been using for 15 years when teaching OTP. >> >> /F >> >> On 08/07/2014 19:14, Lee Sylvester wrote: >>> >>> Wow, $50? Yeah, that is steep. If this were some black arts compiler >>> book or video encoding bible, then that would be something else. But a book >>> on Erlang/OTP just doesn?t fall into that bracket? >>> >>> I may wait til it falls in the bargain bucket ;-) >>> >>> Lee >>> >>> On 8 Jul 2014, at 19:08, Miles Fidelman >>> wrote: >>> >>>> On 8 Jul 2014, at 16:05, Francesco Cesarini >>>> wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> a shameless plug. Steve Vinoski and I are working on a book focused on >>>>>> distributed, scalable systems with OTP. It is available from O'Reilly as an >>>>>> Early Release from their website. The first eight chapters we released cover >>>>>> (in great detail) all other behaviours. We started writing, and before we >>>>>> knew it, we had several hundred pages on behaviours alone. Last week, two >>>>>> new chapters, including the Introduction & Special processes and >>>>>> Implementing your own behaviours were released. We are now focusing on >>>>>> release handling (hgg), code upgrade and architectural patterns. >>>>>> >>>>>> What is available is an unedited draft, with new chapters and >>>>>> improvements published as they become available. You can find more info >>>>>> here: >>>>>> >>>>>> >>>>>> http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>>>>> >>>>>> If you use discount code authd, you will get 50% off the Early Release, >>>>>> and 40% on pre-orders of the the printed copy. >>>>>> >>>>>> Looking forward to your feedback, >>>>>> >>>> Not for nothing, but $50 for the undedited work-in-progress seems a bit >>>> steep (and yes, I saw the discount code). I also note that "rough cuts" >>>> aren't included in my Safari subscription. >>>> >>>> Usual practice that I've seen is for works-in-progress to be free, with >>>> tools that support comments from early readers. >>>> >>>> Sounds interesting - but, from the TOC (all that's available without >>>> paying), it sure looks like it covers the same ground as "Erlang and OTP in >>>> Action." >>>> >>>> Miles Fidelman >>>> >>>> -- >>>> In theory, there is no difference between theory and practice. >>>> In practice, there is. .... Yogi Berra >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> -- >> Erlang Solutions Ltd. >> http://www.erlang-solutions.com >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mfidelman@REDACTED Tue Jul 8 20:44:40 2014 From: mfidelman@REDACTED (Miles Fidelman) Date: Tue, 08 Jul 2014 14:44:40 -0400 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <53BC3A25.8030708@erlang-solutions.com> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> Message-ID: <53BC3C18.70507@meetinghouse.net> I was actually thinking of the CouchDB book, when I wrote my comment. - also published by O'Reilly - also available free on-line - in both work-in-progress and final forms - and there's a paid-for hardcopy on my bookshelf See http://guide.couchdb.org/index.html Francesco Cesarini wrote: > Alas, that is how O'Reilly price their books. Having a high price and > then discounting is not the approach I would pick. That is why > discount codes are being handed out on public mailing lists and social > media. I recommend you use them (Read, no one pays full price for an > O'Reilly book). As an unedited book, the cost is for the final book > which we hope will complement what is already out there. It is a > different approach to OTP in action. One I've been using for 15 years > when teaching OTP. > > /F > > On 08/07/2014 19:14, Lee Sylvester wrote: >> Wow, $50? Yeah, that is steep. If this were some black arts >> compiler book or video encoding bible, then that would be something >> else. But a book on Erlang/OTP just doesn?t fall into that bracket? >> >> I may wait til it falls in the bargain bucket ;-) >> >> Lee >> >> On 8 Jul 2014, at 19:08, Miles Fidelman >> wrote: >> >>> On 8 Jul 2014, at 16:05, Francesco Cesarini >>> wrote: >>>>> Hi all, >>>>> >>>>> a shameless plug. Steve Vinoski and I are working on a book >>>>> focused on distributed, scalable systems with OTP. It is available >>>>> from O'Reilly as an Early Release from their website. The first >>>>> eight chapters we released cover (in great detail) all other >>>>> behaviours. We started writing, and before we knew it, we had >>>>> several hundred pages on behaviours alone. Last week, two new >>>>> chapters, including the Introduction & Special processes and >>>>> Implementing your own behaviours were released. We are now >>>>> focusing on release handling (hgg), code upgrade and architectural >>>>> patterns. >>>>> >>>>> What is available is an unedited draft, with new chapters and >>>>> improvements published as they become available. You can find more >>>>> info here: >>>>> >>>>> http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>>>> >>>>> >>>>> If you use discount code authd, you will get 50% off the Early >>>>> Release, and 40% on pre-orders of the the printed copy. >>>>> >>>>> Looking forward to your feedback, >>>>> >>> Not for nothing, but $50 for the undedited work-in-progress seems a >>> bit steep (and yes, I saw the discount code). I also note that >>> "rough cuts" aren't included in my Safari subscription. >>> >>> Usual practice that I've seen is for works-in-progress to be free, >>> with tools that support comments from early readers. >>> >>> Sounds interesting - but, from the TOC (all that's available without >>> paying), it sure looks like it covers the same ground as "Erlang and >>> OTP in Action." >>> >>> Miles Fidelman >>> >>> -- >>> In theory, there is no difference between theory and practice. >>> In practice, there is. .... Yogi Berra >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From lee.sylvester@REDACTED Tue Jul 8 20:45:18 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 8 Jul 2014 19:45:18 +0100 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> Message-ID: Well, yeah. Where fairness is due, many publishing companies are struggling these days. Odd, really. On 8 Jul 2014, at 19:44, Raoul Duke wrote: >> :-) That?s why I wouldn?t write a book for O?Reilly. Wrox were bad enough. I vowed my next book would be open source material. No one writes a book for money, as anyone who?s written a book will know it?s not worth it from the second book onward and if you wrote a second book, you would be of a mind where the money wasn?t important. > > > > or start another publishing company ;-) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From elbrujohalcon@REDACTED Tue Jul 8 20:47:37 2014 From: elbrujohalcon@REDACTED (Brujo Benavides @ Inaka) Date: Tue, 8 Jul 2014 15:47:37 -0300 Subject: [erlang-questions] Records referring to each other In-Reply-To: References: Message-ID: <1F0C2E00-5FA4-46FA-B65E-952C289CBA40@inaka.net> This seems to work fine for me: -module(corec). -record(corec1, {other :: corec2()}). -record(corec2, {other :: corec1()}). -type corec1() :: #corec1{}. -type corec2() :: #corec2{}. Best regards, On Jul 8, 2014, at 15:41, ANTHONY MOLINARO wrote: > Hi, > > I came across it when trying to compile some generated test code in the thrift code base. The record definition looks like this, > > -ifndef(_recursive_types_included). > -define(_recursive_types_included, yeah). > > %% struct coRec > > -record(coRec, {other :: #coRec2{}}). > > %% struct coRec2 > > -record(coRec2, {other :: #coRec{}}). > > -endif. > > This returns the error > > test/recursive_types.hrl:16: record coRec2 undefined > > I?m wondering if there is any way to get this to work and include the types, or whether I should modify the generation to not include record types? > > Thanks, > > -Anthony > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From francesco@REDACTED Tue Jul 8 20:51:48 2014 From: francesco@REDACTED (Francesco Cesarini) Date: Tue, 08 Jul 2014 19:51:48 +0100 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> Message-ID: <53BC3DC4.7000600@erlang-solutions.com> You do not write books because you believe you are going to make money out of them. You write them because you are passionate about the subject. If you calculate what Simon and I made from our first book, I am not sure we've hit minimum wage yet. From my side, I want to document my approach to teaching OTP, as I think it will work in writing as well as it does in the classroom. From the feedback Steve and I have received so far, we are right on track. Reiterating an email on this list from 2008, I want to see a whole bookshelf of Erlang/OTP books out there. I found the experience of working with O'Reilly really positive the first time around. From the editor, the production team (Graphics, copy editor, proof readers, etc) as well as their marketing and conferences. And this time around, it is just as good, if not better. As an author, I could not recommend them more highly. I want to write books, I do not want to do all of the other stuff associated with getting it out. It is just a false economy. /F PS. For those who can't afford 25$, try before you buy. It is called BitTorrent. On 08/07/2014 19:39, Raoul Duke wrote: > i haven't looked at the books here, but having been somebody who > produced things and wondered how anybody could ever make a living at > it, and knowing that writers throughout history rarely made much after > the publisher etc. got a cut, i didn't have the same reaction to the > $50 price tag. can't say i'd *spend* the $50 since i'm broke and > there's a zillion other things to spend $50 on first. i'd try to get > it via inter library loan or something :-). > > On Tue, Jul 8, 2014 at 11:36 AM, Francesco Cesarini > wrote: >> Alas, that is how O'Reilly price their books. Having a high price and then >> discounting is not the approach I would pick. That is why discount codes are >> being handed out on public mailing lists and social media. I recommend you >> use them (Read, no one pays full price for an O'Reilly book). As an unedited >> book, the cost is for the final book which we hope will complement what is >> already out there. It is a different approach to OTP in action. One I've >> been using for 15 years when teaching OTP. >> >> /F >> >> On 08/07/2014 19:14, Lee Sylvester wrote: >>> Wow, $50? Yeah, that is steep. If this were some black arts compiler >>> book or video encoding bible, then that would be something else. But a book >>> on Erlang/OTP just doesn?t fall into that bracket? >>> >>> I may wait til it falls in the bargain bucket ;-) >>> >>> Lee >>> >>> On 8 Jul 2014, at 19:08, Miles Fidelman >>> wrote: >>> >>>> On 8 Jul 2014, at 16:05, Francesco Cesarini >>>> wrote: >>>>>> Hi all, >>>>>> >>>>>> a shameless plug. Steve Vinoski and I are working on a book focused on >>>>>> distributed, scalable systems with OTP. It is available from O'Reilly as an >>>>>> Early Release from their website. The first eight chapters we released cover >>>>>> (in great detail) all other behaviours. We started writing, and before we >>>>>> knew it, we had several hundred pages on behaviours alone. Last week, two >>>>>> new chapters, including the Introduction & Special processes and >>>>>> Implementing your own behaviours were released. We are now focusing on >>>>>> release handling (hgg), code upgrade and architectural patterns. >>>>>> >>>>>> What is available is an unedited draft, with new chapters and >>>>>> improvements published as they become available. You can find more info >>>>>> here: >>>>>> >>>>>> >>>>>> http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>>>>> >>>>>> If you use discount code authd, you will get 50% off the Early Release, >>>>>> and 40% on pre-orders of the the printed copy. >>>>>> >>>>>> Looking forward to your feedback, >>>>>> >>>> Not for nothing, but $50 for the undedited work-in-progress seems a bit >>>> steep (and yes, I saw the discount code). I also note that "rough cuts" >>>> aren't included in my Safari subscription. >>>> >>>> Usual practice that I've seen is for works-in-progress to be free, with >>>> tools that support comments from early readers. >>>> >>>> Sounds interesting - but, from the TOC (all that's available without >>>> paying), it sure looks like it covers the same ground as "Erlang and OTP in >>>> Action." >>>> >>>> Miles Fidelman >>>> >>>> -- >>>> In theory, there is no difference between theory and practice. >>>> In practice, there is. .... Yogi Berra >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> -- >> Erlang Solutions Ltd. >> http://www.erlang-solutions.com >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Erlang Solutions Ltd. http://www.erlang-solutions.com From mfidelman@REDACTED Tue Jul 8 20:54:20 2014 From: mfidelman@REDACTED (Miles Fidelman) Date: Tue, 08 Jul 2014 14:54:20 -0400 Subject: [erlang-questions] on writing [was: ANN: Designing for Scalability with Erlang/OTP by O'Reilly] In-Reply-To: References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> Message-ID: <53BC3E5C.8020207@meetinghouse.net> Yeah... agree on that. In a previous life, I wrote two books: First was on implementing Internet services in libraries (kind of dates me, don't it :-). Started out as the final report on a funded project, ended up as a book for the American Library Association Press. Phenomenal experience - an advance, an incredible editor, and it ended up as a book club selection (who knew - the ALA has a book club for their members - a copy ended up in every mid- and large- library in the country). I ended up making money on the book (not a lot, but worth the effort). Second was on municipal networks. Started out as a very high-priced handbook, sold by Government Technology Press - something like the third book they published. Intent was to co-market with their weekly trade rag, conferences and seminars. Horrible experience: Horrible editor (kept making changes that changed the meaning of things - had to fight to fix things), then they decided they didn't want to publish books after all - shut down the book division. Then again, I ended up with a few hundred remainders, and camera-ready originals -- ended up selling a bunch at $125 each (instead of the original $200+) through our web site, and bundling them with a dozen seminars given over a year-long period. Made out very well, particularly on the seminars and some consulting gigs that resulted from the seminars. Next book will definitely be self-published - not sure if will be open source, though. Definitely a "strange thing, books." Cheers, Miles Lee Sylvester wrote: > :-) That?s why I wouldn?t write a book for O?Reilly. Wrox were bad enough. I vowed my next book would be open source material. No one writes a book for money, as anyone who?s written a book will know it?s not worth it from the second book onward and if you wrote a second book, you would be of a mind where the money wasn?t important. > > Strange thing, books. > > Lee > > > On 8 Jul 2014, at 19:36, Francesco Cesarini wrote: > >> Alas, that is how O'Reilly price their books. Having a high price and then discounting is not the approach I would pick. That is why discount codes are being handed out on public mailing lists and social media. I recommend you use them (Read, no one pays full price for an O'Reilly book). As an unedited book, the cost is for the final book which we hope will complement what is already out there. It is a different approach to OTP in action. One I've been using for 15 years when teaching OTP. >> >> /F >> >> On 08/07/2014 19:14, Lee Sylvester wrote: >>> Wow, $50? Yeah, that is steep. If this were some black arts compiler book or video encoding bible, then that would be something else. But a book on Erlang/OTP just doesn?t fall into that bracket? >>> >>> I may wait til it falls in the bargain bucket ;-) >>> >>> Lee >>> >>> On 8 Jul 2014, at 19:08, Miles Fidelman wrote: >>> >>>> On 8 Jul 2014, at 16:05, Francesco Cesarini wrote: >>>>>> Hi all, >>>>>> >>>>>> a shameless plug. Steve Vinoski and I are working on a book focused on distributed, scalable systems with OTP. It is available from O'Reilly as an Early Release from their website. The first eight chapters we released cover (in great detail) all other behaviours. We started writing, and before we knew it, we had several hundred pages on behaviours alone. Last week, two new chapters, including the Introduction & Special processes and Implementing your own behaviours were released. We are now focusing on release handling (hgg), code upgrade and architectural patterns. >>>>>> >>>>>> What is available is an unedited draft, with new chapters and improvements published as they become available. You can find more info here: >>>>>> >>>>>> http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>>>>> >>>>>> If you use discount code authd, you will get 50% off the Early Release, and 40% on pre-orders of the the printed copy. >>>>>> >>>>>> Looking forward to your feedback, >>>>>> >>>> Not for nothing, but $50 for the undedited work-in-progress seems a bit steep (and yes, I saw the discount code). I also note that "rough cuts" aren't included in my Safari subscription. >>>> >>>> Usual practice that I've seen is for works-in-progress to be free, with tools that support comments from early readers. >>>> >>>> Sounds interesting - but, from the TOC (all that's available without paying), it sure looks like it covers the same ground as "Erlang and OTP in Action." >>>> >>>> Miles Fidelman >>>> >>>> -- >>>> In theory, there is no difference between theory and practice. >>>> In practice, there is. .... Yogi Berra >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> -- >> Erlang Solutions Ltd. >> http://www.erlang-solutions.com >> -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From lee.sylvester@REDACTED Tue Jul 8 20:55:44 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 8 Jul 2014 19:55:44 +0100 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <53BC3DC4.7000600@erlang-solutions.com> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3DC4.7000600@erlang-solutions.com> Message-ID: <01D5657C-8AEE-47AD-8C76-2BDD1F2A8C35@gmail.com> It?s not about the $25. With the effort you would have put into this, it?s certainly worth it, I?m sure. This is merely a debate of the business of publishing and authoring. In fact, I would feel better giving you $25 knowing you received it all than pay $25 to O?Reilly knowing you will see about $0.25 of it :-) On 8 Jul 2014, at 19:51, Francesco Cesarini wrote: > You do not write books because you believe you are going to make money out of them. You write them because you are passionate about the subject. If you calculate what Simon and I made from our first book, I am not sure we've hit minimum wage yet. From my side, I want to document my approach to teaching OTP, as I think it will work in writing as well as it does in the classroom. From the feedback Steve and I have received so far, we are right on track. Reiterating an email on this list from 2008, I want to see a whole bookshelf of Erlang/OTP books out there. > > I found the experience of working with O'Reilly really positive the first time around. From the editor, the production team (Graphics, copy editor, proof readers, etc) as well as their marketing and conferences. And this time around, it is just as good, if not better. As an author, I could not recommend them more highly. I want to write books, I do not want to do all of the other stuff associated with getting it out. It is just a false economy. > > /F > > PS. For those who can't afford 25$, try before you buy. It is called BitTorrent. > > On 08/07/2014 19:39, Raoul Duke wrote: >> i haven't looked at the books here, but having been somebody who >> produced things and wondered how anybody could ever make a living at >> it, and knowing that writers throughout history rarely made much after >> the publisher etc. got a cut, i didn't have the same reaction to the >> $50 price tag. can't say i'd *spend* the $50 since i'm broke and >> there's a zillion other things to spend $50 on first. i'd try to get >> it via inter library loan or something :-). >> >> On Tue, Jul 8, 2014 at 11:36 AM, Francesco Cesarini >> wrote: >>> Alas, that is how O'Reilly price their books. Having a high price and then >>> discounting is not the approach I would pick. That is why discount codes are >>> being handed out on public mailing lists and social media. I recommend you >>> use them (Read, no one pays full price for an O'Reilly book). As an unedited >>> book, the cost is for the final book which we hope will complement what is >>> already out there. It is a different approach to OTP in action. One I've >>> been using for 15 years when teaching OTP. >>> >>> /F >>> >>> On 08/07/2014 19:14, Lee Sylvester wrote: >>>> Wow, $50? Yeah, that is steep. If this were some black arts compiler >>>> book or video encoding bible, then that would be something else. But a book >>>> on Erlang/OTP just doesn?t fall into that bracket? >>>> >>>> I may wait til it falls in the bargain bucket ;-) >>>> >>>> Lee >>>> >>>> On 8 Jul 2014, at 19:08, Miles Fidelman >>>> wrote: >>>> >>>>> On 8 Jul 2014, at 16:05, Francesco Cesarini >>>>> wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> a shameless plug. Steve Vinoski and I are working on a book focused on >>>>>>> distributed, scalable systems with OTP. It is available from O'Reilly as an >>>>>>> Early Release from their website. The first eight chapters we released cover >>>>>>> (in great detail) all other behaviours. We started writing, and before we >>>>>>> knew it, we had several hundred pages on behaviours alone. Last week, two >>>>>>> new chapters, including the Introduction & Special processes and >>>>>>> Implementing your own behaviours were released. We are now focusing on >>>>>>> release handling (hgg), code upgrade and architectural patterns. >>>>>>> >>>>>>> What is available is an unedited draft, with new chapters and >>>>>>> improvements published as they become available. You can find more info >>>>>>> here: >>>>>>> >>>>>>> >>>>>>> http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>>>>>> >>>>>>> If you use discount code authd, you will get 50% off the Early Release, >>>>>>> and 40% on pre-orders of the the printed copy. >>>>>>> >>>>>>> Looking forward to your feedback, >>>>>>> >>>>> Not for nothing, but $50 for the undedited work-in-progress seems a bit >>>>> steep (and yes, I saw the discount code). I also note that "rough cuts" >>>>> aren't included in my Safari subscription. >>>>> >>>>> Usual practice that I've seen is for works-in-progress to be free, with >>>>> tools that support comments from early readers. >>>>> >>>>> Sounds interesting - but, from the TOC (all that's available without >>>>> paying), it sure looks like it covers the same ground as "Erlang and OTP in >>>>> Action." >>>>> >>>>> Miles Fidelman >>>>> >>>>> -- >>>>> In theory, there is no difference between theory and practice. >>>>> In practice, there is. .... Yogi Berra >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> -- >>> Erlang Solutions Ltd. >>> http://www.erlang-solutions.com >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -- > Erlang Solutions Ltd. > http://www.erlang-solutions.com > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mfidelman@REDACTED Tue Jul 8 20:58:58 2014 From: mfidelman@REDACTED (Miles Fidelman) Date: Tue, 08 Jul 2014 14:58:58 -0400 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <01D5657C-8AEE-47AD-8C76-2BDD1F2A8C35@gmail.com> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3DC4.7000600@erlang-solutions.com> <01D5657C-8AEE-47AD-8C76-2BDD1F2A8C35@gmail.com> Message-ID: <53BC3F72.6060304@meetinghouse.net> Likewise. Kickstarter or Indiegogo seems like a much better model for the writing period. Or maybe these guys: www.patreon.com - interesting model for writers, musicians, and such - instead of funding a project, you subscribe to a person - at $x/chapter or /song. Lee Sylvester wrote: > It?s not about the $25. With the effort you would have put into this, it?s certainly worth it, I?m sure. This is merely a debate of the business of publishing and authoring. In fact, I would feel better giving you $25 knowing you received it all than pay $25 to O?Reilly knowing you will see about $0.25 of it :-) > > > On 8 Jul 2014, at 19:51, Francesco Cesarini wrote: > >> You do not write books because you believe you are going to make money out of them. You write them because you are passionate about the subject. If you calculate what Simon and I made from our first book, I am not sure we've hit minimum wage yet. From my side, I want to document my approach to teaching OTP, as I think it will work in writing as well as it does in the classroom. From the feedback Steve and I have received so far, we are right on track. Reiterating an email on this list from 2008, I want to see a whole bookshelf of Erlang/OTP books out there. >> >> I found the experience of working with O'Reilly really positive the first time around. From the editor, the production team (Graphics, copy editor, proof readers, etc) as well as their marketing and conferences. And this time around, it is just as good, if not better. As an author, I could not recommend them more highly. I want to write books, I do not want to do all of the other stuff associated with getting it out. It is just a false economy. >> >> /F >> >> PS. For those who can't afford 25$, try before you buy. It is called BitTorrent. >> >> On 08/07/2014 19:39, Raoul Duke wrote: >>> i haven't looked at the books here, but having been somebody who >>> produced things and wondered how anybody could ever make a living at >>> it, and knowing that writers throughout history rarely made much after >>> the publisher etc. got a cut, i didn't have the same reaction to the >>> $50 price tag. can't say i'd *spend* the $50 since i'm broke and >>> there's a zillion other things to spend $50 on first. i'd try to get >>> it via inter library loan or something :-). >>> >>> On Tue, Jul 8, 2014 at 11:36 AM, Francesco Cesarini >>> wrote: >>>> Alas, that is how O'Reilly price their books. Having a high price and then >>>> discounting is not the approach I would pick. That is why discount codes are >>>> being handed out on public mailing lists and social media. I recommend you >>>> use them (Read, no one pays full price for an O'Reilly book). As an unedited >>>> book, the cost is for the final book which we hope will complement what is >>>> already out there. It is a different approach to OTP in action. One I've >>>> been using for 15 years when teaching OTP. >>>> >>>> /F >>>> >>>> On 08/07/2014 19:14, Lee Sylvester wrote: >>>>> Wow, $50? Yeah, that is steep. If this were some black arts compiler >>>>> book or video encoding bible, then that would be something else. But a book >>>>> on Erlang/OTP just doesn?t fall into that bracket? >>>>> >>>>> I may wait til it falls in the bargain bucket ;-) >>>>> >>>>> Lee >>>>> >>>>> On 8 Jul 2014, at 19:08, Miles Fidelman >>>>> wrote: >>>>> >>>>>> On 8 Jul 2014, at 16:05, Francesco Cesarini >>>>>> wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> a shameless plug. Steve Vinoski and I are working on a book focused on >>>>>>>> distributed, scalable systems with OTP. It is available from O'Reilly as an >>>>>>>> Early Release from their website. The first eight chapters we released cover >>>>>>>> (in great detail) all other behaviours. We started writing, and before we >>>>>>>> knew it, we had several hundred pages on behaviours alone. Last week, two >>>>>>>> new chapters, including the Introduction & Special processes and >>>>>>>> Implementing your own behaviours were released. We are now focusing on >>>>>>>> release handling (hgg), code upgrade and architectural patterns. >>>>>>>> >>>>>>>> What is available is an unedited draft, with new chapters and >>>>>>>> improvements published as they become available. You can find more info >>>>>>>> here: >>>>>>>> >>>>>>>> >>>>>>>> http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>>>>>>> >>>>>>>> If you use discount code authd, you will get 50% off the Early Release, >>>>>>>> and 40% on pre-orders of the the printed copy. >>>>>>>> >>>>>>>> Looking forward to your feedback, >>>>>>>> >>>>>> Not for nothing, but $50 for the undedited work-in-progress seems a bit >>>>>> steep (and yes, I saw the discount code). I also note that "rough cuts" >>>>>> aren't included in my Safari subscription. >>>>>> >>>>>> Usual practice that I've seen is for works-in-progress to be free, with >>>>>> tools that support comments from early readers. >>>>>> >>>>>> Sounds interesting - but, from the TOC (all that's available without >>>>>> paying), it sure looks like it covers the same ground as "Erlang and OTP in >>>>>> Action." >>>>>> >>>>>> Miles Fidelman >>>>>> >>>>>> -- >>>>>> In theory, there is no difference between theory and practice. >>>>>> In practice, there is. .... Yogi Berra >>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> -- >>>> Erlang Solutions Ltd. >>>> http://www.erlang-solutions.com >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> -- >> Erlang Solutions Ltd. >> http://www.erlang-solutions.com >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From lee.sylvester@REDACTED Tue Jul 8 20:59:34 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 8 Jul 2014 19:59:34 +0100 Subject: [erlang-questions] on writing [was: ANN: Designing for Scalability with Erlang/OTP by O'Reilly] In-Reply-To: <53BC3E5C.8020207@meetinghouse.net> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3E5C.8020207@meetinghouse.net> Message-ID: <628D4972-FE84-4E9D-9D99-B72259F586E7@gmail.com> I also vowed my next book wouldn?t be a programming book. If there?s a reason to write a book, its that it?s the best way to learn a topic inside out :-) So, my next book is based on FPGA design and implementations for everyday projects (which is a programming book, I guess :-( ) On 8 Jul 2014, at 19:54, Miles Fidelman wrote: > Yeah... agree on that. In a previous life, I wrote two books: > > First was on implementing Internet services in libraries (kind of dates me, don't it :-). Started out as the final report on a funded project, ended up as a book for the American Library Association Press. Phenomenal experience - an advance, an incredible editor, and it ended up as a book club selection (who knew - the ALA has a book club for their members - a copy ended up in every mid- and large- library in the country). I ended up making money on the book (not a lot, but worth the effort). > > Second was on municipal networks. Started out as a very high-priced handbook, sold by Government Technology Press - something like the third book they published. Intent was to co-market with their weekly trade rag, conferences and seminars. Horrible experience: Horrible editor (kept making changes that changed the meaning of things - had to fight to fix things), then they decided they didn't want to publish books after all - shut down the book division. Then again, I ended up with a few hundred remainders, and camera-ready originals -- ended up selling a bunch at $125 each (instead of the original $200+) through our web site, and bundling them with a dozen seminars given over a year-long period. Made out very well, particularly on the seminars and some consulting gigs that resulted from the seminars. > > Next book will definitely be self-published - not sure if will be open source, though. > > Definitely a "strange thing, books." > > Cheers, > > Miles > > Lee Sylvester wrote: >> :-) That?s why I wouldn?t write a book for O?Reilly. Wrox were bad enough. I vowed my next book would be open source material. No one writes a book for money, as anyone who?s written a book will know it?s not worth it from the second book onward and if you wrote a second book, you would be of a mind where the money wasn?t important. >> >> Strange thing, books. >> >> Lee >> >> >> On 8 Jul 2014, at 19:36, Francesco Cesarini wrote: >> >>> Alas, that is how O'Reilly price their books. Having a high price and then discounting is not the approach I would pick. That is why discount codes are being handed out on public mailing lists and social media. I recommend you use them (Read, no one pays full price for an O'Reilly book). As an unedited book, the cost is for the final book which we hope will complement what is already out there. It is a different approach to OTP in action. One I've been using for 15 years when teaching OTP. >>> >>> /F >>> >>> On 08/07/2014 19:14, Lee Sylvester wrote: >>>> Wow, $50? Yeah, that is steep. If this were some black arts compiler book or video encoding bible, then that would be something else. But a book on Erlang/OTP just doesn?t fall into that bracket? >>>> >>>> I may wait til it falls in the bargain bucket ;-) >>>> >>>> Lee >>>> >>>> On 8 Jul 2014, at 19:08, Miles Fidelman wrote: >>>> >>>>> On 8 Jul 2014, at 16:05, Francesco Cesarini wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> a shameless plug. Steve Vinoski and I are working on a book focused on distributed, scalable systems with OTP. It is available from O'Reilly as an Early Release from their website. The first eight chapters we released cover (in great detail) all other behaviours. We started writing, and before we knew it, we had several hundred pages on behaviours alone. Last week, two new chapters, including the Introduction & Special processes and Implementing your own behaviours were released. We are now focusing on release handling (hgg), code upgrade and architectural patterns. >>>>>>> >>>>>>> What is available is an unedited draft, with new chapters and improvements published as they become available. You can find more info here: >>>>>>> >>>>>>> http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>>>>>> >>>>>>> If you use discount code authd, you will get 50% off the Early Release, and 40% on pre-orders of the the printed copy. >>>>>>> >>>>>>> Looking forward to your feedback, >>>>>>> >>>>> Not for nothing, but $50 for the undedited work-in-progress seems a bit steep (and yes, I saw the discount code). I also note that "rough cuts" aren't included in my Safari subscription. >>>>> >>>>> Usual practice that I've seen is for works-in-progress to be free, with tools that support comments from early readers. >>>>> >>>>> Sounds interesting - but, from the TOC (all that's available without paying), it sure looks like it covers the same ground as "Erlang and OTP in Action." >>>>> >>>>> Miles Fidelman >>>>> >>>>> -- >>>>> In theory, there is no difference between theory and practice. >>>>> In practice, there is. .... Yogi Berra >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> -- >>> Erlang Solutions Ltd. >>> http://www.erlang-solutions.com >>> > > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From francesco@REDACTED Tue Jul 8 21:09:27 2014 From: francesco@REDACTED (Francesco Cesarini) Date: Tue, 08 Jul 2014 20:09:27 +0100 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <01D5657C-8AEE-47AD-8C76-2BDD1F2A8C35@gmail.com> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3DC4.7000600@erlang-solutions.com> <01D5657C-8AEE-47AD-8C76-2BDD1F2A8C35@gmail.com> Message-ID: <53BC41E7.8070407@erlang-solutions.com> What I am being paid does not bother me, else I would not be putting every spare hour of the day into this project. I have seen the advantages of working with O'Reilly vs other publishing houses. They deserve their cut, else I would not have gone back. I were to self publish, the project would never get finished... And I would be forced to deal with things I do not like doing. Not worth it for me. I would rather have the backing and support O'Reilly are good at. In regards to how books are released, everyone has their choice. For the CouchDB book, it was already being written before O'Reilly came on board. The first book, Simon did not want an early release. In this one, Steve did not mind. There is no one size fits all. Yup, strange things books. I think my next one will be a guide book. /F On 08/07/2014 19:55, Lee Sylvester wrote: > It?s not about the $25. With the effort you would have put into this, it?s certainly worth it, I?m sure. This is merely a debate of the business of publishing and authoring. In fact, I would feel better giving you $25 knowing you received it all than pay $25 to O?Reilly knowing you will see about $0.25 of it :-) > > > On 8 Jul 2014, at 19:51, Francesco Cesarini wrote: > >> You do not write books because you believe you are going to make money out of them. You write them because you are passionate about the subject. If you calculate what Simon and I made from our first book, I am not sure we've hit minimum wage yet. From my side, I want to document my approach to teaching OTP, as I think it will work in writing as well as it does in the classroom. From the feedback Steve and I have received so far, we are right on track. Reiterating an email on this list from 2008, I want to see a whole bookshelf of Erlang/OTP books out there. >> >> I found the experience of working with O'Reilly really positive the first time around. From the editor, the production team (Graphics, copy editor, proof readers, etc) as well as their marketing and conferences. And this time around, it is just as good, if not better. As an author, I could not recommend them more highly. I want to write books, I do not want to do all of the other stuff associated with getting it out. It is just a false economy. >> >> /F >> >> PS. For those who can't afford 25$, try before you buy. It is called BitTorrent. >> >> On 08/07/2014 19:39, Raoul Duke wrote: >>> i haven't looked at the books here, but having been somebody who >>> produced things and wondered how anybody could ever make a living at >>> it, and knowing that writers throughout history rarely made much after >>> the publisher etc. got a cut, i didn't have the same reaction to the >>> $50 price tag. can't say i'd *spend* the $50 since i'm broke and >>> there's a zillion other things to spend $50 on first. i'd try to get >>> it via inter library loan or something :-). >>> >>> On Tue, Jul 8, 2014 at 11:36 AM, Francesco Cesarini >>> wrote: >>>> Alas, that is how O'Reilly price their books. Having a high price and then >>>> discounting is not the approach I would pick. That is why discount codes are >>>> being handed out on public mailing lists and social media. I recommend you >>>> use them (Read, no one pays full price for an O'Reilly book). As an unedited >>>> book, the cost is for the final book which we hope will complement what is >>>> already out there. It is a different approach to OTP in action. One I've >>>> been using for 15 years when teaching OTP. >>>> >>>> /F >>>> >>>> On 08/07/2014 19:14, Lee Sylvester wrote: >>>>> Wow, $50? Yeah, that is steep. If this were some black arts compiler >>>>> book or video encoding bible, then that would be something else. But a book >>>>> on Erlang/OTP just doesn?t fall into that bracket? >>>>> >>>>> I may wait til it falls in the bargain bucket ;-) >>>>> >>>>> Lee >>>>> >>>>> On 8 Jul 2014, at 19:08, Miles Fidelman >>>>> wrote: >>>>> >>>>>> On 8 Jul 2014, at 16:05, Francesco Cesarini >>>>>> wrote: >>>>>>>> Hi all, >>>>>>>> >>>>>>>> a shameless plug. Steve Vinoski and I are working on a book focused on >>>>>>>> distributed, scalable systems with OTP. It is available from O'Reilly as an >>>>>>>> Early Release from their website. The first eight chapters we released cover >>>>>>>> (in great detail) all other behaviours. We started writing, and before we >>>>>>>> knew it, we had several hundred pages on behaviours alone. Last week, two >>>>>>>> new chapters, including the Introduction & Special processes and >>>>>>>> Implementing your own behaviours were released. We are now focusing on >>>>>>>> release handling (hgg), code upgrade and architectural patterns. >>>>>>>> >>>>>>>> What is available is an unedited draft, with new chapters and >>>>>>>> improvements published as they become available. You can find more info >>>>>>>> here: >>>>>>>> >>>>>>>> >>>>>>>> http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>>>>>>> >>>>>>>> If you use discount code authd, you will get 50% off the Early Release, >>>>>>>> and 40% on pre-orders of the the printed copy. >>>>>>>> >>>>>>>> Looking forward to your feedback, >>>>>>>> >>>>>> Not for nothing, but $50 for the undedited work-in-progress seems a bit >>>>>> steep (and yes, I saw the discount code). I also note that "rough cuts" >>>>>> aren't included in my Safari subscription. >>>>>> >>>>>> Usual practice that I've seen is for works-in-progress to be free, with >>>>>> tools that support comments from early readers. >>>>>> >>>>>> Sounds interesting - but, from the TOC (all that's available without >>>>>> paying), it sure looks like it covers the same ground as "Erlang and OTP in >>>>>> Action." >>>>>> >>>>>> Miles Fidelman >>>>>> >>>>>> -- >>>>>> In theory, there is no difference between theory and practice. >>>>>> In practice, there is. .... Yogi Berra >>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> -- >>>> Erlang Solutions Ltd. >>>> http://www.erlang-solutions.com >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> -- >> Erlang Solutions Ltd. >> http://www.erlang-solutions.com >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -- Erlang Solutions Ltd. http://www.erlang-solutions.com From z@REDACTED Tue Jul 8 21:20:27 2014 From: z@REDACTED (Danil Zagoskin) Date: Tue, 8 Jul 2014 23:20:27 +0400 Subject: [erlang-questions] Distribution: small packets Message-ID: Hi! There is enormous packet-per-second in my distributed erlang setup. Under heavy load in single distribution socket and in one direction tcpdump shows rates like 40..100 packets with tcp-push flag set per millisecond. Size of majority of packets is 47..112 bytes while MTU on network interface is 8950 (jumbo-frames). If distribution driver aggregated messages to fit MTU it would be 100 times less packets in network. Given cluster of 3 nodes (very small) and both directions we get about extra 300K PPS which causes packet drops and tcp retransmits (thus increasing latency a lot). Is it possible to make erlang distribution push packets less often (one millisecond would be enough for me)? I'm sure this is not of net ticks because tracing on dist_util:con_loop process shows quite low {_, tick} message rate. Erlang/OTP version used is 17.0. -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Tue Jul 8 21:49:10 2014 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 8 Jul 2014 21:49:10 +0200 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <53BC3F72.6060304@meetinghouse.net> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3DC4.7000600@erlang-solutions.com> <01D5657C-8AEE-47AD-8C76-2BDD1F2A8C35@gmail.com> <53BC3F72.6060304@meetinghouse.net> Message-ID: On Tue, Jul 8, 2014 at 8:58 PM, Miles Fidelman wrote: > Likewise. Kickstarter or Indiegogo seems like a much better model for the > writing period. Or maybe these guys: www.patreon.com - interesting model > for writers, musicians, and such - instead of funding a project, you > subscribe to a person - at $x/chapter or /song. > > The problem is not financing the writing of the book - it's all the other stuff. A decent book needs: - an good technical editor - a copy editor - layout/typography - marketing - etc. Now while the author of a book will do this as a labour of love and is just happy if the book gets read the editors and so on actually want to get paid for their work - seen from O'Reillys POV any Erlang book is pretty risky - they probably need to sell 5,00 copies to break even. O'Reilly (and the prags etc.) re taking a big commercial risk by publishing FP/Erlang books - what they hope is that one day these books will reach the mainstream - Now of course, Francesco et al (and myself) could self publish, publishing through Lulu etc. is really easy (an cheap) so does not even need any kickstarter money - the problem is finding decent technical editors and proof readers. Self-published books will not find themselves on the bookshelves of my local bookstore. > > Lee Sylvester wrote: > >> It?s not about the $25. With the effort you would have put into this, >> it?s certainly worth it, I?m sure. This is merely a debate of the business >> of publishing and authoring. In fact, I would feel better giving you $25 >> knowing you received it all than pay $25 to O?Reilly knowing you will see >> about $0.25 of it :-) >> >> >> What about the money for Simon St. Laurent? - Let's assume that you thought 25$ was a fair amount to pay for Francesco and Simmons first Erlang book - would you pay equal amounts to Francesco and Simon. But wait a moment their editor who I think was Simon St Laurent will have made significant contributions to the book - should his contribution be unpaid and unrecognised? > On 8 Jul 2014, at 19:51, Francesco Cesarini > com> wrote: >> >> You do not write books because you believe you are going to make money >>> out of them. You write them because you are passionate about the subject. >>> If you calculate what Simon and I made from our first book, I am not sure >>> we've hit minimum wage yet. From my side, I want to document my approach to >>> teaching OTP, as I think it will work in writing as well as it does in the >>> classroom. From the feedback Steve and I have received so far, we are right >>> on track. Reiterating an email on this list from 2008, I want to see a >>> whole bookshelf of Erlang/OTP books out there. >>> >>> I found the experience of working with O'Reilly really positive the >>> first time around. From the editor, the production team (Graphics, copy >>> editor, proof readers, etc) as well as their marketing and conferences. >>> And this time around, it is just as good, if not better. As an author, I >>> could not recommend them more highly. I want to write books, I do not want >>> to do all of the other stuff associated with getting it out. It is just a >>> false economy. >>> >> I agree 100% - my experience with the prags was the same - publishers do not just screw the authors - they add value to the product and they take a financial risk. (I actually find it strange to be arguing this. It would be easy to say that all a publisher does is screw the authors but this is not true - they actually improve the product - my books have been immensely improved by my editors - Dave Thomas and Susannah Pfalzer) Books published by reputable publishers are an essentially part of becoming mainstream - the cost does not reflect the production cost but rather what the publisher decides what to price the book at - too little and they go bankrupt too much and they have no income. All decent technical books are available on file sharing networks for free anyway. > >>> /F >>> >>> PS. For those who can't afford 25$, try before you buy. It is called >>> BitTorrent. >>> >> > >>> On 08/07/2014 19:39, Raoul Duke wrote: >>> >>>> i haven't looked at the books here, but having been somebody who >>>> produced things and wondered how anybody could ever make a living at >>>> it, and knowing that writers throughout history rarely made much after >>>> the publisher etc. got a cut, i didn't have the same reaction to the >>>> $50 price tag. can't say i'd *spend* the $50 since i'm broke and >>>> there's a zillion other things to spend $50 on first. i'd try to get >>>> it via inter library loan or something :-). >>>> >>>> On Tue, Jul 8, 2014 at 11:36 AM, Francesco Cesarini >>>> wrote: >>>> >>>>> Alas, that is how O'Reilly price their books. Having a high price and >>>>> then >>>>> discounting is not the approach I would pick. That is why discount >>>>> codes are >>>>> being handed out on public mailing lists and social media. I recommend >>>>> you >>>>> use them (Read, no one pays full price for an O'Reilly book). As an >>>>> unedited >>>>> book, the cost is for the final book which we hope will complement >>>>> what is >>>>> already out there. It is a different approach to OTP in action. One >>>>> I've >>>>> been using for 15 years when teaching OTP. >>>>> >>>>> /F >>>>> >>>>> On 08/07/2014 19:14, Lee Sylvester wrote: >>>>> >>>>>> Wow, $50? Yeah, that is steep. If this were some black arts compiler >>>>>> book or video encoding bible, then that would be something else. But >>>>>> a book >>>>>> on Erlang/OTP just doesn?t fall into that bracket? >>>>>> >>>>>> I may wait til it falls in the bargain bucket ;-) >>>>>> >>>>>> Lee >>>>>> >>>>>> On 8 Jul 2014, at 19:08, Miles Fidelman < >>>>>> mfidelman@REDACTED> >>>>>> wrote: >>>>>> >>>>>> On 8 Jul 2014, at 16:05, Francesco Cesarini >>>>>>> wrote: >>>>>>> >>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> a shameless plug. Steve Vinoski and I are working on a book >>>>>>>>> focused on >>>>>>>>> distributed, scalable systems with OTP. It is available from >>>>>>>>> O'Reilly as an >>>>>>>>> Early Release from their website. The first eight chapters we >>>>>>>>> released cover >>>>>>>>> (in great detail) all other behaviours. We started writing, and >>>>>>>>> before we >>>>>>>>> knew it, we had several hundred pages on behaviours alone. Last >>>>>>>>> week, two >>>>>>>>> new chapters, including the Introduction & Special processes and >>>>>>>>> Implementing your own behaviours were released. We are now >>>>>>>>> focusing on >>>>>>>>> release handling (hgg), code upgrade and architectural patterns. >>>>>>>>> >>>>>>>>> What is available is an unedited draft, with new chapters and >>>>>>>>> improvements published as they become available. You can find more >>>>>>>>> info >>>>>>>>> here: >>>>>>>>> >>>>>>>>> >>>>>>>>> http://shop.oreilly.com/product/0636920024149.do? >>>>>>>>> intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>>>>>>>> >>>>>>>>> If you use discount code authd, you will get 50% off the Early >>>>>>>>> Release, >>>>>>>>> and 40% on pre-orders of the the printed copy. >>>>>>>>> >>>>>>>>> Looking forward to your feedback, >>>>>>>>> >>>>>>>>> Not for nothing, but $50 for the undedited work-in-progress seems >>>>>>> a bit >>>>>>> steep (and yes, I saw the discount code). I also note that "rough >>>>>>> cuts" >>>>>>> aren't included in my Safari subscription. >>>>>>> >>>>>>> Usual practice that I've seen is for works-in-progress to be free, >>>>>>> with >>>>>>> tools that support comments from early readers. >>>>>>> >>>>>>> Sounds interesting - but, from the TOC (all that's available without >>>>>>> paying), it sure looks like it covers the same ground as "Erlang and >>>>>>> OTP in >>>>>>> Action." >>>>>>> >>>>>>> Miles Fidelman >>>>>>> >>>>>>> -- >>>>>>> In theory, there is no difference between theory and practice. >>>>>>> In practice, there is. .... Yogi Berra >>>>>>> >>>>>>> _______________________________________________ >>>>>>> erlang-questions mailing list >>>>>>> erlang-questions@REDACTED >>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>> >>>>> -- >>>>> Erlang Solutions Ltd. >>>>> http://www.erlang-solutions.com >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> -- >>> Erlang Solutions Ltd. >>> http://www.erlang-solutions.com >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Tue Jul 8 21:55:18 2014 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 8 Jul 2014 21:55:18 +0200 Subject: [erlang-questions] on writing [was: ANN: Designing for Scalability with Erlang/OTP by O'Reilly] In-Reply-To: <53BC3E5C.8020207@meetinghouse.net> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3E5C.8020207@meetinghouse.net> Message-ID: On Tue, Jul 8, 2014 at 8:54 PM, Miles Fidelman wrote: > Yeah... agree on that. In a previous life, I wrote two books: > > First was on implementing Internet services in libraries (kind of dates > me, don't it :-). Started out as the final report on a funded project, > ended up as a book for the American Library Association Press. Phenomenal > experience - an advance, an incredible editor, and it ended up as a book > club selection (who knew - the ALA has a book club for their members - a > copy ended up in every mid- and large- library in the country). I ended up > making money on the book (not a lot, but worth the effort). > > Second was on municipal networks. Started out as a very high-priced > handbook, sold by Government Technology Press - something like the third > book they published. Intent was to co-market with their weekly trade rag, > conferences and seminars. Horrible experience: Horrible editor (kept > making changes that changed the meaning of things - had to fight to fix > things), then they decided they didn't want to publish books after all - > shut down the book division. Then again, I ended up with a few hundred > remainders, and camera-ready originals -- ended up selling a bunch at $125 > each (instead of the original $200+) through our web site, and bundling > them with a dozen seminars given over a year-long period. Made out very > well, particularly on the seminars and some consulting gigs that resulted > from the seminars. > > Next book will definitely be self-published - not sure if will be open > source, though. > So how will you find a good editor? Your book one had "a Phenomenal Editor" and book two a "Horrible editor" I guess I would self-publish if I could find a phenomenal editor, and a phenomenal proof reader and marketer but if I could do all these things I guess I'd just start a publishing company. Just "writing the damn text" is only a part of the story ... /Joe > > Definitely a "strange thing, books." > Very > > Cheers, > > Miles > > Lee Sylvester wrote: > >> :-) That?s why I wouldn?t write a book for O?Reilly. Wrox were bad >> enough. I vowed my next book would be open source material. No one writes >> a book for money, as anyone who?s written a book will know it?s not worth >> it from the second book onward and if you wrote a second book, you would be >> of a mind where the money wasn?t important. >> >> Strange thing, books. >> >> Lee >> >> >> On 8 Jul 2014, at 19:36, Francesco Cesarini > com> wrote: >> >> Alas, that is how O'Reilly price their books. Having a high price and >>> then discounting is not the approach I would pick. That is why discount >>> codes are being handed out on public mailing lists and social media. I >>> recommend you use them (Read, no one pays full price for an O'Reilly book). >>> As an unedited book, the cost is for the final book which we hope will >>> complement what is already out there. It is a different approach to OTP in >>> action. One I've been using for 15 years when teaching OTP. >>> >>> /F >>> >>> On 08/07/2014 19:14, Lee Sylvester wrote: >>> >>>> Wow, $50? Yeah, that is steep. If this were some black arts compiler >>>> book or video encoding bible, then that would be something else. But a >>>> book on Erlang/OTP just doesn?t fall into that bracket? >>>> >>>> I may wait til it falls in the bargain bucket ;-) >>>> >>>> Lee >>>> >>>> On 8 Jul 2014, at 19:08, Miles Fidelman >>>> wrote: >>>> >>>> On 8 Jul 2014, at 16:05, Francesco Cesarini < >>>>> francesco@REDACTED> wrote: >>>>> >>>>>> Hi all, >>>>>>> >>>>>>> a shameless plug. Steve Vinoski and I are working on a book focused >>>>>>> on distributed, scalable systems with OTP. It is available from O'Reilly as >>>>>>> an Early Release from their website. The first eight chapters we released >>>>>>> cover (in great detail) all other behaviours. We started writing, and >>>>>>> before we knew it, we had several hundred pages on behaviours alone. Last >>>>>>> week, two new chapters, including the Introduction & Special processes and >>>>>>> Implementing your own behaviours were released. We are now focusing on >>>>>>> release handling (hgg), code upgrade and architectural patterns. >>>>>>> >>>>>>> What is available is an unedited draft, with new chapters and >>>>>>> improvements published as they become available. You can find more info >>>>>>> here: >>>>>>> >>>>>>> http://shop.oreilly.com/product/0636920024149.do? >>>>>>> intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>>>>>> >>>>>>> If you use discount code authd, you will get 50% off the Early >>>>>>> Release, and 40% on pre-orders of the the printed copy. >>>>>>> >>>>>>> Looking forward to your feedback, >>>>>>> >>>>>>> Not for nothing, but $50 for the undedited work-in-progress seems a >>>>> bit steep (and yes, I saw the discount code). I also note that "rough >>>>> cuts" aren't included in my Safari subscription. >>>>> >>>>> Usual practice that I've seen is for works-in-progress to be free, >>>>> with tools that support comments from early readers. >>>>> >>>>> Sounds interesting - but, from the TOC (all that's available without >>>>> paying), it sure looks like it covers the same ground as "Erlang and OTP in >>>>> Action." >>>>> >>>>> Miles Fidelman >>>>> >>>>> -- >>>>> In theory, there is no difference between theory and practice. >>>>> In practice, there is. .... Yogi Berra >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> -- >>> Erlang Solutions Ltd. >>> http://www.erlang-solutions.com >>> >>> > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Tue Jul 8 21:55:34 2014 From: lloyd@REDACTED (lloyd@REDACTED) Date: Tue, 8 Jul 2014 15:55:34 -0400 (EDT) Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <53BC3DC4.7000600@erlang-solutions.com> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3DC4.7000600@erlang-solutions.com> Message-ID: <1404849334.63757001@apps.rackspace.com> Hello, My thoughts as an a novelist and army-of-one small press publisher: - Several sites discuss how to develop a proforma profit & loss state for a book project http://pimpmynovel.blogspot.com/2009/10/p-1-of-4-basics.html - Even in my bare-bones world where I often stringently discount my time, expenses add up fast. O'Reilly carries considerable overhead and brings many paid person hours to a book project. - Sad fact is, few books sell enough to cover the author's advance. - I too gasped when I saw the combined price of the physical and e-book editions of Cesarini/Vinoski. But that said, even at full price, I seriously doubt that anyone involved with Franceso and Steve's book project will make a killing. - Like others on this thread, I could not justify purchasing the book at this point without Franceso's discount code. Not that I consider the two editiorns overpriced for value delivered. But rather, I simply couldn't squeeze the purchase into my book-buying budget. But with the discount code, purchasing both was a no-brainer given the hard-won expertise that Francesco and Steve are sharing so generously through their book. Just imagine how fast the consulting fees would mount if we had to hire either to help with our software projects. - The wonderful thing about the web is that so much invaluable information is available for the price of a few mouse clicks. It's spoiled us into thinking that information is free. But generating worthwhile information entails considerable time and cost. I, for one, benefit considerably and feel boundless gratitude for the wonderful sharing economy that has emerged through the open-source movement. For me it's a privilege to pay back in any way I can. Buying a book that supports the titans of our field who've contributed so much is the least I can do. All the best, Lloyd -----Original Message----- From: "Francesco Cesarini" Sent: Tuesday, July 8, 2014 2:51pm To: erlang-questions@REDACTED Subject: Re: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly You do not write books because you believe you are going to make money out of them. You write them because you are passionate about the subject. If you calculate what Simon and I made from our first book, I am not sure we've hit minimum wage yet. From my side, I want to document my approach to teaching OTP, as I think it will work in writing as well as it does in the classroom. From the feedback Steve and I have received so far, we are right on track. Reiterating an email on this list from 2008, I want to see a whole bookshelf of Erlang/OTP books out there. I found the experience of working with O'Reilly really positive the first time around. From the editor, the production team (Graphics, copy editor, proof readers, etc) as well as their marketing and conferences. And this time around, it is just as good, if not better. As an author, I could not recommend them more highly. I want to write books, I do not want to do all of the other stuff associated with getting it out. It is just a false economy. /F PS. For those who can't afford 25$, try before you buy. It is called BitTorrent. On 08/07/2014 19:39, Raoul Duke wrote: > i haven't looked at the books here, but having been somebody who > produced things and wondered how anybody could ever make a living at > it, and knowing that writers throughout history rarely made much after > the publisher etc. got a cut, i didn't have the same reaction to the > $50 price tag. can't say i'd *spend* the $50 since i'm broke and > there's a zillion other things to spend $50 on first. i'd try to get > it via inter library loan or something :-). > > On Tue, Jul 8, 2014 at 11:36 AM, Francesco Cesarini > wrote: >> Alas, that is how O'Reilly price their books. Having a high price and then >> discounting is not the approach I would pick. That is why discount codes are >> being handed out on public mailing lists and social media. I recommend you >> use them (Read, no one pays full price for an O'Reilly book). As an unedited >> book, the cost is for the final book which we hope will complement what is >> already out there. It is a different approach to OTP in action. One I've >> been using for 15 years when teaching OTP. >> >> /F >> >> On 08/07/2014 19:14, Lee Sylvester wrote: >>> Wow, $50? Yeah, that is steep. If this were some black arts compiler >>> book or video encoding bible, then that would be something else. But a book >>> on Erlang/OTP just doesn?t fall into that bracket? >>> >>> I may wait til it falls in the bargain bucket ;-) >>> >>> Lee >>> >>> On 8 Jul 2014, at 19:08, Miles Fidelman >>> wrote: >>> >>>> On 8 Jul 2014, at 16:05, Francesco Cesarini >>>> wrote: >>>>>> Hi all, >>>>>> >>>>>> a shameless plug. Steve Vinoski and I are working on a book focused on >>>>>> distributed, scalable systems with OTP. It is available from O'Reilly as an >>>>>> Early Release from their website. The first eight chapters we released cover >>>>>> (in great detail) all other behaviours. We started writing, and before we >>>>>> knew it, we had several hundred pages on behaviours alone. Last week, two >>>>>> new chapters, including the Introduction & Special processes and >>>>>> Implementing your own behaviours were released. We are now focusing on >>>>>> release handling (hgg), code upgrade and architectural patterns. >>>>>> >>>>>> What is available is an unedited draft, with new chapters and >>>>>> improvements published as they become available. You can find more info >>>>>> here: >>>>>> >>>>>> >>>>>> http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>>>>> >>>>>> If you use discount code authd, you will get 50% off the Early Release, >>>>>> and 40% on pre-orders of the the printed copy. >>>>>> >>>>>> Looking forward to your feedback, >>>>>> >>>> Not for nothing, but $50 for the undedited work-in-progress seems a bit >>>> steep (and yes, I saw the discount code). I also note that "rough cuts" >>>> aren't included in my Safari subscription. >>>> >>>> Usual practice that I've seen is for works-in-progress to be free, with >>>> tools that support comments from early readers. >>>> >>>> Sounds interesting - but, from the TOC (all that's available without >>>> paying), it sure looks like it covers the same ground as "Erlang and OTP in >>>> Action." >>>> >>>> Miles Fidelman >>>> >>>> -- >>>> In theory, there is no difference between theory and practice. >>>> In practice, there is. .... Yogi Berra >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> -- >> Erlang Solutions Ltd. >> http://www.erlang-solutions.com >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Erlang Solutions Ltd. http://www.erlang-solutions.com _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From mark.nijhof@REDACTED Tue Jul 8 21:56:01 2014 From: mark.nijhof@REDACTED (Mark Nijhof) Date: Tue, 8 Jul 2014 21:56:01 +0200 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <53BC41E7.8070407@erlang-solutions.com> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3DC4.7000600@erlang-solutions.com> <01D5657C-8AEE-47AD-8C76-2BDD1F2A8C35@gmail.com> <53BC41E7.8070407@erlang-solutions.com> Message-ID: It is not funny anymore how many people think that someone writes a book for money. Even when self publishing there is not a lot of money to be made, I use LeanPub and am very happy with the platform/service they offer. And the royalties are good as well, but I need to pay for the cover design, the website (if you want something else then the default) and for editors to help me improve my content. So far it has cost me money. Personally I write because by writing about and explaining Erlang my own understanding grows so much more then by just using. And eventually I hope others find my writing useful as well. The early access mode is very useful for getting early feedback, it is much easier to change content while you are still writing, we all know the feedback loop in coding, while writing there is no difference. Purely looking at price, if by reading this book you save half a day of work you are already in the black (for some we are only talking about an hour of work). Writing is a huge amount of work, and it is a shame writers don't see more of the results. -Mark On Jul 8, 2014 9:09 PM, "Francesco Cesarini" < francesco@REDACTED> wrote: > What I am being paid does not bother me, else I would not be putting every > spare hour of the day into this project. I have seen the advantages of > working with O'Reilly vs other publishing houses. They deserve their cut, > else I would not have gone back. I were to self publish, the project would > never get finished... And I would be forced to deal with things I do not > like doing. Not worth it for me. I would rather have the backing and > support O'Reilly are good at. > > In regards to how books are released, everyone has their choice. For the > CouchDB book, it was already being written before O'Reilly came on board. > The first book, Simon did not want an early release. In this one, Steve did > not mind. There is no one size fits all. > > Yup, strange things books. I think my next one will be a guide book. > > /F > > On 08/07/2014 19:55, Lee Sylvester wrote: > >> It?s not about the $25. With the effort you would have put into this, >> it?s certainly worth it, I?m sure. This is merely a debate of the business >> of publishing and authoring. In fact, I would feel better giving you $25 >> knowing you received it all than pay $25 to O?Reilly knowing you will see >> about $0.25 of it :-) >> >> >> On 8 Jul 2014, at 19:51, Francesco Cesarini > com> wrote: >> >> You do not write books because you believe you are going to make money >>> out of them. You write them because you are passionate about the subject. >>> If you calculate what Simon and I made from our first book, I am not sure >>> we've hit minimum wage yet. From my side, I want to document my approach to >>> teaching OTP, as I think it will work in writing as well as it does in the >>> classroom. From the feedback Steve and I have received so far, we are right >>> on track. Reiterating an email on this list from 2008, I want to see a >>> whole bookshelf of Erlang/OTP books out there. >>> >>> I found the experience of working with O'Reilly really positive the >>> first time around. From the editor, the production team (Graphics, copy >>> editor, proof readers, etc) as well as their marketing and conferences. >>> And this time around, it is just as good, if not better. As an author, I >>> could not recommend them more highly. I want to write books, I do not want >>> to do all of the other stuff associated with getting it out. It is just a >>> false economy. >>> >>> /F >>> >>> PS. For those who can't afford 25$, try before you buy. It is called >>> BitTorrent. >>> >>> On 08/07/2014 19:39, Raoul Duke wrote: >>> >>>> i haven't looked at the books here, but having been somebody who >>>> produced things and wondered how anybody could ever make a living at >>>> it, and knowing that writers throughout history rarely made much after >>>> the publisher etc. got a cut, i didn't have the same reaction to the >>>> $50 price tag. can't say i'd *spend* the $50 since i'm broke and >>>> there's a zillion other things to spend $50 on first. i'd try to get >>>> it via inter library loan or something :-). >>>> >>>> On Tue, Jul 8, 2014 at 11:36 AM, Francesco Cesarini >>>> wrote: >>>> >>>>> Alas, that is how O'Reilly price their books. Having a high price and >>>>> then >>>>> discounting is not the approach I would pick. That is why discount >>>>> codes are >>>>> being handed out on public mailing lists and social media. I recommend >>>>> you >>>>> use them (Read, no one pays full price for an O'Reilly book). As an >>>>> unedited >>>>> book, the cost is for the final book which we hope will complement >>>>> what is >>>>> already out there. It is a different approach to OTP in action. One >>>>> I've >>>>> been using for 15 years when teaching OTP. >>>>> >>>>> /F >>>>> >>>>> On 08/07/2014 19:14, Lee Sylvester wrote: >>>>> >>>>>> Wow, $50? Yeah, that is steep. If this were some black arts compiler >>>>>> book or video encoding bible, then that would be something else. But >>>>>> a book >>>>>> on Erlang/OTP just doesn?t fall into that bracket? >>>>>> >>>>>> I may wait til it falls in the bargain bucket ;-) >>>>>> >>>>>> Lee >>>>>> >>>>>> On 8 Jul 2014, at 19:08, Miles Fidelman < >>>>>> mfidelman@REDACTED> >>>>>> wrote: >>>>>> >>>>>> On 8 Jul 2014, at 16:05, Francesco Cesarini >>>>>>> wrote: >>>>>>> >>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> a shameless plug. Steve Vinoski and I are working on a book >>>>>>>>> focused on >>>>>>>>> distributed, scalable systems with OTP. It is available from >>>>>>>>> O'Reilly as an >>>>>>>>> Early Release from their website. The first eight chapters we >>>>>>>>> released cover >>>>>>>>> (in great detail) all other behaviours. We started writing, and >>>>>>>>> before we >>>>>>>>> knew it, we had several hundred pages on behaviours alone. Last >>>>>>>>> week, two >>>>>>>>> new chapters, including the Introduction & Special processes and >>>>>>>>> Implementing your own behaviours were released. We are now >>>>>>>>> focusing on >>>>>>>>> release handling (hgg), code upgrade and architectural patterns. >>>>>>>>> >>>>>>>>> What is available is an unedited draft, with new chapters and >>>>>>>>> improvements published as they become available. You can find more >>>>>>>>> info >>>>>>>>> here: >>>>>>>>> >>>>>>>>> >>>>>>>>> http://shop.oreilly.com/product/0636920024149.do? >>>>>>>>> intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>>>>>>>> >>>>>>>>> If you use discount code authd, you will get 50% off the Early >>>>>>>>> Release, >>>>>>>>> and 40% on pre-orders of the the printed copy. >>>>>>>>> >>>>>>>>> Looking forward to your feedback, >>>>>>>>> >>>>>>>>> Not for nothing, but $50 for the undedited work-in-progress seems >>>>>>> a bit >>>>>>> steep (and yes, I saw the discount code). I also note that "rough >>>>>>> cuts" >>>>>>> aren't included in my Safari subscription. >>>>>>> >>>>>>> Usual practice that I've seen is for works-in-progress to be free, >>>>>>> with >>>>>>> tools that support comments from early readers. >>>>>>> >>>>>>> Sounds interesting - but, from the TOC (all that's available without >>>>>>> paying), it sure looks like it covers the same ground as "Erlang and >>>>>>> OTP in >>>>>>> Action." >>>>>>> >>>>>>> Miles Fidelman >>>>>>> >>>>>>> -- >>>>>>> In theory, there is no difference between theory and practice. >>>>>>> In practice, there is. .... Yogi Berra >>>>>>> >>>>>>> _______________________________________________ >>>>>>> erlang-questions mailing list >>>>>>> erlang-questions@REDACTED >>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>> >>>>> -- >>>>> Erlang Solutions Ltd. >>>>> http://www.erlang-solutions.com >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> -- >>> Erlang Solutions Ltd. >>> http://www.erlang-solutions.com >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> > -- > Erlang Solutions Ltd. > http://www.erlang-solutions.com > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raould@REDACTED Tue Jul 8 21:59:27 2014 From: raould@REDACTED (Raoul Duke) Date: Tue, 8 Jul 2014 12:59:27 -0700 Subject: [erlang-questions] on writing [was: ANN: Designing for Scalability with Erlang/OTP by O'Reilly] In-Reply-To: References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3E5C.8020207@meetinghouse.net> Message-ID: i'm actually conceited/self-aware enough to claim that i'm a quite good editor (as is my wife, she's probably even a bit better). i've done it for free in the past since i got to read early copies of things for free and educate myself thusly. :-) i should go sign myself up for a service that farms chapters of good technical stuff for me to voluntarily mark up. :-) it is just hard to make a commitment to people with some kind of turn-around time since it isn't my day job. From lee.sylvester@REDACTED Tue Jul 8 22:22:38 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 8 Jul 2014 21:22:38 +0100 Subject: [erlang-questions] on writing [was: ANN: Designing for Scalability with Erlang/OTP by O'Reilly] In-Reply-To: References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3E5C.8020207@meetinghouse.net> Message-ID: <46031E70-D1CC-4329-8524-D0C20768CAC7@gmail.com> I?ve edited a book or two before (as well as tens if not hundreds of magazine articles). I?d gladly edit someones book on this list if they would do the same for me :-) When I wrote my last book, I did have a great editor, but two of my technical editors (free contributors) were by far my greatest aids. Often, the people you want are those that have been there and done that. Also, just to put the record straight, my point was never that I didn?t agree that publishing companies weren?t ?worth it?, but that I feel authors could do with a fairer share of the proceeds. Lee On 8 Jul 2014, at 20:55, Joe Armstrong wrote: > > > > On Tue, Jul 8, 2014 at 8:54 PM, Miles Fidelman wrote: > Yeah... agree on that. In a previous life, I wrote two books: > > First was on implementing Internet services in libraries (kind of dates me, don't it :-). Started out as the final report on a funded project, ended up as a book for the American Library Association Press. Phenomenal experience - an advance, an incredible editor, and it ended up as a book club selection (who knew - the ALA has a book club for their members - a copy ended up in every mid- and large- library in the country). I ended up making money on the book (not a lot, but worth the effort). > > Second was on municipal networks. Started out as a very high-priced handbook, sold by Government Technology Press - something like the third book they published. Intent was to co-market with their weekly trade rag, conferences and seminars. Horrible experience: Horrible editor (kept making changes that changed the meaning of things - had to fight to fix things), then they decided they didn't want to publish books after all - shut down the book division. Then again, I ended up with a few hundred remainders, and camera-ready originals -- ended up selling a bunch at $125 each (instead of the original $200+) through our web site, and bundling them with a dozen seminars given over a year-long period. Made out very well, particularly on the seminars and some consulting gigs that resulted from the seminars. > > Next book will definitely be self-published - not sure if will be open source, though. > > So how will you find a good editor? > > Your book one had "a Phenomenal Editor" and book two a "Horrible editor" > > I guess I would self-publish if I could find a phenomenal editor, and a > phenomenal proof reader and marketer but if I could do all these > things I guess I'd just start a publishing company. > > Just "writing the damn text" is only a part of the story ... > > /Joe > > > Definitely a "strange thing, books." > > Very > > > Cheers, > > Miles > > Lee Sylvester wrote: > :-) That?s why I wouldn?t write a book for O?Reilly. Wrox were bad enough. I vowed my next book would be open source material. No one writes a book for money, as anyone who?s written a book will know it?s not worth it from the second book onward and if you wrote a second book, you would be of a mind where the money wasn?t important. > > Strange thing, books. > > Lee > > > On 8 Jul 2014, at 19:36, Francesco Cesarini wrote: > > Alas, that is how O'Reilly price their books. Having a high price and then discounting is not the approach I would pick. That is why discount codes are being handed out on public mailing lists and social media. I recommend you use them (Read, no one pays full price for an O'Reilly book). As an unedited book, the cost is for the final book which we hope will complement what is already out there. It is a different approach to OTP in action. One I've been using for 15 years when teaching OTP. > > /F > > On 08/07/2014 19:14, Lee Sylvester wrote: > Wow, $50? Yeah, that is steep. If this were some black arts compiler book or video encoding bible, then that would be something else. But a book on Erlang/OTP just doesn?t fall into that bracket? > > I may wait til it falls in the bargain bucket ;-) > > Lee > > On 8 Jul 2014, at 19:08, Miles Fidelman wrote: > > On 8 Jul 2014, at 16:05, Francesco Cesarini wrote: > Hi all, > > a shameless plug. Steve Vinoski and I are working on a book focused on distributed, scalable systems with OTP. It is available from O'Reilly as an Early Release from their website. The first eight chapters we released cover (in great detail) all other behaviours. We started writing, and before we knew it, we had several hundred pages on behaviours alone. Last week, two new chapters, including the Introduction & Special processes and Implementing your own behaviours were released. We are now focusing on release handling (hgg), code upgrade and architectural patterns. > > What is available is an unedited draft, with new chapters and improvements published as they become available. You can find more info here: > > http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct > > If you use discount code authd, you will get 50% off the Early Release, and 40% on pre-orders of the the printed copy. > > Looking forward to your feedback, > > Not for nothing, but $50 for the undedited work-in-progress seems a bit steep (and yes, I saw the discount code). I also note that "rough cuts" aren't included in my Safari subscription. > > Usual practice that I've seen is for works-in-progress to be free, with tools that support comments from early readers. > > Sounds interesting - but, from the TOC (all that's available without paying), it sure looks like it covers the same ground as "Erlang and OTP in Action." > > Miles Fidelman > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- > Erlang Solutions Ltd. > http://www.erlang-solutions.com > > > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Tue Jul 8 22:25:02 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 8 Jul 2014 21:25:02 +0100 Subject: [erlang-questions] on writing [was: ANN: Designing for Scalability with Erlang/OTP by O'Reilly] In-Reply-To: References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3E5C.8020207@meetinghouse.net> Message-ID: <1FA4CACE-A06B-428F-A46F-F424FB0D15DC@gmail.com> The problem is, different works require different amounts of contribution. I?ve edited works from good writers who?s first language is English and had an easy time, because most of the work was technical in nature. I?ve also had work where the literature was from a non-English speaker where most of the effort involved was simply trying to make sense of what the author was trying to say, let alone merit its correctness. Lee On 8 Jul 2014, at 20:59, Raoul Duke wrote: > i'm actually conceited/self-aware enough to claim that i'm a quite > good editor (as is my wife, she's probably even a bit better). i've > done it for free in the past since i got to read early copies of > things for free and educate myself thusly. :-) i should go sign myself > up for a service that farms chapters of good technical stuff for me to > voluntarily mark up. :-) it is just hard to make a commitment to > people with some kind of turn-around time since it isn't my day job. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From z@REDACTED Tue Jul 8 22:48:18 2014 From: z@REDACTED (Danil Zagoskin) Date: Wed, 9 Jul 2014 00:48:18 +0400 Subject: [erlang-questions] Distribution: small packets In-Reply-To: References: Message-ID: Maybe this problem is related to how the application works. Each node may host a shard of service. Application contains a process (shard manager) which registers in global registry with a key like {shard, 2}. Each shard contains thousands (millions planned) of processes, registered locally in gproc as {n, l, {worker, <<"worker_name">>}} Access to the local worker (say, worker:access/2) is pair of gproc:lookup_pid and gen_server:call Access to any worker is determining shard number (crc32(Name) rem Count), N = node(global:whereis_name({shard, 2})), rpc:call(N, worker, access, [...]) So there are two operations related to distribution ? global:whereis_name and rpc:call. Does any of them force sending of tcp-push? If yes, how can I change this behavior by cost of couple of millisecods latency? On Tue, Jul 8, 2014 at 11:20 PM, Danil Zagoskin wrote: > Hi! > > There is enormous packet-per-second in my distributed erlang setup. > > Under heavy load in single distribution socket and in one direction > tcpdump shows rates like 40..100 packets with tcp-push flag set per > millisecond. > Size of majority of packets is 47..112 bytes while MTU on network > interface is 8950 (jumbo-frames). > > If distribution driver aggregated messages to fit MTU it would be 100 > times less packets in network. > Given cluster of 3 nodes (very small) and both directions we get about > extra 300K PPS which causes packet drops and tcp retransmits (thus > increasing latency a lot). > > Is it possible to make erlang distribution push packets less often (one > millisecond would be enough for me)? > > I'm sure this is not of net ticks because tracing on dist_util:con_loop > process shows quite low {_, tick} message rate. > > Erlang/OTP version used is 17.0. > > -- > Danil Zagoskin | z@REDACTED > -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From josh.rubyist@REDACTED Tue Jul 8 23:07:28 2014 From: josh.rubyist@REDACTED (Josh Adams) Date: Tue, 8 Jul 2014 16:07:28 -0500 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <53BC08AB.1030605@erlang-solutions.com> References: <53BC08AB.1030605@erlang-solutions.com> Message-ID: > > If you use discount code authd, you will get 50% off the Early Release, > and 40% on pre-orders of the the printed copy. > Bought it, looking forward to its completion and print delivery :) Looking forward to your feedback, > Looking forward to learning how to build scalable systems :) -- Josh Adams -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfidelman@REDACTED Tue Jul 8 23:10:54 2014 From: mfidelman@REDACTED (Miles Fidelman) Date: Tue, 08 Jul 2014 17:10:54 -0400 Subject: [erlang-questions] on writing [was: ANN: Designing for Scalability with Erlang/OTP by O'Reilly] In-Reply-To: References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3E5C.8020207@meetinghouse.net> Message-ID: <53BC5E5E.6000707@meetinghouse.net> Joe Armstrong wrote: > > So how will you find a good editor? > > Your book one had "a Phenomenal Editor" and book two a "Horrible editor" > > I guess I would self-publish if I could find a phenomenal editor, and a > phenomenal proof reader and marketer but if I could do all these > things I guess I'd just start a publishing company. Well... I'm in a funny position, in that I write pretty good prose, and get called on to edit a lot of stuff (I spend about half my work life writing proposals for a living). But... having said that - my observation, from my limited sample of two books, and two publishers, is that if one signs with a publisher, you're stuck with whomever they assign - so it's a pretty good idea to interview and sign off on an editor before signing the book deal. Not sure if that's practical or not unless you're a name author. So... if O'Reilly routinely assigns good editors, that's a pretty good reason to sign with them. Beyond that, an awful lot of people I know rely on peers to review and edit their writings - quid pro quo and all that. Putting drafts online, and soliciting review and comment, seems like another model that has worked for some. The other choice is to hire an editor - but that starts to get expensive. As to marketing - my observation is that most publishers look to their authors to promote their works - maybe they'll arrange a book tour for you, and maybe they'll pick up some expenses, but probably not for small market items. The value add of traditional publishers seems to go down day-by-day. > > Just "writing the damn text" is only a part of the story ... > Well yeah, but ain't that always the case. :-) Miles -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From mfidelman@REDACTED Tue Jul 8 23:35:50 2014 From: mfidelman@REDACTED (Miles Fidelman) Date: Tue, 08 Jul 2014 17:35:50 -0400 Subject: [erlang-questions] on writing [was: ANN: Designing for Scalability with Erlang/OTP by O'Reilly] In-Reply-To: <53BC5E5E.6000707@meetinghouse.net> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3E5C.8020207@meetinghouse.net> <53BC5E5E.6000707@meetinghouse.net> Message-ID: <53BC6436.5000201@meetinghouse.net> Should have added.... finding layout and graphics support is what I find hardest. Miles Fidelman wrote: > Joe Armstrong wrote: >> >> So how will you find a good editor? >> >> Your book one had "a Phenomenal Editor" and book two a "Horrible editor" >> >> I guess I would self-publish if I could find a phenomenal editor, and a >> phenomenal proof reader and marketer but if I could do all these >> things I guess I'd just start a publishing company. > > Well... I'm in a funny position, in that I write pretty good prose, > and get called on to edit a lot of stuff (I spend about half my work > life writing proposals for a living). > > But... having said that - my observation, from my limited sample of > two books, and two publishers, is that if one signs with a publisher, > you're stuck with whomever they assign - so it's a pretty good idea to > interview and sign off on an editor before signing the book deal. Not > sure if that's practical or not unless you're a name author. So... if > O'Reilly routinely assigns good editors, that's a pretty good reason > to sign with them. > > Beyond that, an awful lot of people I know rely on peers to review and > edit their writings - quid pro quo and all that. Putting drafts > online, and soliciting review and comment, seems like another model > that has worked for some. > > The other choice is to hire an editor - but that starts to get expensive. > > As to marketing - my observation is that most publishers look to their > authors to promote their works - maybe they'll arrange a book tour for > you, and maybe they'll pick up some expenses, but probably not for > small market items. The value add of traditional publishers seems to > go down day-by-day. >> >> Just "writing the damn text" is only a part of the story ... >> > Well yeah, but ain't that always the case. :-) > > Miles > -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From mfidelman@REDACTED Tue Jul 8 23:38:31 2014 From: mfidelman@REDACTED (Miles Fidelman) Date: Tue, 08 Jul 2014 17:38:31 -0400 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <1404849334.63757001@apps.rackspace.com> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3DC4.7000600@erlang-solutions.com> <1404849334.63757001@apps.rackspace.com> Message-ID: <53BC64D7.2020802@meetinghouse.net> lloyd@REDACTED wrote: > I, for one, benefit considerably and feel boundless gratitude for the wonderful sharing economy that has emerged through the open-source movement. For me it's a privilege to pay back in any way I can. Buying a book that supports the titans of our field who've contributed so much is the least I can do. > > By the way - nobody should take anything I say as a statement against the book or Francesco, et. al. Lloyd is spot on here. I just made my way over to O'Reilly and bought a copy (yes, with the discount code) - looking forward to reading it. Miles -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From lee.sylvester@REDACTED Tue Jul 8 23:40:11 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 8 Jul 2014 22:40:11 +0100 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <53BC64D7.2020802@meetinghouse.net> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3DC4.7000600@erlang-solutions.com> <1404849334.63757001@apps.rackspace.com> <53BC64D7.2020802@meetinghouse.net> Message-ID: <59C6871A-F68F-499A-A6F3-29B3C143E1C1@gmail.com> I?ll buy it when I get paid this month ;-) Lee On 8 Jul 2014, at 22:38, Miles Fidelman wrote: > lloyd@REDACTED wrote: >> I, for one, benefit considerably and feel boundless gratitude for the wonderful sharing economy that has emerged through the open-source movement. For me it's a privilege to pay back in any way I can. Buying a book that supports the titans of our field who've contributed so much is the least I can do. >> >> > By the way - nobody should take anything I say as a statement against the book or Francesco, et. al. Lloyd is spot on here. I just made my way over to O'Reilly and bought a copy (yes, with the discount code) - looking forward to reading it. > > Miles > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mfidelman@REDACTED Tue Jul 8 23:46:35 2014 From: mfidelman@REDACTED (Miles Fidelman) Date: Tue, 08 Jul 2014 17:46:35 -0400 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3DC4.7000600@erlang-solutions.com> <01D5657C-8AEE-47AD-8C76-2BDD1F2A8C35@gmail.com> <53BC41E7.8070407@erlang-solutions.com> Message-ID: <53BC66BB.4030301@meetinghouse.net> Mark Nijhof wrote: > > It is not funny anymore how many people think that someone writes a > book for money. Even when self publishing there is not a lot of money > to be made, I use LeanPub and am very happy with the platform/service > they offer. And the royalties are good as well, but I need to pay for > the cover design, the website (if you want something else then the > default) and for editors to help me improve my content. So far it has > cost me money. > That does require a comment. A lot of people DO write for money, though not necessarily looking for all that money to come from book sales. Academics "publish or perish." Consultants as a marketing vehicle. And yes, there are people who actually make a living from writing books. But yes, there are also non-monetary reasons for writing a book - consolidating learning, as an excuse to do some personal research, putting experience or thoughts on paper, promoting a position, and so forth. Even then, few have the luxury to do that without some means of support - we all have to eat. If one is retired, has a job that includes writing as a part of it (can you say academia?), an understanding employer (or an employer who benefits from your writing), that's great. Otherwise, a grant, a fellowship, or some form of income is required. > Writing is a huge amount of work, and it is a shame writers don't see > more of the results. Absolutely. Writing is real work. One of the reasons my next book will be self-published. Cheers, Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From michael.santos@REDACTED Tue Jul 8 23:47:10 2014 From: michael.santos@REDACTED (Michael Santos) Date: Tue, 8 Jul 2014 17:47:10 -0400 Subject: [erlang-questions] UDP Headers In-Reply-To: References: Message-ID: <20140708214710.GB3111@ioctl> On Tue, Jul 08, 2014 at 07:04:12PM +0100, Lee Sylvester wrote: > Hey guys, > > So, I?m using gen_udp for a server I?m building, but I really need to be able to update the headers of incoming packets. Does anyone know how I can do this with gen_udp without having to resort to using raw sockets? It depends which header needs to be changed: * for UDP (source/destination port, length, checksum), the simplest method is to receive and resend the packet * for IPv4/IPv6, some of the headers may be able to be influenced by setting socket options. For example, by using inet:setopts/2 or by using setsockopt(2) from an NIF. * otherwise, there's the raw socket interface. For IPv4, the socket can be opened in raw mode and passed back into gen_udp. For IPv6 headers, if the headers can be changed, you generally have to use sendmsg(2)/recvmsg(2) which means using a port or an NIF. * for ethernet, it will depend on the OS: PF_PACKET for linux, BPF for the BSDs * another option is routing the packets through a tun/tap device and having some erlang code re-write the headers From lee.sylvester@REDACTED Tue Jul 8 23:53:18 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 8 Jul 2014 22:53:18 +0100 Subject: [erlang-questions] UDP Headers In-Reply-To: <20140708214710.GB3111@ioctl> References: <20140708214710.GB3111@ioctl> Message-ID: <663C513B-E105-4536-A3DA-A33560B869F8@gmail.com> Hi Michael, Thanks for responding. I was actually just looking through your repositories to see if you had written something to work with this. What I?d like to do is to be able to set the DF or TTL flags. What would you suggest for such things? Thanks, Lee On 8 Jul 2014, at 22:47, Michael Santos wrote: > On Tue, Jul 08, 2014 at 07:04:12PM +0100, Lee Sylvester wrote: >> Hey guys, >> >> So, I?m using gen_udp for a server I?m building, but I really need to be able to update the headers of incoming packets. Does anyone know how I can do this with gen_udp without having to resort to using raw sockets? > > It depends which header needs to be changed: > > * for UDP (source/destination port, length, checksum), the simplest > method is to receive and resend the packet > > * for IPv4/IPv6, some of the headers may be able to be influenced by setting > socket options. For example, by using inet:setopts/2 or by using > setsockopt(2) from an NIF. > > * otherwise, there's the raw socket interface. > > For IPv4, the socket can be opened in raw mode and passed back into > gen_udp. > > For IPv6 headers, if the headers can be changed, you generally have to > use sendmsg(2)/recvmsg(2) which means using a port or an NIF. > > * for ethernet, it will depend on the OS: PF_PACKET for linux, BPF for > the BSDs > > * another option is routing the packets through a tun/tap device and > having some erlang code re-write the headers > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From michael.santos@REDACTED Wed Jul 9 00:20:00 2014 From: michael.santos@REDACTED (Michael Santos) Date: Tue, 8 Jul 2014 18:20:00 -0400 Subject: [erlang-questions] UDP Headers In-Reply-To: <663C513B-E105-4536-A3DA-A33560B869F8@gmail.com> References: <20140708214710.GB3111@ioctl> <663C513B-E105-4536-A3DA-A33560B869F8@gmail.com> Message-ID: <20140708221959.GC3111@ioctl> On Tue, Jul 08, 2014 at 10:53:18PM +0100, Lee Sylvester wrote: > Hi Michael, > > Thanks for responding. I was actually just looking through your repositories to see if you had written something to work with this. > > What I?d like to do is to be able to set the DF or TTL flags. What would you suggest for such things? Raw sockets would be overkill for this. Have a look at the raw option to inet:setopts/2. There's an example in the docs for inet of setting TCP_LINGER2 on a TCP socket: inet:setopts(Sock,[{raw,6,8,<<30:32/native>>}]) The values will depend on the OS. So for setting the TTL, on linux/x86_64, maybe something like: setsockopt(send_socket, IPPROTO_IP, IP_TTL, &ttl, sizeof(ttl)); % IPPROTO_IP = 0 % IP_TTL = 2 % TTL = 64 inet:setopts(Sock,[{raw,0,2,<<64:32/native>>}]) DF is interesting. There's a good discussion about the DF bit here: http://stackoverflow.com/questions/973439/how-to-set-the-dont-fragment-df-flag-on-a-socket So on a linux system, maybe this would work: int val = IP_PMTUDISC_DO; setsockopt(sd, IPPROTO_IP, IP_MTU_DISCOVER, &val, sizeof(val)); % IPPROTO = 0 % IP_MTU_DISCOVER = 10 % IP_PMTUDISC_DO = 2 inet:setopts(Sock,[{raw,0,10,<<2:32/native>>}]) I haven't tested if these work so let us know if you run into problems and I'll try to come up with some working code! > > On Tue, Jul 08, 2014 at 07:04:12PM +0100, Lee Sylvester wrote: > >> Hey guys, > >> > >> So, I?m using gen_udp for a server I?m building, but I really need to be able to update the headers of incoming packets. Does anyone know how I can do this with gen_udp without having to resort to using raw sockets? > > > > It depends which header needs to be changed: > > > > * for UDP (source/destination port, length, checksum), the simplest > > method is to receive and resend the packet > > > > * for IPv4/IPv6, some of the headers may be able to be influenced by setting > > socket options. For example, by using inet:setopts/2 or by using > > setsockopt(2) from an NIF. > > > > * otherwise, there's the raw socket interface. > > > > For IPv4, the socket can be opened in raw mode and passed back into > > gen_udp. > > > > For IPv6 headers, if the headers can be changed, you generally have to > > use sendmsg(2)/recvmsg(2) which means using a port or an NIF. > > > > * for ethernet, it will depend on the OS: PF_PACKET for linux, BPF for > > the BSDs > > > > * another option is routing the packets through a tun/tap device and > > having some erlang code re-write the headers > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > From lee.sylvester@REDACTED Wed Jul 9 00:25:49 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 8 Jul 2014 23:25:49 +0100 Subject: [erlang-questions] UDP Headers In-Reply-To: <20140708221959.GC3111@ioctl> References: <20140708214710.GB3111@ioctl> <663C513B-E105-4536-A3DA-A33560B869F8@gmail.com> <20140708221959.GC3111@ioctl> Message-ID: <1AE1FD04-4539-467E-B6FF-23256E5DE6AC@gmail.com> Thank you for your help, Michael. I?ll give these a test in the morning and will let you know. Regards, Lee On 8 Jul 2014, at 23:20, Michael Santos wrote: > On Tue, Jul 08, 2014 at 10:53:18PM +0100, Lee Sylvester wrote: >> Hi Michael, >> >> Thanks for responding. I was actually just looking through your repositories to see if you had written something to work with this. >> >> What I?d like to do is to be able to set the DF or TTL flags. What would you suggest for such things? > > Raw sockets would be overkill for this. Have a look at the raw option > to inet:setopts/2. There's an example in the docs for inet of setting > TCP_LINGER2 on a TCP socket: > > inet:setopts(Sock,[{raw,6,8,<<30:32/native>>}]) > > The values will depend on the OS. So for setting the TTL, on linux/x86_64, > maybe something like: > > setsockopt(send_socket, IPPROTO_IP, IP_TTL, &ttl, sizeof(ttl)); > > % IPPROTO_IP = 0 > % IP_TTL = 2 > % TTL = 64 > inet:setopts(Sock,[{raw,0,2,<<64:32/native>>}]) > > DF is interesting. There's a good discussion about the DF bit here: > > http://stackoverflow.com/questions/973439/how-to-set-the-dont-fragment-df-flag-on-a-socket > > So on a linux system, maybe this would work: > > int val = IP_PMTUDISC_DO; > setsockopt(sd, IPPROTO_IP, IP_MTU_DISCOVER, &val, sizeof(val)); > > % IPPROTO = 0 > % IP_MTU_DISCOVER = 10 > % IP_PMTUDISC_DO = 2 > inet:setopts(Sock,[{raw,0,10,<<2:32/native>>}]) > > I haven't tested if these work so let us know if you run into problems > and I'll try to come up with some working code! > >>> On Tue, Jul 08, 2014 at 07:04:12PM +0100, Lee Sylvester wrote: >>>> Hey guys, >>>> >>>> So, I?m using gen_udp for a server I?m building, but I really need to be able to update the headers of incoming packets. Does anyone know how I can do this with gen_udp without having to resort to using raw sockets? >>> >>> It depends which header needs to be changed: >>> >>> * for UDP (source/destination port, length, checksum), the simplest >>> method is to receive and resend the packet >>> >>> * for IPv4/IPv6, some of the headers may be able to be influenced by setting >>> socket options. For example, by using inet:setopts/2 or by using >>> setsockopt(2) from an NIF. >>> >>> * otherwise, there's the raw socket interface. >>> >>> For IPv4, the socket can be opened in raw mode and passed back into >>> gen_udp. >>> >>> For IPv6 headers, if the headers can be changed, you generally have to >>> use sendmsg(2)/recvmsg(2) which means using a port or an NIF. >>> >>> * for ethernet, it will depend on the OS: PF_PACKET for linux, BPF for >>> the BSDs >>> >>> * another option is routing the packets through a tun/tap device and >>> having some erlang code re-write the headers >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.santos@REDACTED Wed Jul 9 00:37:56 2014 From: michael.santos@REDACTED (Michael Santos) Date: Tue, 8 Jul 2014 18:37:56 -0400 Subject: [erlang-questions] UDP Headers In-Reply-To: <1AE1FD04-4539-467E-B6FF-23256E5DE6AC@gmail.com> References: <20140708214710.GB3111@ioctl> <663C513B-E105-4536-A3DA-A33560B869F8@gmail.com> <20140708221959.GC3111@ioctl> <1AE1FD04-4539-467E-B6FF-23256E5DE6AC@gmail.com> Message-ID: <20140708223756.GD3111@ioctl> On Tue, Jul 08, 2014 at 11:25:49PM +0100, Lee Sylvester wrote: > Thank you for your help, Michael. I?ll give these a test in the morning and will let you know. FYI, just confirmed it works: 1> {ok,S} = gen_udp:open(0). {ok,#Port<0.800>} 2> gen_udp:send(S, {8,8,8,8}, 7777, <<"hello">>). tcpdump shows a default TTL of 64 and the DF bit is set: 18:33:28.006857 IP (tos 0x0, ttl 64, id 22899, offset 0, flags [DF], proto UDP (17), length 33) Setting the TTL to 1 and generating another packet: 3> inet:setopts(S,[{raw,0,2,<<1:32/native>>}]). ok 4> gen_udp:send(S, {8,8,8,8}, 7777, <<"hello">>). ok Checking tcpdump, the TTL is now set to 1. 18:34:03.735254 IP (tos 0x0, ttl 1, id 22900, offset 0, flags [DF], proto UDP (17), length 33) Since the DF bit is set, I'll unset it by passing in 0 (instead of 2): 5> inet:setopts(S,[{raw,0,10,<<0:32/native>>}]). ok And tcpdump shows no flags are now set on the packet: 18:35:01.542090 IP (tos 0x0, ttl 1, id 22901, offset 0, flags [none], proto UDP (17), length 33) > On 8 Jul 2014, at 23:20, Michael Santos wrote: > > > On Tue, Jul 08, 2014 at 10:53:18PM +0100, Lee Sylvester wrote: > >> Hi Michael, > >> > >> Thanks for responding. I was actually just looking through your repositories to see if you had written something to work with this. > >> > >> What I?d like to do is to be able to set the DF or TTL flags. What would you suggest for such things? > > > > Raw sockets would be overkill for this. Have a look at the raw option > > to inet:setopts/2. There's an example in the docs for inet of setting > > TCP_LINGER2 on a TCP socket: > > > > inet:setopts(Sock,[{raw,6,8,<<30:32/native>>}]) > > > > The values will depend on the OS. So for setting the TTL, on linux/x86_64, > > maybe something like: > > > > setsockopt(send_socket, IPPROTO_IP, IP_TTL, &ttl, sizeof(ttl)); > > > > % IPPROTO_IP = 0 > > % IP_TTL = 2 > > % TTL = 64 > > inet:setopts(Sock,[{raw,0,2,<<64:32/native>>}]) > > > > DF is interesting. There's a good discussion about the DF bit here: > > > > http://stackoverflow.com/questions/973439/how-to-set-the-dont-fragment-df-flag-on-a-socket > > > > So on a linux system, maybe this would work: > > > > int val = IP_PMTUDISC_DO; > > setsockopt(sd, IPPROTO_IP, IP_MTU_DISCOVER, &val, sizeof(val)); > > > > % IPPROTO = 0 > > % IP_MTU_DISCOVER = 10 > > % IP_PMTUDISC_DO = 2 > > inet:setopts(Sock,[{raw,0,10,<<2:32/native>>}]) > > > > I haven't tested if these work so let us know if you run into problems > > and I'll try to come up with some working code! > > > >>> On Tue, Jul 08, 2014 at 07:04:12PM +0100, Lee Sylvester wrote: > >>>> Hey guys, > >>>> > >>>> So, I?m using gen_udp for a server I?m building, but I really need to be able to update the headers of incoming packets. Does anyone know how I can do this with gen_udp without having to resort to using raw sockets? > >>> > >>> It depends which header needs to be changed: > >>> > >>> * for UDP (source/destination port, length, checksum), the simplest > >>> method is to receive and resend the packet > >>> > >>> * for IPv4/IPv6, some of the headers may be able to be influenced by setting > >>> socket options. For example, by using inet:setopts/2 or by using > >>> setsockopt(2) from an NIF. > >>> > >>> * otherwise, there's the raw socket interface. > >>> > >>> For IPv4, the socket can be opened in raw mode and passed back into > >>> gen_udp. > >>> > >>> For IPv6 headers, if the headers can be changed, you generally have to > >>> use sendmsg(2)/recvmsg(2) which means using a port or an NIF. > >>> > >>> * for ethernet, it will depend on the OS: PF_PACKET for linux, BPF for > >>> the BSDs > >>> > >>> * another option is routing the packets through a tun/tap device and > >>> having some erlang code re-write the headers > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >> > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > From mark.nijhof@REDACTED Wed Jul 9 01:15:25 2014 From: mark.nijhof@REDACTED (Mark Nijhof) Date: Wed, 9 Jul 2014 01:15:25 +0200 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <53BC66BB.4030301@meetinghouse.net> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3DC4.7000600@erlang-solutions.com> <01D5657C-8AEE-47AD-8C76-2BDD1F2A8C35@gmail.com> <53BC41E7.8070407@erlang-solutions.com> <53BC66BB.4030301@meetinghouse.net> Message-ID: I should have said "technical" writers :) but I see your point. But just looking at the time spend and what one could earn by doing that work vs writing about the work then it usually is a clear case what activity wins. On Tue, Jul 8, 2014 at 11:46 PM, Miles Fidelman wrote: > Mark Nijhof wrote: > >> >> It is not funny anymore how many people think that someone writes a book >> for money. Even when self publishing there is not a lot of money to be >> made, I use LeanPub and am very happy with the platform/service they offer. >> And the royalties are good as well, but I need to pay for the cover design, >> the website (if you want something else then the default) and for editors >> to help me improve my content. So far it has cost me money. >> >> > That does require a comment. A lot of people DO write for money, though > not necessarily looking for all that money to come from book sales. > Academics "publish or perish." Consultants as a marketing vehicle. And > yes, there are people who actually make a living from writing books. > > But yes, there are also non-monetary reasons for writing a book - > consolidating learning, as an excuse to do some personal research, putting > experience or thoughts on paper, promoting a position, and so forth. Even > then, few have the luxury to do that without some means of support - we all > have to eat. If one is retired, has a job that includes writing as a part > of it (can you say academia?), an understanding employer (or an employer > who benefits from your writing), that's great. Otherwise, a grant, a > fellowship, or some form of income is required. > > Writing is a huge amount of work, and it is a shame writers don't see >> more of the results. >> > > Absolutely. Writing is real work. One of the reasons my next book will > be self-published. > > Cheers, > > > Miles Fidelman > > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Mark Nijhof t: @MarkNijhof s: marknijhof -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Wed Jul 9 01:58:35 2014 From: felixgallo@REDACTED (Felix Gallo) Date: Tue, 8 Jul 2014 16:58:35 -0700 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3DC4.7000600@erlang-solutions.com> <01D5657C-8AEE-47AD-8C76-2BDD1F2A8C35@gmail.com> <53BC41E7.8070407@erlang-solutions.com> <53BC66BB.4030301@meetinghouse.net> Message-ID: On the topic of Francesco and Steve's book, as someone who bought the early access version some time ago, it was at the time quite solid and informative; I can only imagine it got even better from there; and I look forward to catching up on it tonight. Although O'Reilly's sins are many and egregious, supporting quality authors and helping to promote the general availability of strong and accessible Erlang documentation is in this case entirely worth the price. Of course, the true Erlang communitarian Nordic-style socialist bookshelf would also include the work of the tireless Mr. Hebert ( http://www.nostarch.com/erlang), the selfless Joe Armstrong ( http://pragprog.com/book/jaerlang2/programming-erlang), and the triple threat of Logan, Merritt and Carlsson (http://www.manning.com/logan/). Why not buy five copies of each for your friends and neighbors? Christmas is just around the corner. And what if one of your copies crashed without trapping exit? F. On Tue, Jul 8, 2014 at 4:15 PM, Mark Nijhof wrote: > I should have said "technical" writers :) but I see your point. But just > looking at the time spend and what one could earn by doing that work vs > writing about the work then it usually is a clear case what activity wins. > > > On Tue, Jul 8, 2014 at 11:46 PM, Miles Fidelman < > mfidelman@REDACTED> wrote: > >> Mark Nijhof wrote: >> >>> >>> It is not funny anymore how many people think that someone writes a book >>> for money. Even when self publishing there is not a lot of money to be >>> made, I use LeanPub and am very happy with the platform/service they offer. >>> And the royalties are good as well, but I need to pay for the cover design, >>> the website (if you want something else then the default) and for editors >>> to help me improve my content. So far it has cost me money. >>> >>> >> That does require a comment. A lot of people DO write for money, though >> not necessarily looking for all that money to come from book sales. >> Academics "publish or perish." Consultants as a marketing vehicle. And >> yes, there are people who actually make a living from writing books. >> >> But yes, there are also non-monetary reasons for writing a book - >> consolidating learning, as an excuse to do some personal research, putting >> experience or thoughts on paper, promoting a position, and so forth. Even >> then, few have the luxury to do that without some means of support - we all >> have to eat. If one is retired, has a job that includes writing as a part >> of it (can you say academia?), an understanding employer (or an employer >> who benefits from your writing), that's great. Otherwise, a grant, a >> fellowship, or some form of income is required. >> >> Writing is a huge amount of work, and it is a shame writers don't see >>> more of the results. >>> >> >> Absolutely. Writing is real work. One of the reasons my next book will >> be self-published. >> >> Cheers, >> >> >> Miles Fidelman >> >> >> -- >> In theory, there is no difference between theory and practice. >> In practice, there is. .... Yogi Berra >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Mark Nijhof > t: @MarkNijhof > s: marknijhof > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.nijhof@REDACTED Wed Jul 9 02:03:59 2014 From: mark.nijhof@REDACTED (Mark Nijhof) Date: Wed, 9 Jul 2014 02:03:59 +0200 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3DC4.7000600@erlang-solutions.com> <01D5657C-8AEE-47AD-8C76-2BDD1F2A8C35@gmail.com> <53BC41E7.8070407@erlang-solutions.com> <53BC66BB.4030301@meetinghouse.net> Message-ID: hehe paper copies don't just crash, they crash and burn On Wed, Jul 9, 2014 at 1:58 AM, Felix Gallo wrote: > On the topic of Francesco and Steve's book, as someone who bought the > early access version some time ago, it was at the time quite solid and > informative; I can only imagine it got even better from there; and I look > forward to catching up on it tonight. Although O'Reilly's sins are many > and egregious, supporting quality authors and helping to promote the > general availability of strong and accessible Erlang documentation is in > this case entirely worth the price. > > Of course, the true Erlang communitarian Nordic-style socialist bookshelf > would also include the work of the tireless Mr. Hebert ( > http://www.nostarch.com/erlang), the selfless Joe Armstrong ( > http://pragprog.com/book/jaerlang2/programming-erlang), and the triple > threat of Logan, Merritt and Carlsson (http://www.manning.com/logan/). > Why not buy five copies of each for your friends and neighbors? Christmas > is just around the corner. And what if one of your copies crashed without > trapping exit? > > F. > > > > > On Tue, Jul 8, 2014 at 4:15 PM, Mark Nijhof < > mark.nijhof@REDACTED> wrote: > >> I should have said "technical" writers :) but I see your point. But just >> looking at the time spend and what one could earn by doing that work vs >> writing about the work then it usually is a clear case what activity wins. >> >> >> On Tue, Jul 8, 2014 at 11:46 PM, Miles Fidelman < >> mfidelman@REDACTED> wrote: >> >>> Mark Nijhof wrote: >>> >>>> >>>> It is not funny anymore how many people think that someone writes a >>>> book for money. Even when self publishing there is not a lot of money to be >>>> made, I use LeanPub and am very happy with the platform/service they offer. >>>> And the royalties are good as well, but I need to pay for the cover design, >>>> the website (if you want something else then the default) and for editors >>>> to help me improve my content. So far it has cost me money. >>>> >>>> >>> That does require a comment. A lot of people DO write for money, though >>> not necessarily looking for all that money to come from book sales. >>> Academics "publish or perish." Consultants as a marketing vehicle. And >>> yes, there are people who actually make a living from writing books. >>> >>> But yes, there are also non-monetary reasons for writing a book - >>> consolidating learning, as an excuse to do some personal research, putting >>> experience or thoughts on paper, promoting a position, and so forth. Even >>> then, few have the luxury to do that without some means of support - we all >>> have to eat. If one is retired, has a job that includes writing as a part >>> of it (can you say academia?), an understanding employer (or an employer >>> who benefits from your writing), that's great. Otherwise, a grant, a >>> fellowship, or some form of income is required. >>> >>> Writing is a huge amount of work, and it is a shame writers don't see >>>> more of the results. >>>> >>> >>> Absolutely. Writing is real work. One of the reasons my next book will >>> be self-published. >>> >>> Cheers, >>> >>> >>> Miles Fidelman >>> >>> >>> -- >>> In theory, there is no difference between theory and practice. >>> In practice, there is. .... Yogi Berra >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> >> -- >> Mark Nijhof >> t: @MarkNijhof >> s: marknijhof >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Mark Nijhof t: @MarkNijhof s: marknijhof -------------- next part -------------- An HTML attachment was scrubbed... URL: From fritchie@REDACTED Wed Jul 9 05:30:52 2014 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Tue, 08 Jul 2014 22:30:52 -0500 Subject: [erlang-questions] Distribution: small packets In-Reply-To: Message of "Wed, 09 Jul 2014 00:48:18 +0400." Message-ID: <50158.1404876652@snookles.snookles.com> Danil Zagoskin wrote: dz> If yes, how can I change this behavior by cost of couple of dz> millisecods latency? Hi, Danil. Have you tried the configuration setting mentioned in the 4th paragraph of the Description of http://www.erlang.org/doc/man/inet.html ? Using the Kernel configuration parameters mentioned above, one can set default options for all TCP sockets on a node. This should be used with care, but options like {delay_send,true} might be specified in this way. An example of starting an Erlang node with all sockets using delayed send could look like this: $ erl -sname test -kernel \ inet_default_connect_options '[{delay_send,true}]' \ inet_default_listen_options '[{delay_send,true}]' Note that 'sndbuf' and 'recbuf' can be set this way also. -Scott From zxq9@REDACTED Wed Jul 9 11:40:45 2014 From: zxq9@REDACTED (zxq9) Date: Wed, 09 Jul 2014 18:40:45 +0900 Subject: [erlang-questions] Microservices vs. Erlang processes In-Reply-To: References: <298EAE28-4128-46BC-B981-154A128760F4@writersglen.com> <10329767.WaXFFmkT2h@jalapeno> Message-ID: <5599975.7mYOjNOmpj@jalapeno> On Monday 07 July 2014 23:54:41 you wrote: > Hi Craig, > > >The idea of using actual micros for "micro"-sized services is > > > > intriguing, but the implications seem to be way outside the realm of the > > average web-style VC's imagination. > > This is an "intriguing" statement. Can you please elaborate on the > implications? Hi Lloyd. I appologize in advance for the huge and unorganized nature of the post below. Anyone not really interested in this, please skip reading it instead of flaming me for verbosity. And so... Blurring the lines between physical infrastructure and software infrastructure can become very natural, but it requires a mode of thinking to which neither software people and, say, building architects are accustomed. This is what I am getting at when I vaguely refer to "implications". Imagine a bridge where each truss, buttress, suspension cable socket, surface segment, etc. are aware of its location in relation to other pieces, and where each joint was an input device to the larger segment. When actuating joints are used (not unusual at all in large structures, but I've never seen any actuating joints that took readings on their current angle, load/stress, etc.), it suddenly becomes possible for a structure to deduce what external forces must be acting on it to cause its recent collective actuation history -- and if even some of the joints are powered (this can take many forms) it would be possible for a suspension bridge to actively dampen the torquing effect of wind, windmills to actively feather, segmented road designs to react to ground saturation, etc. These ideas have occurred to other people, of course, but the approach to solving the implementation problem has always been procedural at the code level, centralized at the processing level, and ultimately unsafe to build because handling partial failure involves too many cases to handle when looking from the top-level. The same ideas necessary for large structure segments to do things well together turn out to be the same ideas necessary very small structure coordination -- spray-on screens, microcontrollers mixed into concrete aggregate, etc. Self discovery, "relative -> definite" orientation discovery, etc. permit a very "sci-fi right now" type design mindset where the difference between physical architecture of everyday devices and the software architecture that makes those devices smart melts away. (I wince at using the term "smart" here -- it is indeed what I mean, but the term has been already so abused that it requires a semantic rebirth.) Any future where we can mimic organic brain activity by recalling ad-hoc paths through physical, incidental, aggregate storage instead of directly retreiving data sets of a known/expected type from a dedicated block device requires this sort of thinking. Consider a current-day "smart" coffee maker. It is first a coffee maker, entirely dumb. Then the "smart" bits are tacked on as an afterthought. They are entirely alien to the actual coffee maker. There is very little difference between having an old-fashioned coffee maker controlled by a robot and a "smart" coffee maker. Mr. Bean's morning routine is an equivalent solution from a technical point of view. A truly smart coffee maker, on the other hand, would have a point of view. It would not just know how much water was in the reservoir, it would *ask* the reservoir how much water it had available. This means the reservoir would be "smart" as well. If it provides water, why is it only providing water to the coffee maker? Why isn't this part of the building's water supply utility? This makes a lot more sense, actually, but so far nobody has approached kitchen appliances in this way and so a water provision protocol has not yet needed to occur to anyone. And so on. There are numerous chicken-and-egg problems involved here, of course, but handling chicken-and-egg situations gracefully is part of what true robustness is really about anyway. Incidentally, chicken-and-egg problems also provide an opportunity to sell two products in place of one: a standalone water device, or "smart provision" add-on to existing water coolers or sinks, in addition to the coffee maker itself. This last point and and consumer case along the lines of the coffee maker example is probably the only hope we could possibly have today of interesting VCs in relooking "micro" services of this form. (I just pulled the coffee maker thing out of my nose, by the way, its not a well thought out idea -- but its sort of a fun example to toy with, because it turns out that finding points of interface among common problems in living/working space management can indeed begin with something so trivial.) These are just elementary examples, my point is that truly "micro" sized services embedded in devices which are themselves micro-sized opens a new style of thinking about the concrete problems of the physical world that computers alone are incapable of dealing with and inflexibly built "smart" systems are incapable of evolving around without inordinate amounts of constant retrofit/reprogramming work. A lot of time is spent today telling a microcontroller to do exactly procedure "start; A(); B(); C(); end". Usually in C, assembly, or some special extension of C that embraces special hardware features directly. All of that work is obsolete when the scope of the job changes, of course. A lot of time is spent elsewhere writing Java or Objective-C programs for our "smart" phones. Neither system is adequate to take the state of computing a step further than it has already been in the last several decades, because (ironically in the phone case) the languages themselves and the runtime environments within which they execute are simply not designed around the realities of massive concurrency or even particularly well suited to elementary message passing! These languages and environments *still* require special encantations be uttered to take advantage of more than a single processing device or storage location! This whole mode of thinking is drastically underpowered in the face of problem spaces larger than "send an email", "open a web page", "run game X", "open_valve()", "print_board()" etc. Sadly, these are the same problems we've already been stubbing our toes on for the last 50 years in software. The only thing that makes the world seem any "smarter" or "more connected" today is that the boob-tube has been renamed to "youtube" and Bruce Wayne isn't the only one who can afford powerful computers in handheld form. What all this computing power is doing, though, is simply wasting cycles in social distractions, mostly as a cover to sell things nobody knew they wanted (or even existed) before the internet or as a cover to track what they do. The problem of creating truly intelligent infrastructure, reactive environments, self-recovering *physical* systems, etc. remains, and it remains a space that has been so far only scratched at around the edges, and never really tackled head-on. Since the advent of the web this set of problems seems to have been forgotten entirely, replaced by dreams of quick cash and notoriety. "Entirely" forgotten outside military research, I should add. But they are stubbing their toes as well. In Ada. Something like Erlang/OTP (and I really have to include the OTP part here) enables a different way not just of thinking about concurrency, inter-process communication, "process" defined the way "object" used to be, etc. but a whole different culture of thought. The Erlang culture hasn't interfaced much with hardware culture, and "hardware" culture hasn't interfaced much with mechanical, civil, etc. engineering culture, so there hasn't been a lot of cross-pollination yet. Someday this sort of thing will be natural, but for now its very unusual to meet a programmer who is also a hardware developer who is also a construction engineer. The magic blocking heisenproblems that have really been holding things up have been concurrency, embedded processing, and routing/addressing across heterogenous, autonomous devices. If this is what we would take "microservices architecture" to mean, which is very nearly what more than one member of this list took the phrase to mean on first reading, we would be headed toward a closer relationship with the physical world and perhaps less enthused by the possibility of using Erlang/OTP to back-end yet another market analytics site or social media platform or CMS platform, etc. on the web. OTOH, the brain is the largest erogenous zone and the web targets it readily. Much like rats given the choice between simulated pleasure stimuli and food, it is quite possible that most humans are more interested in being spoon-fed simulated realities than actually dealing with the real world on its own terms in the most powerful ways we can muster. I'll close this huge and disorganized post with an excerpt from Wikipedia's page on the Fermi Paradox https://en.wikipedia.org/wiki/Fermi_paradox#They_do_exist.2C_but_we_see_no_evidence: > They are too busy online > > It may be that intelligent alien life forms cause their own "increasing > disinterest" in the outside world. Perhaps any sufficiently advanced > society will develop highly engaging media and entertainment well before > the capacity for advanced space travel, and that the rate of appeal of > these social contrivances is destined, because of their inherent reduced > complexity, to overtake any desire for complex, expensive endeavors such as > space exploration and communication. Once any sufficiently advanced > civilization becomes able to master its environment, and most of its > physical needs are met through technology, various "social and > entertainment technologies", including virtual reality, are postulated to > become the primary drivers and motivations of that civilization. -Craig Everett From thomasl_erlang@REDACTED Wed Jul 9 11:56:36 2014 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 9 Jul 2014 02:56:36 -0700 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <1404849334.63757001@apps.rackspace.com> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3DC4.7000600@erlang-solutions.com> <1404849334.63757001@apps.rackspace.com> Message-ID: <1404899796.12850.YahooMailNeo@web163606.mail.gq1.yahoo.com> C'mon, does no one here purchase books through their employer? If you're a multi-employee company working with Erlang, or even a one man band who is actually getting paid, it's a no-brainer to buy a copy of basically every erlang book published. The dent in your budget will be forgotten in days if not hours. Minutes. Best, Thomas On Tuesday, July 8, 2014 9:55 PM, "lloyd@REDACTED" wrote: > > >Hello, > >My thoughts as an a novelist and army-of-one small press publisher: > >- Several sites discuss how to develop a proforma profit & loss state for a book project > >http://pimpmynovel.blogspot.com/2009/10/p-1-of-4-basics.html > >- Even in my bare-bones world where I often stringently discount my time, expenses add up fast. O'Reilly carries considerable overhead and brings many paid person hours to a book project. > >- Sad fact is, few books sell enough to cover the author's advance. > >- I too gasped when I saw the combined price of the physical and e-book editions of Cesarini/Vinoski. But that said, even at full price, I seriously doubt that anyone involved with Franceso and Steve's book project will make a killing. > >- Like others on this thread, I could not justify purchasing the book at this point without Franceso's discount code. Not that I consider the two editiorns overpriced for value delivered. But rather, I simply couldn't squeeze the purchase into my book-buying budget. > >But with the discount code, purchasing both was a no-brainer given the hard-won expertise that Francesco and Steve are sharing so generously through their book. Just imagine how fast the consulting fees would mount if we had to hire either to help with our software projects. > >- The wonderful thing about the web is that so much invaluable information is available for the price of a few mouse clicks. It's spoiled us into thinking that information is free. But generating worthwhile information entails considerable time and cost. > >I, for one, benefit considerably and feel boundless gratitude for the wonderful sharing economy that has emerged through the open-source movement. For me it's a privilege to pay back in any way I can. Buying a book that supports the titans of our field who've contributed so much is the least I can do. > >All the best, > >Lloyd > >-----Original Message----- >From: "Francesco Cesarini" >Sent: Tuesday, July 8, 2014 2:51pm >To: erlang-questions@REDACTED >Subject: Re: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly > >You do not write books because you believe you are going to make money >out of them. You write them because you are passionate about the >subject. If you calculate what Simon and I made from our first book, I >am not sure we've hit minimum wage yet. From my side, I want to document >my approach to teaching OTP, as I think it will work in writing as well >as it does in the classroom. From the feedback Steve and I have received >so far, we are right on track. Reiterating an email on this list from >2008, I want to see a whole bookshelf of Erlang/OTP books out there. > >I found the experience of working with O'Reilly really positive the >first time around. From the editor, the production team (Graphics, copy >editor, proof readers, etc) as well as their marketing and conferences.? >And this time around, it is just as good, if not better. As an author, I >could not recommend them more highly. I want to write books, I do not >want to do all of the other stuff associated with getting it out. It is >just a false economy. > >/F > >PS. For those who can't afford 25$, try before you buy. It is called >BitTorrent. > >On 08/07/2014 19:39, Raoul Duke wrote: >> i haven't looked at the books here, but having been somebody who >> produced things and wondered how anybody could ever make a living at >> it, and knowing that writers throughout history rarely made much after >> the publisher etc. got a cut, i didn't have the same reaction to the >> $50 price tag. can't say i'd *spend* the $50 since i'm broke and >> there's a zillion other things to spend $50 on first. i'd try to get >> it via inter library loan or something :-). >> >> On Tue, Jul 8, 2014 at 11:36 AM, Francesco Cesarini >> wrote: >>> Alas, that is how O'Reilly price their books. Having a high price and then >>> discounting is not the approach I would pick. That is why discount codes are >>> being handed out on public mailing lists and social media. I recommend you >>> use them (Read, no one pays full price for an O'Reilly book). As an unedited >>> book, the cost is for the final book which we hope will complement what is >>> already out there. It is a different approach to OTP in action. One I've >>> been using for 15 years when teaching OTP. >>> >>> /F >>> >>> On 08/07/2014 19:14, Lee Sylvester wrote: >>>> Wow, $50?? Yeah, that is steep.? If this were some black arts compiler >>>> book or video encoding bible, then that would be something else.? But a book >>>> on Erlang/OTP just doesn?t fall into that bracket? >>>> >>>> I may wait til it falls in the bargain bucket ;-) >>>> >>>> Lee >>>> >>>>? ? On 8 Jul 2014, at 19:08, Miles Fidelman >>>> wrote: >>>> >>>>> On 8 Jul 2014, at 16:05, Francesco Cesarini >>>>> wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> a shameless plug. Steve Vinoski and I are working on a book focused on >>>>>>> distributed, scalable systems with OTP. It is available from O'Reilly as an >>>>>>> Early Release from their website. The first eight chapters we released cover >>>>>>> (in great detail) all other behaviours. We started writing, and before we >>>>>>> knew it, we had several hundred pages on behaviours alone. Last week, two >>>>>>> new chapters, including the Introduction & Special processes and >>>>>>> Implementing your own behaviours were released. We are now focusing on >>>>>>> release handling (hgg), code upgrade and architectural patterns. >>>>>>> >>>>>>> What is available is an unedited draft, with new chapters and >>>>>>> improvements published as they become available. You can find more info >>>>>>> here: >>>>>>> >>>>>>> >>>>>>> http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>>>>>> >>>>>>> If you use discount code authd, you will get 50% off the Early Release, >>>>>>> and 40% on pre-orders of the the printed copy. >>>>>>> >>>>>>> Looking forward to your feedback, >>>>>>> >>>>> Not for nothing, but $50 for the undedited work-in-progress seems a bit >>>>> steep (and yes, I saw the discount code).? I also note that "rough cuts" >>>>> aren't included in my Safari subscription. >>>>> >>>>> Usual practice that I've seen is for works-in-progress to be free, with >>>>> tools that support comments from early readers. >>>>> >>>>> Sounds interesting - but, from the TOC (all that's available without >>>>> paying), it sure looks like it covers the same ground as "Erlang and OTP in >>>>> Action." >>>>> >>>>> Miles Fidelman >>>>> >>>>> -- >>>>> In theory, there is no difference between theory and practice. >>>>> In practice, there is.? .... Yogi Berra >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> -- >>> Erlang Solutions Ltd. >>> http://www.erlang-solutions.com > >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > >-- >Erlang Solutions Ltd. >http://www.erlang-solutions.com > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions > > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zweicmu@REDACTED Wed Jul 9 10:49:31 2014 From: zweicmu@REDACTED (Zhibo Wei) Date: Wed, 9 Jul 2014 01:49:31 -0700 Subject: [erlang-questions] filelib:fold_files doesnt work for symlink anymore (on Erlang/OTP 17)? Message-ID: Here is how my src dict look like: src/ ??? 1.erl ??? 2.erl ??? 3.src ??? symlink -> ../../xxx/yyy/ and ' ../../xxx/yyy/' contains several '.proto' files Here is what I did in erl console: 1> filelib:fold_files("src", ".*\\.proto$", true, fun(F, Acc) -> [F | Acc] end, []). [] 2> filelib:fold_files("src", ".*\\.erl$", true, fun(F, Acc) -> [F | Acc] end, []). ["/home/xxx/src/1.erl", "/home/xxx/src/2.erl"] 3> filelib:fold_files("src/symlink", ".*\\.proto$", true, fun(F, Acc) -> [F | Acc] end, []). ["/home/xxx/src/symlink/1.proto", "/home/xxx/src/symlink/2.proto"] But as I remembered, filelib:fold_files works fine before I upgrade erlang. My erlang version is 'Erlang/OTP 17 [erts-6.1] [source] [64-bit]' and it's running on Ubuntu 14.04 Is this a known issue of Erlang? Is there any work around to make this working other than downgrade erlang? Please let me know, thanks a lot! Thanks, Zhibo -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Wed Jul 9 12:39:24 2014 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Wed, 9 Jul 2014 12:39:24 +0200 Subject: [erlang-questions] filelib:fold_files doesnt work for symlink anymore (on Erlang/OTP 17)? In-Reply-To: References: Message-ID: It is a regression on Erlang 17.1. It has been reported here: http://erlang.org/pipermail/erlang-bugs/2014-June/004465.html And a pull request to fix it is here: https://github.com/erlang/otp/pull/417 *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Lead Developer On Wed, Jul 9, 2014 at 10:49 AM, Zhibo Wei wrote: > Here is how my src dict look like: > src/ > ??? 1.erl > ??? 2.erl > ??? 3.src > ??? symlink -> ../../xxx/yyy/ > > and ' ../../xxx/yyy/' contains several '.proto' files > > Here is what I did in erl console: > 1> filelib:fold_files("src", ".*\\.proto$", true, fun(F, Acc) -> [F | Acc] > end, []). > [] > 2> filelib:fold_files("src", ".*\\.erl$", true, fun(F, Acc) -> [F | Acc] > end, []). > ["/home/xxx/src/1.erl", > "/home/xxx/src/2.erl"] > 3> filelib:fold_files("src/symlink", ".*\\.proto$", true, fun(F, Acc) -> > [F | Acc] end, []). > ["/home/xxx/src/symlink/1.proto", > "/home/xxx/src/symlink/2.proto"] > > But as I remembered, filelib:fold_files works fine before I upgrade erlang. > My erlang version is 'Erlang/OTP 17 [erts-6.1] [source] [64-bit]' and it's > running on Ubuntu 14.04 > > Is this a known issue of Erlang? Is there any work around to make this > working other than downgrade erlang? Please let me know, thanks a lot! > > Thanks, > Zhibo > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From francesco@REDACTED Wed Jul 9 12:47:53 2014 From: francesco@REDACTED (Francesco Cesarini) Date: Wed, 09 Jul 2014 11:47:53 +0100 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <1404899796.12850.YahooMailNeo@web163606.mail.gq1.yahoo.com> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3DC4.7000600@erlang-solutions.com> <1404849334.63757001@apps.rackspace.com> <1404899796.12850.YahooMailNeo@web163606.mail.gq1.yahoo.com> Message-ID: <53BD1DD9.1050109@erlang-solutions.com> Come on Thomas, if an employer pays for the book, the employee learns a new set of skills and moves on to a new job. Ignorance is bliss! /F On 09/07/2014 10:56, Thomas Lindgren wrote: > C'mon, does no one here purchase books through their employer? If > you're a multi-employee company working with Erlang, or even a one man > band who is actually getting paid, it's a no-brainer to buy a copy of > basically every erlang book published. The dent in your budget will be > forgotten in days if not hours. Minutes. > > Best, > Thomas > > > > On Tuesday, July 8, 2014 9:55 PM, "lloyd@REDACTED" > wrote: > > > > Hello, > > My thoughts as an a novelist and army-of-one small press publisher: > > - Several sites discuss how to develop a proforma profit & loss > state for a book project > > http://pimpmynovel.blogspot.com/2009/10/p-1-of-4-basics.html > > - Even in my bare-bones world where I often stringently discount > my time, expenses add up fast. O'Reilly carries considerable > overhead and brings many paid person hours to a book project. > > - Sad fact is, few books sell enough to cover the author's advance. > > - I too gasped when I saw the combined price of the physical and > e-book editions of Cesarini/Vinoski. But that said, even at full > price, I seriously doubt that anyone involved with Franceso and > Steve's book project will make a killing. > > - Like others on this thread, I could not justify purchasing the > book at this point without Franceso's discount code. Not that I > consider the two editiorns overpriced for value delivered. But > rather, I simply couldn't squeeze the purchase into my book-buying > budget. > > But with the discount code, purchasing both was a no-brainer given > the hard-won expertise that Francesco and Steve are sharing so > generously through their book. Just imagine how fast the > consulting fees would mount if we had to hire either to help with > our software projects. > > - The wonderful thing about the web is that so much invaluable > information is available for the price of a few mouse clicks. It's > spoiled us into thinking that information is free. But generating > worthwhile information entails considerable time and cost. > > I, for one, benefit considerably and feel boundless gratitude for > the wonderful sharing economy that has emerged through the > open-source movement. For me it's a privilege to pay back in any > way I can. Buying a book that supports the titans of our field > who've contributed so much is the least I can do. > > All the best, > > Lloyd > > -----Original Message----- > From: "Francesco Cesarini" > > Sent: Tuesday, July 8, 2014 2:51pm > To: erlang-questions@REDACTED > Subject: Re: [erlang-questions] ANN: Designing for Scalability > with Erlang/OTP by O'Reilly > > You do not write books because you believe you are going to make > money > out of them. You write them because you are passionate about the > subject. If you calculate what Simon and I made from our first > book, I > am not sure we've hit minimum wage yet. From my side, I want to > document > my approach to teaching OTP, as I think it will work in writing as > well > as it does in the classroom. From the feedback Steve and I have > received > so far, we are right on track. Reiterating an email on this list from > 2008, I want to see a whole bookshelf of Erlang/OTP books out there. > > I found the experience of working with O'Reilly really positive the > first time around. From the editor, the production team (Graphics, > copy > editor, proof readers, etc) as well as their marketing and > conferences. > And this time around, it is just as good, if not better. As an > author, I > could not recommend them more highly. I want to write books, I do not > want to do all of the other stuff associated with getting it out. > It is > just a false economy. > > /F > > PS. For those who can't afford 25$, try before you buy. It is called > BitTorrent. > > On 08/07/2014 19:39, Raoul Duke wrote: > > i haven't looked at the books here, but having been somebody who > > produced things and wondered how anybody could ever make a living at > > it, and knowing that writers throughout history rarely made much > after > > the publisher etc. got a cut, i didn't have the same reaction to the > > $50 price tag. can't say i'd *spend* the $50 since i'm broke and > > there's a zillion other things to spend $50 on first. i'd try to get > > it via inter library loan or something :-). > > > > On Tue, Jul 8, 2014 at 11:36 AM, Francesco Cesarini > > > wrote: > >> Alas, that is how O'Reilly price their books. Having a high > price and then > >> discounting is not the approach I would pick. That is why > discount codes are > >> being handed out on public mailing lists and social media. I > recommend you > >> use them (Read, no one pays full price for an O'Reilly book). > As an unedited > >> book, the cost is for the final book which we hope will > complement what is > >> already out there. It is a different approach to OTP in action. > One I've > >> been using for 15 years when teaching OTP. > >> > >> /F > >> > >> On 08/07/2014 19:14, Lee Sylvester wrote: > >>> Wow, $50? Yeah, that is steep. If this were some black arts > compiler > >>> book or video encoding bible, then that would be something > else. But a book > >>> on Erlang/OTP just doesn?t fall into that bracket? > >>> > >>> I may wait til it falls in the bargain bucket ;-) > >>> > >>> Lee > >>> > >>> On 8 Jul 2014, at 19:08, Miles Fidelman > > > >>> wrote: > >>> > >>>> On 8 Jul 2014, at 16:05, Francesco Cesarini > >>>> > wrote: > >>>>>> Hi all, > >>>>>> > >>>>>> a shameless plug. Steve Vinoski and I are working on a book > focused on > >>>>>> distributed, scalable systems with OTP. It is available > from O'Reilly as an > >>>>>> Early Release from their website. The first eight chapters > we released cover > >>>>>> (in great detail) all other behaviours. We started writing, > and before we > >>>>>> knew it, we had several hundred pages on behaviours alone. > Last week, two > >>>>>> new chapters, including the Introduction & Special > processes and > >>>>>> Implementing your own behaviours were released. We are now > focusing on > >>>>>> release handling (hgg), code upgrade and architectural > patterns. > >>>>>> > >>>>>> What is available is an unedited draft, with new chapters and > >>>>>> improvements published as they become available. You can > find more info > >>>>>> here: > >>>>>> > >>>>>> > >>>>>> > http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct > >>>>>> > >>>>>> If you use discount code authd, you will get 50% off the > Early Release, > >>>>>> and 40% on pre-orders of the the printed copy. > >>>>>> > >>>>>> Looking forward to your feedback, > >>>>>> > >>>> Not for nothing, but $50 for the undedited work-in-progress > seems a bit > >>>> steep (and yes, I saw the discount code). I also note that > "rough cuts" > >>>> aren't included in my Safari subscription. > >>>> > >>>> Usual practice that I've seen is for works-in-progress to be > free, with > >>>> tools that support comments from early readers. > >>>> > >>>> Sounds interesting - but, from the TOC (all that's available > without > >>>> paying), it sure looks like it covers the same ground as > "Erlang and OTP in > >>>> Action." > >>>> > >>>> Miles Fidelman > >>>> > >>>> -- > >>>> In theory, there is no difference between theory and practice. > >>>> In practice, there is. .... Yogi Berra > >>>> > >>>> _______________________________________________ > >>>> erlang-questions mailing list > >>>> erlang-questions@REDACTED > >>>> http://erlang.org/mailman/listinfo/erlang-questions > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> -- > >> Erlang Solutions Ltd. > >> http://www.erlang-solutions.com > > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -- > Erlang Solutions Ltd. > http://www.erlang-solutions.com > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Erlang Solutions Ltd. http://www.erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From zweicmu@REDACTED Wed Jul 9 12:42:30 2014 From: zweicmu@REDACTED (Zhibo Wei) Date: Wed, 9 Jul 2014 03:42:30 -0700 Subject: [erlang-questions] filelib:fold_files doesnt work for symlink anymore (on Erlang/OTP 17)? In-Reply-To: References: Message-ID: Ah.... thats why Thanks for lettingme know! Thanks, Zhibo On Wed, Jul 9, 2014 at 3:39 AM, Jos? Valim wrote: > It is a regression on Erlang 17.1. > > It has been reported here: > http://erlang.org/pipermail/erlang-bugs/2014-June/004465.html > > And a pull request to fix it is here: > https://github.com/erlang/otp/pull/417 > > > > *Jos? Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Lead Developer > > > On Wed, Jul 9, 2014 at 10:49 AM, Zhibo Wei wrote: > >> Here is how my src dict look like: >> src/ >> ??? 1.erl >> ??? 2.erl >> ??? 3.src >> ??? symlink -> ../../xxx/yyy/ >> >> and ' ../../xxx/yyy/' contains several '.proto' files >> >> Here is what I did in erl console: >> 1> filelib:fold_files("src", ".*\\.proto$", true, fun(F, Acc) -> [F | >> Acc] end, []). >> [] >> 2> filelib:fold_files("src", ".*\\.erl$", true, fun(F, Acc) -> [F | Acc] >> end, []). >> ["/home/xxx/src/1.erl", >> "/home/xxx/src/2.erl"] >> 3> filelib:fold_files("src/symlink", ".*\\.proto$", true, fun(F, Acc) -> >> [F | Acc] end, []). >> ["/home/xxx/src/symlink/1.proto", >> "/home/xxx/src/symlink/2.proto"] >> >> But as I remembered, filelib:fold_files works fine before I upgrade >> erlang. >> My erlang version is 'Erlang/OTP 17 [erts-6.1] [source] [64-bit]' and >> it's running on Ubuntu 14.04 >> >> Is this a known issue of Erlang? Is there any work around to make this >> working other than downgrade erlang? Please let me know, thanks a lot! >> >> Thanks, >> Zhibo >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From z@REDACTED Wed Jul 9 14:57:01 2014 From: z@REDACTED (Danil Zagoskin) Date: Wed, 9 Jul 2014 16:57:01 +0400 Subject: [erlang-questions] Distribution: small packets In-Reply-To: <50158.1404876652@snookles.snookles.com> References: <50158.1404876652@snookles.snookles.com> Message-ID: Thank you, Scott! I performed some tests and here are my results (5000 concurrent HTTP clients with cowboy on every node): * {delay_send, true} has almost no effect * kernel parameter {dist_nodelay, false} (disables nodelay socket option on dist port) gives 25% more service rps (32K to 40K) with large latency (98% is at 30 ms vs 4 ms in default setup) * latency slowly increases up to 250 ms, then jumps to 550..650 ms. Possibly related to tcp retransmit. On Wed, Jul 9, 2014 at 7:30 AM, Scott Lystig Fritchie wrote: > Danil Zagoskin wrote: > > dz> If yes, how can I change this behavior by cost of couple of > dz> millisecods latency? > > Hi, Danil. Have you tried the configuration setting mentioned in the > 4th paragraph of the Description of > http://www.erlang.org/doc/man/inet.html ? > > Using the Kernel configuration parameters mentioned above, one can > set default options for all TCP sockets on a node. This should be > used with care, but options like {delay_send,true} might be > specified in this way. An example of starting an Erlang node with > all sockets using delayed send could look like this: > > $ erl -sname test -kernel \ > inet_default_connect_options '[{delay_send,true}]' \ > inet_default_listen_options '[{delay_send,true}]' > > Note that 'sndbuf' and 'recbuf' can be set this way also. > > -Scott > -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Wed Jul 9 18:04:44 2014 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Wed, 9 Jul 2014 12:04:44 -0400 Subject: [erlang-questions] Microservices vs. Erlang processes In-Reply-To: <5599975.7mYOjNOmpj@jalapeno> References: <298EAE28-4128-46BC-B981-154A128760F4@writersglen.com> <10329767.WaXFFmkT2h@jalapeno> <5599975.7mYOjNOmpj@jalapeno> Message-ID: <86D029F0-2D6D-47A2-9D51-6ADA4B466D46@writersglen.com> Hi Craig, You are exploring deep and rich ideas and questions. What a treat! Some may find the discussion too far afield for this list, in which case, we can take it off list. But, in my view, it cuts to the philosophical heart of Erlang system design philosophy and architecture. Your speculations entwine both profound technical and cultural issues. Indeed, they come across as a social critique. I'm wondering if it might not help to tease out the essential questions and their implications before second-guessing what VCs might consider or reject. As one of those fuzzy divergent thinkers coming out of a liberal arts education I don't know how much I can contribute. But here are some questions and associations that pop to mind: - What is the intrinsic difference between software and hardware; e.g. What is the difference, say, between a digital pid controller, say, and a Watts governor? - What do the linkages of the Watts governor need to "know" to perform their role in a fully functioning mechanism? - What does an ant need to "know" to forage for food for it's colony? - What does a cell need to "know" to contribute to the health and functionality of the larger organism? - What does a consumer need to "know" to participate in an "economy?" - What does stone need to "know" to start an avalanche? - What do we mean by "know?" Ludwig von Bertalanffy tackled many issues such as these in his General Systems Theory. http://www.nwlink.com/~donclark/history_isd/bertalanffy.html I do take issue with your characterization of the Internet. As one whose career has been devoted to human communication, I see the web as a multiple-node network or system of bridges, if you will, between human minds. People process and respond to content they receive through the web in more ways than we can enumerate. It's too easy, in my view, to say that they are simply narcotized and turned into consuming robots. One last thought: Chuck Moore, the mind behind Forth and one of my heroes, has been striving his whole career with but partial success to make the computer "invisible." One wonders how different in degree and function is a processor in his GA144 and an Erlang process? http://www.greenarraychips.com/home/products/ Do hope we can keep the line of thought going either on or off list. All the best, Lloyd Sent from my iPad > On Jul 9, 2014, at 5:40 AM, zxq9 wrote: > >> On Monday 07 July 2014 23:54:41 you wrote: >> Hi Craig, >> >>> The idea of using actual micros for "micro"-sized services is >>> >>> intriguing, but the implications seem to be way outside the realm of the >>> average web-style VC's imagination. >> >> This is an "intriguing" statement. Can you please elaborate on the >> implications? > > Hi Lloyd. > > I appologize in advance for the huge and unorganized nature of the post below. > Anyone not really interested in this, please skip reading it instead of > flaming me for verbosity. > > And so... > > Blurring the lines between physical infrastructure and software infrastructure > can become very natural, but it requires a mode of thinking to which neither > software people and, say, building architects are accustomed. This is what I > am getting at when I vaguely refer to "implications". > > Imagine a bridge where each truss, buttress, suspension cable socket, surface > segment, etc. are aware of its location in relation to other pieces, and where > each joint was an input device to the larger segment. When actuating joints > are used (not unusual at all in large structures, but I've never seen any > actuating joints that took readings on their current angle, load/stress, > etc.), it suddenly becomes possible for a structure to deduce what external > forces must be acting on it to cause its recent collective actuation history > -- and if even some of the joints are powered (this can take many forms) it > would be possible for a suspension bridge to actively dampen the torquing > effect of wind, windmills to actively feather, segmented road designs to react > to ground saturation, etc. These ideas have occurred to other people, of > course, but the approach to solving the implementation problem has always been > procedural at the code level, centralized at the processing level, and > ultimately unsafe to build because handling partial failure involves too many > cases to handle when looking from the top-level. > > The same ideas necessary for large structure segments to do things well > together turn out to be the same ideas necessary very small structure > coordination -- spray-on screens, microcontrollers mixed into concrete > aggregate, etc. Self discovery, "relative -> definite" orientation discovery, > etc. permit a very "sci-fi right now" type design mindset where the difference > between physical architecture of everyday devices and the software > architecture that makes those devices smart melts away. (I wince at using the > term "smart" here -- it is indeed what I mean, but the term has been already > so abused that it requires a semantic rebirth.) Any future where we can mimic > organic brain activity by recalling ad-hoc paths through physical, incidental, > aggregate storage instead of directly retreiving data sets of a known/expected > type from a dedicated block device requires this sort of thinking. > > Consider a current-day "smart" coffee maker. It is first a coffee maker, > entirely dumb. Then the "smart" bits are tacked on as an afterthought. They > are entirely alien to the actual coffee maker. There is very little difference > between having an old-fashioned coffee maker controlled by a robot and a > "smart" coffee maker. Mr. Bean's morning routine is an equivalent solution > from a technical point of view. > > A truly smart coffee maker, on the other hand, would have a point of view. It > would not just know how much water was in the reservoir, it would *ask* the > reservoir how much water it had available. This means the reservoir would be > "smart" as well. If it provides water, why is it only providing water to the > coffee maker? Why isn't this part of the building's water supply utility? This > makes a lot more sense, actually, but so far nobody has approached kitchen > appliances in this way and so a water provision protocol has not yet needed to > occur to anyone. And so on. > > There are numerous chicken-and-egg problems involved here, of course, but > handling chicken-and-egg situations gracefully is part of what true robustness > is really about anyway. Incidentally, chicken-and-egg problems also provide an > opportunity to sell two products in place of one: a standalone water device, > or "smart provision" add-on to existing water coolers or sinks, in addition to > the coffee maker itself. This last point and and consumer case along the lines > of the coffee maker example is probably the only hope we could possibly have > today of interesting VCs in relooking "micro" services of this form. (I just > pulled the coffee maker thing out of my nose, by the way, its not a well > thought out idea -- but its sort of a fun example to toy with, because it > turns out that finding points of interface among common problems in > living/working space management can indeed begin with something so trivial.) > > These are just elementary examples, my point is that truly "micro" sized > services embedded in devices which are themselves micro-sized opens a new > style of thinking about the concrete problems of the physical world that > computers alone are incapable of dealing with and inflexibly built "smart" > systems are incapable of evolving around without inordinate amounts of > constant retrofit/reprogramming work. > > A lot of time is spent today telling a microcontroller to do exactly procedure > "start; A(); B(); C(); end". Usually in C, assembly, or some special extension > of C that embraces special hardware features directly. All of that work is > obsolete when the scope of the job changes, of course. A lot of time is spent > elsewhere writing Java or Objective-C programs for our "smart" phones. Neither > system is adequate to take the state of computing a step further than it has > already been in the last several decades, because (ironically in the phone > case) the languages themselves and the runtime environments within which they > execute are simply not designed around the realities of massive concurrency or > even particularly well suited to elementary message passing! These languages > and environments *still* require special encantations be uttered to take > advantage of more than a single processing device or storage location! > > This whole mode of thinking is drastically underpowered in the face of problem > spaces larger than "send an email", "open a web page", "run game X", > "open_valve()", "print_board()" etc. Sadly, these are the same problems we've > already been stubbing our toes on for the last 50 years in software. The only > thing that makes the world seem any "smarter" or "more connected" today is > that the boob-tube has been renamed to "youtube" and Bruce Wayne isn't the > only one who can afford powerful computers in handheld form. What all this > computing power is doing, though, is simply wasting cycles in social > distractions, mostly as a cover to sell things nobody knew they wanted (or > even existed) before the internet or as a cover to track what they do. > > The problem of creating truly intelligent infrastructure, reactive > environments, self-recovering *physical* systems, etc. remains, and it remains > a space that has been so far only scratched at around the edges, and never > really tackled head-on. Since the advent of the web this set of problems seems > to have been forgotten entirely, replaced by dreams of quick cash and > notoriety. "Entirely" forgotten outside military research, I should add. But > they are stubbing their toes as well. In Ada. > > Something like Erlang/OTP (and I really have to include the OTP part here) > enables a different way not just of thinking about concurrency, inter-process > communication, "process" defined the way "object" used to be, etc. but a whole > different culture of thought. The Erlang culture hasn't interfaced much with > hardware culture, and "hardware" culture hasn't interfaced much with > mechanical, civil, etc. engineering culture, so there hasn't been a lot of > cross-pollination yet. Someday this sort of thing will be natural, but for now > its very unusual to meet a programmer who is also a hardware developer who is > also a construction engineer. > > The magic blocking heisenproblems that have really been holding things up have > been concurrency, embedded processing, and routing/addressing across > heterogenous, autonomous devices. If this is what we would take "microservices > architecture" to mean, which is very nearly what more than one member of this > list took the phrase to mean on first reading, we would be headed toward a > closer relationship with the physical world and perhaps less enthused by the > possibility of using Erlang/OTP to back-end yet another market analytics site > or social media platform or CMS platform, etc. on the web. > > OTOH, the brain is the largest erogenous zone and the web targets it readily. > Much like rats given the choice between simulated pleasure stimuli and food, > it is quite possible that most humans are more interested in being spoon-fed > simulated realities than actually dealing with the real world on its own terms > in the most powerful ways we can muster. > > I'll close this huge and disorganized post with an excerpt from Wikipedia's > page on the Fermi Paradox > https://en.wikipedia.org/wiki/Fermi_paradox#They_do_exist.2C_but_we_see_no_evidence: > >> They are too busy online >> >> It may be that intelligent alien life forms cause their own "increasing >> disinterest" in the outside world. Perhaps any sufficiently advanced >> society will develop highly engaging media and entertainment well before >> the capacity for advanced space travel, and that the rate of appeal of >> these social contrivances is destined, because of their inherent reduced >> complexity, to overtake any desire for complex, expensive endeavors such as >> space exploration and communication. Once any sufficiently advanced >> civilization becomes able to master its environment, and most of its >> physical needs are met through technology, various "social and >> entertainment technologies", including virtual reality, are postulated to >> become the primary drivers and motivations of that civilization. > > -Craig Everett > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mallen@REDACTED Wed Jul 9 22:19:09 2014 From: mallen@REDACTED (Mark Allen) Date: Wed, 9 Jul 2014 15:19:09 -0500 Subject: [erlang-questions] What factors correlate with programming language popularity/adoption? Message-ID: In this very excellent blog post http://www.pl-enthusiast.net/?p=235 Michael Hicks summarizes some recent research into what makes certain programming languages popular and others less so. It especially includes recommendations on how to improve adoption of functional programming languages. While I highly recommend reading the whole blog post and the linked-to research paper http://www.cs.princeton.edu/~asrabkin/papers/oopsla13.pdf The tl;dr summary is below: > The paper shows convincingly that the single factor that most strongly > correlates with both preferring and actually using a language is good > libraries, particularly open source libraries. This is an area, if I may soapbox for a moment, in which Erlang is conspicuously lacking and it suggests a focus area for improvement if increasing adoption and use of Erlang is a community goal. In includes a namedrop of Joe Armstrong for good clickbait measure. Mark From ivan@REDACTED Wed Jul 9 22:32:09 2014 From: ivan@REDACTED (Ivan Uemlianin) Date: Wed, 9 Jul 2014 21:32:09 +0100 Subject: [erlang-questions] What factors correlate with programming language popularity/adoption? In-Reply-To: References: Message-ID: <9F2350ED-9B52-40E7-A01F-ED22566A2B38@llaisdy.com> Do you think erlang is lacking good libraries? -- festina lente > On 9 Jul 2014, at 21:19, Mark Allen wrote: > > In this very excellent blog post > > http://www.pl-enthusiast.net/?p=235 > > Michael Hicks summarizes some recent research into what makes certain > programming languages popular and others less so. It especially includes > recommendations on how to improve adoption of functional programming > languages. > > While I highly recommend reading the whole blog post and the linked-to > research paper > > http://www.cs.princeton.edu/~asrabkin/papers/oopsla13.pdf > > The tl;dr summary is below: > >> The paper shows convincingly that the single factor that most > strongly >> correlates with both preferring and actually using a language is > good >> libraries, particularly open source libraries. > > This is an area, if I may soapbox for a moment, in which Erlang is > conspicuously lacking and it suggests a focus area for improvement if > increasing adoption and use of Erlang is a community goal. > > In includes a namedrop of Joe Armstrong for good clickbait measure. > > Mark > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mfidelman@REDACTED Wed Jul 9 22:36:27 2014 From: mfidelman@REDACTED (Miles Fidelman) Date: Wed, 09 Jul 2014 16:36:27 -0400 Subject: [erlang-questions] What factors correlate with programming language popularity/adoption? In-Reply-To: References: Message-ID: <53BDA7CB.4090600@meetinghouse.net> Mark Allen wrote: > In this very excellent blog post > > http://www.pl-enthusiast.net/?p=235 > > Michael Hicks summarizes some recent research into what makes certain > programming languages popular and others less so. It especially includes > recommendations on how to improve adoption of functional programming > languages. > > One critical note: The metric used was "The *six most popular languages* used in Sourceforge projects" - as opposed to, say, "lines of production code." I expect that very little large-scale system development, or mission-critical system development, shows up on Sourceforge. Not sure how to measure it, but I expect if one looked at MIS and Financial systems, one would (still) find a predominance of COBOL. For mission-critical systems you'd probably find a lot of Ada in aerospace (can you say fly-by-wire?) and SCADA, and a lot of Erlang in telecom. I expect that very different factors influence language selection for such applications. Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From mallen@REDACTED Wed Jul 9 22:57:48 2014 From: mallen@REDACTED (Mark Allen) Date: Wed, 9 Jul 2014 15:57:48 -0500 Subject: [erlang-questions] What factors correlate with programming language popularity/adoption? In-Reply-To: <9F2350ED-9B52-40E7-A01F-ED22566A2B38@llaisdy.com> References: <9F2350ED-9B52-40E7-A01F-ED22566A2B38@llaisdy.com> Message-ID: On 7/9/14 3:32 PM, "Ivan Uemlianin" wrote: >Do you think erlang is lacking good libraries? Yes. Slightly more nuanced: It is very difficult to understand *what* libraries to use even if they are available. Mark From raould@REDACTED Wed Jul 9 23:06:09 2014 From: raould@REDACTED (Raoul Duke) Date: Wed, 9 Jul 2014 14:06:09 -0700 Subject: [erlang-questions] What factors correlate with programming language popularity/adoption? In-Reply-To: References: <9F2350ED-9B52-40E7-A01F-ED22566A2B38@llaisdy.com> Message-ID: (even when ocaml or haskell have libraries, they are kind of a pain to figure out and decide if they are actually trustworthy. :-{ whereas the jdk has stuff "from sun/oracle the horse's mouth" so you have a little more faith, even if they suck a lot at times. so ocaml did the "batteries included" stuff to try to help.) From ivan@REDACTED Wed Jul 9 23:23:03 2014 From: ivan@REDACTED (Ivan Uemlianin) Date: Wed, 9 Jul 2014 22:23:03 +0100 Subject: [erlang-questions] What factors correlate with programming language popularity/adoption? In-Reply-To: References: <9F2350ED-9B52-40E7-A01F-ED22566A2B38@llaisdy.com> Message-ID: I haven't found it to be a problem. When there's a choice, generally one lib will fit your use-case better than the others. Ivan -- festina lente > On 9 Jul 2014, at 21:57, Mark Allen wrote: > >> On 7/9/14 3:32 PM, "Ivan Uemlianin" wrote: >> >> Do you think erlang is lacking good libraries? > > Yes. > > Slightly more nuanced: It is very difficult to understand *what* libraries > to use even if they are available. > > > Mark > From tty.erlang@REDACTED Wed Jul 9 23:52:38 2014 From: tty.erlang@REDACTED (T Ty) Date: Wed, 9 Jul 2014 22:52:38 +0100 Subject: [erlang-questions] What factors correlate with programming language popularity/adoption? In-Reply-To: References: <9F2350ED-9B52-40E7-A01F-ED22566A2B38@llaisdy.com> Message-ID: In general I am comfortable using libraries from Mochi Games, Basho, Klarna, Heroku and Boundary. Anything that gets tack onto Yaws, Riak, Ejabberd, and RabbitMQ. And of course from well known individuals in the community, U Wiger, Klacke, S Vinoski, F Hebert, R Carlsson et al. That isn't to say all these libraries are complete, some assembly required. On Wed, Jul 9, 2014 at 9:57 PM, Mark Allen wrote: > On 7/9/14 3:32 PM, "Ivan Uemlianin" wrote: > > >Do you think erlang is lacking good libraries? > > Yes. > > Slightly more nuanced: It is very difficult to understand *what* libraries > to use even if they are available. > > > Mark > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From s.j.thompson@REDACTED Thu Jul 10 00:25:16 2014 From: s.j.thompson@REDACTED (Simon Thompson) Date: Wed, 9 Jul 2014 23:25:16 +0100 Subject: [erlang-questions] Erlang researcher position at the University of Kent Message-ID: We're looking for a researcher to work on 2 EU Erlang projects: PROWESS and RELEASE http://jobs.kent.ac.uk/fe/tpl_kent01.asp?s=4A515F4E5A565B1A&jobid=38625,6534548758&key=40778707&c=822348874065&pagestamp=sehvlluqknpwujmuwv happy to discuss further by email Simon Sent from my iPad -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Thu Jul 10 00:45:38 2014 From: zxq9@REDACTED (zxq9) Date: Thu, 10 Jul 2014 07:45:38 +0900 Subject: [erlang-questions] Microservices vs. Erlang processes In-Reply-To: <86D029F0-2D6D-47A2-9D51-6ADA4B466D46@writersglen.com> References: <298EAE28-4128-46BC-B981-154A128760F4@writersglen.com> <5599975.7mYOjNOmpj@jalapeno> <86D029F0-2D6D-47A2-9D51-6ADA4B466D46@writersglen.com> Message-ID: <2400287.vHaXOTKVIk@jalapeno> On Wednesday 09 July 2014 12:04:44 you wrote: > Do hope we can keep the line of thought going either on or off list. I'll take this off-list. Its a very Erlang-ish topic, but Erlang is not essential to it. I'm writing a response which is forcing me to clarify a few of my own intuitions on the subject, an opportunity for which I must thank you. -Craig From raould@REDACTED Thu Jul 10 03:04:03 2014 From: raould@REDACTED (Raoul Duke) Date: Wed, 9 Jul 2014 18:04:03 -0700 Subject: [erlang-questions] a question about pragmatic deadlock solutions Message-ID: https://github.com/laforge49/JActor2 "Unfortunately, this problem [deadlock among actors] rarely comes up in simple designs or initial implementations. It is only as the code matures and the complexity increases that deadlocks begin to occur. Supervisors detect such failed actors and restart them, which in turn gives rise to the increased chance that messages will be lost. So timeouts are often added, further increasing the complexity of the code and potentially giving rise to an increasing frequency of deadlocks." Do people see this in real world Erlang? thanks for any thoughts. From mononcqc@REDACTED Thu Jul 10 05:02:22 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 9 Jul 2014 23:02:22 -0400 Subject: [erlang-questions] a question about pragmatic deadlock solutions In-Reply-To: References: Message-ID: <20140710030221.GC794@ferdmbp.local> On 07/09, Raoul Duke wrote: > "Unfortunately, this problem [deadlock among actors] rarely comes up > in simple designs or initial implementations. It is only as the code > matures and the complexity increases that deadlocks begin to occur. > Supervisors detect such failed actors and restart them, which in turn > gives rise to the increased chance that messages will be lost. So > timeouts are often added, further increasing the complexity of the > code and potentially giving rise to an increasing frequency of > deadlocks." > > Do people see this in real world Erlang? > Yes and no. When designing a system, I try to avoid having processes requiring to exchange information directly in a one-on-one manner. This should really happen only in limited cases, when they represent peers with equal decisioning power talking to each other, for example. The most frequent pattern I encounter is often one of authority, where a process gets to either lead progress, or where one asks for information to another one. These have few chances of deadlocking there. Peer-to-peer cases (a chat system, a trading system, etc.) can often be transformed into an authoritative one by adding a third process that will act as a mediator and will lead things (a chat room, a marketplace). These will tend to move the information flow in a way that has a clear leader or coordinator. When they can't be, that's when fun begins, and yes, it's entirely possible to get deadlock situations if you're not careful. The example of a trading system FSM that can act peer-to-peer asynchronously is an example of the complexity that arises there. The copy in the book still has one of these in there (jlouis spotted it, but it happens so early in the protocol that fixing it would have made everything even less readable). In practice, I've seen these situations light years ahead of when they'd happen by just how frustrating design becomes. Defering to a third party to lead the dance is often much, much simpler. Regards, Fred. From shian5@REDACTED Thu Jul 10 03:21:50 2014 From: shian5@REDACTED (Shian Wu) Date: Thu, 10 Jul 2014 09:21:50 +0800 Subject: [erlang-questions] scheduler usage vs cpu usage In-Reply-To: <53BAC09C.2040706@erlang.org> References: <53BAC09C.2040706@erlang.org> Message-ID: Hi Thanks for your reply. How do I find out which os resource block vm? My iowait is also not so high and no network traffic. Thanks --- Shian Wu On Mon, Jul 7, 2014 at 11:45 PM, Bj?rn-Egil Dahlberg wrote: > On 2014-07-07 17:17, Shian Wu wrote: > > Hi > > I call recon:scheduler_usage(5000) when cpu usage is low (100% ~ 200% > from top, with 32 cores) > It show all scheduler usage is 1.0 > Is this possible? all schedulers are busy but cpu usage is low > > > Yes, it is possible. > > Schedulers waiting for os resources are considered utilized (they cannot > handle more work). > > This is why scheduler utilization exists. CPU utilization is often used as > a load measuring device but it is a very poor estimate load. > Scheduler utilization is a better estimate. > > // Bj?rn-Egil > > > thanks > --- > Shian Wu > > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdl.web@REDACTED Thu Jul 10 09:07:43 2014 From: sdl.web@REDACTED (Leo Liu) Date: Thu, 10 Jul 2014 15:07:43 +0800 Subject: [erlang-questions] What is Cookie in distribution message as in {2, Cookie, ToPid}? Message-ID: http://www.erlang.org/doc/apps/erts/erl_dist_protocol.html Mentions `cookie' in the handshake and also in the messages between connected nodes. Since cookie is not sent in clear text it is odd to see it used in: SEND {2, Cookie, ToPid} REG_SEND {6, FromPid, Cookie, ToName} Ideas? Also what are the differences between EXIT and EXIT2: EXIT {3, FromPid, ToPid, Reason} EXIT2 {8, FromPid, ToPid, Reason} How to instruct an Erlang node to log distribution messages it receives? Thanks, Leo From lukas@REDACTED Thu Jul 10 14:21:21 2014 From: lukas@REDACTED (Lukas Larsson) Date: Thu, 10 Jul 2014 14:21:21 +0200 Subject: [erlang-questions] What is Cookie in distribution message as in {2, Cookie, ToPid}? In-Reply-To: References: Message-ID: Hello, On Thu, Jul 10, 2014 at 9:07 AM, Leo Liu wrote: > http://www.erlang.org/doc/apps/erts/erl_dist_protocol.html Mentions > `cookie' in the handshake and also in the messages between connected > nodes. > > Since cookie is not sent in clear text it is odd to see it used in: > > SEND > > {2, Cookie, ToPid} > > REG_SEND > > {6, FromPid, Cookie, ToName} > The cookies in the messages are currently unused and should always be set to the atom ''. It is afaik there for the possibility to use cookie authentication in non-stream based protocols. > > Ideas? Also what are the differences between EXIT and EXIT2: > > EXIT > > {3, FromPid, ToPid, Reason} > > EXIT2 > > {8, FromPid, ToPid, Reason} EXIT is generated when an exit signal is sent because a process that was linked over distribution crashed. EXIT2 is send when the bif exit/2 is used on a remote pid. > How to instruct an Erlang node to log distribution messages it receives? > There are a bunch of dtrace/systemtap probes available that you can use to inspect the distribution messages. Also there is a define called ERTS_DIST_MSG_DBG in dist.c that you can set to make the code printout any messages that it receive. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From community-manager@REDACTED Thu Jul 10 15:15:32 2014 From: community-manager@REDACTED (Bruce Yinhe) Date: Thu, 10 Jul 2014 06:15:32 -0700 (PDT) Subject: [erlang-questions] Erlang researcher position at the University of Kent In-Reply-To: References: Message-ID: <53cc221b-2c86-49e4-b489-2edac850e3d9@googlegroups.com> Hi Simon, Did you know that you can advertise your job ad here on our official Erlang community site Erlang Central? It's easy to post and completely free. All applications are sent to your preferred inbox, OR redirected to a URL you prefer. You can go to the site and add the jobs yourself, or tell me so we'll take care it! erlangcentral.org/jobs Feel free to ping me or redirect me to the responsible person, we really look forward to your job posting on Erlang Central! Have a good day! Bruce -- Bruce Yinhe Erlang Community Manager +46 72 311 43 89 On Thursday, July 10, 2014 12:25:55 AM UTC+2, Simon Thompson wrote: > > We're looking for a researcher to work on 2 EU Erlang projects: PROWESS > and RELEASE > http://jobs.kent.ac.uk/fe/tpl_kent01.asp?s=4A515F4E5A565B1A&jobid=38625,6534548758&key=40778707&c=822348874065&pagestamp=sehvlluqknpwujmuwv > happy to discuss further by email > > > Simon > > Sent from my iPad > -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Thu Jul 10 15:50:07 2014 From: n.oxyde@REDACTED (Anthony Ramine) Date: Thu, 10 Jul 2014 15:50:07 +0200 Subject: [erlang-questions] Erlang researcher position at the University of Kent In-Reply-To: <53cc221b-2c86-49e4-b489-2edac850e3d9@googlegroups.com> References: <53cc221b-2c86-49e4-b489-2edac850e3d9@googlegroups.com> Message-ID: <803F1E88-3006-406B-B5C4-6D87E3689C14@gmail.com> That bit probably requires an explanation. Since when did it become an official website for the whole community? -- Anthony Ramine Le 10 juil. 2014 ? 15:15, Bruce Yinhe a ?crit : > Hi Simon, > > Did you know that you can advertise your job ad here on our official Erlang community site Erlang Central? It's easy to post and completely free. All applications are sent to your preferred inbox, OR redirected to a URL you prefer. > > You can go to the site and add the jobs yourself, or tell me so we'll take care it! > erlangcentral.org/jobs > > Feel free to ping me or redirect me to the responsible person, we really look forward to your job posting on Erlang Central! > > Have a good day! > Bruce > > -- > Bruce Yinhe > Erlang Community Manager > +46 72 311 43 89 > > > On Thursday, July 10, 2014 12:25:55 AM UTC+2, Simon Thompson wrote: > We're looking for a researcher to work on 2 EU Erlang projects: PROWESS and RELEASE http://jobs.kent.ac.uk/fe/tpl_kent01.asp?s=4A515F4E5A565B1A&jobid=38625,6534548758&key=40778707&c=822348874065&pagestamp=sehvlluqknpwujmuwv happy to discuss further by email > > Simon > > Sent from my iPad > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From sdl.web@REDACTED Thu Jul 10 19:41:58 2014 From: sdl.web@REDACTED (Leo Liu) Date: Fri, 11 Jul 2014 01:41:58 +0800 Subject: [erlang-questions] What is Cookie in distribution message as in {2, Cookie, ToPid}? In-Reply-To: (Lukas Larsson's message of "Thu, 10 Jul 2014 14:21:21 +0200") References: Message-ID: On 2014-07-10 14:21 +0200, Lukas Larsson wrote: > There are a bunch of dtrace/systemtap probes available that you can use to > inspect the distribution messages. Also there is a define called > ERTS_DIST_MSG_DBG in dist.c that you can set to make the code printout any > messages that it receive. Lukas, many thanks for answering all my questions. Leo From duncan@REDACTED Thu Jul 10 20:52:21 2014 From: duncan@REDACTED (Duncan McGreggor) Date: Thu, 10 Jul 2014 11:52:21 -0700 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3DC4.7000600@erlang-solutions.com> <01D5657C-8AEE-47AD-8C76-2BDD1F2A8C35@gmail.com> <53BC41E7.8070407@erlang-solutions.com> <53BC66BB.4030301@meetinghouse.net> Message-ID: Felix, thanks for bringing it back :-) I couldn't agree more -- simply having the opportunity to read -- in very clear, unambiguous text -- what is essentially a guide on OTP best practices and how to think about distributed apps/services, has been amazing. This is true not only for the things that I didn't know about OTP and its internals (which is a lot), but also for the things I had already managed to learn. You know it's a good job when you read something that causes you to think more deeply and richly about a topic with which you already have some intimate knowledge. Thanks Francesco and Steve! d P.S. The the $25 discount was deeply appreciated :-) On Tue, Jul 8, 2014 at 4:58 PM, Felix Gallo wrote: > On the topic of Francesco and Steve's book, as someone who bought the > early access version some time ago, it was at the time quite solid and > informative; I can only imagine it got even better from there; and I look > forward to catching up on it tonight. Although O'Reilly's sins are many > and egregious, supporting quality authors and helping to promote the > general availability of strong and accessible Erlang documentation is in > this case entirely worth the price. > > Of course, the true Erlang communitarian Nordic-style socialist bookshelf > would also include the work of the tireless Mr. Hebert ( > http://www.nostarch.com/erlang), the selfless Joe Armstrong ( > http://pragprog.com/book/jaerlang2/programming-erlang), and the triple > threat of Logan, Merritt and Carlsson (http://www.manning.com/logan/). > Why not buy five copies of each for your friends and neighbors? Christmas > is just around the corner. And what if one of your copies crashed without > trapping exit? > > F. > > > > > On Tue, Jul 8, 2014 at 4:15 PM, Mark Nijhof < > mark.nijhof@REDACTED> wrote: > >> I should have said "technical" writers :) but I see your point. But just >> looking at the time spend and what one could earn by doing that work vs >> writing about the work then it usually is a clear case what activity wins. >> >> >> On Tue, Jul 8, 2014 at 11:46 PM, Miles Fidelman < >> mfidelman@REDACTED> wrote: >> >>> Mark Nijhof wrote: >>> >>>> >>>> It is not funny anymore how many people think that someone writes a >>>> book for money. Even when self publishing there is not a lot of money to be >>>> made, I use LeanPub and am very happy with the platform/service they offer. >>>> And the royalties are good as well, but I need to pay for the cover design, >>>> the website (if you want something else then the default) and for editors >>>> to help me improve my content. So far it has cost me money. >>>> >>>> >>> That does require a comment. A lot of people DO write for money, though >>> not necessarily looking for all that money to come from book sales. >>> Academics "publish or perish." Consultants as a marketing vehicle. And >>> yes, there are people who actually make a living from writing books. >>> >>> But yes, there are also non-monetary reasons for writing a book - >>> consolidating learning, as an excuse to do some personal research, putting >>> experience or thoughts on paper, promoting a position, and so forth. Even >>> then, few have the luxury to do that without some means of support - we all >>> have to eat. If one is retired, has a job that includes writing as a part >>> of it (can you say academia?), an understanding employer (or an employer >>> who benefits from your writing), that's great. Otherwise, a grant, a >>> fellowship, or some form of income is required. >>> >>> Writing is a huge amount of work, and it is a shame writers don't see >>>> more of the results. >>>> >>> >>> Absolutely. Writing is real work. One of the reasons my next book will >>> be self-published. >>> >>> Cheers, >>> >>> >>> Miles Fidelman >>> >>> >>> -- >>> In theory, there is no difference between theory and practice. >>> In practice, there is. .... Yogi Berra >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> >> -- >> Mark Nijhof >> t: @MarkNijhof >> s: marknijhof >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From derekbrown121@REDACTED Thu Jul 10 20:29:42 2014 From: derekbrown121@REDACTED (Derek Brown) Date: Thu, 10 Jul 2014 14:29:42 -0400 Subject: [erlang-questions] dict vs. ETS choice Message-ID: Let's say you want to store on the order of tens of thousands key/value pairs, with integer keys and proplist values (less than 10 simple key/value pairs per proplist). Read-heavy with few writes. What factors would you consider when going with a dict or an ETS table, or perhaps something else? Derek -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Thu Jul 10 22:56:25 2014 From: zxq9@REDACTED (zxq9) Date: Fri, 11 Jul 2014 05:56:25 +0900 Subject: [erlang-questions] dict vs. ETS choice In-Reply-To: References: Message-ID: <2564755.Ugm5G3bPpx@jalapeno> On Thursday 10 July 2014 14:29:42 Derek Brown wrote: > Let's say you want to store on the order of tens of thousands key/value > pairs, with integer keys and proplist values (less than 10 simple key/value > pairs per proplist). Read-heavy with few writes. > > What factors would you consider when going with a dict or an ETS table, or > perhaps something else? Factors to consider? Whatever benchmarking both cases tells me. I would run each test case alongside a few different workloads on the same machine to figure out what good/bad things to expect from each approach. Someone will probably come along who actually has done this both ways and tell you there is indeed a Right Way, but usually I've found that taking the time to run some informal benchmarks provides more insight (which winds up useful elsewhere) than adherence to advice/data structure guidelines. Also, data structure advice often turns out to be a wordy version of "it depends..." and then a request for more information about the higher level problem you're actually trying to solve. What is the overall effect you're trying to achieve? It would be helpful to spell out the problem you're trying to solve (at the higher level like "how to best store 100k player's stats at once" instead of at the implementation detail of "is a dict or an ETS table better for 100k values?"). Very often you'll find someone has already tackled something similar and can give you some good advice you'd never get when asking just about the details of data structures. This prevents X->Y problem discussions. From pierrefenoll@REDACTED Thu Jul 10 23:26:38 2014 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Thu, 10 Jul 2014 23:26:38 +0200 Subject: [erlang-questions] dict vs. ETS choice In-Reply-To: <2564755.Ugm5G3bPpx@jalapeno> References: <2564755.Ugm5G3bPpx@jalapeno> Message-ID: Naturally, measure first. Implement both methods. Switching from one to the other is trivial anyway. For a hashtable store that is read-mostly I would suggest you use the process dictionary instead of ETS. I have measured a 100x speedup compared to ETS (details and implementation here [1]). Though you have to be careful with your writes as proc.dict. is a mutable datastructure. You might be interested in reading this [2]. [1]: https://bitbucket.org/fenollp/tmln-google/src [2]: http://ferd.ca/on-the-use-of-the-process-dictionary-in-erlang.html Cheers, -- Pierre Fenoll On 10 July 2014 22:56, zxq9 wrote: > On Thursday 10 July 2014 14:29:42 Derek Brown wrote: > > Let's say you want to store on the order of tens of thousands key/value > > pairs, with integer keys and proplist values (less than 10 simple > key/value > > pairs per proplist). Read-heavy with few writes. > > > > What factors would you consider when going with a dict or an ETS table, > or > > perhaps something else? > > Factors to consider? Whatever benchmarking both cases tells me. I would run > each test case alongside a few different workloads on the same machine to > figure out what good/bad things to expect from each approach. > > Someone will probably come along who actually has done this both ways and > tell > you there is indeed a Right Way, but usually I've found that taking the > time > to run some informal benchmarks provides more insight (which winds up > useful > elsewhere) than adherence to advice/data structure guidelines. Also, data > structure advice often turns out to be a wordy version of "it depends..." > and > then a request for more information about the higher level problem you're > actually trying to solve. > > What is the overall effect you're trying to achieve? It would be helpful to > spell out the problem you're trying to solve (at the higher level like > "how to > best store 100k player's stats at once" instead of at the implementation > detail of "is a dict or an ETS table better for 100k values?"). Very often > you'll find someone has already tackled something similar and can give you > some good advice you'd never get when asking just about the details of data > structures. This prevents X->Y problem discussions. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Jul 11 01:06:47 2014 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 11 Jul 2014 11:06:47 +1200 Subject: [erlang-questions] dict vs. ETS choice In-Reply-To: References: Message-ID: <8B2E441E-B921-4456-949E-A9A9FC76F19C@cs.otago.ac.nz> On 11/07/2014, at 6:29 AM, Derek Brown wrote: > Let's say you want to store on the order of tens of thousands key/value pairs, with integer keys and proplist values (less than 10 simple key/value pairs per proplist). Read-heavy with few writes. Key factors: (1) Scope : do the data need to be accessible from many processes or just one? (2) Persistence : can the data go away when some one process dies / when a node shuts down / when the disc is decommissioned / never if possible? (3) Volume. "10s of 1000s" could be a very small amount of data or an extremely large amount depending on the size of the values. This should be considered relative to available storage. (4) Traffic. "Ready-heavy" means how many reads per second? "Few" writes means how many writes per second? (5) Operations. What are _all_ the operations you want? (6) Unknown unknowns. From community-manager@REDACTED Fri Jul 11 09:30:15 2014 From: community-manager@REDACTED (Bruce Yinhe) Date: Fri, 11 Jul 2014 00:30:15 -0700 (PDT) Subject: [erlang-questions] Erlang researcher position at the University of Kent In-Reply-To: <803F1E88-3006-406B-B5C4-6D87E3689C14@gmail.com> References: <53cc221b-2c86-49e4-b489-2edac850e3d9@googlegroups.com> <803F1E88-3006-406B-B5C4-6D87E3689C14@gmail.com> Message-ID: <40b350f9-6779-4013-bda8-94a806afc190@googlegroups.com> Hi Anthony, My apologies that was a wrong choice of words. Erlang Central is a community site sponsored by companies using Erlang. It is one of the few places that is trying to be both a complete Erlang resource hub (incl. jobs) and a social hub for the community, filling the void created when trapexit was decomissioned. It is NOT the official Erlang website, but we do hope it can complement erlang.org and all of the other sites out there for Erlang related information. Erlang Central represents no one but the voice of the its sponsors, a group of companies using Erlang. Everyone is welcome to contribute and provide feedback and feature requests throught the forum or tutorials, howtos and articles through the wiki. Best regards, Bruce On Thursday, July 10, 2014 3:50:29 PM UTC+2, Anthony Ramine wrote: > > That bit probably requires an explanation. > > Since when did it become an official website for the whole community? > > -- > Anthony Ramine > > Le 10 juil. 2014 ? 15:15, Bruce Yinhe > a ?crit : > > > Hi Simon, > > > > Did you know that you can advertise your job ad here on our official > Erlang community site Erlang Central? It's easy to post and completely > free. All applications are sent to your preferred inbox, OR redirected to a > URL you prefer. > > > > You can go to the site and add the jobs yourself, or tell me so we'll > take care it! > > erlangcentral.org/jobs > > > > Feel free to ping me or redirect me to the responsible person, we really > look forward to your job posting on Erlang Central! > > > > Have a good day! > > Bruce > > > > -- > > Bruce Yinhe > > Erlang Community Manager > > +46 72 311 43 89 > > > > > > On Thursday, July 10, 2014 12:25:55 AM UTC+2, Simon Thompson wrote: > > We're looking for a researcher to work on 2 EU Erlang projects: PROWESS > and RELEASE > http://jobs.kent.ac.uk/fe/tpl_kent01.asp?s=4A515F4E5A565B1A&jobid=38625,6534548758&key=40778707&c=822348874065&pagestamp=sehvlluqknpwujmuwv > happy to discuss further by email > > > > Simon > > > > Sent from my iPad > > _______________________________________________ > > erlang-questions mailing list > > erlang-q...@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Fri Jul 11 10:54:51 2014 From: zxq9@REDACTED (zxq9) Date: Fri, 11 Jul 2014 17:54:51 +0900 Subject: [erlang-questions] Modernizing code Message-ID: <1922529.Go0BOQZynY@jalapeno> After the (somewhat incessant) discussions about documentation and code examples over the last month, I've been checking over the examples that are available at erlang.org. I see hints (almost warnings) on the link pages that the code examples are "somewhat out of date, but...". I'm trying to figure out what about these small examples is in fact out of date -- mostly to educate myself, but incidentally if my inquiry could help modernize the examples that might be a good thing for us anyway. Taking getty.erl (http://www.erlang.org/article/7) as an example, other than the deliberate decision to not use any OTP behaviors, what about it is so out of date? I'm blind to seeing how that should be written differently for R17. Also, how would a gen_server or gen_fsm version of this need to look to be "best practices", "modern" R17 code? How would that differ from "best practices" R14 code? (Short of becoming a non-trivial example, I mean. If doing things "right" would mean destroying the code's utility as a learning example, then no; OTOH, what does that imply about writing OTP example code as opposed to writing just Erlang example code?) -Craig From zkessin@REDACTED Fri Jul 11 12:32:37 2014 From: zkessin@REDACTED (Zachary Kessin) Date: Fri, 11 Jul 2014 13:32:37 +0300 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <5DFC2AF5-8AEC-4520-85A3-CF06ADEF7605@gmail.com> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <5DFC2AF5-8AEC-4520-85A3-CF06ADEF7605@gmail.com> Message-ID: <53BFBD45.2000403@gmail.com> It generally a percentage of the sale price (more for the ebook actually). That being said no one gets rich writing books on Erlang. I just got my most recent royalty check for my book and well, I could buy a car with it (and by that I mean a toy car for my kids!) That being said if you buy the early release you will also get the updates and the final book --Zach On 7/8/14, 9:41 PM, Lee Sylvester wrote: > Well that?s the thing. I?d hazard a guess (and could be wrong) that Francesco?s and Steve?s contract offers them $1 for every printed book sold and 50 cents for every ebook. > > Was I close? ;-) > > Lee > > > > On 8 Jul 2014, at 19:39, Raoul Duke wrote: > >> i haven't looked at the books here, but having been somebody who >> produced things and wondered how anybody could ever make a living at >> it, and knowing that writers throughout history rarely made much after >> the publisher etc. got a cut, i didn't have the same reaction to the >> $50 price tag. can't say i'd *spend* the $50 since i'm broke and >> there's a zillion other things to spend $50 on first. i'd try to get >> it via inter library loan or something :-). >> >> On Tue, Jul 8, 2014 at 11:36 AM, Francesco Cesarini >> wrote: >>> Alas, that is how O'Reilly price their books. Having a high price and then >>> discounting is not the approach I would pick. That is why discount codes are >>> being handed out on public mailing lists and social media. I recommend you >>> use them (Read, no one pays full price for an O'Reilly book). As an unedited >>> book, the cost is for the final book which we hope will complement what is >>> already out there. It is a different approach to OTP in action. One I've >>> been using for 15 years when teaching OTP. >>> >>> /F >>> >>> On 08/07/2014 19:14, Lee Sylvester wrote: >>>> Wow, $50? Yeah, that is steep. If this were some black arts compiler >>>> book or video encoding bible, then that would be something else. But a book >>>> on Erlang/OTP just doesn?t fall into that bracket? >>>> >>>> I may wait til it falls in the bargain bucket ;-) >>>> >>>> Lee >>>> >>>> On 8 Jul 2014, at 19:08, Miles Fidelman >>>> wrote: >>>> >>>>> On 8 Jul 2014, at 16:05, Francesco Cesarini >>>>> wrote: >>>>>>> Hi all, >>>>>>> >>>>>>> a shameless plug. Steve Vinoski and I are working on a book focused on >>>>>>> distributed, scalable systems with OTP. It is available from O'Reilly as an >>>>>>> Early Release from their website. The first eight chapters we released cover >>>>>>> (in great detail) all other behaviours. We started writing, and before we >>>>>>> knew it, we had several hundred pages on behaviours alone. Last week, two >>>>>>> new chapters, including the Introduction & Special processes and >>>>>>> Implementing your own behaviours were released. We are now focusing on >>>>>>> release handling (hgg), code upgrade and architectural patterns. >>>>>>> >>>>>>> What is available is an unedited draft, with new chapters and >>>>>>> improvements published as they become available. You can find more info >>>>>>> here: >>>>>>> >>>>>>> >>>>>>> http://shop.oreilly.com/product/0636920024149.do?intcmp=il-prog-books-videos-product-intsrch_erlang_ct >>>>>>> >>>>>>> If you use discount code authd, you will get 50% off the Early Release, >>>>>>> and 40% on pre-orders of the the printed copy. >>>>>>> >>>>>>> Looking forward to your feedback, >>>>>>> >>>>> Not for nothing, but $50 for the undedited work-in-progress seems a bit >>>>> steep (and yes, I saw the discount code). I also note that "rough cuts" >>>>> aren't included in my Safari subscription. >>>>> >>>>> Usual practice that I've seen is for works-in-progress to be free, with >>>>> tools that support comments from early readers. >>>>> >>>>> Sounds interesting - but, from the TOC (all that's available without >>>>> paying), it sure looks like it covers the same ground as "Erlang and OTP in >>>>> Action." >>>>> >>>>> Miles Fidelman >>>>> >>>>> -- >>>>> In theory, there is no difference between theory and practice. >>>>> In practice, there is. .... Yogi Berra >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> -- >>> Erlang Solutions Ltd. >>> http://www.erlang-solutions.com >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Zachary Kessin Mostly Erlang Podcast Skype: zachkessin Twitter: @zkessin -------------- next part -------------- An HTML attachment was scrubbed... URL: From zkessin@REDACTED Fri Jul 11 12:37:22 2014 From: zkessin@REDACTED (Zachary Kessin) Date: Fri, 11 Jul 2014 13:37:22 +0300 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3DC4.7000600@erlang-solutions.com> <01D5657C-8AEE-47AD-8C76-2BDD1F2A8C35@gmail.com> <53BC3F72.6060304@meetinghouse.net> Message-ID: <53BFBE62.1030802@gmail.com> > What about the money for Simon St. Laurent? - Let's assume that you > thought 25$ was a fair amount to pay for Francesco and Simmons first > Erlang book - would you pay equal amounts to Francesco and Simon. > > But wait a moment their editor who I think was Simon St Laurent > will have made significant contributions to the book - should his > contribution > be unpaid and unrecognised? > > Simon did not edit that one, but he did edit my Erlang book, as well as writing one on his own (and editing a bunch of other FP Books). It took many hours of work from him and others at O'Reilly to do this, this is what they are for. Also I did not get an advance on that book, so I got nothing until it started to sell. Yes If I had self published I would have gotten a bigger percent, but really there are a million details from finding an editor to cover art and distribution that I really did not want to deal with --Zach -- Zachary Kessin Mostly Erlang Podcast Skype: zachkessin Twitter: @zkessin -------------- next part -------------- An HTML attachment was scrubbed... URL: From zkessin@REDACTED Fri Jul 11 12:41:22 2014 From: zkessin@REDACTED (Zachary Kessin) Date: Fri, 11 Jul 2014 13:41:22 +0300 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> Message-ID: <53BFBF52.6080806@gmail.com> O'Reilly is happy to let you Open Source your Book btw. Michael Snoyman's book on Yesod and Haskell is open source and published via O'Reilly PS the "AUTHD" discount code will work on my book too --Zach -- Zachary Kessin Mostly Erlang Podcast Skype: zachkessin Twitter: @zkessin -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Fri Jul 11 14:26:23 2014 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Fri, 11 Jul 2014 07:26:23 -0500 Subject: [erlang-questions] dict vs. ETS choice In-Reply-To: References: Message-ID: Large dicts complicate debugging because the whole thing gets printed/logged when you have a crash or want to do some tracing. For this reason alone I would lean towards ETS instead of dict when the use-case allows it. Perhaps there are ways to mitigate the huge-printed-dict problem. I've never taken a close look at it. On Thu, Jul 10, 2014 at 1:29 PM, Derek Brown wrote: > Let's say you want to store on the order of tens of thousands key/value > pairs, with integer keys and proplist values (less than 10 simple key/value > pairs per proplist). Read-heavy with few writes. > > What factors would you consider when going with a dict or an ETS table, or > perhaps something else? > > Derek > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Fri Jul 11 14:36:08 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 11 Jul 2014 14:36:08 +0200 Subject: [erlang-questions] dict vs. ETS choice In-Reply-To: <8B2E441E-B921-4456-949E-A9A9FC76F19C@cs.otago.ac.nz> References: <8B2E441E-B921-4456-949E-A9A9FC76F19C@cs.otago.ac.nz> Message-ID: On Fri, Jul 11, 2014 at 1:06 AM, Richard A. O'Keefe wrote: > (2) Persistence : can the data go away when some one process > dies / when a node shuts down / when the disc is decommissioned > / never if possible? > Persistence, the other way: Do you need the ability to "version" your data and go back to an earlier version of the data, or can you live with ephemeral data (i.e., where data gets destructively updated) -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Fri Jul 11 14:52:42 2014 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 11 Jul 2014 14:52:42 +0200 Subject: [erlang-questions] dict vs. ETS choice In-Reply-To: References: Message-ID: <74CF737B-9B36-4489-AD6F-9FCC8D2D10B2@feuerlabs.com> On 11 Jul 2014, at 14:26, Daniel Goertzen wrote: > Perhaps there are ways to mitigate the huge-printed-dict problem. I've never taken a close look at it. Even small dicts tend to look huge when printed. ;-) I prototyped a slight improvement for dict (et al) printing in the shell, in: http://ulf.wiger.net/weblog/2007/11/20/extending-the-erlang-shell-part-1/ Eshell V5.5.4 (abort with ^G) 1> rr(code:which(dict)). [dict] 2> fa(dict,fun(#dict{}=D) -> {custom,dict,true,dict:dict_to_list(D)}; (_) -> no end). true 3> dict:from_list([{N,a} || N <- lists:seq(1,5)]). <|dict:[{3,a},{2,a},{5,a},{1,a},{4,a}]|> That particular code is of course outdated now, since it was a patch on OTP sources from 2007. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From lee.sylvester@REDACTED Fri Jul 11 15:52:26 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Fri, 11 Jul 2014 14:52:26 +0100 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <53BFBE62.1030802@gmail.com> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3DC4.7000600@erlang-solutions.com> <01D5657C-8AEE-47AD-8C76-2BDD1F2A8C35@gmail.com> <53BC3F72.6060304@meetinghouse.net> <53BFBE62.1030802@gmail.com> Message-ID: <969AC6A8-7D38-4861-9990-A0FE76A40EAB@gmail.com> Okay, I bought the book too :-) Thanks Francesco. Lee On 11 Jul 2014, at 11:37, Zachary Kessin wrote: > >> What about the money for Simon St. Laurent? - Let's assume that you >> thought 25$ was a fair amount to pay for Francesco and Simmons first >> Erlang book - would you pay equal amounts to Francesco and Simon. >> >> But wait a moment their editor who I think was Simon St Laurent >> will have made significant contributions to the book - should his contribution >> be unpaid and unrecognised? >> >> > Simon did not edit that one, but he did edit my Erlang book, as well as writing one on his own (and editing a bunch of other FP Books). It took many hours of work from him and others at O'Reilly to do this, this is what they are for. > > Also I did not get an advance on that book, so I got nothing until it started to sell. Yes If I had self published I would have gotten a bigger percent, but really there are a million details from finding an editor to cover art and distribution that I really did not want to deal with > > --Zach > > -- > Zachary Kessin > Mostly Erlang Podcast > Skype: zachkessin > Twitter: @zkessin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Fri Jul 11 16:08:07 2014 From: comptekki@REDACTED (Wes James) Date: Fri, 11 Jul 2014 08:08:07 -0600 Subject: [erlang-questions] dict vs. ETS choice In-Reply-To: References: Message-ID: Have you heard of or looked at RIAK? http://basho.com/riak/ -wes On Thu, Jul 10, 2014 at 12:29 PM, Derek Brown wrote: > Let's say you want to store on the order of tens of thousands key/value > pairs, with integer keys and proplist values (less than 10 simple key/value > pairs per proplist). Read-heavy with few writes. > > What factors would you consider when going with a dict or an ETS table, or > perhaps something else? > > Derek > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Fri Jul 11 16:28:55 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 11 Jul 2014 10:28:55 -0400 Subject: [erlang-questions] dict vs. ETS choice In-Reply-To: References: Message-ID: <20140711142854.GA89097@ferdair.local> On 07/11, Wes James wrote: > Have you heard of or looked at RIAK? > > http://basho.com/riak/ > > -wes > Well that sounds a lot like bringing a tank to a dodgeball game. Let's say for example that you're going to store 10 million keys to make the 10s of thousands of value small in comparison. These keys are mostly read-only so likely easy to just have local on each node. Assuming 10 simple key-value pairs per proplists, you may say that you're gonna have at most 10kb per row in the table as a very conservative estimate. At 10 million rows of 10kb, you're gonna have roughly 95GB of data to store. That's the kind of stuff that easily sits on a regular laptop and can be made to work fine with any good old regular SQL storage, or most single-node DBs that are not DETS. If you go down and figure out you may average out 1kb of data per row, you can now fit the entire thing in memory on some servers or virtual instances. Have 100,000 keys instead of 10 millions, and you now fit everything under 100MB of RAM. You don't really need to go look at Riak when the question, for all reasonable purposes, likely doesn't even need distribution to begin with. I'd personally look at what Richard O'Keefe and Jesper mentioned as selection criteria at this question. A note about the heavy size of log messages for dicts and other data structures on failures: check out the 'format_status' callback offered by gen_server: http://www.erlang.org/doc/man/gen_server.html#Module:format_status-2 Module:format_status(Opt, [PDict, State]) -> Status Opt = normal | terminate PDict = [{Key, Value}] This callback is optional, so callback modules need not export it. The gen_server module provides a default implementation of this function that returns the callback module state. This function is called by a gen_server process when: - One of sys:get_status/1,2 is invoked to get the gen_server status. Opt is set to the atom normal for this case. - The gen_server terminates abnormally and logs an error. Opt is set to the atom terminate for this case. This function is useful for customising the form and appearance of the gen_server status for these cases. A callback module wishing to customise the sys:get_status/1,2 return value as well as how its status appears in termination error logs exports an instance of format_status/2 that returns a term describing the current status of the gen_server. [...] One use for this function is to return compact alternative state representations to avoid having large state terms printed in logfiles. From roger@REDACTED Fri Jul 11 16:37:57 2014 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 11 Jul 2014 15:37:57 +0100 Subject: [erlang-questions] badarg when registering httpd process? Message-ID: I'm attempting to register an httpd process with a name (mainly so that I can figure out the dynamically-assigned port number later). I get badarg when I call register, as follows: Erlang R16B03-1 (erts-5.10.4) [source] [64-bit] [smp:12:12] [async-threads:10] [kernel-poll:false] Eshell V5.10.4 (abort with ^G) 1> application:start(inets). ok 2> {ok, Pid} = inets:start(httpd, [{port, 0}, {server_name, "localhost"}, {server_root, "/tmp"}, {document_root, "/tmp"}]). {ok,<0.48.0>} 3> register(my_httpd, Pid). ** exception error: bad argument in function register/2 called as register(my_httpd,<0.48.0>) ...and yet, register works fine otherwise: 4> register(my_httpd, self()). true Is there something special about httpd processes that means that they can't be registered? Cheers, Roger. From mononcqc@REDACTED Fri Jul 11 16:44:44 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 11 Jul 2014 10:44:44 -0400 Subject: [erlang-questions] badarg when registering httpd process? In-Reply-To: References: Message-ID: <20140711144442.GB89097@ferdair.local> On 07/11, Roger Lipscombe wrote: > I'm attempting to register an httpd process with a name (mainly so > that I can figure out the dynamically-assigned port number later). > > I get badarg when I call register, as follows: > > [snip] > > Is there something special about httpd processes that means that they > can't be registered? > A process can't be registered twice: 1> application:ensure_all_started(inets). {ok,[inets]} 2> {ok, Pid} = inets:start(httpd, [{port, 0}, {server_name, 2> "localhost"}, {server_root, "/tmp"}, {document_root, "/tmp"}]). {ok,<0.67.0>} 3> process_info(Pid, registered_name). {registered_name,httpd_instance_sup_54506} It already has a name and therefore you can't register it a second time. Regards, Fred. From roger@REDACTED Fri Jul 11 18:18:56 2014 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 11 Jul 2014 17:18:56 +0100 Subject: [erlang-questions] badarg when registering httpd process? In-Reply-To: <20140711144442.GB89097@ferdair.local> References: <20140711144442.GB89097@ferdair.local> Message-ID: Ah. I knew that I couldn't have two processes with the same name. Didn't realise that it was constrained the other way as well. Thanks. On 11 July 2014 15:44, Fred Hebert wrote: > On 07/11, Roger Lipscombe wrote: >> I'm attempting to register an httpd process with a name (mainly so >> that I can figure out the dynamically-assigned port number later). >> >> I get badarg when I call register, as follows: >> >> [snip] >> >> Is there something special about httpd processes that means that they >> can't be registered? >> > > A process can't be registered twice: > > 1> application:ensure_all_started(inets). > {ok,[inets]} > 2> {ok, Pid} = inets:start(httpd, [{port, 0}, {server_name, > 2> "localhost"}, {server_root, "/tmp"}, {document_root, "/tmp"}]). > {ok,<0.67.0>} > 3> process_info(Pid, registered_name). > {registered_name,httpd_instance_sup_54506} > > It already has a name and therefore you can't register it a second time. > > Regards, > Fred. From kunthar@REDACTED Sun Jul 13 01:49:13 2014 From: kunthar@REDACTED (Gokhan Boranalp) Date: Sun, 13 Jul 2014 02:49:13 +0300 Subject: [erlang-questions] ANN: Designing for Scalability with Erlang/OTP by O'Reilly In-Reply-To: <969AC6A8-7D38-4861-9990-A0FE76A40EAB@gmail.com> References: <53BC08AB.1030605@erlang-solutions.com> <53BC33A3.4000502@meetinghouse.net> <53BC3A25.8030708@erlang-solutions.com> <53BC3DC4.7000600@erlang-solutions.com> <01D5657C-8AEE-47AD-8C76-2BDD1F2A8C35@gmail.com> <53BC3F72.6060304@meetinghouse.net> <53BFBE62.1030802@gmail.com> <969AC6A8-7D38-4861-9990-A0FE76A40EAB@gmail.com> Message-ID: as a reader my 2 cents. this is still a small community. i have to give a hand whoever brings detailed and useful information to our shortdated lives. i salute you the team of this brave book. i will buy a copy right now -- period On 11 Jul 2014 16:52, "Lee Sylvester" wrote: > Okay, I bought the book too :-) > > Thanks Francesco. > > Lee > > > On 11 Jul 2014, at 11:37, Zachary Kessin wrote: > > > What about the money for Simon St. Laurent? - Let's assume that you > thought 25$ was a fair amount to pay for Francesco and Simmons first > Erlang book - would you pay equal amounts to Francesco and Simon. > > But wait a moment their editor who I think was Simon St Laurent > will have made significant contributions to the book - should his > contribution > be unpaid and unrecognised? > > > Simon did not edit that one, but he did edit my Erlang book, as well as > writing one on his own (and editing a bunch of other FP Books). It took > many hours of work from him and others at O'Reilly to do this, this is what > they are for. > > Also I did not get an advance on that book, so I got nothing until it > started to sell. Yes If I had self published I would have gotten a bigger > percent, but really there are a million details from finding an editor to > cover art and distribution that I really did not want to deal with > > --Zach > > -- > Zachary Kessin > Mostly Erlang Podcast > Skype: zachkessin > Twitter: @zkessin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zkessin@REDACTED Sun Jul 13 10:37:09 2014 From: zkessin@REDACTED (Zachary Kessin) Date: Sun, 13 Jul 2014 11:37:09 +0300 Subject: [erlang-questions] Projects for a Hackathon Message-ID: <53C24535.2000203@gmail.com> It appears that Erlang Factory light Beer Sheva is going to be pretty small, so we are going to turn it into a hackathon. I am hoping that we can get some Erlang (and other FP) pros and some students together and have a good day making new contacts and such. In light of Garretts talk at EUC and much of the conversation about how we could make the erlang onramp a bit easier I thought it might be a good idea to come up with some project ideas to offer at the hackathon. Please note these should be something that 3-4 students should be able to get a good start on in 1 day. -- Zachary Kessin Mostly Erlang Podcast Skype: zachkessin Twitter: @zkessin -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean@REDACTED Sun Jul 13 19:34:04 2014 From: sean@REDACTED (Sean Cribbs) Date: Sun, 13 Jul 2014 12:34:04 -0500 Subject: [erlang-questions] Projects for a Hackathon In-Reply-To: <53C24535.2000203@gmail.com> References: <53C24535.2000203@gmail.com> Message-ID: Hi Zachary, One thing we did a number of years ago at the Triangle Erlang meetup was to make a "game of telephone": Everyone starts a distributed Erlang node and connects to someone next to them. They each write a registered process that takes a message, prints it to the console, adds their node name to it, and passes it to the next person. When the originator receives its own message, it prints it out and doesn't forward. I think it's nice exercise because it teaches about the basics of message passing between processes, constructing protocols, and standing up clusters of Erlang nodes. One thing that was fun (and instructive) at that meetup is that one participant changed his process to forward a different data structure, causing the chain to be disrupted and some people's processes to crash. One could extend the exercise to encompass more messaging patterns like scatter-gather, etc (see the thread from this spring for ideas). On Sun, Jul 13, 2014 at 3:37 AM, Zachary Kessin wrote: > It appears that Erlang Factory light Beer Sheva is going to be pretty > small, so we are going to turn it into a hackathon. I am hoping that we can > get some Erlang (and other FP) pros and some students together and have a > good day making new contacts and such. > > In light of Garretts talk at EUC and much of the conversation about how we > could make the erlang onramp a bit easier I thought it might be a good idea > to come up with some project ideas to offer at the hackathon. > > Please note these should be something that 3-4 students should be able to > get a good start on in 1 day. > > -- > Zachary Kessin > Mostly Erlang Podcast > Skype: zachkessin > Twitter: @zkessin > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Sean Cribbs Software Engineer Basho Technologies, Inc. http://basho.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From yoursurrogategod@REDACTED Mon Jul 14 01:59:36 2014 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Sun, 13 Jul 2014 19:59:36 -0400 Subject: [erlang-questions] Question about behaviours in Erlang/OTP Message-ID: Hello, I'm looking at some code that was written where a user uses a behaviour of gen_server and then uses a process_flag(trap_exit, true). To me, this seems like a supervisor behaviour would work better. What would be a better approach in this case? To declare this module a supervisor? Is it possible (sensible? logical?) to give the same module gen_server and supervisor behaviours? Thanks in advance! -------------- next part -------------- An HTML attachment was scrubbed... URL: From freza@REDACTED Mon Jul 14 03:58:21 2014 From: freza@REDACTED (Jachym Holecek) Date: Sun, 13 Jul 2014 21:58:21 -0400 Subject: [erlang-questions] Question about behaviours in Erlang/OTP In-Reply-To: References: Message-ID: <20140714015821.GA9823@circlewave.net> # Yves S. Garret 2014-07-13: > I'm looking at some code that was written where a user uses a behaviour of > gen_server and then uses a process_flag(trap_exit, true). Nothing wrong with that, they probably just need detailed control over child process' faults. But you don't say what the code is really up to. > To me, this seems like a supervisor behaviour would work better. Supervisors work well at controlling the relatively small & long-lived "spine" of a complex control system. The lower you go the more subleties emerge, thus the more low-level tools you need to tame them; generally speaking. There's only so far pre-baked patterns will go, and that's fine as they were never meant to go much further. > What would be a better approach in this case? To understand the functional requirements at hand and write the simplest possible code that complies with them. You're allowed to cheat on both accounts as long as the customer couldn't possibly tell in live operation. > To declare this module a supervisor? What good would a supervisor do in a case where you have one process spawning jobs supplied on request by a subordinate worker process, whilst limiting job issue rate against configured maximum, whilst engaging in exponential backoff of said activity when job fault rate breaches other configured threshold, whilst merely temporarily slowing down job issue rate if the faults are of certain kinds; all the time making sure the same logical job isn't active more than once at a time? That was a real use case BTW, a simplified form thereof. And that is why you have all those low-level tools available (like trapping exits, various ways of spawning processes, monitors etc etc). Use them. ;-) > Is it possible (sensible? logical?) to give the same module gen_server > and supervisor behaviours? One module can adhere to multiple behaviors in principle, though it is only very rarely sensible, and this particular combination certainly doesn't look like one of those cases (it probably wouldn't even work, considering 'supervisor' already is a 'gen_server' if memory serves). BR, -- Jachym PS: Oh no, now hipster kids will be all over me. Whoopsy daisy. :-) From zxq9@REDACTED Mon Jul 14 03:48:07 2014 From: zxq9@REDACTED (zxq9) Date: Mon, 14 Jul 2014 10:48:07 +0900 Subject: [erlang-questions] Question about behaviours in Erlang/OTP In-Reply-To: References: Message-ID: <1894008.GfzEXuoA7X@jalapeno> On Sunday 13 July 2014 19:59:36 Yves S. Garret wrote: > Hello, > > I'm looking at some code that was written where a user uses a behaviour of > gen_server and then uses a process_flag(trap_exit, true). To me, this > seems like a supervisor behaviour would work better. > > What would be a better approach in this case? To declare this module a > supervisor? Is it possible (sensible? logical?) to give the same module > gen_server and supervisor behaviours? I'm certainly no expert at designing supervisor trees, but the thing I usually see is more layers of supervision, each with a distinct type of responsibility (hence "tree" and not master/slave). So a top-level supervisor that can trap exits (and might do other top-level stuff), which supervises another supervisor which monitors whatever work is going on, and working processes like gen_servers at the bottom. This can get pretty wide and deep depending on the application. In other words, a layer of supervision for each conceptual level of supervisory tasks. Its a bit more indirection/layering than you might expect coming from another language, but this seems to be the way OTP expects systems to be built. OTOH, if the program is not a permanent service -- like a program designed to be run from a command line, do its job, and terminate when it completes -- then it doesn't really matter. In this case even using gen_server might have been overkill compared to writing the code in a more direct manner. The whole issue of supervision hinges on the nature of the program, and not every program benefits from this, as many programs are not long-running services. From rtrlists@REDACTED Mon Jul 14 08:31:04 2014 From: rtrlists@REDACTED (Robert Raschke) Date: Mon, 14 Jul 2014 07:31:04 +0100 Subject: [erlang-questions] Question about behaviours in Erlang/OTP In-Reply-To: References: Message-ID: Hi Yves, one classic scenario where you have a gen_server that needs to trap exits is a server that wraps around a port program. If your server starts an external program that it makes available as resource to Erlang, then it will want to know about crashes explicitly, since normal supervision cannot easily deal with restarting something external. Robby On Jul 14, 2014 12:59 AM, "Yves S. Garret" wrote: > Hello, > > I'm looking at some code that was written where a user uses a behaviour of > gen_server and then uses a process_flag(trap_exit, true). To me, this > seems like a supervisor behaviour would work better. > > What would be a better approach in this case? To declare this module a > supervisor? Is it possible (sensible? logical?) to give the same module > gen_server and supervisor behaviours? > > Thanks in advance! > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qrilka@REDACTED Mon Jul 14 08:40:56 2014 From: qrilka@REDACTED (Kirill Zaborsky) Date: Sun, 13 Jul 2014 23:40:56 -0700 (PDT) Subject: [erlang-questions] Data structures are strange things In-Reply-To: <9b6fc3a1-ff75-45b3-b182-835cc89d495e@knuth> References: <4a0b389f-5cb3-4768-8eba-acf95d03cce3@knuth> <9b6fc3a1-ff75-45b3-b182-835cc89d495e@knuth> Message-ID: <8ea865f7-17df-4254-ba3c-732c0cefd074@googlegroups.com> Hi Robert, I have came across this a bit old topic and I wonder - did you publish any of those modules? I have only found your repo https://github.com/rvirding/rb which is even 3 years older than this thread :) Regards, Kirill ???????, 8 ???? 2012 ?., 4:55:29 UTC+4 ???????????? Robert Virding ???????: > > I was finally going to release some alternate modules with the dict > interface using tree structures, either AA trees or Red Black trees. These > are relatively well-known binary tree structures which are binary tree > versions of 2-3 trees and 2-3-4 trees respectively. These contain codes > with either 2, 3 or 4 sub-nodes. Generally the binary versions are > recommended as being better/easier/more space efficient. > > Before I released my modules I thought it would be interesting to try the > non-binary versions to see how they behaved and what it was lie to program > them. I was lucky to find a very good description of 2-3 trees and the > rules for inserting/deleting elements; it was literally just translate the > rules straight to Erlang and it worked. The code was a little more complex > than working with the binary versions, not much. In some cases actually > simpler. The I decided to measure speed, and the fun started. > > All tests were done with sequences of random numbers as keys, the same for > each implementation. > > What I found was that the lookup times were about the same as for the > binary versions, which was expected, but the insertion/deletion times were > about 20% faster. This compared to my AA and RB tree versions. To check > that it wasn't just bad coding on my part (not bloody likely :-)) I > compared it with gb_trees and got the similar results. Also checking the > data sizes I found that the 2-3 and 2-3-4 trees actually used *less* space > than their binary counterparts. > > Here are the figures. I have also included dict, which is the slowest on > insertion/deletion but significantly faster lookup times. All times are in > usecs for the whole set: > > Operation Load Fetch Load Fetch > # of elements 100000 10000 > Overhead time 43000 3700 > > ttdict 2-3 trees 310000 161000 24000 15500 > ttfdict 2-3-4 trees 312000 173000 24200 16500 > llrbdict Left Leaning RB trees 390000 164000 27100 14900 > > rbdict Red Black trees 380000 167000 27100 15000 > aadict AA trees 423000 167000 30000 15000 > > gb_trees using faster insert/3 370000 190000 26500 16200 > dict 522000 107000 28000 11000 > > > (I hope the table is readable) > Gb_trees are a variant of AA trees with a slightly different algorithm. > One should be careful interpreting exact values here but the trend is clear. > > I don't know why the straight multi-size versions are faster but my guess > is that the pattern-matching/data construction is faster on the flat bigger > nodes rather than on the nested pairs. My guess is that when they comment > on the 2-3(-4) trees using more space it is because they use the same size > node for all cases which in the cases with fewer sub-nodes wastes space, > while just used different sized tuples which didn't waste space; it also > made the pattern matching much easier. > > It looks like I will be submitting the 2-3 tree version, ttdict, as > reasonable choice. Next I will try AVL trees which were discarded long ago > and see how they fare. > > Robert > > P.S. This is the reason why there was hoped to be many different dict > modules with the same API but different data structures. So as it to make > it easier to choose the most suitable version for each occasion. There is > no generally "best" choice, even among trees. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Mon Jul 14 12:49:29 2014 From: ivan@REDACTED (Ivan Uemlianin) Date: Mon, 14 Jul 2014 11:49:29 +0100 Subject: [erlang-questions] reltool/rebar vs. ssl certificates In-Reply-To: References: Message-ID: <53C3B5B9.6020804@llaisdy.com> Dear All (esp. Felix Gallo) I am seeing almost the same strange problem with ssl certificates as this post from last April. To recap: a rebar-packaged release with ssl running gets a "certificate unknown" error when connecting over https: > httpc:request(get, {"https://www.example.org/etc/", []}, [], []). {error,{failed_connect, [{to_address,{"www.example.org",443}}, {inet,[inet],{tls_alert,"certificate unknown"}}]}} The same code running with my local erl will make the above request successfully. Different to Felix's situation: running the code using the release's erl will make the request successfully too. In the original thread, Ingela Andin recommened [1] a specific commit to OTP R16. @Felix did that work? Or did you otherwise fix the problem? Here, the same erl binary will make the https call successfully on its own, but not if running the release. Could the package configs (eg reltool.config) be leaving out something that should be included? Any pointers appreciated. With thanks and best wishes Ivan [1] http://erlang.org/pipermail/erlang-questions/2013-April/073238.html -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From ivan@REDACTED Mon Jul 14 12:59:28 2014 From: ivan@REDACTED (Ivan Uemlianin) Date: Mon, 14 Jul 2014 11:59:28 +0100 Subject: [erlang-questions] reltool/rebar vs. ssl certificates In-Reply-To: <53C3B5B9.6020804@llaisdy.com> References: <53C3B5B9.6020804@llaisdy.com> Message-ID: <53C3B810.9050701@llaisdy.com> Dear All Might have been a reltool.config issue. If I add ssl to the startup apps in reltool.config and then repackage, the release seems to behave properly. Thanks/Apologies Ivan On 14/07/2014 11:49, Ivan Uemlianin wrote: > Dear All (esp. Felix Gallo) > > I am seeing almost the same strange problem with ssl certificates as > this post from last April. > > To recap: a rebar-packaged release with ssl running gets a "certificate > unknown" error when connecting over https: > > > httpc:request(get, {"https://www.example.org/etc/", []}, [], []). > {error,{failed_connect, > [{to_address,{"www.example.org",443}}, > {inet,[inet],{tls_alert,"certificate unknown"}}]}} > > The same code running with my local erl will make the above request > successfully. > > Different to Felix's situation: running the code using the release's erl > will make the request successfully too. > > In the original thread, Ingela Andin recommened [1] a specific commit to > OTP R16. @Felix did that work? Or did you otherwise fix the problem? > > Here, the same erl binary will make the https call successfully on its > own, but not if running the release. Could the package configs (eg > reltool.config) be leaving out something that should be included? > > Any pointers appreciated. > > With thanks and best wishes > > Ivan > > > [1] http://erlang.org/pipermail/erlang-questions/2013-April/073238.html > > -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From dpezely@REDACTED Mon Jul 14 20:19:23 2014 From: dpezely@REDACTED (Daniel Pezely) Date: Mon, 14 Jul 2014 18:19:23 +0000 Subject: [erlang-questions] Emacs 'erlang-mode-hook Message-ID: I might be doing this the hard-way, but it works well enough to share and ask for feedback: having Emacs erlang-shell play nice with multiple components built by rebar (invoked from Makefiles). First, the elisp code followed by more explanation. ;; Add paths to ../../*/deps/ebin for running & compiling: (add-hook 'erlang-mode-hook (lambda () (setq inferior-erlang-machine-options (append (remove nil (mapcan (lambda (app-path) (let ((ebin (concat app-path "/ebin"))) (when (file-readable-p ebin) (list "-pa" ebin)))) (directory-files "../.." t "[^.]$"))) (remove nil (mapcan (lambda (dep) (let ((dep-path (concat dep "/deps"))) (when (file-readable-p dep-path) (mapcan (lambda (dir) (list "-pa" (concat dir "/ebin"))) (directory-files dep-path t "[^.]$"))))) (directory-files "../.." t "[^.]$")))) erlang-compile-extra-opts (append (remove nil (mapcan (lambda (app-path) (let ((ebin (concat app-path "/ebin"))) (when (file-readable-p ebin) (list "-pa" ebin)))) (directory-files "../.." t "[^.]$"))) (remove nil (mapcan (lambda (dep) (let ((dep-path (concat dep "/deps"))) (when (file-readable-p dep-path) (mapcar (lambda (dir) (cons 'i (concat dir "/ebin"))) (directory-files dep-path t "[^.]$"))))) (directory-files "../.." t "[^.]$"))))))) We have a source tree on disk that roughly looks like the following with names changed to protect the innocent, etc. ls app1/ebin app1/deps app1/src app2/ebin app2/deps app2/src Our top-level application relies upon Cowboy & friends, and the division between app1 versus app2 is largely to isolate HTTP-related logic for a future alternate Erlang-specific entry point. This division introduces problems for *erlang-shell* within Emacs because default values for erlang-mode.el are such that it can't find dependencies across ../app* boundaries without telling the inferior-shell about paths. The 'erlang-mode-hook above relies upon rebar related directory tree and should work for an arbitrary number of sibling apps. (Of course, it will require restarting *erlang-shell* if any new dependencies are added.) The beauty of this is that you can compile everything via `make` and then do incremental file compilation from within Emacs and run within *erlang-shell*. The generated .beam files from an Emacs-initiated compilation also appear within ../ebin subdirectories rather than the src directory. There may be subtle interaction issues depending upon which app's source is compiled and possibly depositing the .beam file into the wrong ../ebin, but an occasional 'make clean' is useful during development anyway, I suppose. From my early C days, I tend to `make clean` at least once daily. Why I believe that there may be a cleaner or more concise way is that this is likely to be a common Erlang idiom if not also shared by other languages for which there are Emacs functions like inferior-erlang-compile. Anyone have suggestions about this? Thanks, -Daniel -- dpezely@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonym@REDACTED Tue Jul 15 01:20:41 2014 From: anthonym@REDACTED (ANTHONY MOLINARO) Date: Mon, 14 Jul 2014 16:20:41 -0700 Subject: [erlang-questions] Question about behaviours in Erlang/OTP In-Reply-To: References: Message-ID: Hi Yves, In the gen_server manual, in the Module:terminate/2 it states If the gen_server is part of a supervision tree and is ordered by its supervisor to terminate, this function will be called with Reason=shutdown if the following conditions apply: * the gen_server has been set to trap exit signals, and * the shutdown strategy as defined in the supervisor?s child specification is an integer timeout value, not brutal_kill So, if want to ensure that your gen_server terminate/2 function is call you need to trap exits. This is most common if there is some amount of cleanup you want to make sure happens before the gen_server terminates. HTH, -Anthony On Jul 13, 2014, at 4:59 PM, Yves S. Garret wrote: > Hello, > > I'm looking at some code that was written where a user uses a behaviour of gen_server and then uses a process_flag(trap_exit, true). To me, this seems like a supervisor behaviour would work better. > > What would be a better approach in this case? To declare this module a supervisor? Is it possible (sensible? logical?) to give the same module gen_server and supervisor behaviours? > > Thanks in advance! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From wbin00@REDACTED Tue Jul 15 02:27:09 2014 From: wbin00@REDACTED (Bin Wang) Date: Tue, 15 Jul 2014 08:27:09 +0800 Subject: [erlang-questions] A website to show popular erlang modules Message-ID: Hi, I've wrote this website some months ago. I will be happy if you find it helpful. http://erlang-modules.binwang.me/ It will fetch popular Erlang projects on Github and see how many other Erlang projects are depended on it. Details could be found here: http://www.binwang.me/2013-11-16-Fetch-Popular-Erlang-Modules-by-Coffee-Script.html --- Bin Wang From daocuong@REDACTED Tue Jul 15 02:46:47 2014 From: daocuong@REDACTED (DAO CUONG) Date: Mon, 14 Jul 2014 20:46:47 -0400 Subject: [erlang-questions] A website to show popular erlang modules In-Reply-To: References: Message-ID: Thanks Bin i like it On Mon, Jul 14, 2014 at 8:27 PM, Bin Wang wrote: > Hi, > > I've wrote this website some months ago. I will be happy if you find it > helpful. > > http://erlang-modules.binwang.me/ > > It will fetch popular Erlang projects on Github and see how many other > Erlang projects are depended on it. Details could be found here: > > http://www.binwang.me/2013-11-16-Fetch-Popular-Erlang-Modules-by-Coffee-Script.html > > --- > > Bin Wang > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jpsiyu@REDACTED Tue Jul 15 10:43:36 2014 From: jpsiyu@REDACTED (Fisher) Date: Tue, 15 Jul 2014 16:43:36 +0800 (CST) Subject: [erlang-questions] A website to show popular erlang modules In-Reply-To: References: Message-ID: <156f1092.c791.1473930fa7d.Coremail.jpsiyu@163.com> It's helpful! At 2014-07-15 08:27:09, "Bin Wang" wrote: >Hi, > >I've wrote this website some months ago. I will be happy if you find it helpful. > >http://erlang-modules.binwang.me/ > >It will fetch popular Erlang projects on Github and see how many other >Erlang projects are depended on it. Details could be found here: >http://www.binwang.me/2013-11-16-Fetch-Popular-Erlang-Modules-by-Coffee-Script.html > >--- > >Bin Wang >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Tue Jul 15 12:58:09 2014 From: ivan@REDACTED (Ivan Uemlianin) Date: Tue, 15 Jul 2014 11:58:09 +0100 Subject: [erlang-questions] ssl certificate verification in httpc Message-ID: <53C50941.5030408@llaisdy.com> Dear All I am using httpc to access a web server over https. From reading the erlang ssl docs, this looks like the right way to do it: Url = "https://bla.org/bla", SSLOpts = [{verify, verify_peer}, {cacertfile,"certificates.crt"}], httpc:request(get, {Url, []}, [{ssl,SSLOpts}], []). But this returns an unknown ca error: {error,{failed_connect,[{to_address,{"bla.org", 443}}, {inet,[inet],{tls_alert,"unknown ca"}}]}} 11:06:04.942 [error] SSL: certify: ssl_handshake.erl:1344:Fatal error: unknown ca The certificates file is in pem format, and I can't see from the docs page what ssl options I might add. Is there somewhere else I need to add a reference to the CA? In short, what am I missing to be able to verify this certificate? With thanks and best wishes Ivan -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From rtrlists@REDACTED Tue Jul 15 13:18:43 2014 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 15 Jul 2014 12:18:43 +0100 Subject: [erlang-questions] ssl certificate verification in httpc In-Reply-To: <53C50941.5030408@llaisdy.com> References: <53C50941.5030408@llaisdy.com> Message-ID: Hi Ivan, are you sure you need to verify the certificate? If SSL is only used for encryption, then a simple request will do the trick, that is, just leave out the ssl option. I've never verified a cert, so if you do need that, hopefully someone else can be of assistance. Regards, Robby On 15 July 2014 11:58, Ivan Uemlianin wrote: > Dear All > > I am using httpc to access a web server over https. From reading the > erlang ssl docs, this looks like the right way to do it: > > Url = "https://bla.org/bla", > SSLOpts = [{verify, verify_peer}, > {cacertfile,"certificates.crt"}], > httpc:request(get, {Url, []}, [{ssl,SSLOpts}], []). > > But this returns an unknown ca error: > > {error,{failed_connect,[{to_address,{"bla.org", 443}}, > {inet,[inet],{tls_alert,"unknown ca"}}]}} > 11:06:04.942 [error] SSL: certify: ssl_handshake.erl:1344:Fatal error: > unknown ca > > The certificates file is in pem format, and I can't see from the docs page > what ssl options I might add. Is there somewhere else I need to add a > reference to the CA? > > In short, what am I missing to be able to verify this certificate? > > With thanks and best wishes > > Ivan > > > -- > ============================================================ > Ivan A. Uemlianin PhD > Llaisdy > Speech Technology Research and Development > > ivan@REDACTED > www.llaisdy.com > llaisdy.wordpress.com > github.com/llaisdy > www.linkedin.com/in/ivanuemlianin > > festina lente > ============================================================ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Tue Jul 15 13:27:34 2014 From: ivan@REDACTED (Ivan Uemlianin) Date: Tue, 15 Jul 2014 12:27:34 +0100 Subject: [erlang-questions] ssl certificate verification in httpc In-Reply-To: References: <53C50941.5030408@llaisdy.com> Message-ID: <53C51026.7070207@llaisdy.com> Dear Robby Thanks for your comment. I'm not sure if I need to verify the certificate either :D. I was assuming the client needs to verify the certificate otherwise how is the connection "secure"? [more or less completely ignorant about ssl] Of course if the client doesn't need to verify the certificate that would be most convenient. Best wishes Ivan On 15/07/2014 12:18, Robert Raschke wrote: > Hi Ivan, > > are you sure you need to verify the certificate? If SSL is only used for > encryption, then a simple request will do the trick, that is, just leave > out the ssl option. > > I've never verified a cert, so if you do need that, hopefully someone > else can be of assistance. > > Regards, > Robby > > > > On 15 July 2014 11:58, Ivan Uemlianin > wrote: > > Dear All > > I am using httpc to access a web server over https. From reading > the erlang ssl docs, this looks like the right way to do it: > > Url = "https://bla.org/bla", > SSLOpts = [{verify, verify_peer}, > {cacertfile,"certificates.crt"__}], > httpc:request(get, {Url, []}, [{ssl,SSLOpts}], []). > > But this returns an unknown ca error: > > {error,{failed_connect,[{to___address,{"bla.org > ", 443}}, > {inet,[inet],{tls_alert,"__unknown ca"}}]}} > 11:06:04.942 [error] SSL: certify: ssl_handshake.erl:1344:Fatal > error: unknown ca > > The certificates file is in pem format, and I can't see from the > docs page what ssl options I might add. Is there somewhere else I > need to add a reference to the CA? > > In short, what am I missing to be able to verify this certificate? > > With thanks and best wishes > > Ivan > > > -- > ==============================__============================== > Ivan A. Uemlianin PhD > Llaisdy > Speech Technology Research and Development > > ivan@REDACTED > www.llaisdy.com > llaisdy.wordpress.com > github.com/llaisdy > www.linkedin.com/in/__ivanuemlianin > > > festina lente > ==============================__============================== > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/__listinfo/erlang-questions > > > -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From rtrlists@REDACTED Tue Jul 15 13:41:43 2014 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 15 Jul 2014 12:41:43 +0100 Subject: [erlang-questions] ssl certificate verification in httpc In-Reply-To: <53C51026.7070207@llaisdy.com> References: <53C50941.5030408@llaisdy.com> <53C51026.7070207@llaisdy.com> Message-ID: Hi Ivan, SSL essentially does two things at once: encrypts the data and checks if client and/or server are who they say they are. The latter is where certificate verification comes into play, the encryption part is always done and usually automatically negotiated between client and server. So, if all you are aiming for is encrypting the data travelling between client and server, then you don't need the ssl option. Just point your httpc:request at an "https://..." URL and the encryption is handled for you without you having to do anything more. Hope this helps, Robby On 15 July 2014 12:27, Ivan Uemlianin wrote: > Dear Robby > > Thanks for your comment. I'm not sure if I need to verify the certificate > either :D. I was assuming the client needs to verify the certificate > otherwise how is the connection "secure"? > > [more or less completely ignorant about ssl] > > Of course if the client doesn't need to verify the certificate that would > be most convenient. > > Best wishes > > Ivan > > > > On 15/07/2014 12:18, Robert Raschke wrote: > >> Hi Ivan, >> >> are you sure you need to verify the certificate? If SSL is only used for >> encryption, then a simple request will do the trick, that is, just leave >> out the ssl option. >> >> I've never verified a cert, so if you do need that, hopefully someone >> else can be of assistance. >> >> Regards, >> Robby >> >> >> >> On 15 July 2014 11:58, Ivan Uemlianin > > wrote: >> >> Dear All >> >> I am using httpc to access a web server over https. From reading >> the erlang ssl docs, this looks like the right way to do it: >> >> Url = "https://bla.org/bla", >> SSLOpts = [{verify, verify_peer}, >> {cacertfile,"certificates.crt"__}], >> >> httpc:request(get, {Url, []}, [{ssl,SSLOpts}], []). >> >> But this returns an unknown ca error: >> >> {error,{failed_connect,[{to___address,{"bla.org >> ", 443}}, >> {inet,[inet],{tls_alert,"__unknown >> ca"}}]}} >> >> 11:06:04.942 [error] SSL: certify: ssl_handshake.erl:1344:Fatal >> error: unknown ca >> >> The certificates file is in pem format, and I can't see from the >> docs page what ssl options I might add. Is there somewhere else I >> need to add a reference to the CA? >> >> In short, what am I missing to be able to verify this certificate? >> >> With thanks and best wishes >> >> Ivan >> >> >> -- >> ==============================__============================== >> >> Ivan A. Uemlianin PhD >> Llaisdy >> Speech Technology Research and Development >> >> ivan@REDACTED >> www.llaisdy.com >> llaisdy.wordpress.com >> github.com/llaisdy >> www.linkedin.com/in/__ivanuemlianin >> >> >> festina lente >> ==============================__============================== >> _________________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/__listinfo/erlang-questions >> >> >> >> > -- > ============================================================ > Ivan A. Uemlianin PhD > Llaisdy > Speech Technology Research and Development > > ivan@REDACTED > www.llaisdy.com > llaisdy.wordpress.com > github.com/llaisdy > www.linkedin.com/in/ivanuemlianin > > festina lente > ============================================================ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Tue Jul 15 15:02:33 2014 From: ivan@REDACTED (Ivan Uemlianin) Date: Tue, 15 Jul 2014 14:02:33 +0100 Subject: [erlang-questions] ssl certificate verification in httpc In-Reply-To: References: <53C50941.5030408@llaisdy.com> <53C51026.7070207@llaisdy.com> Message-ID: <53C52669.101@llaisdy.com> Dear Robby Thanks that is very helpful. Unfortunately, I think verifying the server will be required. I have an SSL book somewhere I keep meaning to look at, perhaps now is the time. Best wishes Ivan On 15/07/2014 12:41, Robert Raschke wrote: > Hi Ivan, > > SSL essentially does two things at once: encrypts the data and checks if > client and/or server are who they say they are. The latter is where > certificate verification comes into play, the encryption part is always > done and usually automatically negotiated between client and server. > > So, if all you are aiming for is encrypting the data travelling between > client and server, then you don't need the ssl option. Just point your > httpc:request at an "https://..." URL and the encryption is handled for > you without you having to do anything more. > > Hope this helps, > Robby > > > > On 15 July 2014 12:27, Ivan Uemlianin > wrote: > > Dear Robby > > Thanks for your comment. I'm not sure if I need to verify the > certificate either :D. I was assuming the client needs to verify > the certificate otherwise how is the connection "secure"? > > [more or less completely ignorant about ssl] > > Of course if the client doesn't need to verify the certificate that > would be most convenient. > > Best wishes > > Ivan > > > > On 15/07/2014 12:18, Robert Raschke wrote: > > Hi Ivan, > > are you sure you need to verify the certificate? If SSL is only > used for > encryption, then a simple request will do the trick, that is, > just leave > out the ssl option. > > I've never verified a cert, so if you do need that, hopefully > someone > else can be of assistance. > > Regards, > Robby > > > > On 15 July 2014 11:58, Ivan Uemlianin > >> wrote: > > Dear All > > I am using httpc to access a web server over https. From > reading > the erlang ssl docs, this looks like the right way to do it: > > Url = "https://bla.org/bla", > SSLOpts = [{verify, verify_peer}, > {cacertfile,"certificates.crt"____}], > > httpc:request(get, {Url, []}, [{ssl,SSLOpts}], []). > > But this returns an unknown ca error: > > {error,{failed_connect,[{to_____address,{"bla.org > > ", 443}}, > > {inet,[inet],{tls_alert,"____unknown ca"}}]}} > > 11:06:04.942 [error] SSL: certify: > ssl_handshake.erl:1344:Fatal > error: unknown ca > > The certificates file is in pem format, and I can't see > from the > docs page what ssl options I might add. Is there somewhere > else I > need to add a reference to the CA? > > In short, what am I missing to be able to verify this > certificate? > > With thanks and best wishes > > Ivan > > > -- > > ==============================____============================__== > > Ivan A. Uemlianin PhD > Llaisdy > Speech Technology Research and Development > > ivan@REDACTED > > > www.llaisdy.com > llaisdy.wordpress.com > > github.com/llaisdy > > www.linkedin.com/in/____ivanuemlianin > > > > > festina lente > > ==============================____============================__== > ___________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > > > http://erlang.org/mailman/____listinfo/erlang-questions > > > > > > > -- > ==============================__============================== > Ivan A. Uemlianin PhD > Llaisdy > Speech Technology Research and Development > > ivan@REDACTED > www.llaisdy.com > llaisdy.wordpress.com > github.com/llaisdy > www.linkedin.com/in/__ivanuemlianin > > > festina lente > ==============================__============================== > > -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From francesco@REDACTED Tue Jul 15 16:30:03 2014 From: francesco@REDACTED (Francesco Cesarini) Date: Tue, 15 Jul 2014 15:30:03 +0100 Subject: [erlang-questions] Erlang at HighLoad++ 2014 (Moscow) In-Reply-To: References: Message-ID: <53C53AEB.4050401@erlang-solutions.com> HigLoad in Moscow is looking for Erlang talks. The conference is towards the end of October, if anyone is interested in speaking, please contact Anastasiya directly. Francesco -------- Original Message -------- Subject: Erlang at HighLoad++ 2014 Date: Tue, 17 Jun 2014 23:59:52 +0400 From: ????????? ????????? To: Speakers Dear colleagues, *My name is Anastasiya Raspopina, and I?d like to invite your representative as a speaker to our conference on development of high-loaded web projects and systems. We respect your experience in IT so we ask an Erlang Solutions' employee to kindly offer a talk for HighLoad++ Conference (Moscow). We?ll pass the abstracts of this talk to HighLoad++ 2014 Program Committee immediately on receipt.* This year HighLoad++ is expected to become one of the biggest conferences in Europe with its 1600 attendees. We would be more than grateful if you virtually attend HighLoad++ 2014 at Lanyrd.com: http://lanyrd.com/2014/highload2014/ - you can use either Twitter or LinkedIn account. Many thanks in advance! Best regards, Anastasiya Raspopina, HighLoad++ Conference Manager -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists@REDACTED Tue Jul 15 16:41:49 2014 From: lists@REDACTED (Camille Troillard) Date: Tue, 15 Jul 2014 16:41:49 +0200 Subject: [erlang-questions] ssl certificate verification in httpc In-Reply-To: References: <53C50941.5030408@llaisdy.com> <53C51026.7070207@llaisdy.com> Message-ID: Hi Robert, > SSL essentially does two things at once: encrypts the data and checks if client and/or server are who they say they are. The latter is where certificate verification comes into play, the encryption part is always done and usually automatically negotiated between client and server. > > So, if all you are aiming for is encrypting the data travelling between client and server, then you don't need the ssl option. Just point your httpc:request at an "https://..." URL and the encryption is handled for you without you having to do anything more. I think this works only if you trust your DNS, otherwise you have to check that the certificate matches the host. Please correct me if I?m wrong. Cam From pablo.platt@REDACTED Tue Jul 15 17:53:59 2014 From: pablo.platt@REDACTED (pablo platt) Date: Tue, 15 Jul 2014 18:53:59 +0300 Subject: [erlang-questions] Replace ERL_BIN_ATOM with binary/list in a C node. Message-ID: Hi, I'm using erlycairo to draw on a 2D canvas. There are several functions that leak atoms: erlycairo_server:write_to_png/2 erlycairo_server:show_text/2 erlycairo_server:text_extents/2 erlycairo_server:surface_create_from_png/2 I tried to pass a binary to the C node and use (char *)ERL_BIN_PTR(file) but I'm getting another char at the end of the binary. https://code.google.com/p/erlycairo/source/browse/trunk/src/erlycairo/erlycairo_server.erl#143 https://code.google.com/p/erlycairo/source/browse/trunk/c_src/erlycairo.c#337 What is the correct way to use binaries or lists instead of atoms for the write_to_png function? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From rtrlists@REDACTED Tue Jul 15 18:21:45 2014 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 15 Jul 2014 17:21:45 +0100 Subject: [erlang-questions] Replace ERL_BIN_ATOM with binary/list in a C node. In-Reply-To: References: Message-ID: Hi Pablo, I'm not 100% certain, but I think binaries are not NULL terminated. So, you need to take the length of the binary into account. That obviously makes the code change a bit harder :-( Looking at the erl_interface code for the atom conversion, I see quite a bit of shuffling to do with character encodings and explicit setting of \0 terminator. I think you'll need to take a copy of the binary data and slap the \0 onto the end yourself. And you need to be aware of the character encoding you are expecting to exchange and, if required, convert accordingly. Regards, Robby On 15 July 2014 16:53, pablo platt wrote: > Hi, > > I'm using erlycairo to draw on a 2D canvas. > There are several functions that leak atoms: > erlycairo_server:write_to_png/2 > erlycairo_server:show_text/2 > erlycairo_server:text_extents/2 > erlycairo_server:surface_create_from_png/2 > > I tried to pass a binary to the C node and use (char *)ERL_BIN_PTR(file) > but I'm getting another char at the end of the binary. > > https://code.google.com/p/erlycairo/source/browse/trunk/src/erlycairo/erlycairo_server.erl#143 > > https://code.google.com/p/erlycairo/source/browse/trunk/c_src/erlycairo.c#337 > > What is the correct way to use binaries or lists instead of atoms for the > write_to_png function? > > Thanks > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Tue Jul 15 19:03:44 2014 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Tue, 15 Jul 2014 13:03:44 -0400 Subject: [erlang-questions] What is best way to maintain state in a web-based multiplayer game? Message-ID: <54A8BBD3-428E-4140-A40C-5D4CEB1F1546@writersglen.com> I'm thinking through the design of a web-based multiple-player game. Each player maintains a private database--- most likely dets. Each player moves through nested finite state machines. Each player may be logging in or out at any state of play. At this point my grasp of Erlang architecture breaks down. I don't understand how best to: - parse game structure across players, processes, and directories - maintain player state between sessions At this point in my Erlang education I feel like I know the words but can't play the music. I'd much grateful for any and all ideas and suggestions. Many thanks, LRP Sent from my iPad From felixgallo@REDACTED Tue Jul 15 19:50:30 2014 From: felixgallo@REDACTED (Felix Gallo) Date: Tue, 15 Jul 2014 10:50:30 -0700 Subject: [erlang-questions] What is best way to maintain state in a web-based multiplayer game? In-Reply-To: <54A8BBD3-428E-4140-A40C-5D4CEB1F1546@writersglen.com> References: <54A8BBD3-428E-4140-A40C-5D4CEB1F1546@writersglen.com> Message-ID: I think you have some more marble to cut away before the shape of the problem begins to reveal itself. * do the players interact with each other directly? If so, what state do they share and how much? Is there a separation between 'state for everyone' (e.g. world of warcraft: the entire world geometry and all 3d models are gigabytes of static data that are stored on the local filesystem and not kept 'live' in memory) and 'player state'? What is the size and nature of each type of state? * what are the constraints around latency -- does it look more like a real time game (e.g. street fighter, counterstrike, league of legends) where, e.g., 250 ms of latency is unacceptable to the experience, or more like a turn based game (e.g. chess, poker, backgammon) where latency of 3s is tolerable? This drives certain considerations; e.g., you may be able to use a slightly lower performance but more reliable database system for the latter case. * what is the shape and nature of your cost envelope? If you are running server infrastructure for a game, then you may need to take special measures to ensure that your costs remain beneath your gross profit. Since, as you say, managing game state is such a giant pain in the ass, my general best practice when I do exactly this thing as part of my day job is to first write down all of the different kinds of state, their approximate quantity in bytes per player/world/universe/shard/whatever, and then do some spreadsheet math to figure out what I can get away with. Example: Game is Facebook-style web game like Farmville. State can be boiled down to a binary data structure that is 4 kilobytes in size per player. Expected profit per user is on the order of pennies. Tens of millions of users expected. Solution: save state in the user's own browser via encrypted cookie, use tiny database (sqlite, mysql, etc.) for leaderboards and sending seeds between players. Example: Game is World of Warcraft style MMO but you're trying to do it in the browser so can't download gigabytes. So you must shard each zone into, say, a 50 megabyte downloadable size. Zone shards are static so can be files. Private player data (e.g. inventory, raid history, message log) is maybe 1 megabyte; public player data (e.g., x/y/z coordinates, avatar skin) is maybe 30K and shared across the entire world (but usually limited to a shard), needs fast access though as the world ticks at 30 frames per second and you need to do distance calculations, etc. Store the public player data in memory, dets (per shard) or redis depending on whether you want raw speed, resiliency, or tooling. Store the private data in memory, dets, redis, or mysql, depending on speed, resiliency, tooling, cost and analytics requirements. Generally I use ets for fast-access random-lookup state that has to get shared between processes, redis for fast-enough (still plenty fast) random-lookup state that has to be able to get large and/or survive system crashes or writeoffs (via slave replication and aof, rather than clustering), mysql for mass market mediocre state storage where the client has pre-existing infrastructure and ad hoc querying is important, postgres for green fields state storage where ad hoc querying is important, and thankfully so far I have been able to avoid having to use inconsistent data stores in production. I wouldn't ever consider dets or mnesia because equivalently-functional tools exist that are radically more popular and have better tooling, capability, characteristics or safety guarantees. F. On Tue, Jul 15, 2014 at 10:03 AM, Lloyd R. Prentice wrote: > > I'm thinking through the design of a web-based multiple-player game. Each > player maintains a private database--- most likely dets. Each player moves > through nested finite state machines. Each player may be logging in or out > at any state of play. > > At this point my grasp of Erlang architecture breaks down. I don't > understand how best to: > > - parse game structure across players, processes, and directories > - maintain player state between sessions > > At this point in my Erlang education I feel like I know the words but > can't play the music. > > I'd much grateful for any and all ideas and suggestions. > > Many thanks, > > LRP > > Sent from my iPad > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Tue Jul 15 19:55:51 2014 From: g@REDACTED (Garrett Smith) Date: Tue, 15 Jul 2014 11:55:51 -0600 Subject: [erlang-questions] What is best way to maintain state in a web-based multiplayer game? In-Reply-To: <54A8BBD3-428E-4140-A40C-5D4CEB1F1546@writersglen.com> References: <54A8BBD3-428E-4140-A40C-5D4CEB1F1546@writersglen.com> Message-ID: On Tue, Jul 15, 2014 at 11:03 AM, Lloyd R. Prentice wrote: > > I'm thinking through the design of a web-based multiple-player game. Each player maintains a private database--- most likely dets. Each player moves through nested finite state machines. Each player may be logging in or out at any state of play. > > At this point my grasp of Erlang architecture breaks down. I don't understand how best to: > > - parse game structure across players, processes, and directories > - maintain player state between sessions > > At this point in my Erlang education I feel like I know the words but can't play the music. Out of curiosity, and as a possible valuable exercise, how would you tackle this problem in another platform -- one that you're more versed in for this type of application? Garrett From lloyd@REDACTED Tue Jul 15 20:42:58 2014 From: lloyd@REDACTED (lloyd@REDACTED) Date: Tue, 15 Jul 2014 14:42:58 -0400 (EDT) Subject: [erlang-questions] =?utf-8?q?What_is_best_way_to_maintain_state_i?= =?utf-8?q?n_a_web-based_multiplayer_game=3F?= In-Reply-To: References: <54A8BBD3-428E-4140-A40C-5D4CEB1F1546@writersglen.com> Message-ID: <1405449778.198729759@apps.rackspace.com> Hi Felix, Thanks for helping me think this through. - Players do not interact. Each player is playing against the "real world," e.g. results of their actions. Think of it as project planning with many many contingencies. I'd say "expert system," but that sounds rather hoity toity. I say "game" only in the sense that there are reward points for successful action. Loop is basically: - gather facts - make plan - take action - evaluate results - There are universal resources accessed by player state but these, as a whole, are static text, links, and images; wild-eyed estimate: 10 megabytes - I estimate that each player will store less than three megabytes of private data. - We're not dealing with elaborate graphics or animations. Only latency is a logic tree or inference engine evaluating facts; e.g. (Looking at Erlang eres/seres; but I may be able get away with something more simple minded). Latency, in other words, is a function of how many private and shared database lookups are necessary to evaluate a set of condition -> action rules. Effective state management should help me keep rule sets fairly minimal. Prospects for this game are in the 200K range, but I'm successful with low hundreds of players. Ad hoc querying is not necessary. Thanks again for your help. Lloyd -----Original Message----- From: "Felix Gallo" Sent: Tuesday, July 15, 2014 1:50pm To: "Lloyd R. Prentice" Cc: "Erlang-Questions Questions" Subject: Re: [erlang-questions] What is best way to maintain state in a web-based multiplayer game? I think you have some more marble to cut away before the shape of the problem begins to reveal itself. * do the players interact with each other directly? If so, what state do they share and how much? Is there a separation between 'state for everyone' (e.g. world of warcraft: the entire world geometry and all 3d models are gigabytes of static data that are stored on the local filesystem and not kept 'live' in memory) and 'player state'? What is the size and nature of each type of state? * what are the constraints around latency -- does it look more like a real time game (e.g. street fighter, counterstrike, league of legends) where, e.g., 250 ms of latency is unacceptable to the experience, or more like a turn based game (e.g. chess, poker, backgammon) where latency of 3s is tolerable? This drives certain considerations; e.g., you may be able to use a slightly lower performance but more reliable database system for the latter case. * what is the shape and nature of your cost envelope? If you are running server infrastructure for a game, then you may need to take special measures to ensure that your costs remain beneath your gross profit. Since, as you say, managing game state is such a giant pain in the ass, my general best practice when I do exactly this thing as part of my day job is to first write down all of the different kinds of state, their approximate quantity in bytes per player/world/universe/shard/whatever, and then do some spreadsheet math to figure out what I can get away with. Example: Game is Facebook-style web game like Farmville. State can be boiled down to a binary data structure that is 4 kilobytes in size per player. Expected profit per user is on the order of pennies. Tens of millions of users expected. Solution: save state in the user's own browser via encrypted cookie, use tiny database (sqlite, mysql, etc.) for leaderboards and sending seeds between players. Example: Game is World of Warcraft style MMO but you're trying to do it in the browser so can't download gigabytes. So you must shard each zone into, say, a 50 megabyte downloadable size. Zone shards are static so can be files. Private player data (e.g. inventory, raid history, message log) is maybe 1 megabyte; public player data (e.g., x/y/z coordinates, avatar skin) is maybe 30K and shared across the entire world (but usually limited to a shard), needs fast access though as the world ticks at 30 frames per second and you need to do distance calculations, etc. Store the public player data in memory, dets (per shard) or redis depending on whether you want raw speed, resiliency, or tooling. Store the private data in memory, dets, redis, or mysql, depending on speed, resiliency, tooling, cost and analytics requirements. Generally I use ets for fast-access random-lookup state that has to get shared between processes, redis for fast-enough (still plenty fast) random-lookup state that has to be able to get large and/or survive system crashes or writeoffs (via slave replication and aof, rather than clustering), mysql for mass market mediocre state storage where the client has pre-existing infrastructure and ad hoc querying is important, postgres for green fields state storage where ad hoc querying is important, and thankfully so far I have been able to avoid having to use inconsistent data stores in production. I wouldn't ever consider dets or mnesia because equivalently-functional tools exist that are radically more popular and have better tooling, capability, characteristics or safety guarantees. F. On Tue, Jul 15, 2014 at 10:03 AM, Lloyd R. Prentice <[ lloyd@REDACTED ]( mailto:lloyd@REDACTED )> wrote: I'm thinking through the design of a web-based multiple-player game. Each player maintains a private database--- most likely dets. Each player moves through nested finite state machines. Each player may be logging in or out at any state of play. At this point my grasp of Erlang architecture breaks down. I don't understand how best to: - parse game structure across players, processes, and directories - maintain player state between sessions At this point in my Erlang education I feel like I know the words but can't play the music. I'd much grateful for any and all ideas and suggestions. Many thanks, LRP Sent from my iPad _______________________________________________ erlang-questions mailing list [ erlang-questions@REDACTED ]( mailto:erlang-questions@REDACTED ) [ http://erlang.org/mailman/listinfo/erlang-questions ]( http://erlang.org/mailman/listinfo/erlang-questions ) -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Tue Jul 15 21:05:21 2014 From: lloyd@REDACTED (lloyd@REDACTED) Date: Tue, 15 Jul 2014 15:05:21 -0400 (EDT) Subject: [erlang-questions] =?utf-8?q?What_is_best_way_to_maintain_state_i?= =?utf-8?q?n_a_web-based_multiplayer_game=3F?= In-Reply-To: References: <54A8BBD3-428E-4140-A40C-5D4CEB1F1546@writersglen.com> Message-ID: <1405451121.240129172@apps.rackspace.com> Hi Garrett, Back in antediluvian times I designed and built with help of much more talented programmers than I several hundred educational and consumer games and applications for the Apple II series, Commodore 64s and, dare I say it, IBM PC Jr. --- all in Forth. We didn't have to worry about networks or multiusers since in our markets there were none. Biggest problem was getting graphics up fast enough and synchronizing with beep beep sounds. Oh, and fitting all the game play and graphics into onerous memory and disc constraints. I did design and build an algebra equation solver that was kind of cool and a fun promotional game for Illinois Power based on finite state machines. The Forth representation of FSMs is really really elegant. It now occurs to me that I also designed and built a promotional game for AT&T running on the IBM PC that had a high-level loop not unlike what I'm thinking about. But we had to keep the game part fairly easy to win since we were trying to sell AT&T services. The conference management stuff I worked on for many dreary years was written in Cold Fusion with an MS SQL Server backend. Not sure how much of that carries over to my current problem/interest--- other than certainty that I don't want to build in Cold Fusion and MS SQL Server. I played with Python, but only enough to be dangerous. In short, consider me an enthusiastic wet-behind-the-ears rank noobie. I truly do want to become journeyman competent with Erlang. Progress is slow, but this wonderful Erlang community and folks like you helps goose it along. Thanks, Lloyd -----Original Message----- From: "Garrett Smith" Sent: Tuesday, July 15, 2014 1:55pm To: "Lloyd R. Prentice" Cc: "Erlang-Questions Questions" Subject: Re: [erlang-questions] What is best way to maintain state in a web-based multiplayer game? On Tue, Jul 15, 2014 at 11:03 AM, Lloyd R. Prentice wrote: > > I'm thinking through the design of a web-based multiple-player game. Each player maintains a private database--- most likely dets. Each player moves through nested finite state machines. Each player may be logging in or out at any state of play. > > At this point my grasp of Erlang architecture breaks down. I don't understand how best to: > > - parse game structure across players, processes, and directories > - maintain player state between sessions > > At this point in my Erlang education I feel like I know the words but can't play the music. Out of curiosity, and as a possible valuable exercise, how would you tackle this problem in another platform -- one that you're more versed in for this type of application? Garrett -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Tue Jul 15 21:07:32 2014 From: felixgallo@REDACTED (Felix Gallo) Date: Tue, 15 Jul 2014 12:07:32 -0700 Subject: [erlang-questions] What is best way to maintain state in a web-based multiplayer game? In-Reply-To: <1405449778.198729759@apps.rackspace.com> References: <54A8BBD3-428E-4140-A40C-5D4CEB1F1546@writersglen.com> <1405449778.198729759@apps.rackspace.com> Message-ID: for the static stuff -- I would just use nginx as a web server and serve that separately. for the game states -- if you don't need to query across all the game states (which you will eventually want to do, so that you have a game design feedback loop), then all you need to do is find a way to get the game state to be alive when the player is around, and to hibernate into storage when the player isn't. If you are in pure erlangland or largely pure erlangland, then you might pick BERT (binary erlang term representation (http://bert-rpc.org/)) as your serialization/deserialization standard for ease of use (all you have to do is erlang:term_to_binary/1) and then save that binary off to some binary storage facility (files, mysql or postgres blob fields, redis values, s3 buckets...). Then, when a player comes back, all your gen_server for that player has to do is to figure out which file is theirs, erlang:binary_to_term/2 it, and hey presto, state restored seamlessly. 3M is a really big size for a single player game state by the way -- I would carefully evaluate how tight you can package your state, because every byte will be multiplied by the number of concurrent users you have per server. At 3M, 1K concurrent users is 3 gigabytes of memory; 10K concurrent (feasibly if you have 200K total users) is 30G and that's starting to become a serious server investment. If you instead had 3K worth of hard state to carry around, you could fit your entire playerbase in a 4G server, which is quite inexpensive. F. On Tue, Jul 15, 2014 at 11:42 AM, wrote: > Hi Felix, > > > > Thanks for helping me think this through. > > > > - Players do not interact. Each player is playing against the "real > world," e.g. results of their actions. Think of it as project planning with > many many contingencies. I'd say "expert system," but that sounds rather > hoity toity. I say "game" only in the sense that there are reward points > for successful action. > > > > Loop is basically: > > > > - gather facts > > - make plan > > - take action > > - evaluate results > > > > - There are universal resources accessed by player state but these, as a > whole, are static text, links, and images; wild-eyed estimate: 10 megabytes > > - I estimate that each player will store less than three megabytes of > private data. > > > > - We're not dealing with elaborate graphics or animations. Only latency is > a logic tree or inference engine evaluating facts; e.g. (Looking at Erlang > eres/seres; but I may be able get away with something more simple minded). > > > > Latency, in other words, is a function of how many private and shared > database lookups are necessary to evaluate a set of condition -> action > rules. Effective state management should help me keep rule sets fairly > minimal. > > > > Prospects for this game are in the 200K range, but I'm successful with low > hundreds of players. > > > > Ad hoc querying is not necessary. > > > > Thanks again for your help. > > > > Lloyd > > > > > > > > -----Original Message----- > From: "Felix Gallo" > Sent: Tuesday, July 15, 2014 1:50pm > To: "Lloyd R. Prentice" > Cc: "Erlang-Questions Questions" > Subject: Re: [erlang-questions] What is best way to maintain state in a > web-based multiplayer game? > > I think you have some more marble to cut away before the shape of the > problem begins to reveal itself. > * do the players interact with each other directly? If so, what state do > they share and how much? Is there a separation between 'state for > everyone' (e.g. world of warcraft: the entire world geometry and all 3d > models are gigabytes of static data that are stored on the local filesystem > and not kept 'live' in memory) and 'player state'? What is the size and > nature of each type of state? > * what are the constraints around latency -- does it look more like a real > time game (e.g. street fighter, counterstrike, league of legends) where, > e.g., 250 ms of latency is unacceptable to the experience, or more like a > turn based game (e.g. chess, poker, backgammon) where latency of 3s is > tolerable? This drives certain considerations; e.g., you may be able to > use a slightly lower performance but more reliable database system for the > latter case. > > * what is the shape and nature of your cost envelope? If you are running > server infrastructure for a game, then you may need to take special > measures to ensure that your costs remain beneath your gross profit. > Since, as you say, managing game state is such a giant pain in the ass, my > general best practice when I do exactly this thing as part of my day job is > to first write down all of the different kinds of state, their approximate > quantity in bytes per player/world/universe/shard/whatever, and then do > some spreadsheet math to figure out what I can get away with. > Example: Game is Facebook-style web game like Farmville. State can be > boiled down to a binary data structure that is 4 kilobytes in size per > player. Expected profit per user is on the order of pennies. Tens of > millions of users expected. Solution: save state in the user's own browser > via encrypted cookie, use tiny database (sqlite, mysql, etc.) for > leaderboards and sending seeds between players. > Example: Game is World of Warcraft style MMO but you're trying to do it in > the browser so can't download gigabytes. So you must shard each zone into, > say, a 50 megabyte downloadable size. Zone shards are static so can be > files. Private player data (e.g. inventory, raid history, message log) is > maybe 1 megabyte; public player data (e.g., x/y/z coordinates, avatar skin) > is maybe 30K and shared across the entire world (but usually limited to a > shard), needs fast access though as the world ticks at 30 frames per second > and you need to do distance calculations, etc. Store the public player > data in memory, dets (per shard) or redis depending on whether you want raw > speed, resiliency, or tooling. Store the private data in memory, dets, > redis, or mysql, depending on speed, resiliency, tooling, cost and > analytics requirements. > Generally I use ets for fast-access random-lookup state that has to get > shared between processes, redis for fast-enough (still plenty fast) > random-lookup state that has to be able to get large and/or survive system > crashes or writeoffs (via slave replication and aof, rather than > clustering), mysql for mass market mediocre state storage where the client > has pre-existing infrastructure and ad hoc querying is important, postgres > for green fields state storage where ad hoc querying is important, and > thankfully so far I have been able to avoid having to use inconsistent data > stores in production. I wouldn't ever consider dets or mnesia because > equivalently-functional tools exist that are radically more popular and > have better tooling, capability, characteristics or safety guarantees. > F. > > > On Tue, Jul 15, 2014 at 10:03 AM, Lloyd R. Prentice > wrote: > >> >> I'm thinking through the design of a web-based multiple-player game. Each >> player maintains a private database--- most likely dets. Each player moves >> through nested finite state machines. Each player may be logging in or out >> at any state of play. >> >> At this point my grasp of Erlang architecture breaks down. I don't >> understand how best to: >> >> - parse game structure across players, processes, and directories >> - maintain player state between sessions >> >> At this point in my Erlang education I feel like I know the words but >> can't play the music. >> >> I'd much grateful for any and all ideas and suggestions. >> >> Many thanks, >> >> LRP >> >> Sent from my iPad >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Tue Jul 15 22:30:55 2014 From: rvirding@REDACTED (Robert Virding) Date: Tue, 15 Jul 2014 22:30:55 +0200 Subject: [erlang-questions] Data structures are strange things In-Reply-To: <8ea865f7-17df-4254-ba3c-732c0cefd074@googlegroups.com> References: <4a0b389f-5cb3-4768-8eba-acf95d03cce3@knuth> <9b6fc3a1-ff75-45b3-b182-835cc89d495e@knuth> <8ea865f7-17df-4254-ba3c-732c0cefd074@googlegroups.com> Message-ID: No, I never published them. I still have them of course. The one I prefer is the 2-3 trees which I use in my luerl implementation. Maps could replace them but they are lacking some functions I need. It will be interesting to see how they compare in speed. Robert On 14 July 2014 08:40, Kirill Zaborsky wrote: > Hi Robert, > I have came across this a bit old topic and I wonder - did you publish any > of those modules? > I have only found your repo https://github.com/rvirding/rb which is even > 3 years older than this thread :) > > Regards, > Kirill > > ???????, 8 ???? 2012 ?., 4:55:29 UTC+4 ???????????? Robert Virding ???????: > >> I was finally going to release some alternate modules with the dict >> interface using tree structures, either AA trees or Red Black trees. These >> are relatively well-known binary tree structures which are binary tree >> versions of 2-3 trees and 2-3-4 trees respectively. These contain codes >> with either 2, 3 or 4 sub-nodes. Generally the binary versions are >> recommended as being better/easier/more space efficient. >> >> Before I released my modules I thought it would be interesting to try the >> non-binary versions to see how they behaved and what it was lie to program >> them. I was lucky to find a very good description of 2-3 trees and the >> rules for inserting/deleting elements; it was literally just translate the >> rules straight to Erlang and it worked. The code was a little more complex >> than working with the binary versions, not much. In some cases actually >> simpler. The I decided to measure speed, and the fun started. >> >> All tests were done with sequences of random numbers as keys, the same >> for each implementation. >> >> What I found was that the lookup times were about the same as for the >> binary versions, which was expected, but the insertion/deletion times were >> about 20% faster. This compared to my AA and RB tree versions. To check >> that it wasn't just bad coding on my part (not bloody likely :-)) I >> compared it with gb_trees and got the similar results. Also checking the >> data sizes I found that the 2-3 and 2-3-4 trees actually used *less* space >> than their binary counterparts. >> >> Here are the figures. I have also included dict, which is the slowest on >> insertion/deletion but significantly faster lookup times. All times are in >> usecs for the whole set: >> >> Operation Load Fetch Load Fetch >> >> # of elements 100000 10000 >> Overhead time 43000 3700 >> >> ttdict 2-3 trees 310000 161000 24000 15500 >> ttfdict 2-3-4 trees 312000 173000 24200 16500 >> llrbdict Left Leaning RB trees 390000 164000 27100 14900 >> >> rbdict Red Black trees 380000 167000 27100 15000 >> aadict AA trees 423000 167000 30000 15000 >> >> gb_trees using faster insert/3 370000 190000 26500 16200 >> dict 522000 107000 28000 11000 >> >> >> (I hope the table is readable) >> Gb_trees are a variant of AA trees with a slightly different algorithm. >> One should be careful interpreting exact values here but the trend is clear. >> >> I don't know why the straight multi-size versions are faster but my guess >> is that the pattern-matching/data construction is faster on the flat bigger >> nodes rather than on the nested pairs. My guess is that when they comment >> on the 2-3(-4) trees using more space it is because they use the same size >> node for all cases which in the cases with fewer sub-nodes wastes space, >> while just used different sized tuples which didn't waste space; it also >> made the pattern matching much easier. >> >> It looks like I will be submitting the 2-3 tree version, ttdict, as >> reasonable choice. Next I will try AVL trees which were discarded long ago >> and see how they fare. >> >> Robert >> >> P.S. This is the reason why there was hoped to be many different dict >> modules with the same API but different data structures. So as it to make >> it easier to choose the most suitable version for each occasion. There is >> no generally "best" choice, even among trees. >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knutin@REDACTED Tue Jul 15 22:38:59 2014 From: knutin@REDACTED (Knut Nesheim) Date: Tue, 15 Jul 2014 22:38:59 +0200 Subject: [erlang-questions] What is best way to maintain state in a web-based multiplayer game? In-Reply-To: <1405449778.198729759@apps.rackspace.com> References: <54A8BBD3-428E-4140-A40C-5D4CEB1F1546@writersglen.com> <1405449778.198729759@apps.rackspace.com> Message-ID: Hey, I've had the great pleasure to build a few such systems and help out other teams with their systems. Some of my colleagues and me have presented about this exact topic over the last few years at the Erlang Factory in London, San Francisco, Berlin and the Erlang User Conference in Stockholm. Here's one such presentation: http://www.erlang-factory.com/upload/presentations/469/wooga-euc2011-v2-111103095416-phpapp01.pdf If each user can only affect state owned by that user, you have your parallelism and state management right there. Simplest case is to store all state in a database (relational or not) and on every request, load up all or parts, mutate it and write it back. If you're expecting intensive (1 request every few seconds) and last long (minutes to hours) it makes sense to try to be smarter. You could for example load up the user state at session start in an (unsupervised) Erlang process, mutate it in memory (using purely functional code and plain Erlang data structures) and when the session ends write the full state back and stop the process. You could do this with any of the database options: relational, document or plain key-value. If you want to make life easy for yourself, going for a service provided by let's say Amazon like S3 would be a great choice. Scaling the application servers are then mostly about coordinating sessions (static sharding or distributed coordination, we eventually ended up writing a distributed lock: https://github.com/wooga/locker, presentation here: http://www.erlang-factory.com/conference/SFBay2013/speakers/KnutNesheim) Pro tip: if using plain key-value, use a serialization format easy to digest by other tools, such a json or protobuffers or whatever floats your boat. If your game is a multiplayer game where each user can affect the state of other users (or a shared world), the problem is much more difficult. I've had the pleasure to work on one such game. Each user was separate, but users could join the world of their friends and mess around. This is a much easier problem than the one-single-huge-persistent world problem. We solved this by having one Erlang process for the world of a user, one process for each user and a client that understood how to roll back state updates (needed for conflict resolution). As the world is single-threaded and processes events in the order they arrive, that's the order that is used to resolve conflicting updates. Ie, if we both pick up an apple from the ground at the same time we will see the UI of the game updating, but only the user that reaches the server first (ie faster network) will actually receive the apple and when the second user reaches the server there's no apple on the ground any longer. The client needs to figure out a graceful way of rolling back the state (to avoid a full reload) in a nice way (to avoid pissing off users). We also presented this system: see https://www.youtube.com/watch?v=AUeFutXPojU and http://techmeshconf.com/techmesh-london-2012/presentation/You%20are%20not%20Alone%20-%20Scaling%20Multiplayer%20Games%20for%20the%20Web In the case of MMO, RTS or FPS games they use the same concept as outlined above, but with some important variations. The worlds are always sharded, sometimes this is visible to the user (you need to join a particular "instance" of a world, you pick a map to play, the world is restricted to only one round, there's a limit on players, the game gets slow when many players join, etc) or it's not explicitly visible, but smoothed over as "you can't see or affect users really really far away" and when you move there, you're actually moving shards. Conflict resolution is very important, you want clients to concurrently proceed and only later resolve the conflict, if any. Conflict in this case might be player death in a FPS game, in which case you need consider the effect of latency on player states. For example, if I'm moving in a figure 8 shape users looking at me will see me at different places at exactly the same wallclock time depending on the latency of their connection. If a user shoots me, it might be that in their version of the world the game engine considers it a hit, but in my version of the world it was a miss. One way to solve this is that the server keeps track of the game state of each client and runs the game logic with the state the shooting client should have seen. It's a very interesting topic! Good luck with your game! Knut On Tue, Jul 15, 2014 at 8:42 PM, wrote: > Hi Felix, > > > > Thanks for helping me think this through. > > > > - Players do not interact. Each player is playing against the "real world," > e.g. results of their actions. Think of it as project planning with many > many contingencies. I'd say "expert system," but that sounds rather hoity > toity. I say "game" only in the sense that there are reward points for > successful action. > > > > Loop is basically: > > > > - gather facts > > - make plan > > - take action > > - evaluate results > > > > - There are universal resources accessed by player state but these, as a > whole, are static text, links, and images; wild-eyed estimate: 10 megabytes > > - I estimate that each player will store less than three megabytes of > private data. > > > > - We're not dealing with elaborate graphics or animations. Only latency is a > logic tree or inference engine evaluating facts; e.g. (Looking at Erlang > eres/seres; but I may be able get away with something more simple minded). > > > > Latency, in other words, is a function of how many private and shared > database lookups are necessary to evaluate a set of condition -> action > rules. Effective state management should help me keep rule sets fairly > minimal. > > > > Prospects for this game are in the 200K range, but I'm successful with low > hundreds of players. > > > > Ad hoc querying is not necessary. > > > > Thanks again for your help. > > > > Lloyd > > > > > > > > -----Original Message----- > From: "Felix Gallo" > Sent: Tuesday, July 15, 2014 1:50pm > To: "Lloyd R. Prentice" > Cc: "Erlang-Questions Questions" > Subject: Re: [erlang-questions] What is best way to maintain state in a > web-based multiplayer game? > > I think you have some more marble to cut away before the shape of the > problem begins to reveal itself. > * do the players interact with each other directly? If so, what state do > they share and how much? Is there a separation between 'state for everyone' > (e.g. world of warcraft: the entire world geometry and all 3d models are > gigabytes of static data that are stored on the local filesystem and not > kept 'live' in memory) and 'player state'? What is the size and nature of > each type of state? > * what are the constraints around latency -- does it look more like a real > time game (e.g. street fighter, counterstrike, league of legends) where, > e.g., 250 ms of latency is unacceptable to the experience, or more like a > turn based game (e.g. chess, poker, backgammon) where latency of 3s is > tolerable? This drives certain considerations; e.g., you may be able to use > a slightly lower performance but more reliable database system for the > latter case. > > * what is the shape and nature of your cost envelope? If you are running > server infrastructure for a game, then you may need to take special measures > to ensure that your costs remain beneath your gross profit. > Since, as you say, managing game state is such a giant pain in the ass, my > general best practice when I do exactly this thing as part of my day job is > to first write down all of the different kinds of state, their approximate > quantity in bytes per player/world/universe/shard/whatever, and then do some > spreadsheet math to figure out what I can get away with. > Example: Game is Facebook-style web game like Farmville. State can be > boiled down to a binary data structure that is 4 kilobytes in size per > player. Expected profit per user is on the order of pennies. Tens of > millions of users expected. Solution: save state in the user's own browser > via encrypted cookie, use tiny database (sqlite, mysql, etc.) for > leaderboards and sending seeds between players. > Example: Game is World of Warcraft style MMO but you're trying to do it in > the browser so can't download gigabytes. So you must shard each zone into, > say, a 50 megabyte downloadable size. Zone shards are static so can be > files. Private player data (e.g. inventory, raid history, message log) is > maybe 1 megabyte; public player data (e.g., x/y/z coordinates, avatar skin) > is maybe 30K and shared across the entire world (but usually limited to a > shard), needs fast access though as the world ticks at 30 frames per second > and you need to do distance calculations, etc. Store the public player data > in memory, dets (per shard) or redis depending on whether you want raw > speed, resiliency, or tooling. Store the private data in memory, dets, > redis, or mysql, depending on speed, resiliency, tooling, cost and analytics > requirements. > Generally I use ets for fast-access random-lookup state that has to get > shared between processes, redis for fast-enough (still plenty fast) > random-lookup state that has to be able to get large and/or survive system > crashes or writeoffs (via slave replication and aof, rather than > clustering), mysql for mass market mediocre state storage where the client > has pre-existing infrastructure and ad hoc querying is important, postgres > for green fields state storage where ad hoc querying is important, and > thankfully so far I have been able to avoid having to use inconsistent data > stores in production. I wouldn't ever consider dets or mnesia because > equivalently-functional tools exist that are radically more popular and have > better tooling, capability, characteristics or safety guarantees. > F. > > > On Tue, Jul 15, 2014 at 10:03 AM, Lloyd R. Prentice > wrote: >> >> >> I'm thinking through the design of a web-based multiple-player game. Each >> player maintains a private database--- most likely dets. Each player moves >> through nested finite state machines. Each player may be logging in or out >> at any state of play. >> >> At this point my grasp of Erlang architecture breaks down. I don't >> understand how best to: >> >> - parse game structure across players, processes, and directories >> - maintain player state between sessions >> >> At this point in my Erlang education I feel like I know the words but >> can't play the music. >> >> I'd much grateful for any and all ideas and suggestions. >> >> Many thanks, >> >> LRP >> >> Sent from my iPad >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From g@REDACTED Wed Jul 16 05:19:10 2014 From: g@REDACTED (Garrett Smith) Date: Tue, 15 Jul 2014 21:19:10 -0600 Subject: [erlang-questions] What is best way to maintain state in a web-based multiplayer game? In-Reply-To: <1405449778.198729759@apps.rackspace.com> References: <54A8BBD3-428E-4140-A40C-5D4CEB1F1546@writersglen.com> <1405449778.198729759@apps.rackspace.com> Message-ID: On Tue, Jul 15, 2014 at 12:42 PM, wrote: > Hi Felix, > > Thanks for helping me think this through. > > - Players do not interact. Each player is playing against the "real world," > e.g. results of their actions. Think of it as project planning with many > many contingencies. I'd say "expert system," but that sounds rather hoity > toity. I say "game" only in the sense that there are reward points for > successful action. > > Loop is basically: > > - gather facts > - make plan > - take action > - evaluate results This is a concrete actionable problem - build the loop. This is something you could build with a simple process loop/server just to see how things are working (Joe's book is a great resource for taking this lower level/direct approach). Later you can convert this over to a gen_server or e2 service, plugging it into a proper OTP app. But I would not worry about "proper" anything until you have made more progress on the core problem, which maybe you're describing in these bullet points. > - There are universal resources accessed by player state but these, as a > whole, are static text, links, and images; wild-eyed estimate: 10 megabytes I'd hack and slash this -- hard code stuff into source files -- just get it working. In the abstract you can serve things via named services very easily. > - I estimate that each player will store less than three megabytes of > private data. What's the easiest, most direct imaginable way to do this in Erlang? Do that. (hint: dets) > - We're not dealing with elaborate graphics or animations. Only latency is a > logic tree or inference engine evaluating facts; e.g. (Looking at Erlang > eres/seres; but I may be able get away with something more simple minded). > > Latency, in other words, is a function of how many private and shared > database lookups are necessary to evaluate a set of condition -> action > rules. Effective state management should help me keep rule sets fairly > minimal. If you can get this stuff working in some fashion -- forget databases, keep it in memory, loaded from dev/test source files or config files that you can file:consult -- you can back your way into persistence/recovery later. And "latency optimization" problems even later. > Prospects for this game are in the 200K range, but I'm successful with low > hundreds of players. How about stating with one player? Then two. > Ad hoc querying is not necessary. I suspect though your problem is where to start. What's the most *obvious* thing that you should see working first? It's certainly not scaling to 200K anything. It's not latency. Is it the loop above? Something else? Forget design - stumble through! Erlang is great for that :) Garrett From ddosia@REDACTED Wed Jul 16 14:56:29 2014 From: ddosia@REDACTED (Daniil Churikov) Date: Wed, 16 Jul 2014 16:56:29 +0400 Subject: [erlang-questions] dialyzer and behaviours Message-ID: %% I apologize if you get this message second time, but initially I posted it through google-groups %% interface and discovered that it is not replicated to this mailing list. Is there any way to write a spec for dialyzer that will tell that I expect module of certain behaviour? Simplified example: -module(my_db) -callback table() -> iolist(). %% Instead of `Mod :: atom()' I want something that will reflect that only %% modules which implements `my_db' behaviour are allowed. -spec create(Mod :: atom(), Vals :: [iolist(), ...]) -> iolist(). create(Mod, Vals) -> Table = Mod:table(), ["INSERT INTO ", Table, " VALUES (", string:join(Vals, ", "), ");"]. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean@REDACTED Wed Jul 16 15:41:57 2014 From: sean@REDACTED (Sean Cribbs) Date: Wed, 16 Jul 2014 08:41:57 -0500 Subject: [erlang-questions] dialyzer and behaviours In-Reply-To: References: Message-ID: Your example will work if you are implementing some other module which uses my_db as its behavior: -module(db_impl). -behaviour(my_db). -export([table/0]). table -> ["hello, world"]. On Wed, Jul 16, 2014 at 7:56 AM, Daniil Churikov wrote: > %% I apologize if you get this message second time, but initially I posted > it through google-groups > %% interface and discovered that it is not replicated to this mailing list. > > Is there any way to write a spec for dialyzer that will tell that I expect > module of certain behaviour? > > Simplified example: > > -module(my_db) > > -callback table() -> iolist(). > > %% Instead of `Mod :: atom()' I want something that will reflect that only > %% modules which implements `my_db' behaviour are allowed. > -spec create(Mod :: atom(), Vals :: [iolist(), ...]) -> iolist(). > create(Mod, Vals) -> > Table = Mod:table(), > ["INSERT INTO ", Table, " VALUES (", string:join(Vals, ", "), ");"]. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Sean Cribbs Software Engineer Basho Technologies, Inc. http://basho.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ddosia@REDACTED Wed Jul 16 15:55:12 2014 From: ddosia@REDACTED (Daniil Churikov) Date: Wed, 16 Jul 2014 17:55:12 +0400 Subject: [erlang-questions] dialyzer and behaviours In-Reply-To: References: Message-ID: Thank you Sean for your response. The problem is, how to tell to dialyzer that `my_db:create(Mod, Vals)' is accepts only Mod's which implements `-behaviour(my_db)'? and not an arbitrary atom? 2014-07-16 17:41 GMT+04:00 Sean Cribbs : > Your example will work if you are implementing some other module which > uses my_db as its behavior: > > -module(db_impl). > -behaviour(my_db). > -export([table/0]). > > table -> ["hello, world"]. > > > On Wed, Jul 16, 2014 at 7:56 AM, Daniil Churikov wrote: > >> %% I apologize if you get this message second time, but initially I >> posted it through google-groups >> %% interface and discovered that it is not replicated to this mailing >> list. >> >> Is there any way to write a spec for dialyzer that will tell that I >> expect module of certain behaviour? >> >> Simplified example: >> >> -module(my_db) >> >> -callback table() -> iolist(). >> >> %% Instead of `Mod :: atom()' I want something that will reflect that only >> %% modules which implements `my_db' behaviour are allowed. >> -spec create(Mod :: atom(), Vals :: [iolist(), ...]) -> iolist(). >> create(Mod, Vals) -> >> Table = Mod:table(), >> ["INSERT INTO ", Table, " VALUES (", string:join(Vals, ", "), ");"]. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > Sean Cribbs > Software Engineer > Basho Technologies, Inc. > http://basho.com/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Wed Jul 16 16:15:49 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 16 Jul 2014 10:15:49 -0400 Subject: [erlang-questions] dialyzer and behaviours In-Reply-To: References: Message-ID: <20140716141548.GD89097@ferdair.local> On 07/16, Daniil Churikov wrote: > Thank you Sean for your response. The problem is, how to tell to dialyzer > that `my_db:create(Mod, Vals)' > is accepts only Mod's which implements `-behaviour(my_db)'? and not an > arbitrary atom? > As far as I know, there's no good way to do that in Erlang. Just document it, and blame the caller when they don't implement the behaviour properly (their stuff will crash, or they'll get a compile warning when the behaviour is missing functions). Regards, Fred. From lloyd@REDACTED Wed Jul 16 17:50:22 2014 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Wed, 16 Jul 2014 11:50:22 -0400 Subject: [erlang-questions] What is best way to maintain state in a web-based multiplayer game? In-Reply-To: References: <54A8BBD3-428E-4140-A40C-5D4CEB1F1546@writersglen.com> <1405449778.198729759@apps.rackspace.com> Message-ID: <1F5F502A-287D-4CA8-9E3C-1B92A7EFC9F2@writersglen.com> Hi Knut, Your experience is truly impressive! I just reviewed the Wooga site and many of the slides on the backend. An education in itself. I misspoke, I think, when I called my project a multiplayer game. It's really a solitary game played, I hope, by many. I say game, but it's really serious business from the perspective of the player. Thus, the simple case that you noted would apply. I doubt that I will have the latency issues confronted by many of the Wooga games. Player state data, on the other hand, will be precious to the player and must be protected like Crown Jewels at my end. I'll also be providing a set of tools that must work together seamlessly. Knut, I much appreciate your thoughtful response. I hope I can call on you if I paint myself into a corner. All the best, Lloyd Sent from my iPad > On Jul 15, 2014, at 4:38 PM, Knut Nesheim wrote: > > Hey, > > I've had the great pleasure to build a few such systems and help out > other teams with their systems. Some of my colleagues and me have > presented about this exact topic over the last few years at the Erlang > Factory in London, San Francisco, Berlin and the Erlang User > Conference in Stockholm. Here's one such presentation: > http://www.erlang-factory.com/upload/presentations/469/wooga-euc2011-v2-111103095416-phpapp01.pdf > > If each user can only affect state owned by that user, you have your > parallelism and state management right there. Simplest case is to > store all state in a database (relational or not) and on every > request, load up all or parts, mutate it and write it back. > > If you're expecting intensive (1 request every few seconds) and last > long (minutes to hours) it makes sense to try to be smarter. You could > for example load up the user state at session start in an > (unsupervised) Erlang process, mutate it in memory (using purely > functional code and plain Erlang data structures) and when the session > ends write the full state back and stop the process. You could do this > with any of the database options: relational, document or plain > key-value. If you want to make life easy for yourself, going for a > service provided by let's say Amazon like S3 would be a great choice. > Scaling the application servers are then mostly about coordinating > sessions (static sharding or distributed coordination, we eventually > ended up writing a distributed lock: https://github.com/wooga/locker, > presentation here: > http://www.erlang-factory.com/conference/SFBay2013/speakers/KnutNesheim) > Pro tip: if using plain key-value, use a serialization format easy to > digest by other tools, such a json or protobuffers or whatever floats > your boat. > > If your game is a multiplayer game where each user can affect the > state of other users (or a shared world), the problem is much more > difficult. I've had the pleasure to work on one such game. Each user > was separate, but users could join the world of their friends and mess > around. This is a much easier problem than the > one-single-huge-persistent world problem. We solved this by having one > Erlang process for the world of a user, one process for each user and > a client that understood how to roll back state updates (needed for > conflict resolution). As the world is single-threaded and processes > events in the order they arrive, that's the order that is used to > resolve conflicting updates. Ie, if we both pick up an apple from the > ground at the same time we will see the UI of the game updating, but > only the user that reaches the server first (ie faster network) will > actually receive the apple and when the second user reaches the server > there's no apple on the ground any longer. The client needs to figure > out a graceful way of rolling back the state (to avoid a full reload) > in a nice way (to avoid pissing off users). We also presented this > system: see https://www.youtube.com/watch?v=AUeFutXPojU and > http://techmeshconf.com/techmesh-london-2012/presentation/You%20are%20not%20Alone%20-%20Scaling%20Multiplayer%20Games%20for%20the%20Web > > In the case of MMO, RTS or FPS games they use the same concept as > outlined above, but with some important variations. The worlds are > always sharded, sometimes this is visible to the user (you need to > join a particular "instance" of a world, you pick a map to play, the > world is restricted to only one round, there's a limit on players, the > game gets slow when many players join, etc) or it's not explicitly > visible, but smoothed over as "you can't see or affect users really > really far away" and when you move there, you're actually moving > shards. Conflict resolution is very important, you want clients to > concurrently proceed and only later resolve the conflict, if any. > Conflict in this case might be player death in a FPS game, in which > case you need consider the effect of latency on player states. For > example, if I'm moving in a figure 8 shape users looking at me will > see me at different places at exactly the same wallclock time > depending on the latency of their connection. If a user shoots me, it > might be that in their version of the world the game engine considers > it a hit, but in my version of the world it was a miss. One way to > solve this is that the server keeps track of the game state of each > client and runs the game logic with the state the shooting client > should have seen. > > It's a very interesting topic! Good luck with your game! > > Knut > > >> On Tue, Jul 15, 2014 at 8:42 PM, wrote: >> Hi Felix, >> >> >> >> Thanks for helping me think this through. >> >> >> >> - Players do not interact. Each player is playing against the "real world," >> e.g. results of their actions. Think of it as project planning with many >> many contingencies. I'd say "expert system," but that sounds rather hoity >> toity. I say "game" only in the sense that there are reward points for >> successful action. >> >> >> >> Loop is basically: >> >> >> >> - gather facts >> >> - make plan >> >> - take action >> >> - evaluate results >> >> >> >> - There are universal resources accessed by player state but these, as a >> whole, are static text, links, and images; wild-eyed estimate: 10 megabytes >> >> - I estimate that each player will store less than three megabytes of >> private data. >> >> >> >> - We're not dealing with elaborate graphics or animations. Only latency is a >> logic tree or inference engine evaluating facts; e.g. (Looking at Erlang >> eres/seres; but I may be able get away with something more simple minded). >> >> >> >> Latency, in other words, is a function of how many private and shared >> database lookups are necessary to evaluate a set of condition -> action >> rules. Effective state management should help me keep rule sets fairly >> minimal. >> >> >> >> Prospects for this game are in the 200K range, but I'm successful with low >> hundreds of players. >> >> >> >> Ad hoc querying is not necessary. >> >> >> >> Thanks again for your help. >> >> >> >> Lloyd >> >> >> >> >> >> >> >> -----Original Message----- >> From: "Felix Gallo" >> Sent: Tuesday, July 15, 2014 1:50pm >> To: "Lloyd R. Prentice" >> Cc: "Erlang-Questions Questions" >> Subject: Re: [erlang-questions] What is best way to maintain state in a >> web-based multiplayer game? >> >> I think you have some more marble to cut away before the shape of the >> problem begins to reveal itself. >> * do the players interact with each other directly? If so, what state do >> they share and how much? Is there a separation between 'state for everyone' >> (e.g. world of warcraft: the entire world geometry and all 3d models are >> gigabytes of static data that are stored on the local filesystem and not >> kept 'live' in memory) and 'player state'? What is the size and nature of >> each type of state? >> * what are the constraints around latency -- does it look more like a real >> time game (e.g. street fighter, counterstrike, league of legends) where, >> e.g., 250 ms of latency is unacceptable to the experience, or more like a >> turn based game (e.g. chess, poker, backgammon) where latency of 3s is >> tolerable? This drives certain considerations; e.g., you may be able to use >> a slightly lower performance but more reliable database system for the >> latter case. >> >> * what is the shape and nature of your cost envelope? If you are running >> server infrastructure for a game, then you may need to take special measures >> to ensure that your costs remain beneath your gross profit. >> Since, as you say, managing game state is such a giant pain in the ass, my >> general best practice when I do exactly this thing as part of my day job is >> to first write down all of the different kinds of state, their approximate >> quantity in bytes per player/world/universe/shard/whatever, and then do some >> spreadsheet math to figure out what I can get away with. >> Example: Game is Facebook-style web game like Farmville. State can be >> boiled down to a binary data structure that is 4 kilobytes in size per >> player. Expected profit per user is on the order of pennies. Tens of >> millions of users expected. Solution: save state in the user's own browser >> via encrypted cookie, use tiny database (sqlite, mysql, etc.) for >> leaderboards and sending seeds between players. >> Example: Game is World of Warcraft style MMO but you're trying to do it in >> the browser so can't download gigabytes. So you must shard each zone into, >> say, a 50 megabyte downloadable size. Zone shards are static so can be >> files. Private player data (e.g. inventory, raid history, message log) is >> maybe 1 megabyte; public player data (e.g., x/y/z coordinates, avatar skin) >> is maybe 30K and shared across the entire world (but usually limited to a >> shard), needs fast access though as the world ticks at 30 frames per second >> and you need to do distance calculations, etc. Store the public player data >> in memory, dets (per shard) or redis depending on whether you want raw >> speed, resiliency, or tooling. Store the private data in memory, dets, >> redis, or mysql, depending on speed, resiliency, tooling, cost and analytics >> requirements. >> Generally I use ets for fast-access random-lookup state that has to get >> shared between processes, redis for fast-enough (still plenty fast) >> random-lookup state that has to be able to get large and/or survive system >> crashes or writeoffs (via slave replication and aof, rather than >> clustering), mysql for mass market mediocre state storage where the client >> has pre-existing infrastructure and ad hoc querying is important, postgres >> for green fields state storage where ad hoc querying is important, and >> thankfully so far I have been able to avoid having to use inconsistent data >> stores in production. I wouldn't ever consider dets or mnesia because >> equivalently-functional tools exist that are radically more popular and have >> better tooling, capability, characteristics or safety guarantees. >> F. >> >> >> On Tue, Jul 15, 2014 at 10:03 AM, Lloyd R. Prentice >> wrote: >>> >>> >>> I'm thinking through the design of a web-based multiple-player game. Each >>> player maintains a private database--- most likely dets. Each player moves >>> through nested finite state machines. Each player may be logging in or out >>> at any state of play. >>> >>> At this point my grasp of Erlang architecture breaks down. I don't >>> understand how best to: >>> >>> - parse game structure across players, processes, and directories >>> - maintain player state between sessions >>> >>> At this point in my Erlang education I feel like I know the words but >>> can't play the music. >>> >>> I'd much grateful for any and all ideas and suggestions. >>> >>> Many thanks, >>> >>> LRP >>> >>> Sent from my iPad >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From lloyd@REDACTED Wed Jul 16 18:04:09 2014 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Wed, 16 Jul 2014 12:04:09 -0400 Subject: [erlang-questions] What is best way to maintain state in a web-based multiplayer game? In-Reply-To: References: <54A8BBD3-428E-4140-A40C-5D4CEB1F1546@writersglen.com> <1405449778.198729759@apps.rackspace.com> Message-ID: Ah, Garrett, you are so damned smart! You hit the nail dead on the head: "I suspect though your problem is where to start." I can see most of the pieces, but they are many and each distracts in my mind from the other. This, I suppose, is why I have been obsessed with the idea of architecture; e.g. How the pieces fit together. But your slice-and-dice the big problems into little problems makes imminent good sense. I shall do exactly that. Thank you for dispersing one layer of fog. All the best, Lloyd Sent from my iPad > On Jul 15, 2014, at 11:19 PM, Garrett Smith wrote: > >> On Tue, Jul 15, 2014 at 12:42 PM, wrote: >> Hi Felix, >> >> Thanks for helping me think this through. >> >> - Players do not interact. Each player is playing against the "real world," >> e.g. results of their actions. Think of it as project planning with many >> many contingencies. I'd say "expert system," but that sounds rather hoity >> toity. I say "game" only in the sense that there are reward points for >> successful action. >> >> Loop is basically: >> >> - gather facts >> - make plan >> - take action >> - evaluate results > > This is a concrete actionable problem - build the loop. This is > something you could build with a simple process loop/server just to > see how things are working (Joe's book is a great resource for taking > this lower level/direct approach). Later you can convert this over to > a gen_server or e2 service, plugging it into a proper OTP app. > > But I would not worry about "proper" anything until you have made more > progress on the core problem, which maybe you're describing in these > bullet points. > >> - There are universal resources accessed by player state but these, as a >> whole, are static text, links, and images; wild-eyed estimate: 10 megabytes > > I'd hack and slash this -- hard code stuff into source files -- just > get it working. > > In the abstract you can serve things via named services very easily. > >> - I estimate that each player will store less than three megabytes of >> private data. > > What's the easiest, most direct imaginable way to do this in Erlang? > Do that. (hint: dets) > >> - We're not dealing with elaborate graphics or animations. Only latency is a >> logic tree or inference engine evaluating facts; e.g. (Looking at Erlang >> eres/seres; but I may be able get away with something more simple minded). >> >> Latency, in other words, is a function of how many private and shared >> database lookups are necessary to evaluate a set of condition -> action >> rules. Effective state management should help me keep rule sets fairly >> minimal. > > If you can get this stuff working in some fashion -- forget databases, > keep it in memory, loaded from dev/test source files or config files > that you can file:consult -- you can back your way into > persistence/recovery later. And "latency optimization" problems even > later. > >> Prospects for this game are in the 200K range, but I'm successful with low >> hundreds of players. > > How about stating with one player? Then two. > >> Ad hoc querying is not necessary. > > I suspect though your problem is where to start. What's the most > *obvious* thing that you should see working first? It's certainly not > scaling to 200K anything. It's not latency. Is it the loop above? > Something else? > > Forget design - stumble through! Erlang is great for that :) > > Garrett From ok@REDACTED Thu Jul 17 03:03:00 2014 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 17 Jul 2014 13:03:00 +1200 Subject: [erlang-questions] dialyzer and behaviours In-Reply-To: References: Message-ID: On 17/07/2014, at 12:56 AM, Daniil Churikov wrote: > Is there any way to write a spec for dialyzer that will tell that I expect module of certain behaviour? The types that the Dialyzer works with have no connection with the *state* of the Erlang system; it can only describe the shape of a term in and of itself. Types do not vary with time. You cannot have "registered process name" as a type. You cannot have "defined key in the process dictionary" as a type. You cannot have "loaded module" as a type. You cannot have "open port" as a type. Whether a module conforms to a behaviour is something that can change either way when a new version of the module is loaded. Being time-dependent, it can't be part of a type. This is not to say that *some* verification tool not utterly unlike a type checker might not be possible that could get closer to what you want. It's just that time-dependent properties are not commonly thought of as 'types'. Even in the languages I know about with dependent types, it's value dependency not time dependency that's in question. From siraaj@REDACTED Thu Jul 17 07:12:47 2014 From: siraaj@REDACTED (Siraaj Khandkar) Date: Thu, 17 Jul 2014 01:12:47 -0400 Subject: [erlang-questions] dialyzer and behaviours In-Reply-To: References: Message-ID: <53C75B4F.5050001@khandkar.net> On 07/16/2014 09:03 PM, Richard A. O'Keefe wrote: > > On 17/07/2014, at 12:56 AM, Daniil Churikov wrote: >> Is there any way to write a spec for dialyzer that will tell that I expect module of certain behaviour? > > The types that the Dialyzer works with have no connection > with the *state* of the Erlang system; it can only describe > the shape of a term in and of itself. Types do not vary > with time. > > You cannot have "registered process name" as a type. > You cannot have "defined key in the process dictionary" as a type. > You cannot have "loaded module" as a type. > You cannot have "open port" as a type. > > Whether a module conforms to a behaviour is something that > can change either way when a new version of the module is > loaded. Being time-dependent, it can't be part of a type. > > This is not to say that *some* verification tool not utterly > unlike a type checker might not be possible that could get > closer to what you want. It's just that time-dependent > properties are not commonly thought of as 'types'. Even in > the languages I know about with dependent types, it's > value dependency not time dependency that's in question. > Richard, what I think Daniil is asking for is only time-depended in a language like Erlang. In a statically typed language with a module system it'd be something like: signature MY_DB = sig val table : unit -> iolist end functor Foo (Db_impl : MY_DB) = struct ... end Daniil, is that what you had in mind? Unfortunately this is not possible in Erlang, because, as Richard said, calls to module names in Erlang are like calls to mutable, global reference cells. The only thing statically checkable is that the module name is of type atom(), while the implementation of a named module is always swappable at runtime. From ahe.sanath@REDACTED Thu Jul 17 08:07:05 2014 From: ahe.sanath@REDACTED (Sanath Prasanna) Date: Thu, 17 Jul 2014 11:37:05 +0530 Subject: [erlang-questions] Erlang phash == function in PHP? Message-ID: Hi all, Is there function in php, which is equal to Erlang phash function? Basically I write data to Redis db nodes through Erlang & Read data from Redis through PHP application. Br, Sanath -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Thu Jul 17 09:08:43 2014 From: roger@REDACTED (Roger Lipscombe) Date: Thu, 17 Jul 2014 08:08:43 +0100 Subject: [erlang-questions] ssl certificate verification in httpc In-Reply-To: References: <53C50941.5030408@llaisdy.com> <53C51026.7070207@llaisdy.com> Message-ID: SSL encryption is done by the client creating a session key, and using the server's public key (from the server's certificate) to securely send it to the server. If you don't verify the server certificate, you have no way of knowing whether you're subject to a man-in-the-middle attack, where the attacker can give you a fake certificate, extract the session key, read/manipulate all your traffic, and then (optionally) send it to the real server. Encryption is only half of the story. SSL also aims to guarantee safe key exchange, but to do that, you need to verify the server certificate. This means verifying that the names match, that it's trusted (or issued by a trusted CA), and that it hasn't expired, been revoked, etc. And there's more than one way (also including DNS spoofing) to intercept the traffic: transparent proxies, pwned WiFi router, etc.. On 15 July 2014 15:41, Camille Troillard wrote: > Hi Robert, > >> SSL essentially does two things at once: encrypts the data and checks if client and/or server are who they say they are. The latter is where certificate verification comes into play, the encryption part is always done and usually automatically negotiated between client and server. >> >> So, if all you are aiming for is encrypting the data travelling between client and server, then you don't need the ssl option. Just point your httpc:request at an "https://..." URL and the encryption is handled for you without you having to do anything more. > > I think this works only if you trust your DNS, otherwise you have to check that the certificate matches the host. > Please correct me if I?m wrong. > > Cam > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ddosia@REDACTED Thu Jul 17 09:46:18 2014 From: ddosia@REDACTED (Daniil Churikov) Date: Thu, 17 Jul 2014 11:46:18 +0400 Subject: [erlang-questions] dialyzer and behaviours In-Reply-To: References: Message-ID: (Sorry, initially replied directly) Replying inline > Whether a module conforms to a behaviour is something that > can change either way when a new version of the module is > loaded. Being time-dependent, it can't be part of a type. Whether a function returns an iolist or an atom can change with a next version of code too. But you expect that dialyzer run will reveal this spec incompatibility. So if next version of the module will not implement behaviour it used to, why not to notify about this? I understand that this will not guarantee anything in runtime, you still could load anything you want, but at least it could guarantee me certain level of safety at dialyzer-run time. 2014-07-17 5:03 GMT+04:00 Richard A. O'Keefe : > > On 17/07/2014, at 12:56 AM, Daniil Churikov wrote: > > Is there any way to write a spec for dialyzer that will tell that I > expect module of certain behaviour? > > The types that the Dialyzer works with have no connection > with the *state* of the Erlang system; it can only describe > the shape of a term in and of itself. Types do not vary > with time. > > You cannot have "registered process name" as a type. > You cannot have "defined key in the process dictionary" as a type. > You cannot have "loaded module" as a type. > You cannot have "open port" as a type. > > Whether a module conforms to a behaviour is something that > can change either way when a new version of the module is > loaded. Being time-dependent, it can't be part of a type. > > This is not to say that *some* verification tool not utterly > unlike a type checker might not be possible that could get > closer to what you want. It's just that time-dependent > properties are not commonly thought of as 'types'. Even in > the languages I know about with dependent types, it's > value dependency not time dependency that's in question. > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ddosia@REDACTED Thu Jul 17 09:47:53 2014 From: ddosia@REDACTED (Daniil Churikov) Date: Thu, 17 Jul 2014 11:47:53 +0400 Subject: [erlang-questions] dialyzer and behaviours In-Reply-To: <53C75B4F.5050001@khandkar.net> References: <53C75B4F.5050001@khandkar.net> Message-ID: Thanks Siraaj, this is pretty close to what I meant. 2014-07-17 9:12 GMT+04:00 Siraaj Khandkar : > > > On 07/16/2014 09:03 PM, Richard A. O'Keefe wrote: > > > > On 17/07/2014, at 12:56 AM, Daniil Churikov wrote: > >> Is there any way to write a spec for dialyzer that will tell that I > expect module of certain behaviour? > > > > The types that the Dialyzer works with have no connection > > with the *state* of the Erlang system; it can only describe > > the shape of a term in and of itself. Types do not vary > > with time. > > > > You cannot have "registered process name" as a type. > > You cannot have "defined key in the process dictionary" as a type. > > You cannot have "loaded module" as a type. > > You cannot have "open port" as a type. > > > > Whether a module conforms to a behaviour is something that > > can change either way when a new version of the module is > > loaded. Being time-dependent, it can't be part of a type. > > > > This is not to say that *some* verification tool not utterly > > unlike a type checker might not be possible that could get > > closer to what you want. It's just that time-dependent > > properties are not commonly thought of as 'types'. Even in > > the languages I know about with dependent types, it's > > value dependency not time dependency that's in question. > > > > Richard, what I think Daniil is asking for is only time-depended in a > language like Erlang. In a statically typed language with a module > system it'd be something like: > > signature MY_DB = > sig > val table : unit -> iolist > end > > functor Foo (Db_impl : MY_DB) = > struct > ... > end > > Daniil, is that what you had in mind? Unfortunately this is not possible > in Erlang, because, as Richard said, calls to module names in Erlang are > like calls to mutable, global reference cells. The only thing statically > checkable is that the module name is of type atom(), while the > implementation of a named module is always swappable at runtime. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Thu Jul 17 10:18:17 2014 From: roger@REDACTED (Roger Lipscombe) Date: Thu, 17 Jul 2014 09:18:17 +0100 Subject: [erlang-questions] ssl certificate verification in httpc In-Reply-To: References: <53C50941.5030408@llaisdy.com> <53C51026.7070207@llaisdy.com> Message-ID: On 17 July 2014 08:42, Graham Hay wrote: > https://www.owasp.org/index.php/Certificate_and_Public_Key_Pinning Absolutely. Certificate pinning helps to mitigate the risk from "bad" CAs: either a compromised CA issues bogus certificates, or you've got a bogus CA installed in your root store. But it still doesn't detract from my point: **Always verify the server certificate** From ivan@REDACTED Thu Jul 17 11:30:10 2014 From: ivan@REDACTED (Ivan Uemlianin) Date: Thu, 17 Jul 2014 10:30:10 +0100 Subject: [erlang-questions] ssl certificate verification in httpc In-Reply-To: References: <53C50941.5030408@llaisdy.com> <53C51026.7070207@llaisdy.com> Message-ID: <53C797A2.6030201@llaisdy.com> Dear Roger Thank you for your comments. Your comments take us back to my initial question: > In short, what am I missing to be able to verify this certificate? Just providing the CA certificate doesn't seem to be enough: > ... > SSLOpts = [{verify, verify_peer}, > {cacertfile,"certificates.crt"}], > httpc:request(get, {Url, []}, [{ssl,SSLOpts}], []). > > But this returns an unknown ca error: > > {error,{failed_connect,[{to_address,{"bla.org", 443}}, > {inet,[inet],{tls_alert,"unknown ca"}}]}} > 11:06:04.942 [error] SSL: certify: ssl_handshake.erl:1344: > Fatal error: unknown ca None of the other ssl options in the erlang ssl documentation look likely, and I have found no examples of verifying an ssl certification from erlang on the web. - are any of the other ssl options relevant here? Are there other undocumented ssl options? - failing that, it looks like I need to write a verify_fun and access parts of the OTPCertificate and Extension records. Is that correct, or is more required? - I have scoured the usual sources on the web but found nothing. If anyone knows of relevant work in this area please let me know. Thanks Ivan On 17/07/2014 08:08, Roger Lipscombe wrote: > SSL encryption is done by the client creating a session key, and using > the server's public key (from the server's certificate) to securely > send it to the server. If you don't verify the server certificate, you > have no way of knowing whether you're subject to a man-in-the-middle > attack, where the attacker can give you a fake certificate, extract > the session key, read/manipulate all your traffic, and then > (optionally) send it to the real server. > > Encryption is only half of the story. SSL also aims to guarantee safe > key exchange, but to do that, you need to verify the server > certificate. This means verifying that the names match, that it's > trusted (or issued by a trusted CA), and that it hasn't expired, been > revoked, etc. > > And there's more than one way (also including DNS spoofing) to > intercept the traffic: transparent proxies, pwned WiFi router, etc.. > > On 15 July 2014 15:41, Camille Troillard wrote: >> Hi Robert, >> >>> SSL essentially does two things at once: encrypts the data and checks if client and/or server are who they say they are. The latter is where certificate verification comes into play, the encryption part is always done and usually automatically negotiated between client and server. >>> >>> So, if all you are aiming for is encrypting the data travelling between client and server, then you don't need the ssl option. Just point your httpc:request at an "https://..." URL and the encryption is handled for you without you having to do anything more. >> >> I think this works only if you trust your DNS, otherwise you have to check that the certificate matches the host. >> Please correct me if I?m wrong. >> >> Cam >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From ahe.sanath@REDACTED Thu Jul 17 11:30:24 2014 From: ahe.sanath@REDACTED (Sanath Prasanna) Date: Thu, 17 Jul 2014 15:00:24 +0530 Subject: [erlang-questions] Erlang phash == function in PHP? In-Reply-To: References: Message-ID: Any update on this? On Thu, Jul 17, 2014 at 11:37 AM, Sanath Prasanna wrote: > Hi all, > Is there function in php, which is equal to Erlang phash function? > Basically I write data to Redis db nodes through Erlang & Read data from > Redis through PHP application. > Br, > Sanath > -------------- next part -------------- An HTML attachment was scrubbed... URL: From freeakk@REDACTED Thu Jul 17 11:48:49 2014 From: freeakk@REDACTED (Michael Uvarov) Date: Thu, 17 Jul 2014 11:48:49 +0200 Subject: [erlang-questions] Erlang phash == function in PHP? In-Reply-To: References: Message-ID: Hi, Why not to use md5 or sha1? If you want phash in php, look at the code of make_hash in < https://github.com/erlang/otp/blob/master/erts/emulator/beam/utils.c> On Thu, Jul 17, 2014 at 11:30 AM, Sanath Prasanna wrote: > Any update on this? > > > On Thu, Jul 17, 2014 at 11:37 AM, Sanath Prasanna > wrote: > >> Hi all, >> Is there function in php, which is equal to Erlang phash function? >> Basically I write data to Redis db nodes through Erlang & Read data from >> Redis through PHP application. >> Br, >> Sanath >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- ? ?????????, ?????? ??????. Best regards, Uvarov Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists@REDACTED Thu Jul 17 13:20:02 2014 From: lists@REDACTED (Camille Troillard) Date: Thu, 17 Jul 2014 13:20:02 +0200 Subject: [erlang-questions] ssl certificate verification in httpc In-Reply-To: <53C797A2.6030201@llaisdy.com> References: <53C50941.5030408@llaisdy.com> <53C51026.7070207@llaisdy.com> <53C797A2.6030201@llaisdy.com> Message-ID: <68FA2911-2857-49CF-AAB3-08C2D35344B9@tuli.pe> Hello Ivan, I have the same issue, here is what I came up to. This may not be correct and I would be very happy to discuss the issues anyone may find. My goal is to check the server certificate is valid using verify_peer, and also implement a check on the server hostname I connect to. Here are the SSL options I use: % I know the certificate used by the server requires a depth of 2, you may not need this option in your code. {ssl, [ {cacertfile, cacert_path()}, {depth, 2}, {verify_fun, ssl_verify_fun(?SERVER_HOSTNAME)}, {verify, verify_peer} ]} And the verify fun: ssl_verify_fun(Hostname) -> { fun (_, {bad_cert, _} = Reason, _) -> {fail, Reason}; (_, {extension, _}, UserState) -> {unknown, UserState}; (_, valid, UserState) -> {valid, UserState}; (Cert, valid_peer, UserState) -> TBSCert = Cert#'OTPCertificate'.tbsCertificate, Extensions = TBSCert#'OTPTBSCertificate'.extensions, case lists:keysearch(?'id-ce-subjectAltName', #'Extension'.extnID, Extensions) of {value, #'Extension'{extnValue = [{dNSName, Hostname}]}} -> {valid, UserState}; false -> {fail, invalid_certificate_hostname} end end, [] }. As you can see, I verify the host name matches the one I am connecting to. Best, Camille On 17 Jul 2014, at 11:30, Ivan Uemlianin wrote: > Dear Roger > > Thank you for your comments. > > Your comments take us back to my initial question: > > > In short, what am I missing to be able to verify this certificate? > > Just providing the CA certificate doesn't seem to be enough: > > > ... >> SSLOpts = [{verify, verify_peer}, >> {cacertfile,"certificates.crt"}], >> httpc:request(get, {Url, []}, [{ssl,SSLOpts}], []). >> >> But this returns an unknown ca error: >> >> {error,{failed_connect,[{to_address,{"bla.org", 443}}, >> {inet,[inet],{tls_alert,"unknown ca"}}]}} >> 11:06:04.942 [error] SSL: certify: ssl_handshake.erl:1344: > > Fatal error: unknown ca > > None of the other ssl options in the erlang ssl documentation look likely, and I have found no examples of verifying an ssl certification from erlang on the web. > > - are any of the other ssl options relevant here? Are there other undocumented ssl options? > > - failing that, it looks like I need to write a verify_fun and access parts of the OTPCertificate and Extension records. Is that correct, or is more required? > > - I have scoured the usual sources on the web but found nothing. If anyone knows of relevant work in this area please let me know. > > Thanks > > Ivan > > > On 17/07/2014 08:08, Roger Lipscombe wrote: >> SSL encryption is done by the client creating a session key, and using >> the server's public key (from the server's certificate) to securely >> send it to the server. If you don't verify the server certificate, you >> have no way of knowing whether you're subject to a man-in-the-middle >> attack, where the attacker can give you a fake certificate, extract >> the session key, read/manipulate all your traffic, and then >> (optionally) send it to the real server. >> >> Encryption is only half of the story. SSL also aims to guarantee safe >> key exchange, but to do that, you need to verify the server >> certificate. This means verifying that the names match, that it's >> trusted (or issued by a trusted CA), and that it hasn't expired, been >> revoked, etc. >> >> And there's more than one way (also including DNS spoofing) to >> intercept the traffic: transparent proxies, pwned WiFi router, etc.. >> >> On 15 July 2014 15:41, Camille Troillard wrote: >>> Hi Robert, >>> >>>> SSL essentially does two things at once: encrypts the data and checks if client and/or server are who they say they are. The latter is where certificate verification comes into play, the encryption part is always done and usually automatically negotiated between client and server. >>>> >>>> So, if all you are aiming for is encrypting the data travelling between client and server, then you don't need the ssl option. Just point your httpc:request at an "https://..." URL and the encryption is handled for you without you having to do anything more. >>> >>> I think this works only if you trust your DNS, otherwise you have to check that the certificate matches the host. >>> Please correct me if I?m wrong. >>> >>> Cam >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -- > ============================================================ > Ivan A. Uemlianin PhD > Llaisdy > Speech Technology Research and Development > > ivan@REDACTED > www.llaisdy.com > llaisdy.wordpress.com > github.com/llaisdy > www.linkedin.com/in/ivanuemlianin > > festina lente > ============================================================ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From sina.samv@REDACTED Thu Jul 17 15:28:44 2014 From: sina.samv@REDACTED (Sina Samavati) Date: Thu, 17 Jul 2014 17:58:44 +0430 Subject: [erlang-questions] A website to show popular erlang modules In-Reply-To: References: Message-ID: That's cool, nice job! I was thinking of something similar, but with the consideration of the possibility to write reviews, search, and rank. Because, sometimes there are still questions like "which one to use between this and that?" and/or "which one you prefer and why?" I suppose such features will probably make the community happier. Thanks, Sina On Tue, Jul 15, 2014 at 4:57 AM, Bin Wang wrote: > Hi, > > I've wrote this website some months ago. I will be happy if you find it > helpful. > > http://erlang-modules.binwang.me/ > > It will fetch popular Erlang projects on Github and see how many other > Erlang projects are depended on it. Details could be found here: > > http://www.binwang.me/2013-11-16-Fetch-Popular-Erlang-Modules-by-Coffee-Script.html > > --- > > Bin Wang > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Sina Samavati Software engineer https://github.com/s1n4 https://twitter.com/sinasamavati -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Thu Jul 17 15:31:53 2014 From: ivan@REDACTED (Ivan Uemlianin) Date: Thu, 17 Jul 2014 14:31:53 +0100 Subject: [erlang-questions] ssl certificate verification in httpc In-Reply-To: <68FA2911-2857-49CF-AAB3-08C2D35344B9@tuli.pe> References: <53C50941.5030408@llaisdy.com> <53C51026.7070207@llaisdy.com> <53C797A2.6030201@llaisdy.com> <68FA2911-2857-49CF-AAB3-08C2D35344B9@tuli.pe> Message-ID: <53C7D049.3060808@llaisdy.com> Dear Camille Thanks very much for this! I'll have a go with it this afternoon and report back. Best wishes Ivan On 17/07/2014 12:20, Camille Troillard wrote: > Hello Ivan, > > I have the same issue, here is what I came up to. This may not be correct and I would be very happy to discuss the issues anyone may find. My goal is to check the server certificate is valid using verify_peer, and also implement a check on the server hostname I connect to. > > Here are the SSL options I use: > > % I know the certificate used by the server requires a depth of 2, you may not need this option in your code. > > {ssl, [ > {cacertfile, cacert_path()}, > {depth, 2}, > {verify_fun, ssl_verify_fun(?SERVER_HOSTNAME)}, > {verify, verify_peer} > ]} > > And the verify fun: > > ssl_verify_fun(Hostname) -> > { > fun (_, {bad_cert, _} = Reason, _) -> > {fail, Reason}; > (_, {extension, _}, UserState) -> > {unknown, UserState}; > (_, valid, UserState) -> > {valid, UserState}; > (Cert, valid_peer, UserState) -> > TBSCert = Cert#'OTPCertificate'.tbsCertificate, > Extensions = TBSCert#'OTPTBSCertificate'.extensions, > case lists:keysearch(?'id-ce-subjectAltName', #'Extension'.extnID, Extensions) of > {value, #'Extension'{extnValue = [{dNSName, Hostname}]}} -> > {valid, UserState}; > false -> > {fail, invalid_certificate_hostname} > end > end, > [] > }. > > > As you can see, I verify the host name matches the one I am connecting to. > > > Best, > Camille > > > > > On 17 Jul 2014, at 11:30, Ivan Uemlianin wrote: > >> Dear Roger >> >> Thank you for your comments. >> >> Your comments take us back to my initial question: >> >>> In short, what am I missing to be able to verify this certificate? >> >> Just providing the CA certificate doesn't seem to be enough: >> >>> ... >>> SSLOpts = [{verify, verify_peer}, >>> {cacertfile,"certificates.crt"}], >>> httpc:request(get, {Url, []}, [{ssl,SSLOpts}], []). >>> >>> But this returns an unknown ca error: >>> >>> {error,{failed_connect,[{to_address,{"bla.org", 443}}, >>> {inet,[inet],{tls_alert,"unknown ca"}}]}} >>> 11:06:04.942 [error] SSL: certify: ssl_handshake.erl:1344: >>> Fatal error: unknown ca >> >> None of the other ssl options in the erlang ssl documentation look likely, and I have found no examples of verifying an ssl certification from erlang on the web. >> >> - are any of the other ssl options relevant here? Are there other undocumented ssl options? >> >> - failing that, it looks like I need to write a verify_fun and access parts of the OTPCertificate and Extension records. Is that correct, or is more required? >> >> - I have scoured the usual sources on the web but found nothing. If anyone knows of relevant work in this area please let me know. >> >> Thanks >> >> Ivan >> >> >> On 17/07/2014 08:08, Roger Lipscombe wrote: >>> SSL encryption is done by the client creating a session key, and using >>> the server's public key (from the server's certificate) to securely >>> send it to the server. If you don't verify the server certificate, you >>> have no way of knowing whether you're subject to a man-in-the-middle >>> attack, where the attacker can give you a fake certificate, extract >>> the session key, read/manipulate all your traffic, and then >>> (optionally) send it to the real server. >>> >>> Encryption is only half of the story. SSL also aims to guarantee safe >>> key exchange, but to do that, you need to verify the server >>> certificate. This means verifying that the names match, that it's >>> trusted (or issued by a trusted CA), and that it hasn't expired, been >>> revoked, etc. >>> >>> And there's more than one way (also including DNS spoofing) to >>> intercept the traffic: transparent proxies, pwned WiFi router, etc.. >>> >>> On 15 July 2014 15:41, Camille Troillard wrote: >>>> Hi Robert, >>>> >>>>> SSL essentially does two things at once: encrypts the data and checks if client and/or server are who they say they are. The latter is where certificate verification comes into play, the encryption part is always done and usually automatically negotiated between client and server. >>>>> >>>>> So, if all you are aiming for is encrypting the data travelling between client and server, then you don't need the ssl option. Just point your httpc:request at an "https://..." URL and the encryption is handled for you without you having to do anything more. >>>> >>>> I think this works only if you trust your DNS, otherwise you have to check that the certificate matches the host. >>>> Please correct me if I?m wrong. >>>> >>>> Cam >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> -- >> ============================================================ >> Ivan A. Uemlianin PhD >> Llaisdy >> Speech Technology Research and Development >> >> ivan@REDACTED >> www.llaisdy.com >> llaisdy.wordpress.com >> github.com/llaisdy >> www.linkedin.com/in/ivanuemlianin >> >> festina lente >> ============================================================ >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From essen@REDACTED Thu Jul 17 15:33:53 2014 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Thu, 17 Jul 2014 15:33:53 +0200 Subject: [erlang-questions] A website to show popular erlang modules In-Reply-To: References: Message-ID: <53C7D0C1.70707@ninenines.eu> Two things could make it better. Have 2 numbers: direct dependencies (what you have now) and total dependencies (including deps of deps of deps...). Have a togglable list of the projects that use it, with a link to the github. Food for thoughts. On 07/15/2014 02:27 AM, Bin Wang wrote: > Hi, > > I've wrote this website some months ago. I will be happy if you find it helpful. > > http://erlang-modules.binwang.me/ > > It will fetch popular Erlang projects on Github and see how many other > Erlang projects are depended on it. Details could be found here: > http://www.binwang.me/2013-11-16-Fetch-Popular-Erlang-Modules-by-Coffee-Script.html > > --- > > Bin Wang > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu From siraaj@REDACTED Thu Jul 17 15:57:08 2014 From: siraaj@REDACTED (Siraaj Khandkar) Date: Thu, 17 Jul 2014 09:57:08 -0400 Subject: [erlang-questions] dialyzer and behaviours In-Reply-To: References: Message-ID: <53C7D634.8060500@khandkar.net> On 07/17/2014 03:46 AM, Daniil Churikov wrote:> (Sorry, initially replied directly) > > Replying inline > >> Whether a module conforms to a behaviour is something that >> can change either way when a new version of the module is >> loaded. Being time-dependent, it can't be part of a type. > > Whether a function returns an iolist or an atom can change with > a next version of code too. But you expect that dialyzer run will > reveal this spec incompatibility. So if next version of the module > will not implement behaviour it used to, why not to notify about > this? It isn't so much about runtime possibilities but more about a lack of static capabilities given the semantics. The difference is in constructing a reference to a module (or function) statically or dynamically. The bellow reference to the module is not constructed statically, but dynamically, so at the time of analysis it does not actually exist: -spec create(Mod :: atom(), Vals :: [iolist(), ...]) -> iolist(). create(Mod, Vals) -> Table = Mod:table(), ... . In the face of such ambiguities, Dialyzer really has no choice but to assume innocence until guilt can be proven with certainty. When you say `baz = foo:bar()`, Dialyzer can _try_ to follow the path and match the expressions, but when you say `baz = Foo:bar()` - the rug is swept from underneath Dialyzer's feet and (at that call site) it can go no further than _try_ to find a case where Foo does not result in being an atom. So in other words, Dialyzer is not a type checker - it only seeks _definite_ failure cases in the call graph and sometimes success-or-failure depends _only_ on runtime state (such as dynamic references). > I understand that this will not guarantee anything in runtime, you > still could load anything you want, but at least it could guarantee > me certain level of safety at dialyzer-run time. > It isn't just about guarantees, it is just that Dialyzer has no concrete path to follow at the time it sees `Foo:bar()`. > 2014-07-17 5:03 GMT+04:00 Richard A. O'Keefe : > >> >> On 17/07/2014, at 12:56 AM, Daniil Churikov wrote: >>> Is there any way to write a spec for dialyzer that will tell that I >> expect module of certain behaviour? >> >> The types that the Dialyzer works with have no connection >> with the *state* of the Erlang system; it can only describe >> the shape of a term in and of itself. Types do not vary >> with time. >> >> You cannot have "registered process name" as a type. >> You cannot have "defined key in the process dictionary" as a type. >> You cannot have "loaded module" as a type. >> You cannot have "open port" as a type. >> >> Whether a module conforms to a behaviour is something that >> can change either way when a new version of the module is >> loaded. Being time-dependent, it can't be part of a type. >> >> This is not to say that *some* verification tool not utterly >> unlike a type checker might not be possible that could get >> closer to what you want. It's just that time-dependent >> properties are not commonly thought of as 'types'. Even in >> the languages I know about with dependent types, it's >> value dependency not time dependency that's in question. >> From mononcqc@REDACTED Thu Jul 17 16:29:07 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 17 Jul 2014 10:29:07 -0400 Subject: [erlang-questions] dialyzer and behaviours In-Reply-To: <53C7D634.8060500@khandkar.net> References: <53C7D634.8060500@khandkar.net> Message-ID: <20140717142906.GE89097@ferdair.local> On 07/17, Siraaj Khandkar wrote: > In the face of such ambiguities, Dialyzer really has no choice but to > assume innocence until guilt can be proven with certainty. > > When you say `baz = foo:bar()`, Dialyzer can _try_ to follow the path > and match the expressions, but when you say `baz = Foo:bar()` - the rug > is swept from underneath Dialyzer's feet and (at that call site) it can > go no further than _try_ to find a case where Foo does not result in > being an atom. > > So in other words, Dialyzer is not a type checker - it only seeks > _definite_ failure cases in the call graph and sometimes > success-or-failure depends _only_ on runtime state (such as dynamic > references). > Someone who knows better about types, type systems, and type checking than me may correct me, but I believe that Dialyzer still is a type checker. The difference is that it checks a type system that isn't axiomatic (such as Hindley-Milner) -- those that tend to *prove* that there is no type errors in the program, sometimes at the cost of forbidding certain otherwise valid programs. The canonical example for this is is: 'and'(true, true) -> true; 'and'(_, _) -> false. being called as: 'and'(5,6) A type inference for an axiomatic type system might decide that the 'and'/2 function's signature is -spec 'and'(boolean(), boolean()) -> boolean()' and therefore ban the call. Dialyzer can in fact ban this one if you explicitly write that signature down, but won't infer this. If you have no signature and call `$ typer `, the output is: -spec 'and'(_,_) -> boolean(). Erlang's semantics allow that call, and that is the correct type signature that will correspond to its behavior at runtime. So the reason for this is that Dialyzer type checks on a concept called 'success typing'. To simplify a lot (maybe too much), an axiomatic type checker will build its list of what is valid or not by starting with a null set (nothing is valid), and expanding it as it finds more information in your program. This leads to denying programs that are: 1. wrong all of the time for an execution path 2. wrong some of the time for an execution path (and therefore right some of the time) 3. do not provide sufficient information to know something will always be right (false negative) Success typing basically works the other way around. It starts assuming that everything is good, and then restricts what it thinks is valid as it finds more information. This leads to denying programs that are: 1. wrong all the time for an execution path Programs that may or may not fail, either due to lack of information or succeeding some of the time (or a combination of both) will be accepted by Dialyzer. Dialyzer still definitely checks your types, does analysis, does inference, etc. It just analyzes them differently to give you a different diagnostic, but for all intents and purposes, it is a type checker (on top of being able to find other discrepancies in your programs). Regards, Fred. From ivan@REDACTED Thu Jul 17 16:29:23 2014 From: ivan@REDACTED (Ivan Uemlianin) Date: Thu, 17 Jul 2014 15:29:23 +0100 Subject: [erlang-questions] ssl certificate verification in httpc In-Reply-To: <68FA2911-2857-49CF-AAB3-08C2D35344B9@tuli.pe> References: <53C50941.5030408@llaisdy.com> <53C51026.7070207@llaisdy.com> <53C797A2.6030201@llaisdy.com> <68FA2911-2857-49CF-AAB3-08C2D35344B9@tuli.pe> Message-ID: <53C7DDC3.8060600@llaisdy.com> Dear Cammille Thanks for your help. However, I'm still failing at bad_cert (unknown ca). It seems to be finding the cacertfile ok. ... I guess there's no alternative but to go in. If/when I get it working I'll report back. Best wishes Ivan On 17/07/2014 12:20, Camille Troillard wrote: > Hello Ivan, > > I have the same issue, here is what I came up to. This may not be correct and I would be very happy to discuss the issues anyone may find. My goal is to check the server certificate is valid using verify_peer, and also implement a check on the server hostname I connect to. > > Here are the SSL options I use: > > % I know the certificate used by the server requires a depth of 2, you may not need this option in your code. > > {ssl, [ > {cacertfile, cacert_path()}, > {depth, 2}, > {verify_fun, ssl_verify_fun(?SERVER_HOSTNAME)}, > {verify, verify_peer} > ]} > > And the verify fun: > > ssl_verify_fun(Hostname) -> > { > fun (_, {bad_cert, _} = Reason, _) -> > {fail, Reason}; > (_, {extension, _}, UserState) -> > {unknown, UserState}; > (_, valid, UserState) -> > {valid, UserState}; > (Cert, valid_peer, UserState) -> > TBSCert = Cert#'OTPCertificate'.tbsCertificate, > Extensions = TBSCert#'OTPTBSCertificate'.extensions, > case lists:keysearch(?'id-ce-subjectAltName', #'Extension'.extnID, Extensions) of > {value, #'Extension'{extnValue = [{dNSName, Hostname}]}} -> > {valid, UserState}; > false -> > {fail, invalid_certificate_hostname} > end > end, > [] > }. > > > As you can see, I verify the host name matches the one I am connecting to. > > > Best, > Camille > > > > > On 17 Jul 2014, at 11:30, Ivan Uemlianin wrote: > >> Dear Roger >> >> Thank you for your comments. >> >> Your comments take us back to my initial question: >> >>> In short, what am I missing to be able to verify this certificate? >> >> Just providing the CA certificate doesn't seem to be enough: >> >>> ... >>> SSLOpts = [{verify, verify_peer}, >>> {cacertfile,"certificates.crt"}], >>> httpc:request(get, {Url, []}, [{ssl,SSLOpts}], []). >>> >>> But this returns an unknown ca error: >>> >>> {error,{failed_connect,[{to_address,{"bla.org", 443}}, >>> {inet,[inet],{tls_alert,"unknown ca"}}]}} >>> 11:06:04.942 [error] SSL: certify: ssl_handshake.erl:1344: >>> Fatal error: unknown ca >> >> None of the other ssl options in the erlang ssl documentation look likely, and I have found no examples of verifying an ssl certification from erlang on the web. >> >> - are any of the other ssl options relevant here? Are there other undocumented ssl options? >> >> - failing that, it looks like I need to write a verify_fun and access parts of the OTPCertificate and Extension records. Is that correct, or is more required? >> >> - I have scoured the usual sources on the web but found nothing. If anyone knows of relevant work in this area please let me know. >> >> Thanks >> >> Ivan >> >> >> On 17/07/2014 08:08, Roger Lipscombe wrote: >>> SSL encryption is done by the client creating a session key, and using >>> the server's public key (from the server's certificate) to securely >>> send it to the server. If you don't verify the server certificate, you >>> have no way of knowing whether you're subject to a man-in-the-middle >>> attack, where the attacker can give you a fake certificate, extract >>> the session key, read/manipulate all your traffic, and then >>> (optionally) send it to the real server. >>> >>> Encryption is only half of the story. SSL also aims to guarantee safe >>> key exchange, but to do that, you need to verify the server >>> certificate. This means verifying that the names match, that it's >>> trusted (or issued by a trusted CA), and that it hasn't expired, been >>> revoked, etc. >>> >>> And there's more than one way (also including DNS spoofing) to >>> intercept the traffic: transparent proxies, pwned WiFi router, etc.. >>> >>> On 15 July 2014 15:41, Camille Troillard wrote: >>>> Hi Robert, >>>> >>>>> SSL essentially does two things at once: encrypts the data and checks if client and/or server are who they say they are. The latter is where certificate verification comes into play, the encryption part is always done and usually automatically negotiated between client and server. >>>>> >>>>> So, if all you are aiming for is encrypting the data travelling between client and server, then you don't need the ssl option. Just point your httpc:request at an "https://..." URL and the encryption is handled for you without you having to do anything more. >>>> >>>> I think this works only if you trust your DNS, otherwise you have to check that the certificate matches the host. >>>> Please correct me if I?m wrong. >>>> >>>> Cam >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> -- >> ============================================================ >> Ivan A. Uemlianin PhD >> Llaisdy >> Speech Technology Research and Development >> >> ivan@REDACTED >> www.llaisdy.com >> llaisdy.wordpress.com >> github.com/llaisdy >> www.linkedin.com/in/ivanuemlianin >> >> festina lente >> ============================================================ >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From shayan@REDACTED Thu Jul 17 22:42:48 2014 From: shayan@REDACTED (Shayan Pooya) Date: Thu, 17 Jul 2014 16:42:48 -0400 Subject: [erlang-questions] Using ram-only mnesia in place of gen_server+ets table Message-ID: Disco filesystem gc involves a step in which a process builds an ets table on the master and the slave nodes query this ets table regularly afterwards. The ets table is owned by a gen_server which responds to the queries from the slaves using this ets table. I am experimenting with using a mnesia table instead of this ets table so that the gen_server is not involved in this operation (There are a lot of queries and this master process is a bottleneck). This is a ram-only mnesia table which is created and then made read-only. All of the operations use mnesia:dirty_* functions and the slave nodes never modify the table. 1. Is this going to avoid the gen_server bottleneck? 2. I am thinking of adding all of the nodes (master + slaves) as the ram-copies for this table in order to have local lookups on the slave nodes. Is this going to slow down the creation of the mnesia table? (as said before, it is only using dirty_* functions for modifying the table). -------------- next part -------------- An HTML attachment was scrubbed... URL: From sedrik@REDACTED Thu Jul 17 15:32:54 2014 From: sedrik@REDACTED (Fredrik Andersson) Date: Thu, 17 Jul 2014 15:32:54 +0200 Subject: [erlang-questions] A website to show popular erlang modules In-Reply-To: References: Message-ID: Really cool stuff. How did you find the projects that you analyze? Seems to me that there should be bigger numbers all around. Also I would recommend that you change the wordings to Erlang Applications instead of modules as modules is something else in the Erlang lingo. Great effort! On Thu, Jul 17, 2014 at 3:28 PM, Sina Samavati wrote: > That's cool, nice job! > > I was thinking of something similar, but with the consideration of the > possibility to write reviews, search, and rank. Because, sometimes there > are still questions like "which one to use between this and that?" and/or > "which one you prefer and why?" > I suppose such features will probably make the community happier. > > Thanks, > Sina > > > On Tue, Jul 15, 2014 at 4:57 AM, Bin Wang wrote: > >> Hi, >> >> I've wrote this website some months ago. I will be happy if you find it >> helpful. >> >> http://erlang-modules.binwang.me/ >> >> It will fetch popular Erlang projects on Github and see how many other >> Erlang projects are depended on it. Details could be found here: >> >> http://www.binwang.me/2013-11-16-Fetch-Popular-Erlang-Modules-by-Coffee-Script.html >> >> --- >> >> Bin Wang >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Sina Samavati > Software engineer > > https://github.com/s1n4 > https://twitter.com/sinasamavati > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From grahamrhay@REDACTED Thu Jul 17 09:42:43 2014 From: grahamrhay@REDACTED (Graham Hay) Date: Thu, 17 Jul 2014 08:42:43 +0100 Subject: [erlang-questions] ssl certificate verification in httpc In-Reply-To: References: <53C50941.5030408@llaisdy.com> <53C51026.7070207@llaisdy.com> Message-ID: https://www.owasp.org/index.php/Certificate_and_Public_Key_Pinning On 17 July 2014 08:08, Roger Lipscombe wrote: > SSL encryption is done by the client creating a session key, and using > the server's public key (from the server's certificate) to securely > send it to the server. If you don't verify the server certificate, you > have no way of knowing whether you're subject to a man-in-the-middle > attack, where the attacker can give you a fake certificate, extract > the session key, read/manipulate all your traffic, and then > (optionally) send it to the real server. > > Encryption is only half of the story. SSL also aims to guarantee safe > key exchange, but to do that, you need to verify the server > certificate. This means verifying that the names match, that it's > trusted (or issued by a trusted CA), and that it hasn't expired, been > revoked, etc. > > And there's more than one way (also including DNS spoofing) to > intercept the traffic: transparent proxies, pwned WiFi router, etc.. > > On 15 July 2014 15:41, Camille Troillard wrote: > > Hi Robert, > > > >> SSL essentially does two things at once: encrypts the data and checks > if client and/or server are who they say they are. The latter is where > certificate verification comes into play, the encryption part is always > done and usually automatically negotiated between client and server. > >> > >> So, if all you are aiming for is encrypting the data travelling between > client and server, then you don't need the ssl option. Just point your > httpc:request at an "https://..." URL and the encryption is handled for > you without you having to do anything more. > > > > I think this works only if you trust your DNS, otherwise you have to > check that the certificate matches the host. > > Please correct me if I?m wrong. > > > > Cam > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Thu Jul 17 19:44:10 2014 From: t@REDACTED (Tristan Sloughter) Date: Thu, 17 Jul 2014 12:44:10 -0500 Subject: [erlang-questions] OTP Version Message-ID: <1405619050.32590.142801741.733CC21C@webmail.messagingengine.com> If for all OTP 17.x.y the runtime is going to consider it 17: 1> erlang:system_info(otp_release). "17" Why do you say, "Erlang/OTP 17.1 has been released" I know this has been brought up before, but really odd to claim both OTP Release is 17 and 17.1, consistency either way would be nice. Not sure how you could say a new version of Erlang/OTP 17 is released without adding a minor version number to it... but if the minor version is considered not part of the 'otp_release' it shouldn't be there. - Tristan From kunthar@REDACTED Thu Jul 17 23:17:31 2014 From: kunthar@REDACTED (Gokhan Boranalp) Date: Fri, 18 Jul 2014 00:17:31 +0300 Subject: [erlang-questions] OTP Version In-Reply-To: <1405619050.32590.142801741.733CC21C@webmail.messagingengine.com> References: <1405619050.32590.142801741.733CC21C@webmail.messagingengine.com> Message-ID: It shouldn't be that complex. Please use the way of semver[1] [1] http://semver.org/ On Thu, Jul 17, 2014 at 8:44 PM, Tristan Sloughter wrote: > If for all OTP 17.x.y the runtime is going to consider it 17: > > 1> erlang:system_info(otp_release). > "17" > > Why do you say, "Erlang/OTP 17.1 has been released" > > I know this has been brought up before, but really odd to claim both OTP > Release is 17 and 17.1, consistency either way would be nice. Not sure > how you could say a new version of Erlang/OTP 17 is released without > adding a minor version number to it... but if the minor version is > considered not part of the 'otp_release' it shouldn't be there. > > - Tristan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- BR, \|/ Kunthar -------------- next part -------------- An HTML attachment was scrubbed... URL: From ludovic@REDACTED Thu Jul 17 23:21:39 2014 From: ludovic@REDACTED (Ludovic Demblans) Date: Thu, 17 Jul 2014 23:21:39 +0200 Subject: [erlang-questions] Using ram-only mnesia in place of gen_server+ets table In-Reply-To: References: Message-ID: Hi, If you plan to use dirty read-only acces, (and if this is the only problem), any process can acces the table data if it is in 'protected' mode (which is the default. Cheers lud Le Thu, 17 Jul 2014 22:42:48 +0200, Shayan Pooya a ?crit: > Disco filesystem gc involves a step in which a process builds an ets > table > on the master and > the slave nodes query this ets table regularly afterwards. > The ets table is owned by a gen_server which responds to the queries from > the > slaves using this ets table. > > I am experimenting with using a mnesia table instead of this ets table so > that > the gen_server is not involved in this operation (There are a lot of > queries and > this master process is a bottleneck). > This is a ram-only mnesia table which is created and then made read-only. > All of > the operations use mnesia:dirty_* functions and the slave nodes never > modify the > table. > > 1. Is this going to avoid the gen_server bottleneck? > 2. I am thinking of adding all of the nodes (master + slaves) as the > ram-copies for > this table in order to have local lookups on the slave nodes. > Is this going to slow down the creation of the mnesia table? (as > said before, it is only using dirty_* functions for modifying the table). -- Utilisant le logiciel de courrier r?volutionnaire d'Opera : http://www.opera.com/mail/ From mononcqc@REDACTED Thu Jul 17 23:21:44 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 17 Jul 2014 17:21:44 -0400 Subject: [erlang-questions] OTP Version In-Reply-To: <1405619050.32590.142801741.733CC21C@webmail.messagingengine.com> References: <1405619050.32590.142801741.733CC21C@webmail.messagingengine.com> Message-ID: <20140717212143.GE794@ferdmbp.local> On 07/17, Tristan Sloughter wrote: > If for all OTP 17.x.y the runtime is going to consider it 17: > > 1> erlang:system_info(otp_release). > "17" > For reverence, the new method to get the version according to the docs is: http://www.erlang.org/doc/system_principles/versions.html In an OTP source code tree, the OTP version can be read from the text file /OTP_VERSION. The absolute path to the file can be constructed by calling filename:join([code:root_dir(), "OTP_VERSION"]). In an installed OTP development system, the OTP version can be read from the text file /releases//OTP_VERSION. The absolute path to the file can by constructed by calling filename:join([code:root_dir(), "releases", erlang:system_info(otp_release), "OTP_VERSION"]). Do note that the version number can be followed by '**', meaning it was patched with otp_patch_apply. From richard.youngkin@REDACTED Thu Jul 17 23:32:54 2014 From: richard.youngkin@REDACTED (Youngkin, Rich) Date: Thu, 17 Jul 2014 15:32:54 -0600 Subject: [erlang-questions] Prevent overwriting sasl log on VM restart? Message-ID: Is there a way, other than using the built-in log rotate mechanism or something like lager, to avoid having the sasl log overwritten when the VM is re/started? Thanks, Rich -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Thu Jul 17 23:35:34 2014 From: t@REDACTED (Tristan Sloughter) Date: Thu, 17 Jul 2014 16:35:34 -0500 Subject: [erlang-questions] runtime_dependencies part 2 Message-ID: <1405632934.26969.142876397.6358DFD1@webmail.messagingengine.com> I wrote the list about runtime_dependencies in .app files now in 17+ a couple months ago, http://www.erlang.org/doc/man/app.html Today I was informed that if you were to take all the runtime_dependencies defined for kernel-3.x and their runtime_dependencies you'd end up having wx as one, even for OTP built without wx support. Yes, I know the second warning on the app man page says that just because it is in there today doesn't mean it will be in the future and is actively being worked on. My question is, wouldn't it be nice if the OTP team made an EEP that the community could comment on for this? :) Tristan From lee.sylvester@REDACTED Thu Jul 17 23:47:41 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Thu, 17 Jul 2014 22:47:41 +0100 Subject: [erlang-questions] Elastic Erlang Message-ID: <08146F1A-D2C0-404D-AB0D-60B892186896@gmail.com> Hi guys, I was wondering, who out there in the list is using Erlang in an elastic cloud environment, and what platforms (hypervisor, monitoring, etc.) are you using? Would be good to see how people are deploying Erlang in real world situations. Cheers, Lee From raould@REDACTED Thu Jul 17 23:48:45 2014 From: raould@REDACTED (Raoul Duke) Date: Thu, 17 Jul 2014 14:48:45 -0700 Subject: [erlang-questions] non-erlang integration? Message-ID: when do people start to reach outside the erlang system and use things like cassandra or zookerper or whatever else? i'd be interested to hear how people have fared with such integrations, or have used erlang in some way to avoid having to do them at all. From mjtruog@REDACTED Fri Jul 18 00:43:07 2014 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 17 Jul 2014 15:43:07 -0700 Subject: [erlang-questions] OTP Version In-Reply-To: <20140717212143.GE794@ferdmbp.local> References: <1405619050.32590.142801741.733CC21C@webmail.messagingengine.com> <20140717212143.GE794@ferdmbp.local> Message-ID: <53C8517B.90202@gmail.com> On 07/17/2014 02:21 PM, Fred Hebert wrote: > On 07/17, Tristan Sloughter wrote: >> If for all OTP 17.x.y the runtime is going to consider it 17: >> >> 1> erlang:system_info(otp_release). >> "17" >> > For reverence, the new method to get the version according to the docs > is: http://www.erlang.org/doc/system_principles/versions.html > > In an OTP source code tree, the OTP version can be read from the text > file /OTP_VERSION. The absolute path to the file can be > constructed by calling filename:join([code:root_dir(), "OTP_VERSION"]). > > In an installed OTP development system, the OTP version can be read from > the text file /releases/ number>/OTP_VERSION. The absolute path to the file can by constructed by > calling filename:join([code:root_dir(), "releases", > erlang:system_info(otp_release), "OTP_VERSION"]). > > Do note that the version number can be followed by '**', meaning it was > patched with otp_patch_apply. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > So, the idea is that only the source code has a version. The binary installation has no minor or release version, unless you check individual erlang applications. The reason given so far (http://erlang.org/pipermail/erlang-questions/2014-April/078590.html), is it is too hard to provide the minor and release version because different installations are likely to use different source tree dependencies at different versions, so we are likely to lose any dependable information about what version set of erlang applications are most stable together. From t@REDACTED Fri Jul 18 00:49:01 2014 From: t@REDACTED (Tristan Sloughter) Date: Thu, 17 Jul 2014 17:49:01 -0500 Subject: [erlang-questions] OTP Version In-Reply-To: <53C8517B.90202@gmail.com> References: <1405619050.32590.142801741.733CC21C@webmail.messagingengine.com> <20140717212143.GE794@ferdmbp.local> <53C8517B.90202@gmail.com> Message-ID: <1405637341.11464.142897873.731BCC5E@webmail.messagingengine.com> Right, but it gets confusing because the OTP version IS 17.x.y not 17. So I'd argue if it can't be determined by the running system it shouldn't be part of system_info at all. And to make it more confusing, how does it know it is 17? Could there not be a mix of 17 and 18 apps in there? Maybe that is rejected? On Thu, Jul 17, 2014, at 05:43 PM, Michael Truog wrote: > On 07/17/2014 02:21 PM, Fred Hebert wrote: > > On 07/17, Tristan Sloughter wrote: > >> If for all OTP 17.x.y the runtime is going to consider it 17: > >> > >> 1> erlang:system_info(otp_release). > >> "17" > >> > > For reverence, the new method to get the version according to the docs > > is: http://www.erlang.org/doc/system_principles/versions.html > > > > In an OTP source code tree, the OTP version can be read from the text > > file /OTP_VERSION. The absolute path to the file can be > > constructed by calling filename:join([code:root_dir(), "OTP_VERSION"]). > > > > In an installed OTP development system, the OTP version can be read from > > the text file /releases/ > number>/OTP_VERSION. The absolute path to the file can by constructed by > > calling filename:join([code:root_dir(), "releases", > > erlang:system_info(otp_release), "OTP_VERSION"]). > > > > Do note that the version number can be followed by '**', meaning it was > > patched with otp_patch_apply. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > So, the idea is that only the source code has a version. The binary > installation has no minor or release version, unless you check individual > erlang applications. The reason given so far > (http://erlang.org/pipermail/erlang-questions/2014-April/078590.html), is > it is too hard to provide the minor and release version because different > installations are likely to use different source tree dependencies at > different versions, so we are likely to lose any dependable information > about what version set of erlang applications are most stable together. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From lloyd@REDACTED Fri Jul 18 01:34:13 2014 From: lloyd@REDACTED (lloyd@REDACTED) Date: Thu, 17 Jul 2014 19:34:13 -0400 (EDT) Subject: [erlang-questions] A website to show popular erlang modules In-Reply-To: References: Message-ID: <1405640053.194325314@apps.rackspace.com> Hi Bin, Your site is a valuable contribution to Erlang documentation. Much appreciate your hard work. In a dream world I imagine a link to a noobie-proof tutorial accompanying every listing--- why and when one would consider using the application and how to use it. Some kind of constructive community rating would be great as well. Nevertheless, this is a fine step forward. As they say in Mexico paso a paso. All the best, Lloyd ----Original Message----- From: "Bin Wang" Sent: Monday, July 14, 2014 8:27pm To: "erlang-questions" Subject: [erlang-questions] A website to show popular erlang modules Hi, I've wrote this website some months ago. I will be happy if you find it helpful. http://erlang-modules.binwang.me/ It will fetch popular Erlang projects on Github and see how many other Erlang projects are depended on it. Details could be found here: http://www.binwang.me/2013-11-16-Fetch-Popular-Erlang-Modules-by-Coffee-Script.html --- Bin Wang _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Jul 18 02:02:32 2014 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 18 Jul 2014 12:02:32 +1200 Subject: [erlang-questions] dialyzer and behaviours In-Reply-To: <53C75B4F.5050001@khandkar.net> References: <53C75B4F.5050001@khandkar.net> Message-ID: <5C2210C7-E79B-45AD-A026-740FE4EC587D@cs.otago.ac.nz> On 17/07/2014, at 5:12 PM, Siraaj Khandkar wrote: > Richard, what I think Daniil is asking for is only time-depended in a > language like Erlang. Indeed. But C lets you load and unload modules, Fortran has done that on mainframes since the 60s, Lisp lets you load new stuff, Python lets you load new stuff, Smalltalk (even Animorphic Smalltalk) lets you load and replace stuff, Prolog (even Prologs with threads and web servers) let you load and replace stuff, .... "Languages *like* Erlang" in the relevant sense are abundant. It would make sense for Erlang to offer something like old-fashioned Lisp "block compilation" where a group of source files are compiled together and loaded together. That would permit not just things like cross-module optimisation but cross-module verification. > In a statically typed language with a module > system it'd be something like: > > signature MY_DB = > sig > val table : unit -> iolist > end > > functor Foo (Db_impl : MY_DB) = > struct > ... > end The tricky thing here is that ML has been implemented in two ways: ML > > Daniil, is that what you had in mind? Unfortunately this is not possible > in Erlang, because, as Richard said, calls to module names in Erlang are > like calls to mutable, global reference cells. The only thing statically > checkable is that the module name is of type atom(), while the > implementation of a named module is always swappable at runtime. From lloyd@REDACTED Fri Jul 18 02:11:07 2014 From: lloyd@REDACTED (lloyd@REDACTED) Date: Thu, 17 Jul 2014 20:11:07 -0400 (EDT) Subject: [erlang-questions] block comments Message-ID: <1405642267.25285466@apps.rackspace.com> Hello, Since it hasn't been done I'd guess a compiler constraint. But why doesn't Erlang have a method for commenting out a block of lines in a module without having to tag every line with %? Or did I miss that page? It would be so convenient at times. Thanks, LRP -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Jul 18 02:58:26 2014 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 18 Jul 2014 12:58:26 +1200 Subject: [erlang-questions] block comments In-Reply-To: <1405642267.25285466@apps.rackspace.com> References: <1405642267.25285466@apps.rackspace.com> Message-ID: On 18/07/2014, at 12:11 PM, wrote: > But why doesn't Erlang have a method for commenting out a block of lines in a module without > having to tag every line with %? See the Ada rationale. > > Or did I miss that page? > > It would be so convenient at times. In my text editor, writing Prolog, I can comment out a region in one of two ways: Meta-/ Meta-% It's the same number of keystrokes to get /**/ around or % on every line. From wbin00@REDACTED Fri Jul 18 03:44:27 2014 From: wbin00@REDACTED (Bin Wang) Date: Fri, 18 Jul 2014 09:44:27 +0800 Subject: [erlang-questions] A website to show popular erlang modules In-Reply-To: <1405640053.194325314@apps.rackspace.com> References: <1405640053.194325314@apps.rackspace.com> Message-ID: Hi, all The Erlang projects I'm analyzing is all come from Github. I get the top 1000 stared Erlang projects from the Github's API. (I can only get 1000). Then analyze rebar.config to find the dependencies. So the result is all done by machine. So it's not as complex as a forum. If you are more interested in a forum that could review, comment and vote , I may write a new project based on it. If Erlang has a package manager such as gem or npm, things could be done much easier. --- Bin Wang 2014-07-18 7:34 GMT+08:00 : > Hi Bin, > > > > Your site is a valuable contribution to Erlang documentation. Much > appreciate your hard work. > > > > In a dream world I imagine a link to a noobie-proof tutorial accompanying > every listing--- why and when one would consider using the application and > how to use it. Some kind of constructive community rating would be great as > well. Nevertheless, this is a fine step forward. As they say in Mexico paso > a paso. > > > > All the best, > > > > Lloyd > > > > ----Original Message----- > From: "Bin Wang" > Sent: Monday, July 14, 2014 8:27pm > To: "erlang-questions" > Subject: [erlang-questions] A website to show popular erlang modules > > Hi, > > I've wrote this website some months ago. I will be happy if you find it > helpful. > > http://erlang-modules.binwang.me/ > > It will fetch popular Erlang projects on Github and see how many other > Erlang projects are depended on it. Details could be found here: > http://www.binwang.me/2013-11-16-Fetch-Popular-Erlang-Modules-by-Coffee-Script.html > > --- > > Bin Wang > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From serge@REDACTED Fri Jul 18 05:02:17 2014 From: serge@REDACTED (Serge Aleynikov) Date: Thu, 17 Jul 2014 23:02:17 -0400 Subject: [erlang-questions] Prevent overwriting sasl log on VM restart? In-Reply-To: References: Message-ID: I wrote the attached patch when I dealt with the same issue last year. Haven't had time to submit this in a "proper" way. It adds the following option to the sasl config: {file, Filename, Modes} where Modes are the modes the filename is open with. Use [append] for appending on startup. Regards, Serge On Thu, Jul 17, 2014 at 5:32 PM, Youngkin, Rich < richard.youngkin@REDACTED> wrote: > Is there a way, other than using the built-in log rotate mechanism or > something like lager, to avoid having the sasl log overwritten when the VM > is re/started? > > Thanks, > Rich > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: sasl.patch Type: application/octet-stream Size: 3702 bytes Desc: not available URL: From schneider@REDACTED Fri Jul 18 10:01:00 2014 From: schneider@REDACTED (Frans Schneider) Date: Fri, 18 Jul 2014 10:01:00 +0200 Subject: [erlang-questions] Suppressing error_logger output in escript Message-ID: <53C8D43C.5000704@xs4all.nl> Hi list, In my code I use some error_logger:info_msg's for debugging, which is great. Using escript, I implemented a few command line tools. The output of these tools is cluttered with the debugging output from the main program code. How can I suppress the info messages? I tried starting SASL and setting sasl_error_logger to {file, "/dev/null"} as in: #!/usr/bin/env escript %% -*- erlang -*- %%! -pa deps/getopt/ebin -mode(compile). main() -> ... application:load(sasl), application:set_env(sasl, sasl_error_logger, {file, "/dev/null"}), application:start(sasl), ...start main program... but the messages keep coming. According to the kernel documentation, suppressing error_logger output can also be done by setting error_logger to silent. Any suggestions? Frans From dm.klionsky@REDACTED Fri Jul 18 10:58:55 2014 From: dm.klionsky@REDACTED (Dmitry Klionsky) Date: Fri, 18 Jul 2014 11:58:55 +0300 Subject: [erlang-questions] Suppressing error_logger output in escript In-Reply-To: <53C8D43C.5000704@xs4all.nl> References: <53C8D43C.5000704@xs4all.nl> Message-ID: <53C8E1CF.3070808@gmail.com> On 07/18/2014 11:01 AM, Frans Schneider wrote: > Hi list, > > In my code I use some error_logger:info_msg's for debugging, which is > great. Using escript, I implemented a few command line tools. The > output of these tools is cluttered with the debugging output from the > main program code. How can I suppress the info messages? > I tried starting SASL and setting sasl_error_logger to {file, > "/dev/null"} as in: > > #!/usr/bin/env escript > %% -*- erlang -*- > %%! -pa deps/getopt/ebin > > -mode(compile). > > main() -> > ... > application:load(sasl), > application:set_env(sasl, sasl_error_logger, {file, "/dev/null"}), > application:start(sasl), > ...start main program... > > but the messages keep coming. > According to the kernel documentation, suppressing error_logger output > can also be done by setting error_logger to silent. > > Any suggestions? > > Frans > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Have you tried error_logger:tty(false) ? http://erlang.org/doc/man/error_logger.html#tty-1 -- Best regards, Dmitry Klionsky From jesper.louis.andersen@REDACTED Fri Jul 18 11:14:37 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 18 Jul 2014 11:14:37 +0200 Subject: [erlang-questions] block comments In-Reply-To: <1405642267.25285466@apps.rackspace.com> References: <1405642267.25285466@apps.rackspace.com> Message-ID: On Fri, Jul 18, 2014 at 2:11 AM, wrote: > But why doesn't Erlang have a method for commenting out a block of lines > in a module without > > having to tag every line with %? > Can can abuse the Erlang Pre-Processor and do some -ifdef(DISABLED). magic. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Fri Jul 18 11:45:31 2014 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Fri, 18 Jul 2014 11:45:31 +0200 Subject: [erlang-questions] block comments In-Reply-To: <1405642267.25285466@apps.rackspace.com> References: <1405642267.25285466@apps.rackspace.com> Message-ID: 2014-07-18 2:11 GMT+02:00 : > Hello, > > Since it hasn't been done I'd guess a compiler constraint. > > But why doesn't Erlang have a method for commenting out a block of lines in > a module without > > having to tag every line with %? > > Or did I miss that page? > > It would be so convenient at times. These are my vim macros: "To comment/uncomment vmap c :s/^/%/:nohl vmap u :s/^%//:nohl Select the lines, then press \c to comment the lines, \u to uncomment. From pierrefenoll@REDACTED Fri Jul 18 11:52:52 2014 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Fri, 18 Jul 2014 11:52:52 +0200 Subject: [erlang-questions] block comments In-Reply-To: References: <1405642267.25285466@apps.rackspace.com> Message-ID: > > Can can abuse the Erlang Pre-Processor and do some -ifdef(DISABLED). magic. > That will not always work: https://gist.github.com/fenollp/8315947 -------------- next part -------------- An HTML attachment was scrubbed... URL: From schneider@REDACTED Fri Jul 18 12:10:06 2014 From: schneider@REDACTED (Frans Schneider) Date: Fri, 18 Jul 2014 12:10:06 +0200 Subject: [erlang-questions] Suppressing error_logger output in escript Message-ID: <53C8F27E.1050505@xs4all.nl> That's it! Thanks, From n.oxyde@REDACTED Fri Jul 18 13:33:19 2014 From: n.oxyde@REDACTED (Anthony Ramine) Date: Fri, 18 Jul 2014 13:33:19 +0200 Subject: [erlang-questions] block comments In-Reply-To: <1405642267.25285466@apps.rackspace.com> References: <1405642267.25285466@apps.rackspace.com> Message-ID: <1A57F7E4-B820-4D27-8225-7F52CA551AD4@gmail.com> Why do you need to comment that many lines exactly? -- Anthony Ramine Le 18 juil. 2014 ? 02:11, lloyd@REDACTED a ?crit : > But why doesn't Erlang have a method for commenting out a block of lines in a module without > having to tag every line with %? From egil@REDACTED Fri Jul 18 14:34:31 2014 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Fri, 18 Jul 2014 14:34:31 +0200 Subject: [erlang-questions] block comments In-Reply-To: References: <1405642267.25285466@apps.rackspace.com> Message-ID: <53C91457.1060401@erlang.org> I agree with Richard. A side note: My work is about evenly divided between C code and Erlang code. In C we only use /* block comments */ and // coments are frowned upon (you are actually in error if you use them in our code). In Erlang we have the opposite, only % line comments. I'm originally from a complete C-world and I really missed the /* block comments */ in the beginning when writing Erlang code. Today this is completely reversed. I use my editor comment out large sections of code if needed. /* block comments */ are actually a hindrance here since I need to keep track of the beginning and end of a comment. I frequently use // instead - but never in committed code. For me, it's much easier to use line comments but I had to unlearn and learn anew. To answer your question, no block comments in Erlang and you'll likely never see them either. // Bj?rn-Egil On 2014-07-18 02:58, Richard A. O'Keefe wrote: > On 18/07/2014, at 12:11 PM, wrote: >> But why doesn't Erlang have a method for commenting out a block of lines in a module without >> having to tag every line with %? > See the Ada rationale. >> >> Or did I miss that page? >> >> It would be so convenient at times. > In my text editor, writing Prolog, I can comment out > a region in one of two ways: > > > > > Meta-/ Meta-% > > It's the same number of keystrokes to get /**/ around > or % on every line. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From n.oxyde@REDACTED Fri Jul 18 14:46:08 2014 From: n.oxyde@REDACTED (Anthony Ramine) Date: Fri, 18 Jul 2014 14:46:08 +0200 Subject: [erlang-questions] block comments In-Reply-To: <53C91457.1060401@erlang.org> References: <1405642267.25285466@apps.rackspace.com> <53C91457.1060401@erlang.org> Message-ID: <4BE7D646-3FE7-4661-9883-BC8B54492019@gmail.com> Maybe one day we will stop to pretend that C99 and C11 do not exist. :( -- Anthony Ramine Le 18 juil. 2014 ? 14:34, Bj?rn-Egil Dahlberg a ?crit : > In C we only use /* block comments */ and // coments are frowned upon (you are actually in error if you use them in our code). > In Erlang we have the opposite, only % line comments. From richard.youngkin@REDACTED Fri Jul 18 15:17:43 2014 From: richard.youngkin@REDACTED (Youngkin, Rich) Date: Fri, 18 Jul 2014 07:17:43 -0600 Subject: [erlang-questions] Prevent overwriting sasl log on VM restart? In-Reply-To: References: Message-ID: Hi Serge, Thanks for the patch. Do you know if this will work on R15B? I'll probably give it a test spin in any case. Thanks again, Rich On Thu, Jul 17, 2014 at 9:02 PM, Serge Aleynikov wrote: > I wrote the attached patch when I dealt with the same issue last year. > Haven't had time to submit this in a "proper" way. > > It adds the following option to the sasl config: > > {file, Filename, Modes} > > where Modes are the modes the filename is open with. Use [append] for > appending on startup. > > Regards, > > Serge > > > On Thu, Jul 17, 2014 at 5:32 PM, Youngkin, Rich < > richard.youngkin@REDACTED> wrote: > >> Is there a way, other than using the built-in log rotate mechanism or >> something like lager, to avoid having the sasl log overwritten when the VM >> is re/started? >> >> Thanks, >> Rich >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shayan@REDACTED Fri Jul 18 15:45:26 2014 From: shayan@REDACTED (Shayan Pooya) Date: Fri, 18 Jul 2014 09:45:26 -0400 Subject: [erlang-questions] Using ram-only mnesia in place of gen_server+ets table In-Reply-To: References: Message-ID: Hello Ludovic, I was more curious about the best practices of 'shipping' the dbs to the slaves. Creating the table, marking it as read-only and then shipping it to the slave nodes makes sense but it seems like it is not possible and we have to set the hosting nodes of a table at its creation time. Thanks. On Thu, Jul 17, 2014 at 5:21 PM, Ludovic Demblans wrote: > Hi, > > If you plan to use dirty read-only acces, (and if this is the only > problem), any process can acces the table data if it is in 'protected' mode > (which is the default. > > Cheers > > lud > > Le Thu, 17 Jul 2014 22:42:48 +0200, Shayan Pooya a > ?crit: > > > Disco filesystem gc involves a step in which a process builds an ets table >> on the master and >> the slave nodes query this ets table regularly afterwards. >> The ets table is owned by a gen_server which responds to the queries from >> the >> slaves using this ets table. >> >> I am experimenting with using a mnesia table instead of this ets table so >> that >> the gen_server is not involved in this operation (There are a lot of >> queries and >> this master process is a bottleneck). >> This is a ram-only mnesia table which is created and then made read-only. >> All of >> the operations use mnesia:dirty_* functions and the slave nodes never >> modify the >> table. >> >> 1. Is this going to avoid the gen_server bottleneck? >> 2. I am thinking of adding all of the nodes (master + slaves) as the >> ram-copies for >> this table in order to have local lookups on the slave nodes. >> Is this going to slow down the creation of the mnesia table? (as >> said before, it is only using dirty_* functions for modifying the table). >> > > > -- > Utilisant le logiciel de courrier r?volutionnaire d'Opera : > http://www.opera.com/mail/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andreas@REDACTED Fri Jul 18 17:38:28 2014 From: andreas@REDACTED (Andreas Schumacher) Date: Fri, 18 Jul 2014 17:38:28 +0200 Subject: [erlang-questions] Fwd: FW: runtime_dependencies part 2 In-Reply-To: References: <1405632934.26969.142876397.6358DFD1@webmail.messagingengine.com> Message-ID: Hi Tristan, As Kenneth responded to your post [1] a couple of months ago, there is room for improvement wrt to the handling of runtime_dependencies. We will continue to work on that issue, and we may write an EEP for collecting community input; although, I won't promise the latter. In any case, we appreciate your input and will consider it, regardless of an EEP. [1]: http://erlang.org/pipermail/erlang-questions/2014-April/078730.html Andreas --- Andreas Schumacher, Erlang/OTP, Ericsson AB On 17/07/14 23:35, "Tristan Sloughter" wrote: >I wrote the list about runtime_dependencies in .app files now in 17+ a >couple months ago, http://www.erlang.org/doc/man/app.html > >Today I was informed that if you were to take all the >runtime_dependencies defined for kernel-3.x and their >runtime_dependencies you'd end up having wx as one, even for OTP built >without wx support. > >Yes, I know the second warning on the app man page says that just >because it is in there today doesn't mean it will be in the future and >is actively being worked on. > >My question is, wouldn't it be nice if the OTP team made an EEP that the >community could comment on for this? :) > >Tristan >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Fri Jul 18 20:32:40 2014 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 18 Jul 2014 11:32:40 -0700 Subject: [erlang-questions] Fwd: FW: runtime_dependencies part 2 In-Reply-To: References: <1405632934.26969.142876397.6358DFD1@webmail.messagingengine.com> Message-ID: <53C96848.7050203@gmail.com> On 07/18/2014 08:38 AM, Andreas Schumacher wrote: > Hi Tristan, > > As Kenneth responded to your post [1] a couple of months ago, there is room for improvement wrt to the handling of runtime_dependencies. We will continue to work on that issue, and we may write an EEP for collecting community input; although, I won't promise the latter. In any case, we appreciate your input and will consider it, regardless of an EEP. > > [1]: http://erlang.org/pipermail/erlang-questions/2014-April/078730.html > > Andreas > > --- > > Andreas Schumacher, Erlang/OTP, Ericsson AB > > > On 17/07/14 23:35, "Tristan Sloughter" > wrote: > > >I wrote the list about runtime_dependencies in .app files now in 17+ a > >couple months ago, http://www.erlang.org/doc/man/app.html > > > >Today I was informed that if you were to take all the > >runtime_dependencies defined for kernel-3.x and their > >runtime_dependencies you'd end up having wx as one, even for OTP built > >without wx support. > > > >Yes, I know the second warning on the app man page says that just > >because it is in there today doesn't mean it will be in the future and > >is actively being worked on. > > > >My question is, wouldn't it be nice if the OTP team made an EEP that the > >community could comment on for this? :) > > > >Tristan > >_______________________________________________ > >erlang-questions mailing list > >erlang-questions@REDACTED > >http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions One suggestion is that it would be nice to not have to parse a string to get the version or the constraint from the entry. So, keeping the application name as an atom and the version as a string, the relational operator could be done as an atom: http://www.erlang.org/doc/man/app.html example of "kernel-3.0" becomes {kernel, '>=', "3.0"} If the normal relational operators are handled like atoms, it seems like this would become explicit and easier to use (no extra parsing required). This is similar to what was in the older email thread, but keeps the version separate from the relational operator. It would be nice to clarify whether it is an error for 'runtime_dependencies' to provide a constraint for an application that is not a dependency listed in either 'included_applications' or 'applications'. It seems like it should be limited to providing constraints on its dependencies (when considering the whole tree of dependencies for that particular application), since otherwise it could impact unrelated applications with invalid constraints. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Fri Jul 18 20:39:30 2014 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 18 Jul 2014 11:39:30 -0700 Subject: [erlang-questions] OTP Version In-Reply-To: <1405637341.11464.142897873.731BCC5E@webmail.messagingengine.com> References: <1405619050.32590.142801741.733CC21C@webmail.messagingengine.com> <20140717212143.GE794@ferdmbp.local> <53C8517B.90202@gmail.com> <1405637341.11464.142897873.731BCC5E@webmail.messagingengine.com> Message-ID: <53C969E2.2040609@gmail.com> On 07/17/2014 03:49 PM, Tristan Sloughter wrote: > Right, but it gets confusing because the OTP version IS 17.x.y not 17. > So I'd argue if it can't be determined by the running system it > shouldn't be part of system_info at all. > > And to make it more confusing, how does it know it is 17? Could there > not be a mix of 17 and 18 apps in there? Maybe that is rejected? My main concern is that being unable to version all erlang applications together helps to obscure what selection of erlang applications should be used in production, making the open source project less transparent and more of a proprietary project. That can help generate consulting revenue for the few people that may track what erlang applications are best together, but would further limit the growth of the community due to a decrease in transparency. That pursues an idea that the open source community should only be a community of alpha/beta testers. I understand the source repo has a single version file which is meant to track a version, but if that version is never used by anything in production, it becomes meaningless. > > On Thu, Jul 17, 2014, at 05:43 PM, Michael Truog wrote: >> On 07/17/2014 02:21 PM, Fred Hebert wrote: >>> On 07/17, Tristan Sloughter wrote: >>>> If for all OTP 17.x.y the runtime is going to consider it 17: >>>> >>>> 1> erlang:system_info(otp_release). >>>> "17" >>>> >>> For reverence, the new method to get the version according to the docs >>> is: http://www.erlang.org/doc/system_principles/versions.html >>> >>> In an OTP source code tree, the OTP version can be read from the text >>> file /OTP_VERSION. The absolute path to the file can be >>> constructed by calling filename:join([code:root_dir(), "OTP_VERSION"]). >>> >>> In an installed OTP development system, the OTP version can be read from >>> the text file /releases/>> number>/OTP_VERSION. The absolute path to the file can by constructed by >>> calling filename:join([code:root_dir(), "releases", >>> erlang:system_info(otp_release), "OTP_VERSION"]). >>> >>> Do note that the version number can be followed by '**', meaning it was >>> patched with otp_patch_apply. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> So, the idea is that only the source code has a version. The binary >> installation has no minor or release version, unless you check individual >> erlang applications. The reason given so far >> (http://erlang.org/pipermail/erlang-questions/2014-April/078590.html), is >> it is too hard to provide the minor and release version because different >> installations are likely to use different source tree dependencies at >> different versions, so we are likely to lose any dependable information >> about what version set of erlang applications are most stable together. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From moxford@REDACTED Fri Jul 18 22:26:57 2014 From: moxford@REDACTED (Mike Oxford) Date: Fri, 18 Jul 2014 13:26:57 -0700 Subject: [erlang-questions] block comments In-Reply-To: <1405642267.25285466@apps.rackspace.com> References: <1405642267.25285466@apps.rackspace.com> Message-ID: In emacs, mark the region (-space, move to the other end) and then C-c-C-c (Ctrl-C twice) to comment out and then C-c-C-u to uncomment. I dislike /* */ because you cannot nest them. I like per-line, though I will admit to abusing ifdefs in C/C++ in a past life. -mox On Thu, Jul 17, 2014 at 5:11 PM, wrote: > Hello, > > > > Since it hasn't been done I'd guess a compiler constraint. > > > > But why doesn't Erlang have a method for commenting out a block of lines > in a module without > > having to tag every line with %? > > > > Or did I miss that page? > > > > It would be so convenient at times. > > > > Thanks, > > > > LRP > > > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Fri Jul 18 22:44:26 2014 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Fri, 18 Jul 2014 22:44:26 +0200 Subject: [erlang-questions] block comments In-Reply-To: <1405642267.25285466@apps.rackspace.com> References: <1405642267.25285466@apps.rackspace.com> Message-ID: <53C9872A.6060800@wanadoo.fr> Hi, Using vi : Comment : Esc :x,ys/^/%/g Uncomment : Esc :x,ys/^%//g Where x=starting line number, y= ending line number regards From jj@REDACTED Fri Jul 18 09:23:45 2014 From: jj@REDACTED (Giovanni Giorgi) Date: Fri, 18 Jul 2014 09:23:45 +0200 Subject: [erlang-questions] My Erlang experience (and Introducing myself) Message-ID: Hi all, My name is Giovanni Giorgi and I am new to the Erlang mailing list, so I will introduce my self shortly. I am a 40-years old Software architect, started to coding when I was eleven, inspired by the "Socratic daim?n" if you like. In 2000, after my Master degree at DSI University of Milan, Italy I started to work in the banking and financial field, mostly coding in Java J2EE (and ...tired by now). I like very much learning programming languages and Unix operating systems. I have started looking at Erlang after taking a peek look at Scala, Haskell, Emacs-Lisp, and Nodejs/JavaScript. I am surprised from how much is easy using Erlang compared to EmacsLisp/Scala or Haskell. I am not doing holy war here, I am just saying I am very comfortable with Erlang compared to other functional languages. I have started a small project which is a search engine tailored to code search. I have found is easier to maintain the erlang version compared to the ruby one (!) and my knowledge of the two languages (and paired std libraries) is quite the same. I enjoy Squeak Smalltalk for a long time, and I think is Erlang is even more productive then Smalltalk, a quite hard to beat environment. --- Gio's Blog http://gioorgi.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangerintraining@REDACTED Fri Jul 18 22:35:07 2014 From: erlangerintraining@REDACTED (Kristoffer Brown) Date: Fri, 18 Jul 2014 14:35:07 -0600 Subject: [erlang-questions] Mnesia error when switching fro clustered to un-clustered mode Message-ID: I have deployed my application to three nodes with mnesia in standalone mode. I have three tables, one of which recently changed. I have all three upgraded with the correct table schema and running fine in standalone mode. When I switch them to clustered I receive the following error {merge_schema_failed, "Bad cookie in table definition From derekbrown121@REDACTED Fri Jul 18 23:46:34 2014 From: derekbrown121@REDACTED (Derek Brown) Date: Fri, 18 Jul 2014 17:46:34 -0400 Subject: [erlang-questions] Problem running observer with 17.1 Message-ID: I'm having a problem running observer with 17.1. It previously ran ok with 17.0. This is OS X 10.9.3. Apparently to do with the wx lib. As the error suggests, there's no /usr/local/lib/libwx_osx_cocoau_stc-3.0.dylib on the system. ~ $ erl -sname observer -hidden -setcookie foo -run observer Erlang/OTP 17 [erts-6.1] [source-d2a4c20] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] Eshell V6.1 (abort with ^G) (observer@REDACTED)1> =ERROR REPORT==== 18-Jul-2014::13:06:51 === WX Failed loading "wxe_driver"@"/usr/local/lib/erlang/lib/wx-1.3/priv" (observer@REDACTED)1> observer:start(). =ERROR REPORT==== 18-Jul-2014::13:06:56 === WX Failed loading "wxe_driver"@"/usr/local/lib/erlang/lib/wx-1.3/priv" {error,{{load_driver,"dlopen(/usr/local/lib/erlang/lib/wx-1.3/priv/wxe_driver.so, 2): Library not loaded: /usr/local/lib/libwx_osx_cocoau_stc-3.0.dylib\n Referenced from: /usr/local/lib/erlang/lib/wx-1.3/priv/wxe_driver.so\n Reason: image not found"}, [{wxe_server,start,1,[{file,"wxe_server.erl"},{line,64}]}, {wx,new,1,[{file,"wx.erl"},{line,114}]}, {observer_wx,init,1,[{file,"observer_wx.erl"},{line,91}]}, {wx_object,init_it,6,[{file,"wx_object.erl"},{line,299}]}, {proc_lib,init_p_do_apply,3, [{file,"proc_lib.erl"},{line,239}]}]}} (observer@REDACTED)2> -------------- next part -------------- An HTML attachment was scrubbed... URL: From z@REDACTED Sat Jul 19 08:28:15 2014 From: z@REDACTED (Danil Zagoskin) Date: Sat, 19 Jul 2014 10:28:15 +0400 Subject: [erlang-questions] Problem running observer with 17.1 In-Reply-To: References: Message-ID: Hi Derek! Is your OTP instance downloaded as binary Mac OS package from https://www.erlang-solutions.com/downloads/download-erlang-otp? Did you try downloading the current version (with same OTP version number package content sometimes change there)? Try installing fresh wxmac from brew (brew update && brew install wxmac). Current version in brew repository is 3.0.1. Currently I have the latest esl-erlang 17.1 and observer works fine out-of-the-box with wxmac 3.0.1. On Sat, Jul 19, 2014 at 1:46 AM, Derek Brown wrote: > I'm having a problem running observer with 17.1. It previously ran ok with > 17.0. This is OS X 10.9.3. > > Apparently to do with the wx lib. As the error suggests, there's no > /usr/local/lib/libwx_osx_cocoau_stc-3.0.dylib on the system. > > > ~ $ erl -sname observer -hidden -setcookie foo -run observer > Erlang/OTP 17 [erts-6.1] [source-d2a4c20] [64-bit] [smp:4:4] > [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V6.1 (abort with ^G) > (observer@REDACTED)1> > =ERROR REPORT==== 18-Jul-2014::13:06:51 === > WX Failed loading "wxe_driver"@"/usr/local/lib/erlang/lib/wx-1.3/priv" > > (observer@REDACTED)1> observer:start(). > > =ERROR REPORT==== 18-Jul-2014::13:06:56 === > WX Failed loading "wxe_driver"@"/usr/local/lib/erlang/lib/wx-1.3/priv" > {error,{{load_driver,"dlopen(/usr/local/lib/erlang/lib/wx-1.3/priv/wxe_driver.so, > 2): Library not loaded: /usr/local/lib/libwx_osx_cocoau_stc-3.0.dylib\n > Referenced from: /usr/local/lib/erlang/lib/wx-1.3/priv/wxe_driver.so\n > Reason: image not found"}, > [{wxe_server,start,1,[{file,"wxe_server.erl"},{line,64}]}, > {wx,new,1,[{file,"wx.erl"},{line,114}]}, > {observer_wx,init,1,[{file,"observer_wx.erl"},{line,91}]}, > {wx_object,init_it,6,[{file,"wx_object.erl"},{line,299}]}, > {proc_lib,init_p_do_apply,3, > [{file,"proc_lib.erl"},{line,239}]}]}} > (observer@REDACTED)2> > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From schneider@REDACTED Sat Jul 19 14:47:57 2014 From: schneider@REDACTED (Frans Schneider) Date: Sat, 19 Jul 2014 14:47:57 +0200 Subject: [erlang-questions] Cowboy based XML-RPC server Message-ID: <53CA68FD.6010505@xs4all.nl> Hi, I have to implement XML-RPC to get my system to communicate with ROS [1]. To implement a ROS node, I need both an XML-RPC client and XML-RPC server. Nothing fancy and preferable very light-weight. Cowboy seems to be a very nice framework, but since I am totally new to implementing XML-RPC and I can't find anything on Cowboy + XML-RPC, I could use some advice or pointers to examples. Also, what would be the best approach to implementing the client? Currently the system is fully functional but uses Erlang message passing directly instead of the ROS XML-RPC calls and TCP/UDPROS protocols. So, when exposing the system to ROS, I would like to use some middleman to implement the ROS parts, much liek Joe Armstrong describes in his book [2]. Thanks, Frans [1] http://wiki.ros.org/ [2] Programming Erlang, Second Edition, 24.1 Maintaining the Erlang View of the World From vinoski@REDACTED Sat Jul 19 16:43:02 2014 From: vinoski@REDACTED (Steve Vinoski) Date: Sat, 19 Jul 2014 10:43:02 -0400 Subject: [erlang-questions] Cowboy based XML-RPC server In-Reply-To: <53CA68FD.6010505@xs4all.nl> References: <53CA68FD.6010505@xs4all.nl> Message-ID: On Sat, Jul 19, 2014 at 8:47 AM, Frans Schneider wrote: > Hi, > > I have to implement XML-RPC to get my system to communicate with ROS [1]. > To implement a ROS node, I need both an XML-RPC client and XML-RPC server. > Nothing fancy and preferable very light-weight. Cowboy seems to be a very > nice framework, but since I am totally new to implementing XML-RPC and I > can't find anything on Cowboy + XML-RPC, I could use some advice or > pointers to examples. > The yaws web server provides XML-RPC support: https://github.com/klacke/yaws --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From schneider@REDACTED Sat Jul 19 17:18:26 2014 From: schneider@REDACTED (Schneider) Date: Sat, 19 Jul 2014 17:18:26 +0200 Subject: [erlang-questions] Cowboy based XML-RPC server In-Reply-To: References: <53CA68FD.6010505@xs4all.nl> Message-ID: <6B74C590-A697-4386-8CE7-12A58797F1B5@xs4all.nl> I know, but it is not as light weight as Cowboy I guess. It seems overkill for what a typical ROS node does, i.e. Registering with the ROS master and protocol necotiation with other nodes. > Op 19 jul. 2014 om 16:43 heeft Steve Vinoski het volgende geschreven: > > > > >> On Sat, Jul 19, 2014 at 8:47 AM, Frans Schneider wrote: >> Hi, >> >> I have to implement XML-RPC to get my system to communicate with ROS [1]. To implement a ROS node, I need both an XML-RPC client and XML-RPC server. Nothing fancy and preferable very light-weight. Cowboy seems to be a very nice framework, but since I am totally new to implementing XML-RPC and I can't find anything on Cowboy + XML-RPC, I could use some advice or pointers to examples. > > The yaws web server provides XML-RPC support: > > https://github.com/klacke/yaws > > --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Sat Jul 19 18:05:02 2014 From: vinoski@REDACTED (Steve Vinoski) Date: Sat, 19 Jul 2014 12:05:02 -0400 Subject: [erlang-questions] Cowboy based XML-RPC server In-Reply-To: <6B74C590-A697-4386-8CE7-12A58797F1B5@xs4all.nl> References: <53CA68FD.6010505@xs4all.nl> <6B74C590-A697-4386-8CE7-12A58797F1B5@xs4all.nl> Message-ID: "Seems like" and "is" often differ by a significant amount. The only real way to know is to measure. Just because a system is full featured does not mean that all those features are all active at all times. A number of Yaws users use it embedded situations and rely on only the parts they need. --steve On Sat, Jul 19, 2014 at 11:18 AM, Schneider wrote: > I know, but it is not as light weight as Cowboy I guess. It seems overkill > for what a typical ROS node does, i.e. Registering with the ROS master and > protocol necotiation with other nodes. > > Op 19 jul. 2014 om 16:43 heeft Steve Vinoski het > volgende geschreven: > > > > > On Sat, Jul 19, 2014 at 8:47 AM, Frans Schneider > wrote: > >> Hi, >> >> I have to implement XML-RPC to get my system to communicate with ROS [1]. >> To implement a ROS node, I need both an XML-RPC client and XML-RPC server. >> Nothing fancy and preferable very light-weight. Cowboy seems to be a very >> nice framework, but since I am totally new to implementing XML-RPC and I >> can't find anything on Cowboy + XML-RPC, I could use some advice or >> pointers to examples. >> > > The yaws web server provides XML-RPC support: > > https://github.com/klacke/yaws > > --steve > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From schneider@REDACTED Sat Jul 19 18:08:19 2014 From: schneider@REDACTED (Schneider) Date: Sat, 19 Jul 2014 18:08:19 +0200 Subject: [erlang-questions] Cowboy based XML-RPC server In-Reply-To: References: <53CA68FD.6010505@xs4all.nl> <6B74C590-A697-4386-8CE7-12A58797F1B5@xs4all.nl> Message-ID: <9C44F2BF-EFF4-4662-9FFC-4B814DBDEAC2@xs4all.nl> Point taken. I'll have a look at Yaws then. Any advice on the client side? Thanks, Frans > Op 19 jul. 2014 om 18:05 heeft Steve Vinoski het volgende geschreven: > > "Seems like" and "is" often differ by a significant amount. The only real way to know is to measure. > > Just because a system is full featured does not mean that all those features are all active at all times. A number of Yaws users use it embedded situations and rely on only the parts they need. > > --steve > > >> On Sat, Jul 19, 2014 at 11:18 AM, Schneider wrote: >> I know, but it is not as light weight as Cowboy I guess. It seems overkill for what a typical ROS node does, i.e. Registering with the ROS master and protocol necotiation with other nodes. >> >>> Op 19 jul. 2014 om 16:43 heeft Steve Vinoski het volgende geschreven: >>> >> >>> >>> >>> >>>> On Sat, Jul 19, 2014 at 8:47 AM, Frans Schneider wrote: >>>> Hi, >>>> >>>> I have to implement XML-RPC to get my system to communicate with ROS [1]. To implement a ROS node, I need both an XML-RPC client and XML-RPC server. Nothing fancy and preferable very light-weight. Cowboy seems to be a very nice framework, but since I am totally new to implementing XML-RPC and I can't find anything on Cowboy + XML-RPC, I could use some advice or pointers to examples. >>> >>> The yaws web server provides XML-RPC support: >>> >>> https://github.com/klacke/yaws >>> >>> --steve > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Sat Jul 19 18:12:44 2014 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Sat, 19 Jul 2014 18:12:44 +0200 Subject: [erlang-questions] Cowboy based XML-RPC server In-Reply-To: References: <53CA68FD.6010505@xs4all.nl> Message-ID: <53CA98FC.4030203@ninenines.eu> I would advise using Yaws for XML-RPC too. It's XML-RPC, there is very little to gain from one server or another. It's a pain in all cases, even more so if you have to reimplement it yourself. On 07/19/2014 04:43 PM, Steve Vinoski wrote: > > > > On Sat, Jul 19, 2014 at 8:47 AM, Frans Schneider > wrote: > > Hi, > > I have to implement XML-RPC to get my system to communicate with ROS > [1]. To implement a ROS node, I need both an XML-RPC client and > XML-RPC server. Nothing fancy and preferable very light-weight. > Cowboy seems to be a very nice framework, but since I am totally new > to implementing XML-RPC and I can't find anything on Cowboy + > XML-RPC, I could use some advice or pointers to examples. > > > The yaws web server provides XML-RPC support: > > https://github.com/klacke/yaws > > --steve > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu From siraaj@REDACTED Sat Jul 19 20:16:16 2014 From: siraaj@REDACTED (Siraaj Khandkar) Date: Sat, 19 Jul 2014 14:16:16 -0400 Subject: [erlang-questions] dialyzer and behaviours In-Reply-To: <20140717142906.GE89097@ferdair.local> References: <53C7D634.8060500@khandkar.net> <20140717142906.GE89097@ferdair.local> Message-ID: <53CAB5F0.9050009@khandkar.net> On 07/17/2014 10:29 AM, Fred Hebert wrote: > On 07/17, Siraaj Khandkar wrote: >> In the face of such ambiguities, Dialyzer really has no choice but to >> assume innocence until guilt can be proven with certainty. >> >> When you say `baz = foo:bar()`, Dialyzer can _try_ to follow the path >> and match the expressions, but when you say `baz = Foo:bar()` - the rug >> is swept from underneath Dialyzer's feet and (at that call site) it can >> go no further than _try_ to find a case where Foo does not result in >> being an atom. >> >> So in other words, Dialyzer is not a type checker - it only seeks >> _definite_ failure cases in the call graph and sometimes >> success-or-failure depends _only_ on runtime state (such as dynamic >> references). >> > > Someone who knows better about types, type systems, and type checking > than me may correct me, but I believe that Dialyzer still is a type > checker. > > The difference is that it checks a type system that isn't axiomatic > (such as Hindley-Milner) -- those that tend to *prove* that there is no > type errors in the program, sometimes at the cost of forbidding certain > otherwise valid programs. > > The canonical example for this is is: > > 'and'(true, true) -> true; > 'and'(_, _) -> false. > > being called as: > > 'and'(5,6) > > A type inference for an axiomatic type system might decide that the > 'and'/2 function's signature is > > -spec 'and'(boolean(), boolean()) -> boolean()' Very interesting point! To explore it, the key question to answer is: "What is a type?" I struggled with that question for sometime, because I noticed that I use similar reasoning, of creating building blocks, in languages with _and_ without static type enforcement. So one way to think about "type" is as a synonym for "abstraction". Going further, I think the most enlightening definition of "type" was given by John Reynolds as: "a syntactic discipline for maintaining levels of abstraction." [1] So, while related, a type is not the abstraction itself, but a tool for enforcing it. More specifically, a _syntactic_ tool, that is - static. The 'and' example, I think, perfectly demonstrates why the enforcement necessarily has to be static - the catch-all case breaks the abstraction and by the time it hits runtime - it is too late and there's no principled way to prevent it other than adding checks _syntactically_. This is a somewhat controversial view that will probably upset some people, but _statistically_ creating and enforcing axiomatic abstractions is the only thing that qualifies as a type system. Dynamic typing just isn't typing (going even further, which is even more controversial (but accurate IMO), it is uni-typing). [1]: http://www.cse.chalmers.se/edu/year/2010/course/DAT140_Types/Reynolds_typesabpara.pdf > > and therefore ban the call. Dialyzer can in fact ban this one if you > explicitly write that signature down, but won't infer this. If you have > no signature and call `$ typer `, the output is: > > -spec 'and'(_,_) -> boolean(). And this is the key point that makes me feel that Dialyzer is not quite a type checker. Except... it DOES give you a way to syntactically define and enforce abstraction, so it DOES do type checking, but _optionally_ and _incompletely_ (as in the case of dynamic references discussed earlier). So what to do? :) IMO it is an awesome static analysis tool that I cannot imagine using Erlang without, but because of the unavoidable typing holes - it cannot be thought of as a full-blown type checker in my mind. > Erlang's semantics allow that call, and that is the correct type > signature that will correspond to its behavior at runtime. Yes, Dialyzer is true to Erlang semantics, but Erlang semantics is exactly what creates the abstraction holes in the first place. > So the reason for this is that Dialyzer type checks on a concept called > 'success typing'. To simplify a lot (maybe too much), an axiomatic type > checker will build its list of what is valid or not by starting with a > null set (nothing is valid), and expanding it as it finds more > information in your program. This leads to denying programs that are: > > 1. wrong all of the time for an execution path > 2. wrong some of the time for an execution path (and therefore right > some of the time) > 3. do not provide sufficient information to know something will always > be right (false negative) > > Success typing basically works the other way around. It starts assuming > that everything is good, and then restricts what it thinks is valid as > it finds more information. This leads to denying programs that are: > > 1. wrong all the time for an execution path > > Programs that may or may not fail, either due to lack of information or > succeeding some of the time (or a combination of both) will be accepted > by Dialyzer. > > Dialyzer still definitely checks your types, does analysis, does > inference, etc. It just analyzes them differently to give you a > different diagnostic, but for all intents and purposes, it is a type > checker (on top of being able to find other discrepancies in your > programs). From eric.pailleau@REDACTED Sat Jul 19 21:56:13 2014 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sat, 19 Jul 2014 21:56:13 +0200 Subject: [erlang-questions] block comments In-Reply-To: <53C9872A.6060800@wanadoo.fr> References: <1405642267.25285466@apps.rackspace.com> <53C9872A.6060800@wanadoo.fr> Message-ID: <53CACD5D.2080407@wanadoo.fr> Hi, I remember a mail about Here Documents in Erlang, and found it : http://erlang.org/pipermail/erlang-questions/2010-December/055118.html MHO This would be a good idea for many uses, including multiline comments. For instance, I personnally embed perldocs in bash scripts into Here documents. The documentation is in scripts, human readable, follows versionning of the code, and man pages are created with pod2man. http://bahut.alma.ch/2007/08/embedding-documentation-in-shell-script_16.html Enjoy. Regards From siraaj@REDACTED Sun Jul 20 00:09:08 2014 From: siraaj@REDACTED (Siraaj Khandkar) Date: Sat, 19 Jul 2014 18:09:08 -0400 Subject: [erlang-questions] dialyzer and behaviours In-Reply-To: <53CAB5F0.9050009@khandkar.net> References: <53C7D634.8060500@khandkar.net> <20140717142906.GE89097@ferdair.local> <53CAB5F0.9050009@khandkar.net> Message-ID: <53CAEC84.1090608@khandkar.net> On 07/19/2014 02:16 PM, Siraaj Khandkar wrote: >... _statistically_ creating and enforcing axiomatic abstractions ... > Gah! Mindless acceptance of spell checker suggestion - I meant "syntactically", of course, not "statistically"! :-D From siraaj@REDACTED Sun Jul 20 00:10:43 2014 From: siraaj@REDACTED (Siraaj Khandkar) Date: Sat, 19 Jul 2014 18:10:43 -0400 Subject: [erlang-questions] dialyzer and behaviours In-Reply-To: <5C2210C7-E79B-45AD-A026-740FE4EC587D@cs.otago.ac.nz> References: <53C75B4F.5050001@khandkar.net> <5C2210C7-E79B-45AD-A026-740FE4EC587D@cs.otago.ac.nz> Message-ID: <53CAECE3.6090409@khandkar.net> On 07/17/2014 08:02 PM, Richard A. O'Keefe wrote: > > On 17/07/2014, at 5:12 PM, Siraaj Khandkar wrote: >> Richard, what I think Daniil is asking for is only time-depended in a >> language like Erlang. > > Indeed. But C lets you load and unload modules, > Fortran has done that on mainframes since the 60s, > Lisp lets you load new stuff, Python lets you load > new stuff, Smalltalk (even Animorphic Smalltalk) > lets you load and replace stuff, Prolog (even Prologs > with threads and web servers) let you load and replace > stuff, .... > > "Languages *like* Erlang" in the relevant sense are > abundant. > > It would make sense for Erlang to offer something like > old-fashioned Lisp "block compilation" where a group > of source files are compiled together and loaded together. > That would permit not just things like cross-module > optimisation but cross-module verification. > >> In a statically typed language with a module >> system it'd be something like: >> >> signature MY_DB = >> sig >> val table : unit -> iolist >> end >> >> functor Foo (Db_impl : MY_DB) = >> struct >> ... >> end > > The tricky thing here is that ML has been implemented > in two ways: > ML Richard, the suspense is killing me! :-) From erlangerintraining@REDACTED Sun Jul 20 00:26:07 2014 From: erlangerintraining@REDACTED (Kristoffer Brown) Date: Sat, 19 Jul 2014 16:26:07 -0600 Subject: [erlang-questions] Remove running node from mnesia cluster Message-ID: Hello all, I have a requirement to remove a running node from an mnesia cluster. This is a legitimate node that needs to have some maintenance performed. However, we want to keep this node running and servicing requests. I found this post. Which helps remove it from the additional nodes. However, once you re-start mnesia on the orphan node, it returns to the other nodes in the cluster. >From each of the non-orphan nodes, I run a script that does the following: rpc:call('node_to_be_orphaned', mnesia, stop, []), mnesia:del_table_copy(schema, 'node_to_be_orphaned'), ^^ At this point mnesia:system_info(db_nodes) shows that the node has indeed been removed. rpc:call('node_to_be_orphaned', mnesia, start, []), Now it's back. Ugh! So, I then tried to flip it and remove the other nodes from the orphan first adding the following. rpc:call(ThisNode, mnesia, stop, []), rpc:call('node_to_be_orphaned', mnesia, del_table_copy, [schema, node()]), rpc:call(ThisNode, mnesia, start, []), This just creates a loop with no difference. Is there a way to take a node out of mnesia clustering while leaving it up-and-running? Any and all guidance is greatly appreciated. Best. -------------- next part -------------- An HTML attachment was scrubbed... URL: From derekbrown121@REDACTED Sun Jul 20 02:14:27 2014 From: derekbrown121@REDACTED (Derek Brown) Date: Sat, 19 Jul 2014 20:14:27 -0400 Subject: [erlang-questions] Erlang projects for contributors? Message-ID: Any suggestions on projects for anyone looking to contribute? One list here: https://erlangcentral.org/erlang-projects Derek -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomasl_erlang@REDACTED Sun Jul 20 10:57:54 2014 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Sun, 20 Jul 2014 01:57:54 -0700 Subject: [erlang-questions] dialyzer and behaviours In-Reply-To: <53CAB5F0.9050009@khandkar.net> References: <53C7D634.8060500@khandkar.net> <20140717142906.GE89097@ferdair.local> <53CAB5F0.9050009@khandkar.net> Message-ID: <1405846674.89503.YahooMailNeo@web163602.mail.gq1.yahoo.com> "This is a somewhat controversial view that will probably upset somepeople, but _statistically_ creating and enforcing axiomatic abstractions is the only thing that qualifies as a type system. Dynamic typing just isn't typing (going even further, which is even more controversial (but accurate IMO), it is uni-typing)." Static typing people as a rule want to reject dynamic typing from consideration, usually reserving "typing" for their own work. (Do I detect a hint of Bob Harper in your text above?) But dynamic typing does however, for example, work well in Milner's sense of "does not go wrong", which is sufficient to many of us. There are other advantages too, like trivially handling incomplete or changing programs. Or for that matter allowing programs that are correct and do not go wrong, yet still are rejected by the type inference algorithm. In the larger scheme of things, we also have to consider other issues than typing purity tests. While there are many aspects to this, two points have remained with me. First, it's a family of restrictive disciplines combined with various inference algorithms that buys you some assurances of correctness, but utterly ignores other potentially important issues, including partial correctness. Do the gains outweigh the costs? It's a trade off. Second, in some sense, the preference for static vs dynamic seems more than anything else to be an issue of personality (perhaps reinforced by upbringing). For example, some love it that the type checker finds their "tons of errors" (as they often put it), while others prefer the shortcuts available when there are less restrictions, like having PIDs instead of typed channels. And since this same discussion has been recurring for me since the early 90s, with no apparent progress, and since this is not the mailing list to rehash these old issues, with that I'll try to leave the bait and unhook myself ;-) Best, Thomas On Saturday, July 19, 2014 8:16 PM, Siraaj Khandkar wrote: > > >On 07/17/2014 10:29 AM, Fred Hebert wrote: >> On 07/17, Siraaj Khandkar wrote: >>> In the face of such ambiguities, Dialyzer really has no choice but to >>> assume innocence until guilt can be proven with certainty. >>> >>> When you say `baz = foo:bar()`, Dialyzer can _try_ to follow the path >>> and match the expressions, but when you say `baz = Foo:bar()` - the rug >>> is swept from underneath Dialyzer's feet and (at that call site) it can >>> go no further than _try_ to find a case where Foo does not result in >>> being an atom. >>> >>> So in other words, Dialyzer is not a type checker - it only seeks >>> _definite_ failure cases in the call graph and sometimes >>> success-or-failure depends _only_ on runtime state (such as dynamic >>> references). >>> >> >> Someone who knows better about types, type systems, and type checking >> than me may correct me, but I believe that Dialyzer still is a type >> checker. >> >> The difference is that it checks a type system that isn't axiomatic >> (such as Hindley-Milner) -- those that tend to *prove* that there is no >> type errors in the program, sometimes at the cost of forbidding certain >> otherwise valid programs. >> >> The canonical example for this is is: >> >>? ? 'and'(true, true) -> true; >>? ? 'and'(_, _) -> false. >> >> being called as: >> >>? ? 'and'(5,6) >> >> A type inference for an axiomatic type system might decide that the >> 'and'/2 function's signature is >> >>? ? -spec 'and'(boolean(), boolean()) -> boolean()' > > >Very interesting point! To explore it, the key question to answer is: >"What is a type?" > >I struggled with that question for sometime, because I noticed that I >use similar reasoning, of creating building blocks, in languages with >_and_ without static type enforcement. So one way to think about "type" >is as a synonym for "abstraction". > >Going further, I think the most enlightening definition of "type" was >given by John Reynolds as: "a syntactic discipline for maintaining >levels of abstraction." [1] > >So, while related, a type is not the abstraction itself, but a tool for >enforcing it. More specifically, a _syntactic_ tool, that is - static. > >The 'and' example, I think, perfectly demonstrates why the enforcement >necessarily has to be static - the catch-all case breaks the abstraction >and by the time it hits runtime - it is too late and there's no >principled way to prevent it other than adding checks _syntactically_. > >This is a somewhat controversial view that will probably upset some >people, but _statistically_ creating and enforcing axiomatic >abstractions is the only thing that qualifies as a type system. Dynamic >typing just isn't typing (going even further, which is even more >controversial (but accurate IMO), it is uni-typing). > > >[1]: >http://www.cse.chalmers.se/edu/year/2010/course/DAT140_Types/Reynolds_typesabpara.pdf > >> >> and therefore ban the call. Dialyzer can in fact ban this one if you >> explicitly write that signature down, but won't infer this. If you have >> no signature and call `$ typer `, the output is: >> >>? ? -spec 'and'(_,_) -> boolean(). > >And this is the key point that makes me feel that Dialyzer is not quite >a type checker. Except... it DOES give you a way to syntactically define >and enforce abstraction, so it DOES do type checking, but _optionally_ >and _incompletely_ (as in the case of dynamic references discussed >earlier). So what to do? :) > >IMO it is an awesome static analysis tool that I cannot imagine using >Erlang without, but because of the unavoidable typing holes - it cannot >be thought of as a full-blown type checker in my mind. > > >> Erlang's semantics allow that call, and that is the correct type >> signature that will correspond to its behavior at runtime. > >Yes, Dialyzer is true to Erlang semantics, but Erlang semantics is >exactly what creates the abstraction holes in the first place. > > > >> So the reason for this is that Dialyzer type checks on a concept called >> 'success typing'. To simplify a lot (maybe too much), an axiomatic type >> checker will build its list of what is valid or not by starting with a >> null set (nothing is valid), and expanding it as it finds more >> information in your program. This leads to denying programs that are: >> >> 1. wrong all of the time for an execution path >> 2. wrong some of the time for an execution path (and therefore right >>? ? some of the time) >> 3. do not provide sufficient information to know something will always >>? ? be right (false negative) >> >> Success typing basically works the other way around. It starts assuming >> that everything is good, and then restricts what it thinks is valid as >> it finds more information. This leads to denying programs that are: >> >> 1. wrong all the time for an execution path >> >> Programs that may or may not fail, either due to lack of information or >> succeeding some of the time (or a combination of both) will be accepted >> by Dialyzer. >> >> Dialyzer still definitely checks your types, does analysis, does >> inference, etc. It just analyzes them differently to give you a >> different diagnostic, but for all intents and purposes, it is a type >> checker (on top of being able to find other discrepancies in your >> programs). >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serge@REDACTED Sun Jul 20 14:59:18 2014 From: serge@REDACTED (Serge Aleynikov) Date: Sun, 20 Jul 2014 08:59:18 -0400 Subject: [erlang-questions] Prevent overwriting sasl log on VM restart? In-Reply-To: References: Message-ID: I haven't tested it in R15B, but see no reason it shouldn't work as that SASL functionality has been in the distribution for ages. On Fri, Jul 18, 2014 at 9:17 AM, Youngkin, Rich < richard.youngkin@REDACTED> wrote: > Hi Serge, > > Thanks for the patch. Do you know if this will work on R15B? I'll > probably give it a test spin in any case. > > Thanks again, > Rich > > > On Thu, Jul 17, 2014 at 9:02 PM, Serge Aleynikov > wrote: > >> I wrote the attached patch when I dealt with the same issue last year. >> Haven't had time to submit this in a "proper" way. >> >> It adds the following option to the sasl config: >> >> {file, Filename, Modes} >> >> where Modes are the modes the filename is open with. Use [append] for >> appending on startup. >> >> Regards, >> >> Serge >> >> >> On Thu, Jul 17, 2014 at 5:32 PM, Youngkin, Rich < >> richard.youngkin@REDACTED> wrote: >> >>> Is there a way, other than using the built-in log rotate mechanism or >>> something like lager, to avoid having the sasl log overwritten when the VM >>> is re/started? >>> >>> Thanks, >>> Rich >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon Jul 21 02:25:49 2014 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 21 Jul 2014 12:25:49 +1200 Subject: [erlang-questions] dialyzer and behaviours In-Reply-To: <53CAECE3.6090409@khandkar.net> References: <53C75B4F.5050001@khandkar.net> <5C2210C7-E79B-45AD-A026-740FE4EC587D@cs.otago.ac.nz> <53CAECE3.6090409@khandkar.net> Message-ID: <6486D225-BF5F-44B4-8EAE-CE74092CDDB9@cs.otago.ac.nz> On 20/07/2014, at 10:10 AM, Siraaj Khandkar wrote: > On 07/17/2014 08:02 PM, Richard A. O'Keefe wrote: >> >> On 17/07/2014, at 5:12 PM, Siraaj Khandkar wrote: >>> In a statically typed language with a module >>> system it'd be something like: >>> >>> signature MY_DB = >>> sig >>> val table : unit -> iolist >>> end >>> >>> functor Foo (Db_impl : MY_DB) = >>> struct >>> ... >>> end >> >> The tricky thing here is that ML has been implemented >> in two ways: >> ML > > Richard, the suspense is killing me! :-) I did _write_ the second half of the message.... ML has been implemented in (at least) two ways: MLton: static whole-program compilation. SML/NJ, Poplog, &c: dynamic incremental loading. The incremental loading approach is the one that's relevant here, BUT 'loading' is assimilated to 'lexically nested inside'. So if you do use "A.ml"; this introduces a number of bindings into the static environment, then use "B.ml"; uses those bindings and introduces more, then use "A.ml"; introduces *new* bindings for A which *hide* the old ones from any subsequent loading, but DO NOT REPLACE the old bindings. B is still linked to the bindings from the old version of A. Here's an actual example. m% sml Standard ML of New Jersey v110.73 [built: Sun May 15 21:34:53 2011] - (* A.ml contains 'val x = 1' *) - use "A.ml"; [opening A.ml] val x = 1 : int val it = () : unit - (* B.ml contains 'val y = x' *) - use "B.ml"; [opening B.ml] val y = 1 : int val it = () : unit - (* in another window, A is now changed to 'val x = true' *) - use "A.ml"; [opening A.ml] val x = true : bool val it = () : unit - (* we see that y still refers to the old x *) - y; val it = 1 : int - So ML is not an example of a hot-loading system with strong types. It's close enough to hot loading for interactive development, provided you remember that when you "replace" a module you have to "replace" every module that depended on it, which is why CM exists. From martink@REDACTED Mon Jul 21 03:21:03 2014 From: martink@REDACTED (Martin Karlsson) Date: Mon, 21 Jul 2014 13:21:03 +1200 Subject: [erlang-questions] Remove running node from mnesia cluster In-Reply-To: References: Message-ID: > > > Is there a way to take a node out of mnesia clustering while leaving it > up-and-running? > > Hi Kristoffer, I'm no mnesia expert but I remember reading that to completely remove a node you need to delete the schema on the "orphaned" node before starting it again. Look under chapter 5.6 in mnesia user guide ( http://www.erlang.org/doc/apps/mnesia/Mnesia_chap5.html) So your script would look like this: rpc:call('node_to_be_orphaned', mnesia, stop, []), mnesia:del_table_copy(schema, 'node_to_be_orphaned'), mnesia:delete_schema(['node_to_be_orphaned']), rpc:call('node_to_be_orphaned', mnesia, start, []). Note that all data on the orphaned node will be lost. Cheers, Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From andreas@REDACTED Mon Jul 21 18:35:04 2014 From: andreas@REDACTED (Andreas Schumacher) Date: Mon, 21 Jul 2014 18:35:04 +0200 Subject: [erlang-questions] Fwd: FW: OTP Version In-Reply-To: References: <1405619050.32590.142801741.733CC21C@webmail.messagingengine.com> <20140717212143.GE794@ferdmbp.local> <53C8517B.90202@gmail.com> <1405637341.11464.142897873.731BCC5E@webmail.messagingengine.com> <53C969E2.2040609@gmail.com> Message-ID: Hi Michael, I am not sure how to address your concerns, especially your comments regarding "making the open source project [...] more of a proprietary project" and "the open source community should only be a community of alpha/beta testers", other than by referring to Rickard Green's motivations of the new versioning scheme and OTP_VERSION file on this mailing list a few months ago [1][2][3]. In general, as long as you use an unmodified / tagged / official OTP version, you can be sure that the OTP version number in OTP_VERSION is correct and that you are running a set of Erlang/OTP applications that work well together, because that combination has been thoroughly tested. Any change to that release, by patching or changing one or more applications, results in a modified / untagged / unofficial OTP version for which there is no official OTP version number, either; in that case, you neither can nor should rely on the OTP_VERSION file. Furthermore, the one who modifies an official OTP system is responsible for making its (modified) parts work together; that has nothing to do with generating consulting revenue. Although, this could be improved by providing better and more correct runtime dependency information. As you are well aware of, this is something we are trying to improve in future releases. [1] http://erlang.org/pipermail/erlang-questions/2014-February/077675.html [2] http://erlang.org/pipermail/erlang-questions/2014-April/078585.html [3] http://erlang.org/pipermail/erlang-questions/2014-April/078590.html Andreas --- Andreas Schumacher, Erlang/OTP, Ericsson AB On 18/07/14 20:39, "Michael Truog" wrote: >On 07/17/2014 03:49 PM, Tristan Sloughter wrote: >> Right, but it gets confusing because the OTP version IS 17.x.y not 17. >> So I'd argue if it can't be determined by the running system it >> shouldn't be part of system_info at all. >> >> And to make it more confusing, how does it know it is 17? Could there >> not be a mix of 17 and 18 apps in there? Maybe that is rejected? > >My main concern is that being unable to version all erlang applications >together helps to obscure what selection of erlang applications should be >used in production, making the open source project less transparent and >more of a proprietary project. That can help generate consulting revenue >for the few people that may track what erlang applications are best >together, but would further limit the growth of the community due to a >decrease in transparency. That pursues an idea that the open source >community should only be a community of alpha/beta testers. I understand >the source repo has a single version file which is meant to track a >version, but if that version is never used by anything in production, it >becomes meaningless. > >> >> On Thu, Jul 17, 2014, at 05:43 PM, Michael Truog wrote: >>> On 07/17/2014 02:21 PM, Fred Hebert wrote: >>>> On 07/17, Tristan Sloughter wrote: >>>>> If for all OTP 17.x.y the runtime is going to consider it 17: >>>>> >>>>> 1> erlang:system_info(otp_release). >>>>> "17" >>>>> >>>> For reverence, the new method to get the version according to the docs >>>> is: http://www.erlang.org/doc/system_principles/versions.html >>>> >>>> In an OTP source code tree, the OTP version can be read from >>>>the text >>>> file /OTP_VERSION. The absolute path to the >>>>file can be >>>> constructed by calling filename:join([code:root_dir(), >>>>"OTP_VERSION"]). >>>> >>>> In an installed OTP development system, the OTP version can be >>>>read from >>>> the text file /releases/>>> number>/OTP_VERSION. The absolute path to the file can by >>>>constructed by >>>> calling filename:join([code:root_dir(), "releases", >>>> erlang:system_info(otp_release), "OTP_VERSION"]). >>>> >>>> Do note that the version number can be followed by '**', meaning it >>>>was >>>> patched with otp_patch_apply. >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> So, the idea is that only the source code has a version. The binary >>> installation has no minor or release version, unless you check >>>individual >>> erlang applications. The reason given so far >>> (http://erlang.org/pipermail/erlang-questions/2014-April/078590.html), >>>is >>> it is too hard to provide the minor and release version because >>>different >>> installations are likely to use different source tree dependencies at >>> different versions, so we are likely to lose any dependable information >>> about what version set of erlang applications are most stable together. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay@REDACTED Mon Jul 21 23:07:50 2014 From: jay@REDACTED (Jay Nelson) Date: Mon, 21 Jul 2014 14:07:50 -0700 Subject: [erlang-questions] Recommendations on optional callbacks Message-ID: We have created a behaviour with -callbacks, but would like to simplify the implementation by providing default implementations of some of the callbacks. Using erlang:function_exported/3 to test for a user-supplied callback and instead calling a default implementation may work, but the compiler will complain for the user-defined module if no callback is supplied. What is the recommended best practice for this situation? The best I can come up with is to document the default implementation and leave it up to the user-defined module to call that function or provide its own so that the callback is always present. jay From steven.charles.davis@REDACTED Tue Jul 22 02:11:05 2014 From: steven.charles.davis@REDACTED (Steve Davis) Date: Mon, 21 Jul 2014 17:11:05 -0700 (PDT) Subject: [erlang-questions] Recommendations on optional callbacks In-Reply-To: References: Message-ID: I still prefer behaviors... src/my_evt.erl:5: Warning: undefined callback function handle_info/2 (behaviour 'gen_event') ...sometimes "newer" isn't necessarily "better". /s On Monday, July 21, 2014 4:08:06 PM UTC-5, Jay Nelson wrote: > > We have created a behaviour with -callbacks, but would like to simplify > the implementation by providing default implementations of some of the > callbacks. Using erlang:function_exported/3 to test for a user-supplied > callback and instead calling a default implementation may work, but the > compiler will complain for the user-defined module if no callback is > supplied. What is the recommended best practice for this situation? > The best I can come up with is to document the default implementation > and leave it up to the user-defined module to call that function or > provide its own so that the callback is always present. > > jay > > _______________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Tue Jul 22 05:05:12 2014 From: g@REDACTED (Garrett Smith) Date: Mon, 21 Jul 2014 20:05:12 -0700 Subject: [erlang-questions] Recommendations on optional callbacks In-Reply-To: References: Message-ID: On Mon, Jul 21, 2014 at 2:07 PM, Jay Nelson wrote: > We have created a behaviour with -callbacks, but would like to simplify > the implementation by providing default implementations of some of the > callbacks. Using erlang:function_exported/3 to test for a user-supplied > callback and instead calling a default implementation may work, but the > compiler will complain for the user-defined module if no callback is > supplied. What is the recommended best practice for this situation? > The best I can come up with is to document the default implementation > and leave it up to the user-defined module to call that function or > provide its own so that the callback is always present. e2 has lots of optional callbacks - they're just left out of the behaviour_info/1 list. I don't know what else you do here. You don't want to generate warnings for optional callbacks. Having optional callbacks is just plain common sense. Making people suffer boiler plate is awful. If there's a possible default behavior, the higher order module should implement it. For performance sake, the higher order module notes the list of exports on start_link (or other entry point) and stores them in state rather than test on each call. This is sort of obvious though. Garrett From bengt.kleberg@REDACTED Tue Jul 22 06:56:24 2014 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 22 Jul 2014 06:56:24 +0200 Subject: [erlang-questions] Recommendations on optional callbacks In-Reply-To: References: Message-ID: <1406004984.4667.3.camel@sekic1152.rnd.ki.sw.ericsson.se> Greetings, How is loading of a new callback handled, when checking for exported functions on start_link()? bengt On Mon, 2014-07-21 at 20:05 -0700, Garrett Smith wrote: > On Mon, Jul 21, 2014 at 2:07 PM, Jay Nelson wrote: > > We have created a behaviour with -callbacks, but would like to simplify > > the implementation by providing default implementations of some of the > > callbacks. Using erlang:function_exported/3 to test for a user-supplied > > callback and instead calling a default implementation may work, but the > > compiler will complain for the user-defined module if no callback is > > supplied. What is the recommended best practice for this situation? > > The best I can come up with is to document the default implementation > > and leave it up to the user-defined module to call that function or > > provide its own so that the callback is always present. > > e2 has lots of optional callbacks - they're just left out of the > behaviour_info/1 list. > > I don't know what else you do here. You don't want to generate > warnings for optional callbacks. > > Having optional callbacks is just plain common sense. Making people > suffer boiler plate is awful. If there's a possible default behavior, > the higher order module should implement it. > > For performance sake, the higher order module notes the list of > exports on start_link (or other entry point) and stores them in state > rather than test on each call. This is sort of obvious though. > > Garrett > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From a.shneyderman@REDACTED Tue Jul 22 10:28:41 2014 From: a.shneyderman@REDACTED (Alex Shneyderman) Date: Tue, 22 Jul 2014 04:28:41 -0400 Subject: [erlang-questions] Recommendations on optional callbacks In-Reply-To: References: Message-ID: elixir has a very nice solution to this. not sure if elixir is an option on your project. If not, at least in theory, you could use parse transforms (which are a bit of a pain - then again elixir will get rid of a lot of these pains) to add the missing methods and redirect to default implementations. On Mon, Jul 21, 2014 at 5:07 PM, Jay Nelson wrote: > We have created a behaviour with -callbacks, but would like to simplify > the implementation by providing default implementations of some of the > callbacks. Using erlang:function_exported/3 to test for a user-supplied > callback and instead calling a default implementation may work, but the > compiler will complain for the user-defined module if no callback is > supplied. What is the recommended best practice for this situation? > The best I can come up with is to document the default implementation > and leave it up to the user-defined module to call that function or > provide its own so that the callback is always present. > > jay > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue Jul 22 10:41:15 2014 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 22 Jul 2014 10:41:15 +0200 Subject: [erlang-questions] Recommendations on optional callbacks In-Reply-To: References: Message-ID: <53CE23AB.1000504@ninenines.eu> On 07/22/2014 05:05 AM, Garrett Smith wrote: > On Mon, Jul 21, 2014 at 2:07 PM, Jay Nelson wrote: >> We have created a behaviour with -callbacks, but would like to simplify >> the implementation by providing default implementations of some of the >> callbacks. Using erlang:function_exported/3 to test for a user-supplied >> callback and instead calling a default implementation may work, but the >> compiler will complain for the user-defined module if no callback is >> supplied. What is the recommended best practice for this situation? >> The best I can come up with is to document the default implementation >> and leave it up to the user-defined module to call that function or >> provide its own so that the callback is always present. Don't use -behaviour. That functionality gives you two things: a check that the functions are exported (pointless for optional callbacks), and a *default* -spec that is generally much larger than what the function actually does. This does *not* saves you having to write the -spec for them if you care about typespecs. > For performance sake, the higher order module notes the list of > exports on start_link (or other entry point) and stores them in state > rather than test on each call. This is sort of obvious though. Terrible idea. This screws code reloading and isn't actually optimizing anything in a real world scenario. -- Lo?c Hoguin http://ninenines.eu From davidnwelton@REDACTED Tue Jul 22 11:57:55 2014 From: davidnwelton@REDACTED (David Welton) Date: Tue, 22 Jul 2014 11:57:55 +0200 Subject: [erlang-questions] HEART_COMMAND scripts Message-ID: Hi, I've been doing some digging into a few minor problems with the excellent relx (Erlang needs more tools like this!), and came across one thing that I filed a bug report for: https://github.com/erlware/relx/issues/213 I wanted to ask a wider audience about HEART_COMMAND scripts and how they are used. You you just use it to restart the Erlang node? Or do people do other things in there? I'm thinking of doing some fancier scripting, like attempting to notify the service team of the problem, checking how many restarts have happened in the past N minutes, and of course also restarting Erlang if there haven't been too many restarts. Steve Vinoski has some good advice here: http://steve.vinoski.net/blog/2009/02/22/controlling-erlangs-heart/ which I think I'll be integrating in the script we end up with. Thanks -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From z@REDACTED Tue Jul 22 13:02:54 2014 From: z@REDACTED (Danil Zagoskin) Date: Tue, 22 Jul 2014 15:02:54 +0400 Subject: [erlang-questions] Memory consuption: what is 'system' an how to compact Message-ID: Hi! Our application has allocated over 11 GB of memory and I don't understand why. The application contains: - HTTPS server using old mochiweb and OTP SSL - interfaces to internal services using lhttpc - caches using ETS First problem: over 11 GB is allocated but only 5 GB are used: > erlang:memory(). [{total,5154952544}, {processes,1893901080}, {processes_used,1893832024}, {system,3261051464}, {atom,553569}, {atom_used,533811}, {binary,520767552}, {code,12580030}, {ets,154178408}] I understand that 50% allocator usage ratio is subject to changing allocation strategy. Question: Are there any tips on choosing one apart of "try and see what happens"? Second problem: as you can see above, system consumes 3.2 GB when atom, binary, code, ets sum up to less than 1 GB. Question: Where did 2.5 GB go? Earlier (R16B+) this application could run for weeks with half (13K) of current (28K) connection count and use less than 2 GB. The most significant change about memory is a small binary kept in each worker's dictionary. Third problem: MBCSs = [proplists:get_value(mbcs_usage, Info) || {{binary_alloc,_}, Info} <- recon_alloc:fragmentation(current)], lists:sum(MBCSs)/length(MBCSs). 0.13687142195007596 Question: is there a way to reallocate long-living binaries without changing the code so that they don't hold this much memory? Question: How can I list binaries in single carrier? I want to see contents and referencing pids. Question: Given an entry from process_info(Pid, binary) like this: {139767104723144,19,1} how can I see carrier ID and contents of this binary? I have recon_alloc snapshot and can send it if needed. -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From s.j.thompson@REDACTED Tue Jul 22 13:42:56 2014 From: s.j.thompson@REDACTED (Simon Thompson) Date: Tue, 22 Jul 2014 12:42:56 +0100 Subject: [erlang-questions] Research job: closing date today Message-ID: Today's the closing date for this one year researcher job at Kent. Applications very welcome! http://www.jobs.ac.uk/job/AJD705/research-associate-in-functional-and-concurrent-programming/ Simon Simon Thompson | Professor of Logic and Computation School of Computing | University of Kent | Canterbury, CT2 7NF, UK s.j.thompson@REDACTED | M +44 7986 085754 | W www.cs.kent.ac.uk/~sjt From mfidelman@REDACTED Tue Jul 22 13:53:05 2014 From: mfidelman@REDACTED (Miles Fidelman) Date: Tue, 22 Jul 2014 07:53:05 -0400 Subject: [erlang-questions] design pattern question for messaging system Message-ID: <53CE50A1.3030400@meetinghouse.net> Hi Folks, So far, I've mostly been experimenting w/ Erlang, and using Erlang-based technology (notably CouchDB). As I'm thinking about a new application, I'm having trouble getting my hands around an appropriate design pattern. I wonder if anybody might be able to point me in the right direction. The application is message handling (back to that in a minute). I realize that I have a pretty good idea how to handle some kinds of applications in a highly concurrent fashion, such as: - modeling/simulation (obviously, each entity - such as a vehicle - is a process) - this is what led me to Erlang in the first place - protocol engines as state machines - e.g., spawn a process for each tcp connection - transaction systems - spawn a process for each transaction - transaction oriented But I'm looking at a work flow application that maps onto a paper-forms-based model. It's a classic queuing system - work elements move from queue to queue as they're worked on. The obvious first thought is: - a process for each queue - a worker process for each work step - a message for each piece of work-in-process -- moving from queue to queue via the worker processes Except, that model kind of falls down because Erlang message are unreliable by design, and don't persist in the event of a process crash (much less a node crash). My first two thoughts are: - spawn a process for each queue entry, pass around the PIDs - use Mnesia to hold the queues But neither of those feels quite right. This must be a solved problem, but I'm hitting a blind spot. So... what is the design pattern for queuing systems and/or reliable message passing in Erlang? Any good examples to look at? Good presentation slides or reference materials to review? Thanks very much, Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From jesper.louis.andersen@REDACTED Tue Jul 22 14:00:42 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 22 Jul 2014 14:00:42 +0200 Subject: [erlang-questions] design pattern question for messaging system In-Reply-To: <53CE50A1.3030400@meetinghouse.net> References: <53CE50A1.3030400@meetinghouse.net> Message-ID: On Tue, Jul 22, 2014 at 1:53 PM, Miles Fidelman wrote: > But neither of those feels quite right. This must be a solved problem, > but I'm hitting a blind spot. So... what is the design pattern for queuing > systems and/or reliable message passing in Erlang? Make each message a process. Skip the queues. Each message runs its own state to completion. If you need some kind of capacity constraint, use something like uwiger/jobs or jlouis/safetyvalve, poolboy, basho/sidejob or likewise. The trick is to dualize the world since Erlang has no channel concept which naturally maps to a queue. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Tue Jul 22 14:51:50 2014 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 22 Jul 2014 08:51:50 -0400 Subject: [erlang-questions] Recommendations on optional callbacks In-Reply-To: References: Message-ID: <20140722125149.GF55558@ferdmbp.local> Erlang had a feature for this before Elixir even existed, and it was taken out for various reasons (see http://www.erlang.org/news/35) For backwards compatibility reasons, a parse transform supporting it (the -extends(...). attribute) and parametrized modules was released as a parse transform library: https://github.com/erlang/pmod_transform The way to do it in OTP right now is just not to put the optional call into the list of -callback or -behaviour_info, and dynamically check if it's available. The thing then gets documented, as is done for `format_status` for all worker behaviours in OTP: - http://www.erlang.org/doc/man/gen_server.html#Module:format_status-2 - http://www.erlang.org/doc/man/gen_fsm.html#Module:format_status-2 - http://www.erlang.org/doc/man/gen_event.html#Module:format_status-2 This does have the downside of not being able to provide type information throught the -callback attributes however. Regards, Fred. On 07/22, Alex Shneyderman wrote: > elixir has a very nice solution to this. not sure if elixir is an option on > your project. If not, at least in theory, you could use parse transforms > (which are a bit of a pain - then again elixir will get rid of a lot of > these pains) to add the missing methods and redirect to default > implementations. > > > On Mon, Jul 21, 2014 at 5:07 PM, Jay Nelson wrote: > > > We have created a behaviour with -callbacks, but would like to simplify > > the implementation by providing default implementations of some of the > > callbacks. Using erlang:function_exported/3 to test for a user-supplied > > callback and instead calling a default implementation may work, but the > > compiler will complain for the user-defined module if no callback is > > supplied. What is the recommended best practice for this situation? > > The best I can come up with is to document the default implementation > > and leave it up to the user-defined module to call that function or > > provide its own so that the callback is always present. > > > > jay > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From federico.carrone@REDACTED Tue Jul 22 17:16:11 2014 From: federico.carrone@REDACTED (Federico Carrone) Date: Tue, 22 Jul 2014 12:16:11 -0300 Subject: [erlang-questions] Recommendations on optional callbacks In-Reply-To: References: Message-ID: I do not know if this information might help you, but it might be useful for somebody else: I am using mixer (https://github.com/opscode/mixer) from the Chef guys for solving this issue when coding cowboy handlers. I have a default cowboy rest handler that implements all the default implementation. All the handlers use the default cowboy rest handler implementation if I they do not implement functions like is_authorized or forbidden. Regards, Federico. On Mon, Jul 21, 2014 at 6:07 PM, Jay Nelson wrote: > We have created a behaviour with -callbacks, but would like to simplify > the implementation by providing default implementations of some of the > callbacks. Using erlang:function_exported/3 to test for a user-supplied > callback and instead calling a default implementation may work, but the > compiler will complain for the user-defined module if no callback is > supplied. What is the recommended best practice for this situation? > The best I can come up with is to document the default implementation > and leave it up to the user-defined module to call that function or > provide its own so that the callback is always present. > > jay > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- http://federicocarrone.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Tue Jul 22 17:40:34 2014 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Tue, 22 Jul 2014 11:40:34 -0400 Subject: [erlang-questions] Blog :: Building an ARMy of Xen unikernels Message-ID: Hello, Given that ARM dev boards are ever cheaper and more capable, wouldn't it be cool to have something like this in Erlang? http://openmirage.org/blog/introducing-xen-minios-arm Best, LRP Sent from my iPad From dpezely@REDACTED Tue Jul 22 19:35:03 2014 From: dpezely@REDACTED (Daniel Pezely) Date: Tue, 22 Jul 2014 17:35:03 +0000 Subject: [erlang-questions] design pattern question for messaging system In-Reply-To: <53CE50A1.3030400@meetinghouse.net> References: <53CE50A1.3030400@meetinghouse.net> Message-ID: > Except, that model kind of falls down because Erlang message are unreliable by design, and don't persist in the event of a process crash (much less a node crash). You can get the best of what Erlang offers while also adding a journal mechanism in much the same way that many traditional databases handle transactions reliably. There is a cascade effect of features: 1. For every message that you send, first log it to disk. Then in event of a complete system crash you can replay the journal. 2. Next is to simultaneously write two pieces of information: a flag to be set later upon completion of action following message delivery as well as writing the original message itself, of course. 3. This in turn requires updating the status flag upon completion of action which should also be a distinct node on your sequence diagram. (Any I/O operation should be noted.) These could be all in one file or two depending upon various factors: speed of which you generate messages, speed of which you process messages, number & nature of storage devices, whether you are on AWS style ephemeral server instances where semantics of service provider's local versus network attached storage is critical to understand, etc. Some tricks include writing a header for every N messages such that you can perform bulk updates, thereby iterating at a granularity of large blocks rather than several bytes at a time and minimize seek operations during nominal operations. It also depends upon how you track state internally, such that you may be able to completely forgo the status flag on disk. Lots of directions that you could take this... -Daniel -- dpezely@REDACTED ________________________________________ From: erlang-questions-bounces@REDACTED [erlang-questions-bounces@REDACTED] on behalf of Miles Fidelman [mfidelman@REDACTED] Sent: Tuesday, July 22, 2014 4:53 AM To: erlang-questions Questions Subject: [erlang-questions] design pattern question for messaging system Hi Folks, So far, I've mostly been experimenting w/ Erlang, and using Erlang-based technology (notably CouchDB). As I'm thinking about a new application, I'm having trouble getting my hands around an appropriate design pattern. I wonder if anybody might be able to point me in the right direction. The application is message handling (back to that in a minute). I realize that I have a pretty good idea how to handle some kinds of applications in a highly concurrent fashion, such as: - modeling/simulation (obviously, each entity - such as a vehicle - is a process) - this is what led me to Erlang in the first place - protocol engines as state machines - e.g., spawn a process for each tcp connection - transaction systems - spawn a process for each transaction - transaction oriented But I'm looking at a work flow application that maps onto a paper-forms-based model. It's a classic queuing system - work elements move from queue to queue as they're worked on. The obvious first thought is: - a process for each queue - a worker process for each work step - a message for each piece of work-in-process -- moving from queue to queue via the worker processes Except, that model kind of falls down because Erlang message are unreliable by design, and don't persist in the event of a process crash (much less a node crash). My first two thoughts are: - spawn a process for each queue entry, pass around the PIDs - use Mnesia to hold the queues But neither of those feels quite right. This must be a solved problem, but I'm hitting a blind spot. So... what is the design pattern for queuing systems and/or reliable message passing in Erlang? Any good examples to look at? Good presentation slides or reference materials to review? Thanks very much, Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From dmkolesnikov@REDACTED Tue Jul 22 21:58:08 2014 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Tue, 22 Jul 2014 22:58:08 +0300 Subject: [erlang-questions] design pattern question for messaging system In-Reply-To: <53CE50A1.3030400@meetinghouse.net> References: <53CE50A1.3030400@meetinghouse.net> Message-ID: Hello, "Except, that model kind of falls down because Erlang message are unreliable by design, and don't persist in the event of a process crash (much less a node crash).? This is a trade off you have to accept. If the message persistency is MUST then you have to look into reliable message broker solution then. I think you already articulate the basic pattern, which is used in many applications ?[ mq ]?[ worker ]?[ mq ]-[ worker ]? You are right if you keep messages within mailbox then crash of process destroys mailbox. Thus, you need an intermediate process to hold messages. The intermediate process does not do any work except enqueue / dequeue operation. Therefore, the number of failures is limited. The worker is pool of processes Jesper Louis gave you list pool libraries, I can give one more if you need ;-) I am using a similar pattern at one of my application. The biggest problem is node crash due to OOM or external factors. I am trying to solve it by duplicating the processing path into N-distinct node (my app uses last-write wins but you might use other conflict resolution technique). You can extend the pattern by having persistent mq to check-point intermediate results to survive node crash but I found it complicated. ?[ p-mq ]?[ worker ]?[ mq ]?[ worker ]?[ mq ]?[ worker ]?[ p-mq ]- Best Regards, Dmitry On 22 Jul 2014, at 14:53, Miles Fidelman wrote: > Hi Folks, > > So far, I've mostly been experimenting w/ Erlang, and using Erlang-based technology (notably CouchDB). As I'm thinking about a new application, I'm having trouble getting my hands around an appropriate design pattern. I wonder if anybody might be able to point me in the right direction. > > The application is message handling (back to that in a minute). I realize that I have a pretty good idea how to handle some kinds of applications in a highly concurrent fashion, such as: > - modeling/simulation (obviously, each entity - such as a vehicle - is a process) - this is what led me to Erlang in the first place > - protocol engines as state machines - e.g., spawn a process for each tcp connection > - transaction systems - spawn a process for each transaction > - transaction oriented > > But I'm looking at a work flow application that maps onto a paper-forms-based model. It's a classic queuing system - work elements move from queue to queue as they're worked on. The obvious first thought is: > - a process for each queue > - a worker process for each work step > - a message for each piece of work-in-process -- moving from queue to queue via the worker processes > > Except, that model kind of falls down because Erlang message are unreliable by design, and don't persist in the event of a process crash (much less a node crash). > > My first two thoughts are: > - spawn a process for each queue entry, pass around the PIDs > - use Mnesia to hold the queues > > But neither of those feels quite right. This must be a solved problem, but I'm hitting a blind spot. So... what is the design pattern for queuing systems and/or reliable message passing in Erlang? > > Any good examples to look at? Good presentation slides or reference materials to review? > > Thanks very much, > > Miles Fidelman > > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From gleber.p@REDACTED Wed Jul 23 06:52:24 2014 From: gleber.p@REDACTED (Gleb Peregud) Date: Wed, 23 Jul 2014 06:52:24 +0200 Subject: [erlang-questions] Generate .core files for all .erl files in OTP Message-ID: Hey I am toying with my Erlang interpreter and I am trying to make it more compatible with OTP. The best way I know would be to actually run OTP test suite using my interpreter, but my interpreter works on Core Erlang instead of BEAM bytecode or plain Erlang source code. Hence I need to compile all of OTP to produce Core Erlang for each .erl file. Any ideas how to do it nicely? Cheers, Gleb From erlang-programming+noreply@REDACTED Wed Jul 23 08:31:32 2014 From: erlang-programming+noreply@REDACTED (Webmaster Erlang/OTP (Google Groups)) Date: Wed, 23 Jul 2014 06:31:32 +0000 Subject: [erlang-questions] Google Groups: You've been added to Erlang Programming Message-ID: <53cf56c4.26548c0a.2c3a.0024.GMR@mx.google.com> The mailing list erlang-questions@REDACTED is now added as a member to the Google Group erlang-programming@REDACTED Hopefully this means that the Google Group members will not have to cross post to both lists. -- erlang-questions-owner@REDACTED About this group: Erlang Programming, mirror of erlang-questions@REDACTED Subscribe here to be able to post: http://www.erlang.org/mailman/listinfo/erlang-questions Remember to send posts to the list address. ----------------------- Google Groups Information ---------------------- The owner of the group has set your subscription type as "Email", meaning that you'll receive a copy of every message posted to the group as they are posted. Visit this group: http://groups.google.com/d/forum/erlang-programming?hl=en You can unsubscribe from this group using the following URL: http://groups.google.com/d/forum/erlang-programming/unsubscribe/iWJKdhQAAADvQ36zzcHOS5uHsuYcY-SrIyy_I5cLHxn_3M5TwkwAvg?hl=en If you feel that this message is abuse, please inform the Google Groups staff by using the URL below. http://groups.google.com/d/abuse/YQAAACC_wbPlAAAAx6m8fEUAAAC4RhNdTfF2H-BZBamLGeQNtXniVwY?hl=en -------------- next part -------------- An HTML attachment was scrubbed... URL: From slashfoo@REDACTED Wed Jul 23 11:36:40 2014 From: slashfoo@REDACTED (Jamiel Almeida) Date: Wed, 23 Jul 2014 02:36:40 -0700 (PDT) Subject: [erlang-questions] Hiding user input in escript In-Reply-To: <5148EFDB.8040909@go-factory.net> References: <5148D2DA.6080604@go-factory.net> <5148EFDB.8040909@go-factory.net> Message-ID: Was this ever submitted as a bug? and if so, do you have a URL to track? On Tuesday, March 19, 2013 4:08:11 PM UTC-7, Bryan Hughes wrote: > > Hi Bob, > > Thanks for checking it out! Guess not too many people are writing scripts > that need to deal with passwords. I will go ahead and submit it as a bug. > > Cheers, > Bryan > > On 3/19/13 4:01 PM, Bob Ippolito wrote: > > R16B does not change anything for escript as far as I can tell. > > On Tue, Mar 19, 2013 at 3:17 PM, Bob Ippolito > wrote: > >> I see io:get_password() in my io module in R15B03, but it doesn't appear >> to work with escript because it uses the old 'user' rather than 'group' >> which doesn't understand the get_password io request. Haven't looked at >> R16B yet. >> >> >> >> On Tue, Mar 19, 2013 at 2:04 PM, Bryan Hughes > > wrote: >> >>> Hi Everyone, >>> >>> I found this old thread from 2009 about trying to hide user input in an >>> escript (e.g. entering a password). >>> >>> http://erlang.org/pipermail/erlang-questions/2009-April/043311.html >>> >>> It concluded with: >>> >>> The problem seems to be that escript uses the old shell, which >>> does not support any way to control the terminal. >>> >>> We'll have a look at escript for the next release and see whether >>> it would be possible to have it run with the standard shell. >>> >>> >>> Does anyone know if this has been fixed. I looked through the docs and >>> do not find io:get_password/0, or any options for reading stdin to not echo >>> the characters back. >>> >>> Looking forward to meeting those of you attending Erlang Factory in a >>> couple of days. >>> >>> Cheers, >>> B >>> >>> -- >>> >>> Bryan Hughes >>> *Go Factory* >>> http://www.go-factory.net >>> >>> *"Art is never finished, only abandoned. - Leonardo da Vinci"* >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-q...@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > > -- > > Bryan Hughes > CTO and Founder / *Go Factory* > (415) 515-7916 > http://www.go-factory.net > > *"Art is never finished, only abandoned. - Leonardo da Vinci"* > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard.youngkin@REDACTED Wed Jul 23 16:26:54 2014 From: richard.youngkin@REDACTED (Youngkin, Rich) Date: Wed, 23 Jul 2014 08:26:54 -0600 Subject: [erlang-questions] design pattern question for messaging Message-ID: Hi, An application I'm working on has a similar requirement regarding message persistence. We elected to use RabbitMQ. One notable difference between this app and the one you're describing is we're not moving a message between states as in a work-flow model. But as Dmitry said below, "if the message persistency is a MUST then you have to look into a reliable message broker..." or some other way to manage persistence. I may have missed it, but in Jesper's reply I didn't see the persistence requirement addressed. Jesper has a nice write-up "On Erlang, State and Crashes that you might find useful ( http://jlouisramblings.blogspot.com/2010/11/on-erlang-state-and-crashes.html ). Cheers, Rich Date: Tue, 22 Jul 2014 22:58:08 +0300 From: Dmitry Kolesnikov To: Miles Fidelman Cc: erlang-questions Questions Subject: Re: [erlang-questions] design pattern question for messaging system Message-ID: Content-Type: text/plain; charset=windows-1252 Hello, "Except, that model kind of falls down because Erlang message are unreliable by design, and don't persist in the event of a process crash (much less a node crash).? This is a trade off you have to accept. If the message persistency is MUST then you have to look into reliable message broker solution then. I think you already articulate the basic pattern, which is used in many applications ?[ mq ]?[ worker ]?[ mq ]-[ worker ]? You are right if you keep messages within mailbox then crash of process destroys mailbox. Thus, you need an intermediate process to hold messages. The intermediate process does not do any work except enqueue / dequeue operation. Therefore, the number of failures is limited. The worker is pool of processes Jesper Louis gave you list pool libraries, I can give one more if you need ;-) I am using a similar pattern at one of my application. The biggest problem is node crash due to OOM or external factors. I am trying to solve it by duplicating the processing path into N-distinct node (my app uses last-write wins but you might use other conflict resolution technique). You can extend the pattern by having persistent mq to check-point intermediate results to survive node crash but I found it complicated. ?[ p-mq ]?[ worker ]?[ mq ]?[ worker ]?[ mq ]?[ worker ]?[ p-mq ]- Best Regards, Dmitry On 22 Jul 2014, at 14:53, Miles Fidelman wrote: > Hi Folks, > > So far, I've mostly been experimenting w/ Erlang, and using Erlang-based technology (notably CouchDB). As I'm thinking about a new application, I'm having trouble getting my hands around an appropriate design pattern. I wonder if anybody might be able to point me in the right direction. > > The application is message handling (back to that in a minute). I realize that I have a pretty good idea how to handle some kinds of applications in a highly concurrent fashion, such as: > - modeling/simulation (obviously, each entity - such as a vehicle - is a process) - this is what led me to Erlang in the first place > - protocol engines as state machines - e.g., spawn a process for each tcp connection > - transaction systems - spawn a process for each transaction > - transaction oriented > > But I'm looking at a work flow application that maps onto a paper-forms-based model. It's a classic queuing system - work elements move from queue to queue as they're worked on. The obvious first thought is: > - a process for each queue > - a worker process for each work step > - a message for each piece of work-in-process -- moving from queue to queue via the worker processes > > Except, that model kind of falls down because Erlang message are unreliable by design, and don't persist in the event of a process crash (much less a node crash). > > My first two thoughts are: > - spawn a process for each queue entry, pass around the PIDs > - use Mnesia to hold the queues > > But neither of those feels quite right. This must be a solved problem, but I'm hitting a blind spot. So... what is the design pattern for queuing systems and/or reliable message passing in Erlang? > > Any good examples to look at? Good presentation slides or reference materials to review? > > Thanks very much, > > Miles Fidelman -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfidelman@REDACTED Wed Jul 23 16:34:46 2014 From: mfidelman@REDACTED (Miles Fidelman) Date: Wed, 23 Jul 2014 10:34:46 -0400 Subject: [erlang-questions] design pattern question for messaging In-Reply-To: References: Message-ID: <53CFC806.1050205@meetinghouse.net> Hi Rich, Jesper's writeup is interesting. I guess a question that now comes to mind is: How does RabbitMQ implement message persistence? After all, it's written in Erlang. Off to do some research (unless someone can point to a good architectural reference). Thanks, Miles Youngkin, Rich wrote: > Hi, > > An application I'm working on has a similar requirement regarding > message persistence. We elected to use RabbitMQ. One notable > difference between this app and the one you're describing is we're not > moving a message between states as in a work-flow model. But as Dmitry > said below, "if the message persistency is a MUST then you have to > look into a reliable message broker..." or some other way to manage > persistence. > > I may have missed it, but in Jesper's reply I didn't see the > persistence requirement addressed. Jesper has a nice write-up "On > Erlang, State and Crashes that you might find useful > (http://jlouisramblings.blogspot.com/2010/11/on-erlang-state-and-crashes.html). > > > Cheers, > Rich > > Date: Tue, 22 Jul 2014 22:58:08 +0300 > From: Dmitry Kolesnikov > > To: Miles Fidelman > > Cc: erlang-questions Questions > > Subject: Re: [erlang-questions] design pattern question for messaging > system > Message-ID: > > Content-Type: text/plain; charset=windows-1252 > > Hello, > > "Except, that model kind of falls down because Erlang message are > unreliable by design, and don't persist in the event of a process > crash (much less a node crash).? > > This is a trade off you have to accept. If the message persistency is > MUST then you have to look into reliable message broker solution then. > > I think you already articulate the basic pattern, which is used in > many applications > > ?[ mq ]?[ worker ]?[ mq ]-[ worker ]? > > You are right if you keep messages within mailbox then crash of > process destroys mailbox. Thus, you need an intermediate process to > hold messages. The intermediate process does not do any work except > enqueue / dequeue operation. Therefore, the number of failures is > limited. The worker is pool of processes Jesper Louis gave you list > pool libraries, I can give one more if you need ;-) > > I am using a similar pattern at one of my application. The biggest > problem is node crash due to OOM or external factors. I am trying to > solve it by duplicating the processing path into N-distinct node (my > app uses last-write wins but you might use other conflict resolution > technique). > > You can extend the pattern by having persistent mq to check-point > intermediate results to survive node crash but I found it complicated. > > ?[ p-mq ]?[ worker ]?[ mq ]?[ worker ]?[ mq ]?[ worker ]?[ p-mq ]- > > Best Regards, > Dmitry > > On 22 Jul 2014, at 14:53, Miles Fidelman > wrote: > > > Hi Folks, > > > > So far, I've mostly been experimenting w/ Erlang, and using > Erlang-based technology (notably CouchDB). As I'm thinking about a > new application, I'm having trouble getting my hands around an > appropriate design pattern. I wonder if anybody might be able to > point me in the right direction. > > > > The application is message handling (back to that in a minute). I > realize that I have a pretty good idea how to handle some kinds of > applications in a highly concurrent fashion, such as: > > - modeling/simulation (obviously, each entity - such as a vehicle - is > a process) - this is what led me to Erlang in the first place > > - protocol engines as state machines - e.g., spawn a process for each > tcp connection > > - transaction systems - spawn a process for each transaction > > - transaction oriented > > > > But I'm looking at a work flow application that maps onto a > paper-forms-based model. It's a classic queuing system - work > elements move from queue to queue as they're worked on. The obvious > first thought is: > > - a process for each queue > > - a worker process for each work step > > - a message for each piece of work-in-process -- moving from queue to > queue via the worker processes > > > > Except, that model kind of falls down because Erlang message are > unreliable by design, and don't persist in the event of a process > crash (much less a node crash). > > > > My first two thoughts are: > > - spawn a process for each queue entry, pass around the PIDs > > - use Mnesia to hold the queues > > > > But neither of those feels quite right. This must be a solved > problem, but I'm hitting a blind spot. So... what is the design > pattern for queuing systems and/or reliable message passing in Erlang? > > > > Any good examples to look at? Good presentation slides or reference > materials to review? > > > > Thanks very much, > > > > Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From jesper.louis.andersen@REDACTED Wed Jul 23 16:53:00 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 23 Jul 2014 16:53:00 +0200 Subject: [erlang-questions] design pattern question for messaging In-Reply-To: References: Message-ID: On Wed, Jul 23, 2014 at 4:26 PM, Youngkin, Rich < richard.youngkin@REDACTED> wrote: > I may have missed it, but in Jesper's reply I didn't see the persistence > requirement addressed. One way to approach persistence is to have each process track its state on disk whenever it is sure its state is consistent. The trick is to keep the amount of persist-calls low so most of the way forward moves on volatile messages which are fast. In the event of a crash you reset to the last known stable point and continue on from there. For smaller systems, tracking state in mnesia is an obvious candidate. For larger systems, you can use something like Riak. RabbitMQ implements its own persistence layer and just makes sure that data are sync'ed to disk before it accepts the message from the client. In the event of an error, RabbitMQ can then reconstruct its queue state from the disk log. This is one of the reasons never-persisted messages are much faster in RabbitMQ than messages you persist on disk. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bryan@REDACTED Wed Jul 23 16:54:56 2014 From: bryan@REDACTED (Bryan) Date: Wed, 23 Jul 2014 07:54:56 -0700 Subject: [erlang-questions] Hiding user input in escript In-Reply-To: References: <5148D2DA.6080604@go-factory.net> <5148EFDB.8040909@go-factory.net> Message-ID: Howdy. I believe I submitted a bug, but never followed up and no longer have my sent emails from back then ? so not 100% sure. Cheers, Bryan ---- Bryan Hughes CTO and Founder / Go Factory (415) 515-7916 http://www.go-factory.net Connecting the Internet of Things On Jul 23, 2014, at 2:36 AM, Jamiel Almeida wrote: > Was this ever submitted as a bug? and if so, do you have a URL to track? > > On Tuesday, March 19, 2013 4:08:11 PM UTC-7, Bryan Hughes wrote: > Hi Bob, > > Thanks for checking it out! Guess not too many people are writing scripts that need to deal with passwords. I will go ahead and submit it as a bug. > > Cheers, > Bryan > > On 3/19/13 4:01 PM, Bob Ippolito wrote: >> R16B does not change anything for escript as far as I can tell. >> >> On Tue, Mar 19, 2013 at 3:17 PM, Bob Ippolito wrote: >> I see io:get_password() in my io module in R15B03, but it doesn't appear to work with escript because it uses the old 'user' rather than 'group' which doesn't understand the get_password io request. Haven't looked at R16B yet. >> >> >> >> On Tue, Mar 19, 2013 at 2:04 PM, Bryan Hughes wrote: >> Hi Everyone, >> >> I found this old thread from 2009 about trying to hide user input in an escript (e.g. entering a password). >> >> http://erlang.org/pipermail/erlang-questions/2009-April/043311.html >> >> It concluded with: >>> The problem seems to be that escript uses the old shell, which >>> does not support any way to control the terminal. >>> >>> We'll have a look at escript for the next release and see whether >>> it would be possible to have it run with the standard shell. >> >> Does anyone know if this has been fixed. I looked through the docs and do not find io:get_password/0, or any options for reading stdin to not echo the characters back. >> >> Looking forward to meeting those of you attending Erlang Factory in a couple of days. >> >> Cheers, >> B >> >> -- >> Bryan Hughes >> Go Factory >> http://www.go-factory.net >> >> "Art is never finished, only abandoned. - Leonardo da Vinci" >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-q...@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > > -- > Bryan Hughes > CTO and Founder / Go Factory > (415) 515-7916 > http://www.go-factory.net > > "Art is never finished, only abandoned. - Leonardo da Vinci" > -------------- next part -------------- An HTML attachment was scrubbed... URL: From leonard.boyce@REDACTED Wed Jul 23 17:08:06 2014 From: leonard.boyce@REDACTED (Leonard Boyce) Date: Wed, 23 Jul 2014 11:08:06 -0400 Subject: [erlang-questions] design pattern question for messaging In-Reply-To: References: Message-ID: On Wed, Jul 23, 2014 at 10:53 AM, Jesper Louis Andersen wrote: > > On Wed, Jul 23, 2014 at 4:26 PM, Youngkin, Rich > wrote: >> >> I may have missed it, but in Jesper's reply I didn't see the persistence >> requirement addressed. > > > One way to approach persistence is to have each process track its state on > disk whenever it is sure its state is consistent. The trick is to keep the > amount of persist-calls low so most of the way forward moves on volatile > messages which are fast. In the event of a crash you reset to the last known > stable point and continue on from there. > > For smaller systems, tracking state in mnesia is an obvious candidate. For > larger systems, you can use something like Riak. > > RabbitMQ implements its own persistence layer and just makes sure that data > are sync'ed to disk before it accepts the message from the client. In the > event of an error, RabbitMQ can then reconstruct its queue state from the > disk log. This is one of the reasons never-persisted messages are much > faster in RabbitMQ than messages you persist on disk. > Just thought I'd throw in my 2c as we use RabbitMQ exactly for this. We use durable queues/exchanges and delivery_mode = 2 (persisted) messages. Each message carries its own 'State' as it moves between nodes and queues. Messages are only ack'd after the results are sent onto the next stage (yes, there is a possible race condition with duplicate delivery bu that's handled elsewhere) so that crashes in nodes or processes allow for another worker on some other node to handle the message. While it's not the 'fastest' method it has been incredibly stable for almost 3 years of 24/7 operation. Leonard > -- > J. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mattevans123@REDACTED Wed Jul 23 17:09:26 2014 From: mattevans123@REDACTED (Matthew Evans) Date: Wed, 23 Jul 2014 11:09:26 -0400 Subject: [erlang-questions] 17.1 vs R16 Message-ID: Hi Team, Are there any issues with running code compiled on a 17.1 Erlang release on an R16 release? Obviously I understand we can't use any of the new features (maps etc). Thanks Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Wed Jul 23 17:28:14 2014 From: g@REDACTED (Guilherme Andrade) Date: Wed, 23 Jul 2014 16:28:14 +0100 Subject: [erlang-questions] 17.1 vs R16 In-Reply-To: References: Message-ID: <53CFD48E.3040002@gandrade.net> Hi, Independently of feature (in)compatibility, AFAIK the newer BEAM bytecode will not be compatible itself and R16 will outright refuse to run it with such an error message (concerning newer, unrecognised opcodes or something of the sort). Cheers, On 23/07/14 16:09, Matthew Evans wrote: > Hi Team, > > Are there any issues with running code compiled on a 17.1 Erlang > release on an R16 release? > > Obviously I understand we can't use any of the new features (maps etc). > > Thanks > > Matt > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Guilherme https://www.gandrade.net/ PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A D67A 9330 79B1 35CB 8191 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: OpenPGP digital signature URL: From mattevans123@REDACTED Wed Jul 23 17:32:15 2014 From: mattevans123@REDACTED (Matthew Evans) Date: Wed, 23 Jul 2014 11:32:15 -0400 Subject: [erlang-questions] 17.1 vs R16 In-Reply-To: <53CFD48E.3040002@gandrade.net> References: , <53CFD48E.3040002@gandrade.net> Message-ID: Well it doesn't appear to fail to run. It runs just fine, but I'm worried about odd behaviors. Matt Date: Wed, 23 Jul 2014 16:28:14 +0100 From: g@REDACTED To: mattevans123@REDACTED; erlang-questions@REDACTED Subject: Re: [erlang-questions] 17.1 vs R16 Hi, Independently of feature (in)compatibility, AFAIK the newer BEAM bytecode will not be compatible itself and R16 will outright refuse to run it with such an error message (concerning newer, unrecognised opcodes or something of the sort). Cheers, On 23/07/14 16:09, Matthew Evans wrote: Hi Team, Are there any issues with running code compiled on a 17.1 Erlang release on an R16 release? Obviously I understand we can't use any of the new features (maps etc). Thanks Matt _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -- Guilherme https://www.gandrade.net/ PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A D67A 9330 79B1 35CB 8191 -------------- next part -------------- An HTML attachment was scrubbed... URL: From elixmoon@REDACTED Wed Jul 23 18:29:17 2014 From: elixmoon@REDACTED (Dmitry Aleksandrov) Date: Wed, 23 Jul 2014 18:29:17 +0200 Subject: [erlang-questions] 17.1 vs R16 In-Reply-To: References: , <53CFD48E.3040002@gandrade.net> Message-ID: <53CFE2DD.4020200@gmail.com> I'll tried to use R17.1 compiled code on R15(on R16 too), it works, but lager handler was crashing, as it can't print some characteres. Thats, because of different encoding changes in erlang 17, I'll guess. > Matthew Evans > 23 Jul 2014 17:32 > Well it doesn't appear to fail to run. It runs just fine, but I'm > worried about odd behaviors. > > Matt > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > Matthew Evans > 23 Jul 2014 17:09 > Hi Team, > > Are there any issues with running code compiled on a 17.1 Erlang > release on an R16 release? > > Obviously I understand we can't use any of the new features (maps etc). > > Thanks > > Matt > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: compose-unknown-contact.jpg Type: image/jpeg Size: 770 bytes Desc: not available URL: From ryankbrown@REDACTED Wed Jul 23 21:58:08 2014 From: ryankbrown@REDACTED (Ryan Brown) Date: Wed, 23 Jul 2014 13:58:08 -0600 Subject: [erlang-questions] design pattern question for messaging Message-ID: Two more cents. I work on the same system as Rich and I can see a hybrid approach here being a very nice way to go. You could use rabbit (or really any MQ of your choice) for persistence with one queue per step in the process. Your application would then just listen to messages from each queue, spin a process to handle it and enqueue it in the next step. As Jesper mentioned, this could all be tracked nicely via an fsm. Miles, do you feel like your questions have been answered for this thread? I know it has spawned some creative thoughts on my end! -- -rb -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfidelman@REDACTED Wed Jul 23 22:02:59 2014 From: mfidelman@REDACTED (Miles Fidelman) Date: Wed, 23 Jul 2014 16:02:59 -0400 Subject: [erlang-questions] architectural/design overview of Erlang/OTP network stack Message-ID: <53D014F3.1040506@meetinghouse.net> I wonder if someone can point me to any architectural/design documentation of Erlang's network stack. I'm interested in seeing how things are coded in Erlang, as opposed to c implementation (to the extent that Erlang is not making use of underling operating system network code). I'm assuming that network programming in Erlang lends itself to a different style than network coding in other languages (I'm guessing some kinds of packets/transactions are handled by spawning a process as opposed to creating and linking a buffer.) Sort of trying to get a feel for how this all works. It would be cool beyond belief if someone has an overview of "the life of an http transaction," or some such, that illustrates all the moving parts, how events and data flow through various modules, and so forth. Failing that, some pointers into the code. Thanks very much, Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From mfidelman@REDACTED Wed Jul 23 22:31:08 2014 From: mfidelman@REDACTED (Miles Fidelman) Date: Wed, 23 Jul 2014 16:31:08 -0400 Subject: [erlang-questions] design pattern question for messaging In-Reply-To: References: Message-ID: <53D01B8C.6070008@meetinghouse.net> Ryan Brown wrote: > Two more cents. I work on the same system as Rich and I can see a > hybrid approach here being a very nice way to go. You could use rabbit > (or really any MQ of your choice) for persistence with one queue per > step in the process. Your application would then just listen to > messages from each queue, spin a process to handle it and enqueue it > in the next step. As Jesper mentioned, this could all be tracked > nicely via an fsm. > > Miles, do you feel like your questions have been answered for this > thread? I know it has spawned some creative thoughts on my end! > > Well... partially - as you say, spawned some thoughts. Also looked at Rabbit and exchanged some email with one of its designers, who I conveniently happen to know. The thing is, I'm thinking of things that are a lot longer-lived, and larger, than MQ messages - think email messages with attachments that are getting routed around, worked on, held while waiting for events, worked on some more, moved around some more, archived, etc. Best analogy is the way forms and documents move through an organization. What I'm coming to realize is that maybe a document store - CouchDB comes to mind - might be the best thing to use to hold queues; using it's replication mechanisms for resiliency. Then again, part of me is thinking that each document should just be its process. And what I'm really looking for is a conceptual and computational model that combines Erlang-like actors, with Smalltalk-style objects (which is really what Alan Kay had in mind before all the Smalltalk implementation came out single-threaded). Massively concurrent Smalltalk could be a really sweet environment. By way of context: I've been working on a family of applications that involve large groups working on shared and linked documents (business plans, proposals, military mission plans - that kind of thing) - with a distributed versioning approach (i.e., git for documents rather than CVS or Sharepoint). Workflow is much like the way we do paper - documents get copies, distributed, handed off from one person to the next, merged, split up, etc. Changes, and approvals for changes will be through publish subscribe. I keep coming back to wishing for an environment where the basic "unit of computation" has both actor-like properties and object-like properties. Long term persistence, data encapsulation, plus message-passing concurrency. Erlang seems to be the best starting point that I've found. Cheers, Miles -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From dmkolesnikov@REDACTED Wed Jul 23 22:37:58 2014 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Wed, 23 Jul 2014 23:37:58 +0300 Subject: [erlang-questions] architectural/design overview of Erlang/OTP network stack In-Reply-To: <53D014F3.1040506@meetinghouse.net> References: <53D014F3.1040506@meetinghouse.net> Message-ID: <6063F77B-BB10-44E0-80BB-EA8309E00FC6@gmail.com> Hello, I would recommend you to look into design (source code) of following OTP applications * ssl * ssh I would say that they reflect a ?typical? scenario of network programming using Erlang. Best Regards, Dmitry On 23 Jul 2014, at 23:02, Miles Fidelman wrote: > I wonder if someone can point me to any architectural/design documentation of Erlang's network stack. > > I'm interested in seeing how things are coded in Erlang, as opposed to c implementation (to the extent that Erlang is not making use of underling operating system network code). I'm assuming that network programming in Erlang lends itself to a different style than network coding in other languages (I'm guessing some kinds of packets/transactions are handled by spawning a process as opposed to creating and linking a buffer.) Sort of trying to get a feel for how this all works. > > It would be cool beyond belief if someone has an overview of "the life of an http transaction," or some such, that illustrates all the moving parts, how events and data flow through various modules, and so forth. Failing that, some pointers into the code. > > Thanks very much, > > Miles Fidelman > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mfidelman@REDACTED Wed Jul 23 22:43:53 2014 From: mfidelman@REDACTED (Miles Fidelman) Date: Wed, 23 Jul 2014 16:43:53 -0400 Subject: [erlang-questions] architectural/design overview of Erlang/OTP network stack In-Reply-To: <6063F77B-BB10-44E0-80BB-EA8309E00FC6@gmail.com> References: <53D014F3.1040506@meetinghouse.net> <6063F77B-BB10-44E0-80BB-EA8309E00FC6@gmail.com> Message-ID: <53D01E89.3070809@meetinghouse.net> Thanks! Dmitry Kolesnikov wrote: > Hello, > > I would recommend you to look into design (source code) of following OTP applications > * ssl > * ssh > > I would say that they reflect a ?typical? scenario of network programming using Erlang. > > Best Regards, > Dmitry > > > On 23 Jul 2014, at 23:02, Miles Fidelman wrote: > >> I wonder if someone can point me to any architectural/design documentation of Erlang's network stack. >> >> I'm interested in seeing how things are coded in Erlang, as opposed to c implementation (to the extent that Erlang is not making use of underling operating system network code). I'm assuming that network programming in Erlang lends itself to a different style than network coding in other languages (I'm guessing some kinds of packets/transactions are handled by spawning a process as opposed to creating and linking a buffer.) Sort of trying to get a feel for how this all works. >> >> It would be cool beyond belief if someone has an overview of "the life of an http transaction," or some such, that illustrates all the moving parts, how events and data flow through various modules, and so forth. Failing that, some pointers into the code. >> >> Thanks very much, >> >> Miles Fidelman >> >> -- >> In theory, there is no difference between theory and practice. >> In practice, there is. .... Yogi Berra >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From timmcgil@REDACTED Wed Jul 23 23:43:44 2014 From: timmcgil@REDACTED (Tim McGilchrist) Date: Thu, 24 Jul 2014 07:43:44 +1000 Subject: [erlang-questions] State of Erlang on PPC Message-ID: What's the current state of the Erlang VM on PPC? Specifically I'm interested in running it on one of the Freescale embedded boards with one of these CPUs. (http://en.m.wikipedia.org/wiki/PowerQUICC) Searching through the mailing list archives suggests that erlang has previously run on these CPUs. But there isn't much recent activity suggesting people are using it. So who's running Erlang on PPC Thanks ----------------------------------------------------- Tim McGilchrist @lambda_foo http://github.com/tmcgilchrist -------------- next part -------------- An HTML attachment was scrubbed... URL: From mallen@REDACTED Thu Jul 24 00:18:38 2014 From: mallen@REDACTED (Mark Allen) Date: Wed, 23 Jul 2014 17:18:38 -0500 Subject: [erlang-questions] design pattern question for messaging In-Reply-To: <53D01B8C.6070008@meetinghouse.net> References: <53D01B8C.6070008@meetinghouse.net> Message-ID: On 7/23/14 3:31 PM, "Miles Fidelman" wrote: >I keep coming back to wishing for an environment where the basic "unit >of computation" has both actor-like properties and object-like >properties. Long term persistence, data encapsulation, plus >message-passing concurrency. Erlang seems to be the best starting point >that I've found. Have you considered Microsoft's Orleans run-time environment? https://orleans.codeplex.com/wikipage?title=Orleans%20Background&referringT itle=Home https://orleans.codeplex.com/ It sounds like it might fit into the concepts you're thinking about. I think it's pretty cool. I'd love to see some of these concepts in the Erlang run-time, especially as a update/alternative for the current distributed Erlang model. Mark From raould@REDACTED Thu Jul 24 00:22:59 2014 From: raould@REDACTED (Raoul Duke) Date: Wed, 23 Jul 2014 15:22:59 -0700 Subject: [erlang-questions] design pattern question for messaging In-Reply-To: References: <53D01B8C.6070008@meetinghouse.net> Message-ID: >>I keep coming back to wishing for an environment where the basic "unit >>of computation" has both actor-like properties and object-like >>properties. Long term persistence, data encapsulation, plus >>message-passing concurrency. so everything comes with problems, so there's no perfect answer, but the longer i use OO and stuff the more i think we're all just brainwashed into using it when it is just plain bad. actors are maybe less bad somehow (not that Erlang is actors per Hewitt et. al.) but even if they are less bad than shared mutable state, i hardly think they are the best. dataflow a la gpars or oz or fbp whatever might be a step up, as might any number of other things like functional-relational. (yes, Erlang exists and has been proven to work yadda.) tho if i had to do things along these lines, i'd wish that clusterken would be a viable product. From mfidelman@REDACTED Thu Jul 24 00:35:39 2014 From: mfidelman@REDACTED (Miles Fidelman) Date: Wed, 23 Jul 2014 18:35:39 -0400 Subject: [erlang-questions] design pattern question for messaging In-Reply-To: References: <53D01B8C.6070008@meetinghouse.net> Message-ID: <53D038BB.7080208@meetinghouse.net> Mark Allen wrote: > On 7/23/14 3:31 PM, "Miles Fidelman" wrote: >> I keep coming back to wishing for an environment where the basic "unit >> of computation" has both actor-like properties and object-like >> properties. Long term persistence, data encapsulation, plus >> message-passing concurrency. Erlang seems to be the best starting point >> that I've found. > Have you considered Microsoft's Orleans run-time environment? > > https://orleans.codeplex.com/wikipage?title=Orleans%20Background&referringT > itle=Home > > > https://orleans.codeplex.com/ > > > It sounds like it might fit into the concepts you're thinking about. I > think it's pretty cool. I'd love to see some of these concepts in the > Erlang run-time, especially as a update/alternative for the current > distributed Erlang model. > > This is the first I've heard of it. Thanks for the pointers! Some interesting concepts - particularly the notion of persistent actors that never go away. As soon as I see .net, though, I start cringing. And Microsoft is known for developing interesting stuff in the lab, that then goes away (at least they publish it though, Apple tends to deep six things.) As you say - some of those concepts might fit, and might be nice to pull into the Erlang environment. Thanks! Miles -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From mfidelman@REDACTED Thu Jul 24 00:39:18 2014 From: mfidelman@REDACTED (Miles Fidelman) Date: Wed, 23 Jul 2014 18:39:18 -0400 Subject: [erlang-questions] design pattern question for messaging In-Reply-To: References: <53D01B8C.6070008@meetinghouse.net> Message-ID: <53D03996.7040409@meetinghouse.net> Raoul Duke wrote: >>> I keep coming back to wishing for an environment where the basic "unit >>> of computation" has both actor-like properties and object-like >>> properties. Long term persistence, data encapsulation, plus >>> message-passing concurrency. > so everything comes with problems, so there's no perfect answer, but > the longer i use OO and stuff the more i think we're all just > brainwashed into using it when it is just plain bad. actors are maybe > less bad somehow (not that Erlang is actors per Hewitt et. al.) but > even if they are less bad than shared mutable state, i hardly think > they are the best. dataflow a la gpars or oz or fbp whatever might be > a step up, as might any number of other things like > functional-relational. (yes, Erlang exists and has been proven to work > yadda.) tho if i had to do things along these lines, i'd wish that > clusterken would be a viable product. > no, no, no, .... not problems - but design challenges and on opportunity to work on new concepts! :-) -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From chandrashekhar.mullaparthi@REDACTED Thu Jul 24 07:42:06 2014 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Thu, 24 Jul 2014 06:42:06 +0100 Subject: [erlang-questions] Mnesia error when switching fro clustered to un-clustered mode In-Reply-To: References: Message-ID: Hi Kris, When you say "When I switch them to clustered", how exactly did you do this? Chandru On 18 July 2014 21:35, Kristoffer Brown wrote: > I have deployed my application to three nodes with mnesia in standalone > mode. I have three tables, one of which recently changed. I have all three > upgraded with the correct table schema and running fine in standalone mode. > When I switch them to clustered I receive the following error > > {merge_schema_failed, > > "Bad cookie in table definition changed table?: > > > I cannot determine what is causing this. I can delete the mnesia directory > and re-start and it works fine. However, this is obviously not an acceptable > solution for a production deployment. > > What could cause this error? All three nodes have the exact same schema and > my understanding was that this was the cause of this issue. I think this > just is the result of a newbie not understanding the purpose of the table > cookie. > > Thanks. > > Kris > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From glasper9@REDACTED Thu Jul 24 08:17:21 2014 From: glasper9@REDACTED (Jason Stokes) Date: Wed, 23 Jul 2014 23:17:21 -0700 Subject: [erlang-questions] State of Erlang on PPC In-Reply-To: References: Message-ID: <1406182641.82440.YahooMailNeo@web125905.mail.ne1.yahoo.com> Configures, compiles, installs and runs without special effort or modification on MacOS 10.5.8 PPC, because I happen to be using one for development right now. I imaginethe story is similar for Linux PPC.? I have no idea how easy it is to utilize it for embedded use, but it is (surprisingly enough) still supported. ________________________________ From: Tim McGilchrist To: "erlang-questions@REDACTED" Sent: Thursday, 24 July 2014 7:43 AM Subject: [erlang-questions] State of Erlang on PPC What's the current state of the Erlang VM on PPC? Specifically I'm interested in running it on one of the Freescale embedded boards with one of these CPUs. (http://en.m.wikipedia.org/wiki/PowerQUICC) Searching through the mailing list archives suggests that erlang has previously run on these CPUs. But there isn't much recent activity suggesting people are using it.? So who's running Erlang on PPC Thanks ----------------------------------------------------- Tim McGilchrist @lambda_foo http://github.com/tmcgilchrist _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitrii.bushmelev@REDACTED Thu Jul 24 10:54:05 2014 From: dmitrii.bushmelev@REDACTED (Dmitry Bushmelev) Date: Thu, 24 Jul 2014 12:54:05 +0400 Subject: [erlang-questions] Web Framework Benchmarks Message-ID: Hello. I know that: web framework benchmarks it is bad and useless, but I interested in. I tried extract common logic to lib: https://github.com/b0oh/erl_bench, to see only frameworks overhead. Please see https://github.com/TechEmpower/FrameworkBenchmarks/pull/938, and if someone can run benchmarks for checking everything is ok, it'll be nice. Thanks. From jesper.louis.andersen@REDACTED Thu Jul 24 11:35:18 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 24 Jul 2014 11:35:18 +0200 Subject: [erlang-questions] design pattern question for messaging In-Reply-To: <53D038BB.7080208@meetinghouse.net> References: <53D01B8C.6070008@meetinghouse.net> <53D038BB.7080208@meetinghouse.net> Message-ID: On Thu, Jul 24, 2014 at 12:35 AM, Miles Fidelman wrote: > This is the first I've heard of it. Thanks for the pointers! > > Some interesting concepts - particularly the notion of persistent actors > that never go away. > [Orleans...] The thing about Orleans is frameworkizm. It provides you with a framework where actors are persisted, dynamically spawned and so on. But by doing so, it also trades off certain things. You can't (per CAP) guarantee an actor will be run on once machine only. Thus, you have to resolve conflicts in other ways, usually by having transactions in a subsystem somewhere. And then using those transactions to lift safety into the system as a whole. Imagine you run eventually consistent and asynchronous but once you do something with money, you become strongly consistent by using a database connection. When I read the paper my guess was that you could build something like Orleans in 2-3 weeks in Erlang tailored to your needs. Depending on the fit of Orleans to your problem space this might be slower or faster than understanding the framework in the first place. I don't particularly like that you have to program in a certain mode of operation in .NET for your code to be safe. While the same can be said for Node.js or OCaml+Core.Async it is still not as safe as Erlang, where the default mode of operation is the one you seek. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zerthurd@REDACTED Thu Jul 24 15:05:09 2014 From: zerthurd@REDACTED (Max Treskin) Date: Thu, 24 Jul 2014 15:05:09 +0200 Subject: [erlang-questions] Type spec for application:get_env and set_env Message-ID: Hello It is possible to use any term() as a key in application:set_env and then retrieve a value by this term using application:get_env (what is very conveniently), but only atom() is allowed as a key in type spec of these functions. My proposal is to change type spec in favour of any term usage as a key. Any cons? --? Max Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Thu Jul 24 15:50:37 2014 From: vinoski@REDACTED (Steve Vinoski) Date: Thu, 24 Jul 2014 09:50:37 -0400 Subject: [erlang-questions] Type spec for application:get_env and set_env In-Reply-To: References: Message-ID: On Thu, Jul 24, 2014 at 9:05 AM, Max Treskin wrote: > Hello > > It is possible to use any term() as a key in application:set_env and then > retrieve a value by this term using application:get_env (what is very > conveniently), but only atom() is allowed as a key in type spec of these > functions. > My proposal is to change type spec in favour of any term usage as a key. > Any cons? > I previously brought this up with the OTP team and was reminded that this also affects config files, .app files, and also setting env vars from the command line, with the point being that there could well be unintended side effects of such a change. Some Riak code used to take advantage of what you've described, but after the discussion we modified our code to honor the type spec. --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From slashfoo@REDACTED Thu Jul 24 18:20:28 2014 From: slashfoo@REDACTED (Jamiel Almeida) Date: Thu, 24 Jul 2014 09:20:28 -0700 Subject: [erlang-questions] Hiding user input in escript In-Reply-To: References: <5148D2DA.6080604@go-factory.net> <5148EFDB.8040909@go-factory.net> Message-ID: Ah thank you, I'll see what I can find or ask around. On Wed, Jul 23, 2014 at 7:54 AM, Bryan wrote: > Howdy. I believe I submitted a bug, but never followed up and no longer > have my sent emails from back then ? so not 100% sure. > > Cheers, > Bryan > > ---- > > Bryan Hughes > CTO and Founder / *Go Factory* > (415) 515-7916 > > http://www.go-factory.net > > *Connecting the Internet of Things* > > On Jul 23, 2014, at 2:36 AM, Jamiel Almeida wrote: > > Was this ever submitted as a bug? and if so, do you have a URL to track? > > On Tuesday, March 19, 2013 4:08:11 PM UTC-7, Bryan Hughes wrote: >> >> Hi Bob, >> >> Thanks for checking it out! Guess not too many people are writing >> scripts that need to deal with passwords. I will go ahead and submit it as >> a bug. >> >> Cheers, >> Bryan >> >> On 3/19/13 4:01 PM, Bob Ippolito wrote: >> >> R16B does not change anything for escript as far as I can tell. >> >> On Tue, Mar 19, 2013 at 3:17 PM, Bob Ippolito wrote: >> >>> I see io:get_password() in my io module in R15B03, but it doesn't appear >>> to work with escript because it uses the old 'user' rather than 'group' >>> which doesn't understand the get_password io request. Haven't looked at >>> R16B yet. >>> >>> >>> >>> On Tue, Mar 19, 2013 at 2:04 PM, Bryan Hughes >>> wrote: >>> >>>> Hi Everyone, >>>> >>>> I found this old thread from 2009 about trying to hide user input in an >>>> escript (e.g. entering a password). >>>> >>>> http://erlang.org/pipermail/erlang-questions/2009-April/043311.html >>>> >>>> It concluded with: >>>> >>>> The problem seems to be that escript uses the old shell, which >>>> does not support any way to control the terminal. >>>> >>>> We'll have a look at escript for the next release and see whether >>>> it would be possible to have it run with the standard shell. >>>> >>>> >>>> Does anyone know if this has been fixed. I looked through the docs and >>>> do not find io:get_password/0, or any options for reading stdin to not echo >>>> the characters back. >>>> >>>> Looking forward to meeting those of you attending Erlang Factory in a >>>> couple of days. >>>> >>>> Cheers, >>>> B >>>> >>>> -- >>>> >>>> Bryan Hughes >>>> *Go Factory* >>>> http://www.go-factory.net >>>> >>>> *"Art is never finished, only abandoned. - Leonardo da Vinci"* >>>> >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-q...@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >> >> -- >> >> Bryan Hughes >> CTO and Founder / *Go Factory* >> (415) 515-7916 >> http://www.go-factory.net >> >> *"Art is never finished, only abandoned. - Leonardo da Vinci"* >> >> >> > -- Jamiel Almeida -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitrii.bushmelev@REDACTED Thu Jul 24 22:27:57 2014 From: dmitrii.bushmelev@REDACTED (Dmitry Bushmelev) Date: Fri, 25 Jul 2014 00:27:57 +0400 Subject: [erlang-questions] Web Framework Benchmarks In-Reply-To: References: Message-ID: For now there are only cowboy and elli, so I added mochiweb, yaws and misultin in pull request. 2014-07-24 12:54 GMT+04:00 Dmitry Bushmelev : > Hello. > I know that: web framework benchmarks it is bad and useless, but I > interested in. I tried extract common logic to lib: > https://github.com/b0oh/erl_bench, to see only frameworks overhead. > > Please see https://github.com/TechEmpower/FrameworkBenchmarks/pull/938, > and if someone can run benchmarks for checking everything is ok, it'll > be nice. > > Thanks. From timmcgil@REDACTED Fri Jul 25 00:32:16 2014 From: timmcgil@REDACTED (Tim McGilchrist) Date: Fri, 25 Jul 2014 08:32:16 +1000 Subject: [erlang-questions] State of Erlang on PPC In-Reply-To: <1406182641.82440.YahooMailNeo@web125905.mail.ne1.yahoo.com> References: <1406182641.82440.YahooMailNeo@web125905.mail.ne1.yahoo.com> Message-ID: <1F9F0CAE-D986-46C5-8F23-F470E9A81259@gmail.com> Thanks Jason. I used to work on a PowerBook and used Erlang there but that was a number of years ago now. It's good to know the code still works there. I expected it would/should work I think the devil will be in getting it compiled to whatever Linux version comes with the embedded board. Now to get my hands on some real hardware or a decent enough emulator to test it out. Cheers, Tim ----------------------------------------------------- Tim McGilchrist @lambda_foo http://github.com/tmcgilchrist On 24 Jul 2014, at 4:17 pm, Jason Stokes wrote: > Configures, compiles, installs and runs without special effort or modification on MacOS 10.5.8 PPC, because I happen to be using one for development right now. I imagine the story is similar for Linux PPC. I have no idea how easy it is to utilize it for embedded use, but it is (surprisingly enough) still supported. > > From: Tim McGilchrist > To: "erlang-questions@REDACTED" > Sent: Thursday, 24 July 2014 7:43 AM > Subject: [erlang-questions] State of Erlang on PPC > > What's the current state of the Erlang VM on PPC? Specifically I'm interested in running it on one of the Freescale embedded boards with one of these CPUs. (http://en.m.wikipedia.org/wiki/PowerQUICC) > > Searching through the mailing list archives suggests that erlang has previously run on these CPUs. But there isn't much recent activity suggesting people are using it. > > So who's running Erlang on PPC > > Thanks > > > > ----------------------------------------------------- > Tim McGilchrist > > @lambda_foo > http://github.com/tmcgilchrist > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From zerthurd@REDACTED Fri Jul 25 09:48:23 2014 From: zerthurd@REDACTED (Max Treskin) Date: Fri, 25 Jul 2014 09:48:23 +0200 Subject: [erlang-questions] Type spec for application:get_env and set_env In-Reply-To: References: Message-ID: So what is the best native way to do this? Only to create an additional ETS to keep parameters with an arbitrary term key? --? Max Treskin From:?Steve Vinoski Reply:?Steve Vinoski > Date:?24 Jul 2014 at 15:50:37 To:?Max Treskin > Cc:?erlang-questions@REDACTED > Subject:? Re: [erlang-questions] Type spec for application:get_env and set_env On Thu, Jul 24, 2014 at 9:05 AM, Max Treskin wrote: Hello It is possible to use any term() as a key in application:set_env and then retrieve a value by this term using application:get_env (what is very conveniently), but only atom() is allowed as a key in type spec of these functions. My proposal is to change type spec in favour of any term usage as a key. Any cons? I previously brought this up with the OTP team and was reminded that this also affects config files, .app files, and also setting env vars from the command line, with the point being that there could well be unintended side effects of such a change. Some Riak code used to take advantage of what you've described, but after the discussion we modified our code to honor the type spec. --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Jul 25 10:29:51 2014 From: ok@REDACTED (ok@REDACTED) Date: Fri, 25 Jul 2014 20:29:51 +1200 Subject: [erlang-questions] Type spec for application:get_env and set_env In-Reply-To: References: Message-ID: > On Thu, Jul 24, 2014 at 9:05 AM, Max Treskin asked how to use "any term" as a key in application:set_env &c. My proposal of many years for 'frames' restricted keys to atoms. I still think that *as a replacement for records*, that's the optimal design decision. But what Erlang actually *has* these days is 'maps'. And those things accept any term as a key. So why not have 'extra' as a single application key with the value of that key being a map? To set what you want, you'd have to get the old map (with #{} as the default), set/replace the subkey you have in mind, and then put the whole map back. From schneider@REDACTED Fri Jul 25 12:51:46 2014 From: schneider@REDACTED (Frans Schneider) Date: Fri, 25 Jul 2014 12:51:46 +0200 Subject: [erlang-questions] Performance inets httpd versus comboy Message-ID: <53D236C2.6070109@xs4all.nl> Hi list, Running Cowboy versus inets httpd gives a huge performance difference. Cowboy handles 700+ requests per second and httpds only 26. Cowboy takes 88 % CPU but httpd only 5%. Why is it that httpd does not run at full speed? System specs: Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz Memory 8GB Linux debian 3.2.0-4-amd64 #1 SMP Debian 3.2.60-1+deb7u1 x86_64 GNU/Linux Erlang/OTP 17 [erts-6.1] [source] [64-bit] [smp:2:2] [async-threads:10] [kernel-poll:false] Cowboy is started like this: application:start(crypto), application:start(ranch), application:start(cowlib), application:start(cowboy), Dispatch = cowboy_router:compile([ %% {URIHost, list({URIPath, Handler, Opts})} {'_', [{'_', empty_handler, []}]} ]), %% Name, NbAcceptors, TransOpts, ProtoOpts cowboy:start_http(my_http_listener, 100, [{port, 8080}], [{env, [{dispatch, Dispatch}]}] ), And httpd is started like this application:start(inets), inets:start(httpd, [{modules, [ mod_empty ]}, {port,8080}, {max_clients, 1000}, {server_root,"log"}, {server_name,"hello_world"}, {document_root,"www"} ]), Both empty_handler and mod_empty return status 200 plus a proper XML-RPC reply. The client is a Python script: server = xmlrpclib.ServerProxy('http://localhost:8080') while True: server.ping(1234) Thanks, Frans From zkessin@REDACTED Fri Jul 25 13:15:19 2014 From: zkessin@REDACTED (Zachary Kessin) Date: Fri, 25 Jul 2014 14:15:19 +0300 Subject: [erlang-questions] Web Framework Benchmarks In-Reply-To: References: Message-ID: <53D23C47.9060703@gmail.com> Just a note, mitultin is no longer maintained and the creators suggest using one of the other web servers. --Zach On 7/24/14, 11:27 PM, Dmitry Bushmelev wrote: > For now there are only cowboy and elli, so I added mochiweb, yaws and > misultin in pull request. > > 2014-07-24 12:54 GMT+04:00 Dmitry Bushmelev : >> Hello. >> I know that: web framework benchmarks it is bad and useless, but I >> interested in. I tried extract common logic to lib: >> https://github.com/b0oh/erl_bench, to see only frameworks overhead. >> >> Please see https://github.com/TechEmpower/FrameworkBenchmarks/pull/938, >> and if someone can run benchmarks for checking everything is ok, it'll >> be nice. >> >> Thanks. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Zachary Kessin Mostly Erlang Podcast Skype: zachkessin Twitter: @zkessin -------------- next part -------------- An HTML attachment was scrubbed... URL: From kunthar@REDACTED Sat Jul 26 00:54:01 2014 From: kunthar@REDACTED (Gokhan Boranalp) Date: Sat, 26 Jul 2014 01:54:01 +0300 Subject: [erlang-questions] =?utf-8?q?Ch=C3=A9ri_Cart_-_Creating_a_Web_App?= =?utf-8?q?_in_Erlang_with_Chicago_Boss_and_Riak_from_Scratch_-_I?= Message-ID: http://blog.drlinux.org/creating-a-web-app-in-erlang-with-chicago-boss-and-riak-from-scratch-i/ just wanted to let you know. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rapidkaju@REDACTED Sat Jul 26 10:17:34 2014 From: rapidkaju@REDACTED (Kaju) Date: Sat, 26 Jul 2014 13:47:34 +0530 Subject: [erlang-questions] erlang with sctp doesn't build on armv7 from source Message-ID: Hi, I have been trying get erlang build on armv7 embedded platform radxa-rock with sctp support, but it still doesnt work. What's done so far. 1. Install all the development packages as indicated on erlang website for debian based system (I am running Ubuntu/Linaro 4.6.3-1ubuntu5). 2. lksctp-tools, libsctp1, libsctp-dev all are installed correctly. 3. Used --enable-sctp configure option, which didnt work, so had to add LIBS='-lsctp' before running the configure. LIBS='-lsctp' CFLAGS='-O0' ./configure --disable-hipe --enable-smp-support --enable-threads --enable-kernel-poll --enable-sctp --without-wx --prefix=/media/sd/local > /media/sd/erlang/configure.log 2>&1 LIBS='-lsctp' make -j3 > /media/sd/erlang/make.log 2>&1 LIBS='-lsctp' make install > /media/sd/erlang/make-install.log 2>&1 The configure.log, make.log and make-install.log are all attached in compressed format. Note: I have built erlang from source with sctp support on linux-x86_64 and Mac OS X without any issues, so its surprising why it wont build on embedded board while the only difference being running on arm. Additionally, I am building on the target board directly. uname = Linux radxa 3.0.36+ #2 SMP PREEMPT Sat Mar 15 11:18:05 CST 2014 armv7l armv7l armv7l GNU/Linux Thanks, Kaju -------------- next part -------------- A non-text attachment was scrubbed... Name: erlang-sctp-armv7-issue.tar.gz Type: application/x-gzip Size: 45861 bytes Desc: not available URL: -------------- next part -------------- From carlosj.gf@REDACTED Sun Jul 27 01:05:55 2014 From: carlosj.gf@REDACTED (=?ISO-8859-1?Q?Carlos_Gonz=E1lez_Florido?=) Date: Sun, 27 Jul 2014 01:05:55 +0200 Subject: [erlang-questions] [ANN] NkSIP v0.4.0 Message-ID: Hello list, I want to announce the release of version v0.4.0 of NkSIP, a SIP Erlang Framework to build scalable SIP elements very easily. About a year ago, I announced the release of the first open source version. A lot of work has been done during this year, and NkSIP is now quite stable and fully featured: - Full support for all currently defined SIP methods: INVITE, ACK, REGISTER, OPTIONS, INFO, UPDATE, PRACK, SUBSCRIBE, NOTIFY, REFER, PUBLISH and MESSAGE, as an UAC, an UAS or a Proxy. - Can be used to develop very easily any possible SIP application: endpoints, stateful proxies with serial and parallel forking, stateless proxies, B2BUAs, application servers, registrars, SBCs, load generators, etc. - Transports UDP, TCP, TLS, SCTP, WS and WSS (websockets) are available, all of them capable of handling thousands of simultaneous sessions. - Full SIP Event support. - A written from scratch, fully typed 100% Erlang code with few external dependencies. - Robust and highly scalable, using all available processor cores automatically. - More than 150 tests covering nearly all of the functionality. - Sophisticated plugin mechanism, that adds very low overhead to the core. - Hot, on-the-fly core and application configuration and code upgrades. - IPv6 support and IPv4 <-> IPv6 bridge. - Full support for NAPTR and SRV location, including priority and weights. - Dialog and SDP processing utility functions, including media start and stop detection. A lot of functionality has been moved into plugins: - Digest Authentication. - Registrar Server and Automatic Registrations. - Event State Compositor. - Reliable provisional responses. - Session Timers - SIP REFER. - Outbound and GRUU. - Trace and Statistics. This release also includes comprehensive documentation in the project page. Now, the next target is NkCore, a distributed, generic, and easy to use Erlang application server based on the NkSIP code. NkSIP will then become a plugin of NkCore. As always, any feedback is welcomed. Thank you! https://github.com/kalta/nksip Carlos Gonz?lez @carlosjgf -------------- next part -------------- An HTML attachment was scrubbed... URL: From rapidkaju@REDACTED Sun Jul 27 03:22:16 2014 From: rapidkaju@REDACTED (Kaju) Date: Sun, 27 Jul 2014 06:52:16 +0530 Subject: [erlang-questions] erlang with sctp doesn't build on armv7 from source In-Reply-To: References: Message-ID: <32859219-A27B-4439-A1FC-698FEB392D3A@gmail.com> Hi, I wrote a sample sctp client and server in c-lang and that failed with error "Protocol not supported". It appears that my kernel do not support sctp protocol. The issue is with kernel sctp support and probably not with erlang build system. On 26-Jul-2014, at 1:47 pm, Kaju wrote: > Hi, > > I have been trying get erlang build on armv7 embedded platform radxa-rock with sctp support, but it still doesnt work. > > What's done so far. > > 1. Install all the development packages as indicated on erlang website for debian based system (I am running Ubuntu/Linaro 4.6.3-1ubuntu5). > 2. lksctp-tools, libsctp1, libsctp-dev all are installed correctly. > 3. Used --enable-sctp configure option, which didnt work, so had to add LIBS='-lsctp' before running the configure. > > LIBS='-lsctp' CFLAGS='-O0' ./configure --disable-hipe --enable-smp-support --enable-threads --enable-kernel-poll --enable-sctp --without-wx --prefix=/media/sd/local > /media/sd/erlang/configure.log 2>&1 > LIBS='-lsctp' make -j3 > /media/sd/erlang/make.log 2>&1 > LIBS='-lsctp' make install > /media/sd/erlang/make-install.log 2>&1 > > The configure.log, make.log and make-install.log are all attached in compressed format. > > Note: I have built erlang from source with sctp support on linux-x86_64 and Mac OS X without any issues, so its surprising why it wont build on embedded board while the only difference being running on arm. Additionally, I am building on the target board directly. > > > uname = Linux radxa 3.0.36+ #2 SMP PREEMPT Sat Mar 15 11:18:05 CST 2014 armv7l armv7l armv7l GNU/Linux > > Thanks, > Kaju > > > From vladimir.ralev@REDACTED Sun Jul 27 11:17:51 2014 From: vladimir.ralev@REDACTED (Vladimir Ralev) Date: Sun, 27 Jul 2014 12:17:51 +0300 Subject: [erlang-questions] [ANN] NkSIP v0.4.0 In-Reply-To: References: Message-ID: Cool. Would be really interesting if you can make it fully distributed. If a failure in the middle of transaction can be recovered on another node to send retransmissions and continue. Distributed timers and state all together. I will be checking it out. On Sun, Jul 27, 2014 at 2:05 AM, Carlos Gonz?lez Florido < carlosj.gf@REDACTED> wrote: > Hello list, > > I want to announce the release of version v0.4.0 of NkSIP, a SIP Erlang > Framework to build scalable SIP elements very easily. About a year ago, I > announced the release of the first open source version. A lot of work has > been done during this year, and NkSIP is now quite stable and fully > featured: > > > - Full support for all currently defined SIP methods: INVITE, ACK, > REGISTER, OPTIONS, INFO, UPDATE, PRACK, SUBSCRIBE, NOTIFY, REFER, PUBLISH > and MESSAGE, as an UAC, an UAS or a Proxy. > - Can be used to develop very easily any possible SIP application: > endpoints, stateful proxies with serial and parallel forking, stateless > proxies, B2BUAs, application servers, registrars, SBCs, load generators, > etc. > - Transports UDP, TCP, TLS, SCTP, WS and WSS (websockets) are > available, all of them capable of handling thousands of simultaneous > sessions. > - Full SIP Event support. > - A written from scratch, fully typed 100% Erlang code with few > external dependencies. > - Robust and highly scalable, using all available processor cores > automatically. > - More than 150 tests covering nearly all of the functionality. > - Sophisticated plugin mechanism, that adds very low overhead to the > core. > - Hot, on-the-fly core and application configuration and code upgrades. > - IPv6 support and IPv4 <-> IPv6 bridge. > - Full support for NAPTR and SRV location, including priority and > weights. > - Dialog and SDP processing utility functions, including media start > and stop detection. > > A lot of functionality has been moved into plugins: > > - Digest Authentication. > - Registrar Server and Automatic Registrations. > - Event State Compositor. > - Reliable provisional responses. > - Session Timers > - SIP REFER. > - Outbound and GRUU. > - Trace and Statistics. > > This release also includes comprehensive documentation in the project page. > > Now, the next target is NkCore, a distributed, generic, and easy to use > Erlang application server based on the NkSIP code. NkSIP will then become a > plugin of NkCore. > > As always, any feedback is welcomed. Thank you! > > https://github.com/kalta/nksip > > Carlos Gonz?lez > @carlosjgf > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joaohf@REDACTED Mon Jul 28 01:26:03 2014 From: joaohf@REDACTED (=?windows-1252?Q?Jo=E3o_Henrique_Ferreira_de_Freitas?=) Date: Sun, 27 Jul 2014 20:26:03 -0300 Subject: [erlang-questions] State of Erlang on PPC In-Reply-To: <1F9F0CAE-D986-46C5-8F23-F470E9A81259@gmail.com> References: <1406182641.82440.YahooMailNeo@web125905.mail.ne1.yahoo.com> <1F9F0CAE-D986-46C5-8F23-F470E9A81259@gmail.com> Message-ID: <53D58A8B.3000700@gmail.com> Hi, On 24-07-2014 19:32, Tim McGilchrist wrote: > I expected it would/should work I think the devil will be in getting > it compiled to whatever Linux version comes with the embedded board. > Now to get my hands on some real hardware or a decent enough emulator > to test it out. > Besides the fact of 'Linux version comes with the embedded board', you could try the Yocto Project (www.yoctoproject.org) and meta-erlang (http://layers.openembedded.org/layerindex/branch/master/layer/meta-erlang/). -- Jo?o Henrique Ferreira de Freitas - joaohf_at_gmail.com Campinas-SP-Brasil -------------- next part -------------- An HTML attachment was scrubbed... URL: From spasennyi4ert@REDACTED Fri Jul 25 19:31:03 2014 From: spasennyi4ert@REDACTED (Anastasia) Date: Fri, 25 Jul 2014 10:31:03 -0700 Subject: [erlang-questions] [erlang-programming] Re: Source specific multicast In-Reply-To: <5f30a1d8-16f3-4de3-a385-f4aa5151097f@googlegroups.com> References: <5f30a1d8-16f3-4de3-a385-f4aa5151097f@googlegroups.com> Message-ID: -module(udp_bin_ssm). -compile([export_all]). start([Source,Group,Port]) -> {ok,SourceAddress} = inet:parse_address(Source), {ok,GroupAddress} = inet:parse_address(Group), LocalIp = ip_to_binary({0,0,0,0}), GroupIp = ip_to_binary(GroupAddress), SourceIp = ip_to_binary(SourceAddress), Bin = << GroupIp/binary,LocalIp/binary,SourceIp/binary >>, {ok,Socket} = gen_udp:open(erlang:list_to_integer(Port), [ inet, binary, {active,true}, {reuseaddr,true}, {multicast_ttl, 30}, {raw, 0, 39, Bin} ]), io:format("Socket ~p:~n", [Socket]), Pid = spawn(fun() -> loop(Socket) end), io:format("Pid :~p~n",[Pid]), ok = gen_udp:controlling_process(Socket,Pid). ip_to_binary(Ip) -> list_to_binary(tuple_to_list(Ip)). loop(Socket) -> receive {udp,_Socket,_SrcAddr,_Port,Bin} -> io:format("Bin ~p:~n", [Bin]), loop(Socket); Msg -> io:format("Msg ~p:~n", [Msg]) end. ???????, 15 ???? 2014 ?., 8:09:37 UTC+4 ???????????? Anastasia ???????: > > What should I change or add to the code in order to run the code with > source specific multicast? > > -module(udp_bin). > -compile([export_all]). > > start([Host,Port]) -> > {ok,IpAddress} = inet:parse_address(Host), > {ok,Socket} = gen_udp:open(erlang:list_to_integer(Port), > [ > inet, > binary, > {active,true}, > {reuseaddr,true}, > {add_membership,{IpAddress,{0,0,0,0}}} > ]), > io:format("Socket ~p:~n", [Socket]), > Pid = spawn(fun() -> loop(Socket) end), > io:format("Pid :~p~n",[Pid]), %%15 > ok = gen_udp:controlling_process(Socket,Pid). > > > loop(Socket) -> > receive > {udp,_Socket,_SrcAddr,_Port,Bin} -> > io:format("Bin ~p:~n", [Bin]), %%25 > loop(Socket); > Msg -> > io:format("Msg ~p:~n", [Msg]) > end. > -- You received this message because you are subscribed to the Google Groups "Erlang Programming" group. To unsubscribe from this group and stop receiving emails from it, send an email to erlang-programming+unsubscribe@REDACTED To post to this group, send email to erlang-programming@REDACTED Visit this group at http://groups.google.com/group/erlang-programming. For more options, visit https://groups.google.com/d/optout. -------------- next part -------------- An HTML attachment was scrubbed... URL: From yilmazhuseyin@REDACTED Mon Jul 28 13:24:33 2014 From: yilmazhuseyin@REDACTED (Huseyin Yilmaz) Date: Mon, 28 Jul 2014 14:24:33 +0300 Subject: [erlang-questions] mnesia initialization code review Message-ID: Hello, I tried to use mnesia couple of times before. But it always seemed too hard to get it working right. So at last I think I got a good initialization. Can you guys spend couple minutes to review my code. My code is aimed to run as a single node (because multinode mnesia setup seemed a lot of work). So only thing I do is to check if my tables are there and if they aren't create them. My code that initializes mnesia: https://github.com/huseyinyilmaz/machete/blob/master/apps/machete/src/machete_mnesia.erl Backup and restore functions: https://github.com/huseyinyilmaz/machete/blob/master/apps/machete/src/machete_console.erl Thanks in advance. -------------- next part -------------- An HTML attachment was scrubbed... URL: From neerajsharma.live@REDACTED Sat Jul 26 09:24:45 2014 From: neerajsharma.live@REDACTED (Kaju) Date: Sat, 26 Jul 2014 00:24:45 -0700 Subject: [erlang-questions] [erlang-programming] unable to compile erlang with sctp on armv7 Message-ID: I have compiled erlang from source with sctp on linux x86_64 and Mac OS X successfully but somehow repeated attempts to do the same for Armv7 fails. I get a nagging configure issue where the sctp_bindx is undefined when I pass the '--enable-sctp' configure option. In order to overcome this issue I started using the LIBS='-lsctp' before configure and make, this worked but when I try to open sctp socket then it fails as follows: # /media/sd/local/bin/erl +K true Erlang/OTP 17 [erts-6.1] [source] [smp:4:4] [async-threads:10] [kernel-poll:true] Eshell V6.1 (abort with ^G) 1> gen_sctp:open(). {error,esocktnosupport} 2> Note that I built erlang on the target hardware (armv7 linaro 13.09) without any issues but just that sctp is not enabled. Any pointers to debug this issue will be of great help. uname -a Linux radxa 3.0.36+ #2 SMP PREEMPT Sat Mar 15 11:18:05 CST 2014 armv7l armv7l armv7l GNU/Linux Full configure (pretty big) is at https://gist.github.com/anonymous/8f2f92df146c02f59593 -- You received this message because you are subscribed to the Google Groups "Erlang Programming" group. To unsubscribe from this group and stop receiving emails from it, send an email to erlang-programming+unsubscribe@REDACTED To post to this group, send email to erlang-programming@REDACTED Visit this group at http://groups.google.com/group/erlang-programming. For more options, visit https://groups.google.com/d/optout. -------------- next part -------------- An HTML attachment was scrubbed... URL: From arif@REDACTED Mon Jul 28 22:48:44 2014 From: arif@REDACTED (arif@REDACTED) Date: Mon, 28 Jul 2014 13:48:44 -0700 Subject: [erlang-questions] [ANN] NkSIP v0.4.0 Message-ID: Great work. The tests fail with R15B because you are using crypto:hash/2 which does not seem to be defined in R15B. Best regards Arif From sdl.web@REDACTED Tue Jul 29 21:19:34 2014 From: sdl.web@REDACTED (Leo Liu) Date: Wed, 30 Jul 2014 03:19:34 +0800 Subject: [erlang-questions] erlang:exit/1 and erlang:exit/2 Message-ID: Hi there, Could someone help me understand the implementation of erlang:exit/1? BIF_RETTYPE exit_1(BIF_ALIST_1) { BIF_P->fvalue = BIF_ARG_1; /* exit value */ BIF_ERROR(BIF_P, EXC_EXIT); } which is pretty simple and I am not sure how it propagates the exit signal. erlang:exit/2 on the contrary is full of details. Thanks, Leo From tony@REDACTED Tue Jul 29 23:11:10 2014 From: tony@REDACTED (Tony Rogvall) Date: Tue, 29 Jul 2014 23:11:10 +0200 Subject: [erlang-questions] erlang:exit/1 and erlang:exit/2 In-Reply-To: References: Message-ID: <871EB3A0-F9ED-4229-B4D4-C90C77A8953D@rogvall.se> On 29 jul 2014, at 21:19, Leo Liu wrote: > Hi there, > Hello. > Could someone help me understand the implementation of erlang:exit/1? > > BIF_RETTYPE exit_1(BIF_ALIST_1) > { > BIF_P->fvalue = BIF_ARG_1; /* exit value */ > BIF_ERROR(BIF_P, EXC_EXIT); > } > > which is pretty simple and I am not sure how it propagates the exit > signal. erlang:exit/2 on the contrary is full of details. > BIFs with one (and more) argument are called from emulator loop in beam_emu.c (R17 source) beam_emu.c line 2779 in opcode "call_bif_e" and line 3571 in opcode "apply_bif" and line 2610 in "i_gc_bif1_jIsId" and the rest is just a matter of reading code. Hint. The function handler_error may be a good start to look in. Have fun :-) /Tony > Thanks, > Leo > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdl.web@REDACTED Wed Jul 30 10:04:25 2014 From: sdl.web@REDACTED (Leo Liu) Date: Wed, 30 Jul 2014 16:04:25 +0800 Subject: [erlang-questions] erlang:exit/1 and erlang:exit/2 References: <871EB3A0-F9ED-4229-B4D4-C90C77A8953D@rogvall.se> Message-ID: On 2014-07-30 05:11 +0800, Tony Rogvall wrote: > BIFs with one (and more) argument are called from emulator loop in beam_emu.c > (R17 source) > beam_emu.c line 2779 in opcode "call_bif_e" > and line 3571 in opcode "apply_bif" > and line 2610 in "i_gc_bif1_jIsId" > > and the rest is just a matter of reading code. > Hint. The function handler_error may be a good start to look in. > > Have fun :-) > > /Tony Thank you for the hint and pointers. exit/1 is handled in OpCase(raise_ss) in beam_emu.c => handle_error => terminate_proc (when the exception is not caught). Leo From carlsson.richard@REDACTED Wed Jul 30 12:40:59 2014 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 30 Jul 2014 12:40:59 +0200 Subject: [erlang-questions] erlang:exit/1 and erlang:exit/2 In-Reply-To: References: Message-ID: Note that exit/1 and exit/2 are completely different functions, hence the difference in implementation complexity. exit/1 is similar to error/1 and throw/1, while exit/2 would have been better named signal/2. /Richard On Tue, Jul 29, 2014 at 9:19 PM, Leo Liu wrote: > Hi there, > > Could someone help me understand the implementation of erlang:exit/1? > > BIF_RETTYPE exit_1(BIF_ALIST_1) > { > BIF_P->fvalue = BIF_ARG_1; /* exit value */ > BIF_ERROR(BIF_P, EXC_EXIT); > } > > which is pretty simple and I am not sure how it propagates the exit > signal. erlang:exit/2 on the contrary is full of details. > > Thanks, > Leo > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marcos.quesadas@REDACTED Wed Jul 30 11:53:16 2014 From: marcos.quesadas@REDACTED (marcos quesada) Date: Wed, 30 Jul 2014 02:53:16 -0700 (PDT) Subject: [erlang-questions] Cluster Functional test using Slave Nodes Message-ID: <7d709a3b-ffe5-40c3-9420-c0f9f0aff0ed@googlegroups.com> Hi there! I'm developing some kind of cluster (toy project), and writing a functional tests using slave:start it's being more dificult than expected... It's just a test where some nodes are started and joinned between them, so that,cluster creation can be asserted. I'm getting crazy trying to find an explanation to an unexpected behavior, that is, starting first node (using make start), and then executing manually cluster_test:cluster_formation() the whole tests works fine. The same test integrated in the app test suite, executed as make quicktests, fails responding timeout: ... module 'cluster_test' cluster_test: cluster_formation_suite_test_...*timed out* undefined [done in 5.001 s] Configuration files between those configurations are identical, when running make start, cluster app is first started, so when i run make quicktests setup step takes care to start up cluster app. I don't understand why slave:start is failing returning a timeout error, it has been hard to solve step by step some minor config errors, so, when finally slaves where working fine executing the test manually i was expecting to complete that functional test suite... Any clue will be great , thanks!! Makefile: start: @$(ERL) -config releases/dev/dev1/etc/app.config -args_file releases/dev/dev1/etc/vm.args \ -pa $PWD apps/*/ebin deps/*/ebin -rsh ssh -boot start_sasl -s cluster quicktests: rm -rf apps/cluster/.eunit/ ERL_FLAGS="-args_file apps/cluster/test/conf/vm.eunit.args -config apps/cluster/test/conf/app.eunit.config -pa $PWD apps/*/ebin deps/*/ebin -rsh ssh" \ ./rebar skip_deps=true compile eunit --verbose cluster_test.erl : ... cluster_formation_suite_test_() -> { foreach, fun setup/0, fun teardown/1, [ fun cluster_formation/0 ] }. setup() -> application:ensure_all_started(cluster, permanent), %% Go outside eunit folder /home/marcos/code/cluster/apps/cluster/.eunit c:cd('../../../'), ok. teardown(_) -> application:stop(cluster), ok. cluster_formation() -> ?ioLog("Pwd is ~p Cookie ~p node ~p ~n", [c:pwd(), erlang:get_cookie(), node()]), Host = '127.0.0.1', Args = " -pa apps/*/ebin deps/*/ebin -setcookie secret -rsh ssh", {ok, Slave1} = slave:start(Host, slave1, Args), {ok, Slave2} = slave:start(Host, slave2, Args), ?assertEqual([], c:pwd()), rpc:call(Slave1, application ,ensure_all_started,[cluster,permanent]), rpc:call(Slave2, application ,ensure_all_started,[cluster,permanent]), ?assertEqual(pong, net_adm:ping(Slave1)), ?assertEqual(pong, net_adm:ping(Slave2)), ?assertEqual(Slave1, 'slave1@REDACTED'), ?assertEqual(Slave2, 'slave2@REDACTED'), Result1 = rpc:call(Slave1, cluster, join,[node()]), ?assertEqual(ok, Result1), Result2 = rpc:call(Slave2, cluster, join,[node()]), ?assertEqual(ok, Result2), ?assertEqual( ['node1@REDACTED','slave1@REDACTED','slave2@REDACTED'], cluster:status() ), slave:stop(Slave1), slave:stop(Slave2). -- You received this message because you are subscribed to the Google Groups "Erlang Programming" group. To unsubscribe from this group and stop receiving emails from it, send an email to erlang-programming+unsubscribe@REDACTED To post to this group, send email to erlang-programming@REDACTED Visit this group at http://groups.google.com/group/erlang-programming. For more options, visit https://groups.google.com/d/optout. From danilagamma@REDACTED Wed Jul 30 18:49:14 2014 From: danilagamma@REDACTED (=?UTF-8?B?0JTQsNC90LjQu9CwINCk0LXQtNGP0YnQuNC9?=) Date: Wed, 30 Jul 2014 20:49:14 +0400 Subject: [erlang-questions] R16 + Centos + Crypto + ESL Package Message-ID: Hello, During our work on migration from R14B04 to R16B03-1 we hit the problem with `crypto` on `CentOS (details here: https://groups.google.com/forum/#!topic/erlang-programming/wW6Uuz4VO2w), We decided to create our own build, where configure line in spec file would look like: './configure ... CFLAGS="-DOPENSSL_NO_EC=1"' That made `crypto` work, but tests showed significant slowdown for _everything_ in VM (every operation became ~2x slower in comparison with R14). We tried ErlangSolutions R16B03-1 CentOS package (was found here: http://packages.erlang-solutions.com/site/esl/esl-erlang/FLAVOUR_1_general/esl-erlang_16.b.3-1~centos~6_amd64.rpm) which was also slow as ours. After some digging we found that `CFLAGS="-DOPENSSL_NO_EC=1"` rewrites `CFLAGS="-g -O2"`, which was the culprit of slowdown. So, we are assuming that Erlang Solutions package was compiled in same way we compiled ours. So the question is: what is the best way to 'append' CFLAGS option, rather then providing '-DOPENSSL_NO_EC=1 -g -O2' assuming that we've appended correct default options? -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Wed Jul 30 21:00:53 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 30 Jul 2014 20:00:53 +0100 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: References: Message-ID: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> So, I set a bunch of flags to update the buffer size: {:buffer, 1024*1024*16}, {:recbuf, 1024*1024*16}, {:sndbuf, 1024*1024*16} Giving quite a hefty amount of buffer space. When I connect 5 clients to the server with an echo peer, and send about 1000 packets from each with 1500 bytes per packet, only 50% of the packets even get registered in the message stack. I?ve conferred with other Erlang devs and have heard their similar servers can support around 1000 such clients. Why, then, do I get this issue? :-( Any idea?s or help would be greatly appreciated. Regards, Lee On 30 Jul 2014, at 10:02, Lee Sylvester wrote: > Hey guys, > > I have a simple UDP server set up, which is experiencing packet loss. If I send it 20 packets of 1000 bytes (1 packet each 20ms), only 16 will reach the handle_info genserver handler. The packet loss gets worse as I increase their number. If I send 100 packets of 1500 bytes each, one every 150ms, I get 75% packet loss. This makes the app totally useless for streaming. Has anyone else experienced this? Is there something I can do to see what's entering the message stack? I?m using statistics to tally the messages at the end of the tests and I?m watching network traffic with Wireshark. The messages are being sent, but many of them never seem to enter the message stack. > > I am setting the active value of the socket to once, and reset it to once with each message. I have also tried just setting it to true, but the results are the same. > > Thanks, > Lee From jesper.louis.andersen@REDACTED Wed Jul 30 21:03:37 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 30 Jul 2014 21:03:37 +0200 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> Message-ID: On Wed, Jul 30, 2014 at 9:00 PM, Lee Sylvester wrote: > So, I set a bunch of flags to update the buffer size: > > {:buffer, 1024*1024*16}, {:recbuf, 1024*1024*16}, {:sndbuf, 1024*1024*16} > > Giving quite a hefty amount of buffer space. When I connect 5 clients to > the server with an echo peer, and send about 1000 packets from each with > 1500 bytes per packet, only 50% of the packets even get registered in the > message stack. I?ve conferred with other Erlang devs and have heard their > similar servers can support around 1000 such clients. Why, then, do I get > this issue? :-( > > Any idea?s or help would be greatly appreciated. > > Regards, > Lee > Start by verifying your kernel is not dropping the packets at its end. If your kernels buffer is full, then things will not work. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Wed Jul 30 21:04:52 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 30 Jul 2014 20:04:52 +0100 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> Message-ID: <4374AF17-07C3-42C9-8356-B0A761B75B9A@gmail.com> Thanks Jesper. How exactly do I check the Kernel?s buffer? Thanks, Lee On 30 Jul 2014, at 20:03, Jesper Louis Andersen wrote: > > On Wed, Jul 30, 2014 at 9:00 PM, Lee Sylvester wrote: > So, I set a bunch of flags to update the buffer size: > > {:buffer, 1024*1024*16}, {:recbuf, 1024*1024*16}, {:sndbuf, 1024*1024*16} > > Giving quite a hefty amount of buffer space. When I connect 5 clients to the server with an echo peer, and send about 1000 packets from each with 1500 bytes per packet, only 50% of the packets even get registered in the message stack. I?ve conferred with other Erlang devs and have heard their similar servers can support around 1000 such clients. Why, then, do I get this issue? :-( > > Any idea?s or help would be greatly appreciated. > > Regards, > Lee > > Start by verifying your kernel is not dropping the packets at its end. If your kernels buffer is full, then things will not work. > > > -- > J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Jul 30 21:06:42 2014 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 30 Jul 2014 23:06:42 +0400 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> Message-ID: I've spent lot of time to make capturing of 600 mbit/s of udp traffic without loss. Don't specify buffer size in erlang, use system variables. sysctl -w net.core.rmem_max=1048576 sysctl -w net.core.rmem_default=1048576 sysctl -w net.ipv4.udp_mem="8388608 12582912 16777216" -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Wed Jul 30 21:09:14 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 30 Jul 2014 20:09:14 +0100 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> Message-ID: <7779829B-C198-48B9-B828-AEEE5D884315@gmail.com> Thanks. How does that make a difference over setting it in the app, though? Thanks, Lee On 30 Jul 2014, at 20:06, Max Lapshin wrote: > I've spent lot of time to make capturing of 600 mbit/s of udp traffic without loss. > > Don't specify buffer size in erlang, use system variables. > > > > > > > sysctl -w net.core.rmem_max=1048576 > sysctl -w net.core.rmem_default=1048576 > sysctl -w net.ipv4.udp_mem="8388608 12582912 16777216" -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Jul 30 21:12:28 2014 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 30 Jul 2014 21:12:28 +0200 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: <7779829B-C198-48B9-B828-AEEE5D884315@gmail.com> References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> <7779829B-C198-48B9-B828-AEEE5D884315@gmail.com> Message-ID: There are two buffers here. The kernel has a buffer and the Erlang VM has a buffer. You are setting the Erlang VMs buffers only. If the kernels buffer fills up, it drops packets, because it is allowed to do so with UDP (UDP is not reliable). Linux doesn't have too many easy ways to see this. But you can do netstat -c --udp -an and watch for large recv-q's. On Wed, Jul 30, 2014 at 9:09 PM, Lee Sylvester wrote: > Thanks. How does that make a difference over setting it in the app, > though? > > Thanks, > Lee > > > On 30 Jul 2014, at 20:06, Max Lapshin wrote: > > I've spent lot of time to make capturing of 600 mbit/s of udp traffic > without loss. > > Don't specify buffer size in erlang, use system variables. > > > > > > > > sysctl -w net.core.rmem_max=1048576 > sysctl -w net.core.rmem_default=1048576 > sysctl -w net.ipv4.udp_mem="8388608 12582912 16777216" > > > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Wed Jul 30 21:14:33 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 30 Jul 2014 20:14:33 +0100 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> <7779829B-C198-48B9-B828-AEEE5D884315@gmail.com> Message-ID: Okay, I?ll try that. Just so you know, though, I also receive packet loss with TCP, only at 35 clients +, rather than a dismal 3+ Lee On 30 Jul 2014, at 20:12, Jesper Louis Andersen wrote: > There are two buffers here. The kernel has a buffer and the Erlang VM has a buffer. You are setting the Erlang VMs buffers only. If the kernels buffer fills up, it drops packets, because it is allowed to do so with UDP (UDP is not reliable). Linux doesn't have too many easy ways to see this. But you can do netstat -c --udp -an and watch for large recv-q's. > > > On Wed, Jul 30, 2014 at 9:09 PM, Lee Sylvester wrote: > Thanks. How does that make a difference over setting it in the app, though? > > Thanks, > Lee > > > On 30 Jul 2014, at 20:06, Max Lapshin wrote: > >> I've spent lot of time to make capturing of 600 mbit/s of udp traffic without loss. >> >> Don't specify buffer size in erlang, use system variables. >> >> >> >> >> >> >> >> >> >> sysctl -w net.core.rmem_max=1048576 >> sysctl -w net.core.rmem_default=1048576 >> sysctl -w net.ipv4.udp_mem="8388608 12582912 16777216" > > > > > -- > J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From develop7@REDACTED Wed Jul 30 21:16:03 2014 From: develop7@REDACTED (Andrei Dziahel) Date: Wed, 30 Jul 2014 22:16:03 +0300 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> <7779829B-C198-48B9-B828-AEEE5D884315@gmail.com> Message-ID: you also could check last column of `cat /proc/net/udp` output On Wed, Jul 30, 2014 at 10:12 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > There are two buffers here. The kernel has a buffer and the Erlang VM has > a buffer. You are setting the Erlang VMs buffers only. If the kernels > buffer fills up, it drops packets, because it is allowed to do so with UDP > (UDP is not reliable). Linux doesn't have too many easy ways to see this. > But you can do netstat -c --udp -an and watch for large recv-q's. > > > On Wed, Jul 30, 2014 at 9:09 PM, Lee Sylvester > wrote: > >> Thanks. How does that make a difference over setting it in the app, >> though? >> >> Thanks, >> Lee >> >> >> On 30 Jul 2014, at 20:06, Max Lapshin wrote: >> >> I've spent lot of time to make capturing of 600 mbit/s of udp traffic >> without loss. >> >> Don't specify buffer size in erlang, use system variables. >> >> >> >> >> >> >> >> >> >> >> sysctl -w net.core.rmem_max=1048576 >> sysctl -w net.core.rmem_default=1048576 >> sysctl -w net.ipv4.udp_mem="8388608 12582912 16777216" >> >> >> > > > -- > J. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Regards, Andrei Dziahel -------------- next part -------------- An HTML attachment was scrubbed... URL: From develop7@REDACTED Wed Jul 30 21:16:35 2014 From: develop7@REDACTED (Andrei Dziahel) Date: Wed, 30 Jul 2014 22:16:35 +0300 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> Message-ID: Do you think fetching packets from passive socket (created with {active, false}) with gen_udp:recv (in separate process ofc) would help prevent packet drops? On Wed, Jul 30, 2014 at 10:06 PM, Max Lapshin wrote: > I've spent lot of time to make capturing of 600 mbit/s of udp traffic > without loss. > > Don't specify buffer size in erlang, use system variables. > > > > > > > > sysctl -w net.core.rmem_max=1048576 > sysctl -w net.core.rmem_default=1048576 > sysctl -w net.ipv4.udp_mem="8388608 12582912 16777216" > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Regards, Andrei Dziahel -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Wed Jul 30 21:18:03 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 30 Jul 2014 20:18:03 +0100 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> Message-ID: I don?t think that would make any difference, since the packets never show up in the message stack, anyway. Lee On 30 Jul 2014, at 20:16, Andrei Dziahel wrote: > Do you think fetching packets from passive socket (created with {active, false}) with gen_udp:recv (in separate process ofc) would help prevent packet drops? > > > On Wed, Jul 30, 2014 at 10:06 PM, Max Lapshin wrote: > I've spent lot of time to make capturing of 600 mbit/s of udp traffic without loss. > > Don't specify buffer size in erlang, use system variables. > > > > > > > > > > sysctl -w net.core.rmem_max=1048576 > sysctl -w net.core.rmem_default=1048576 > sysctl -w net.ipv4.udp_mem="8388608 12582912 16777216" > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Regards, > Andrei Dziahel > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Wed Jul 30 21:21:33 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 30 Jul 2014 20:21:33 +0100 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> Message-ID: Okay, so, I?m having trouble with this. Mainly because I?m on a Mac. When I update sysctl.conf and reload it, it doesn?t show the changes for net.core. In fact, it doesn?t show anything for net.core. Lee On 30 Jul 2014, at 20:06, Max Lapshin wrote: > I've spent lot of time to make capturing of 600 mbit/s of udp traffic without loss. > > Don't specify buffer size in erlang, use system variables. > > > > > > > sysctl -w net.core.rmem_max=1048576 > sysctl -w net.core.rmem_default=1048576 > sysctl -w net.ipv4.udp_mem="8388608 12582912 16777216" -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Wed Jul 30 21:24:56 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 30 Jul 2014 20:24:56 +0100 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> Message-ID: <78C596CA-6CCD-474B-8A4F-594DAE56B590@gmail.com> Okay, so I?m updating them using sysctl -w kern.ipc.maxsockbuf=16777216 sysctl -w net.inet.tcp.sendspace=1048576 sysctl -w net.inet.tcp.recvspace=1048576 Apparently that produces the same result. 16777216 is reported as too large for the maxsockbuf, though. I?m on a Macbook Pro with 8Gb. :rolleyes: Lee On 30 Jul 2014, at 20:06, Max Lapshin wrote: > I've spent lot of time to make capturing of 600 mbit/s of udp traffic without loss. > > Don't specify buffer size in erlang, use system variables. > > > > > > > sysctl -w net.core.rmem_max=1048576 > sysctl -w net.core.rmem_default=1048576 > sysctl -w net.ipv4.udp_mem="8388608 12582912 16777216" -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Jul 30 21:32:47 2014 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 30 Jul 2014 23:32:47 +0400 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: <78C596CA-6CCD-474B-8A4F-594DAE56B590@gmail.com> References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> <78C596CA-6CCD-474B-8A4F-594DAE56B590@gmail.com> Message-ID: Well, Mac is a great platform for surfing and developing, but not for running server software. Can you launch your server on linux/freebsd? -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Wed Jul 30 21:33:54 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 30 Jul 2014 20:33:54 +0100 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> <78C596CA-6CCD-474B-8A4F-594DAE56B590@gmail.com> Message-ID: <37A3435A-F892-4DEC-BFE0-6A9EBCE77C15@gmail.com> Tomorrow I will be able to. How is it, though, that my C++ server works fine? Surely, if one does, the other should, unless it?s VM limitations? Lee On 30 Jul 2014, at 20:32, Max Lapshin wrote: > > Well, Mac is a great platform for surfing and developing, but not for running server software. > > Can you launch your server on linux/freebsd? > From max.lapshin@REDACTED Wed Jul 30 21:44:59 2014 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 30 Jul 2014 23:44:59 +0400 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: <37A3435A-F892-4DEC-BFE0-6A9EBCE77C15@gmail.com> References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> <78C596CA-6CCD-474B-8A4F-594DAE56B590@gmail.com> <37A3435A-F892-4DEC-BFE0-6A9EBCE77C15@gmail.com> Message-ID: inet:setopts(Socket, [{active,once}]), process_flag(priority, high), this will help you -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Wed Jul 30 21:48:20 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 30 Jul 2014 20:48:20 +0100 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> <78C596CA-6CCD-474B-8A4F-594DAE56B590@gmail.com> <37A3435A-F892-4DEC-BFE0-6A9EBCE77C15@gmail.com> Message-ID: <0B1D035F-5A41-4772-8769-37D5FD779127@gmail.com> Sorry, still 50% packet loss :-( On 30 Jul 2014, at 20:44, Max Lapshin wrote: > inet:setopts(Socket, [{active,once}]), > process_flag(priority, high), > > > this will help you From felixgallo@REDACTED Wed Jul 30 21:55:31 2014 From: felixgallo@REDACTED (Felix Gallo) Date: Wed, 30 Jul 2014 12:55:31 -0700 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: <0B1D035F-5A41-4772-8769-37D5FD779127@gmail.com> References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> <78C596CA-6CCD-474B-8A4F-594DAE56B590@gmail.com> <37A3435A-F892-4DEC-BFE0-6A9EBCE77C15@gmail.com> <0B1D035F-5A41-4772-8769-37D5FD779127@gmail.com> Message-ID: you mention a vm, are you running osx in a vm? On Wed, Jul 30, 2014 at 12:48 PM, Lee Sylvester wrote: > Sorry, still 50% packet loss :-( > > > > On 30 Jul 2014, at 20:44, Max Lapshin wrote: > > > inet:setopts(Socket, [{active,once}]), > > process_flag(priority, high), > > > > > > this will help you > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Wed Jul 30 21:56:21 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 30 Jul 2014 20:56:21 +0100 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> <78C596CA-6CCD-474B-8A4F-594DAE56B590@gmail.com> <37A3435A-F892-4DEC-BFE0-6A9EBCE77C15@gmail.com> <0B1D035F-5A41-4772-8769-37D5FD779127@gmail.com> Message-ID: Oh, sorry, no I meant the EVM. :-) Lee On 30 Jul 2014, at 20:55, Felix Gallo wrote: > you mention a vm, are you running osx in a vm? > > > On Wed, Jul 30, 2014 at 12:48 PM, Lee Sylvester wrote: > Sorry, still 50% packet loss :-( > > > > On 30 Jul 2014, at 20:44, Max Lapshin wrote: > > > inet:setopts(Socket, [{active,once}]), > > process_flag(priority, high), > > > > > > this will help you > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From per@REDACTED Wed Jul 30 22:07:08 2014 From: per@REDACTED (Per Hedeland) Date: Wed, 30 Jul 2014 22:07:08 +0200 (CEST) Subject: [erlang-questions] R16 + Centos + Crypto + ESL Package In-Reply-To: Message-ID: <201407302007.s6UK7859028049@pluto.hedeland.org> ???????????? ?????????????? wrote: > >After some digging we found that `CFLAGS="-DOPENSSL_NO_EC=1"` rewrites >`CFLAGS="-g -O2"`, which was the culprit of slowdown. Ouch, I knew there was a reason I always try to avoid passing C compiler flags to OTP's 'configure'.:-) >So the question is: what is the best way to 'append' CFLAGS option, rather >then providing '-DOPENSSL_NO_EC=1 -g -O2' assuming that we've appended >correct default options? The general, semi-ugly method I use when I need to pass some C compiler flags is to create a wrapper, a la $ cat /usr/local/softfloat/bin/gcc #!/bin/sh exec /usr/local/gcc-4.2.4/bin/gcc -msoft-float "$@" $ - and then make sure to have the directory with the wrapper early on in $PATH. If I'm really ambitious I may name the wrapper gcc-- and pass that as CC (IIRC) to 'configure'. For this particular case though, if you aren't afraid to touch the source, you can apply this diff instead: --- lib/crypto/c_src/crypto.c (revision 69183) +++ lib/crypto/c_src/crypto.c (revision 69184) @@ -77,7 +77,8 @@ #if OPENSSL_VERSION_NUMBER >= 0x009080ffL \ && !defined(OPENSSL_NO_EC) \ && !defined(OPENSSL_NO_ECDH) \ - && !defined(OPENSSL_NO_ECDSA) + && !defined(OPENSSL_NO_ECDSA) \ + && !defined(OPENSSL_NO_EC2M) # define HAVE_EC #endif I was actually about to report it as a bug fix, but found that R17 - uh, 17? - has much better one (it disables *only* the EC2M stuff which is all that is missing on "current" CentOS/RHEL at least) - thanks to the OTP team for that! - "in advance", we're not ready for "17" yet. If you are, I suggest that you try it. --Per Hedeland From Michael.K.Schmidt@REDACTED Wed Jul 30 22:07:14 2014 From: Michael.K.Schmidt@REDACTED (Michael.K.Schmidt@REDACTED) Date: Wed, 30 Jul 2014 15:07:14 -0500 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: <78C596CA-6CCD-474B-8A4F-594DAE56B590@gmail.com> References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> <78C596CA-6CCD-474B-8A4F-594DAE56B590@gmail.com> Message-ID: Don't you want to adjust the UDP (not TCP) settings? something like sysctl -w net.inet.udp.recvspace=1048576 I don't know if this still applies, but there is a writeup here on how to increase maxsockbuf: http://lists.apple.com/archives/darwin-kernel/2012/Apr/msg00042.html # tell the system to use more than the # default 32768 mbufclusters via the ncl boot arg nvram boot-args="ncl=131072" (reboot) sysctl -w kern.ipc.maxsockbuf=16777216 sysctl -w net.inet.udp.recvspace=8388608 Hope that helps, Mike erlang-questions-bounces@REDACTED wrote on 07/30/2014 02:24:56 PM: > Lee Sylvester > Sent by: erlang-questions-bounces@REDACTED > > 07/30/2014 02:24 PM > > To > > Max Lapshin , > > cc > > "Erlang Users' List" > > Subject > > Re: [erlang-questions] Gen_UDP Packet Loss > > Okay, so I?m updating them using > > sysctl -w kern.ipc.maxsockbuf=16777216 > sysctl -w net.inet.tcp.sendspace=1048576 > sysctl -w net.inet.tcp.recvspace=1048576 > > Apparently that produces the same result. 16777216 is reported as > too large for the maxsockbuf, though. I?m on a Macbook Pro with > 8Gb. :rolleyes: > > Lee > > On 30 Jul 2014, at 20:06, Max Lapshin wrote: > > I've spent lot of time to make capturing of 600 mbit/s of udp > traffic without loss. > > Don't specify buffer size in erlang, use system variables. > > > > > sysctl -w net.core.rmem_max=1048576 > sysctl -w net.core.rmem_default=1048576 > sysctl -w net.ipv4.udp_mem="8388608 12582912 16777216" > > > ______________________________________________________________________ > This email has been scanned by the Symantec Email Security.cloud service. > ______________________________________________________________________ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From lee.sylvester@REDACTED Wed Jul 30 22:12:47 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 30 Jul 2014 21:12:47 +0100 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> <78C596CA-6CCD-474B-8A4F-594DAE56B590@gmail.com> Message-ID: <012CCC8B-D678-4D49-99F6-D8E3D83650B8@gmail.com> Thanks, Mike. I tried that, but it hasn?t made any changes :-( Lee On 30 Jul 2014, at 21:07, Michael.K.Schmidt@REDACTED wrote: > > Don't you want to adjust the UDP (not TCP) settings? something like > > sysctl -w net.inet.udp.recvspace=1048576 > > I don't know if this still applies, but there is a writeup here on how to > increase maxsockbuf: > http://lists.apple.com/archives/darwin-kernel/2012/Apr/msg00042.html > > # tell the system to use more than the > # default 32768 mbufclusters via the ncl boot arg > nvram boot-args="ncl=131072" > > (reboot) > > sysctl -w kern.ipc.maxsockbuf=16777216 > sysctl -w net.inet.udp.recvspace=8388608 > > > Hope that helps, > Mike > > > erlang-questions-bounces@REDACTED wrote on 07/30/2014 02:24:56 PM: > >> Lee Sylvester >> Sent by: erlang-questions-bounces@REDACTED >> >> 07/30/2014 02:24 PM >> >> To >> >> Max Lapshin , >> >> cc >> >> "Erlang Users' List" >> >> Subject >> >> Re: [erlang-questions] Gen_UDP Packet Loss >> >> Okay, so I?m updating them using >> >> sysctl -w kern.ipc.maxsockbuf=16777216 >> sysctl -w net.inet.tcp.sendspace=1048576 >> sysctl -w net.inet.tcp.recvspace=1048576 >> >> Apparently that produces the same result. 16777216 is reported as >> too large for the maxsockbuf, though. I?m on a Macbook Pro with >> 8Gb. :rolleyes: >> >> Lee >> >> On 30 Jul 2014, at 20:06, Max Lapshin wrote: >> >> I've spent lot of time to make capturing of 600 mbit/s of udp >> traffic without loss. >> >> Don't specify buffer size in erlang, use system variables. >> >> >> >> >> sysctl -w net.core.rmem_max=1048576 >> sysctl -w net.core.rmem_default=1048576 >> sysctl -w net.ipv4.udp_mem="8388608 12582912 16777216" >> >> >> ______________________________________________________________________ >> This email has been scanned by the Symantec Email Security.cloud service. >> ______________________________________________________________________ >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From Michael.K.Schmidt@REDACTED Wed Jul 30 22:24:47 2014 From: Michael.K.Schmidt@REDACTED (Michael.K.Schmidt@REDACTED) Date: Wed, 30 Jul 2014 15:24:47 -0500 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: <012CCC8B-D678-4D49-99F6-D8E3D83650B8@gmail.com> References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> <78C596CA-6CCD-474B-8A4F-594DAE56B590@gmail.com> <012CCC8B-D678-4D49-99F6-D8E3D83650B8@gmail.com> Message-ID: One other thing to keep in mind is that the ethernet switch will have a big impact on your test. If a packet collision occurs inside the switch (when a packet is being transferred from one port queue to another), it is can be hard dropped. The sending NIC thinks the packet was transmitted, so no attempt is made to retransmit on the wire. Higher-End switches handle this better (more queue space,etc), but it can still be an issue. Lee Sylvester wrote on 07/30/2014 03:12:47 PM: > Lee Sylvester > 07/30/2014 03:12 PM > > To > > Michael.K.Schmidt@REDACTED, > > cc > > "Erlang Users' List" , erlang- > questions-bounces@REDACTED, Max Lapshin > > Subject > > Re: [erlang-questions] Gen_UDP Packet Loss > > Thanks, Mike. I tried that, but it hasn?t made any changes :-( > > Lee > > > On 30 Jul 2014, at 21:07, Michael.K.Schmidt@REDACTED wrote: > > > > > Don't you want to adjust the UDP (not TCP) settings? something like > > > > sysctl -w net.inet.udp.recvspace=1048576 > > > > I don't know if this still applies, but there is a writeup here on how to > > increase maxsockbuf: > > http://lists.apple.com/archives/darwin-kernel/2012/Apr/msg00042.html > > > > # tell the system to use more than the > > # default 32768 mbufclusters via the ncl boot arg > > nvram boot-args="ncl=131072" > > > > (reboot) > > > > sysctl -w kern.ipc.maxsockbuf=16777216 > > sysctl -w net.inet.udp.recvspace=8388608 > > > > > > Hope that helps, > > Mike > > > > > > erlang-questions-bounces@REDACTED wrote on 07/30/2014 02:24:56 PM: > > > >> Lee Sylvester > >> Sent by: erlang-questions-bounces@REDACTED > >> > >> 07/30/2014 02:24 PM > >> > >> To > >> > >> Max Lapshin , > >> > >> cc > >> > >> "Erlang Users' List" > >> > >> Subject > >> > >> Re: [erlang-questions] Gen_UDP Packet Loss > >> > >> Okay, so I?m updating them using > >> > >> sysctl -w kern.ipc.maxsockbuf=16777216 > >> sysctl -w net.inet.tcp.sendspace=1048576 > >> sysctl -w net.inet.tcp.recvspace=1048576 > >> > >> Apparently that produces the same result. 16777216 is reported as > >> too large for the maxsockbuf, though. I?m on a Macbook Pro with > >> 8Gb. :rolleyes: > >> > >> Lee > >> > >> On 30 Jul 2014, at 20:06, Max Lapshin wrote: > >> > >> I've spent lot of time to make capturing of 600 mbit/s of udp > >> traffic without loss. > >> > >> Don't specify buffer size in erlang, use system variables. > >> > >> > >> > >> > >> sysctl -w net.core.rmem_max=1048576 > >> sysctl -w net.core.rmem_default=1048576 > >> sysctl -w net.ipv4.udp_mem="8388608 12582912 16777216" > >> > >> > >> ______________________________________________________________________ > >> This email has been scanned by the Symantec Email Security.cloud service. > >> ______________________________________________________________________ > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > ______________________________________________________________________ > This email has been scanned by the Symantec Email Security.cloud service. > ______________________________________________________________________ From tony@REDACTED Thu Jul 31 00:20:01 2014 From: tony@REDACTED (Tony Rogvall) Date: Thu, 31 Jul 2014 00:20:01 +0200 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> <7779829B-C198-48B9-B828-AEEE5D884315@gmail.com> Message-ID: On 30 jul 2014, at 21:14, Lee Sylvester wrote: > Okay, I?ll try that. Just so you know, though, I also receive packet loss with TCP, only at 35 clients +, rather than a dismal 3+ > Could please explain what you mean with "receive packet loss with TCP" ? Do you mean that you see the packet error on the network stat ? Have you checked your equipment, connected the cables correctly, power cycled everything :-) /Tony > Lee > > > On 30 Jul 2014, at 20:12, Jesper Louis Andersen wrote: > >> There are two buffers here. The kernel has a buffer and the Erlang VM has a buffer. You are setting the Erlang VMs buffers only. If the kernels buffer fills up, it drops packets, because it is allowed to do so with UDP (UDP is not reliable). Linux doesn't have too many easy ways to see this. But you can do netstat -c --udp -an and watch for large recv-q's. >> >> >> On Wed, Jul 30, 2014 at 9:09 PM, Lee Sylvester wrote: >> Thanks. How does that make a difference over setting it in the app, though? >> >> Thanks, >> Lee >> >> >> On 30 Jul 2014, at 20:06, Max Lapshin wrote: >> >>> I've spent lot of time to make capturing of 600 mbit/s of udp traffic without loss. >>> >>> Don't specify buffer size in erlang, use system variables. >>> >>> >>> >>> >>> >>> >>> >>> >>> sysctl -w net.core.rmem_max=1048576 >>> sysctl -w net.core.rmem_default=1048576 >>> sysctl -w net.ipv4.udp_mem="8388608 12582912 16777216" >> >> >> >> >> -- >> J. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Thu Jul 31 01:45:03 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Thu, 31 Jul 2014 00:45:03 +0100 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> <7779829B-C198-48B9-B828-AEEE5D884315@gmail.com> Message-ID: <007B063E-77F4-46BB-9E9A-ACE42447B1AC@gmail.com> What I mean is that, Wireshark will show 1000 packets of 1500 bytes over 5 connections being sent to the server on a given port. The EVM statistics show only around 50% of those messages every being recorded. I?ve now tried this on a remote server and the results are even worse, so it can?t be my dev machine. The server uses 4 Xeon processors, SSD?s and runs CentOS with a huge fat bandwidth. If I run the beam files directly in the EVM (using Erl), can I expect different results? What exactly is iex -S mix doing under the covers? Thanks, Lee On 30 Jul 2014, at 23:20, Tony Rogvall wrote: > > On 30 jul 2014, at 21:14, Lee Sylvester wrote: > >> Okay, I?ll try that. Just so you know, though, I also receive packet loss with TCP, only at 35 clients +, rather than a dismal 3+ >> > Could please explain what you mean with "receive packet loss with TCP" ? > > Do you mean that you see the packet error on the network stat ? > > Have you checked your equipment, connected the cables correctly, power cycled everything :-) > > /Tony > > >> Lee >> >> >> On 30 Jul 2014, at 20:12, Jesper Louis Andersen wrote: >> >>> There are two buffers here. The kernel has a buffer and the Erlang VM has a buffer. You are setting the Erlang VMs buffers only. If the kernels buffer fills up, it drops packets, because it is allowed to do so with UDP (UDP is not reliable). Linux doesn't have too many easy ways to see this. But you can do netstat -c --udp -an and watch for large recv-q's. >>> >>> >>> On Wed, Jul 30, 2014 at 9:09 PM, Lee Sylvester wrote: >>> Thanks. How does that make a difference over setting it in the app, though? >>> >>> Thanks, >>> Lee >>> >>> >>> On 30 Jul 2014, at 20:06, Max Lapshin wrote: >>> >>>> I've spent lot of time to make capturing of 600 mbit/s of udp traffic without loss. >>>> >>>> Don't specify buffer size in erlang, use system variables. >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> sysctl -w net.core.rmem_max=1048576 >>>> sysctl -w net.core.rmem_default=1048576 >>>> sysctl -w net.ipv4.udp_mem="8388608 12582912 16777216" >>> >>> >>> >>> >>> -- >>> J. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Thu Jul 31 01:46:29 2014 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Thu, 31 Jul 2014 00:46:29 +0100 Subject: [erlang-questions] Gen_UDP Packet Loss In-Reply-To: <007B063E-77F4-46BB-9E9A-ACE42447B1AC@gmail.com> References: <74DEA7CC-C830-4DA2-AFC7-42976FC05C7F@gmail.com> <7779829B-C198-48B9-B828-AEEE5D884315@gmail.com> <007B063E-77F4-46BB-9E9A-ACE42447B1AC@gmail.com> Message-ID: Oops, sorry, I thought this was the thread in the Elixir group. The app is written in Elixir. Lee On 31 Jul 2014, at 00:45, Lee Sylvester wrote: > What I mean is that, Wireshark will show 1000 packets of 1500 bytes over 5 connections being sent to the server on a given port. The EVM statistics show only around 50% of those messages every being recorded. I?ve now tried this on a remote server and the results are even worse, so it can?t be my dev machine. The server uses 4 Xeon processors, SSD?s and runs CentOS with a huge fat bandwidth. > > If I run the beam files directly in the EVM (using Erl), can I expect different results? What exactly is iex -S mix doing under the covers? > > Thanks, > Lee > > > On 30 Jul 2014, at 23:20, Tony Rogvall wrote: > >> >> On 30 jul 2014, at 21:14, Lee Sylvester wrote: >> >>> Okay, I?ll try that. Just so you know, though, I also receive packet loss with TCP, only at 35 clients +, rather than a dismal 3+ >>> >> Could please explain what you mean with "receive packet loss with TCP" ? >> >> Do you mean that you see the packet error on the network stat ? >> >> Have you checked your equipment, connected the cables correctly, power cycled everything :-) >> >> /Tony >> >> >>> Lee >>> >>> >>> On 30 Jul 2014, at 20:12, Jesper Louis Andersen wrote: >>> >>>> There are two buffers here. The kernel has a buffer and the Erlang VM has a buffer. You are setting the Erlang VMs buffers only. If the kernels buffer fills up, it drops packets, because it is allowed to do so with UDP (UDP is not reliable). Linux doesn't have too many easy ways to see this. But you can do netstat -c --udp -an and watch for large recv-q's. >>>> >>>> >>>> On Wed, Jul 30, 2014 at 9:09 PM, Lee Sylvester wrote: >>>> Thanks. How does that make a difference over setting it in the app, though? >>>> >>>> Thanks, >>>> Lee >>>> >>>> >>>> On 30 Jul 2014, at 20:06, Max Lapshin wrote: >>>> >>>>> I've spent lot of time to make capturing of 600 mbit/s of udp traffic without loss. >>>>> >>>>> Don't specify buffer size in erlang, use system variables. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> sysctl -w net.core.rmem_max=1048576 >>>>> sysctl -w net.core.rmem_default=1048576 >>>>> sysctl -w net.ipv4.udp_mem="8388608 12582912 16777216" >>>> >>>> >>>> >>>> >>>> -- >>>> J. >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aschultz@REDACTED Thu Jul 31 11:33:34 2014 From: aschultz@REDACTED (Andreas Schultz) Date: Thu, 31 Jul 2014 09:33:34 +0000 (UTC) Subject: [erlang-questions] R16 + Centos + Crypto + ESL Package In-Reply-To: <201407302007.s6UK7859028049@pluto.hedeland.org> References: <201407302007.s6UK7859028049@pluto.hedeland.org> Message-ID: <702664485.8947.1406799214774.JavaMail.zimbra@tpip.net> Hi, ----- Original Message ----- > ???????????? ?????????????? wrote: > > > >After some digging we found that `CFLAGS="-DOPENSSL_NO_EC=1"` rewrites > >`CFLAGS="-g -O2"`, which was the culprit of slowdown. > > Ouch, I knew there was a reason I always try to avoid passing C compiler > flags to OTP's 'configure'.:-) > > >So the question is: what is the best way to 'append' CFLAGS option, rather > >then providing '-DOPENSSL_NO_EC=1 -g -O2' assuming that we've appended > >correct default options? > > The general, semi-ugly method I use when I need to pass some C compiler > flags is to create a wrapper, a la > > $ cat /usr/local/softfloat/bin/gcc > #!/bin/sh > > exec /usr/local/gcc-4.2.4/bin/gcc -msoft-float "$@" > $ > > - and then make sure to have the directory with the wrapper early on in > $PATH. If I'm really ambitious I may name the wrapper gcc-- > and pass that as CC (IIRC) to 'configure'. > > For this particular case though, if you aren't afraid to touch the > source, you can apply this diff instead: > > --- lib/crypto/c_src/crypto.c (revision 69183) > +++ lib/crypto/c_src/crypto.c (revision 69184) > @@ -77,7 +77,8 @@ > #if OPENSSL_VERSION_NUMBER >= 0x009080ffL \ > && !defined(OPENSSL_NO_EC) \ > && !defined(OPENSSL_NO_ECDH) \ > - && !defined(OPENSSL_NO_ECDSA) > + && !defined(OPENSSL_NO_ECDSA) \ > + && !defined(OPENSSL_NO_EC2M) > # define HAVE_EC > #endif > > > I was actually about to report it as a bug fix, but found that R17 - uh, > 17? - has much better one (it disables *only* the EC2M stuff which is > all that is missing on "current" CentOS/RHEL at least) - thanks to the > OTP team for that! - "in advance", we're not ready for "17" yet. If you > are, I suggest that you try it. The R17 version was added by this pull request: https://github.com/erlang/otp/pull/120 You could try to apply this to R16 as well. Andreas > > --Per Hedeland > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- -- Dipl. Inform. Andreas Schultz From schneider@REDACTED Thu Jul 31 14:08:52 2014 From: schneider@REDACTED (Frans Schneider) Date: Thu, 31 Jul 2014 14:08:52 +0200 Subject: [erlang-questions] terminate Cowboy Message-ID: <53DA31D4.40401@xs4all.nl> Hi list, I have a problem with stopping Cowboy from a gen_server. I use trap exit. This is the code. terminate(_Reason, _State) -> error_logger:info_msg("Cowboy was here"), try Rv = cowboy:stop_listener(xmlrpc_http_listener), %Rv = 1 / 0, error_logger:info_msg("Cowboy terminate Rv ~p", [Rv]) catch _:_ -> error_logger:info_msg("Cowboy CRASH") end, ok. I get the message "Cowboy was here" and nothing else. When I replace the "cowboy:stop_listener" with the division by 0, the cowboy CRASH shows up. Maybe that's because the compiler already detected the error? Could somebody explain this behaviour? Thanks, Frans From jj@REDACTED Thu Jul 31 14:02:42 2014 From: jj@REDACTED (Giovanni Giorgi) Date: Thu, 31 Jul 2014 14:02:42 +0200 Subject: [erlang-questions] =?utf-8?q?Performance_measurement_for_er=5Fzau?= =?utf-8?q?ker?= Message-ID: Hi all, I am writing a code indexer called er_zaueker: it is based on erlang and redis: https://github.com/daitangio/er_zauker It is also my "erlang learning project". I am trying to optimize it. What are the OTP API I can use to find out where is spent much of the time? I need an aggregate timing value to find the slowest functions. er_zauker breaks the files in "trigram" and push them to redis. This part is done sequentially on a single file, whereas file processing is spread via a lot of processes. The entire work is higly I/O bound, but I hope to break the 10 files per second barrier soon. I'd like tu understang if it make sense to have more processes per single file. To collect trigram I am using the sets module, but I think R17 maps could be even more fast. --- Gio's Blog http://gioorgi.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Thu Jul 31 17:15:18 2014 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Thu, 31 Jul 2014 18:15:18 +0300 Subject: [erlang-questions] Performance measurement for er_zauker In-Reply-To: References: Message-ID: Hi, I am using https://github.com/virtan/eep - Dmitry On 31 Jul 2014, at 15:02, Giovanni Giorgi wrote: > Hi all, > > I am writing a code indexer called er_zaueker: it is based on erlang and redis: https://github.com/daitangio/er_zauker > > > It is also my "erlang learning project". > > I am trying to optimize it. > > What are the OTP API I can use to find out where is spent much of the time? > > I need an aggregate timing value to find the slowest functions. > > er_zauker breaks the files in "trigram" and push them to redis. This part is done sequentially on a single file, whereas file processing is spread via a lot of processes. > > The entire work is higly I/O bound, but I hope to break the 10 files per second barrier soon. > > I'd like tu understang if it make sense to have more processes per single file. > > To collect trigram I am using the sets module, but I think R17 maps could be even more fast. > > > --- > Gio's Blog http://gioorgi.com > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From per@REDACTED Thu Jul 31 18:29:02 2014 From: per@REDACTED (Per Hedeland) Date: Thu, 31 Jul 2014 18:29:02 +0200 (CEST) Subject: [erlang-questions] R16 + Centos + Crypto + ESL Package In-Reply-To: <702664485.8947.1406799214774.JavaMail.zimbra@tpip.net> Message-ID: <201407311629.s6VGT2lS049742@pluto.hedeland.org> Andreas Schultz wrote: > >----- Original Message ----- >> >> I was actually about to report it as a bug fix, but found that R17 - uh, >> 17? - has much better one (it disables *only* the EC2M stuff which is >> all that is missing on "current" CentOS/RHEL at least) - thanks to the >> OTP team for that! - "in advance", we're not ready for "17" yet. If you >> are, I suggest that you try it. > >The R17 version was added by this pull request: >https://github.com/erlang/otp/pull/120 > >You could try to apply this to R16 as well. Thanks! When I looked into backporting - can't remember if I looked at these commits or simply diffed the R16 and R17 sources - it seemed to me that the EC2M-specific changes were dependant on a significant amount of other stuff, and so gave up on it, since I didn't have a pressing need to have EC at all. E.g. looking just at the crypto.c changes in the above, there are algo_xxx[] arrays (with algo_pubkey[] being extended), that don't exist at all in R16. But maybe it is doable with some work. --Per From yoursurrogategod@REDACTED Thu Jul 31 19:08:59 2014 From: yoursurrogategod@REDACTED (yoursurrogategod@REDACTED) Date: Thu, 31 Jul 2014 13:08:59 -0400 Subject: [erlang-questions] How to run erlang code on GPU Message-ID: <15F2F098-EBA2-4203-AFEA-A4C51F803716@gmail.com> Has anyone done this or given the GPU architecture it's impossible? Sent from my iPhone From daniel.goertzen@REDACTED Thu Jul 31 21:28:39 2014 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Thu, 31 Jul 2014 14:28:39 -0500 Subject: [erlang-questions] map module functions, map "cons" idea Message-ID: I just watched Kenneth?s map talk from the EUC (http://vimeo.com/99747090). He asked for feedback on maps, so here?s mine: 1. I ported a few functions from the lists module. The ones I found useful were filter() and mapfold(). I also found with() useful, which does the opposite of maps:without(). I would be very happy to see these 3 functions in the maps module. 2. In the talk there was a question about matching an empty map, and also a question about precise matching like how one would match a list. I have always wondered similar things, so it got me thinking? What if we use a "cons" in a map pattern to allow matching all remaining keys that were not selected in the main part of the pattern? This would be roughly symmetric to what you can do with lists and binaries. For example: #{ a:=A, c:=C | Rest } = #{a=>1, b=>2, c=>3, d=>4}. % Rest = #{b=>2, d=>4} That in itself I think has limited use, but it could be used to exactly match a map (ie, no extra pairs other than those specified in the match): f(#{ a:=A, c:=C | Rest }) when maps:size(Rest)==0 -> ? If the pattern to the right of the bar is elided, we could take it to mean ?match the empty map?. The previous example could be rewritten as: f(#{ a:=A, c:=C |}) -> ? This also lets you match just the empty map as was asked in the talk: f(#{|}) -> ? Apologies if something like this has already been discussed. Dan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablo.polvorin@REDACTED Thu Jul 31 23:01:12 2014 From: pablo.polvorin@REDACTED (Pablo Polvorin) Date: Thu, 31 Jul 2014 18:01:12 -0300 Subject: [erlang-questions] Performance measurement for er_zauker In-Reply-To: References: Message-ID: I wrote a blog post time ago explaining a simple usage of fprof to discover possible bottlenecks, maybe it can help you getting started. http://blog.process-one.net/optimizing-erlang-applications-emysql/ fprof is only one of many available profiling tools, each has different trade-offs. On 31 July 2014 12:15, Dmitry Kolesnikov wrote: > Hi, > > I am using https://github.com/virtan/eep > > - Dmitry > > > On 31 Jul 2014, at 15:02, Giovanni Giorgi wrote: > > Hi all, > > I am writing a code indexer called er_zaueker: it is based on erlang and > redis: https://github.com/daitangio/er_zauker > > > It is also my "erlang learning project". > > I am trying to optimize it. > > What are the OTP API I can use to find out where is spent much of the time? > > I need an aggregate timing value to find the slowest functions. > > er_zauker breaks the files in "trigram" and push them to redis. This part > is done sequentially on a single file, whereas file processing is spread > via a lot of processes. > > The entire work is higly I/O bound, but I hope to break the 10 files per > second barrier soon. > > I'd like tu understang if it make sense to have more processes per single > file. > > To collect trigram I am using the sets module, but I think R17 maps could > be even more fast. > > > --- > Gio's Blog http://gioorgi.com > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Pablo Polvorin ProcessOne -------------- next part -------------- An HTML attachment was scrubbed... URL: