From ok@REDACTED Tue May 1 00:54:45 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 1 May 2012 10:54:45 +1200 Subject: [erlang-questions] Frames proposal In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> Message-ID: <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> On 1/05/2012, at 5:13 AM, Max Lapshin wrote: > Maybe it possible to add JSON notation to frame handling? > > It would be very convenient, because {id : 4135, title : "title2"} is > mapped to [{id, 4135}, {title, "title2"}]. > The only difference is that in all external format handlers binaries > are used to represent external strings. I don't quite understand this question. There is a section in the frames proposal titled "Frames and JSON" which mainly just presents a table JSON Erlang null null false false true true numbers numbers, but JSON has no integers strings binaries arrays lists or tuples dictionaries frames The implied mapping takes JSON {id: 4135, title: "title2"} to <{id ~ 4135, title ~ <<"title2">>}>. The bounded size of the Erlang atom table is a vulnerability but there is an EEP to address that; that in itself is a much more urgent issue than frames. From garret.smith@REDACTED Tue May 1 01:11:59 2012 From: garret.smith@REDACTED (Garret Smith) Date: Mon, 30 Apr 2012 16:11:59 -0700 Subject: [erlang-questions] Debugging apps with dependencies In-Reply-To: <2375676CEBC34724A653F2C69CBDD33C@gmail.com> References: <4BE220475DFC4F5AA43F0637D39ED568@gmail.com> <2375676CEBC34724A653F2C69CBDD33C@gmail.com> Message-ID: Declaring the dependencies in your .app.src file is the most idiomatic way I've seen, and they get started for you by the boot script so you don't have to worry about coding that yourself. See the 'applications' list for the list of applications that should be started before yours. {{modules}} is a rebar macro that will be replaced with the module names from your project. You have to fill in the 'registered' list yourself. example app.src: {application, myapp, [{description, "My Erlang App"}, {vsn, "1.0.0"}, {modules, [{{modules}}]}, {registered, [registered_process_names]}, {applications, [kernel, stdlib, sasl, mnesia, yaws]}, {mod, {app_callback_module, []}} ] }. For the appmon problem, 1st I recommend using 'observer' instead. I just started using it with R15B01 and... wow. The tracing support is phenomenal. Go Erlang devs! To get observer included in your release, add it to your reltool.config like so: {app, observer, [{incl_cond, include}]}, -Garret Smith On Mon, Apr 30, 2012 at 12:00 PM, Josh Black wrote: > Excellent! Thanks Tristan and Bob for the suggestions. I'll try them > out. Is either one considered more idiomatic? > > On Monday, April 30, 2012 at 11:48 AM, Tristan Sloughter wrote: > > Correct about why you can't find appmon, but there is an easy way around > this. Start the release yourself, you probably have to add your libs to > ERL_LIBS first depending how how you have things setup. So like: > > $ export ERL_LIBS=$ERL_LIBS: > $ erl -boot > > This will have your release running in your main erts env, not one created > specifically for your release. > > Someone else can probably give better instructions based on where rebar > places the libs and .boot file, I don't know those. > > Tristan > > On Mon, Apr 30, 2012 at 1:05 PM, Josh Black wrote: > > Hi, erlang beginner here. > > I have a question about the best way to debug apps which include > dependencies. > > I'm working on an app, and so far, I've been testing it by using rebar to > generate a release, then starting a console for the release and going from > there. The problem comes when I want to use something like dbg or appmon > to debug errors or inspect the state of my app. When I try to do, e.g. > appmon:start() from the console for my release, it tells me "undefined > function appmon:start/0". I'm guessing this is because I didn't explicitly > specify appmon in my release configuration? > > Alternatively, I created a start.sh shell script with contents like this: > > erl -pa apps/*/ebin -pa deps/*/ebin -eval 'application:start(dependency1), > application:start(dependency2), application:start(myapp).' > > This lets me start appmon and whatever else I want when I run into errors, > but feels pretty messy and wrong. > > I feel like I'm missing something really obvious here. What's the correct > way to do this? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From janburse@REDACTED Tue May 1 01:13:45 2012 From: janburse@REDACTED (Jan Burse) Date: Tue, 01 May 2012 01:13:45 +0200 Subject: [erlang-questions] Haskell Catching up with Erlang Message-ID: <4F9F1CA9.4090001@fastmail.fm> Scala Days 2012: Simon Peyton Jones on Towards Haskell in the Cloud http://skillsmatter.com/podcast/home/haskell-cloud/js-4179 From amindfv@REDACTED Tue May 1 02:34:12 2012 From: amindfv@REDACTED (Tom Murphy) Date: Mon, 30 Apr 2012 20:34:12 -0400 Subject: [erlang-questions] Haskell Catching up with Erlang In-Reply-To: <4F9F1CA9.4090001@fastmail.fm> References: <4F9F1CA9.4090001@fastmail.fm> Message-ID: Well, Haskell is "catching up," but viewed in another light Erlang and Haskell are helping each other to catch up with imperative languages, by "cross-pollinating" ideas. Haskell taking tools and techniques from Erlang is great for Erlang, and it's great for people who've learned these skills! Don't fear - Haskell is as much of a joy to code in as Erlang :) Tom On 4/30/12, Jan Burse wrote: > Scala Days 2012: Simon Peyton Jones on Towards Haskell in the Cloud > > http://skillsmatter.com/podcast/home/haskell-cloud/js-4179 > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From toby@REDACTED Tue May 1 03:14:58 2012 From: toby@REDACTED (Toby Thain) Date: Mon, 30 Apr 2012 21:14:58 -0400 Subject: [erlang-questions] Haskell Catching up with Erlang In-Reply-To: References: <4F9F1CA9.4090001@fastmail.fm> Message-ID: <4F9F3912.3020902@telegraphics.com.au> On 30/04/12 8:34 PM, Tom Murphy wrote: > Well, Haskell is "catching up," but viewed in another light Erlang and > Haskell are helping each other to catch up with imperative languages, > by "cross-pollinating" ideas. > > Haskell taking tools and techniques from Erlang is great for Erlang, > and it's great for people who've learned these skills! > > Don't fear - Haskell is as much of a joy to code in as Erlang :) > Well said. --Toby > > Tom > > On 4/30/12, Jan Burse wrote: >> Scala Days 2012: Simon Peyton Jones on Towards Haskell in the Cloud >> >> http://skillsmatter.com/podcast/home/haskell-cloud/js-4179 >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From max.lapshin@REDACTED Tue May 1 05:38:10 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 1 May 2012 07:38:10 +0400 Subject: [erlang-questions] Frames proposal In-Reply-To: <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> Message-ID: I mean using JSON directly inside Erlang: function({type : <<"article">>, title : Title, id : Id} = Object) -> Object{id : make_permalink(Id, Title)}. or Article = {id : 523, title : proplists:get_value(<<"title">>, Params)} I mean this. Your syntax may have some historical roots, but they are too ancient. Nowadays such syntax <{key ~ value>} look like inventing bicycle with square wheels. From ok@REDACTED Tue May 1 07:27:20 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 1 May 2012 17:27:20 +1200 Subject: [erlang-questions] Frames proposal In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> Message-ID: <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> On 1/05/2012, at 3:38 PM, Max Lapshin wrote: > I mean using JSON directly inside Erlang: > > > function({type : <<"article">>, title : Title, id : Id} = Object) -> > Object{id : make_permalink(Id, Title)}. > > or > > Article = {id : 523, title : proplists:get_value(<<"title">>, Params)} > > I mean this. Your syntax may have some historical roots, but they are > too ancient. Nowadays such syntax <{key ~ value>} look like inventing > bicycle with square wheels. JSON syntax is ***Javascript*** syntax. The frames proposal has always made it very clear why we cannot copy JSON syntax. {} is already an empty TUPLE, it cannot also be an empty 'dictionary'. We cannot reasonably use unadorned curly braces for frames. {a:f()} already means a tuple whose one element is the value of a call to the f() function in module a. We *CANNOT* even unreasonably use a colon in maplets; it is just not going to work. Have a lollipop. From max.lapshin@REDACTED Tue May 1 07:43:47 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 1 May 2012 09:43:47 +0400 Subject: [erlang-questions] Frames proposal In-Reply-To: <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> Message-ID: On Tue, May 1, 2012 at 9:27 AM, Richard O'Keefe wrote: > > {} ? ? ?is already an empty TUPLE, > {a:f()} already means a tuple whose one element > ? ? ? ?is the value of a call to the f() function Yes, I see. It is a pity =( From michael.eugene.turner@REDACTED Tue May 1 07:58:52 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Tue, 1 May 2012 14:58:52 +0900 Subject: [erlang-questions] Frames proposal In-Reply-To: <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> Message-ID: Max Lapshin: "Your syntax may have some historical roots, but they are too ancient." "Ancient" always trumps "unworkable." Max again: "Nowadays such syntax <{key ~ value>} look like inventing bicycle with square wheels." Bzzt. Analogy fail. Whatever a bicycle with square wheels might *look* like http://3.bp.blogspot.com/_uhH7HeUdvmw/TB7l0F7aGCI/AAAAAAAAACU/lx6z8b6g-XM/s1600/square_wheels.jpg we can agree that such a bicycle couldn't work -- well, except perhaps in a special environment http://www.math.hmc.edu/funfacts/figures/10001.2-3-8.1.gif in which square wheels are actually ideal. If there's any parallel to the ideal-square-wheel special environment, here, it's existing Erlang syntax, which is so weird and annoying that even some ardent fans of the language can't refrain from open criticism. Richard is looking for what would be the "smoothest ride" given what can't be changed. I think he's made the best of some not-very-satisfactory choices. And it's not like he and I exactly agree on syntax issues (but let's not go there again. Ever.) Richard: "Have a lollipop." Curmudgeon points for that one. This proposal is almost 10 years old. Is it really so radical that we couldn't have it in beta within three months? -michael turner On Tue, May 1, 2012 at 2:27 PM, Richard O'Keefe wrote: > > On 1/05/2012, at 3:38 PM, Max Lapshin wrote: > >> I mean using JSON directly inside Erlang: >> >> >> function({type : <<"article">>, title : Title, id : Id} = Object) -> >> ? Object{id : make_permalink(Id, Title)}. >> >> or >> >> ?Article = {id : 523, title : proplists:get_value(<<"title">>, Params)} >> >> I mean this. Your syntax may have some historical roots, but they are >> too ancient. Nowadays such syntax <{key ~ value>} look like inventing >> bicycle with square wheels. > > JSON syntax is ***Javascript*** syntax. > > The frames proposal has always made it very clear why we cannot > copy JSON syntax. > > {} ? ? ?is already an empty TUPLE, > ? ? ? ?it cannot also be an empty 'dictionary'. > ? ? ? ?We cannot reasonably use unadorned curly > ? ? ? ?braces for frames. > > {a:f()} already means a tuple whose one element > ? ? ? ?is the value of a call to the f() function > ? ? ? ?in module a. ?We *CANNOT* even unreasonably > ? ? ? ?use a colon in maplets; it is just not going > ? ? ? ?to work. > > Have a lollipop. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jose.valim@REDACTED Tue May 1 09:37:47 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Tue, 1 May 2012 09:37:47 +0200 Subject: [erlang-questions] Frames proposal In-Reply-To: <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> Message-ID: > > The implied mapping takes JSON {id: 4135, title: "title2"} > to <{id ~ 4135, title ~ <<"title2">>}>. > > The bounded size of the Erlang atom table is a vulnerability > but there is an EEP to address that; that in itself is a much > more urgent issue than frames. > Exactly. In Ruby, for example, since atoms aren't garbage collected, converting a JSON from an external source to a hash using atoms as keys represents a security vulnerability in a web service, as someone could force the "atom table" to fill in completely, so we simply don't. So until the atom limitation is fixed, we would be better on handling JSONs as a dict or something else. I have read the proposal completely and I think everything is well explained and defined. Even though I am not a huge fan of the syntax, I think it fits Erlang well. One option that I haven't seen considered is still using curly brackets as delimiters and use `{~}` to specify an empty frame. This would make the common case (a frame with at least one element) easier on the eyes by sacrificing a bit the not so common case (empty frame). But again, we are in good hands with whatever the Erlang and OTP team decide. -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Tue May 1 10:21:14 2012 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Tue, 1 May 2012 10:21:14 +0200 Subject: [erlang-questions] Can't load crypto module after installing R15 in Solaris 10 x86 In-Reply-To: References: Message-ID: 2012/4/30 Erisa Dervishi : [...] > OpenSSL is installed. As suggested in the Erlang documentation, I installed > it from?http://www.openssl.org/source/ (version:?openssl-0.9.8w.tar.gz) > I tried several suggestions after googling but nothing helped. > Any help to get this straight would be appreciated :) Have you tried the precompiled openssl package from http://www.sunfreeware.com/intel/10/openssl-0.9.8s-sol10-x86-local.gz (this needs libgcc too)? From watson.timothy@REDACTED Tue May 1 10:42:47 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 1 May 2012 09:42:47 +0100 Subject: [erlang-questions] Debugging apps with dependencies In-Reply-To: References: <4BE220475DFC4F5AA43F0637D39ED568@gmail.com> <2375676CEBC34724A653F2C69CBDD33C@gmail.com> Message-ID: <904061C5-2369-42E5-99A4-D938620F0ED7@gmail.com> You might also want to look at github.com/hyperthunk/appstart which is intended for development environment not as alternative to releases. On 1 May 2012, at 00:11, Garret Smith wrote: > Declaring the dependencies in your .app.src file is the most idiomatic way I've seen, and they get started for you by the boot script so you don't have to worry about coding that yourself. See the 'applications' list for the list of applications that should be started before yours. > > {{modules}} is a rebar macro that will be replaced with the module names from your project. > You have to fill in the 'registered' list yourself. > > example app.src: > {application, myapp, > [{description, "My Erlang App"}, > {vsn, "1.0.0"}, > {modules, [{{modules}}]}, > {registered, [registered_process_names]}, > {applications, [kernel, stdlib, sasl, mnesia, yaws]}, > {mod, {app_callback_module, []}} > ] > }. > > For the appmon problem, 1st I recommend using 'observer' instead. I just started using it with R15B01 and... wow. The tracing support is phenomenal. Go Erlang devs! > > To get observer included in your release, add it to your reltool.config like so: > > {app, observer, [{incl_cond, include}]}, > > -Garret Smith > > On Mon, Apr 30, 2012 at 12:00 PM, Josh Black wrote: > Excellent! Thanks Tristan and Bob for the suggestions. I'll try them out. Is either one considered more idiomatic? > On Monday, April 30, 2012 at 11:48 AM, Tristan Sloughter wrote: > >> Correct about why you can't find appmon, but there is an easy way around this. Start the release yourself, you probably have to add your libs to ERL_LIBS first depending how how you have things setup. So like: >> >> $ export ERL_LIBS=$ERL_LIBS: >> $ erl -boot >> >> This will have your release running in your main erts env, not one created specifically for your release. >> >> Someone else can probably give better instructions based on where rebar places the libs and .boot file, I don't know those. >> >> Tristan >> >> On Mon, Apr 30, 2012 at 1:05 PM, Josh Black wrote: >>> Hi, erlang beginner here. >>> >>> I have a question about the best way to debug apps which include dependencies. >>> >>> I'm working on an app, and so far, I've been testing it by using rebar to generate a release, then starting a console for the release and going from there. The problem comes when I want to use something like dbg or appmon to debug errors or inspect the state of my app. When I try to do, e.g. appmon:start() from the console for my release, it tells me "undefined function appmon:start/0". I'm guessing this is because I didn't explicitly specify appmon in my release configuration? >>> >>> Alternatively, I created a start.sh shell script with contents like this: >>> >>> erl -pa apps/*/ebin -pa deps/*/ebin -eval 'application:start(dependency1), application:start(dependency2), application:start(myapp).' >>> >>> This lets me start appmon and whatever else I want when I run into errors, but feels pretty messy and wrong. >>> >>> I feel like I'm missing something really obvious here. What's the correct way to do this? >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From laforge@REDACTED Tue May 1 10:56:21 2012 From: laforge@REDACTED (Harald Welte) Date: Tue, 1 May 2012 10:56:21 +0200 Subject: [erlang-questions] Is there an accepted way of including GPL code in a project? In-Reply-To: References: <14C428C7-89A6-4099-BEA5-B419C776372B@feuerlabs.com> <4F882F3C.6000806@gmail.com> Message-ID: <20120501085621.GW15890@prithivi.gnumonks.org> Hi Alan, apologies for my late response, I don't read the erlang-questions list very frequently, although I'm subscribed to it. As some of you may know, besides writing lots of osmocom.org code I'm also involved in legal aspects of Free Software for many years (including GPL enforcement at gpl-violations.org). As such I also have the pleasure of meeting with lots and lots international legal experts around Free Software licensing. The question is not really one of licenses, but whether or not OTP libraries and 'user applications' will form a 'derivative work' under copyright law. This is not easy to answer, particularly as most of the existing work in this area has been focussing on classinc "C/C++" style development, which is quite a bit different from what you can see in Erlang. I cannot (and don't want to) go into all the detailed arguments here, but suffice it to say that at least in some jurisdictions you will inevitably find some legal experts that consider that using a OTP library from your own code creates a derivative work. In such a case, your own code cannot be licensed under GPLv2/GPLv3/AGPLv3, as the GPL conditions extend to the entire derivative work, including the OTP libraries. As those are under the incompatible Erlang Public License, tihs won't work. The solutoing (for GPLv3 / AGPLv3) is to add an "additional permissive clause" to your own work, stating that _if_ the combination of the Erlang/OTP libraries and your own program form such a derivative work, then you explicitly give permsision to convey/distribute those Erlang/OTP libraries under their license. I've done that for the osmocom.org Erlang code, as you can see at http://cgit.osmocom.org/cgit/erlang/osmo_ss7/diff/src/m3ua_codec.erl?id=f8bf03231c8d708a1fd43dc030df7ce6ea4eabd9 % Additional Permission under GNU AGPL version 3 section 7: % % If you modify this Program, or any covered work, by linking or % combining it with runtime libraries of Erlang/OTP as released by % Ericsson on http://www.erlang.org (or a modified version of these % libraries), containing parts covered by the terms of the Erlang Public % License (http://www.erlang.org/EPLICENSE), the licensors of this % Program grant you additional permission to convey the resulting work % without the need to license the runtime libraries of Erlang/OTP under % the GNU Affero General Public License. Corresponding Source for a % non-source form of such a combination shall include the source code % for the parts of the runtime libraries of Erlang/OTP used as well as % that of the covered work. So my recommendation for anyone doing GPLv3 / AGPLv3 development would be to use a similar clause just to be on the safe side. If there is no derivative work between the OTP library and your code, then you don't have to worry about license compatibility at all. Regards, Harald -- - Harald Welte http://laforge.gnumonks.org/ ============================================================================ "Privacy in residential applications is a desirable marketing option." (ETSI EN 300 175-7 Ch. A6) From lukas@REDACTED Tue May 1 11:28:59 2012 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 1 May 2012 11:28:59 +0200 Subject: [erlang-questions] ei for Apple's iOS? In-Reply-To: <8FCACD73-8E5C-4DF1-979A-DE816BF88155@bignerdranch.com> References: <8FCACD73-8E5C-4DF1-979A-DE816BF88155@bignerdranch.com> Message-ID: Hi, You seem to not have exported ERL_TOP before running make. Try export ERL_TOP=`pwd` && make Lukas On Mon, Apr 30, 2012 at 9:38 PM, Aaron Hillegass wrote: > I'm writing a client that talks to an Erlang server. ? The client runs on iOS. > > I was thinking it would be nice to use the ei_encode* functions to make the data passing back and forth more Erlang-friendly. > > My question: Has anyone compiled ei as a static library for iOS applications? > > I have tried: downloading the source to a Mac with the iOS SDK, cd to the erl_interface directory, configure, and make. ?The output from that is attached. ?Any suggestions are appreciated. > > -- > > Aaron Hillegass > Big Nerd Ranch, Inc. > > $ ./configure CPP="cpp" CXXCPP="cpp" CXX=/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/arm-apple-darwin10-llvm-g++-4.2 CC=/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/arm-apple-darwin10-llvm-gcc-4.2 LD=/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/ld CFLAGS="-O -isysroot /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS5.0.sdk" CXXFLAGS="-O -isysroot /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS5.0.sdk" --disable-dependency-tracking --host=arm-apple-darwin10 LDFLAGS="-isysroot /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS5.0.sdk" AR=/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/ar AS=/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/as LIBTOOL=/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/libtool STRIP=/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/strip RANLIB=/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/ranlib > configure: WARNING: If you wanted to set the --build type, don't use --host. > ? ?If a cross compiler is detected then cross compile mode will be used. > checking build system type... i386-apple-darwin11.3.0 > checking host system type... arm-apple-darwin10 > checking for arm-apple-darwin10-gcc... /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/arm-apple-darwin10-llvm-gcc-4.2 > checking for C compiler default output file name... a.out > checking whether the C compiler works... yes > checking whether we are cross compiling... yes > checking for suffix of executables... > checking for suffix of object files... o > checking whether we are using the GNU C compiler... yes > checking whether /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/arm-apple-darwin10-llvm-gcc-4.2 accepts -g... yes > checking for /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/arm-apple-darwin10-llvm-gcc-4.2 option to accept ANSI C... none needed > checking how to run the C preprocessor... cpp > checking for arm-apple-darwin10-ranlib... /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/ranlib > checking for ld.sh... /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/ld > checking for arm-apple-darwin10-ld... (cached) /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/ld > checking for egrep... grep -E > checking for ANSI C header files... yes > checking for sys/types.h... yes > checking for sys/stat.h... yes > checking for stdlib.h... yes > checking for string.h... yes > checking for memory.h... yes > checking for strings.h... yes > checking for inttypes.h... yes > checking for stdint.h... yes > checking for unistd.h... yes > checking for short... yes > checking size of short... 2 > checking for int... yes > checking size of int... 4 > checking for long... yes > checking size of long... 4 > checking for void *... yes > checking size of void *... 4 > checking for long long... yes > checking size of long long... 8 > checking for arm-apple-darwin10-ar... /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/ar > checking for a BSD-compatible install... /usr/bin/install -c > checking how to create a directory including parents... /usr/bin/install -c -d > checking for gethostbyname in -lnsl... no > checking for getpeername in -lsocket... no > checking for ANSI C header files... (cached) yes > checking for sys/wait.h that is POSIX.1 compatible... yes > checking arpa/inet.h usability... yes > checking arpa/inet.h presence... yes > checking for arpa/inet.h... yes > checking fcntl.h usability... yes > checking fcntl.h presence... yes > checking for fcntl.h... yes > checking limits.h usability... yes > checking limits.h presence... yes > checking for limits.h... yes > checking malloc.h usability... no > checking malloc.h presence... no > checking for malloc.h... no > checking netdb.h usability... yes > checking netdb.h presence... yes > checking for netdb.h... yes > checking netinet/in.h usability... yes > checking netinet/in.h presence... yes > checking for netinet/in.h... yes > checking stddef.h usability... yes > checking stddef.h presence... yes > checking for stddef.h... yes > checking for stdlib.h... (cached) yes > checking for string.h... (cached) yes > checking sys/param.h usability... yes > checking sys/param.h presence... yes > checking for sys/param.h... yes > checking sys/socket.h usability... yes > checking sys/socket.h presence... yes > checking for sys/socket.h... yes > checking sys/select.h usability... yes > checking sys/select.h presence... yes > checking for sys/select.h... yes > checking sys/time.h usability... yes > checking sys/time.h presence... yes > checking for sys/time.h... yes > checking for unistd.h... (cached) yes > checking for sys/types.h... (cached) yes > checking for uid_t in sys/types.h... yes > checking for pid_t... yes > checking for size_t... yes > checking whether time.h and sys/time.h may both be included... yes > checking for socklen_t usability... yes > checking for working alloca.h... yes > checking for alloca... yes > checking whether /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/arm-apple-darwin10-llvm-gcc-4.2 needs -traditional... no > checking for working memcmp... no > checking for dup2... yes > checking for gethostbyaddr... yes > checking for gethostbyname... yes > checking for gethostbyaddr_r... no > checking for gethostbyname_r... no > checking for gethostname... yes > checking for writev... yes > checking for gethrtime... no > checking for gettimeofday... yes > checking for inet_ntoa... yes > checking for memchr... yes > checking for memmove... yes > checking for memset... yes > checking for select... yes > checking for socket... yes > checking for strchr... yes > checking for strerror... yes > checking for strrchr... yes > checking for strstr... yes > checking for uname... yes > checking for res_gethostbyname... no > checking for res_gethostbyname in -lresolv... no > checking for clock_gettime... no > checking for clock_gettime in -lrt... no > checking for mixed cygwin or msys and native VC++ environment... no > checking for mixed cygwin and native MinGW environment... no > checking if we mix cygwin with any native compiler... no > checking if we mix msys with another native compiler... no > checking for native win32 threads... no > checking for pthread_create in -lpthread... yes > checking pthread.h usability... yes > checking pthread.h presence... yes > checking for pthread.h... yes > checking pthread/mit/pthread.h usability... no > checking pthread/mit/pthread.h presence... no > checking for pthread/mit/pthread.h... no > configure: creating ./config.status > config.status: creating src/arm-apple-darwin10/Makefile > config.status: creating src/arm-apple-darwin10/eidefs.mk > config.status: creating src/arm-apple-darwin10/config.h > config.status: src/arm-apple-darwin10/config.h is unchanged > > $ make > Makefile:19: /make/target.mk: No such file or directory > Makefile:20: /make//otp.mk: No such file or directory > Makefile:32: /make/otp_subdir.mk: No such file or directory > make: *** No rule to make target `/make/otp_subdir.mk'. ?Stop. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From lukas@REDACTED Tue May 1 11:56:15 2012 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 1 May 2012 11:56:15 +0200 Subject: [erlang-questions] EUnit error test: limit number of processes at runtime In-Reply-To: References: <20120428170626.GA2692@achilles.local.defunct.ch> <4F9C24A9.9070807@cs.ntua.gr> <20120428171903.GC2692@achilles.local.defunct.ch> Message-ID: Hi, You could also spawn a slave[1] node in the test with your desired parameters to test this behaviour. That is how it is done in the Erlang/OTP test suites. Lukas [1] : http://erlang.org/doc/man/slave.html On Sat, Apr 28, 2012 at 9:24 PM, Geoff Cant wrote: > From memory there's no way to limit the number of processes allowed aside from the size of the process table. The size of the process table is set at startup and can't be changed, so I guess the answer to your question is No - you have to spawn enough processes to run out. spawn(fun () -> erlang:hibernate(erlang, exit, [normal]) end, []) might help with the memory footprint a bit. > > Cheers, > -Geoff > > On 2012-04-28, at 10:19 , Christian Kruse wrote: > >> Hi, >> >> On 28/04/12 19:11, Kostis Sagonas wrote: >>>> for an Erlang project I'm working on (Open Source, >>>> ?in case someone is curious) I'm >>>> trying to test a possible error with spawning a process, e.g. maximum >>>> number of processes reached. Since I don't want to spawn> ?32k >>>> processes for each test run, I'd like to limit the number of possible >>>> processes for this test only. >>>> >>>> Is this possible? Or is there another solution I can use to test such >>>> an error case? >>> >>> Have you considered using the +P option of erl? >> >> Yes, but since I want to reduce this limit only for one specific test >> it is not really a good solution. I would have to set the limit to a >> specific number of processes for all tests and exceed that limit in >> the error test. Of course it can be much lower than 32k processes for >> the test run, but it would be much nicer to limit it only for this one >> test. >> >> Greetings, >> CK >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -- > Geoff Cant > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From andrewt1972@REDACTED Tue May 1 13:39:09 2012 From: andrewt1972@REDACTED (Andrew Thompson) Date: Tue, 1 May 2012 21:39:09 +1000 Subject: [erlang-questions] trapexit.org Message-ID: <-542295322088726156@unknownmsgid> Sorry all I know this isn't the right forum but ... does anyone know how to let these guys know they have a problem? Been like this for a couple of days. phpBB : *Critical Error* Could not connect to the database -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc@REDACTED Tue May 1 13:48:51 2012 From: marc@REDACTED (Marc Worrell) Date: Tue, 1 May 2012 13:48:51 +0200 Subject: [erlang-questions] Erlang Talent Community Message-ID: Maximonster ?we are the people behind Totally Erlang, Zotonic, MaxClass and other Erlang things ? is developing a Erlang Talent Community. If you are interested in becoming part of it, please signup now: http://www.maximonster.com/erlang-talent Regards, Maximonster Interactive Things Michiel Kl?nhammer, Marc Worrell and Arjan Scherpenisse. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Tue May 1 16:28:27 2012 From: bourinov@REDACTED (Max Bourinov) Date: Tue, 1 May 2012 18:28:27 +0400 Subject: [erlang-questions] Frames proposal In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> Message-ID: Hi Richard, Thank you very much for posting your great work! But the title scared me. "Getting rid of records" - what should I do with all my code that uses records? I like the idea of frames a lot. It would significantly simplify my code and I love this idea. Best regards, Max On Tue, May 1, 2012 at 11:37 AM, Jos? Valim wrote: > The implied mapping takes JSON {id: 4135, title: "title2"} >> to <{id ~ 4135, title ~ <<"title2">>}>. >> >> The bounded size of the Erlang atom table is a vulnerability >> but there is an EEP to address that; that in itself is a much >> more urgent issue than frames. >> > > Exactly. In Ruby, for example, since atoms aren't garbage > collected, converting a JSON from an external source to a hash using atoms > as keys represents a security vulnerability in a web service, as someone > could force the "atom table" to fill in completely, so we simply don't. > > So until the atom limitation is fixed, we would be better on handling > JSONs as a dict or something else. > > I have read the proposal completely and I think everything is well > explained and defined. Even though I am not a huge fan of the syntax, I > think it fits Erlang well. One option that I haven't seen considered is > still using curly brackets as delimiters and use `{~}` to specify an empty > frame. This would make the common case (a frame with at least one element) > easier on the eyes by sacrificing a bit the not so common case (empty > frame). But again, we are in good hands with whatever the Erlang and OTP > team decide. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serge@REDACTED Tue May 1 16:30:57 2012 From: serge@REDACTED (Serge Aleynikov) Date: Tue, 01 May 2012 10:30:57 -0400 Subject: [erlang-questions] Float multiplication behaviours In-Reply-To: <20120424103437.GB42675@netch.kiev.ua> References: <2dca6d7b-76fc-436b-ae10-35f4d20c1a38@knuth> <14d07c6c-d645-47d7-b124-0454a938cb56@knuth> <20120424103437.GB42675@netch.kiev.ua> Message-ID: <4F9FF3A1.9000709@aleynikov.org> FWIW, if the issue is displaying the property rounded value of the float, the float_to_list/2 patch that I submitted awhile back, rounds and displays this property (hopefully it'll make it in the release soon): 8> io:fwrite("~p~n", [10.20*100]). 1019.9999999999999 9> float_to_list(10.20*100). "1.01999999999999988631e+03" 10> float_to_list(10.20*100, []). "1020.000000" 11> float_to_list(10.20*100, [compact]). "1020.0" On 4/24/2012 6:34 AM, Valentin Nechayev wrote: > Mon, Apr 23, 2012 at 13:46:14, vinoski wrote about "Re: [erlang-questions] Float multiplication behaviours": > >>> (node@REDACTED)49> 10.20*100. >>> 1019.9999999999999 >> http://floating-point-gui.de/ > > The question is more complex if to compare with different language > system behaviors and Erlang behavior difference between output formats. > For example, Python (2.7) shows exactly the same value as Erlang in the > quote, but C with Glibc (glibc-2.14.1-14.27.1.x86_64@REDACTED), Perl > and Ruby's irb shows simply 1020. The latter one shows rounding which > could be unreasonable, because doesn't differ adjacent values: > > $ cat tx1.c > #include > #include > main() { > double a = 10.20; > a *= 100; > double ap, an; > ap = nextafter(a, 1000.0); > an = nextafter(a, 0.0); > printf("%g%s%g%s%g\n", > an, (an == a) ? "==" : "!=", > a, (a == ap) ? "==" : "!=", ap); > return 0; > } > $ ./tx1 > 1020!=1020!=1020 > > If the thread starter expects the value to be rounded in accustomed way, this > can help in its reading, but can also give confusing results on values which > are looking equal but differs really. Erlang manner to print with full details > with ~p is more debug-oriented than user-oriented: > > 2> io:fwrite("~p~n", [10.20*100]). > 1019.9999999999999 > ok > 3> io:fwrite("~g~n", [10.20*100]). > 1020.00 > ok > > (Sadly, math module is quite thin and doesn't carry IEEE required > functions, as nextafter().) > > For more floating-point details, it's better to go to > > and start reading from the previous topics. > > > -netch- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An embedded message was scrubbed... From: Serge Aleynikov Subject: Re: [erlang-patches] [patch] new float_to_list/2 Date: Sat, 21 Apr 2012 01:19:04 -0400 Size: 12863 URL: From michael.eugene.turner@REDACTED Tue May 1 16:36:56 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Tue, 1 May 2012 23:36:56 +0900 Subject: [erlang-questions] Frames proposal In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> Message-ID: > But the title scared me. "Getting rid of records" - what should I do with > all my code that uses records? EEP stands for Erlang Extension Proposal, not Erlang Excision Proposal. -michael turner On Tue, May 1, 2012 at 11:28 PM, Max Bourinov wrote: > Hi Richard, > > Thank you very much for posting your great work! > > But the title scared me. "Getting rid of records" - what should I do with > all my code that uses records? > > I like the idea of frames a lot. It would significantly simplify my code and > I love this idea. > > > Best regards, > Max > > > > > On Tue, May 1, 2012 at 11:37 AM, Jos? Valim wrote: >>> >>> The implied mapping takes JSON {id: 4135, title: "title2"} >>> to <{id ~ 4135, title ~ <<"title2">>}>. >>> >>> The bounded size of the Erlang atom table is a vulnerability >>> but there is an EEP to address that; that in itself is a much >>> more urgent issue than frames. >> >> >> Exactly. In Ruby, for example, since atoms aren't garbage >> collected,?converting a JSON from an external source to a hash using atoms >> as keys represents a security vulnerability in a web service, as someone >> could force the "atom table" to fill in completely, so we simply don't. >> >> So until the atom limitation is fixed, we would be better on handling >> JSONs as a dict or something else. >> >> I have read the proposal completely and I think everything is well >> explained and defined. Even though I am not a huge fan of the syntax, I >> think it fits Erlang well. One option that I haven't seen considered is >> still using curly brackets as delimiters and use `{~}` to specify an empty >> frame. This would make the common case (a frame with at least one element) >> easier on the eyes by sacrificing a bit the not so common case (empty >> frame). But again, we are in good hands with whatever the Erlang and OTP >> team decide. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From juanjo@REDACTED Tue May 1 17:48:44 2012 From: juanjo@REDACTED (Juan Jose Comellas) Date: Tue, 1 May 2012 12:48:44 -0300 Subject: [erlang-questions] Frames proposal In-Reply-To: <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> Message-ID: And what about borrowing the syntax that Anton Lavrik used for erlson [1]? That syntax looks similar to the one used for records and is much easier to read/write. [1] https://github.com/alavrik/erlson On Tue, May 1, 2012 at 2:27 AM, Richard O'Keefe wrote: > > On 1/05/2012, at 3:38 PM, Max Lapshin wrote: > > > I mean using JSON directly inside Erlang: > > > > > > function({type : <<"article">>, title : Title, id : Id} = Object) -> > > Object{id : make_permalink(Id, Title)}. > > > > or > > > > Article = {id : 523, title : proplists:get_value(<<"title">>, Params)} > > > > I mean this. Your syntax may have some historical roots, but they are > > too ancient. Nowadays such syntax <{key ~ value>} look like inventing > > bicycle with square wheels. > > JSON syntax is ***Javascript*** syntax. > > The frames proposal has always made it very clear why we cannot > copy JSON syntax. > > {} is already an empty TUPLE, > it cannot also be an empty 'dictionary'. > We cannot reasonably use unadorned curly > braces for frames. > > {a:f()} already means a tuple whose one element > is the value of a call to the f() function > in module a. We *CANNOT* even unreasonably > use a colon in maplets; it is just not going > to work. > > Have a lollipop. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue May 1 18:00:57 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 1 May 2012 20:00:57 +0400 Subject: [erlang-questions] Frames proposal In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> Message-ID: I really dream about plain and simple 0 = D1.baz.fum.i, =) From wallentin.dahlberg@REDACTED Tue May 1 18:28:26 2012 From: wallentin.dahlberg@REDACTED (=?ISO-8859-1?Q?Bj=F6rn=2DEgil_Dahlberg?=) Date: Tue, 1 May 2012 18:28:26 +0200 Subject: [erlang-questions] ei for Apple's iOS? In-Reply-To: References: <8FCACD73-8E5C-4DF1-979A-DE816BF88155@bignerdranch.com> Message-ID: 2012/4/30 Jared Kofron > +1 for JSON - Communication between our Erlang nodes and Obj-C is done > entirely via JSON and it works very nicely. > On Mon, Apr 30, 2012 at 1:42 PM, Jack Moffitt wrote: > >> > I was thinking it would be nice to use the ei_encode* functions to make >> the data passing back and forth more Erlang-friendly. >> >> This isn't directly your question, but I did an iOS app that talks to >> an Erlang server and used JSON. I've also done this with another app >> with MsgPack, and both worked great. >> > +1 - I have also done iOS <-> Erlang Server communication with JSON to great success. We used record <-> json encodings on the server-side since records were our data-model (with mnesia). Do not use ei and force a term-format on your Obj-C code. It's not nice and a bad fit. // Bj?rn-Egil -------------- next part -------------- An HTML attachment was scrubbed... URL: From dan@REDACTED Tue May 1 18:39:12 2012 From: dan@REDACTED (Daniel Dormont) Date: Tue, 1 May 2012 12:39:12 -0400 Subject: [erlang-questions] Two questions about open files and file descriptors Message-ID: Hello Erlangers, I'm trying to chase down some 'emfile' errors that showed up in my application. I have two questions about this: 1) Is there a way I can see, in a live system, supposing I have a console open, the list of open files and/or file descriptors? 2) Suppose I have code running in a particular process that calls file:open/2, writes some stuff using io:format/3, and then closes the file using file:close/1. During this time, the descriptor returned by file:open is kept only in local variables. Now, in the middle of computing "some stuff", the process crashes. What happens to my open file? -Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue May 1 18:46:26 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 1 May 2012 20:46:26 +0400 Subject: [erlang-questions] Two questions about open files and file descriptors In-Reply-To: References: Message-ID: lsof -p ulimit [{Pid, Ports} || {Pid,Ports} <- [{Pid, [L || L <- element(2,process_info(Pid, links)), is_port(L)]} || Pid <- processes()], length(Ports) > 0] From wallentin.dahlberg@REDACTED Tue May 1 19:19:03 2012 From: wallentin.dahlberg@REDACTED (=?ISO-8859-1?Q?Bj=F6rn=2DEgil_Dahlberg?=) Date: Tue, 1 May 2012 19:19:03 +0200 Subject: [erlang-questions] Frames proposal In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> Message-ID: 2012/5/1 Juan Jose Comellas > And what about borrowing the syntax that Anton Lavrik used for erlson [1]? > That syntax looks similar to the one used for records and is much easier to > read/write. > > [1] https://github.com/alavrik/erlson > Syntax is more than just finding which characters our parser can identify, like in the frames proposals where ~ is our separator. When I read <{ k ~ value | Is }> it says cons "k is like a value" to Is. *brr* At first I didn't like the syntax at all, but after playing around with it for a couple of hours it kind of grows on you. #{ } vs <{ }>, the second one only uses a single character more. I think we could live with that one. I still don't like ~ .. and I definitely don't like lollipops. Lavriks syntax proposal is very clean and seems like an obvious fit to the current Erlang syntax. However, if I read #{ k = value }, and I know how R#{k = value } works, it feels like the two should have some sort of connection. They would't have one. It is entirely possible we are fooling ourselves with a similar syntax and would be better off with a completely different syntax. Syntax is perhaps the most fun to discuss since it is our interface. It is important to get it right since it hard to change it once implemented. But, It is of minor concern to me. What guarantees should we leave? The underlying implementation of a Hashmap has enormous consequences on the guarantees we can promise. Speed, memory consumption and functionality are all trade offs. From a memory / speed perspective I really like a HAMT-implementation. Very compact, simple garbage collector, persistent and all around nice properties (except ordering). Doing something more fancy complicates the gc which is not unsolvable but irritating. Richard, is the "beam"-implementation for frames around to look at? // Bj?rn-Egil -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremy.hood@REDACTED Tue May 1 19:28:08 2012 From: jeremy.hood@REDACTED (Jeremy Hood) Date: Tue, 1 May 2012 13:28:08 -0400 Subject: [erlang-questions] Global Name Registry Hang Message-ID: <1335893288.27393.425.camel@H123456.site> Customer ID: eid63330888064 There was a hang in the global name registration that we experienced a while back. When I looked it up online at the time, there was a website that identified the problem and had a workaround for it. It was run by Erlang-Solutions(.com), but the web page I saw before is not there any more (http://www.trapexit.org/forum/viewtopic.php?p=47938). It said to look up the system's global locks: [{ResourceId, LockReqId, PidRefs}] = ets:lookup(global_locks, global). [X,Y] = LockReqId. node(X). node(Y). And then to remove it. That got the Erlang VM to work again, but we never figured out what caused it to hang up in the first place. Since then, we upgraded to the latest version of Erlang and did not worry about this problem until it just happened again the other day with Erlang version R15B. When I attached to the Erlang VM and issued the command "global:sync()", it hung for a very long time before I finally killed the VM. Are you guys aware of any such problem? Are we doing something wrong that is causing this problem? Thanks for the help. Jeremy -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Tue May 1 19:29:10 2012 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 1 May 2012 19:29:10 +0200 Subject: [erlang-questions] Frames proposal In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> Message-ID: On 1 May 2012, at 19:19, Bj?rn-Egil Dahlberg wrote: > It is entirely possible we are fooling ourselves with a similar syntax and would be better off with a completely different syntax. I'm in the camp of - similar syntax indicates similar semantics - different syntax indicates different semantics ?or something to that effect. Most importantly, I definitely agree that borrowing a syntax just because it's well-established, is a very bad idea if your intention is to give it different semantics from what one would expect. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From dan@REDACTED Tue May 1 19:40:12 2012 From: dan@REDACTED (Daniel Dormont) Date: Tue, 1 May 2012 13:40:12 -0400 Subject: [erlang-questions] Two questions about open files and file descriptors In-Reply-To: References: Message-ID: Thanks. Will that include ports open from processes that have died? That's the specific case I'm trying to understand - what happens if a process crashes with a file open, and how do I recover from it? -Dan On Tue, May 1, 2012 at 12:46 PM, Max Lapshin wrote: > lsof -p > ulimit > > [{Pid, Ports} || {Pid,Ports} <- [{Pid, [L || L <- > element(2,process_info(Pid, links)), is_port(L)]} || Pid <- > processes()], length(Ports) > 0] > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc@REDACTED Tue May 1 20:12:42 2012 From: marc@REDACTED (Marc Worrell) Date: Tue, 1 May 2012 20:12:42 +0200 Subject: [erlang-questions] Travis-CI - lager_transform undef error Message-ID: <5AA18C1E-08EA-43F8-9874-ADA167307240@worrell.nl> Hi, I am trying to setup some continuous integration with Travis-CI.org for Zotonic. I am running into this error: http://travis-ci.org/#!/zotonic/zotonic/jobs/1220673 erlc -o src/erlydtl src/erlydtl/erlydtl_parser.yrl 202Recompile: src/zotonic_sup 203src/zotonic_sup.erl:none: error in parse transform 'lager_transform': {undef, 204 [{lager_transform, With the following travis definition: language: erlang otp_release: - R15B01 - R14B04 - R14B03 script: make before_install: git submodule update --init --recursive env: - DB=postgres branches: only: - master Does anyone have any idea how to set the correct paths so that the compiler can find the lager_transform in deps/lager? - Marc -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Tue May 1 20:15:42 2012 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 1 May 2012 20:15:42 +0200 Subject: [erlang-questions] Two questions about open files and file descriptors In-Reply-To: References: Message-ID: When the process controlling a port dies the port will get what is equivalent to an exit signal and be stopped. For the specific case of the file driver, the driver will try to flush all written data before shutting down the driver, but eventually it should end with a closed filedescriptor. Note that if you are not opening the file with raw as arguments, it will be the file server which is the controlling process. Lukas On Tue, May 1, 2012 at 7:40 PM, Daniel Dormont wrote: > Thanks. Will that include ports open from processes that have died? That's > the specific case I'm trying to understand - what happens if a process > crashes with a file open, and how do I recover from it? > > -Dan > > > On Tue, May 1, 2012 at 12:46 PM, Max Lapshin wrote: >> >> lsof -p >> ulimit >> >> [{Pid, Ports} || {Pid,Ports} <- [{Pid, [L || L <- >> element(2,process_info(Pid, links)), is_port(L)]} || Pid <- >> processes()], length(Ports) > 0] > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From dmercer@REDACTED Tue May 1 20:50:18 2012 From: dmercer@REDACTED (David Mercer) Date: Tue, 1 May 2012 13:50:18 -0500 Subject: [erlang-questions] Fwd: Erlang Job for $5000 in Saint-Petersburg In-Reply-To: <8FEF2536-1F35-442C-BFD1-7AF4BD2FF281@cs.otago.ac.nz> References: <4F990A78.5060501@rabbitmq.com> <4F9910A3.2030500@llaisdy.com> <8FEF2536-1F35-442C-BFD1-7AF4BD2FF281@cs.otago.ac.nz> Message-ID: <015401cd27cb$418bd5a0$c4a380e0$@com> On Thursday, April 26, Richard O'Keefe wrote: > On 26/04/2012, at 9:08 PM, Ivan Uemlianin wrote: > > > fwiw colloquial British English uses "their" for his/her ("they" for > he/she, and "them" for him/her). It's quite serviceable. > > However some native speakers of English find that unappealing. Indeed, also confusing. Just last week, a colleague was talking about someone who was interested in transferring from another department, but to conceal identity was using gender-neutral plurals. I actually thought she was talking about multiple people until I asked her how many people she was talking about. > Note that English has _almost_ entirely abandoned grammatical > gender, so that people think of pronouns as referring to sex. Almost? Abandoned? (1) I didn't know English ever had grammatical gender; (2) I didn't know there were still remnants of it around. (Unless you're talking about referring to ships in the feminine third person, though I thought that was sailor tradition rather than grammar.) Cheers, DBM From gumm@REDACTED Tue May 1 20:59:48 2012 From: gumm@REDACTED (Jesse Gumm) Date: Tue, 1 May 2012 13:59:48 -0500 Subject: [erlang-questions] Fwd: Erlang Job for $5000 in Saint-Petersburg In-Reply-To: <015401cd27cb$418bd5a0$c4a380e0$@com> References: <4F990A78.5060501@rabbitmq.com> <4F9910A3.2030500@llaisdy.com> <8FEF2536-1F35-442C-BFD1-7AF4BD2FF281@cs.otago.ac.nz> <015401cd27cb$418bd5a0$c4a380e0$@com> Message-ID: It's too bad English doesn't maintain a public repo where we can submit a patch to deal with fact that the language lacks support for such a fundamental and critical language feature as third person, gender-neutral, singulars to indicate a person (as opposed to just using "it"). -Jesse On Tue, May 1, 2012 at 1:50 PM, David Mercer wrote: > On Thursday, April 26, Richard O'Keefe wrote: > >> On 26/04/2012, at 9:08 PM, Ivan Uemlianin wrote: >> >> > fwiw colloquial British English uses "their" for his/her ("they" for >> he/she, and "them" for him/her). ?It's quite serviceable. >> >> However some native speakers of English find that unappealing. > > Indeed, also confusing. ?Just last week, a colleague was talking about > someone who was interested in transferring from another department, but to > conceal identity was using gender-neutral plurals. ?I actually thought she > was talking about multiple people until I asked her how many people she was > talking about. > >> Note that English has _almost_ entirely abandoned grammatical >> gender, so that people think of pronouns as referring to sex. > > Almost? ?Abandoned? ?(1) I didn't know English ever had grammatical gender; > (2) I didn't know there were still remnants of it around. ?(Unless you're > talking about referring to ships in the feminine third person, though I > thought that was sailor tradition rather than grammar.) > > Cheers, > > DBM > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm From gumm@REDACTED Tue May 1 21:03:20 2012 From: gumm@REDACTED (Jesse Gumm) Date: Tue, 1 May 2012 14:03:20 -0500 Subject: [erlang-questions] Fwd: Erlang Job for $5000 in Saint-Petersburg In-Reply-To: References: <4F990A78.5060501@rabbitmq.com> <4F9910A3.2030500@llaisdy.com> <8FEF2536-1F35-442C-BFD1-7AF4BD2FF281@cs.otago.ac.nz> <015401cd27cb$418bd5a0$c4a380e0$@com> Message-ID: Or, for that matter, a non-ambiguous second person plural. I'm personally fine with "y'all" despite not being from the American South. On Tue, May 1, 2012 at 1:59 PM, Jesse Gumm wrote: > It's too bad English doesn't maintain a public repo where we can > submit a patch to deal with fact that the language lacks support for > such a fundamental and critical language feature as third person, > gender-neutral, singulars to indicate a person (as opposed to just > using "it"). > > -Jesse > > On Tue, May 1, 2012 at 1:50 PM, David Mercer wrote: >> On Thursday, April 26, Richard O'Keefe wrote: >> >>> On 26/04/2012, at 9:08 PM, Ivan Uemlianin wrote: >>> >>> > fwiw colloquial British English uses "their" for his/her ("they" for >>> he/she, and "them" for him/her). ?It's quite serviceable. >>> >>> However some native speakers of English find that unappealing. >> >> Indeed, also confusing. ?Just last week, a colleague was talking about >> someone who was interested in transferring from another department, but to >> conceal identity was using gender-neutral plurals. ?I actually thought she >> was talking about multiple people until I asked her how many people she was >> talking about. >> >>> Note that English has _almost_ entirely abandoned grammatical >>> gender, so that people think of pronouns as referring to sex. >> >> Almost? ?Abandoned? ?(1) I didn't know English ever had grammatical gender; >> (2) I didn't know there were still remnants of it around. ?(Unless you're >> talking about referring to ships in the feminine third person, though I >> thought that was sailor tradition rather than grammar.) >> >> Cheers, >> >> DBM >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Jesse Gumm > Owner, Sigma Star Systems > 414.940.4866?|| sigma-star.com || @jessegumm -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm From mononcqc@REDACTED Tue May 1 21:07:28 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 01 May 2012 15:07:28 -0400 Subject: [erlang-questions] Fwd: Erlang Job for $5000 in Saint-Petersburg In-Reply-To: References: <4F990A78.5060501@rabbitmq.com> <4F9910A3.2030500@llaisdy.com> <8FEF2536-1F35-442C-BFD1-7AF4BD2FF281@cs.otago.ac.nz> <015401cd27cb$418bd5a0$c4a380e0$@com> Message-ID: <4FA03470.3050408@ferd.ca> What I remember a lot of French texts from the government (in Quebec) saying is just add a note somewhere at the beginning or end of the text that says 'The masculine form in this document is employed to make the text lighter and isn't meant to discriminate' or something to that effect. Then nobody really complains, though French does have a tradition of going masculine by default. On 12-05-01 2:59 PM, Jesse Gumm wrote: > It's too bad English doesn't maintain a public repo where we can > submit a patch to deal with fact that the language lacks support for > such a fundamental and critical language feature as third person, > gender-neutral, singulars to indicate a person (as opposed to just > using "it"). > > -Jesse > > On Tue, May 1, 2012 at 1:50 PM, David Mercer wrote: >> On Thursday, April 26, Richard O'Keefe wrote: >> >>> On 26/04/2012, at 9:08 PM, Ivan Uemlianin wrote: >>> >>>> fwiw colloquial British English uses "their" for his/her ("they" for >>> he/she, and "them" for him/her). It's quite serviceable. >>> >>> However some native speakers of English find that unappealing. >> Indeed, also confusing. Just last week, a colleague was talking about >> someone who was interested in transferring from another department, but to >> conceal identity was using gender-neutral plurals. I actually thought she >> was talking about multiple people until I asked her how many people she was >> talking about. >> >>> Note that English has _almost_ entirely abandoned grammatical >>> gender, so that people think of pronouns as referring to sex. >> Almost? Abandoned? (1) I didn't know English ever had grammatical gender; >> (2) I didn't know there were still remnants of it around. (Unless you're >> talking about referring to ships in the feminine third person, though I >> thought that was sailor tradition rather than grammar.) >> >> Cheers, >> >> DBM >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From ok@REDACTED Wed May 2 00:30:18 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 2 May 2012 10:30:18 +1200 Subject: [erlang-questions] Frames proposal In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> Message-ID: <6F95B5BD-EB88-485A-8090-6C5168B63259@cs.otago.ac.nz> On 1/05/2012, at 7:37 PM, Jos? Valim wrote: > I have read the proposal completely and I think everything is well explained and defined. Even though I am not a huge fan of the syntax, I think it fits Erlang well. One option that I haven't seen considered is still using curly brackets as delimiters and use `{~}` to specify an empty frame. This would make the common case (a frame with at least one element) easier on the eyes by sacrificing a bit the not so common case (empty frame). But again, we are in good hands with whatever the Erlang and OTP team decide. The relevant concept here is *uniformity*: are all cases of a construct generated by the *same* rule(s)? {a~1, b~2}, {a~1}, and {~} are *not* generated by the same rule. The bare ~ in {~} has no counterpart in the others. As I understand it, Joe Armstrong's current proposed syntax is '#{' [maplet {, maplet}...] '}' so you would have #{a~1, b~2}, #{a~1}, and #{} (using ~ for maplets where Joe would actually use =). I prefer symmetric fences, not least because it helps tools to help me get my brackets to match properly. But I could live with Joe's syntax. I have been toying with the idea of a revision to my proposal, namely that {',' }... '|' should only be allowed to replace slots that already exist in the base frame, and not to add new slots. Functional support for adding new slots would remain, but this way we would *know* that the new frame could always share the descriptor of the old one, and it would provide a "safety" assurance to the programmer: if you *intend* to write 'a copy of F except that the ~count slot is zero' then <{ count ~ 0 | F }> would faithfully check your assumption that F _has_ a count slot. This is just like the way records are now, after all. From ok@REDACTED Wed May 2 01:01:56 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 2 May 2012 11:01:56 +1200 Subject: [erlang-questions] Frames proposal In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> Message-ID: On 2/05/2012, at 2:28 AM, Max Bourinov wrote: > Hi Richard, > > Thank you very much for posting your great work! > > But the title scared me. "Getting rid of records" - what should I do with all my code that uses records? > > I like the idea of frames a lot. It would significantly simplify my code and I love this idea. > There are several answers to 'what should I do'. (1) If it ain't broke, don't fix it. Records are NOT going to go away. There's an analogy from Fortran. Fortran 66 had so-called 'Hollerith literals', where you would write e.g., 4HFRED to get a string "FRED". That's right, the length as a decimal integer, the letter capital H, and then that many characters. With the advent of proper character data in Fortran 77, Hollerith literals were not needed any more. They had always been clumsy and error-prone. But they stayed. I can't remember if they were still there in Fortran 90 or not, but by Fortran 95 they were gone. Well, I just tried a Hollerith literal in a tiny program and while the Fortran 95 compiler *complained*, the program *worked*. The Erlang community isn't quite _that_ committed to backwards compatibility, but it's not too far off... (2) Frames are almost drop-in replacements for records. The major exception is default initial values for fields, which have problems anyway. Years ago I wrote a paper 'Delenda est preprocessor'. Abstract patterns and frames are all part of a long-time project to make the preprocessor unnecessary. But, sigh, it will probably never go away. From ok@REDACTED Wed May 2 02:22:57 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 2 May 2012 12:22:57 +1200 Subject: [erlang-questions] Frames proposal In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> Message-ID: <0AC65FD9-71BD-4D04-B0F6-524DB61CF095@cs.otago.ac.nz> On 2/05/2012, at 3:48 AM, Juan Jose Comellas wrote: > And what about borrowing the syntax that Anton Lavrik used for erlson [1]? That syntax looks similar to the one used for records and is much easier to read/write. The frames proposal was written long before erlson was a twinkle in its author's eye. Indeed, it was originally drafted in the year that JSON was invented (2001), so JSON was not an influence. The frames proposal explains why using '=' in maplets is a bad idea. Basically, it is HARDER to read because it makes '=' locally ambiguous. You go searching through a file and hit the line foo = bar(), and you have NO IDEA whether this is a maplet or a matching. BAD! As for #{...} vs <{...}>, it's in one way harder to write (to get <{, press SHIFT with left hand, and then < and { are close for two-fingered right hand typing; the same is true of }>, but #{ requires me to hold down the shift key with my left hand AND press the # key with my left hand at the same time, and while there is a shift key on the right, { requires the use of the right hand). Brackets really ought to be mirror symmetric, but I could live with #{...}. There's a section in the proposal titled "The Colour of the paint on the Ark." why do we find it easier to argue about the colour of the paint on the Ark than to climb aboard? That is the section that provides the reasons for the syntax. Now Jan Burse and I have been arguing with each other about frames, but actually, that was a *lot* more useful, because he focussed on semantics. But here's the most important thing, which appears in the next draft of the frames proposal: This would have been a workable syntax for frames before erlson used it. Now, it is important that the syntax should be distinct from erlson syntax, so that frames can be added to Erlang without breaking erlson. There is also an important pragmatic difference between frames and erlson. Frames are new built in data type with a good deal of thought put into making them economical in memory and time. ``At runtime, Erlson dictionaries are represented as a list of {Name, Value} tuples ordered by Name. This way, each Erlson dictionary is a valid proplist and orddict in terms of the correspond[ing] stdlib modules.'' An entry in an Erlson dictionary thus takes 4 words for the {Name,Value} tuple plus 2 words for the list cell pointing to it, or 6 words compared with a frame's 1 or 2. It therefore matters which one you are using: a programmer needs to be able to tell at a glance whether an expression deals with a frame or with an erlson dictionary. That is, we shouldn't copy Erlson syntax, BECAUSE Erlson IS ALREADY USING IT and we must not break Erlson. From essen@REDACTED Wed May 2 02:35:49 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 02 May 2012 02:35:49 +0200 Subject: [erlang-questions] Frames proposal In-Reply-To: <0AC65FD9-71BD-4D04-B0F6-524DB61CF095@cs.otago.ac.nz> References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> <0AC65FD9-71BD-4D04-B0F6-524DB61CF095@cs.otago.ac.nz> Message-ID: <4FA08165.5000702@ninenines.eu> On 05/02/2012 02:22 AM, Richard O'Keefe wrote: > As for #{...} vs<{...}>, it's in one way harder to write > (to get<{, press SHIFT with left hand, and then< and { are close > for two-fingered right hand typing; the same is true of }>, > but #{ requires me to hold down the shift key with my left hand > AND press the # key with my left hand at the same time, and while > there is a shift key on the right, { requires the use of the right > hand). Brackets really ought to be mirror symmetric, but I could > live with #{...}. Only true for a range of QWERTY keyboards. There's a lot more layouts in the world. Incidentaly on my keyboard writing <{}> is insanely difficult (takes a minimum of 2 seconds when trying to go fast), while #{} is not perfect but a lot easier (maybe half a second, hard to measure at this point). This is because the former requires me to press the sequence "<, ALT GR {, ALT GR }, SHIFT >", while the latter only requires "ALT GR #, ALT GR {, ALT GR }". The keys for # and { are also right next to the other, which isn't true for < and {. So in my case at least it's not only 1 key less, it's also a lot simpler to type. It's also a lot simpler to type = instead of ALT GR ~, regardless of other considerations. -- Lo?c Hoguin Erlang Cowboy Nine Nines From ok@REDACTED Wed May 2 02:46:27 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 2 May 2012 12:46:27 +1200 Subject: [erlang-questions] Fwd: Erlang Job for $5000 in Saint-Petersburg In-Reply-To: <015401cd27cb$418bd5a0$c4a380e0$@com> References: <4F990A78.5060501@rabbitmq.com> <4F9910A3.2030500@llaisdy.com> <8FEF2536-1F35-442C-BFD1-7AF4BD2FF281@cs.otago.ac.nz> <015401cd27cb$418bd5a0$c4a380e0$@com> Message-ID: <1E36331E-D80F-44D9-8A84-FEAE49C5C4D5@cs.otago.ac.nz> On 2/05/2012, at 6:50 AM, David Mercer wrote: > > Almost? Abandoned? (1) I didn't know English ever had grammatical gender; > (2) I didn't know there were still remnants of it around. (Unless you're > talking about referring to ships in the feminine third person, though I > thought that was sailor tradition rather than grammar.) See http://en.wikipedia.org/wiki/Old_English Bearing in mind that there's another thread discussing syntax, you've got to love the last sentence of this paragraph: Gender in nouns was grammatical, as opposed to the natural gender that prevails in modern English. That is, the grammatical gender of a given noun did not necessarily correspond to its natural gender, even for nouns referring to people. For example, s?o sunne (the Sun) was feminine, se m?na (the Moon) was masculine, and ?at w?f "the woman/wife" was neuter. (Compare German cognates die Sonne, der Mond, das Weib.) Pronominal usage could reflect either natural or grammatical gender, when it conflicted. Elsewhere in wikipedia, The pronoun "she" is sometimes used to refer to things which can contain people, such as countries, ships, or vehicles, or when referring to certain other machines. I actually had certain dialects in mind. From ok@REDACTED Wed May 2 03:11:11 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 2 May 2012 13:11:11 +1200 Subject: [erlang-questions] Frames proposal In-Reply-To: <4FA08165.5000702@ninenines.eu> References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> <0AC65FD9-71BD-4D04-B0F6-524DB61CF095@cs.otago.ac.nz> <4FA08165.5000702@ninenines.eu> Message-ID: It would be *so* nice to discuss semantics... > Only true for a range of QWERTY keyboards. There's a lot more layouts in the world. Indeed. It is impossible to optimise for all of them. > Incidentaly on my keyboard writing <{}> is insanely difficult (takes a minimum of 2 seconds when trying to go fast), while #{} is not perfect but a lot easier (maybe half a second, hard to measure at this point). This is because the former requires me to press the sequence "<, ALT GR {, ALT GR }, SHIFT >", while the latter only requires "ALT GR #, ALT GR {, ALT GR }". The keys for # and { are also right next to the other, which isn't true for < and {. You don't say which keyboard. The frames proposal document notes a similar issue for Swedish keyboards, and suggests allowing *both* <{...}> *and* <(...)> as a way of coping with the problem. However, since the problem with <{...}> appears to be due to the use of the curly braces, surely this must make tuples and records already 'insanely difficult'. As for the difficulty of typing ~, code is READ more often than it is WRITTEN, so the readability advantage much outweighs the typing disadvantage. Modern text editors have abbreviation features which mean that you *should* be able to set your text editor up so that (, expands to <{}> with the cursor in the middle =, expands to ~ with the cursor after it so that typing "<{a ~ 1, b ~ 2}>" requires the keystrokes "(,a=,1, b=,2" It took me five minutes to write this code for my home-brew text editor: void comma(void) { if (argvalue != implicit) { putin(); } else switch (fetch(here)) { case ')': /* For Erlang expressions */ case ']': /* For Erlang lists */ here++; break; case '}': /* For Erlang tuples and frames */ case '>': /* For Erlang binaries */ here += fetch(here+1) == '>' ? 2 : 1; break; default: switch (fprev(here)) { case '(': /* For Erlang frames */ delete(NORM, here-1); strinsert(1, 4, "<{}>"); here -= 2; break; case '[': /* For Erlang tuples */ delete(NORM, here-1); strinsert(1, 2, "{}"); here--; break; case ':': /* For Erlang records */ delete(NORM, here-1); insert(1, '#'); break; case '=': /* For Erlang frames */ delete(NORM, here-1); strinsert(1, 3, " ~ "); break; case ' ': here--; insert(-1, ','); here++; break; default: putin(); } } } After which it's just "(,a=,1, b=,2," which can't be _that_ hard to type. From jozsef.berces@REDACTED Wed May 2 05:24:08 2012 From: jozsef.berces@REDACTED (=?iso-8859-1?Q?J=F3zsef_B=E9rces?=) Date: Wed, 2 May 2012 05:24:08 +0200 Subject: [erlang-questions] signal arrival time Message-ID: <3717CEF51B134A4F8073627839A9DF412743678B61@ESESSCMS0353.eemea.ericsson.se> Is there a way to tell the time when a signal arrived to the process mailbox? Thanks, Jozsef -------------- next part -------------- An HTML attachment was scrubbed... URL: From alavrik@REDACTED Wed May 2 06:36:26 2012 From: alavrik@REDACTED (Anton Lavrik) Date: Tue, 1 May 2012 23:36:26 -0500 Subject: [erlang-questions] Frames proposal In-Reply-To: <0AC65FD9-71BD-4D04-B0F6-524DB61CF095@cs.otago.ac.nz> References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> <0AC65FD9-71BD-4D04-B0F6-524DB61CF095@cs.otago.ac.nz> Message-ID: On Tue, May 1, 2012 at 7:22 PM, Richard O'Keefe wrote: > > That is, we shouldn't copy Erlson syntax, BECAUSE Erlson IS ALREADY > USING IT and we must not break Erlson. > I'm not sure if you are serious, but here's my comment to that anyway. To me, the benefits of using Erlson syntax for a native implementation of name-value dictionaries in Erlang outweigh the broken compatibility with Erlson runtime data representation. One of the reasons why I created Erlson was to prove that it is possible to have a nice syntax for Erlang dictionaries that fits with the language (*). While Erlson syntax is similar to records, it is still different enough to avoid confusion and it is compatible with the rest of Erlang's syntax. So, for those who actually consider implementing a native dictionary data type in Erlang and using Erlson syntax for that, please feel free to break Erlson. Anton (*) I realize that this is a highly subjective statement, but it is no worse than any of the purely theoretic arguments about syntax I read in this thread. Language design is not a science, for better or worse. From andrzej.sliwa@REDACTED Wed May 2 07:50:25 2012 From: andrzej.sliwa@REDACTED (Andrzej Sliwa) Date: Wed, 2 May 2012 07:50:25 +0200 Subject: [erlang-questions] Frames proposal In-Reply-To: <4FA08165.5000702@ninenines.eu> References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> <0AC65FD9-71BD-4D04-B0F6-524DB61CF095@cs.otago.ac.nz> <4FA08165.5000702@ninenines.eu> Message-ID: come on, we can't start inventing new syntax only from 'editor' perspective, otherwise we could go to "brainfuck" syntax, which is so compact and easy to write ;), but in 100% is not readable at all Regards, Andrzej On May 2, 2012, at 2:35 AM, Lo?c Hoguin wrote: > On 05/02/2012 02:22 AM, Richard O'Keefe wrote: >> As for #{...} vs<{...}>, it's in one way harder to write >> (to get<{, press SHIFT with left hand, and then< and { are close >> for two-fingered right hand typing; the same is true of }>, >> but #{ requires me to hold down the shift key with my left hand >> AND press the # key with my left hand at the same time, and while >> there is a shift key on the right, { requires the use of the right >> hand). Brackets really ought to be mirror symmetric, but I could >> live with #{...}. > > Only true for a range of QWERTY keyboards. There's a lot more layouts in the world. Incidentaly on my keyboard writing <{}> is insanely difficult (takes a minimum of 2 seconds when trying to go fast), while #{} is not perfect but a lot easier (maybe half a second, hard to measure at this point). This is because the former requires me to press the sequence "<, ALT GR {, ALT GR }, SHIFT >", while the latter only requires "ALT GR #, ALT GR {, ALT GR }". The keys for # and { are also right next to the other, which isn't true for < and {. > > So in my case at least it's not only 1 key less, it's also a lot simpler to type. > > It's also a lot simpler to type = instead of ALT GR ~, regardless of other considerations. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Wed May 2 08:38:17 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 2 May 2012 07:38:17 +0100 Subject: [erlang-questions] Travis-CI - lager_transform undef error In-Reply-To: <5AA18C1E-08EA-43F8-9874-ADA167307240@worrell.nl> References: <5AA18C1E-08EA-43F8-9874-ADA167307240@worrell.nl> Message-ID: <7C536E2B-19DD-4A84-8CFE-F042CF1D7166@gmail.com> Is it '{lib_dirs, ["deps"]}.' in rebar.config or something similar passed to erl_opts? The issue is that the compiler needs to know where code:lib_dir(lager) resides iirc. On 1 May 2012, at 19:12, Marc Worrell wrote: > Hi, > > I am trying to setup some continuous integration with Travis-CI.org for Zotonic. > > I am running into this error: > > http://travis-ci.org/#!/zotonic/zotonic/jobs/1220673 > erlc -o src/erlydtl src/erlydtl/erlydtl_parser.yrl > 202Recompile: src/zotonic_sup > 203src/zotonic_sup.erl:none: error in parse transform 'lager_transform': {undef, > 204 [{lager_transform, > With the following travis definition: > > language: erlang > otp_release: > - R15B01 > - R14B04 > - R14B03 > script: make > before_install: git submodule update --init --recursive > env: > - DB=postgres > branches: > only: > - master > > Does anyone have any idea how to set the correct paths so that the compiler can find the lager_transform in deps/lager? > > - Marc > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Wed May 2 08:50:48 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 02 May 2012 08:50:48 +0200 Subject: [erlang-questions] Debugging apps with dependencies In-Reply-To: <4BE220475DFC4F5AA43F0637D39ED568@gmail.com> References: <4BE220475DFC4F5AA43F0637D39ED568@gmail.com> Message-ID: <4FA0D948.4010100@gmail.com> On 2012-04-30 20:05, Josh Black wrote: > I'm working on an app, and so far, I've been testing it by using rebar > to generate a release, then starting a console for the release and going > from there. The problem comes when I want to use something like dbg or > appmon to debug errors or inspect the state of my app. When I try to do, > e.g. appmon:start() from the console for my release, it tells me > "undefined function appmon:start/0". I'm guessing this is because I > didn't explicitly specify appmon in my release configuration? > > Alternatively, I created a start.sh shell script with contents like this: > > erl -pa apps/*/ebin -pa deps/*/ebin -eval > 'application:start(dependency1), application:start(dependency2), > application:start(myapp).' > > This lets me start appmon and whatever else I want when I run into > errors, but feels pretty messy and wrong. > > I feel like I'm missing something really obvious here. What's the > correct way to do this? It's usually best to not run the debugging tools directly on the node you're inspecting. You shouldn't have to load the code for such tools on the production node. Instead, start another node having all libraries available, and connect to the one you want to debug. Appmon and many other debugging tools are able to inspect the state of another node than the one actually running the GUI. /Richard From dgud@REDACTED Wed May 2 10:12:32 2012 From: dgud@REDACTED (Dan Gudmundsson) Date: Wed, 2 May 2012 10:12:32 +0200 Subject: [erlang-questions] Debugging apps with dependencies In-Reply-To: References: <4BE220475DFC4F5AA43F0637D39ED568@gmail.com> <2375676CEBC34724A653F2C69CBDD33C@gmail.com> Message-ID: On Tue, May 1, 2012 at 1:11 AM, Garret Smith wrote: > > For the appmon problem, 1st I recommend using 'observer' instead.? I just > started using it with R15B01 and... wow.? The tracing support is > phenomenal.? Go Erlang devs! > > To get observer included in your release, add it to your reltool.config like > so: > > {app, observer, [{incl_cond, include}]}, > > -Garret Smith Thanks. But, as Richard said, run observer from a separate node instead. Include the 'runtime_tools' library on your target machine, it should have everything you will need on your target. /Dan From janburse@REDACTED Wed May 2 10:50:32 2012 From: janburse@REDACTED (Jan Burse) Date: Wed, 02 May 2012 10:50:32 +0200 Subject: [erlang-questions] Frames proposal In-Reply-To: <0AC65FD9-71BD-4D04-B0F6-524DB61CF095@cs.otago.ac.nz> References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> <0AC65FD9-71BD-4D04-B0F6-524DB61CF095@cs.otago.ac.nz> Message-ID: <4FA0F558.4080400@fastmail.fm> Richard O'Keefe schrieb: > Now Jan Burse and I have been arguing with each other about frames, > but actually, that was a*lot* more useful, because he focussed on > semantics. No I didn't care about semantics. The semantics is trivial since as I interpret Erlang it does do some flow analysis of the given code and sees to it that everything is ground, compared to normal Prolog. The flow analysis happens mostly implicit by having turned the relation semantics of Prolog into a functional semantics. With the sad result that one way to implement relation semantics in Erlang is via continuation. I was more concerned with usage and complexity. Your frames have been invented many many times. I remember also some proposals that retain the relational character, so that a pattern <{k1~v1, k2~vn|F}> with a tail frame would become a first class citizen of the language. These are then not called frames but rather feature structures. I guess one way to implement real feature structures in a logic programming language would be to extend the unification so that it works on feature terms with tail features. Feature structures are very fashion in certain attribute grammar formalism. Besides unification there can be further operations between feature structures. The simple syntax there for feature structures is [k1:v1, k2:vn|F]. And it seems to be rather the rule than the exception that a feature structure is open. This stems from the relation ship of feature structures to equations, so the following equations G:k1 === v1, G:k2 === v2 would correspond to the following feature structure relation: G===[k1:v1,k2:v2|_] The (===)/2 in the two equations can be viewed as a declarative combined setter/getter. So setter/getter are by no way alien to logic programming and it is not mandatory to go with a feature structure on the surface of a language. What seems to be a problem in the grammar domain is that then practically highly recursive feature structures are used. But some compilation techniques are able to flatten them straight the way to records and removing all open parts that are never closed. So that a feature structure (*): [head:[cat:[n:A,v:B|_],agr:[per:C,num:D|_], case:E,nform:F|_],bar:G|_] Turns into: dag0(A,B,C,D,F,G) In certain circumstances it can be also benefitial to turn open parts into record positions. Maybe the grammar application domain can provide further ideas for representing and optimizing frame terms in Erlang, and vice versa. Most probably the referenced paper below is already hopelessly outdated, and people do even more clever things today. But anyway. (*) Compiling Feature Structures into Terms: an Empirical Study in Prolog (1993) by Andreas Sch?ter , Buccleuch Place http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.24.8407 From erlang@REDACTED Wed May 2 11:40:34 2012 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 2 May 2012 11:40:34 +0200 Subject: [erlang-questions] signal arrival time In-Reply-To: <3717CEF51B134A4F8073627839A9DF412743678B61@ESESSCMS0353.eemea.ericsson.se> References: <3717CEF51B134A4F8073627839A9DF412743678B61@ESESSCMS0353.eemea.ericsson.se> Message-ID: Yes - read the manual pages for the BIF erlang:trace. You basically set up a trace process that watches the process you are interested in, then you ask the system to send you timestamp information for each message that is received. This is described at http://www.erlang.org/doc/man/erlang.html#trace-3 Alternatively you could you fold something like this into your code: loop() -> receive {ts, Time, Pattern1} -> ... {ts, Time, Pattern2} -> ... Any -> self() ! {ts, erlang:now(), Any}, loop() end. /Joe On Wed, May 2, 2012 at 5:24 AM, J?zsef B?rces wrote: > Is there a way to tell the time when a signal arrived to the process > mailbox? > > Thanks, > Jozsef > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ulf@REDACTED Wed May 2 11:52:06 2012 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 2 May 2012 11:52:06 +0200 Subject: [erlang-questions] signal arrival time In-Reply-To: References: <3717CEF51B134A4F8073627839A9DF412743678B61@ESESSCMS0353.eemea.ericsson.se> Message-ID: Another way to do it is to use DTrace (not yet officially supported) http://www.erlang-factory.com/conference/ErlangUserConference2011/speakers/ScottLystigFritchie BR, Ulf W On 2 May 2012, at 11:40, Joe Armstrong wrote: > Yes - read the manual pages for the BIF erlang:trace. You basically > set up a trace process > that watches the process you are interested in, then you ask the > system to send you timestamp > information for each message that is received. > > This is described at http://www.erlang.org/doc/man/erlang.html#trace-3 > > Alternatively you could you fold something like this into your code: > > > loop() -> > receive > {ts, Time, Pattern1} -> > ... > {ts, Time, Pattern2} -> > ... > Any -> > self() ! {ts, erlang:now(), Any}, > loop() > end. > > /Joe > > > On Wed, May 2, 2012 at 5:24 AM, J?zsef B?rces > wrote: >> Is there a way to tell the time when a signal arrived to the process >> mailbox? >> >> Thanks, >> Jozsef >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From juanjo@REDACTED Wed May 2 13:42:54 2012 From: juanjo@REDACTED (Juan Jose Comellas) Date: Wed, 2 May 2012 08:42:54 -0300 Subject: [erlang-questions] Debugging apps with dependencies In-Reply-To: References: <4BE220475DFC4F5AA43F0637D39ED568@gmail.com> Message-ID: BTW, I use a variant of what you showed that also starts dependencies recursively. I normally create a module with the name of the project and export the following functions to start it. That's what I use while testing. -define(APP, ?MODULE). %% @doc Start the application and all its dependencies. -spec start() -> ok. start() -> start_deps(?APP), application:start(?APP). -spec start_deps(App :: atom()) -> ok. start_deps(App) -> application:load(App), {ok, Deps} = application:get_key(App, applications), lists:foreach(fun start_deps/1, Deps), start_app(App). -spec start_app(App :: atom()) -> ok. start_app(App) -> case application:start(App) of {error, {already_started, _}} -> ok; ok -> ok end. Juanjo On Mon, Apr 30, 2012 at 3:39 PM, Bob Ippolito wrote: > I haven't discovered a great workflow for developing with releases either, > but you can start dependencies in a slightly cleaner way by looking at the > app file: > > This would go in some module with the same name as the application, and > you would call it with "erl ... -s modname" > > -export([start/0]). > > start() -> start(?MODULE). > > start(App) -> > application:load(App), > {ok, Deps} = application:get_key(App, applications), > lists:foreach(fun start_app/1, Deps), > application:start(App). > > start_app(App) -> > case application:start(App) of > {error, {already_started, _}} -> ok; > ok -> ok > end. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From juanjo@REDACTED Wed May 2 13:55:20 2012 From: juanjo@REDACTED (Juan Jose Comellas) Date: Wed, 2 May 2012 08:55:20 -0300 Subject: [erlang-questions] Debugging apps with dependencies In-Reply-To: References: <4BE220475DFC4F5AA43F0637D39ED568@gmail.com> Message-ID: Sorry, there was a small typo in my previous copy. Here's the correct version that also includes the function that stops the module and its dependencies: -define(APP, ?MODULE). %% @doc Start the application and all its dependencies. -spec start() -> ok. start() -> start_deps(?APP). -spec start_deps(App :: atom()) -> ok. start_deps(App) -> application:load(App), {ok, Deps} = application:get_key(App, applications), lists:foreach(fun start_deps/1, Deps), start_app(App). -spec start_app(App :: atom()) -> ok. start_app(App) -> case application:start(App) of {error, {already_started, _}} -> ok; ok -> ok end. %% @doc Stop the application and all its dependencies. -spec stop() -> ok. stop() -> stop_deps(?APP). -spec stop_deps(App :: atom()) -> ok. stop_deps(App) -> stop_app(App), {ok, Deps} = application:get_key(App, applications), lists:foreach(fun stop_deps/1, lists:reverse(Deps)). -spec stop_app(App :: atom()) -> ok. stop_app(kernel) -> ok; stop_app(stdlib) -> ok; stop_app(App) -> case application:stop(App) of {error, {not_started, _}} -> ok; ok -> ok end. On Wed, May 2, 2012 at 8:42 AM, Juan Jose Comellas wrote: > BTW, I use a variant of what you showed that also starts dependencies > recursively. I normally create a module with the name of the project and > export the following functions to start it. That's what I use while testing. > > -define(APP, ?MODULE). > > %% @doc Start the application and all its dependencies. > -spec start() -> ok. > start() -> > start_deps(?APP), > application:start(?APP). > > -spec start_deps(App :: atom()) -> ok. > start_deps(App) -> > application:load(App), > {ok, Deps} = application:get_key(App, applications), > lists:foreach(fun start_deps/1, Deps), > start_app(App). > > -spec start_app(App :: atom()) -> ok. > start_app(App) -> > case application:start(App) of > {error, {already_started, _}} -> ok; > ok -> ok > end. > > > Juanjo > > > On Mon, Apr 30, 2012 at 3:39 PM, Bob Ippolito wrote: > >> I haven't discovered a great workflow for developing with releases >> either, but you can start dependencies in a slightly cleaner way by looking >> at the app file: >> >> This would go in some module with the same name as the application, and >> you would call it with "erl ... -s modname" >> >> -export([start/0]). >> >> start() -> start(?MODULE). >> >> start(App) -> >> application:load(App), >> {ok, Deps} = application:get_key(App, applications), >> lists:foreach(fun start_app/1, Deps), >> application:start(App). >> >> start_app(App) -> >> case application:start(App) of >> {error, {already_started, _}} -> ok; >> ok -> ok >> end. >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mazen.harake@REDACTED Wed May 2 13:56:50 2012 From: mazen.harake@REDACTED (Mazen Harake) Date: Wed, 2 May 2012 13:56:50 +0200 Subject: [erlang-questions] signal arrival time In-Reply-To: <3717CEF51B134A4F8073627839A9DF412743678B61@ESESSCMS0353.eemea.ericsson.se> References: <3717CEF51B134A4F8073627839A9DF412743678B61@ESESSCMS0353.eemea.ericsson.se> Message-ID: Yes you can. You can trace the process you want to observe (use the flags 'r' for receive and 'timestamp' for attaching the time to it) and then receive the trace messages using your own trace event handler. I would also recommend that you use matchspecifications in case you are only interested if 1 specific message has arrived, it is much more efficient. I wrote a pretty straightforward tutorial on how to do these sort of things in one of the EUC a couple of years ago. You can find it here: http://www.erlang-factory.com/upload/presentations/316/dbg[1].pdf BR, /M On 2 May 2012 05:24, J?zsef B?rces wrote: > Is there a way to tell the time when a signal arrived to the process > mailbox? > > Thanks, > Jozsef > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From huss01@REDACTED Wed May 2 14:24:13 2012 From: huss01@REDACTED (=?ISO-8859-1?Q?H=E5kan_Huss?=) Date: Wed, 2 May 2012 14:24:13 +0200 Subject: [erlang-questions] Fwd: Erlang Job for $5000 in Saint-Petersburg In-Reply-To: References: <4F990A78.5060501@rabbitmq.com> <4F9910A3.2030500@llaisdy.com> <8FEF2536-1F35-442C-BFD1-7AF4BD2FF281@cs.otago.ac.nz> <015401cd27cb$418bd5a0$c4a380e0$@com> Message-ID: Well, in some southern regions "y'all" denotes second person singular. Second person plural would be "all y'all" :-) /H?kan On Tue, May 1, 2012 at 9:03 PM, Jesse Gumm wrote: > Or, for that matter, a non-ambiguous second person plural. I'm > personally fine with "y'all" despite not being from the American > South. > > On Tue, May 1, 2012 at 1:59 PM, Jesse Gumm wrote: >> It's too bad English doesn't maintain a public repo where we can >> submit a patch to deal with fact that the language lacks support for >> such a fundamental and critical language feature as third person, >> gender-neutral, singulars to indicate a person (as opposed to just >> using "it"). >> >> -Jesse >> >> On Tue, May 1, 2012 at 1:50 PM, David Mercer wrote: >>> On Thursday, April 26, Richard O'Keefe wrote: >>> >>>> On 26/04/2012, at 9:08 PM, Ivan Uemlianin wrote: >>>> >>>> > fwiw colloquial British English uses "their" for his/her ("they" for >>>> he/she, and "them" for him/her). ?It's quite serviceable. >>>> >>>> However some native speakers of English find that unappealing. >>> >>> Indeed, also confusing. ?Just last week, a colleague was talking about >>> someone who was interested in transferring from another department, but to >>> conceal identity was using gender-neutral plurals. ?I actually thought she >>> was talking about multiple people until I asked her how many people she was >>> talking about. >>> >>>> Note that English has _almost_ entirely abandoned grammatical >>>> gender, so that people think of pronouns as referring to sex. >>> >>> Almost? ?Abandoned? ?(1) I didn't know English ever had grammatical gender; >>> (2) I didn't know there were still remnants of it around. ?(Unless you're >>> talking about referring to ships in the feminine third person, though I >>> thought that was sailor tradition rather than grammar.) >>> >>> Cheers, >>> >>> DBM >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> -- >> Jesse Gumm >> Owner, Sigma Star Systems >> 414.940.4866?|| sigma-star.com || @jessegumm > > > > -- > Jesse Gumm > Owner, Sigma Star Systems > 414.940.4866?|| sigma-star.com || @jessegumm > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ivan@REDACTED Wed May 2 14:32:09 2012 From: ivan@REDACTED (Ivan Uemlianin) Date: Wed, 02 May 2012 13:32:09 +0100 Subject: [erlang-questions] Fwd: Erlang Job for $5000 in Saint-Petersburg In-Reply-To: References: <4F990A78.5060501@rabbitmq.com> <4F9910A3.2030500@llaisdy.com> <8FEF2536-1F35-442C-BFD1-7AF4BD2FF281@cs.otago.ac.nz> <015401cd27cb$418bd5a0$c4a380e0$@com> Message-ID: <4FA12949.7030308@llaisdy.com> Some dialects of British English have "youse" (pronounced a bit like "use" but with a shorter vowel). > On Tue, May 1, 2012 at 9:03 PM, Jesse Gumm wrote: >> Or, for that matter, a non-ambiguous second person plural. I'm >> personally fine with "y'all" despite not being from the American >> South. -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From jargon@REDACTED Wed May 2 15:09:37 2012 From: jargon@REDACTED (Johannes =?utf-8?B?V2Vpw59s?=) Date: Wed, 2 May 2012 15:09:37 +0200 Subject: [erlang-questions] unicode chardata and re:run Message-ID: <20120502130937.GB19620@molb.org> Hello List! I have a question regarding the regular expression module: The man page [1] says re:run/3 is accepting a unicode:charlist() [2] as Subject/RE when the "unicode" option is supplied. However, using the function with a UTF-8 binary also works: match = re:run(<<"foo">>, <<"f.o">>, [{capture, none}, unicode]). I even found a test case which relies on re:run/3 accepting unicode:chardata() (charlist + unicode_binary) in re_SUITE.erl [3]. Does this mean I can rely on re:run/3 accepting binaries (in this case the documentation should be changed), or does re:run/3 only accept charlists (in this case the test case needs to be changed)? I found a post from 2010 [4] in which the first option is suggested. [1] http://www.erlang.org/doc/man/re.html#run-3 [2] http://www.erlang.org/doc/man/unicode.html#type-charlist [3] https://github.com/erlang/otp/blob/master/lib/stdlib/test/re_SUITE.erl#L295 [4] http://erlang.org/pipermail/erlang-patches/2010-January/000697.html Greetings, Johannes From michael.eugene.turner@REDACTED Wed May 2 15:16:49 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 2 May 2012 22:16:49 +0900 Subject: [erlang-questions] Fwd: Erlang Job for $5000 in Saint-Petersburg In-Reply-To: <4FA12949.7030308@llaisdy.com> References: <4F990A78.5060501@rabbitmq.com> <4F9910A3.2030500@llaisdy.com> <8FEF2536-1F35-442C-BFD1-7AF4BD2FF281@cs.otago.ac.nz> <015401cd27cb$418bd5a0$c4a380e0$@com> <4FA12949.7030308@llaisdy.com> Message-ID: I believe that the topic drift on this thread has reached the point where a return to the original topic is slowly becoming slightly more probable. This is because of the curvature of the universe. -michael turner On Wed, May 2, 2012 at 9:32 PM, Ivan Uemlianin wrote: > Some dialects of British English have "youse" (pronounced a bit like "use" > but with a shorter vowel). > > >> On Tue, May 1, 2012 at 9:03 PM, Jesse Gumm ?wrote: >>> >>> Or, for that matter, a non-ambiguous second person plural. I'm >>> personally fine with "y'all" despite not being from the American >>> South. > > > -- > ============================================================ > Ivan A. Uemlianin PhD > Llaisdy > Speech Technology Research and Development > > ? ? ? ? ? ? ? ? ? ?ivan@REDACTED > ? ? ? ? ? ? ? ? ? ? www.llaisdy.com > ? ? ? ? ? ? ? ? ? ? ? ? llaisdy.wordpress.com > ? ? ? ? ? ? ?github.com/llaisdy > ? ? ? ? ? ? ? ? ? ? www.linkedin.com/in/ivanuemlianin > > ? ?"Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" > ? ? ? ? ? ? ? ? ? ? (Schiller, Beethoven) > ============================================================ > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From kunthar@REDACTED Wed May 2 15:20:38 2012 From: kunthar@REDACTED (Kunthar) Date: Wed, 2 May 2012 16:20:38 +0300 Subject: [erlang-questions] Fwd: Erlang Job for $5000 in Saint-Petersburg In-Reply-To: <4FA12949.7030308@llaisdy.com> References: <4F990A78.5060501@rabbitmq.com> <4F9910A3.2030500@llaisdy.com> <8FEF2536-1F35-442C-BFD1-7AF4BD2FF281@cs.otago.ac.nz> <015401cd27cb$418bd5a0$c4a380e0$@com> <4FA12949.7030308@llaisdy.com> Message-ID: Hi all, As you can easily realize, we are monitoring this list from all over the world speaking different languages for Erlang based events. I would like to give another example from my own country for audience to have fun: Siz Erlang ile ilgili bir listede siktirik dilbilim yorumlar? yaparken, snob ve en k???k bir katk? sa?lamayan angut mesajlar?n?za makak maymunlar? bile g?t?yle g?l?yor, sizi erkek egemen dallamalar s?r?s? sizi :) Be well brothers On Wed, May 2, 2012 at 3:32 PM, Ivan Uemlianin wrote: > Some dialects of British English have "youse" (pronounced a bit like "use" > but with a shorter vowel). > > >> On Tue, May 1, 2012 at 9:03 PM, Jesse Gumm ?wrote: >>> >>> Or, for that matter, a non-ambiguous second person plural. I'm >>> personally fine with "y'all" despite not being from the American >>> South. > > > -- > ============================================================ > Ivan A. Uemlianin PhD > Llaisdy > Speech Technology Research and Development > > ? ? ? ? ? ? ? ? ? ?ivan@REDACTED > ? ? ? ? ? ? ? ? ? ? www.llaisdy.com > ? ? ? ? ? ? ? ? ? ? ? ? llaisdy.wordpress.com > ? ? ? ? ? ? ?github.com/llaisdy > ? ? ? ? ? ? ? ? ? ? www.linkedin.com/in/ivanuemlianin > > ? ?"Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" > ? ? ? ? ? ? ? ? ? ? (Schiller, Beethoven) > ============================================================ > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- BR, \|/ Kunthar From straightflush@REDACTED Wed May 2 15:30:21 2012 From: straightflush@REDACTED (AD) Date: Wed, 2 May 2012 09:30:21 -0400 Subject: [erlang-questions] Most performant way to store/search a bag Message-ID: Hello, I have a problem where i need to map event_ids to connection objects (for use in websockets to broadcast an event). I am trying to determine the most efficient way to map 1 event_id to possibly several thousand connection objects. Ultimately I would need to find every connection object mapped to an ID and then send an event to each connection object. Is using an ETS bag and then using ets:foldl() to loop through the most optimal ? I was thinking of breaking down each event_type into a separate ETS table and then storing the event_id for that event_type in the respective ETS table with the connection object. Thoughts welcome. Thanks -AD -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenneth.lundin@REDACTED Wed May 2 17:07:51 2012 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 2 May 2012 17:07:51 +0200 Subject: [erlang-questions] signal arrival time In-Reply-To: References: <3717CEF51B134A4F8073627839A9DF412743678B61@ESESSCMS0353.eemea.ericsson.se> Message-ID: Note that the trace mechanisms should not be used as a regular part of production code. Trace is intended for trouble shooting and profiling, if used as part of the regular system it will destroy the possibility to use trace for trouble shooting alternatively trouble shooting will destroy the running system. It would be interesting to know what you are going to use the information about when a message arrived to a mailbox for? Maybe there are other ways of solving your problem? Kenneth, Erlang/OTP Ericsson Den 2 maj 2012 13:56 skrev "Mazen Harake" : > > Yes you can. > > You can trace the process you want to observe (use the flags 'r' for > receive and 'timestamp' for attaching the time to it) and then receive > the trace messages using your own trace event handler. I would also > recommend that you use matchspecifications in case you are only > interested if 1 specific message has arrived, it is much more > efficient. > > I wrote a pretty straightforward tutorial on how to do these sort of > things in one of the EUC a couple of years ago. You can find it here: > http://www.erlang-factory.com/upload/presentations/316/dbg[1].pdf > > BR, > > /M > > On 2 May 2012 05:24, J?zsef B?rces wrote: > > Is there a way to tell the time when a signal arrived to the process > > mailbox? > > > > Thanks, > > Jozsef > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From zhengsyao@REDACTED Wed May 2 17:09:35 2012 From: zhengsyao@REDACTED (=?utf-8?Q?Siyao_Zheng=28=E9=83=91=E6=80=9D=E9=81=A5=29?=) Date: Wed, 2 May 2012 23:09:35 +0800 Subject: [erlang-questions] About Erlang SMP scheduler In-Reply-To: References: <0F4D9489-7755-4FD2-8831-E4BEE2A8C8F1@gmail.com> <2FF9F129-C96C-48C2-BFE9-223F591D624B@gmail.com> Message-ID: <0773DEE9-E2B7-440A-814E-33C55A472A7C@gmail.com> Hi, Yes, that's true. But the "try_steal_task()" afterwards is also controlled by ERTS_SMP macro. It doesn't make any sense to talk about workload balance outside multi-processor environment. I'm just wondering why "check_balance" is needed while there exists work stealing(try_steal_task). Cheers On May 2, 2012, at 12:04 PM, xu yu wrote: > Hi, > > "check_balance" controlled by "ifdef ERTS_SMP", so... > > 2012/4/27 "Siyao Zheng(???)" > Hi, > > schedule(), in which check_balance() is only called. > > In schedule(), when check_balance_reds reaches zero, scheduler decides to "check_balance". But later in schedule(), when scheduler finds run queue being empty, it will try to steal task from other schedulers (by calling try_steal_task()), which is also a load balance mechanism. I'm just wondering what the purpose of check_balance(). > > Cheers > Siyao > > On Apr 27, 2012, at 9:40 PM, Zabrane Mickael wrote: > > > Hi Siyao, > > > > Which "check_balance" you've commented out? > > > > Here is what I found so far: > > > > $ find /otp_src_R15B01 -type f | xargs grep check_balance | grep -v matches > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: int forced_check_balance; > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c:check_balance(ErtsRunQueue *c_rq) > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c:# error check_balance() assumes ERTS_MAX_PROCESS < (1 << 27) > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: c_rq->check_balance_reds = INT_MAX; > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: c_rq->check_balance_reds = INT_MAX; > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: rq->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: * check_balance() is never called in more threads > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: forced = balance_info.forced_check_balance; > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: balance_info.forced_check_balance = 0; > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: c_rq->check_balance_reds = INT_MAX; > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: rq->check_balance_reds = INT_MAX; > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: rq->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: rq->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: balance_info.forced_check_balance = 0; > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: (RQ)->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; \ > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: balance_info.forced_check_balance = 1; > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: ERTS_RUNQ_IX(0)->check_balance_reds = 0; > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: if (rq->check_balance_reds <= 0) > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: check_balance(rq); > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.h: int check_balance_reds; > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.h: (RQ)->check_balance_reds -= (REDS); \ > > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.h: (RQ)->check_balance_reds -= (REDS); > > [...] > > > > For OTP Team: is this dangerous ? > > > > Regards, > > Zabrane > > > > > > On Apr 27, 2012, at 1:23 PM, Siyao Zheng(???) wrote: > > > >> Hi, > >> > >> In SMP version of Erlang, every scheduler periodically call "check_balance()" to check load balance among all schedulers, then it might migrate some processes to balance load, or shut down some schedulers with low load. Does anyone know why scheduler should do this? check_balance() is quite a big one, and it has to lock every run queue when it checks the run queue. I think it's quite a big cost here. The work stealing at each schedule step afterwards actually does work load balance very well. Actually after I comment out the check_balance() step and run BigBang and Hackbench benchmarks, they are really faster! > >> > >> So, I wonder what is the purpose of check_balance() step here, if it is related to scalability on many core processors or something, and if there is any other benchmark I can run to prove its usefulness. > >> > >> Cheers! > >> > >> Siyao > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Wed May 2 17:19:39 2012 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 02 May 2012 16:19:39 +0100 (BST) Subject: [erlang-questions] Artificial Intelligence and Erlang? In-Reply-To: Message-ID: I don't know if you would class this as AI but I have a Prolog interpreter written in Erlang. It is a proper subset of standard Prolog with unification and real database handling and interacts well with Erlang. Prolog is very good for rule-based and knowledge based systems (duh!). An alternative would be to connect to a real prolog system. I also know that there should be a book on using Erlang and neural-nets in the pipeline, though I have no idea when it will be available. I am waiting though. Robert ----- Original Message ----- Does anyone know of a good source on AI and Erlang? A book would be awesome! Most of the sources that I see out there have Lisp as the functional programming language of choice. Erlang seems like a better option given it's nature of distributed nature and ability to place a ton of processes in less space. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From yoursurrogategod@REDACTED Wed May 2 17:22:16 2012 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Wed, 2 May 2012 11:22:16 -0400 Subject: [erlang-questions] Artificial Intelligence and Erlang? In-Reply-To: References: Message-ID: Could you please share some info on this book? Maybe an announcement by the author or publisher online? I'd love to read more. On Wed, May 2, 2012 at 11:19 AM, Robert Virding < robert.virding@REDACTED> wrote: > I don't know if you would class this as AI but I have a Prolog interpreter > written in Erlang. It is a proper subset of standard Prolog with > unification and real database handling and interacts well with Erlang. > Prolog is very good for rule-based and knowledge based systems (duh!). An > alternative would be to connect to a real prolog system. > > I also know that there should be a book on using Erlang and neural-nets in > the pipeline, though I have no idea when it will be available. I am waiting > though. > > Robert > > ------------------------------ > > Does anyone know of a good source on AI and Erlang? A book would be > awesome! Most of the sources that I see out there have Lisp as the > functional programming language of choice. Erlang seems like a better > option given it's nature of distributed nature and ability to place a ton > of processes in less space. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Wed May 2 17:24:40 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Wed, 2 May 2012 17:24:40 +0200 Subject: [erlang-questions] Artificial Intelligence and Erlang? In-Reply-To: <8442A289-D1A9-4DDC-B32E-BECC52FFE846@gmail.com> References: <4F9EAD5F.3010203@meetinghouse.net> <4F9EADC5.7040503@meetinghouse.net> <8442A289-D1A9-4DDC-B32E-BECC52FFE846@gmail.com> Message-ID: On Mon, Apr 30, 2012 at 5:39 PM, Zabrane Mickael wrote: > Hi Gled, > > Where can I find "eXat" and "Sereyse"? https://github.com/gleber/exat https://github.com/afiniate/seresye/ From zabrane3@REDACTED Wed May 2 17:31:05 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Wed, 2 May 2012 17:31:05 +0200 Subject: [erlang-questions] Artificial Intelligence and Erlang? In-Reply-To: References: <4F9EAD5F.3010203@meetinghouse.net> <4F9EADC5.7040503@meetinghouse.net> <8442A289-D1A9-4DDC-B32E-BECC52FFE846@gmail.com> Message-ID: Thanks Gleb ;-) /Zab On May 2, 2012, at 5:24 PM, Gleb Peregud wrote: > On Mon, Apr 30, 2012 at 5:39 PM, Zabrane Mickael wrote: >> Hi Gled, >> >> Where can I find "eXat" and "Sereyse"? > > https://github.com/gleber/exat > https://github.com/afiniate/seresye/ From madtrick@REDACTED Wed May 2 17:50:04 2012 From: madtrick@REDACTED (Farruco Sanjurjo) Date: Wed, 2 May 2012 17:50:04 +0200 Subject: [erlang-questions] Arguments to emulator in escript Message-ID: Hi, There's something with the options passed to the emulator within an escript file that I don't get to work properly. The problem is: If I add paths which contains glob expressions to the code path with -pa argument in the escript file, this paths don't get added. Example: #!/usr/bin/env escript %%! -pa ebin deps/*/ebin % % Escript file to exemplify my problem % main(_) -> lists:foreach(fun(E) -> io:format("Path ~s ~n", [E]) end, code:get_path()). When I execute this escript, this is the output: [madtrick@REDACTED { auth } (mock_auth_service *%) ruby-1.9.2-p290 ] >> ./ecript Path ebin Path . Path /usr/lib/erlang/lib/kernel-2.15/ebin Path /usr/lib/erlang/lib/stdlib-1.18/ebin Path /usr/lib/erlang/lib/xmerl-1.3/ebin Path /usr/lib/erlang/lib/wx-0.99.1/ebin Path /usr/lib/erlang/lib/webtool-0.8.9.1/ebin Path /usr/lib/erlang/lib/typer-0.9.3/ebin Path /usr/lib/erlang/lib/tv-2.1.4.8/ebin Path /usr/lib/erlang/lib/tools-2.6.6.6/ebin ... % output deleted As you can see, the ebin/ paths under deps/ don't get added to the code path. But, If I execute the emulator directly and use the same arguments those paths get added: [madtrick@REDACTED { auth } (mock_auth_service *%) ruby-1.9.2-p290 ] >>erl -pa ebin deps/*/ebin Erlang R15B (erts-5.9) [source] [64-bit] [smp:2:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9 (abort with ^G) 1> lists:foreach(fun(E) -> io:format("Path ~s ~n", [E]) end, code:get_path()). Path deps/reloader/ebin Path deps/rabbit_common/ebin Path deps/proper/ebin Path deps/piqi_src/ebin Path deps/piqi/ebin Path deps/meck/ebin Path deps/jiffy/ebin Path deps/hamcrest/ebin Path deps/espec/ebin Path deps/coello/ebin Path deps/amqp_client/ebin Path ebin Path . Path /usr/lib/erlang/lib/kernel-2.15/ebin Path /usr/lib/erlang/lib/stdlib-1.18/ebin Path /usr/lib/erlang/lib/xmerl-1.3/ebin Path /usr/lib/erlang/lib/wx-0.99.1/ebin Path /usr/lib/erlang/lib/webtool-0.8.9.1/ebin Path /usr/lib/erlang/lib/typer-0.9.3/ebin Path /usr/lib/erlang/lib/tv-2.1.4.8/ebin ... %output deleted As you can see, the ebin/ paths under deps/ get added. Any ideas? Thanks. Farruco Sanjurjo -------------- next part -------------- An HTML attachment was scrubbed... URL: From sgolovan@REDACTED Wed May 2 18:00:10 2012 From: sgolovan@REDACTED (Sergei Golovan) Date: Wed, 2 May 2012 20:00:10 +0400 Subject: [erlang-questions] Arguments to emulator in escript In-Reply-To: References: Message-ID: On Wed, May 2, 2012 at 7:50 PM, Farruco Sanjurjo wrote: > Hi, > > There's something with the options passed to the emulator within an escript > file that I don't get to work properly. The problem is: > > If I add paths which contains glob expressions to the code path with -pa > argument in the escript file, this paths don't get added. Example: > > #!/usr/bin/env escript > %%! -pa ebin deps/*/ebin The argument -pa ebin deps/*/ebin is subject to shell globbing if it's supplied to the erl command in bash/zsh whatever. So, Erlang VM sees -pa ebin deps/amqp_client/ebin ... deps/reloader/ebin and processes the entire list of directories. In escript -pa ebin deps/*/ebin goes to Erlang VM literally, and since there's no deps/*/ebin directory, it isn't appended to the code path at all. I don't know which is the best way of adding these directories to the code path. I'd use code:add_pathsa/1 directly in main/1. Cheers! -- Sergei Golovan From madtrick@REDACTED Wed May 2 18:06:30 2012 From: madtrick@REDACTED (Farruco Sanjurjo) Date: Wed, 2 May 2012 18:06:30 +0200 Subject: [erlang-questions] Arguments to emulator in escript In-Reply-To: References: Message-ID: On 2 May 2012 18:00, Sergei Golovan wrote: > On Wed, May 2, 2012 at 7:50 PM, Farruco Sanjurjo > wrote: > > Hi, > > > > There's something with the options passed to the emulator within an > escript > > file that I don't get to work properly. The problem is: > > > > If I add paths which contains glob expressions to the code path with -pa > > argument in the escript file, this paths don't get added. Example: > > > > #!/usr/bin/env escript > > %%! -pa ebin deps/*/ebin > > The argument -pa ebin deps/*/ebin is subject to shell globbing if it's > supplied to the erl command in bash/zsh whatever. So, Erlang VM sees > -pa ebin deps/amqp_client/ebin ... deps/reloader/ebin > and processes the entire list of directories. Yep, that make sense. Thanks : ) > > In escript -pa ebin deps/*/ebin goes to Erlang VM literally, and since > there's no deps/*/ebin directory, it isn't appended to the code path > at all. > > I don't know which is the best way of adding these directories to the > code path. I'd use code:add_pathsa/1 directly in main/1. > In the meanwhile I tried this: %%! -env ERL_LIBS=deps and it works for me > > Cheers! > -- > Sergei Golovan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Wed May 2 18:54:34 2012 From: bob@REDACTED (Bob Ippolito) Date: Wed, 2 May 2012 09:54:34 -0700 Subject: [erlang-questions] Most performant way to store/search a bag In-Reply-To: References: Message-ID: You could look at how gproc does it, or just use gproc. On Wednesday, May 2, 2012, AD wrote: > Hello, > > I have a problem where i need to map event_ids to connection objects (for > use in websockets to broadcast an event). I am trying to determine the > most efficient way to map 1 event_id to possibly several thousand > connection objects. Ultimately I would need to find every connection > object mapped to an ID and then send an event to each connection object. > Is using an ETS bag and then using ets:foldl() to loop through the most > optimal ? I was thinking of breaking down each event_type into a separate > ETS table and then storing the event_id for that event_type in the > respective ETS table with the connection object. > > Thoughts welcome. > > Thanks > -AD > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ciprian.craciun@REDACTED Wed May 2 20:39:34 2012 From: ciprian.craciun@REDACTED (Ciprian Dorin Craciun) Date: Wed, 2 May 2012 21:39:34 +0300 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: Message-ID: (As no-one replied in 4 days, I'll add my opinion...) On Sat, Apr 28, 2012 at 4:36 PM, Tristan Sloughter wrote: > I started with this problem as something to simply discuss with the > maintainers of certain build and package management projects -- sinan and > agner specifically. But it seems to be more broad and cover how all of us > who keep apps on github handle versioning and dependencies. > > The basic issue is app versions within .app files on branches in github, the > resultant directory from a agner install and the?discrepancies?this causes. > > [...] > > Am I the only one going crazy with this? Nop. I've been faced with this problem myself a couple of times. (I've chosen to ignore it.) > Does anyone have > suggestions/examples of what they do for production projects on teams to > ease these annoyances? For example for my project I have one big Git repository called `myproject-repositories` (replace `myproject` with what you need) with submodules pointing to the dependencies. Then inside my own project repositories I have a symlink `.repositories` poiting to the "current" snapshot of dependencies. I also don't use any of rebar, agner, etc., mainly because I've hacked something on-top of the ninja build system. > I mostly just create my own packages and repos of dependencies, or package > third party deps with the project. Yup. +1 :) P.S.: I hate Java from the bottom of my heart, but boy-o-boy is the Java tooling way better than that of Erlang... I mean what Erlang is missing is something similar to NodeJS's NPM, or Java's Maven (minus the XML baggage and all useless complexities), or Python setup-tools (with virtual environment), etc... From dmkolesnikov@REDACTED Wed May 2 21:14:17 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Wed, 2 May 2012 22:14:17 +0300 Subject: [erlang-questions] Most performant way to store/search a bag In-Reply-To: References: Message-ID: I've been looking into similar problem. I would not advice to partition ets by event type. You would not gain anything... a single ets should work. You are right that foldl over large table might be expensive: * with ordered_set ets and ets:next / ets:prev you might implement efficient "fold" through particular event type only. * with bag you can use select to retrieve a subset of keys according to match spec. You could look either gproc or https://github.com/fogfish/pts they might suite your needs. Dmitry On May 2, 2012, at 7:54 PM, Bob Ippolito wrote: > You could look at how gproc does it, or just use gproc. > > On Wednesday, May 2, 2012, AD wrote: > Hello, > > I have a problem where i need to map event_ids to connection objects (for use in websockets to broadcast an event). I am trying to determine the most efficient way to map 1 event_id to possibly several thousand connection objects. Ultimately I would need to find every connection object mapped to an ID and then send an event to each connection object. Is using an ETS bag and then using ets:foldl() to loop through the most optimal ? I was thinking of breaking down each event_type into a separate ETS table and then storing the event_id for that event_type in the respective ETS table with the connection object. > > Thoughts welcome. > > Thanks > -AD > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Wed May 2 22:03:44 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 2 May 2012 21:03:44 +0100 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: Message-ID: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> Eric Merit and I have had some lengthy discussions about this on the Erlware mailing list and have some ideas that I think are pretty solid. The thing is though, you don't just need tools - you also need people to package their stuff up using the tools. I'm sure this area will continue to improve over time though, especially as new tools (hopefully) emerge. On 2 May 2012, at 19:39, Ciprian Dorin Craciun wrote: > (As no-one replied in 4 days, I'll add my opinion...) > > On Sat, Apr 28, 2012 at 4:36 PM, Tristan Sloughter > wrote: >> I started with this problem as something to simply discuss with the >> maintainers of certain build and package management projects -- sinan and >> agner specifically. But it seems to be more broad and cover how all of us >> who keep apps on github handle versioning and dependencies. >> >> The basic issue is app versions within .app files on branches in github, the >> resultant directory from a agner install and the discrepancies this causes. >> >> [...] >> >> Am I the only one going crazy with this? > > Nop. I've been faced with this problem myself a couple of times. > (I've chosen to ignore it.) > > >> Does anyone have >> suggestions/examples of what they do for production projects on teams to >> ease these annoyances? > > For example for my project I have one big Git repository called > `myproject-repositories` (replace `myproject` with what you need) with > submodules pointing to the dependencies. > > Then inside my own project repositories I have a symlink > `.repositories` poiting to the "current" snapshot of dependencies. > > I also don't use any of rebar, agner, etc., mainly because I've > hacked something on-top of the ninja build system. > > >> I mostly just create my own packages and repos of dependencies, or package >> third party deps with the project. > > Yup. +1 :) > > > P.S.: I hate Java from the bottom of my heart, but boy-o-boy is > the Java tooling way better than that of Erlang... I mean what Erlang > is missing is something similar to NodeJS's NPM, or Java's Maven > (minus the XML baggage and all useless complexities), or Python > setup-tools (with virtual environment), etc... > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From tristan.sloughter@REDACTED Wed May 2 22:12:01 2012 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Wed, 2 May 2012 15:12:01 -0500 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> References: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> Message-ID: Yeah, the discussion between you and Eric seemed much larger of a goal and one that would require mass adoption. It was just more than I've been looking for. I can get packages installed pretty easily, and if people tagged versions (and the version of the app was the same as the name of the tag) I would have almost no problems. So while what has been discussed on the Erlware list is great, I think even just simple changes with no new tools could make a large difference. Tristan On Wed, May 2, 2012 at 3:03 PM, Tim Watson wrote: > Eric Merit and I have had some lengthy discussions about this on the > Erlware mailing list and have some ideas that I think are pretty solid. The > thing is though, you don't just need tools - you also need people to > package their stuff up using the tools. > > I'm sure this area will continue to improve over time though, especially > as new tools (hopefully) emerge. > > On 2 May 2012, at 19:39, Ciprian Dorin Craciun > wrote: > > > (As no-one replied in 4 days, I'll add my opinion...) > > > > On Sat, Apr 28, 2012 at 4:36 PM, Tristan Sloughter > > wrote: > >> I started with this problem as something to simply discuss with the > >> maintainers of certain build and package management projects -- sinan > and > >> agner specifically. But it seems to be more broad and cover how all of > us > >> who keep apps on github handle versioning and dependencies. > >> > >> The basic issue is app versions within .app files on branches in > github, the > >> resultant directory from a agner install and the discrepancies this > causes. > >> > >> [...] > >> > >> Am I the only one going crazy with this? > > > > Nop. I've been faced with this problem myself a couple of times. > > (I've chosen to ignore it.) > > > > > >> Does anyone have > >> suggestions/examples of what they do for production projects on teams to > >> ease these annoyances? > > > > For example for my project I have one big Git repository called > > `myproject-repositories` (replace `myproject` with what you need) with > > submodules pointing to the dependencies. > > > > Then inside my own project repositories I have a symlink > > `.repositories` poiting to the "current" snapshot of dependencies. > > > > I also don't use any of rebar, agner, etc., mainly because I've > > hacked something on-top of the ninja build system. > > > > > >> I mostly just create my own packages and repos of dependencies, or > package > >> third party deps with the project. > > > > Yup. +1 :) > > > > > > P.S.: I hate Java from the bottom of my heart, but boy-o-boy is > > the Java tooling way better than that of Erlang... I mean what Erlang > > is missing is something similar to NodeJS's NPM, or Java's Maven > > (minus the XML baggage and all useless complexities), or Python > > setup-tools (with virtual environment), etc... > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ciprian.craciun@REDACTED Wed May 2 22:13:17 2012 From: ciprian.craciun@REDACTED (Ciprian Dorin Craciun) Date: Wed, 2 May 2012 23:13:17 +0300 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> References: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> Message-ID: On Wed, May 2, 2012 at 11:03 PM, Tim Watson wrote: > Eric Merit and I have had some lengthy discussions about this on the Erlware mailing list and have some ideas that I think are pretty solid. I'm glad to hear this. (I'll give it a look.) > The thing is though, you don't just need tools - you also need people to package their stuff up using the tools. I agree about this. In fact I think currently there are a lot of tools with diverging solutions. Also I don't think that one "blessed" tool would be the final answer. I would have taken a somehow different road, similar maybe to how Go is going (although they do have the "one" tool): i.e. strict conventions. What do I mean is this: a) it would be nice to have a standardized way to specify "extra" options to compile an Erlang module. (We have the module attributes that we could use.) (the same for C sources); b) we already have a standard project layout; (i.e. `./src`, `./include`, etc.) c) we already have a standard project dependencies (i.e. the `app` file); All we need to do is be consistent in this convention, and then all the various build and packaging systems could adapt. From kunthar.gmail@REDACTED Wed May 2 22:15:06 2012 From: kunthar.gmail@REDACTED (kunthar.gmail@REDACTED) Date: Wed, 2 May 2012 13:15:06 -0700 (PDT) Subject: [erlang-questions] Fwd: Erlang Job for $5000 in Saint-Petersburg Message-ID: <1335989706.10016.YahooMailNeo@web121703.mail.ne1.yahoo.com> Hi, I do not think that words like "fucking" and similar are acceptable in mailing lists. Regardless the language English, Turkish they are written... You have to respect others even if some discussion goes off ?topic. +++ -------------- next part -------------- An HTML attachment was scrubbed... URL: From alisdairsullivan@REDACTED Wed May 2 22:31:26 2012 From: alisdairsullivan@REDACTED (alisdair sullivan) Date: Wed, 2 May 2012 13:31:26 -0700 (PDT) Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: Message-ID: <1335990686.53204.YahooMailNeo@web125902.mail.ne1.yahoo.com> > I also don't use any of rebar, agner, etc., mainly because I've > hacked something on-top of the ninja build system. > P.S.: I hate Java from the bottom of my heart, but boy-o-boy is > the Java tooling way better than that of Erlang... I mean what Erlang > is missing is something similar to NodeJS's NPM, or Java's Maven > (minus the XML baggage and all useless complexities), or Python > setup-tools (with virtual environment), etc... the erlang tooling situation is really dire. there's some powerful stuff hidden away in the standard release, but it's accessibility is poor. common_test, in particular, has awful documentation that's only useful if you already know how to use common_test rebar and sinan were a huge leap forward from the previous 'standard' of emakefiles and hand rolled build systems, but why is our primary interface to living images (in the smalltalk sense) a collection of command line scripts? rebar and sinan's functionality should be as easily accessible from within a running node as it is from the command line. we use primitive tools like reloader.erl when we should have build environments that make concerns like reloading code irrelevant. you shouldn't need to roll a full release to manage name clashes in applications, registered processes and loaded code what the erlang ecosystem needs is a distributed repository of applications and releases, and better tools for working with them. forget cramming a bunch of stuff you found on github into your path and hoping for the best i'm mostly all talk, but i did do this: https://github.com/talentdeficit/erlenv that's just a tiny step, but hopefully it helps start the process From gianfranco.alongi@REDACTED Wed May 2 22:50:11 2012 From: gianfranco.alongi@REDACTED (Gianfranco Alongi) Date: Wed, 2 May 2012 22:50:11 +0200 Subject: [erlang-questions] Fwd: Erlang Job for $5000 in Saint-Petersburg In-Reply-To: References: <4F990A78.5060501@rabbitmq.com> <4F9910A3.2030500@llaisdy.com> <8FEF2536-1F35-442C-BFD1-7AF4BD2FF281@cs.otago.ac.nz> <015401cd27cb$418bd5a0$c4a380e0$@com> <4FA12949.7030308@llaisdy.com> Message-ID: Can I please remind everyone that as fun as this might be, the Erlang mailing list is for Erlang topics. Respect this. Please continue your language discussions elsewhere. /G On Wed, May 2, 2012 at 3:20 PM, Kunthar wrote: > Hi all, > > As you can easily realize, we are monitoring this list from all over > the world speaking different languages for Erlang based events. > > I would like to give another example from my own country for audience > to have fun: > Siz Erlang ile ilgili bir listede siktirik dilbilim yorumlar? yaparken, > snob ve en k???k bir katk? sa?lamayan angut mesajlar?n?za > makak maymunlar? bile g?t?yle g?l?yor, sizi erkek egemen dallamalar > s?r?s? sizi :) > > > Be well brothers > > > > > On Wed, May 2, 2012 at 3:32 PM, Ivan Uemlianin wrote: >> Some dialects of British English have "youse" (pronounced a bit like "use" >> but with a shorter vowel). >> >> >>> On Tue, May 1, 2012 at 9:03 PM, Jesse Gumm ?wrote: >>>> >>>> Or, for that matter, a non-ambiguous second person plural. I'm >>>> personally fine with "y'all" despite not being from the American >>>> South. >> >> >> -- >> ============================================================ >> Ivan A. Uemlianin PhD >> Llaisdy >> Speech Technology Research and Development >> >> ? ? ? ? ? ? ? ? ? ?ivan@REDACTED >> ? ? ? ? ? ? ? ? ? ? www.llaisdy.com >> ? ? ? ? ? ? ? ? ? ? ? ? llaisdy.wordpress.com >> ? ? ? ? ? ? ?github.com/llaisdy >> ? ? ? ? ? ? ? ? ? ? www.linkedin.com/in/ivanuemlianin >> >> ? ?"Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" >> ? ? ? ? ? ? ? ? ? ? (Schiller, Beethoven) >> ============================================================ >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > BR, > \|/ Kunthar > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From gleber.p@REDACTED Wed May 2 22:57:28 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Wed, 2 May 2012 22:57:28 +0200 Subject: [erlang-questions] Artificial Intelligence and Erlang? In-Reply-To: References: <4F9EAD5F.3010203@meetinghouse.net> <4F9EADC5.7040503@meetinghouse.net> <8442A289-D1A9-4DDC-B32E-BECC52FFE846@gmail.com> Message-ID: On Wed, May 2, 2012 at 5:24 PM, Gleb Peregud wrote: > On Mon, Apr 30, 2012 at 5:39 PM, Zabrane Mickael wrote: >> Hi Gled, >> >> Where can I find "eXat" and "Sereyse"? > > https://github.com/gleber/exat > https://github.com/afiniate/seresye/ There's one more agent / rule processing engine combo available: https://github.com/vjache/elips Haven't tried it myself. From per.melin@REDACTED Wed May 2 23:09:52 2012 From: per.melin@REDACTED (Per Melin) Date: Wed, 2 May 2012 23:09:52 +0200 Subject: [erlang-questions] Erlang and FIX protocol In-Reply-To: References: Message-ID: <9DDA4A0F-4321-40F7-A896-3B41E737F60D@gmail.com> On Apr 26, 2012, at 07:00 , Max Lapshin wrote: > Seems that since May 2007 (http://erlang.2086793.n4.nabble.com/FIX-protocol-td2096116.html) nothing had changed about erlang and FIX protocol, http://fixprotocol.org/ > > Everyone is rewriting their own parser/encoder? I wrote my own some years ago. I only needed a subset of the functionality but felt ambitious and set out to do a complete implementation of FIX 4.4. By the time the parser was almost done I was so disgusted with the protocol that I lowered my ambition. For the rest I just did the bare minimum of what I needed. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Wed May 2 23:45:10 2012 From: jwatte@REDACTED (Jon Watte) Date: Wed, 2 May 2012 14:45:10 -0700 Subject: [erlang-questions] Frames proposal In-Reply-To: <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> Message-ID: > > The implied mapping takes JSON {id: 4135, title: "title2"} I'd just like to chime in that this is not actually valid JSON. In valid JSON, all keys are quoted strings. Thus, this would be: {"id": 4135, "title": "title2"} Which is not at all as good a match with Erlang idioms, unless you want more magic to turn strings into atoms, only when used as keys... Sincerely, jw Sincerely, Jon Watte -- "I pledge allegiance to the flag of the United States of America, and to the republic for which it stands, one nation indivisible, with liberty and justice for all." ~ Adopted by U.S. Congress, June 22, 1942 -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Wed May 2 23:48:31 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 02 May 2012 22:48:31 +0100 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: <1335990686.53204.YahooMailNeo@web125902.mail.ne1.yahoo.com> References: <1335990686.53204.YahooMailNeo@web125902.mail.ne1.yahoo.com> Message-ID: <4FA1ABAF.20605@gmail.com> On 02/05/2012 21:31, alisdair sullivan wrote: > what the erlang ecosystem needs is a distributed repository of applications > and releases, and better tools for working with them. forget cramming a bunch > of stuff you found on github into your path and hoping for the best This is exactly what Eric Merrit and I have been talking about. Hopefully something will eventually come of it, but in the meanwhile if you care about this stuff, please consider taking a look at the packaging and distribution threads on erlware-questions! > i'm mostly all talk, but i did do this: https://github.com/talentdeficit/erlenv > > that's just a tiny step, but hopefully it helps start the process > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From tuncer.ayaz@REDACTED Thu May 3 00:14:57 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Thu, 3 May 2012 00:14:57 +0200 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: Message-ID: On Sat, Apr 28, 2012 at 3:36 PM, Tristan Sloughter wrote: > I started with this problem as something to simply discuss with the > maintainers of certain build and package management projects -- > sinan and agner specifically. But it seems to be more broad and > cover how all of us who keep apps on github handle versioning and > dependencies. > > The basic issue is app versions within .app files on branches in > github, the resultant directory from a agner install and the > discrepancies this causes. > > For an example take an app like Cowboy that its .app files in the > master branch has {vsn, "0.5.0"} but this is not really 0.5.0. That > is a tagged version. What is in master is a version beyond 0.5.0. > > An agner install of this results in lib/cowboy-@REDACTED with the .app > file still being vsn 0.5.0, of course. > > Rebar has a partial work around for these where you can put {vsn > git} in your .app.src file and it will populate it on build with > part of the git hash for the repo at that time. This I think could > be part of the ultimate solution. Though another problem arises for > cut versions put into tags, like if you simply download > https://github.com/boundary/folsom/zipball/0.6 you can not build it > by default because it is not a git repo. And if you pull the tag and > build it (as Agner does for when you do a 'agner install folsom -v > 0.6' the version is not 0.6 in the .app as you'd expect, but instead > 0.6-. Thus I had to create an agner package for the tag > 0.6-51-gdaa75cb that points to the 0.6 tag of folsom -- other wise > the dir name and the actual app version would not match. > > So I've been doing many work arounds for this for my needs, none of > them yet satisfying. > > For real development of an Erlang project on a team its important to > keep versions of dependencies in sync AND accurate. I've not found a > good flow for this with the current environment of Erlang apps from > their main source, github. > > Am I the only one going crazy with this? haha Does anyone have > suggestions/examples of what they do for production projects on > teams to ease these annoyances? > > I mostly just create my own packages and repos of dependencies, or > package third party deps with the project. Maybe Hackage for Erlang with delta index download and using a mirror network? Maybe use xref and typespec tools to implement advanced search (not same but inspired by Hoogle)? From matthewsackslists@REDACTED Thu May 3 00:40:40 2012 From: matthewsackslists@REDACTED (Matthew Sacks) Date: Wed, 2 May 2012 15:40:40 -0700 Subject: [erlang-questions] Copyright Request to Use OTP Documentation as an Example Message-ID: Hello List, I was wondering if there is anyone who might advise me on using a page of the Erlang documentation as an example for different kinds of documentation to be used in my book http://www.amazon.com/Pro-Website-Development-Operations-Streamlining/dp/1430239697. Please advise if you know whom or where I might make a request. I suppose Ericcson AB legal? Thanks, Matthew -------------- next part -------------- An HTML attachment was scrubbed... URL: From toby@REDACTED Thu May 3 01:09:51 2012 From: toby@REDACTED (Toby Thain) Date: Wed, 02 May 2012 19:09:51 -0400 Subject: [erlang-questions] Artificial Intelligence and Erlang? In-Reply-To: References: Message-ID: <4FA1BEBF.5000405@telegraphics.com.au> On 02/05/12 11:19 AM, Robert Virding wrote: > I don't know if you would class this as AI but I have a Prolog > interpreter written in Erlang. It is a proper subset of standard Prolog Where can we find this? thanks. --Toby > with unification and real database handling and interacts well with > Erlang. Prolog is very good for rule-based and knowledge based systems > (duh!). An alternative would be to connect to a real prolog system. > > I also know that there should be a book on using Erlang and neural-nets > in the pipeline, though I have no idea when it will be available. I am > waiting though. > > Robert > > Does anyone know of a good source on AI and Erlang? A book would be > awesome! Most of the sources that I see out there have Lisp as the > functional programming language of choice. Erlang seems like a > better option given it's nature of distributed nature and ability to > place a ton of processes in less space. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From gleber.p@REDACTED Thu May 3 01:26:54 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Thu, 3 May 2012 01:26:54 +0200 Subject: [erlang-questions] Artificial Intelligence and Erlang? In-Reply-To: <4FA1BEBF.5000405@telegraphics.com.au> References: <4FA1BEBF.5000405@telegraphics.com.au> Message-ID: On Thu, May 3, 2012 at 1:09 AM, Toby Thain wrote: > On 02/05/12 11:19 AM, Robert Virding wrote: >> I don't know if you would class this as AI but I have a Prolog >> interpreter written in Erlang. It is a proper subset of standard Prolog > Where can we find this? I believe it is this one: https://github.com/rvirding/erlog From robert.virding@REDACTED Thu May 3 01:50:44 2012 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 03 May 2012 00:50:44 +0100 (BST) Subject: [erlang-questions] Artificial Intelligence and Erlang? In-Reply-To: Message-ID: <63e07273-9ba1-48d5-b349-52b7104d7d51@knuth> ----- Original Message ----- > On Thu, May 3, 2012 at 1:09 AM, Toby Thain > wrote: > > On 02/05/12 11:19 AM, Robert Virding wrote: > >> I don't know if you would class this as AI but I have a Prolog > >> interpreter written in Erlang. It is a proper subset of standard > >> Prolog > > Where can we find this? > > I believe it is this one: > > https://github.com/rvirding/erlog That's the one. It is dormant at the moment, but not definitely not dead. What it needs is someone to use it and come with suggestions, especially about the interface with Erlang. It is still only an interpreter, though I know how to compile it. Compiling would make it faster though I haven't done any serious tests to work out how much faster. I will gladly put more time into it if someone wants to use it. Robert > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From robert.virding@REDACTED Thu May 3 01:52:20 2012 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 03 May 2012 00:52:20 +0100 (BST) Subject: [erlang-questions] Copyright Request to Use OTP Documentation as an Example In-Reply-To: Message-ID: <9d4a2bed-30a7-4f4c-82cf-6ed0adfba6cd@knuth> First stop is definitely Kenneth Lundin who runs the OTP group at Ericsson. kenneth.lundin@REDACTED Robert ----- Original Message ----- > Hello List, > I was wondering if there is anyone who might advise me on using a > page of the Erlang documentation as an example for different kinds > of documentation to be used in my book > http://www.amazon.com/Pro-Website-Development-Operations-Streamlining/dp/1430239697 > . > Please advise if you know whom or where I might make a request. I > suppose Ericcson AB legal? > Thanks, > Matthew > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Thu May 3 02:04:05 2012 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 03 May 2012 01:04:05 +0100 (BST) Subject: [erlang-questions] Erlounge in Houston? Message-ID: <7fa4790d-d69f-4122-b73f-aac7b5640feb@knuth> Hi, I will be in Houston, Texas 13-26 May giving a course. Are there any erlangers in or around Houston who would like to have an erlounge while I am there. It is a bit of a short notice for a Factory Lite I'm afraid. Also what can you recommend I like go see as a tourist while there? Robert -- Robert Virding, Erlang Solutions Ltd. From watson.timothy@REDACTED Thu May 3 02:06:45 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 03 May 2012 01:06:45 +0100 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> Message-ID: <4FA1CC15.9090305@gmail.com> On 02/05/2012 21:12, Tristan Sloughter wrote: > Yeah, the discussion between you and Eric seemed much larger of a goal > and one that would require mass adoption. > This is why we want to introduce 3rd party signing - so you can build + upload stuff you depend on that isn't necessarily being published by the original author(s). Of course our hope is that if lots of people want to use riak/lager/etc and decide to self-sign them so they can be used as dependencies, then the companies/teams behind those projects will be encouraged to join in. > It was just more than I've been looking for. I can get packages > installed pretty easily, and if people tagged versions (and the > version of the app was the same as the name of the tag) I would have > almost no problems. > > So while what has been discussed on the Erlware list is great, I think > even just simple changes with no new tools could make a large difference. > Sure that seems fair enough. The issue of tagging is absolutely vital and I find it shocking that people aren't maintaining versions of things that intended for general use. I guess that's because I've spend enough time in a 'corporate' environment to become indoctrinated about such things, but generally I think it is The Right Thing To Do (TM). The fact that people are maintaining custom forks of projects in order to be able to rely on them such seems totally wrong to me - a misuse in that forking should be about introducing your own changes (whether they end up going back upstream or not), but not about having to impose some kind of artificial snapshot on external dependencies. > Tristan > > On Wed, May 2, 2012 at 3:03 PM, Tim Watson > wrote: > > Eric Merit and I have had some lengthy discussions about this on > the Erlware mailing list and have some ideas that I think are > pretty solid. The thing is though, you don't just need tools - you > also need people to package their stuff up using the tools. > > I'm sure this area will continue to improve over time though, > especially as new tools (hopefully) emerge. > > On 2 May 2012, at 19:39, Ciprian Dorin Craciun > > wrote: > > > (As no-one replied in 4 days, I'll add my opinion...) > > > > On Sat, Apr 28, 2012 at 4:36 PM, Tristan Sloughter > > > wrote: > >> I started with this problem as something to simply discuss with the > >> maintainers of certain build and package management projects -- > sinan and > >> agner specifically. But it seems to be more broad and cover how > all of us > >> who keep apps on github handle versioning and dependencies. > >> > >> The basic issue is app versions within .app files on branches > in github, the > >> resultant directory from a agner install and the discrepancies > this causes. > >> > >> [...] > >> > >> Am I the only one going crazy with this? > > > > Nop. I've been faced with this problem myself a couple of times. > > (I've chosen to ignore it.) > > > > > >> Does anyone have > >> suggestions/examples of what they do for production projects on > teams to > >> ease these annoyances? > > > > For example for my project I have one big Git repository called > > `myproject-repositories` (replace `myproject` with what you > need) with > > submodules pointing to the dependencies. > > > > Then inside my own project repositories I have a symlink > > `.repositories` poiting to the "current" snapshot of dependencies. > > > > I also don't use any of rebar, agner, etc., mainly because I've > > hacked something on-top of the ninja build system. > > > > > >> I mostly just create my own packages and repos of dependencies, > or package > >> third party deps with the project. > > > > Yup. +1 :) > > > > > > P.S.: I hate Java from the bottom of my heart, but boy-o-boy is > > the Java tooling way better than that of Erlang... I mean what > Erlang > > is missing is something similar to NodeJS's NPM, or Java's Maven > > (minus the XML baggage and all useless complexities), or Python > > setup-tools (with virtual environment), etc... > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Thu May 3 02:18:16 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 03 May 2012 01:18:16 +0100 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> Message-ID: <4FA1CEC8.1060807@gmail.com> On 02/05/2012 21:13, Ciprian Dorin Craciun wrote: > On Wed, May 2, 2012 at 11:03 PM, Tim Watson wrote: >> Eric Merit and I have had some lengthy discussions about this on the Erlware mailing list and have some ideas that I think are pretty solid. > I'm glad to hear this. (I'll give it a look.) Cool thanks - please do leave any feedback you feel is relevant. >> The thing is though, you don't just need tools - you also need people to package their stuff up using the tools. > I agree about this. In fact I think currently there are a lot of > tools with diverging solutions. > > Also I don't think that one "blessed" tool would be the final > answer. I would have taken a somehow different road, similar maybe to > how Go is going (although they do have the "one" tool): i.e. strict > conventions. Yes I agree that '1 tool to rule them all' isn't going to work. Eric and I have discussed building a unix-y pipeline of tools to deal with local/remote package and repository management, dependency resolution, installing and/or generating a code path (or some other structure) for bootstrapping either ERL_LIBS or a call to code:path, etc. The assumption we have is that various parts can be dealt with be different tools providing they respect the APIs, so you can build with rebar or sinan or make or whatever. We are also planning on using reltool (or some alternative/replica of sorts) to deal with the packaging bits, as well as a set of tools for publication, code/package signing and the like. We would also like to support different release types (e.g., embedded vs. non), distinguish between applications and library applications and so on. > > What do I mean is this: > a) it would be nice to have a standardized way to specify "extra" > options to compile an Erlang module. (We have the module attributes > that we could use.) (the same for C sources); > b) we already have a standard project layout; (i.e. `./src`, > `./include`, etc.) > c) we already have a standard project dependencies (i.e. the `app` file); > > All we need to do is be consistent in this convention, and then > all the various build and packaging systems could adapt. Packaging wise, I don't see how any of this really helps so much, although I completely agree that sticking to this consistent layout is a good thing. Using the app file for dependencies is fine at runtime - I do this in https://github.com/hyperthunk/appstart - but if you're connecting to a (possibly remote) artefact repository and fetching stuff, then you need to specify the publisher/signer (because 2 OTP applications could be written with the same name, and of course with FOSS project forking this happens all the time), the application name, and the version. You can't whack all of that into your .app file without breaking various things. From ttmrichter@REDACTED Thu May 3 03:57:13 2012 From: ttmrichter@REDACTED (Michael Richter) Date: Thu, 3 May 2012 09:57:13 +0800 Subject: [erlang-questions] Artificial Intelligence and Erlang? In-Reply-To: <63e07273-9ba1-48d5-b349-52b7104d7d51@knuth> References: <63e07273-9ba1-48d5-b349-52b7104d7d51@knuth> Message-ID: If it's Prolog-in-Erlang you're looking for, take a look at Mercury ( http://mercurylang.org). Mercury is a purely declarative dialect of Prolog, in effect, that compiles to a variety of back-ends including Erlang. -- "Perhaps people don't believe this, but throughout all of the discussions of entering China our focus has really been what's best for the Chinese people. It's not been about our revenue or profit or whatnot." --Sergey Brin, demonstrating the emptiness of the "don't be evil" mantra. -------------- next part -------------- An HTML attachment was scrubbed... URL: From straightflush@REDACTED Thu May 3 05:35:13 2012 From: straightflush@REDACTED (AD) Date: Wed, 2 May 2012 23:35:13 -0400 Subject: [erlang-questions] Most performant way to store/search a bag In-Reply-To: References: Message-ID: gproc seems to use an ordered_set ETS table and then an ets:select() to match against the keys it needs. I did some basic tests with a 5 million row ETS ordered set table and ets:foldl still seems to come out faster. Not sure if my simple test script is bogus or what but the results were consistent. https://gist.github.com/2582918 -AD On Wed, May 2, 2012 at 3:14 PM, Dmitry Kolesnikov wrote: > I've been looking into similar problem. I would not advice to partition > ets by event type. You would not gain anything... a single ets should work. > You are right that foldl over large table might be expensive: > > * with ordered_set ets and ets:next / ets:prev you might implement > efficient "fold" through particular event type only. > > * with bag you can use select to retrieve a subset of keys according to > match spec. > > You could look either gproc or https://github.com/fogfish/pts they might > suite your needs. > > Dmitry > > > > > > > > On May 2, 2012, at 7:54 PM, Bob Ippolito wrote: > > You could look at how gproc does it, or just use gproc. > > On Wednesday, May 2, 2012, AD wrote: > >> Hello, >> >> I have a problem where i need to map event_ids to connection objects >> (for use in websockets to broadcast an event). I am trying to determine >> the most efficient way to map 1 event_id to possibly several thousand >> connection objects. Ultimately I would need to find every connection >> object mapped to an ID and then send an event to each connection object. >> Is using an ETS bag and then using ets:foldl() to loop through the most >> optimal ? I was thinking of breaking down each event_type into a separate >> ETS table and then storing the event_id for that event_type in the >> respective ETS table with the connection object. >> >> Thoughts welcome. >> >> Thanks >> -AD >> > _______________________________________________ > > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu May 3 06:22:50 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 3 May 2012 16:22:50 +1200 Subject: [erlang-questions] New draft of frames proposal (was Re: Frames proposal) In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> Message-ID: There is a new version of http://www.cs.otago.ac.nz/staffpriv/ok/frames.pdf - a couple of diagrams - some new material, including a description of what is wrong with using X.y to access a field - a new section 2 apologising for the rewrite this needs and hasn't had yet - a new section 15 (starting on page 41) which describes in some detail how it works with new BEAM(-ish) instructions put_frame allocate a frame is_frame give true/false result test_frame used in clause heads check_frame used in expressions load_slot load a slot (with inline cache) copy_frame copy a frame for updating store_slot store a slot (with inline cache) On 2/05/2012, at 5:19 AM, Bj?rn-Egil Dahlberg wrote: > Syntax is more than just finding which characters our parser can identify, like in the frames proposals where ~ is our separator. When I read <{ k ~ value | Is }> it says cons "k is like a value" to Is. *brr* In fact the semantics very closely related to the semantics of `((k ,@REDACTED) ,@REDACTED) considered as an association list in Scheme, which really does involve a cons. Let's face it, Is#{k=value} says that k *equals* value, which it doesn't. There is a uniquely right character for the job, and that's ? . > What guarantees should we leave? Any data structure associated with Phil Bagwell is likely to repay study. I do hope there is no patent on "Cache-Aware Lock-Free Concurrent Hash Tries" by Aleksandar Prokopec, Phil Bagwell, and Martin Odersky, because that's _exactly_ what I want for storing frame descriptors. However, the frame implementation described in the frames proposal is meant as a replacement for records, and however good HAMTs are at other things, frames should leave them in the dust *when used as record replacements*. From ulf@REDACTED Thu May 3 06:57:32 2012 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 3 May 2012 06:57:32 +0200 Subject: [erlang-questions] Most performant way to store/search a bag In-Reply-To: References: Message-ID: <609A9F79-20D8-4431-AD30-E552558FC463@feuerlabs.com> 3 maj 2012 kl. 05:35 skrev AD : > gproc seems to use an ordered_set ETS table and then an ets:select() to match against the keys it needs. I did some basic tests with a 5 million row ETS ordered set table and ets:foldl still seems to come out faster. The key to good select() performance in an ordered_set table is to ensure that the first part of the key pattern is bound. In your test, the key pattern was a free variable, forcing the match spec evaluator to test every single record in the table. Gproc strives to do 'bounded selects' as much as possible. BR, Ulf W Ulf Wiger, Feuerlabs, Inc. http://www.feuerlabs.com From ok@REDACTED Thu May 3 07:12:50 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 3 May 2012 17:12:50 +1200 Subject: [erlang-questions] Frames proposal In-Reply-To: <4FA0F558.4080400@fastmail.fm> References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> <0AC65FD9-71BD-4D04-B0F6-524DB61CF095@cs.otago.ac.nz> <4FA0F558.4080400@fastmail.fm> Message-ID: On 2/05/2012, at 8:50 PM, Jan Burse wrote: > I was more concerned with usage and complexity. Your frames have > been invented many many times. I know that. The frames proposal SAYS that with specifics. I haven't seen this particular implementation used for this particular purpose, but it's not novel either. > I remember also some proposals > that retain the relational character, so that a pattern <{k1~v1, > k2~vn|F}> with a tail frame would become a first class citizen > of the language. These are then not called frames but rather > feature structures. Actually, they're called ?-terms. > > I guess one way to implement real feature structures in a logic > programming language It has been done in at least two different ways. LIFE (Logic, Inheritance, Functions, and Equations) was described by Hassan A?t-Kaci and Patrick Lincoln in 1989. It was based on LOGIN, described in 3. A ??t-Kaci, H. and Nasr, R., ?LOGIN: A Logic Programming Language with Built-in Inheritance.? Journal of Logic Programming 3(3), pp. 187?215. 1986. (Bob Carpenter's "The Logic of Typed Feature Structures: With Applications to Unification Grammars, Logic Programs And Constraint Resolution) was published nearly 20 years later. There was an interpreter for LIFE written in C called Wild LIFE which can still be downloaded if you search for it (the link on Peter van Roy's page appears to be dead). I found it at http://rapidlibrary.com/files/life1-01-tar-z_ulzttmxtv8i89on.html I was very unhappy when I learned that LIFE was dead. The Wild LIFE system can't be compiled by an ANSI C compiler. One can figure out by reading the source code that it uses binary trees to represent ?-terms. The other way that I know of is what Michael Covington did in his book about natural language processing in Prolog, which is what I did independently, and which I *suspect* but don't know that Pl-PATR did as well, which is to use equations on paths rather than displaying feature structures as ?-terms, and precompiling the grammar so that the paths work on plain on Prolog terms. >>Feature structures<< may be open, but the set of >>feature names<< is closed. > > The simple syntax there for feature structures is [k1:v1, > k2:vn|F]. The syntax for ?-terms looks like label(a => U, b => V) with the 'tail' implied. > And it seems to be rather the rule than the exception > that a feature structure is open. Yes. But this is >>Erlang<< we are talking about, not Prolog, and not LIFE, and not linguistics. Erlang does not use unification. > (*) > Compiling Feature Structures into Terms: > an Empirical Study in Prolog (1993) > by Andreas Sch?ter , Buccleuch Place > http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.24.8407 You will note that this comes two years _after_ WildLIFE was released and at least seven years after A?t-Kaci's work. "Compiling to flat terms" is the technique I mentioned using above; it is so obvious, so easy, and so effective that it's a wonder to me that anyone ever used anything else. In fact it was SO obvious and SO easy that it never occurred to me that it was worth a publication. (Erlang readers: this is actually very very close in spirit to the way -record declarations are implemented now. Savour the irony!) Anyway, the *POINT* of the frames proposal is not to innovate but to take a well established idea and adapt the details to make it suitable for >>>replacing records<<< in a distributed functional language. It's like a mouse's nest in a cat's ear: only the location is new. From dmkolesnikov@REDACTED Thu May 3 07:38:55 2012 From: dmkolesnikov@REDACTED (dmitry kolesnikov) Date: Thu, 3 May 2012 08:38:55 +0300 Subject: [erlang-questions] Most performant way to store/search a bag In-Reply-To: <609A9F79-20D8-4431-AD30-E552558FC463@feuerlabs.com> References: <609A9F79-20D8-4431-AD30-E552558FC463@feuerlabs.com> Message-ID: <7560917348206531539@unknownmsgid> Hi, In your test script select does a full table scan, emulates foldl. But your use-case is foldl through subset. Should you measure performance of that case? Best Regards, Dmitry >-|-|-*> On 3.5.2012, at 7.57, Ulf Wiger wrote: > 3 maj 2012 kl. 05:35 skrev AD : > >> gproc seems to use an ordered_set ETS table and then an ets:select() to match against the keys it needs. I did some basic tests with a 5 million row ETS ordered set table and ets:foldl still seems to come out faster. > > > The key to good select() performance in an ordered_set table is to ensure that the first part of the key pattern is bound. In your test, the key pattern was a free variable, forcing the match spec evaluator to test every single record in the table. > > Gproc strives to do 'bounded selects' as much as possible. > > BR, > Ulf W > > Ulf Wiger, Feuerlabs, Inc. > http://www.feuerlabs.com From corticalcomputer@REDACTED Thu May 3 10:03:33 2012 From: corticalcomputer@REDACTED (G.S.) Date: Thu, 3 May 2012 04:03:33 -0400 Subject: [erlang-questions] Artificial Intelligence and Erlang? In-Reply-To: References: <63e07273-9ba1-48d5-b349-52b7104d7d51@knuth> Message-ID: Hello, I am the author of the mentioned upcoming book on Neural Network based systems through Erlang, the book title is: Handbook of Neuroevolution Through Erlang. Springer is my publisher, the book will be in print towards the end of 3rd quarter of 2012. The book is currently in the Springer's production pipeline, print version ISBN is: 978-1-4614-4462-6. Finally, I am also very happy to note that the foreword to the book is written by Joe Armstrong. I do research in Universal Learning Networks and Neuroevolution based systems: www.dxnnresearch.com. The following are the slides from this year's Erlang Factory conference: https://www.erlang-factory.com/conference/SFBay2012/speakers/GeneSher. The original DXNN (not a lot of comments are present in the source code, since DXNN1 I primarily use myself) source code is available on GitHub, DXNN2 has a cleaner architecture than V1, significantly more documentation, and is the one built from ground up and explored in my book. Source code for DXNN2 should be available on GitHub shortly. Here are the old videos of my system evolving complex behaviours in artificial life experiments, using my Flatland 2d ALife simulator (built using gs): http://www.youtube.com/watch?v=HzsDZt8EO70&list=UUdBTNtB1C3Jt90X1I26Vmhg&index=2&feature=plcpAlso, a number of preprints on the DXNN system are available on arxiv. If you're interested in computational intelligence systems built in Erlang, DXNN is a solid choice. DXNN has direct and indirect NN encoding, evolutionary/memetic algorithm based learning, neural plasticity (Hebbian, Oja's, Neuromodulation, substrate-freeform rules...), modularity, and is dynamic and has content drift tracking and new sensor/actuator evolution and integration. Thus, it's a strong, versatile, and a highly scalable contender for applications in ALife, Robotics,... Best regards, -Gene ps. I'll be at this year's Genetic and Evolutionary Computation Conference (GECCO), so if any of you plan on going, let me know. On Wed, May 2, 2012 at 9:57 PM, Michael Richter wrote: > If it's Prolog-in-Erlang you're looking for, take a look at Mercury ( > http://mercurylang.org). Mercury is a purely declarative dialect of > Prolog, in effect, that compiles to a variety of back-ends including Erlang. > > -- > "Perhaps people don't believe this, but throughout all of the discussions > of entering China our focus has really been what's best for the Chinese > people. It's not been about our revenue or profit or whatnot." > --Sergey Brin, demonstrating the emptiness of the "don't be evil" mantra. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From silver.buyer@REDACTED Thu May 3 10:29:12 2012 From: silver.buyer@REDACTED (Joseph Goldberg) Date: Thu, 3 May 2012 01:29:12 -0700 (PDT) Subject: [erlang-questions] websockets and streaming quote help Message-ID: <1336033752.71089.YahooMailNeo@web121605.mail.ne1.yahoo.com> Can someone help me getting user authentication done with websockets and rabbitmq to consume and distribute streaming quotes to my subscribed users. I have a wave file of our conversation over skype, where he gives other details. ?http://armstrongonsoftware.blogspot.ca/2009/12/comet-is-dead-long-live-websockets.html http://blog.dberg.org/2012/04/using-gproc-and-cowboy-to-pass-messages.html [6:46:27 PM] Adam Denenberg: Our last step is to actually broadcast a message that contains this format. ?Now, here is where you need to roll your own logic to determine how you want to ingest messages (queue, webservice, etc), but once your app gets the message you are waiting for all you need is one line to send the message: gproc:send({p, l, ?WSKey}, {self(), ?WSKey, Msg}) [6:46:44 PM] Adam Denenberg: This simply sends message "Msg" to all the processes that were registered with the key ?WSKey and then gets handled in websocket_info and thus passed over the websocket connection. also need fix protocol?implementation?done with MQ and sequence number checking which he does not do http://www.codeproject.com/Articles/35439/Fix-Protocol-C-Fix-Engine-Source-Code Yes that's true. but tcpip messages are not erroneus/corrupted. the point of this code is to be fast - eg i do not verify the checksum. in fact i don't even bother to check the incoming message sequence number. i just have two safety checks in the code which check the message starts 8= and then 35=. -------------- next part -------------- An HTML attachment was scrubbed... URL: From janburse@REDACTED Thu May 3 10:56:09 2012 From: janburse@REDACTED (Jan Burse) Date: Thu, 03 May 2012 10:56:09 +0200 Subject: [erlang-questions] New draft of frames proposal (was Re: Frames proposal) In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> Message-ID: <4FA24829.1020305@fastmail.fm> Richard O'Keefe schrieb: > There is a new version of > http://www.cs.otago.ac.nz/staffpriv/ok/frames.pdf Pitty the sections do not have a @date, now I don't know what I have already read. BTW: Inline cache, when your lookup would return a pointer to an immutable {func,class}, than you can cache this in one STORE op, making it amenable to concurrent use. See to it that you first use one LOAD op to retrieve this pointer into a local register, before checking class and then possibly using func. Works with both 64-bit and 32-bit, no need to pack two 32-bit into one 64-bit. (I am using the same for {fun,arity} for a cache inside an atom, works like a charm) > It's like a mouse's nest in a cat's ear LoL, poor kitty From janburse@REDACTED Thu May 3 11:02:31 2012 From: janburse@REDACTED (Jan Burse) Date: Thu, 03 May 2012 11:02:31 +0200 Subject: [erlang-questions] New draft of frames proposal (was Re: Frames proposal) In-Reply-To: <4FA24829.1020305@fastmail.fm> References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> <4FA24829.1020305@fastmail.fm> Message-ID: <4FA249A7.60207@fastmail.fm> Jan Burse schrieb: > > (I am using the same for {fun,arity} for a cache > inside an atom, works like a charm) Actually I am using {fun,arity,abolish_flag}, and abolish_flag might change. But I guess such a flag is not needed for frames, no hot swap problem? From janburse@REDACTED Thu May 3 12:31:54 2012 From: janburse@REDACTED (Jan Burse) Date: Thu, 03 May 2012 12:31:54 +0200 Subject: [erlang-questions] New draft of frames proposal (was Re: Frames proposal) In-Reply-To: <4FA24829.1020305@fastmail.fm> References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> <4FA24829.1020305@fastmail.fm> Message-ID: <4FA25E9A.4060709@fastmail.fm> Jan Burse schrieb: > (I am using the same for {fun,arity} for a cache > inside an atom, works like a charm) Speeds up =../2, call/n, predicate_property ( also when predicate indicator based (sic!)), etc.. in Prolog. But requires one more field in an atom, and to avoid cache trashing when the same arity is overloaded, I don't make atoms unique. Could be also helpful for Erlang apply, or maybe you already usa a cache there in some instruction. From paperless@REDACTED Thu May 3 12:39:21 2012 From: paperless@REDACTED (Tim McNamara) Date: Thu, 3 May 2012 22:39:21 +1200 Subject: [erlang-questions] New draft of frames proposal (was Re: Frames proposal) In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> Message-ID: On 3 May 2012 16:22, Richard O'Keefe wrote: > There is a new version of > http://www.cs.otago.ac.nz/staffpriv/ok/frames.pdf Thank you for this impressive piece of work. Here are some notes that I made that may be of use to people who are intimidated by the size of the article. (Please correct errors!) - Frames are a distinct type. That is, they are not syntactic sugar over tuples and/or lists - They are not declared. They - They have no default value - Only atoms may be keys - No frame identifier, e.g. impossible to label a frame as "person" which represents people I've only been able to make it through the first 8 sections in any detail. The only thing that genuinely worries is relying on atoms as keys. I think this would mean that frames could not be used to deserialise untrustworthy strings. That is, programmers could expose themselves to having their atom table overrun. I disapprove of the tilde though. The association with approximate is too strong for me to overlook. The argument of overloading the equals sign is noted. However, when I am programming, I don't see the equals sign as equality. When I see it, it represents an association, usually from right to left on the page. However, I'm wary of bikeshedding over the syntax. However, I don't see why the Erlang grammar can not support the Python syntax (I'm not calling this JSON, because JSON requires double quoted keys). {a: "0", b: "1"} The comment was made earlier that it's useful to use different syntax for different conceptual things. However, I think that this syntax is sufficiently different from a tuple to be distinguished, while looking unified. We are referring to lots of data which has been unified together into a single frame. I would also make the point that there are very few single-language programmers. Context switching is not free for humans either. It would be great to be able to translate between JS/Python/other and Erlang with as few hurdles as possible. Using familiar syntax for a fairly similar task seems very useful to me. I don't understand this example [p16] foo:bar(Ugh), How is this ambiguous? Are you trying to set the bar element from the foo frame? Your next line is unclear. If, following JSON, you use colon for maplets, your comprehension comes to a juddering halt while you try to gure out if this is a maplet or a remote function call. BAD! JSON is just a serialisation format. It is not JavaScript. JavaScript would require you something closer to: foo["bar"] = Ugh. Is that what you intended? I think this is a strong argument, but I am not persuaded by it: Erlang uses curly braces for sequences, not sets. It would be very confusing for people if {a ~ 1, b ~ 2} = {b ~ 2, a ~ 1} but {{a, 1}, {b, 2}} != {{b, 2}, {a, 1}}. I think that this would certainly be considered a trip for new Erlang programmers. However, this risk is warranted. From max.lapshin@REDACTED Thu May 3 12:44:56 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 3 May 2012 14:44:56 +0400 Subject: [erlang-questions] Erlang and FIX protocol In-Reply-To: <9DDA4A0F-4321-40F7-A896-3B41E737F60D@gmail.com> References: <9DDA4A0F-4321-40F7-A896-3B41E737F60D@gmail.com> Message-ID: I've tried to implement it: http://github.com/maxlapshin/fix Problem with FIX protocol is that it is a low-level protocol with very simple syntax, but rather cryptic semantic. Frames are just what we need for it =)) So parser should be so smart, that it must produce high-level messages from low-level protocol packets. XML description took from quickxml project looks very interesting, because it is a good specification. I've took this XML, added some erlangy stuff: https://github.com/maxlapshin/fix/blob/master/src/fix_template.erl#L213 to make fields convenient to use and now I have simple protocol handler. Next problem is how to handle inlined groups and containers for FIX messages. From paperless@REDACTED Thu May 3 12:45:12 2012 From: paperless@REDACTED (Tim McNamara) Date: Thu, 3 May 2012 22:45:12 +1200 Subject: [erlang-questions] Any Aucklanders around? Message-ID: I've just moved up here and am now based at the University of Auckland, as part of the New Zealand eScience Infrastructure (http://nesi.org.nz). I would be interested in getting to know anyone interested in HPC and/or Erlang in this part of the world. Cheers, Tim From jose.valim@REDACTED Thu May 3 12:50:13 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Thu, 3 May 2012 12:50:13 +0200 Subject: [erlang-questions] New draft of frames proposal (was Re: Frames proposal) In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> Message-ID: > > I don't understand this example [p16] > > foo:bar(Ugh), > > How is this ambiguous? Are you trying to set the bar element from the > foo frame? Your next line is unclear. > It is ambiguous because: { foo:bar(Ugh) } Could be a tuple with one element which is the result of foo:bar(Ugh) or a frame with key foo and value bar(Ugh) (according to your proposal). There is also the "ambiguity" in {}. Is it an empty frame or an empty tuple? <{ foo: bar(Ugh) }> could work because there would be no ambiguity for the compiler, but it would be still ambiguous for humans (damn you, humans). I would shrug if I read a code like <{ foo:bar(Ugh) }> that actually means <{ foo: bar(Ugh) }>. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu May 3 12:55:28 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 3 May 2012 14:55:28 +0400 Subject: [erlang-questions] New draft of frames proposal (was Re: Frames proposal) In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> Message-ID: > ?- They have no default value > ?- Only atoms may be keys These two poins are very interesting to discuss. Problem is with handling external data. After frames will be added, webservers will change their API to represent user query string as a frame. There have been done a lot in Ruby world, because Ruby also have atoms and it has the same problem with their non-GC behaviour. So appeared extension to Hash class, called HashWithIndifferentAccess http_params[:key] is translated to http_params["key"] and strings are used internally to store keys. This is safe in terms of atom table overload and convenient. We will not be able to use frames for handling user data if only atoms can be keys, but if we allow to store atoms, binaries and strings as keys, we will get a mess with different ways to introduce keys. From paperless@REDACTED Thu May 3 13:02:07 2012 From: paperless@REDACTED (Tim McNamara) Date: Thu, 3 May 2012 23:02:07 +1200 Subject: [erlang-questions] New draft of frames proposal (was Re: Frames proposal) In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> Message-ID: On 3 May 2012 22:50, Jos? Valim wrote: >> I don't understand this example [p16] >> >> ? ?foo:bar(Ugh), >> >> How is this ambiguous? Are you trying to set the bar element from the >> foo frame? Your next line is unclear. > > > It is ambiguous because: > > ? ? { foo:bar(Ugh) } > > Could be a tuple with one element which is the result of foo:bar(Ugh) or > a frame with key foo and value bar(Ugh) (according to your proposal). > There is also the "ambiguity"?in {}. Is it an empty frame or an empty tuple? Ah, understood. There is certainly an opportunity for error here. Could {:} represent the empty frame? > > <{ foo: bar(Ugh) }> could work because there would be no ambiguity for > the compiler, but it would be still ambiguous for humans (damn you, humans). > I would shrug if I read a code like <{ foo:bar(Ugh) }> that actually means > <{ foo: bar(Ugh) }>. FWIW, I think this is probably acceptable. If <{ }> was accepted, it implies an intent on the programmer's behalf that they wanted to create a frame, which requires a key and a value. From watson.timothy@REDACTED Thu May 3 13:10:25 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 03 May 2012 12:10:25 +0100 Subject: [erlang-questions] New draft of frames proposal (was Re: Frames proposal) In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> Message-ID: <4FA267A1.50709@gmail.com> On 03/05/2012 11:50, Jos? Valim wrote: > > I don't understand this example [p16] > > foo:bar(Ugh), > > How is this ambiguous? Are you trying to set the bar element from the > foo frame? Your next line is unclear. > > > It is ambiguous because: > > { foo:bar(Ugh) } > > Could be a tuple with one element which is the result of foo:bar(Ugh) or > a frame with key foo and value bar(Ugh) (according to your proposal). > There is also the "ambiguity" in {}. Is it an empty frame or an empty > tuple? > > <{ foo: bar(Ugh) }> could work because there would be no ambiguity for > the compiler, but it would be still ambiguous for humans (damn you, > humans). > I would shrug if I read a code like <{ foo:bar(Ugh) }> that actually means > <{ foo: bar(Ugh) }>. I'm not sure that I would (shrug) at it - #{ foo->bar, baz->2 } seems much less ambiguous to me, and whilst I recognise that #{ 1 -> 2 } appears a bit odd, I would've thought that it's fairly obvious what it's doing. There is no clash with module:function syntax, no clash with #record_name{} syntax and no chance of accidental matching taking place AFAICT. And as the author of erlson mentioned, there is no real problem with trouncing erlson's #{...} syntax as erlson users will immediately migrate to frames. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu May 3 14:53:58 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 3 May 2012 16:53:58 +0400 Subject: [erlang-questions] Erlang and FIX protocol In-Reply-To: References: <9DDA4A0F-4321-40F7-A896-3B41E737F60D@gmail.com> Message-ID: On Thursday, May 3, 2012, Minin Maxim wrote: > ** > I am writing the simple fix-test-engine supported all fix versions- like > Verifix (http://www.greenlinetech.com/products/verifix.php). > And it is closed? > > It is really not difficult to parser/encoder the fix-messages. Definitions > of types and records can be generated from Fix.xml, some utils too and the > rest ist trivial. > This is not trivial, because there are incapsulated groups and components in protocol. Also connamara systems has allready developed the commercially available Erlang FIX engine - http://www.connamara.com > > I've asked some finnish company about their fix implementation, but seems my email was lost. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From straightflush@REDACTED Thu May 3 15:05:50 2012 From: straightflush@REDACTED (AD) Date: Thu, 3 May 2012 09:05:50 -0400 Subject: [erlang-questions] Most performant way to store/search a bag In-Reply-To: <7560917348206531539@unknownmsgid> References: <609A9F79-20D8-4431-AD30-E552558FC463@feuerlabs.com> <7560917348206531539@unknownmsgid> Message-ID: yes of course, i was just measuring worst case across both. @Ulf good point. Will take suggestions and try a few other tests. -AD On Thu, May 3, 2012 at 1:38 AM, dmitry kolesnikov wrote: > Hi, > > In your test script select does a full table scan, emulates foldl. But > your use-case is foldl through subset. Should you measure performance > of that case? > > Best Regards, > Dmitry >-|-|-*> > > > On 3.5.2012, at 7.57, Ulf Wiger wrote: > > > 3 maj 2012 kl. 05:35 skrev AD : > > > >> gproc seems to use an ordered_set ETS table and then an ets:select() to > match against the keys it needs. I did some basic tests with a 5 million > row ETS ordered set table and ets:foldl still seems to come out faster. > > > > > > The key to good select() performance in an ordered_set table is to > ensure that the first part of the key pattern is bound. In your test, the > key pattern was a free variable, forcing the match spec evaluator to test > every single record in the table. > > > > Gproc strives to do 'bounded selects' as much as possible. > > > > BR, > > Ulf W > > > > Ulf Wiger, Feuerlabs, Inc. > > http://www.feuerlabs.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From straightflush@REDACTED Thu May 3 15:10:10 2012 From: straightflush@REDACTED (AD) Date: Thu, 3 May 2012 09:10:10 -0400 Subject: [erlang-questions] websockets and streaming quote help In-Reply-To: <1336033752.71089.YahooMailNeo@web121605.mail.ne1.yahoo.com> References: <1336033752.71089.YahooMailNeo@web121605.mail.ne1.yahoo.com> Message-ID: So you recorded what I thought was a private skype conversation into a wav file and you are sending our private skype chat to an entire mailing list. Interesting. Moving on.... On Thu, May 3, 2012 at 4:29 AM, Joseph Goldberg wrote: > Can someone help me getting user authentication done with websockets and > rabbitmq to consume and distribute streaming quotes to my subscribed users. > I have a wave file of our conversation over skype, where he gives other > details. > > > http://armstrongonsoftware.blogspot.ca/2009/12/comet-is-dead-long-live-websockets.html > > http://blog.dberg.org/2012/04/using-gproc-and-cowboy-to-pass-messages.html > [6:46:27 PM] Adam Denenberg: Our last step is to actually broadcast a > message that contains this format. Now, here is where you need to roll > your own logic to determine how you want to ingest messages (queue, > webservice, etc), but once your app gets the message you are waiting for > all you need is one line to send the message: > > gproc:send({p, l, ?WSKey}, {self(), ?WSKey, Msg}) > [6:46:44 PM] Adam Denenberg: This simply sends message "Msg" to all the > processes that were registered with the key ?WSKey and then gets handled in > websocket_info and thus passed over the websocket connection. > > > also need fix protocol implementation done with MQ and sequence number > checking which he does not do > > > > http://www.codeproject.com/Articles/35439/Fix-Protocol-C-Fix-Engine-Source-Code > > Yes that's true. but tcpip messages are not erroneus/corrupted. the point > of this code is to be fast - eg i do not verify the checksum. in fact i > don't even bother to check the incoming message sequence number. i just > have two safety checks in the code which check the message starts 8= and > then 35=. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hynek@REDACTED Thu May 3 15:14:13 2012 From: hynek@REDACTED (Hynek Vychodil) Date: Thu, 3 May 2012 15:14:13 +0200 Subject: [erlang-questions] New draft of frames proposal (was Re: Frames proposal) In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> Message-ID: If I understand ROK right, the main problem which worries him is code like this { quix % some comment or some more code , <{ a:1 % some comment or some more code , foo:bar(Ugh) , baz:quux(Eh) % some comment or some more code }> % some comment or some more code , [ ...] % some comment or some more code } When you see code snippet/line with foo:bar(Ugh) you have to very carefully scan surrounding code when you are willing to know what the hell is going on and above example is relatively pretty formatted and indented. It can be far worse in real world application and with little bit more levels as {ok, <{ foo:{bar, baz, <{foo:bar(Ugh), baz:quux(Eh)}>, foo:bar(Och)}, a:2 }>} Good luck with this syntax. On Thu, May 3, 2012 at 1:02 PM, Tim McNamara wrote: > On 3 May 2012 22:50, Jos? Valim wrote: >>> I don't understand this example [p16] >>> >>> ? ?foo:bar(Ugh), >>> >>> How is this ambiguous? Are you trying to set the bar element from the >>> foo frame? Your next line is unclear. >> >> >> It is ambiguous because: >> >> ? ? { foo:bar(Ugh) } >> >> Could be a tuple with one element which is the result of foo:bar(Ugh) or >> a frame with key foo and value bar(Ugh) (according to your proposal). >> There is also the "ambiguity"?in {}. Is it an empty frame or an empty tuple? > > Ah, understood. There is certainly an opportunity for error here. > Could {:} represent the empty frame? > >> >> <{ foo: bar(Ugh) }> could work because there would be no ambiguity for >> the compiler, but it would be still ambiguous for humans (damn you, humans). >> I would shrug if I read a code like <{ foo:bar(Ugh) }> that actually means >> <{ foo: bar(Ugh) }>. > > FWIW, I think this is probably acceptable. If <{ }> was accepted, it > implies an intent on the programmer's behalf that they wanted to > create a frame, which requires a key and a value. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Hynek Vychodil BI consultant GoodData n?m?st? 28. ??jna 1104/17, 602 00, Brno - ?ern? Pole Office: ? +420 530 50 7704 E-mail: ?hynek@REDACTED Web: ? ? www.gooddata.com From tristan.sloughter@REDACTED Thu May 3 15:30:25 2012 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Thu, 3 May 2012 08:30:25 -0500 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: <4FA1CEC8.1060807@gmail.com> References: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> <4FA1CEC8.1060807@gmail.com> Message-ID: I feel like someone said Travis-CI wasn't the answer... But I am now having the thought it is, at least to make something I'd like to see without me having to do it. The question is would the community be interested in this if I implement it. Here is what I'm talking about: 1) A github hook that informs X that an update to a branch or tag is pushed. 2) This hook does not only ensure tests pass but also that the version numbers are correct. If you tag a project as 0.1.0 and the .app file has {vsn, 0.1.0} it fails 3) Assuming these criteria are met the agner repo for the project is either updated or created. Instead of @master it would use a piece of the git hash for the branch pointer (using a moving target -- a branch name -- as the version of an app doesn't make sense), for a tag it would use the tag name. Now, this is only part of the large solution people want, and I agree with them we should do. But, I think it is a start and something that can be done in small enough amount of effort to be done soon. Tristan On Wed, May 2, 2012 at 7:18 PM, Tim Watson wrote: > On 02/05/2012 21:13, Ciprian Dorin Craciun wrote: > >> On Wed, May 2, 2012 at 11:03 PMv, Tim Watson> >> wrote: > > Eric Merit and I have had some lengthy discussions about this on the >>> Erlware mailing list and have some ideas that I think are pretty solid. >>> >> I'm glad to hear this. (I'll give it a look.) >> > Cool thanks - please do leave any feedback you feel is relevant. > > The thing is though, you don't just need tools - you also need people to >>> package their stuff up using the tools. >>> >> I agree about this. In fact I think currently there are a lot of >> tools with diverging solutions. >> >> Also I don't think that one "blessed" tool would be the final >> answer. I would have taken a somehow different road, similar maybe to >> how Go is going (although they do have the "one" tool): i.e. strict >> conventions. >> > Yes I agree that '1 tool to rule them all' isn't going to work. Eric and I > have discussed building a unix-y pipeline of tools to deal with > local/remote package and repository management, dependency resolution, > installing and/or generating a code path (or some other structure) for > bootstrapping either ERL_LIBS or a call to code:path, etc. > > The assumption we have is that various parts can be dealt with be > different tools providing they respect the APIs, so you can build with > rebar or sinan or make or whatever. We are also planning on using reltool > (or some alternative/replica of sorts) to deal with the packaging bits, as > well as a set of tools for publication, code/package signing and the like. > > We would also like to support different release types (e.g., embedded vs. > non), distinguish between applications and library applications and so on. > > >> What do I mean is this: >> a) it would be nice to have a standardized way to specify "extra" >> options to compile an Erlang module. (We have the module attributes >> that we could use.) (the same for C sources); >> b) we already have a standard project layout; (i.e. `./src`, >> `./include`, etc.) >> c) we already have a standard project dependencies (i.e. the `app` >> file); >> >> All we need to do is be consistent in this convention, and then >> all the various build and packaging systems could adapt. >> > Packaging wise, I don't see how any of this really helps so much, although > I completely agree that sticking to this consistent layout is a good thing. > > Using the app file for dependencies is fine at runtime - I do this in > https://github.com/hyperthunk/**appstart- but if you're connecting to a (possibly remote) artefact repository and > fetching stuff, then you need to specify the publisher/signer (because 2 > OTP applications could be written with the same name, and of course with > FOSS project forking this happens all the time), the application name, and > the version. You can't whack all of that into your .app file without > breaking various things. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From per.melin@REDACTED Thu May 3 16:31:45 2012 From: per.melin@REDACTED (Per Melin) Date: Thu, 3 May 2012 16:31:45 +0200 Subject: [erlang-questions] Erlang and FIX protocol In-Reply-To: References: <9DDA4A0F-4321-40F7-A896-3B41E737F60D@gmail.com> Message-ID: On May 3, 2012, at 12:44 , Max Lapshin wrote: > I've tried to implement it: http://github.com/maxlapshin/fix > > Problem with FIX protocol is that it is a low-level protocol with very > simple syntax, but rather cryptic semantic. The problem with the syntax is that it is simplistic. It does not even give you enough information to be able to tokenize a message or to find message boundaries. See binary fields below. > Next problem is how to handle inlined groups and containers for FIX messages. It has been a few years since I last looked at FIX so I apologize if anything below is incorrect. When I run test cases that is suppose to fail on your code I mostly get {more, N} back, often {more, 0}. If I remember correctly, one pain of FIX is that when your code finds an error in what it received from the other party you should categorize it (e.g. fields out of order, value out of range) and send a reject message back detailing the error together with some context. I think dealing with that added quite a lot of both lines and complexity to my parser. You are not verifying the message checksum, which in the end is not as straight forward as it may first seem. Your code works on the assumption that SOH (ASCII 01) can be used as a field separator in all contexts, right? That is not true, since it can also be part of a binary field. For example: 1> fix:decode(<<"8=FIX.4.4",1,"9=23",1,"35=0",1,"93=10",1,"89=A",1,"89=234",1,"10=999",1>>). {ok,{heartbeat,undefined,undefined,undefined,undefined, undefined, [{signature_length,10}, {signature,<<"A">>}, {signature,<<"234">>}]}, <<>>} This is incorrect, as far as I can remember. The actual value of the signature is "A|89=234|1" here, not ["A", "234"] (not to mention that your signature length does not match the content). Field tag 89 (signature) is of type data, so you must look at tag 93 (signature_length) to know where the value ends. Because of binary fields I don't see how you can split or tokenize a message first and then parse it in the way you've done. From max.lapshin@REDACTED Thu May 3 16:47:51 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 3 May 2012 18:47:51 +0400 Subject: [erlang-questions] Erlang and FIX protocol In-Reply-To: References: <9DDA4A0F-4321-40F7-A896-3B41E737F60D@gmail.com> Message-ID: On Thu, May 3, 2012 at 6:31 PM, Per Melin wrote: > > The problem with the syntax is that it is simplistic. It does not even give you enough information to be able to tokenize a message or to find message boundaries. See binary fields below. > It is not exactly so. Standard tells that whole packet must start from field 8 with proto version and next field must be byte length without checksum. So this is a good way to detect real byte length. > > It has been a few years since I last looked at FIX so I apologize if anything below is incorrect. > > When I run test cases that is suppose to fail on your code I mostly get {more, N} back, often {more, 0}. If I remember correctly, one pain of FIX is that when your code finds an error in what it received But you must first recevie whole message before saying that it is broken. However, {more, 0} is a problem. > You are not verifying the message checksum, which in the end is not as straight forward as it may first seem. What is bad with it? > > Your code works on the assumption that SOH (ASCII 01) can be used as a field separator in all contexts, right? That is not true, since it can also be part of a binary field. For example: Yes, I haven't parsed data fields yet. But I know how to add this if I need =) In fact a good test suite will really help. > > 1> fix:decode(<<"8=FIX.4.4",1,"9=23",1,"35=0",1,"93=10",1,"89=A",1,"89=234",1,"10=999",1>>). > {ok,{heartbeat,undefined,undefined,undefined,undefined, > ? ? ? ? ? ? ? undefined, > ? ? ? ? ? ? ? [{signature_length,10}, > ? ? ? ? ? ? ? ?{signature,<<"A">>}, > ? ? ? ? ? ? ? ?{signature,<<"234">>}]}, > ? ?<<>>} > > This is incorrect, as far as I can remember. The actual value of the signature is "A|89=234|1" here, not ["A", "234"] (not to mention that your signature length does not match the content). Field tag 89 (signature) is of type data, so you must look at tag 93 (signature_length) to know where the value ends. Because of binary fields I don't see how you can split or tokenize a message first and then parse it in the way you've done. > I understand you. But this can really be fixed. From max.lapshin@REDACTED Thu May 3 16:55:48 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 3 May 2012 18:55:48 +0400 Subject: [erlang-questions] Erlang and FIX protocol In-Reply-To: References: <9DDA4A0F-4321-40F7-A896-3B41E737F60D@gmail.com> Message-ID: While creating fix_parser I tried to solve following problem: create parser that creates records, ready to use in business code so that I don't have to add intermediate layer, that translate decoded field to some structure. This is why I have defined records for all kinds of FIX messages. But there are two ways of handling messages: 1) split by SOH and fill record fields in big lists:foldl 2) go one by one and fill record by pattern matching of binary in function clause. I think that I'll change a bit parsing code so that it should be: 1) non tail-recursive. How many fields can be in a message? I think no more than 100-200? 2) switch to threaded way of parsing 3) precreate mapping of field names to record indexes so that it may be unified parsing function for all records. I think that going on C level is too early =)) From max.lapshin@REDACTED Thu May 3 17:02:02 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 3 May 2012 19:02:02 +0400 Subject: [erlang-questions] Erlang and FIX protocol In-Reply-To: References: <9DDA4A0F-4321-40F7-A896-3B41E737F60D@gmail.com> Message-ID: > > 1> fix:decode(<<"8=FIX.4.4",1,"9=23",1,"35=0",1,"93=10",1,"89=A",1,"89=234",1,"10=999",1>>). > {ok,{heartbeat,undefined,undefined,undefined,undefined, > ? ? ? ? ? ? ? undefined, > ? ? ? ? ? ? ? [{signature_length,10}, > ? ? ? ? ? ? ? ?{signature,<<"A">>}, > ? ? ? ? ? ? ? ?{signature,<<"234">>}]}, > ? ?<<>>} > > This is incorrect, as far as I can remember. The actual value of the signature is "A|89=234|1" here, not ["A", "234"] (not to mention that your signature length does not match the content). Are you really sure, that it is so? Specification tells: Data fields are always immediately preceded by a length field. The length field should specify the number of bytes of the value of the data field (up to but not including the terminating SOH). "A|89=234|1" is 10 bytes but there must be terminating SOH after this. From straightflush@REDACTED Thu May 3 20:59:31 2012 From: straightflush@REDACTED (AD) Date: Thu, 3 May 2012 14:59:31 -0400 Subject: [erlang-questions] supervisor not restarting gen_server Message-ID: I have a gen_server that seems to be crashing for some reason and the supservisor is not restarting it. The child spec is one_for_one, permanent and the error log is below. All i see is the crash report but no attempt at a restart. Is there any way to figure out why the supervisor would not restart it? 2012-05-02 14:54:28.776 [info] <0.889.0>@websocket_pubsubsrv:terminate:198 Shutting down pubsub server... 2012-05-02 14:54:28.776 [error] <0.882.0> Supervisor cowboy_sup had child {cowboy_listener_sup,my_http_listener} started with cowboy_listener_sup:start_link(100, cowboy_tcp_transport, [{port,8000}], cowboy_http_protocol, [{dispatch,[{'_',[{[<<"multiplex">>,'...'],sockjs_cowboy_handler,{service,"/multiplex",sockjs_multiplex,...}},...]}]}]) at <0.909.0> exit with reason killed in context child_terminated 2012-05-02 14:54:28.785 [error] <0.911.0> gen_server <0.911.0> terminated with reason: killed 2012-05-02 14:54:28.788 [error] <0.911.0> CRASH REPORT Process <0.911.0> with 1 neighbours crashed with reason: {killed,[{gen_server,terminate,6},{proc_lib,init_p_do_apply,3}]} Thanks -AD -------------- next part -------------- An HTML attachment was scrubbed... URL: From tristan.sloughter@REDACTED Thu May 3 23:06:25 2012 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Thu, 3 May 2012 16:06:25 -0500 Subject: [erlang-questions] [ANN] Sinan 4.0.0 Message-ID: Sinan (http://erlware.github.com/sinan/) has been through a huge amount of work from its original versions, to 2.0, to 3.0 and now 4.0! Unlike past versions I thought it was important to announce this 4.0.0 because of the lightning talk I gave at Erlang Factory 2012 that interested some. Afterwards I found out that OSX users with Erlang installed through brew were met with less than helpful error messages about dependency on proper. This has been fixed. So give it a go again :) Credit goes to Eric Merritt and the gatekeeper Jordan Wilberding. The change log below does not cover the immense changes from the first version up to 4.0.0 but is included here: CHANGES: commit 6d6239c4ba09a78dcd6f125d017caaa94fd20ea4 Author: Eric Merritt Date: Tue May 1 15:49:28 2012 -0500 fix bug in detecting change in joxa files commit 0fb6ed577ba028e56f3f162aa871aefbae4e6f94 Author: Eric Merritt Date: Wed May 2 10:39:36 2012 -0500 make eunit detection a bit more robust commit af8959a33e0c55062d71e4b7c68f2ea3e5714a9c Author: Eric Merritt Date: Tue May 1 12:06:50 2012 -0500 Make sinan version return just the version and nothing more commit 0ae2a06730fa7a834a819c3cdfd76468ce88cea4 Author: Eric Merritt Date: Tue May 1 09:15:56 2012 -0500 fix version setting problem in makefile this seems to be related to newer versions of awk, but the fix in this patch provides a reasonable solution. commit cbd2520e2671089d4a3fcb26ebda53a343271273 Author: Eric Merritt Date: Tue May 1 08:35:54 2012 -0500 Fix spec in sinan:run_sinan commit fbc2d99b17b34138c7553a634e2b0f674c4b71bc Author: Eric Merritt Date: Mon Apr 30 15:18:39 2012 -0500 make sure that solving happens every run (we cant really know what non-project dependencies changed so we should solve each time) commit d535ddc1980bf267380b7c6449cf36725a7ce88b Author: Eric Merritt Date: Mon Apr 30 15:18:14 2012 -0500 fix recursive copy bug in sin release commit b81d77756ebae2227b537abbaf0a078c050c43f1 Author: Eric Merritt Date: Mon Apr 30 13:50:21 2012 -0500 make sure that apps with a version in the name are 'discoverable' commit ec470c0f6cdd83a4f1a7e5dd8e81711c210e78b4 Author: Eric Merritt Date: Mon Apr 30 13:04:59 2012 -0500 fix dist so that it correctly tars the release dir commit acedbf6291ca9c590ae8a4ee42f1a18877652cfc Author: Eric Merritt Date: Mon Apr 30 12:07:28 2012 -0500 make sure the include_erts is optional in release commit 91ae619e35bf9c61c8ba856069658deb0ecc08e4 Author: Eric Merritt Date: Mon Apr 30 12:07:11 2012 -0500 remove unneeded param from args in release commit f01bb0ea9323b15215c920ee8a92726bc23cf1ec Author: Eric Merritt Date: Mon Apr 30 12:05:17 2012 -0500 fix and simplify how 'is_dir_ignorable' works commit dd5c2f4a6afa3673f024290b43bb93b36019b6fb Author: Eric Merritt Date: Mon Apr 30 10:17:49 2012 -0500 present a more reasonable error message on startup failure commit aa7aa366d476377652e1096a4aeddb63fcf87fb7 Author: Eric Merritt Date: Tue May 1 08:36:31 2012 -0500 remove built in dependencies on eunit and proper commit 45cdaba17975737ec29d8b757e70729926360e81 Author: Eric Merritt Date: Wed May 2 16:42:28 2012 -0500 add missing joxa dependency to the system commit 8da03076b59e510b75532e4ad167f27294120964 Author: Eric Merritt Date: Sun Mar 25 10:56:55 2012 -0500 Remove behaviours from the list of warnings in dialyzer commit 4a8fe7965135dcfe09830ac507b7aa77ac81abbd Author: Eric Merritt Date: Fri Mar 23 20:47:25 2012 -0500 Add missing joxa to debian control commit ae17d9ce73c500f95797058095870bc5bd1485e4 Author: Eric Merritt Date: Fri Mar 23 20:33:58 2012 -0500 Add missing version bump to the debian changelog commit 524898ed33302256c52783fe2bf7184d09fa02ac Author: Jordan Wilberding Date: Wed Mar 14 08:48:17 2012 -0400 Fixed typo. -------------- next part -------------- An HTML attachment was scrubbed... URL: From peerst@REDACTED Thu May 3 23:16:51 2012 From: peerst@REDACTED (Peer Stritzinger) Date: Thu, 3 May 2012 23:16:51 +0200 Subject: [erlang-questions] BIF or NIF for extending Erlang on OS which can't dynamic link Message-ID: Hi, I'm currently porting Erlang to RTEMS a small embedded OS that can't dynamic link. If I want to extend Erlang with functionality written in C (e.g. for accessing hardware or RTEMS features) I always have to statically link this code to the Erlang + RTEMS image -- the OS is just a executive which is linked as a lib to Erlang which together boots directly on the hardware. If I want to use NIFs or linked drivers there is always some dynamic linking going on which fails for me since there is no dlopen etc. functionality in the basic RTEMS system. So I planned to fake the dynamic linking either on the dlopen/dlsym level or at the erts_sys_ddll_* level. The other possibility would be to add the functions to be written in C as BIFs Currently I don't know how to add my own BIFs, so I can't decide if this is the way to go or rather stick with faking dynamic linking. What do you think is best? Cheers, Peer Stritzinger From gumm@REDACTED Thu May 3 23:22:32 2012 From: gumm@REDACTED (Jesse Gumm) Date: Thu, 3 May 2012 16:22:32 -0500 Subject: [erlang-questions] [ANN] Sinan 4.0.0 In-Reply-To: References: Message-ID: Congrats on the release. I haven't played around with sinan much since ErlangCamp Chicago 2010. With this release, I'll need to tinker around with it some. Thanks guys, -Jesse On Thu, May 3, 2012 at 4:06 PM, Tristan Sloughter wrote: > Sinan (http://erlware.github.com/sinan/)?has been through a huge amount of > work from its original versions, to 2.0, to 3.0 and now 4.0! > > Unlike past versions I thought it was important to announce this 4.0.0 > because of the lightning talk I gave at Erlang Factory 2012 that interested > some. Afterwards I found out that OSX users with Erlang installed through > brew were met with less than helpful error messages about dependency on > proper. This has been fixed. So give it a go again :) > > Credit goes to Eric Merritt and the gatekeeper Jordan Wilberding. > > The change log below does not cover the immense changes from the first > version up to 4.0.0 but is included here: > > CHANGES: > > commit 6d6239c4ba09a78dcd6f125d017caaa94fd20ea4 > Author: Eric Merritt > Date: ? Tue May 1 15:49:28 2012 -0500 > > ? ?fix bug in detecting change in joxa files > > commit 0fb6ed577ba028e56f3f162aa871aefbae4e6f94 > Author: Eric Merritt > Date: ? Wed May 2 10:39:36 2012 -0500 > > ? ?make eunit detection a bit more robust > > commit af8959a33e0c55062d71e4b7c68f2ea3e5714a9c > Author: Eric Merritt > Date: ? Tue May 1 12:06:50 2012 -0500 > > ? ?Make sinan version return just the version and nothing more > > commit 0ae2a06730fa7a834a819c3cdfd76468ce88cea4 > Author: Eric Merritt > Date: ? Tue May 1 09:15:56 2012 -0500 > > ? ?fix version setting problem in makefile > > ? ?this seems to be related to newer versions of awk, but the fix in this > ? ?patch provides a reasonable solution. > > commit cbd2520e2671089d4a3fcb26ebda53a343271273 > Author: Eric Merritt > Date: ? Tue May 1 08:35:54 2012 -0500 > > ? ?Fix spec in sinan:run_sinan > > commit fbc2d99b17b34138c7553a634e2b0f674c4b71bc > Author: Eric Merritt > Date: ? Mon Apr 30 15:18:39 2012 -0500 > > ? ?make sure that solving happens every run > > ? ?(we cant really know what non-project dependencies changed so we > should solve each time) > > commit d535ddc1980bf267380b7c6449cf36725a7ce88b > Author: Eric Merritt > Date: ? Mon Apr 30 15:18:14 2012 -0500 > > ? ?fix recursive copy bug in sin release > > commit b81d77756ebae2227b537abbaf0a078c050c43f1 > Author: Eric Merritt > Date: ? Mon Apr 30 13:50:21 2012 -0500 > > ? ?make sure that apps with a version in the name are 'discoverable' > > commit ec470c0f6cdd83a4f1a7e5dd8e81711c210e78b4 > Author: Eric Merritt > Date: ? Mon Apr 30 13:04:59 2012 -0500 > > ? ?fix dist so that it correctly tars the release dir > > commit acedbf6291ca9c590ae8a4ee42f1a18877652cfc > Author: Eric Merritt > Date: ? Mon Apr 30 12:07:28 2012 -0500 > > ? ?make sure the include_erts is optional in release > > commit 91ae619e35bf9c61c8ba856069658deb0ecc08e4 > Author: Eric Merritt > Date: ? Mon Apr 30 12:07:11 2012 -0500 > > ? ?remove unneeded param from args in release > > commit f01bb0ea9323b15215c920ee8a92726bc23cf1ec > Author: Eric Merritt > Date: ? Mon Apr 30 12:05:17 2012 -0500 > > ? ?fix and simplify how 'is_dir_ignorable' works > > commit dd5c2f4a6afa3673f024290b43bb93b36019b6fb > Author: Eric Merritt > Date: ? Mon Apr 30 10:17:49 2012 -0500 > > ? ?present a more reasonable error message on startup failure > > commit aa7aa366d476377652e1096a4aeddb63fcf87fb7 > Author: Eric Merritt > Date: ? Tue May 1 08:36:31 2012 -0500 > > ? ?remove built in dependencies on eunit and proper > > commit 45cdaba17975737ec29d8b757e70729926360e81 > Author: Eric Merritt > Date: ? Wed May 2 16:42:28 2012 -0500 > > ? ?add missing joxa dependency to the system > > commit 8da03076b59e510b75532e4ad167f27294120964 > Author: Eric Merritt > Date: ? Sun Mar 25 10:56:55 2012 -0500 > > ? ?Remove behaviours from the list of warnings in dialyzer > > commit 4a8fe7965135dcfe09830ac507b7aa77ac81abbd > Author: Eric Merritt > Date: ? Fri Mar 23 20:47:25 2012 -0500 > > ? ?Add missing joxa to debian control > > commit ae17d9ce73c500f95797058095870bc5bd1485e4 > Author: Eric Merritt > Date: ? Fri Mar 23 20:33:58 2012 -0500 > > ? ?Add missing version bump to the debian changelog > > commit 524898ed33302256c52783fe2bf7184d09fa02ac > Author: Jordan Wilberding > Date: ? Wed Mar 14 08:48:17 2012 -0400 > > ? ?Fixed typo. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm From torben.lehoff@REDACTED Thu May 3 23:52:53 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Thu, 3 May 2012 23:52:53 +0200 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> <4FA1CEC8.1060807@gmail.com> Message-ID: That actually sounds like a good step forward - it almost, but only almost makes me sad that I have created my own dependency management tool called colombo (alpha vers. is on github/lehoff). I haven't spent much time on it, but it can fetch apps from git repos as well as their dependencies, one can specify a tag or a branch to check out, a simple build can be done and you can get a version tagged version installed in the subdir lib of the dir where you you colombo (vers is taken from the app file or a default is choosen). very much alpha quality, but it shows that small dedicated tools aren't that hard to create for the Erlang eco system. I will continue improving colombo since it helps me solve my own problem, but it would be nice to have "real" users as well that could help form the functionality of the tool. Or tools since I might already have added too much functionality to colombo. Cheers, Torben Sent from my iPad On 03/05/2012, at 15.30, Tristan Sloughter wrote: > I feel like someone said Travis-CI wasn't the answer... But I am now having the thought it is, at least to make something I'd like to see without me having to do it. > > The question is would the community be interested in this if I implement it. Here is what I'm talking about: > > 1) A github hook that informs X that an update to a branch or tag is pushed. > 2) This hook does not only ensure tests pass but also that the version numbers are correct. If you tag a project as 0.1.0 and the .app file has {vsn, 0.1.0} it fails > 3) Assuming these criteria are met the agner repo for the project is either updated or created. Instead of @master it would use a piece of the git hash for the branch pointer (using a moving target -- a branch name -- as the version of an app doesn't make sense), for a tag it would use the tag name. > > Now, this is only part of the large solution people want, and I agree with them we should do. > > But, I think it is a start and something that can be done in small enough amount of effort to be done soon. > > Tristan > > On Wed, May 2, 2012 at 7:18 PM, Tim Watson wrote: > On 02/05/2012 21:13, Ciprian Dorin Craciun wrote: > On Wed, May 2, 2012 at 11:03 PMv, Tim Watson wrote: > Eric Merit and I have had some lengthy discussions about this on the Erlware mailing list and have some ideas that I think are pretty solid. > I'm glad to hear this. (I'll give it a look.) > Cool thanks - please do leave any feedback you feel is relevant. > > The thing is though, you don't just need tools - you also need people to package their stuff up using the tools. > I agree about this. In fact I think currently there are a lot of > tools with diverging solutions. > > Also I don't think that one "blessed" tool would be the final > answer. I would have taken a somehow different road, similar maybe to > how Go is going (although they do have the "one" tool): i.e. strict > conventions. > Yes I agree that '1 tool to rule them all' isn't going to work. Eric and I have discussed building a unix-y pipeline of tools to deal with local/remote package and repository management, dependency resolution, installing and/or generating a code path (or some other structure) for bootstrapping either ERL_LIBS or a call to code:path, etc. > > The assumption we have is that various parts can be dealt with be different tools providing they respect the APIs, so you can build with rebar or sinan or make or whatever. We are also planning on using reltool (or some alternative/replica of sorts) to deal with the packaging bits, as well as a set of tools for publication, code/package signing and the like. > > We would also like to support different release types (e.g., embedded vs. non), distinguish between applications and library applications and so on. > > > What do I mean is this: > a) it would be nice to have a standardized way to specify "extra" > options to compile an Erlang module. (We have the module attributes > that we could use.) (the same for C sources); > b) we already have a standard project layout; (i.e. `./src`, > `./include`, etc.) > c) we already have a standard project dependencies (i.e. the `app` file); > > All we need to do is be consistent in this convention, and then > all the various build and packaging systems could adapt. > Packaging wise, I don't see how any of this really helps so much, although I completely agree that sticking to this consistent layout is a good thing. > > Using the app file for dependencies is fine at runtime - I do this in https://github.com/hyperthunk/appstart - but if you're connecting to a (possibly remote) artefact repository and fetching stuff, then you need to specify the publisher/signer (because 2 OTP applications could be written with the same name, and of course with FOSS project forking this happens all the time), the application name, and the version. You can't whack all of that into your .app file without breaking various things. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From serge@REDACTED Fri May 4 02:58:56 2012 From: serge@REDACTED (Serge Aleynikov) Date: Thu, 03 May 2012 20:58:56 -0400 Subject: [erlang-questions] Erlang and FIX protocol In-Reply-To: References: <9DDA4A0F-4321-40F7-A896-3B41E737F60D@gmail.com> Message-ID: <4FA329D0.7010701@aleynikov.org> Since the time I asked that question on the mailing list, I also played with the idea of implementing FIX in Erlang. Pursued two approaches - one, similar to Quickfix, that used XSLT to auto-generate the parser and message definitions. The end result was something nice that was able to decode FIX logs quite nicely (similar to what this product does: http://www.validfix.com/fix-analyzer.html). The second approach was to implement a simple parser in C and use Erlang just to handle the session and application level layer of the protocol stack. At the end this didn't really go beyond those experiments, as I switched to working on a forked version of Quickfix in C++. Very likely you don't need to support a full set of FIX features and if so, you can make some shortcuts that simplify the implementation. Taking a look at your project, I doubt that defining those large record tuples for messages is a good idea performance wise, since every update of a tuple will require a copy. a better approach would be to have a small tuple with mandatory fields, and the rest stored in one element holding a gb_tree or some other container. E.g.: %% Message type: 8 -record(executionReport, { orderID %% Tag: 37 , execID %% Tag: 17 , execTransType %% Tag: 20 , execType %% Tag: 150 , ordStatus %% Tag: 39 , symbol %% Tag: 55 , side %% Tag: 54 , leavesQty %% Tag: 151 , cumQty %% Tag: 14 , avgPx %% Tag: 6 , opt = [] }). On 5/3/2012 6:44 AM, Max Lapshin wrote: > I've tried to implement it: http://github.com/maxlapshin/fix > > Problem with FIX protocol is that it is a low-level protocol with very > simple syntax, but rather cryptic semantic. > > Frames are just what we need for it =)) > > So parser should be so smart, that it must produce high-level messages > from low-level protocol packets. XML description took from quickxml > project looks very interesting, because it is a good specification. > > I've took this XML, added some erlangy stuff: > https://github.com/maxlapshin/fix/blob/master/src/fix_template.erl#L213 > to make fields convenient to use and now I have simple protocol > handler. > > Next problem is how to handle inlined groups and containers for FIX messages. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From thpr@REDACTED Fri May 4 03:09:37 2012 From: thpr@REDACTED (Tom Parker) Date: Thu, 3 May 2012 18:09:37 -0700 (PDT) Subject: [erlang-questions] Capturing keystrokes in wxWindows Issues + Segmentation Fault Message-ID: <1336093777.24034.YahooMailNeo@web130101.mail.mud.yahoo.com> Background: I'm new to Erlang, so just getting the hang of things...I'm trying to get key events back from a window, and exploring how to do that.? If I do this in the erl shell: Wx=wx:new(). F=wxFrame:new(Wx,-1,"Hello, World!"). wxFrame:show(F). Canvas = wxPanel:new(F, []). wxWindow:setSize(Canvas,200,100). DC=wxPaintDC:new(Canvas). wxDC:clear(DC). wxEvtHandler:connect(F,key_down, [{callback, fun(Rec, Obj) -> io:format("Received:~p~n",[Rec]) end}]). Issue #1: With this setup, I get no events back (regardless of what I select in the UI prior to pressing any keys). If I change to key_up, I will get events, but ... Issue #2: ... the events only show up after I click into the (white due to the wxDC:clear call) Canvas, not just the title bar of the window or the other (gray) part of the Frame (see below for where the focus is before clicking in the Canvas).? I'm unsure why it has to be a child that has focus for me to get a callback. I also get char_hook events if I use that instead of key_down Issue #3: ... but no char_hook for things like the "Shift" key - the ONLY event I seem to be able to get from Shift is "key_up" Issue #4: No char events at all if I connect to those (just like key_down) (I think those are the 4 types of keyboard events I might find) Since I wanted to try to capture events even when the title bar was clicked (vs just contents of the window), I set up a test to identify where the focus is when I select the title bar.? Code not relevant here, but clicking on the title bar of the window puts focus (as reported by wxWindow.findFocus() into a wxWindow that is not Frame F. Focus:{wx_ref,0,wxWindow,[]} Transferring Focus to the Frame doesn't help receipt of events: Focus:{wx_ref,35,wxWindow,[]} Clicking into the Canvas puts focus on Canvas, and allows me to receive key_up and (some) char_hook events: Focus:{wx_ref,36,wxWindow,[]} Attempting the connect the event handler to Wx produces errors, so that doesn't work (fair enough), but it turns out Wx really isn't the parent of F.? The wxWindow (ID 0) that has the focus IS the parent of Frame F as reported by wxWindow.getParent(F).... so I tried this: Wx=wx:new(). F=wxFrame:new(Wx,-1,"Hello, World!"). wxFrame:show(F). Fparent=wxWindow:getParent(F). wxEvtHandler:connect(Fparent,key_down, [{callback, fun(Rec, Obj) -> io:format("Received:~p~n",[Rec]) end}]). Issue #5: This causes a segmentation fault. Erlang 15B01 from a source build (plain untar, make, make install... necessary since I wanted wxWindows) on Ubuntu 10.04-LTS Any thoughts, ideas or feedback?? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ericbmerritt@REDACTED Fri May 4 03:25:16 2012 From: ericbmerritt@REDACTED (Eric Merritt) Date: Thu, 3 May 2012 20:25:16 -0500 Subject: [erlang-questions] [ANN] Sinan 4.0.0 In-Reply-To: References: Message-ID: Jesse, Its changed quite a bit since then. We are working on improving the documentation so I would love to hear how it goes for you. Eric On Thu, May 3, 2012 at 4:22 PM, Jesse Gumm wrote: > Congrats on the release. > > I haven't played around with sinan much since ErlangCamp Chicago 2010. > ?With this release, I'll need to tinker around with it some. > > Thanks guys, > > -Jesse > > On Thu, May 3, 2012 at 4:06 PM, Tristan Sloughter > wrote: >> Sinan (http://erlware.github.com/sinan/)?has been through a huge amount of >> work from its original versions, to 2.0, to 3.0 and now 4.0! >> >> Unlike past versions I thought it was important to announce this 4.0.0 >> because of the lightning talk I gave at Erlang Factory 2012 that interested >> some. Afterwards I found out that OSX users with Erlang installed through >> brew were met with less than helpful error messages about dependency on >> proper. This has been fixed. So give it a go again :) >> >> Credit goes to Eric Merritt and the gatekeeper Jordan Wilberding. >> >> The change log below does not cover the immense changes from the first >> version up to 4.0.0 but is included here: >> >> CHANGES: >> >> commit 6d6239c4ba09a78dcd6f125d017caaa94fd20ea4 >> Author: Eric Merritt >> Date: ? Tue May 1 15:49:28 2012 -0500 >> >> ? ?fix bug in detecting change in joxa files >> >> commit 0fb6ed577ba028e56f3f162aa871aefbae4e6f94 >> Author: Eric Merritt >> Date: ? Wed May 2 10:39:36 2012 -0500 >> >> ? ?make eunit detection a bit more robust >> >> commit af8959a33e0c55062d71e4b7c68f2ea3e5714a9c >> Author: Eric Merritt >> Date: ? Tue May 1 12:06:50 2012 -0500 >> >> ? ?Make sinan version return just the version and nothing more >> >> commit 0ae2a06730fa7a834a819c3cdfd76468ce88cea4 >> Author: Eric Merritt >> Date: ? Tue May 1 09:15:56 2012 -0500 >> >> ? ?fix version setting problem in makefile >> >> ? ?this seems to be related to newer versions of awk, but the fix in this >> ? ?patch provides a reasonable solution. >> >> commit cbd2520e2671089d4a3fcb26ebda53a343271273 >> Author: Eric Merritt >> Date: ? Tue May 1 08:35:54 2012 -0500 >> >> ? ?Fix spec in sinan:run_sinan >> >> commit fbc2d99b17b34138c7553a634e2b0f674c4b71bc >> Author: Eric Merritt >> Date: ? Mon Apr 30 15:18:39 2012 -0500 >> >> ? ?make sure that solving happens every run >> >> ? ?(we cant really know what non-project dependencies changed so we >> should solve each time) >> >> commit d535ddc1980bf267380b7c6449cf36725a7ce88b >> Author: Eric Merritt >> Date: ? Mon Apr 30 15:18:14 2012 -0500 >> >> ? ?fix recursive copy bug in sin release >> >> commit b81d77756ebae2227b537abbaf0a078c050c43f1 >> Author: Eric Merritt >> Date: ? Mon Apr 30 13:50:21 2012 -0500 >> >> ? ?make sure that apps with a version in the name are 'discoverable' >> >> commit ec470c0f6cdd83a4f1a7e5dd8e81711c210e78b4 >> Author: Eric Merritt >> Date: ? Mon Apr 30 13:04:59 2012 -0500 >> >> ? ?fix dist so that it correctly tars the release dir >> >> commit acedbf6291ca9c590ae8a4ee42f1a18877652cfc >> Author: Eric Merritt >> Date: ? Mon Apr 30 12:07:28 2012 -0500 >> >> ? ?make sure the include_erts is optional in release >> >> commit 91ae619e35bf9c61c8ba856069658deb0ecc08e4 >> Author: Eric Merritt >> Date: ? Mon Apr 30 12:07:11 2012 -0500 >> >> ? ?remove unneeded param from args in release >> >> commit f01bb0ea9323b15215c920ee8a92726bc23cf1ec >> Author: Eric Merritt >> Date: ? Mon Apr 30 12:05:17 2012 -0500 >> >> ? ?fix and simplify how 'is_dir_ignorable' works >> >> commit dd5c2f4a6afa3673f024290b43bb93b36019b6fb >> Author: Eric Merritt >> Date: ? Mon Apr 30 10:17:49 2012 -0500 >> >> ? ?present a more reasonable error message on startup failure >> >> commit aa7aa366d476377652e1096a4aeddb63fcf87fb7 >> Author: Eric Merritt >> Date: ? Tue May 1 08:36:31 2012 -0500 >> >> ? ?remove built in dependencies on eunit and proper >> >> commit 45cdaba17975737ec29d8b757e70729926360e81 >> Author: Eric Merritt >> Date: ? Wed May 2 16:42:28 2012 -0500 >> >> ? ?add missing joxa dependency to the system >> >> commit 8da03076b59e510b75532e4ad167f27294120964 >> Author: Eric Merritt >> Date: ? Sun Mar 25 10:56:55 2012 -0500 >> >> ? ?Remove behaviours from the list of warnings in dialyzer >> >> commit 4a8fe7965135dcfe09830ac507b7aa77ac81abbd >> Author: Eric Merritt >> Date: ? Fri Mar 23 20:47:25 2012 -0500 >> >> ? ?Add missing joxa to debian control >> >> commit ae17d9ce73c500f95797058095870bc5bd1485e4 >> Author: Eric Merritt >> Date: ? Fri Mar 23 20:33:58 2012 -0500 >> >> ? ?Add missing version bump to the debian changelog >> >> commit 524898ed33302256c52783fe2bf7184d09fa02ac >> Author: Jordan Wilberding >> Date: ? Wed Mar 14 08:48:17 2012 -0400 >> >> ? ?Fixed typo. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Jesse Gumm > Owner, Sigma Star Systems > 414.940.4866?|| sigma-star.com || @jessegumm > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Fri May 4 03:28:04 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 4 May 2012 13:28:04 +1200 Subject: [erlang-questions] New draft of frames proposal (was Re: Frames proposal) In-Reply-To: <4FA24829.1020305@fastmail.fm> References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> <4FA24829.1020305@fastmail.fm> Message-ID: <879FB186-A3AF-4FD2-BE66-05837BA72A5E@cs.otago.ac.nz> On 3/05/2012, at 8:56 PM, Jan Burse wrote: > Pitty the sections do not have a @date, now > I don't know what I have already read. That's why my message identified the sections that were new. > > BTW: Inline cache, when your lookup would return > a pointer to an immutable {func,class}, Er, in frames.pdf, the pointer *doesn't* point to {func,class}. As section 15 makes clear with a diagram, the cache item in a load_slot or store_slot instruction points to something which is logically {descriptor,offset} but can be stored more compactly. > then you > can cache this in one STORE op, making it amenable > to concurrent use. As indeed section 15 already says. > > > It's like a mouse's nest in a cat's ear > LoL, poor kitty It's from "Rite of Passage" by Alexei Panshin. While in transit between The Ship and Grainau (?) Mia Havero is told a story, and in the store the good guy wins a fortune by asking the ogre this riddle: What is not, never was, and never will be? The answer is a mouse's nest in a cat's ear. Rite is one of the *great* SF stories, and it has dated very well. From ok@REDACTED Fri May 4 03:45:21 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 4 May 2012 13:45:21 +1200 Subject: [erlang-questions] New draft of frames proposal (was Re: Frames proposal) In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> Message-ID: <30436D47-9834-4FC2-AD8B-0BE15AD440BE@cs.otago.ac.nz> On 3/05/2012, at 10:39 PM, Tim McNamara wrote: > The only thing that genuinely worries is relying on atoms as keys. I > think this would mean that frames could not be used to deserialise > untrustworthy strings. That is, programmers could expose themselves to > having their atom table overrun. This is not a problem at all for frames that are created by patterns that the compiler can see, only for dynamically created frames. And there is an EEP showing in sufficient detail how we can fix *that* problem once and for all. The frames proposal presupposes that's been done. > I disapprove of the tilde though. The association with approximate is > too strong for me to overlook. This is a classic "what is good for everybody is what *I* am used to" argument. For a lot of people reading this, "~" is not associated with approximation at all, but with regular expression matching. > However, I'm wary of bikeshedding over the syntax. However, I don't > see why the Erlang grammar can not support the Python syntax (I'm not > calling this JSON, because JSON requires double quoted keys). > > {a: "0", b: "1"} Because {a:f(), b:g()} is already legal Erlang syntax for a tuple with two function calls to compute its elements. > I would also make the point that there are very few single-language > programmers. Context switching is not free for humans either. It would > be great to be able to translate between JS/Python/other and Erlang > with as few hurdles as possible. Using familiar syntax for a fairly > similar task seems very useful to me. > > I don't understand this example [p16] > > foo:bar(Ugh), > > How is this ambiguous? Are you trying to set the bar element from the > foo frame? Your next line is unclear. > > If, following JSON, you use colon for maplets, your comprehension comes > to a juddering halt while you try to gure out if this is a maplet or a > remote function call. BAD! > > JSON is just a serialisation format. It is not JavaScript. JavaScript > would require you something closer to: foo["bar"] = Ugh. Is that what > you intended? Actually, no. {foo:bar(Ugh)} is 100% leval JavaScript. s/JSON/JavasScript in the version I just put up. The whole point here is that foo:bar(Ugh) is a legal Erlang function call. We don't want the same sequence of tokens *also* meaning "bind field foo to value of bar(Ugh)". I'm painfully aware of the multilanguage programmer issue. The last time I counted, I'd used over 200 programming languages and could remember more than I wanted to about each of them. Just today I wrote g: Integer * Integer -> Integer in a Haskell Program, where I should have written g :: Integer -> Integer -> Integer I'd used ML syntax... I get far too confused by R code doing f(x=y=g()) where the first = means "bind argument by name" and the second = means "assign to variable" to be happy about two different meanings for = in Erlang. At least using "~" means (a) ***NOT*** innovating, other people have used ~ for this purposebefore; and (b) *NOT* using tokens ambiguously. From ok@REDACTED Fri May 4 03:54:46 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 4 May 2012 13:54:46 +1200 Subject: [erlang-questions] New draft of frames proposal (was Re: Frames proposal) In-Reply-To: References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> Message-ID: <4DEFE0FF-E839-42CC-B180-DCEA1B3406AD@cs.otago.ac.nz> On 3/05/2012, at 10:55 PM, Max Lapshin wrote: >> - They have no default value >> - Only atoms may be keys > > These two poins are very interesting to discuss. > > > Problem is with handling external data. Response 1: Frames are very explicitly a replacement for RECORDS. You cannot currently use records in external data. Response 2: EEP 20 (http://www.erlang.org/eeps/eep-0020.html) has now been around for nearly four years. It offers a permanent fix to the limited size of the atom table, a fix which has been used before in another concurrent programming language. That's not the only way to fix the atom table problem. SWI Prolog offers true multicore concurrency AND a garbage collected atom table. That's been around for a few years now too. By any engineering standard I can think of, fixing this vulnerability in Erlang should have very high priority. The frames proposal presupposed that this flaw has been fixed somehow. We *cannot* let this flaw warp Erlang programming for the rest of our lives. > > We will not be able to use frames for handling user data if only atoms > can be keys, but if we allow to store atoms, binaries and strings as > keys, we will get a mess with different ways to introduce keys. We *WILL* be able to use frames for handling user data WHEN THE ATOM TABLE BUG IS FIXED. And it is LONG past time for that to happen. From tristan.sloughter@REDACTED Fri May 4 04:06:27 2012 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Thu, 3 May 2012 21:06:27 -0500 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> <4FA1CEC8.1060807@gmail.com> Message-ID: Ah yes, I briefly looked at colombo when you mentioned it on the erlware list. I'll give that a deeper look to see how much of what I need is covered by it and possibly simply start extending it. Tristan On Thu, May 3, 2012 at 4:52 PM, Torben Hoffmann wrote: > That actually sounds like a good step forward - it almost, but only almost > makes me sad that I have created my own dependency management tool called > colombo (alpha vers. is on github/lehoff). > > I haven't spent much time on it, but it can fetch apps from git repos as > well as their dependencies, one can specify a tag or a branch to check > out, a simple build can be done and you can get a version tagged version > installed in the subdir lib of the dir where you you colombo (vers is taken > from the app file or a default is choosen). > > very much alpha quality, but it shows that small dedicated tools aren't > that hard to create for the Erlang eco system. > > I will continue improving colombo since it helps me solve my own problem, > but it would be nice to have "real" users as well that could help form the > functionality of the tool. Or tools since I might already have added too > much functionality to colombo. > > Cheers, > Torben > > Sent from my iPad > > On 03/05/2012, at 15.30, Tristan Sloughter > wrote: > > I feel like someone said Travis-CI wasn't the answer... But I am now > having the thought it is, at least to make something I'd like to see > without me having to do it. > > The question is would the community be interested in this if > I implement it. Here is what I'm talking about: > > 1) A github hook that informs X that an update to a branch or tag is > pushed. > 2) This hook does not only ensure tests pass but also that the version > numbers are correct. If you tag a project as 0.1.0 and the .app file has > {vsn, 0.1.0} it fails > 3) Assuming these criteria are met the agner repo for the project is > either updated or created. Instead of @master it would use a piece of the > git hash for the branch pointer (using a moving target -- a branch name -- > as the version of an app doesn't make sense), for a tag it would use the > tag name. > > Now, this is only part of the large solution people want, and I agree with > them we should do. > > But, I think it is a start and something that can be done in small enough > amount of effort to be done soon. > > Tristan > > On Wed, May 2, 2012 at 7:18 PM, Tim Watson wrote: > >> On 02/05/2012 21:13, Ciprian Dorin Craciun wrote: >> >>> On Wed, May 2, 2012 at 11:03 PMv, Tim Watson> >>> wrote: >> >> Eric Merit and I have had some lengthy discussions about this on the >>>> Erlware mailing list and have some ideas that I think are pretty solid. >>>> >>> I'm glad to hear this. (I'll give it a look.) >>> >> Cool thanks - please do leave any feedback you feel is relevant. >> >> The thing is though, you don't just need tools - you also need people to >>>> package their stuff up using the tools. >>>> >>> I agree about this. In fact I think currently there are a lot of >>> tools with diverging solutions. >>> >>> Also I don't think that one "blessed" tool would be the final >>> answer. I would have taken a somehow different road, similar maybe to >>> how Go is going (although they do have the "one" tool): i.e. strict >>> conventions. >>> >> Yes I agree that '1 tool to rule them all' isn't going to work. Eric and >> I have discussed building a unix-y pipeline of tools to deal with >> local/remote package and repository management, dependency resolution, >> installing and/or generating a code path (or some other structure) for >> bootstrapping either ERL_LIBS or a call to code:path, etc. >> >> The assumption we have is that various parts can be dealt with be >> different tools providing they respect the APIs, so you can build with >> rebar or sinan or make or whatever. We are also planning on using reltool >> (or some alternative/replica of sorts) to deal with the packaging bits, as >> well as a set of tools for publication, code/package signing and the like. >> >> We would also like to support different release types (e.g., embedded vs. >> non), distinguish between applications and library applications and so on. >> >> >>> What do I mean is this: >>> a) it would be nice to have a standardized way to specify "extra" >>> options to compile an Erlang module. (We have the module attributes >>> that we could use.) (the same for C sources); >>> b) we already have a standard project layout; (i.e. `./src`, >>> `./include`, etc.) >>> c) we already have a standard project dependencies (i.e. the `app` >>> file); >>> >>> All we need to do is be consistent in this convention, and then >>> all the various build and packaging systems could adapt. >>> >> Packaging wise, I don't see how any of this really helps so much, >> although I completely agree that sticking to this consistent layout is a >> good thing. >> >> Using the app file for dependencies is fine at runtime - I do this in >> https://github.com/hyperthunk/**appstart- but if you're connecting to a (possibly remote) artefact repository and >> fetching stuff, then you need to specify the publisher/signer (because 2 >> OTP applications could be written with the same name, and of course with >> FOSS project forking this happens all the time), the application name, and >> the version. You can't whack all of that into your .app file without >> breaking various things. >> >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mapandfold@REDACTED Fri May 4 05:08:22 2012 From: mapandfold@REDACTED (Justus) Date: Fri, 4 May 2012 11:08:22 +0800 Subject: [erlang-questions] source file encoding In-Reply-To: References: <4EF078C1.9090105@gmail.com> <20111229080818.GA15295@smp.if.uj.edu.pl> <20111229101517.GC15295@smp.if.uj.edu.pl> Message-ID: Hi all, I have tried to use BOM in source files, if no BOM is found, then fallback to latin1. Patches based on 5.9 are attached. Both strings (lists) and binaries are handled properly. f() -> {"??, ??", <<"??, ??"/utf8>>}. -- Best Regards, Justus -------------- next part -------------- A non-text attachment was scrubbed... Name: sys_pre_expand.patch Type: application/octet-stream Size: 937 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: epp.patch Type: application/octet-stream Size: 1152 bytes Desc: not available URL: From max.lapshin@REDACTED Fri May 4 07:19:45 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 4 May 2012 09:19:45 +0400 Subject: [erlang-questions] Erlang and FIX protocol In-Reply-To: <4FA329D0.7010701@aleynikov.org> References: <9DDA4A0F-4321-40F7-A896-3B41E737F60D@gmail.com> <4FA329D0.7010701@aleynikov.org> Message-ID: On Fri, May 4, 2012 at 4:58 AM, Serge Aleynikov wrote: > http://www.validfix.com/fix-analyzer.html). ?The second approach was to > implement a simple parser in C and use Erlang just to handle the session and > application level layer of the protocol stack. I know how to generate such a parser, so I also thought about it. > Very likely you don't need to support a full set of FIX features and if so, > you can make some shortcuts that simplify the implementation. I also thought about using only those subset, which is supported by my broker. About long structures?? yes. You are right From max.lapshin@REDACTED Fri May 4 09:30:18 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 4 May 2012 11:30:18 +0400 Subject: [erlang-questions] Erlang and FIX protocol In-Reply-To: References: <9DDA4A0F-4321-40F7-A896-3B41E737F60D@gmail.com> <4FA329D0.7010701@aleynikov.org> Message-ID: There is other problem with long structs: it is impossible to dump them with ~p and understand, what is written there. proplist is better in this sense. From watson.timothy@REDACTED Fri May 4 09:32:13 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 4 May 2012 08:32:13 +0100 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> <4FA1CEC8.1060807@gmail.com> Message-ID: Is Columbo somehow different from what rebar already does in this space? On 4 May 2012, at 03:06, Tristan Sloughter wrote: > Ah yes, I briefly looked at colombo when you mentioned it on the erlware list. I'll give that a deeper look to see how much of what I need is covered by it and possibly simply start extending it. > > Tristan > > On Thu, May 3, 2012 at 4:52 PM, Torben Hoffmann wrote: > That actually sounds like a good step forward - it almost, but only almost makes me sad that I have created my own dependency management tool called colombo (alpha vers. is on github/lehoff). > > I haven't spent much time on it, but it can fetch apps from git repos as well as their dependencies, one can specify a tag or a branch to check out, a simple build can be done and you can get a version tagged version installed in the subdir lib of the dir where you you colombo (vers is taken from the app file or a default is choosen). > > very much alpha quality, but it shows that small dedicated tools aren't that hard to create for the Erlang eco system. > > I will continue improving colombo since it helps me solve my own problem, but it would be nice to have "real" users as well that could help form the functionality of the tool. Or tools since I might already have added too much functionality to colombo. > > Cheers, > Torben > > Sent from my iPad > > On 03/05/2012, at 15.30, Tristan Sloughter wrote: > >> I feel like someone said Travis-CI wasn't the answer... But I am now having the thought it is, at least to make something I'd like to see without me having to do it. >> >> The question is would the community be interested in this if I implement it. Here is what I'm talking about: >> >> 1) A github hook that informs X that an update to a branch or tag is pushed. >> 2) This hook does not only ensure tests pass but also that the version numbers are correct. If you tag a project as 0.1.0 and the .app file has {vsn, 0.1.0} it fails >> 3) Assuming these criteria are met the agner repo for the project is either updated or created. Instead of @master it would use a piece of the git hash for the branch pointer (using a moving target -- a branch name -- as the version of an app doesn't make sense), for a tag it would use the tag name. >> >> Now, this is only part of the large solution people want, and I agree with them we should do. >> >> But, I think it is a start and something that can be done in small enough amount of effort to be done soon. >> >> Tristan >> >> On Wed, May 2, 2012 at 7:18 PM, Tim Watson wrote: >> On 02/05/2012 21:13, Ciprian Dorin Craciun wrote: >> On Wed, May 2, 2012 at 11:03 PMv, Tim Watson wrote: >> Eric Merit and I have had some lengthy discussions about this on the Erlware mailing list and have some ideas that I think are pretty solid. >> I'm glad to hear this. (I'll give it a look.) >> Cool thanks - please do leave any feedback you feel is relevant. >> >> The thing is though, you don't just need tools - you also need people to package their stuff up using the tools. >> I agree about this. In fact I think currently there are a lot of >> tools with diverging solutions. >> >> Also I don't think that one "blessed" tool would be the final >> answer. I would have taken a somehow different road, similar maybe to >> how Go is going (although they do have the "one" tool): i.e. strict >> conventions. >> Yes I agree that '1 tool to rule them all' isn't going to work. Eric and I have discussed building a unix-y pipeline of tools to deal with local/remote package and repository management, dependency resolution, installing and/or generating a code path (or some other structure) for bootstrapping either ERL_LIBS or a call to code:path, etc. >> >> The assumption we have is that various parts can be dealt with be different tools providing they respect the APIs, so you can build with rebar or sinan or make or whatever. We are also planning on using reltool (or some alternative/replica of sorts) to deal with the packaging bits, as well as a set of tools for publication, code/package signing and the like. >> >> We would also like to support different release types (e.g., embedded vs. non), distinguish between applications and library applications and so on. >> >> >> What do I mean is this: >> a) it would be nice to have a standardized way to specify "extra" >> options to compile an Erlang module. (We have the module attributes >> that we could use.) (the same for C sources); >> b) we already have a standard project layout; (i.e. `./src`, >> `./include`, etc.) >> c) we already have a standard project dependencies (i.e. the `app` file); >> >> All we need to do is be consistent in this convention, and then >> all the various build and packaging systems could adapt. >> Packaging wise, I don't see how any of this really helps so much, although I completely agree that sticking to this consistent layout is a good thing. >> >> Using the app file for dependencies is fine at runtime - I do this in https://github.com/hyperthunk/appstart - but if you're connecting to a (possibly remote) artefact repository and fetching stuff, then you need to specify the publisher/signer (because 2 OTP applications could be written with the same name, and of course with FOSS project forking this happens all the time), the application name, and the version. You can't whack all of that into your .app file without breaking various things. >> >> >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Fri May 4 09:37:14 2012 From: dangud@REDACTED (Dan Gudmundsson) Date: Fri, 4 May 2012 09:37:14 +0200 Subject: [erlang-questions] Capturing keystrokes in wxWindows Issues + Segmentation Fault In-Reply-To: <1336093777.24034.YahooMailNeo@web130101.mail.mud.yahoo.com> References: <1336093777.24034.YahooMailNeo@web130101.mail.mud.yahoo.com> Message-ID: How about this: Wx = wx:new(), Frame = wxFrame:new(Wx, ?wxID_ANY, "Press any key"), Panel = wxPanel:new(Frame, []), wxFrame:connect(Frame, enter_window, [{callback, fun(_,_) -> io:format("Set focus~n"), wxWindow:setFocus(Panel) end}]), KeyEvent = fun(Ev,Obj) -> io:format("Got ~p~n",[Ev]), wxEvent:skip(Obj) end, [wxWindow:connect(Panel, Types, [{callback,KeyEvent}]) || Types <- [key_down, key_up, char]], wxWindow:connect(Frame, char_hook, [{callback,KeyEvent}]), wxFrame:show(Frame). Regarding Issue #5 yes it will crash since the parent of frame is null pointer, you could have checked with wx:is_null(Obj), the erlang wx api is just wrapper of wxWidgets, so if you do bad things, bad things will happen. /Dan On Fri, May 4, 2012 at 3:09 AM, Tom Parker wrote: > Background: I'm new to Erlang, so just getting the hang of things... I'm > trying to get key events back from a window, and exploring how to do that. > > If I do this in the erl shell: > > Wx=wx:new(). > F=wxFrame:new(Wx,-1,"Hello, World!"). > wxFrame:show(F). > Canvas = wxPanel:new(F, []). > wxWindow:setSize(Canvas,200,100). > DC=wxPaintDC:new(Canvas). > wxDC:clear(DC). > wxEvtHandler:connect(F,key_down, [{callback, fun(Rec, Obj) -> > io:format("Received:~p~n",[Rec]) end}]). > > Issue #1: With this setup, I get no events back (regardless of what I select > in the UI prior to pressing any keys). > > If I change to key_up, I will get events, but ... > > Issue #2: ... the events only show up after I click into the (white due to > the wxDC:clear call) Canvas, not just the title bar of the window or the > other (gray) part of the Frame (see below for where the focus is before > clicking in the Canvas).? I'm unsure why it has to be a child that has focus > for me to get a callback. > > I also get char_hook events if I use that instead of key_down > > Issue #3: ... but no char_hook for things like the "Shift" key - the ONLY > event I seem to be able to get from Shift is "key_up" > > Issue #4: No char events at all if I connect to those (just like key_down) > > (I think those are the 4 types of keyboard events I might find) > > Since I wanted to try to capture events even when the title bar was clicked > (vs just contents of the window), I set up a test to identify where the > focus is when I select the title bar.? Code not relevant here, but clicking > on the title bar of the window puts focus (as reported by > wxWindow.findFocus() into a wxWindow that is not Frame F. > Focus:{wx_ref,0,wxWindow,[]} > Transferring Focus to the Frame doesn't help receipt of events: > Focus:{wx_ref,35,wxWindow,[]} > Clicking into the Canvas puts focus on Canvas, and allows me to receive > key_up and (some) char_hook events: > Focus:{wx_ref,36,wxWindow,[]} > > Attempting the connect the event handler to Wx produces errors, so that > doesn't work (fair enough), but it turns out Wx really isn't the parent of > F.? The wxWindow (ID 0) that has the focus IS the parent of Frame F as > reported by wxWindow.getParent(F).... so I tried this: > > Wx=wx:new(). > F=wxFrame:new(Wx,-1,"Hello, World!"). > wxFrame:show(F). > Fparent=wxWindow:getParent(F). > wxEvtHandler:connect(Fparent,key_down, [{callback, fun(Rec, Obj) -> > io:format("Received:~p~n",[Rec]) end}]). > > Issue #5: This causes a segmentation fault. > > Erlang 15B01 from a source build (plain untar, make, make install... > necessary since I wanted wxWindows) on Ubuntu 10.04-LTS > > Any thoughts, ideas or feedback?? Thanks. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From watson.timothy@REDACTED Fri May 4 10:03:56 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 4 May 2012 09:03:56 +0100 Subject: [erlang-questions] New draft of frames proposal (was Re: Frames proposal) In-Reply-To: <4DEFE0FF-E839-42CC-B180-DCEA1B3406AD@cs.otago.ac.nz> References: <4714288D-1696-49E3-9854-8707CC84CE6F@cs.otago.ac.nz> <6D16F4C6-04E9-4C54-8E04-E7A130D2B129@cs.otago.ac.nz> <73026B30-F32F-4541-9816-B993F60841AD@cs.otago.ac.nz> <4DEFE0FF-E839-42CC-B180-DCEA1B3406AD@cs.otago.ac.nz> Message-ID: <6ADEA065-384A-4EBC-A81E-35DE1D69F677@gmail.com> Anyone on the OTP team wish to comment on whether or not this is part of the roadmap for R16/17? On 4 May 2012, at 02:54, "Richard O'Keefe" wrote: > > On 3/05/2012, at 10:55 PM, Max Lapshin wrote: > >>> - They have no default value >>> - Only atoms may be keys >> >> These two poins are very interesting to discuss. >> >> >> Problem is with handling external data. > > Response 1: > > Frames are very explicitly a replacement for RECORDS. > You cannot currently use records in external data. > > > Response 2: > > EEP 20 (http://www.erlang.org/eeps/eep-0020.html) > has now been around for nearly four years. It offers > a permanent fix to the limited size of the atom table, > a fix which has been used before in another concurrent > programming language. > > That's not the only way to fix the atom table problem. > SWI Prolog offers true multicore concurrency AND a > garbage collected atom table. That's been around for > a few years now too. > > By any engineering standard I can think of, fixing this > vulnerability in Erlang should have very high priority. > The frames proposal presupposed that this flaw has been > fixed somehow. We *cannot* let this flaw warp Erlang > programming for the rest of our lives. > >> >> We will not be able to use frames for handling user data if only atoms >> can be keys, but if we allow to store atoms, binaries and strings as >> keys, we will get a mess with different ways to introduce keys. > > We *WILL* be able to use frames for handling user data > WHEN THE ATOM TABLE BUG IS FIXED. > > And it is LONG past time for that to happen. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From freza@REDACTED Fri May 4 10:14:20 2012 From: freza@REDACTED (Jachym Holecek) Date: Fri, 4 May 2012 04:14:20 -0400 Subject: [erlang-questions] BIF or NIF for extending Erlang on OS which can't dynamic link In-Reply-To: References: Message-ID: <20120504081420.GA19861@circlewave.net> # Peer Stritzinger 2012-05-03: > I'm currently porting Erlang to RTEMS a small embedded OS that can't > dynamic link. > > If I want to extend Erlang with functionality written in C (e.g. for > accessing hardware or RTEMS features) I always have to statically link > this code to the Erlang + RTEMS image -- the OS is just a executive > which is linked as a lib to Erlang which together boots directly on > the hardware. > > If I want to use NIFs or linked drivers there is always some dynamic > linking going on which fails for me since there is no dlopen etc. > functionality in the basic RTEMS system. I thought one can have linked-in drivers linked statically during BEAM compilation? Searching for 'driver_tab' in erts/emulator/Makefile.in should be a good start. HTH, -- Jachym From roberto.majadas@REDACTED Fri May 4 11:38:02 2012 From: roberto.majadas@REDACTED (Roberto Majadas Lopez) Date: Fri, 4 May 2012 11:38:02 +0200 Subject: [erlang-questions] ERL & SIG[TERM|INT|...] Message-ID: Hello people: I'm working in a erlang daemon and i want to trap a sigterm or another signal to launch some "safe-close" functions. I've been looking information about it in the erlang documentation and google and i've not found anything about it. Can anyone help me? Thanks :) Roberto -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Fri May 4 11:46:03 2012 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 4 May 2012 11:46:03 +0200 Subject: [erlang-questions] Proposals for who will become Erlang User of the Year are invited. Message-ID: Proposals for who will become Erlang User of the Year are invited. Please send me proposed names with a short motivation. The selection will be made by a panel consisting of Joe Armstrong, Kenneth Lundin (manager of the Erlang/OTP team at Ericsson), Robert Virding (technical expert at Erlang Solutions Ltd) and some of the last few years' recipients of the award. The award will be presented at the Erlang User Conference in Stockholm on 28 May. In 2011 it was awarded to Dave "Dizzy" Smith at Basho in recognition of his development of Rebar. /Joe From peerst@REDACTED Fri May 4 13:04:53 2012 From: peerst@REDACTED (Peer Stritzinger) Date: Fri, 4 May 2012 13:04:53 +0200 Subject: [erlang-questions] BIF or NIF for extending Erlang on OS which can't dynamic link In-Reply-To: <20120504081420.GA19861@circlewave.net> References: <20120504081420.GA19861@circlewave.net> Message-ID: On Fri, May 4, 2012 at 10:14 AM, Jachym Holecek wrote: > # Peer Stritzinger 2012-05-03: > I thought one can have linked-in drivers linked statically during BEAM > compilation? Searching for 'driver_tab' in erts/emulator/Makefile.in > should be a good start. Oh great this is convenient. Is there any similar functionality for NIF modules? Cheers, -- Peer Stritzinger From max.lapshin@REDACTED Fri May 4 15:22:42 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 4 May 2012 17:22:42 +0400 Subject: [erlang-questions] Safe way to cache atoms in NIF: driver_mk_atom replacement Message-ID: Is there a way to cache atoms in NIF? Is it ok just to store in static variable result of enif_make_atom ? From lukas@REDACTED Fri May 4 15:35:17 2012 From: lukas@REDACTED (Lukas Larsson) Date: Fri, 4 May 2012 15:35:17 +0200 Subject: [erlang-questions] Safe way to cache atoms in NIF: driver_mk_atom replacement In-Reply-To: References: Message-ID: If I understand you correctly the answer is yes. I think the crypto implementation[1] does exactly this and since Sverker wrote both crypto and the NIF support I would say that it is safe :) Lukas [1] https://github.com/erlang/otp/blob/maint/lib/crypto/c_src/crypto.c#L300 On Fri, May 4, 2012 at 3:22 PM, Max Lapshin wrote: > Is there a way to cache atoms in NIF? > > Is it ok just to store in static variable result of enif_make_atom ? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Fri May 4 15:51:58 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Fri, 4 May 2012 15:51:58 +0200 Subject: [erlang-questions] Any video streaming library? Message-ID: <0333AA51-DC5B-4B06-BD38-FB7C6ABF3B07@gmail.com> Hi guys, I'm looking after a simple video streaming library (or tricks). The idea is to used it on Windows to stream a bunch of files. Regards, Zabrane From sverker@REDACTED Fri May 4 16:11:01 2012 From: sverker@REDACTED (Sverker Eriksson) Date: Fri, 4 May 2012 16:11:01 +0200 Subject: [erlang-questions] Safe way to cache atoms in NIF: driver_mk_atom replacement In-Reply-To: References: Message-ID: <4FA3E375.3090200@erix.ericsson.se> Currently atoms are independent of environment (ErlNifEnv). That is, you can cache atoms by storing them in static variables as done in crypto.c. This is an undocumented feature however. To compensate for future possible introduction of atom-GC I have thought of documenting this feature with some restriction. Something like "Atoms created in load/upgrade callbacks are static and can be used in any environment" /Sverker, Erlang/OTP Lukas Larsson wrote: > If I understand you correctly the answer is yes. > > I think the crypto implementation[1] does exactly this and since > Sverker wrote both crypto and the NIF support I would say that it is > safe :) > > Lukas > > [1] https://github.com/erlang/otp/blob/maint/lib/crypto/c_src/crypto.c#L300 > > On Fri, May 4, 2012 at 3:22 PM, Max Lapshin wrote: > >> Is there a way to cache atoms in NIF? >> >> Is it ok just to store in static variable result of enif_make_atom ? >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From comptekki@REDACTED Fri May 4 16:30:47 2012 From: comptekki@REDACTED (Wes James) Date: Fri, 4 May 2012 08:30:47 -0600 Subject: [erlang-questions] Any video streaming library? In-Reply-To: <0333AA51-DC5B-4B06-BD38-FB7C6ABF3B07@gmail.com> References: <0333AA51-DC5B-4B06-BD38-FB7C6ABF3B07@gmail.com> Message-ID: Did you ask google? erlang video streaming server -> http://erlyvideo.org/ -wes On Fri, May 4, 2012 at 7:51 AM, Zabrane Mickael wrote: > Hi guys, > > I'm looking after a simple video streaming library (or tricks). > The idea is to used it on Windows to stream a bunch of files. > > Regards, > Zabrane > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zabrane3@REDACTED Fri May 4 16:41:19 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Fri, 4 May 2012 16:41:19 +0200 Subject: [erlang-questions] Any video streaming library? In-Reply-To: References: <0333AA51-DC5B-4B06-BD38-FB7C6ABF3B07@gmail.com> Message-ID: Yep, I need a "library" not an "app". And something on Windows (erlyvideo used linked-in drivers and I'm not sure if we can compile it on Windows). Regards, Zabrane On May 4, 2012, at 4:30 PM, Wes James wrote: > Did you ask google? > > erlang video streaming server -> http://erlyvideo.org/ > > -wes > > On Fri, May 4, 2012 at 7:51 AM, Zabrane Mickael wrote: >> Hi guys, >> >> I'm looking after a simple video streaming library (or tricks). >> The idea is to used it on Windows to stream a bunch of files. >> >> Regards, >> Zabrane >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From jws@REDACTED Fri May 4 16:51:40 2012 From: jws@REDACTED (Jeff Schultz) Date: Sat, 5 May 2012 00:51:40 +1000 Subject: [erlang-questions] Safe way to cache atoms in NIF: driver_mk_atom replacement In-Reply-To: References: Message-ID: <20120504145140.GA5185@mulga.csse.unimelb.edu.au> On Fri, May 04, 2012 at 03:35:17PM +0200, Lukas Larsson wrote: > If I understand you correctly the answer is yes. > I think the crypto implementation[1] does exactly this and since > Sverker wrote both crypto and the NIF support I would say that it is > safe :) > On Fri, May 4, 2012 at 3:22 PM, Max Lapshin wrote: > > Is there a way to cache atoms in NIF? > > Is it ok just to store in static variable result of enif_make_atom ? Seems unlikely to remain true if EEP20 or atom GC is ever implemented. It would be quite a chore to find all such instances then. Jeff Schultz From trevorw@REDACTED Fri May 4 17:21:14 2012 From: trevorw@REDACTED (Trevor Woollacott) Date: Fri, 4 May 2012 17:21:14 +0200 Subject: [erlang-questions] Strange SSL transport_accept behaviour Message-ID: Hi, I've noticed strange behaviour when ssl:transport_accept/2 returns with a timeout. It appears that certain inet options are reset to internal values and are never changed back if accept encounters an error (such as timeout). In my case, I had set inet option {packet, http}, and after the accept timed out it had changed to 0 (raw). For example: > ssl:setopts( ListenSocket, [{packet, http}] ). ok > ssl:transport_accept( ListenSocket, Timeout ). {error, timeout} > ssl:getopts( ListenSocket, [packet] ). {ok, [{packet,0}]} The problem seems to have been introduced in Erlang R14B. Looking at the source code below, we can see that the initial inet options are stored in InetValues, and then the inet options are reset to internal_inet_values (which includes {packet,0}) and the inet options are only set back to InetValues if the accept was successful. Is this the correct behaviour? I'm thinking that for timeout (and possibly certain other errors) the listener socket should be reset with the user defined inet options? transport_accept(#sslsocket{pid = {ListenSocket, #config{cb=CbInfo, ssl=SslOpts}}, fd = new_ssl}, Timeout) -> %% The setopt could have been invoked on the listen socket %% and options should be inherited. EmOptions = emulated_options(), {ok, InetValues} = inet:getopts(ListenSocket, EmOptions), ok = inet:setopts(ListenSocket, internal_inet_values()), {CbModule,_,_, _} = CbInfo, case CbModule:accept(ListenSocket, Timeout) of {ok, Socket} -> ok = inet:setopts(ListenSocket, InetValues), {ok, Port} = inet:port(Socket), ConnArgs = [server, "localhost", Port, Socket, {SslOpts, socket_options(InetValues)}, self(), CbInfo], case ssl_connection_sup:start_child(ConnArgs) of {ok, Pid} -> ssl_connection:socket_control(Socket, Pid, CbModule); {error, Reason} -> {error, Reason} end; {error, Reason} -> {error, Reason} end; Regards, Trevor From per.melin@REDACTED Fri May 4 17:56:25 2012 From: per.melin@REDACTED (Per Melin) Date: Fri, 4 May 2012 17:56:25 +0200 Subject: [erlang-questions] Erlang and FIX protocol In-Reply-To: References: <9DDA4A0F-4321-40F7-A896-3B41E737F60D@gmail.com> Message-ID: <8BF4260F-7BB1-4088-A72E-DF3AC43834E1@gmail.com> Max Lapshin wrote: > On Thu, May 3, 2012 at 6:31 PM, Per Melin wrote: >> >> The problem with the syntax is that it is simplistic. It does not even give you enough information to be able to tokenize a message or to find message boundaries. See binary fields below. >> > > It is not exactly so. Standard tells that whole packet must start from > field 8 with proto version and next field must be byte length without > checksum. So this is a good way to detect real byte length. When you said that the syntax was VERY simple I assumed you took the view that FIX lives on top of a format of Tag=Value that you can first tokenize independent of the dictionary (which of course has some 900 fields and can be freely extended). It was that basic format I commented on. If we were to define a grammar that covered header/body/trailer separation, optional/required fields, optional/required repeating nested context-dependent groups and components, binary data fields, etc, then it wouldn't be very simple anymore, right? Or do you regard these as semantic concepts, not syntax? Since none of them have any lexical clues I find it weirdly hard to draw a clear line between syntax and semantics. How complete and strict an implementation are you aiming for? For example, if a required field or group is missing, a field appears in a message where it does not belong, a header field appears in the body, fields are out of order in a group, a value is not valid according to the spec, etc, are you going to detect it? Are you going to send a reject message? If you take a more relaxed stance many complexities melt away. My mistake was to be very strict in my implementation. >> You are not verifying the message checksum, which in the end is not as straight forward as it may first seem. > > What is bad with it? I had a distinct memory of a corner case, but now I can't remember what it was, and it doesn't come back to me when I read my own implementation. I must have been mistaken. >> 1> fix:decode(<<"8=FIX.4.4",1,"9=23",1,"35=0",1,"93=10",1,"89=A",1,"89=234",1,"10=999",1>>). >> > Specification tells: > > Data fields are always immediately preceded by a length field. The > length field should specify the number of bytes of the value of the > data field (up to but not > including the terminating SOH). > > "A|89=234|1" is 10 bytes but there must be terminating SOH after this. I picked 10 as length just to demonstrate that you ignore it. If you change signature length from 10 to 8 you should get a message that is valid, but still incorrectly parsed. I just noticed that a length of "ZZ" is parsed as 462 while ".." is parsed as -0.2 and "-1" is parsed as -29. ;) From per.melin@REDACTED Fri May 4 18:15:25 2012 From: per.melin@REDACTED (Per Melin) Date: Fri, 4 May 2012 18:15:25 +0200 Subject: [erlang-questions] Erlang and FIX protocol In-Reply-To: References: <9DDA4A0F-4321-40F7-A896-3B41E737F60D@gmail.com> <4FA329D0.7010701@aleynikov.org> Message-ID: <8840DAD5-17A3-442C-8889-F4CE9ED1316F@gmail.com> On May 4, 2012, at 09:30 , Max Lapshin wrote: > There is other problem with long structs: it is impossible to dump > them with ~p and understand, what is written there. proplist is > better in this sense. I only generate lookup tables from XML, not the parser, so records were out of the question for me. I use proplists. For example: 8=FIX.4.4|9=40|35=7|49=P|555=2|600=A|601=B|600=C|601=D Parses to: {app, % Class of message: application or admin {advertisement, % Message type [{sender_comp_id, "P"}], % Header [{no_legs, [[{leg_symbol, "A"}, {leg_symbol_sfx, "B"}], [{leg_symbol, "C"}, {leg_symbol_sfx, "D"}]]}], []}} % Trailer From max.lapshin@REDACTED Fri May 4 18:19:07 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 4 May 2012 20:19:07 +0400 Subject: [erlang-questions] Erlang and FIX protocol In-Reply-To: <8840DAD5-17A3-442C-8889-F4CE9ED1316F@gmail.com> References: <9DDA4A0F-4321-40F7-A896-3B41E737F60D@gmail.com> <4FA329D0.7010701@aleynikov.org> <8840DAD5-17A3-442C-8889-F4CE9ED1316F@gmail.com> Message-ID: Ok, I've found that FIX is not a simple Field=Value, but it has some problems with "DATA" type. It was not a problem to fix it. But there is a problem: fix is easily splitted into ordered list of Key,Value pairs, which are unuseable without further processing. Semantics of transforming this list into records with groups can be partially generated from XML info. This is exactly where I'm fighting right now: how to generate convenient and fast translator of this list into records. From max.lapshin@REDACTED Fri May 4 18:39:49 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 4 May 2012 20:39:49 +0400 Subject: [erlang-questions] Erlang and FIX protocol In-Reply-To: References: <9DDA4A0F-4321-40F7-A896-3B41E737F60D@gmail.com> <4FA329D0.7010701@aleynikov.org> <8840DAD5-17A3-442C-8889-F4CE9ED1316F@gmail.com> Message-ID: I've updated git repo. Now it has fix_splitter NIF-ed module. binary:split is sooo slow, that it can become a problem. I've moved low-level splitting of FIX data to C and high level translating of KV pairs to records left in erlang. Currently I'm designing how to handle groups, because it seems to be one of the most complicated parts of fix parsing: how can I understand that this field is a part of next group. From daniel.goertzen@REDACTED Fri May 4 19:26:58 2012 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Fri, 4 May 2012 12:26:58 -0500 Subject: [erlang-questions] enif_make_reverse_list() vs lists:reverse() Message-ID: The documentation for?enif_make_reverse_list() says ... "This function should only be used on short lists as a copy will be created of the list which will not be released until after the nif returns." I have a long-ish list. Should I be opting to use lists:reverse() instead of enif_make_reverse_list()? I assumed they were the same and for my situation I can use either, but the documentation suggests the nif version may be worse. Dan. From noiddicle@REDACTED Fri May 4 19:33:29 2012 From: noiddicle@REDACTED (Red Davies) Date: Fri, 4 May 2012 13:33:29 -0400 Subject: [erlang-questions] rebar and the observer application. Message-ID: Greetings, I'm still very new, please forgive me if this is an easy question to answer. I'm trying to use observer to view my application VM. The application is run as follows: rel/xeuler1/bin/xeuler1 console Exec: /home/red/projects/dev--/xeuler1/rel/xeuler1/erts-5.9/bin/erlexec -boot /home/red/projects/dev--/xeuler1/rel/xeuler1/releases/1/xeuler1 -mode embedded -config /home/red/projects/dev--/xeuler1/rel/xeuler1/releases/1/sys.config -args_file /home/red/projects/dev--/xeuler1/rel/xeuler1/releases/1/vm.args -- console Root: /home/red/projects/dev--/xeuler1/rel/xeuler1 Erlang R15B (erts-5.9) [source] [64-bit] [smp:4:4] [async-threads:0] [kernel-poll:false] Eshell V5.9 (abort with ^G) (xeuler1@REDACTED)1> I run up a second erlang instance with my observer application as follows: erl -name observer -hidden -setcookie xeuler1 -run observer The nodes are able to communicate: (observer@REDACTED)1> net:ping(xeuler1@REDACTED). pong However, when I try to connect to the remote node using the observer UI I get the following output: =ERROR REPORT==== 4-May-2012::13:22:17 === node: 'xeuler1@REDACTED' call: {observer_backend,sys_info,[]} reason: {badrpc, {'EXIT', {undef, [{observer_backend,sys_info,[],[]}, {rpc,'-handle_call_call/6-fun-0-',5, [{file,"rpc.erl"},{line,203}]}]}}} So, my understanding of the problem is that my reltool generated environment doesn't have a deployed version of observer. I've added it to reltool.config the best I can understand it and a find rel/ | grep observer does see observer related files in there. If I try to run observer:start() from my actual xeuler1 VM then it hangs indefinately. Any advice on how I to debug this? Thanks, Red -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Fri May 4 20:14:49 2012 From: garazdawi@REDACTED (Lukas Larsson) Date: Fri, 4 May 2012 20:14:49 +0200 Subject: [erlang-questions] enif_make_reverse_list() vs lists:reverse() In-Reply-To: References: Message-ID: <2413D08D-72EC-40DF-9AAD-366E370611B4@gmail.com> Hi, The nif version is blocking while the bif isn't. So for long lists the nif will block the scheduler while the bif will not. If the blocking is acceptable or not is up to you. If unsure using lists:reverse is the safe choice. Sent from me On 4 maj 2012, at 19:26, Daniel Goertzen wrote: > The documentation for enif_make_reverse_list() says ... > > "This function should only be used on short lists as a copy will be > created of the list which will not be released until after the nif > returns." > > I have a long-ish list. Should I be opting to use lists:reverse() > instead of enif_make_reverse_list()? I assumed they were the same and > for my situation I can use either, but the documentation suggests the > nif version may be worse. > > Dan. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From tuncer.ayaz@REDACTED Fri May 4 20:34:56 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 4 May 2012 20:34:56 +0200 Subject: [erlang-questions] rebar and the observer application. In-Reply-To: References: Message-ID: On Fri, May 4, 2012 at 7:33 PM, Red Davies wrote: > Greetings, > > I'm still very new, please forgive me if this is an easy question to answer. > > I'm trying to use observer to view my application VM. The application is > run as follows: > > rel/xeuler1/bin/xeuler1 console > Exec: /home/red/projects/dev--/xeuler1/rel/xeuler1/erts-5.9/bin/erlexec > -boot /home/red/projects/dev--/xeuler1/rel/xeuler1/releases/1/xeuler1 -mode > embedded -config > /home/red/projects/dev--/xeuler1/rel/xeuler1/releases/1/sys.config > -args_file /home/red/projects/dev--/xeuler1/rel/xeuler1/releases/1/vm.args > -- console > Root: /home/red/projects/dev--/xeuler1/rel/xeuler1 > Erlang R15B (erts-5.9) [source] [64-bit] [smp:4:4] [async-threads:0] > [kernel-poll:false] > > Eshell V5.9 (abort with ^G) > (xeuler1@REDACTED)1> > > I run up a second erlang instance with my observer application as follows: > erl -name observer -hidden -setcookie xeuler1 -run observer > > The nodes are able to communicate: > (observer@REDACTED)1> net:ping(xeuler1@REDACTED). > pong > > However, when I try to connect to the remote node using the observer UI I > get the following output: > =ERROR REPORT==== 4-May-2012::13:22:17 === > node: 'xeuler1@REDACTED' > call: {observer_backend,sys_info,[]} > reason: {badrpc, > {'EXIT', > {undef, > [{observer_backend,sys_info,[],[]}, > {rpc,'-handle_call_call/6-fun-0-',5, > [{file,"rpc.erl"},{line,203}]}]}}} > > So, my understanding of the problem is that my reltool generated environment > doesn't have a deployed version of observer. I've added it to > reltool.config the best I can understand it and a find rel/ | grep observer > does see observer related files in there. You should add _only_ appmon and runtime_tools to the release for xeuler1. > If I try to run observer:start() from my actual xeuler1 VM then it hangs > indefinately. Although you shouldn't run observer on xeuler1, does it stop hanging if you add wx? > Any advice on how I to debug this? Use the failed rpc:call('xeuler1@REDACTED', observer, sys_info, []) as a hint for which app is missing. From tuncer.ayaz@REDACTED Fri May 4 20:38:34 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Fri, 4 May 2012 20:38:34 +0200 Subject: [erlang-questions] rebar and the observer application. In-Reply-To: References: Message-ID: On Fri, May 4, 2012 at 8:34 PM, Tuncer Ayaz wrote: > Use the failed > rpc:call('xeuler1@REDACTED', observer, sys_info, []) Minor typo. Correct version: rpc:call('xeuler1@REDACTED', observer_backend, sys_info, []) From ixmatus@REDACTED Sat May 5 04:00:32 2012 From: ixmatus@REDACTED (Parnell Springmeyer) Date: Fri, 04 May 2012 21:00:32 -0500 Subject: [erlang-questions] How do I capture the system pid of an OTP application? Message-ID: I have an OTP application and a release generated. Now, the problem I'm having is capturing the operating system level PID that it is assigned on startup. Using pgrep isn't very useful because the process is listed as "[beam.smp]" and there are more than one OTP applications running on this system! -- Parnell "ixmatus" Springmeyer (http://ixmat.us) From bob@REDACTED Sat May 5 04:03:57 2012 From: bob@REDACTED (Bob Ippolito) Date: Fri, 4 May 2012 19:03:57 -0700 Subject: [erlang-questions] How do I capture the system pid of an OTP application? In-Reply-To: References: Message-ID: On Fri, May 4, 2012 at 7:00 PM, Parnell Springmeyer wrote: > I have an OTP application and a release generated. Now, the problem I'm > having is capturing the operating system level PID that it is assigned > on startup. > > Using pgrep isn't very useful because the process is listed as > "[beam.smp]" and there are more than one OTP applications running on > this system! Well, there's os:getpid/0, you could write a pid file with that... or you could run the process it in the foreground and manage the process with something like daemontools, launchd, etc. -bob -------------- next part -------------- An HTML attachment was scrubbed... URL: From cloudzen@REDACTED Sat May 5 11:18:02 2012 From: cloudzen@REDACTED (skyman) Date: Sat, 5 May 2012 17:18:02 +0800 (CST) Subject: [erlang-questions] Questions about ETS locking Message-ID: <2d34a408.e600.1371c4aa309.Coremail.cloudzen@163.com> Hi all, I know when access an ETS table, there are wo locks that need to be taken. 1) A lock to access the meta table, to convert the numeric table identifier to a pointer to the actual table. 2) The lock for the table itself (either a read or write lock). I know in the SMP mode?an ETS table must be locked when access it, but I don't make sure that must an ETS table be locked in the non-SMP mode(only one scheduler)? Another question is, whether the default ETS table locking is on table-level or on record-level? Thanks in advance! -------------- next part -------------- An HTML attachment was scrubbed... URL: From tristan.sloughter@REDACTED Sat May 5 20:20:34 2012 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Sat, 5 May 2012 13:20:34 -0500 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> <4FA1CEC8.1060807@gmail.com> Message-ID: Tuncer's idea of Hackage for Erlang is interesting. And if it were possible to move something like Hackage to do for Erlang what it does for Haskell, instead of writing from scratch, that could be even better. Tristan On Fri, May 4, 2012 at 2:32 AM, Tim Watson wrote: > Is Columbo somehow different from what rebar already does in this space? > > > > On 4 May 2012, at 03:06, Tristan Sloughter > wrote: > > Ah yes, I briefly looked at colombo when you mentioned it on the erlware > list. I'll give that a deeper look to see how much of what I need is > covered by it and possibly simply start extending it. > > Tristan > > On Thu, May 3, 2012 at 4:52 PM, Torben Hoffmann wrote: > >> That actually sounds like a good step forward - it almost, but only >> almost makes me sad that I have created my own dependency management tool >> called colombo (alpha vers. is on github/lehoff). >> >> I haven't spent much time on it, but it can fetch apps from git repos as >> well as their dependencies, one can specify a tag or a branch to check >> out, a simple build can be done and you can get a version tagged version >> installed in the subdir lib of the dir where you you colombo (vers is taken >> from the app file or a default is choosen). >> >> very much alpha quality, but it shows that small dedicated tools aren't >> that hard to create for the Erlang eco system. >> >> I will continue improving colombo since it helps me solve my own problem, >> but it would be nice to have "real" users as well that could help form the >> functionality of the tool. Or tools since I might already have added too >> much functionality to colombo. >> >> Cheers, >> Torben >> >> Sent from my iPad >> >> On 03/05/2012, at 15.30, Tristan Sloughter >> wrote: >> >> I feel like someone said Travis-CI wasn't the answer... But I am now >> having the thought it is, at least to make something I'd like to see >> without me having to do it. >> >> The question is would the community be interested in this if >> I implement it. Here is what I'm talking about: >> >> 1) A github hook that informs X that an update to a branch or tag is >> pushed. >> 2) This hook does not only ensure tests pass but also that the version >> numbers are correct. If you tag a project as 0.1.0 and the .app file has >> {vsn, 0.1.0} it fails >> 3) Assuming these criteria are met the agner repo for the project is >> either updated or created. Instead of @master it would use a piece of the >> git hash for the branch pointer (using a moving target -- a branch name -- >> as the version of an app doesn't make sense), for a tag it would use the >> tag name. >> >> Now, this is only part of the large solution people want, and I agree >> with them we should do. >> >> But, I think it is a start and something that can be done in small enough >> amount of effort to be done soon. >> >> Tristan >> >> On Wed, May 2, 2012 at 7:18 PM, Tim Watson wrote: >> >>> On 02/05/2012 21:13, Ciprian Dorin Craciun wrote: >>> >>>> On Wed, May 2, 2012 at 11:03 PMv, Tim Watson> >>>> wrote: >>> >>> Eric Merit and I have had some lengthy discussions about this on the >>>>> Erlware mailing list and have some ideas that I think are pretty solid. >>>>> >>>> I'm glad to hear this. (I'll give it a look.) >>>> >>> Cool thanks - please do leave any feedback you feel is relevant. >>> >>> The thing is though, you don't just need tools - you also need people >>>>> to package their stuff up using the tools. >>>>> >>>> I agree about this. In fact I think currently there are a lot of >>>> tools with diverging solutions. >>>> >>>> Also I don't think that one "blessed" tool would be the final >>>> answer. I would have taken a somehow different road, similar maybe to >>>> how Go is going (although they do have the "one" tool): i.e. strict >>>> conventions. >>>> >>> Yes I agree that '1 tool to rule them all' isn't going to work. Eric and >>> I have discussed building a unix-y pipeline of tools to deal with >>> local/remote package and repository management, dependency resolution, >>> installing and/or generating a code path (or some other structure) for >>> bootstrapping either ERL_LIBS or a call to code:path, etc. >>> >>> The assumption we have is that various parts can be dealt with be >>> different tools providing they respect the APIs, so you can build with >>> rebar or sinan or make or whatever. We are also planning on using reltool >>> (or some alternative/replica of sorts) to deal with the packaging bits, as >>> well as a set of tools for publication, code/package signing and the like. >>> >>> We would also like to support different release types (e.g., embedded >>> vs. non), distinguish between applications and library applications and so >>> on. >>> >>> >>>> What do I mean is this: >>>> a) it would be nice to have a standardized way to specify "extra" >>>> options to compile an Erlang module. (We have the module attributes >>>> that we could use.) (the same for C sources); >>>> b) we already have a standard project layout; (i.e. `./src`, >>>> `./include`, etc.) >>>> c) we already have a standard project dependencies (i.e. the `app` >>>> file); >>>> >>>> All we need to do is be consistent in this convention, and then >>>> all the various build and packaging systems could adapt. >>>> >>> Packaging wise, I don't see how any of this really helps so much, >>> although I completely agree that sticking to this consistent layout is a >>> good thing. >>> >>> Using the app file for dependencies is fine at runtime - I do this in >>> https://github.com/hyperthunk/**appstart- but if you're connecting to a (possibly remote) artefact repository and >>> fetching stuff, then you need to specify the publisher/signer (because 2 >>> OTP applications could be written with the same name, and of course with >>> FOSS project forking this happens all the time), the application name, and >>> the version. You can't whack all of that into your .app file without >>> breaking various things. >>> >>> >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Sat May 5 20:44:42 2012 From: bob@REDACTED (Bob Ippolito) Date: Sat, 5 May 2012 11:44:42 -0700 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> <4FA1CEC8.1060807@gmail.com> Message-ID: That's a nice idea, but it's not very practical to add a dependency on GHC to Erlang users, although if we're just talking server-side then that might be an ok solution. I think we just need to clearly identify what needs to be improved and just fix rebar, anger or sinan to do that. I don't care which one it is, as long as there's a de facto standard. Rebar feels pretty close to standard at this point, so if we can fix it then we'll get the most bang for our buck. On Saturday, May 5, 2012, Tristan Sloughter wrote: > Tuncer's idea of Hackage for Erlang is interesting. And if it were > possible to move something like Hackage to do for Erlang what it does for > Haskell, instead of writing from scratch, that could be even better. > > Tristan > > On Fri, May 4, 2012 at 2:32 AM, Tim Watson wrote: > > Is Columbo somehow different from what rebar already does in this space? > > > > On 4 May 2012, at 03:06, Tristan Sloughter > wrote: > > Ah yes, I briefly looked at colombo when you mentioned it on the erlware > list. I'll give that a deeper look to see how much of what I need is > covered by it and possibly simply start extending it. > > Tristan > > On Thu, May 3, 2012 at 4:52 PM, Torben Hoffmann wrote: > > That actually sounds like a good step forward - it almost, but only almost > makes me sad that I have created my own dependency management tool called > colombo (alpha vers. is on github/lehoff). > > I haven't spent much time on it, but it can fetch apps from git repos as > well as their dependencies, one can specify a tag or a branch to check > out, a simple build can be done and you can get a version tagged version > installed in the subdir lib of the dir where you you colombo (vers is taken > from the app file or a default is choosen). > > very much alpha quality, but it shows that small dedicated tools aren't > that hard to create for the Erlang eco system. > > I will continue improving colombo since it helps me solve my own problem, > but it would be nice to have "real" users as well that could help form the > functionality of the tool. Or tools since I might already have added too > much functionality to colombo. > > Cheers, > Torben > > Sent from my iPad > > On 03/05/2012, at 15.30, Tristan Sloughter > wrote: > > I feel like someone said Travis-CI wasn't the answer... But I am now > having the thought it is, at least to make something I'd like to see > without me having to do it. > > The question is would the community be interested in this if > I implement it. Here is what I'm talking about: > > 1) A github hook that informs X that an update to a branch or tag is > pushed. > 2) This hook does not only ensure tests pass but also that the version > numbers are correct. If you tag a project as 0.1.0 and the .app file has > {vsn, 0.1.0} it fails > 3) Assuming these criteria are met the agner repo for the project is > either updated or created. Instead of @master it would use a piece of the > git hash for the branch pointer (using a moving target -- a branch name -- > as the version of an app doesn't make sense), for a tag it would use the > tag name. > > Now, this is only part of the large solution people want, and I agree with > them we should do. > > But, I think it is a start and something that can be done in small enough > amount of effort to be done soon. > > Tristan > > On Wed, May 2, 2012 at 7:18 PM, Tim Watson wrote: > > On 02/05/2012 21:13, Ciprian Dorin Craciun wrote: > > On Wed, May 2, 2012 at 11:03 PMv, Tim Watson > wrote: > > Eric Merit and I have had some lengthy discussions about this on the > Erlware mailing list and have some ideas that I think are pretty solid. > > I'm glad to hear this. (I'll give it a look.) > > Cool than > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Sat May 5 21:07:22 2012 From: robert.virding@REDACTED (Robert Virding) Date: Sat, 05 May 2012 20:07:22 +0100 (BST) Subject: [erlang-questions] supervisor not restarting gen_server In-Reply-To: Message-ID: <17f371d9-faab-47d9-bfe3-dd45f09363d1@knuth> A quick look seems to indicate that both a server and a supervisor, cowboy_listener_sup, crashed. It was another supervisor, cowboy_sup, which reports the death of cowboy_listener_sup. From where is your gene_server started? Robert ----- Original Message ----- > I have a gen_server that seems to be crashing for some reason and the > supservisor is not restarting it. The child spec is one_for_one, > permanent and the error log is below. All i see is the crash report > but no attempt at a restart. Is there any way to figure out why the > supervisor would not restart it? > 2012-05-02 14:54:28.776 [info] > <0.889.0>@websocket_pubsubsrv:terminate:198 Shutting down pubsub > server... > 2012-05-02 14:54:28.776 [error] <0.882.0> Supervisor cowboy_sup had > child {cowboy_listener_sup,my_http_listener} started with > cowboy_listener_sup:start_link(100, cowboy_tcp_transport, > [{port,8000}], cowboy_http_protocol, > [{dispatch,[{'_',[{[<<"multiplex">>,'...'],sockjs_cowboy_handler,{service,"/multiplex",sockjs_multiplex,...}},...]}]}]) > at <0.909.0> exit with reason killed in context child_terminated > 2012-05-02 14:54:28.785 [error] <0.911.0> gen_server <0.911.0> > terminated with reason: killed > 2012-05-02 14:54:28.788 [error] <0.911.0> CRASH REPORT Process > <0.911.0> with 1 neighbours crashed with reason: > {killed,[{gen_server,terminate,6},{proc_lib,init_p_do_apply,3}]} > Thanks > -AD > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Sat May 5 22:58:27 2012 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Sat, 5 May 2012 15:58:27 -0500 Subject: [erlang-questions] enif_make_reverse_list() vs lists:reverse() In-Reply-To: <2413D08D-72EC-40DF-9AAD-366E370611B4@gmail.com> References: <2413D08D-72EC-40DF-9AAD-366E370611B4@gmail.com> Message-ID: Good reason to keep the reverse in Erlang then. Thanks, Dan. On Fri, May 4, 2012 at 1:14 PM, Lukas Larsson wrote: > Hi, > > The nif version is blocking while the bif isn't. So for long lists the nif will block the scheduler while the bif will not. If the blocking is acceptable or not is up to you. If unsure using lists:reverse is the safe choice. > > Sent from me > > On 4 maj 2012, at 19:26, Daniel Goertzen wrote: > >> The documentation for enif_make_reverse_list() says ... >> >> "This function should only be used on short lists as a copy will be >> created of the list which will not be released until after the nif >> returns." >> >> I have a long-ish list. ?Should I be opting to use lists:reverse() >> instead of enif_make_reverse_list()? ?I assumed they were the same and >> for my situation I can use either, but the documentation suggests the >> nif version may be worse. >> >> Dan. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From per.melin@REDACTED Sun May 6 00:17:20 2012 From: per.melin@REDACTED (Per Melin) Date: Sun, 6 May 2012 00:17:20 +0200 Subject: [erlang-questions] supervisor not restarting gen_server In-Reply-To: <17f371d9-faab-47d9-bfe3-dd45f09363d1@knuth> References: <17f371d9-faab-47d9-bfe3-dd45f09363d1@knuth> Message-ID: On May 5, 2012, at 21:07 , Robert Virding wrote: > A quick look seems to indicate that both a server and a supervisor, cowboy_listener_sup, crashed. It was another supervisor, cowboy_sup, which reports the death of cowboy_listener_sup. From where is your gene_server started? Since cowboy_sup received an exit signal of 'killed' from cowboy_listener_sup, doesn't that imply that it was (explicitly) killed with exit(Pid, kill)? >> I have a gen_server that seems to be crashing for some reason and the supservisor is not restarting it. The child spec is one_for_one, permanent and the error log is below. All i see is the crash report but no attempt at a restart. Is there any way to figure out why the supervisor would not restart it? >> >> 2012-05-02 14:54:28.776 [info] <0.889.0>@websocket_pubsubsrv:terminate:198 Shutting down pubsub server... >> 2012-05-02 14:54:28.776 [error] <0.882.0> Supervisor cowboy_sup had child {cowboy_listener_sup,my_http_listener} started with cowboy_listener_sup:start_link(100, cowboy_tcp_transport, [{port,8000}], cowboy_http_protocol, [{dispatch,[{'_',[{[<<"multiplex">>,'...'],sockjs_cowboy_handler,{service,"/multiplex",sockjs_multiplex,...}},...]}]}]) at <0.909.0> exit with reason killed in context child_terminated >> 2012-05-02 14:54:28.785 [error] <0.911.0> gen_server <0.911.0> terminated with reason: killed >> 2012-05-02 14:54:28.788 [error] <0.911.0> CRASH REPORT Process <0.911.0> with 1 neighbours crashed with reason: {killed,[{gen_server,terminate,6},{proc_lib,init_p_do_apply,3}]} >> >> >> Thanks >> -AD >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From robert.virding@REDACTED Sun May 6 00:40:39 2012 From: robert.virding@REDACTED (Robert Virding) Date: Sat, 05 May 2012 23:40:39 +0100 (BST) Subject: [erlang-questions] supervisor not restarting gen_server In-Reply-To: Message-ID: ----- Original Message ----- > On May 5, 2012, at 21:07 , Robert Virding wrote: > > > A quick look seems to indicate that both a server and a supervisor, > > cowboy_listener_sup, crashed. It was another supervisor, > > cowboy_sup, which reports the death of cowboy_listener_sup. From > > where is your gene_server started? > > Since cowboy_sup received an exit signal of 'killed' from > cowboy_listener_sup, doesn't that imply that it was (explicitly) > killed with exit(Pid, kill)? That is most definitely does. And if the supervisor was killed then that would also have killed the gen_server which would not be restarted as there was no supervisor. The question is then why that supervisor was not restarted by cowboy_sup? If it should have been, Robert > > >> I have a gen_server that seems to be crashing for some reason and > >> the supservisor is not restarting it. The child spec is > >> one_for_one, permanent and the error log is below. All i see is > >> the crash report but no attempt at a restart. Is there any way > >> to figure out why the supervisor would not restart it? > >> > >> 2012-05-02 14:54:28.776 [info] > >> <0.889.0>@websocket_pubsubsrv:terminate:198 Shutting down pubsub > >> server... > >> 2012-05-02 14:54:28.776 [error] <0.882.0> Supervisor cowboy_sup > >> had child {cowboy_listener_sup,my_http_listener} started with > >> cowboy_listener_sup:start_link(100, cowboy_tcp_transport, > >> [{port,8000}], cowboy_http_protocol, > >> [{dispatch,[{'_',[{[<<"multiplex">>,'...'],sockjs_cowboy_handler,{service,"/multiplex",sockjs_multiplex,...}},...]}]}]) > >> at <0.909.0> exit with reason killed in context child_terminated > >> 2012-05-02 14:54:28.785 [error] <0.911.0> gen_server <0.911.0> > >> terminated with reason: killed > >> 2012-05-02 14:54:28.788 [error] <0.911.0> CRASH REPORT Process > >> <0.911.0> with 1 neighbours crashed with reason: > >> {killed,[{gen_server,terminate,6},{proc_lib,init_p_do_apply,3}]} > >> > >> > >> Thanks > >> -AD > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > From per.melin@REDACTED Sun May 6 01:26:43 2012 From: per.melin@REDACTED (Per Melin) Date: Sun, 6 May 2012 01:26:43 +0200 Subject: [erlang-questions] supervisor not restarting gen_server In-Reply-To: References: Message-ID: On May 6, 2012, at 00:40 , Robert Virding wrote: >> Since cowboy_sup received an exit signal of 'killed' from >> cowboy_listener_sup, doesn't that imply that it was (explicitly) >> killed with exit(Pid, kill)? > > That is most definitely does. And if the supervisor was killed then that would also have killed the gen_server which would not be restarted as there was no supervisor. The question is then why that supervisor was not restarted by cowboy_sup? I asked myself that too, but are we sure it wasn't? If AD's gen_server was dynamically started (supervisor:start_child/2) as a descendant to cowboy_listener_sup it could not be started again since that information would now be lost. So it's back to your original question; where is that process started? >>>> >>>> I have a gen_server that seems to be crashing for some reason and >>>> the supservisor is not restarting it. The child spec is >>>> one_for_one, permanent and the error log is below. All i see is >>>> the crash report but no attempt at a restart. Is there any way >>>> to figure out why the supervisor would not restart it? >>>> >>>> 2012-05-02 14:54:28.776 [info] >>>> <0.889.0>@websocket_pubsubsrv:terminate:198 Shutting down pubsub >>>> server... >>>> 2012-05-02 14:54:28.776 [error] <0.882.0> Supervisor cowboy_sup >>>> had child {cowboy_listener_sup,my_http_listener} started with >>>> cowboy_listener_sup:start_link(100, cowboy_tcp_transport, >>>> [{port,8000}], cowboy_http_protocol, >>>> [{dispatch,[{'_',[{[<<"multiplex">>,'...'],sockjs_cowboy_handler,{service,"/multiplex",sockjs_multiplex,...}},...]}]}]) >>>> at <0.909.0> exit with reason killed in context child_terminated >>>> 2012-05-02 14:54:28.785 [error] <0.911.0> gen_server <0.911.0> >>>> terminated with reason: killed >>>> 2012-05-02 14:54:28.788 [error] <0.911.0> CRASH REPORT Process >>>> <0.911.0> with 1 neighbours crashed with reason: >>>> {killed,[{gen_server,terminate,6},{proc_lib,init_p_do_apply,3}]} >>>> >>>> >>>> Thanks >>>> -AD From boxoxod@REDACTED Sun May 6 04:36:20 2012 From: boxoxod@REDACTED (allanruin) Date: Sat, 5 May 2012 19:36:20 -0700 (PDT) Subject: [erlang-questions] Where is the erlang:decode_packet implemented? Message-ID: <1336271780387-4612240.post@n4.nabble.com> I heard that decode_packet was implemented in "erts/emulator/beam/erl_bif_port.c and erts/emulator/beam/packet_parser.c " however, in Erlang R14B04, I can't found these files.Besides, I see a page explaining how to extract POST DATA use decode_packet, but it use 'list' in gen_tcp:listen ,if I use 'binary', will it be more efficient? and if I use 'binary' mode, can I still parse the HTTP REQUEST header using decode_packet? I tried to figure this out, but I think the documentation is kind of too complex to understand. Thank you all in advance. -- View this message in context: http://erlang.2086793.n4.nabble.com/Where-is-the-erlang-decode-packet-implemented-tp4612240.html Sent from the Erlang Questions mailing list archive at Nabble.com. From straightflush@REDACTED Sun May 6 04:49:53 2012 From: straightflush@REDACTED (AD) Date: Sat, 5 May 2012 22:49:53 -0400 Subject: [erlang-questions] supervisor not restarting gen_server In-Reply-To: References: Message-ID: i had a supervisor behaviour starting the gen_server and the cowboy_sup as 2 child specs. Each was a standard one_for_one permanent although I now realize I had the cowboy PID set to worker instead of supervisor which probably caused some issues. I also realized I probably dont need to supervise cowboy under my own supervisor as it already has its own supervision. -AD On Sat, May 5, 2012 at 7:26 PM, Per Melin wrote: > On May 6, 2012, at 00:40 , Robert Virding wrote: > > >> Since cowboy_sup received an exit signal of 'killed' from > >> cowboy_listener_sup, doesn't that imply that it was (explicitly) > >> killed with exit(Pid, kill)? > > > > That is most definitely does. And if the supervisor was killed then that > would also have killed the gen_server which would not be restarted as there > was no supervisor. The question is then why that supervisor was not > restarted by cowboy_sup? > > I asked myself that too, but are we sure it wasn't? If AD's gen_server was > dynamically started (supervisor:start_child/2) as a descendant to > cowboy_listener_sup it could not be started again since that information > would now be lost. So it's back to your original question; where is that > process started? > > > > >>>> > >>>> I have a gen_server that seems to be crashing for some reason and > >>>> the supservisor is not restarting it. The child spec is > >>>> one_for_one, permanent and the error log is below. All i see is > >>>> the crash report but no attempt at a restart. Is there any way > >>>> to figure out why the supervisor would not restart it? > >>>> > >>>> 2012-05-02 14:54:28.776 [info] > >>>> <0.889.0>@websocket_pubsubsrv:terminate:198 Shutting down pubsub > >>>> server... > >>>> 2012-05-02 14:54:28.776 [error] <0.882.0> Supervisor cowboy_sup > >>>> had child {cowboy_listener_sup,my_http_listener} started with > >>>> cowboy_listener_sup:start_link(100, cowboy_tcp_transport, > >>>> [{port,8000}], cowboy_http_protocol, > >>>> > [{dispatch,[{'_',[{[<<"multiplex">>,'...'],sockjs_cowboy_handler,{service,"/multiplex",sockjs_multiplex,...}},...]}]}]) > >>>> at <0.909.0> exit with reason killed in context child_terminated > >>>> 2012-05-02 14:54:28.785 [error] <0.911.0> gen_server <0.911.0> > >>>> terminated with reason: killed > >>>> 2012-05-02 14:54:28.788 [error] <0.911.0> CRASH REPORT Process > >>>> <0.911.0> with 1 neighbours crashed with reason: > >>>> {killed,[{gen_server,terminate,6},{proc_lib,init_p_do_apply,3}]} > >>>> > >>>> > >>>> Thanks > >>>> -AD > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Sun May 6 12:14:03 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 6 May 2012 12:14:03 +0200 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> <4FA1CEC8.1060807@gmail.com> Message-ID: On Sat, May 5, 2012 at 8:44 PM, Bob Ippolito wrote: > That's a nice idea, but it's not very practical to add a dependency > on GHC to Erlang users, although if we're just talking server-side > then that might be an ok solution. I've mentioned Hackage as a source of inspiration and didn't mean to say we should depend on GHC on the client-side. If Hackage could be repurposed to host Erlang bits, that would imply a GHC dep on the server-side. > I think we just need to clearly identify what needs to be improved > and just fix rebar, anger or sinan to do that. I don't care which > one it is, as long as there's a de facto standard. Rebar feels > pretty close to standard at this point, so if we can fix it then > we'll get the most bang for our buck. > > On Saturday, May 5, 2012, Tristan Sloughter wrote: > > > > Tuncer's idea of Hackage for Erlang is interesting. And if it were > > possible to move something like Hackage to do for Erlang what it > > does for Haskell, instead of writing from scratch, that could be > > even better. From roberto@REDACTED Sun May 6 22:59:54 2012 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 6 May 2012 13:59:54 -0700 Subject: [erlang-questions] rebar compile In-Reply-To: References: Message-ID: Oops, sorry I missed this response. Thank you Ulf. Too bad, if anyone is already working on this let me know. r. On Mon, Apr 23, 2012 at 9:12 AM, Ulf Wiger wrote: > > There doesn't seem to be. > > It would not be terribly hard to add, I think. E.g. an option to make > rebar_erlc_compiler:doterl_compile/3 and friends return 'skipped' on > compilation error rather than ?FAIL. > > BR, > Ulf W > > On 23 Apr 2012, at 18:02, Roberto Ostinelli wrote: > > Dear all, > > is there a way to make rebar compile skip the files that do not compile > properly and proceed to the next ones, instead of breaking the compile > process asa it encounters one error? > > Thank you, > > r. _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From huangjl@REDACTED Mon May 7 05:11:18 2012 From: huangjl@REDACTED (huangjl) Date: Mon, 7 May 2012 11:11:18 +0800 Subject: [erlang-questions] Connect to Node with multiple IPs Message-ID: <002201cd2bff$12bacfa0$38306ee0$@com> Hi, All I have a question about erlang node connection. There are two machines: MachineA and MachineB, MachineB has two IPs: IP1 and IP2. MachineA want to connect to MachineB using IP, just like net_kernel:connect_node('test@REDACTED') or net_kernel:connect_node('test@REDACTED'). Then , how to start only one test node at MachineB which can receive all messages from test@REDACTED and test@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From witeman.g@REDACTED Mon May 7 08:08:28 2012 From: witeman.g@REDACTED (Zheng Zhibin) Date: Mon, 7 May 2012 14:08:28 +0800 Subject: [erlang-questions] Connect to Node with multiple IPs In-Reply-To: <002201cd2bff$12bacfa0$38306ee0$@com> References: <002201cd2bff$12bacfa0$38306ee0$@com> Message-ID: <5A8DF162-B90D-455E-B5D9-8433C8BE15DE@gmail.com> use domain instead of ip, then config your hosts table ???? iPhone ? 2012-5-7???11:11?"huangjl" ??? > Hi, All > I have a question about erlang node connection. There are two machines: MachineA and MachineB, MachineB has two IPs: IP1 and IP2. MachineA want to connect to MachineB using IP, just like net_kernel:connect_node(?test@REDACTED?) or net_kernel:connect_node(?test@REDACTED?). Then , how to start only one test node at MachineB which can receive all messages from test@REDACTED and test@REDACTED > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcoveney@REDACTED Mon May 7 08:39:35 2012 From: jcoveney@REDACTED (Jonathan Coveney) Date: Mon, 7 May 2012 15:39:35 +0900 Subject: [erlang-questions] Is there a good source for documentation on BEAM? Message-ID: This question seems to come up now and again, and it's surprising to me that a crucial part of the documentation isn't better documented. Is there a reason that it is the case? Is the reason that there is no VM spec to give the devs the flexibility to change the intermediate layer without having to worry about backwards compatibility to the degree that Java does? Thus far I've found a description of the opcodes: http://azunyanmoe.wordpress.com/2011/03/30/erlang-vm-opcodes/ and this resource on the file format: http://www.erlang.se/~bjorn/beam_file_format.html But there doesn't seem to be a lot of high level talk about what the opcodes do (a la the JVM specification, for example). I know it's not impossible, and could always ask the guys at Erjang how they went about it, but thought I'd ask here. Please forgive a newbie question, and thanks in advance Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: From huangjl@REDACTED Mon May 7 08:53:36 2012 From: huangjl@REDACTED (huangjl) Date: Mon, 7 May 2012 14:53:36 +0800 Subject: [erlang-questions] =?utf-8?b?562U5aSNOiAgQ29ubmVjdCB0byBOb2RlIHdp?= =?utf-8?q?th_multiple_IPs?= In-Reply-To: <5A8DF162-B90D-455E-B5D9-8433C8BE15DE@gmail.com> References: <002201cd2bff$12bacfa0$38306ee0$@com> <5A8DF162-B90D-455E-B5D9-8433C8BE15DE@gmail.com> Message-ID: <002f01cd2c1e$20d8c2d0$628a4870$@com> But without DNS, it seems I have to config two domains for two IPs? My purpose is detect which IP is suitable for MachineA to connect to MachineB, but I don?t want to start two nodes for MachineB, is there any way to do this? ???: Zheng Zhibin [mailto:witeman.g@REDACTED] ????: 2012?5?7? 14:08 ???: huangjl ??: ??: Re: [erlang-questions] Connect to Node with multiple IPs use domain instead of ip, then config your hosts table ???? iPhone ? 2012-5-7???11:11?"huangjl" ??? Hi, All I have a question about erlang node connection. There are two machines: MachineA and MachineB, MachineB has two IPs: IP1 and IP2. MachineA want to connect to MachineB using IP, just like net_kernel:connect_node(?test@REDACTED?) or net_kernel:connect_node(?test@REDACTED?). Then , how to start only one test node at MachineB which can receive all messages from test@REDACTED and test@REDACTED _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From witeman.g@REDACTED Mon May 7 09:01:48 2012 From: witeman.g@REDACTED (Zheng Zhibin) Date: Mon, 7 May 2012 15:01:48 +0800 Subject: [erlang-questions] =?gb2312?b?tPC4tDogIENvbm5lY3QgdG8gTm9kZSB3?= =?gb2312?b?aXRoIG11bHRpcGxlIElQcw==?= In-Reply-To: <002f01cd2c1e$20d8c2d0$628a4870$@com> References: <002201cd2bff$12bacfa0$38306ee0$@com> <5A8DF162-B90D-455E-B5D9-8433C8BE15DE@gmail.com> <002f01cd2c1e$20d8c2d0$628a4870$@com> Message-ID: one domain + reverse proxy, such as nginx? ???? iPhone ? 2012-5-7???2:53?"huangjl" ??? > But without DNS, it seems I have to config two domains for two IPs? > My purpose is detect which IP is suitable for MachineA to connect to MachineB, but I don?t want to start two nodes for MachineB, is there any way to do this? > > > ???: Zheng Zhibin [mailto:witeman.g@REDACTED] > ????: 2012?5?7? 14:08 > ???: huangjl > ??: > ??: Re: [erlang-questions] Connect to Node with multiple IPs > > use domain instead of ip, then config your hosts table > > ???? iPhone > > ? 2012-5-7???11:11?"huangjl" ??? > > Hi, All > I have a question about erlang node connection. There are two machines: MachineA and MachineB, MachineB has two IPs: IP1 and IP2. MachineA want to connect to MachineB using IP, just like net_kernel:connect_node(?test@REDACTED?) or net_kernel:connect_node(?test@REDACTED?). Then , how to start only one test node at MachineB which can receive all messages from test@REDACTED and test@REDACTED > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Mon May 7 09:37:48 2012 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 7 May 2012 09:37:48 +0200 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: Message-ID: Hello, I've been watching this thread - but not contributing ... I have a few questions/thoughts/comments 1) NodeJSs NPM looks pretty good to me. I've been toying with the idea of making a clone the advantages of cloning seem to be: - it is very well documented - it has a decent set of commands - it produces a reasonable (target) directory structure 2) What's wrong today? It seems to me that the single biggest problem is the lack of a decent manifest file. NodeJS uses https://github.com/isaacs/npm/blob/master/package.json For comparison Google chrome apps use http://code.google.com/chrome/extensions/manifest.html We (Erlang) use http://www.erlang.org/doc/design_principles/applications.html#id73480 In a sense once you *defined* the manifest file and the file system layout, all the rest follows naturally. (Once you've agreed on the data formats - which program produces the files is irrelevant) Now the .app files date from a time *before* Erlang was open source, so there are no things for licenses, digital signatures and so on. So I think a first step is hammer out the content of the manifest. The manifest could be simple or very complicated - so there are a lot of design questions here. 3) Tricky problems How do we avoid version nightmares? i.e. if A depends upon X.vsn1 and breaks with X.vsn2 and B depends upon X.vsn2 and breaks with X.vsn1 how can we run A and B in the same namespace? Three answers: a) You can't b) Declare A to be broken and fix it (assuming X.vsn2 is the latest and best version of X) c) do some magic name munging (refactoring) that allows A and B to run in the same namspace with X.vsn1 and X.vsn2 d) run in separate nodes with message passing e) add some sort of namespaces to erlang Comments: c) is pie in the sky - I don't see this happening realistically d) yes - but the apps need changing e) is research a) is bad b) is the best option - fix A - now why can't the refactoring guys do this for us :-) In practice I'd mail the authors of A and hope they could help 4) Signing No we want digital signatures - bla bla bla .... signed code 5) Should packages be unpacked on the target machines (I think not - this is abstraction breaking) The way rebar is packaged is nice. 6) The way forward - Define a new manifest (say erlang .app files with inspiration from NPM) - Define the layout of the local repository (ie where things land locally after a package has been fetched) - Make a clone of NPM (or similar) - Make a converter that sucks code from rebar github repository and repackages, throwing away the history inn (.git) 7) Binary or source packages? - Ummm - tricky 8) Community buy-in We need one recognised format that everybody uses. I'd actually proposed a NPM clone as the subject of a hackathon at the next Erlang User conference (in may) - now (of course) this is not a particularly difficult program to write (famous last words) - the tricky bit is agreeing on the manifest and file layout and semantics of updates etc. If any of you guys are coming to the Erlang users conference in Stockholm I'd be happy to discuss this - or even participate in a hackathon session Cheers /Joe On Wed, May 2, 2012 at 8:39 PM, Ciprian Dorin Craciun wrote: > ? ?(As no-one replied in 4 days, I'll add my opinion...) > > On Sat, Apr 28, 2012 at 4:36 PM, Tristan Sloughter > wrote: >> I started with this problem as something to simply discuss with the >> maintainers of certain build and package management projects -- sinan and >> agner specifically. But it seems to be more broad and cover how all of us >> who keep apps on github handle versioning and dependencies. >> >> The basic issue is app versions within .app files on branches in github, the >> resultant directory from a agner install and the?discrepancies?this causes. >> >> [...] >> >> Am I the only one going crazy with this? > > ? ?Nop. I've been faced with this problem myself a couple of times. > (I've chosen to ignore it.) > > >> Does anyone have >> suggestions/examples of what they do for production projects on teams to >> ease these annoyances? > > ? ?For example for my project I have one big Git repository called > `myproject-repositories` (replace `myproject` with what you need) with > submodules pointing to the dependencies. > > ? ?Then inside my own project repositories I have a symlink > `.repositories` poiting to the "current" snapshot of dependencies. > > ? ?I also don't use any of rebar, agner, etc., mainly because I've > hacked something on-top of the ninja build system. > > >> I mostly just create my own packages and repos of dependencies, or package >> third party deps with the project. > > ? ?Yup. +1 :) > > > ? ?P.S.: I hate Java from the bottom of my heart, but boy-o-boy is > the Java tooling way better than that of Erlang... I mean what Erlang > is missing is something similar to NodeJS's NPM, or Java's Maven > (minus the XML baggage and all useless complexities), or Python > setup-tools (with virtual environment), etc... > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vladdu55@REDACTED Mon May 7 10:29:10 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 7 May 2012 10:29:10 +0200 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: Message-ID: Hi, On Mon, May 7, 2012 at 9:37 AM, Joe Armstrong wrote: > ? ? ? ?How do we avoid version nightmares? > > ? ? ? ?i.e. if A depends upon X.vsn1 and breaks with X.vsn2 > ? ? ? ? ? ? and B depends upon X.vsn2 and breaks with X.vsn1 > ? ? ? ?how can we run A and B in the same namespace? (...snip...) > ? ? ? ? b) Declare A to be broken and fix it (assuming X.vsn2 is the > latest and best version of > ? ? ? ? ? ? ?X) This might be an obvious thing to point out (or maybe not), but in real life (as an A developer) one might need to deliver both A.v4 (using X.v1) and A.v5 (using X.v2) because maybe some user/customer can't upgrade to the latest and greatest right away or maybe there is a C app that doesn't work with X.v1 yet. This might be a bit heavy on an open-source project... A remark about the manifest file: for a while ago I proposed some extensions to the .app file: http://erlang.org/pipermail/erlang-questions/2010-September/053354.html. There was no reaction at all at the time, maybe it's not of general interest, but if there will be a process of standardisation of manifest data, I'd like to suggest to have a format open to extensions that will be ignored by tools that don't understand them. regards, Vlad From erlang@REDACTED Mon May 7 10:47:26 2012 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 7 May 2012 10:47:26 +0200 Subject: [erlang-questions] Fwd: Is there a good source for documentation on BEAM? In-Reply-To: References: Message-ID: ---------- Forwarded message ---------- From: Joe Armstrong Date: Mon, May 7, 2012 at 10:46 AM Subject: Re: [erlang-questions] Is there a good source for documentation on BEAM? To: Jonathan Coveney Hi, I did start writing a description but it's not very complete. This is on my list of things-to-do-one-day-when-you-get-time ? See http://dl.dropbox.com/u/4764922/beam.pdf If there is any interest I could up the priority :-) /Joe On Mon, May 7, 2012 at 8:39 AM, Jonathan Coveney wrote: > This question seems to come up now and again, and it's surprising to me that > a crucial part of the documentation isn't better documented. Is there a > reason that it is the case? Is the reason that there is no VM spec to give > the devs the flexibility to change the intermediate layer without having to > worry about backwards compatibility to the degree that Java does? > > Thus far I've found a description of the opcodes: > http://azunyanmoe.wordpress.com/2011/03/30/erlang-vm-opcodes/ > > and this resource on the file format: > http://www.erlang.se/~bjorn/beam_file_format.html > > But there doesn't seem to be a lot of high level talk about what the opcodes > do (a la the JVM specification, for example). I know it's not impossible, > and could always ask the guys at Erjang how they went about it, but thought > I'd ask here. > > Please forgive a newbie question, and thanks in advance > Jon > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From thomasl_erlang@REDACTED Mon May 7 10:46:59 2012 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 7 May 2012 01:46:59 -0700 (PDT) Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: Message-ID: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> >________________________________ > From: Jonathan Coveney >To: erlang-questions@REDACTED >Sent: Monday, May 7, 2012 8:39 AM >Subject: [erlang-questions] Is there a good source for documentation on BEAM? > > >This question seems to come up now and again, and it's surprising to me that a crucial part of the documentation isn't better documented. Is there a reason that it is the case? Is the reason that there is no VM spec to give the devs the flexibility to change the intermediate layer without having to worry about backwards compatibility to the degree that Java does? Actually, I don't think such docs are all _that_ crucial -- who really needs to know, except a small number of VM implementors? (And they should read the source to get at all the goodies.) But perhaps someone on the list might be moved to do a tutorial presentation on an Erlang Factory or something? (By the way, I too assume not doing it is to avoid getting bogged down into minutiae.) If you want to learn more about some of the intellectual roots, try these: http://wambook.sourceforge.net/ http://dl.acm.org/citation.cfm?id=188051 Best regards, Thomas From carlsson.richard@REDACTED Mon May 7 11:17:20 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Mon, 07 May 2012 11:17:20 +0200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: Message-ID: <4FA79320.4070305@gmail.com> On 05/07/2012 08:39 AM, Jonathan Coveney wrote: > This question seems to come up now and again, and it's surprising to me > that a crucial part of the documentation isn't better documented. Is > there a reason that it is the case? Is the reason that there is no VM > spec to give the devs the flexibility to change the intermediate layer > without having to worry about backwards compatibility to the degree that > Java does? Yes, that's probably the reason. The BEAM is not the first VM for Erlang (JAM was used until the late 90s), and might not be the last. In the case of Java, the JVM was central for defining the language semantics and ensure portability across platforms and VM implementations. The Erlang language, on the other hand, is mostly functional and its semantics is better specified in terms of the source level code. The implementation on top of a VM is a detail. Still, the Beam is an interesting VM, and the implementation has been thoroughly battle tested, so using it as a target for other languages is not a bad idea. It deserves to be better documented. /Richard From michael.eugene.turner@REDACTED Mon May 7 11:27:33 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Mon, 7 May 2012 18:27:33 +0900 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> Message-ID: "Actually, I don't think such docs are all _that_ crucial -- who really needs to know, except a small number of VM implementors?" Aren't Erlang's chances of greater mindshare improved by making it easier to become a VM implementor? I doubt very much that Java would be where it is today had it not been for clear VM specification. That's not to say that Erlang should follow in all of Java's footsteps, even if it could. But I have to say I was a boggled to learn that you can't find out what the VM opcodes mean without reading the source (and maybe not even then, if the source contains bugs vis-a-vis some idealized machine model.) -michael turner On Mon, May 7, 2012 at 5:46 PM, Thomas Lindgren wrote: > > > >>________________________________ >> From: Jonathan Coveney >>To: erlang-questions@REDACTED >>Sent: Monday, May 7, 2012 8:39 AM >>Subject: [erlang-questions] Is there a good source for documentation on BEAM? >> >> >>This question seems to come up now and again, and it's surprising to me that a crucial part of the documentation isn't better documented. Is there a reason that it is the case? Is the reason that there is no VM spec to give the devs the flexibility to change the intermediate layer without having to worry about backwards compatibility to the degree that Java does? > > > Actually, I don't think such docs are all _that_ crucial -- who really needs to know, except a small number of VM implementors? (And they should read the source to get at all the goodies.) But perhaps someone on the list might be moved to do a tutorial presentation on an Erlang Factory or something? > > (By the way, I too assume not doing it is to avoid getting bogged down into minutiae.) > > If you want to learn more about some of the intellectual roots, try these: > http://wambook.sourceforge.net/ > > http://dl.acm.org/citation.cfm?id=188051 > > > Best regards, > Thomas > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From gleber.p@REDACTED Mon May 7 11:59:28 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Mon, 7 May 2012 11:59:28 +0200 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: Message-ID: On Mon, May 7, 2012 at 10:29 AM, Vlad Dumitrescu wrote: > Hi, > > On Mon, May 7, 2012 at 9:37 AM, Joe Armstrong wrote: >> ? ? ? ?How do we avoid version nightmares? >> >> ? ? ? ?i.e. if A depends upon X.vsn1 and breaks with X.vsn2 >> ? ? ? ? ? ? and B depends upon X.vsn2 and breaks with X.vsn1 >> ? ? ? ?how can we run A and B in the same namespace? > (...snip...) >> ? ? ? ? b) Declare A to be broken and fix it (assuming X.vsn2 is the >> latest and best version of >> ? ? ? ? ? ? ?X) > > This might be an obvious thing to point out (or maybe not), but in > real life (as an A developer) one might need to deliver both A.v4 > (using X.v1) and A.v5 (using X.v2) because maybe some user/customer > can't upgrade to the latest and greatest right away or maybe there is > a C app that doesn't work with X.v1 yet. This might be a bit heavy on > an open-source project... > > A remark about the manifest file: for a while ago I proposed some > extensions to the .app file: > http://erlang.org/pipermail/erlang-questions/2010-September/053354.html. > There was no reaction at all at the time, maybe it's not of general > interest, but if there will be a process of standardisation of > manifest data, I'd like to suggest to have a format open to extensions > that will be ignored by tools that don't understand them. The good question is - do we want to put all meta data into one file? Some things are useful only at compile-time (e.g. location of dependencies) and some things are useful only at run-time (e.g. application env). I'm a bit vary of putting too many things into one file. Although if most of those fields are optional and reasonable defaults are assumed, this is not a problem. From vladdu55@REDACTED Mon May 7 12:40:24 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 7 May 2012 12:40:24 +0200 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: Message-ID: On Mon, May 7, 2012 at 11:59 AM, Gleb Peregud wrote: > On Mon, May 7, 2012 at 10:29 AM, Vlad Dumitrescu wrote: >> A remark about the manifest file: for a while ago I proposed some >> extensions to the .app file: >> http://erlang.org/pipermail/erlang-questions/2010-September/053354.html. > > The good question is - do we want to put all meta data into one file? > Some things are useful only at compile-time (e.g. location of > dependencies) and some things are useful only at run-time (e.g. > application env). I'm a bit vary of putting too many things into one > file. Although if most of those fields are optional and reasonable > defaults are assumed, this is not a problem. And a good question it is. I have no problem either way, except that some information is be relevant in both situations and it shouldn't be duplicated. regards, Vlad From erlang@REDACTED Mon May 7 13:47:33 2012 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 7 May 2012 13:47:33 +0200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> Message-ID: I think it works like this: 1) first you don't understand how the X works (X=Beam, JVM, X11, ... you name it) 2) You struggle - and think - google and have a hot bath 3) Eureka - bath flows over 4) Now you can understand it - and you can also remember why you could not understand it 5) Now it's easy you understand it 6) You see no reason to document it since it's obvious Round about 4) there is a small window of opportunity to explain to other people how it works. Once you get to 6) it's very difficult to remember what it felt like at point 2) and consequently difficult to write decent documentation. /Joe On Mon, May 7, 2012 at 11:27 AM, Michael Turner wrote: > "Actually, I don't think such docs are all _that_ crucial -- who > really needs to know, except a small number of VM implementors?" > > Aren't Erlang's chances of greater mindshare improved by making it > easier to become a VM implementor? I doubt very much that Java would > be where it is today had it not been for clear VM specification. > That's not to say that Erlang should follow in all of Java's > footsteps, even if it could. But I have to say I was a boggled to > learn that you can't find out what the VM opcodes mean without reading > the source (and maybe not even then, if the source contains bugs > vis-a-vis some idealized machine model.) > > -michael turner > > > > On Mon, May 7, 2012 at 5:46 PM, Thomas Lindgren > wrote: >> >> >> >>>________________________________ >>> From: Jonathan Coveney >>>To: erlang-questions@REDACTED >>>Sent: Monday, May 7, 2012 8:39 AM >>>Subject: [erlang-questions] Is there a good source for documentation on BEAM? >>> >>> >>>This question seems to come up now and again, and it's surprising to me that a crucial part of the documentation isn't better documented. Is there a reason that it is the case? Is the reason that there is no VM spec to give the devs the flexibility to change the intermediate layer without having to worry about backwards compatibility to the degree that Java does? >> >> >> Actually, I don't think such docs are all _that_ crucial -- who really needs to know, except a small number of VM implementors? (And they should read the source to get at all the goodies.) But perhaps someone on the list might be moved to do a tutorial presentation on an Erlang Factory or something? >> >> (By the way, I too assume not doing it is to avoid getting bogged down into minutiae.) >> >> If you want to learn more about some of the intellectual roots, try these: >> http://wambook.sourceforge.net/ >> >> http://dl.acm.org/citation.cfm?id=188051 >> >> >> Best regards, >> Thomas >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From masklinn@REDACTED Mon May 7 13:58:32 2012 From: masklinn@REDACTED (Masklinn) Date: Mon, 7 May 2012 13:58:32 +0200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> Message-ID: <4945B8FA-2780-498F-A510-3D1905AA6CB4@masklinn.net> On 7 mai 2012, at 13:47, Joe Armstrong wrote: > I think it works like this: > > 1) first you don't understand how the X works (X=Beam, JVM, X11, > ... you name it) > 2) You struggle - and think - google and have a hot bath > 3) Eureka - bath flows over > 4) Now you can understand it - and you can also remember why you > could not understand it > 5) Now it's easy you understand it > 6) You see no reason to document it since it's obvious Pretty much the same process which prevents the completion and uptake of alternative git porcelains: by the time the author has a good enough knowledge to finish his implementation he has good enough knowledge to use the default cli without hindrance. From sverker@REDACTED Mon May 7 14:55:39 2012 From: sverker@REDACTED (Sverker Eriksson) Date: Mon, 7 May 2012 14:55:39 +0200 Subject: [erlang-questions] Questions about ETS locking In-Reply-To: <2d34a408.e600.1371c4aa309.Coremail.cloudzen@163.com> References: <2d34a408.e600.1371c4aa309.Coremail.cloudzen@163.com> Message-ID: <4FA7C64B.5090002@erix.ericsson.se> skyman wrote: > Hi all, > > > I know when access an ETS table, there are wo locks that need to be taken. > 1) A lock to access the meta table, to convert the numeric table identifier > to a pointer to the actual table. > > 2) The lock for the table itself (either a read or write lock). > I know in the SMP mode??an ETS table must be locked when access it, but I don't make sure that must an ETS table be locked in the non-SMP mode(only one scheduler)? > An SMP-enabled emulator will do the locking even when running only one scheduler. A non-SMP emulator (erl -smp disable) will not do any locking. > Another question is, whether the default ETS table locking is on table-level or on record-level? > Thanks in advance! > > The default locking is on table-level, allowing only one update operation at a time per table. Table option write_concurrency will enable locking on a more fine grained level, allowing concurrent update operations. In current implementation 16 locks per table is used, which result in a probability of 1/16 that two random keys will collide on the same lock. Option write_concurrency have no effect on ordered_set's. /Sverker, Erlang/OTP Ericsson From tristan.sloughter@REDACTED Mon May 7 15:14:09 2012 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Mon, 7 May 2012 08:14:09 -0500 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: Message-ID: > > I'd actually proposed a NPM clone as the subject of a hackathon at the next Erlang User conference (in may) Or a team could take it on for http://spawnfest.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfidelman@REDACTED Mon May 7 15:18:16 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Mon, 07 May 2012 09:18:16 -0400 Subject: [erlang-questions] Fwd: Is there a good source for documentation on BEAM? In-Reply-To: References: Message-ID: <4FA7CB98.50603@meetinghouse.net> At the very least, it's probably worth putting someone w/ the rest of Erlang documentation, rather than buried on dropbox! Miles Fidelman Joe Armstrong wrote: > ---------- Forwarded message ---------- > From: Joe Armstrong > Date: Mon, May 7, 2012 at 10:46 AM > Subject: Re: [erlang-questions] Is there a good source for > documentation on BEAM? > To: Jonathan Coveney > > > Hi, > > I did start writing a description but it's not very complete. > > This is on my list of things-to-do-one-day-when-you-get-time > > See http://dl.dropbox.com/u/4764922/beam.pdf > > If there is any interest I could up the priority :-) > > /Joe > > > On Mon, May 7, 2012 at 8:39 AM, Jonathan Coveney wrote: >> This question seems to come up now and again, and it's surprising to me that >> a crucial part of the documentation isn't better documented. Is there a >> reason that it is the case? Is the reason that there is no VM spec to give >> the devs the flexibility to change the intermediate layer without having to >> worry about backwards compatibility to the degree that Java does? >> >> Thus far I've found a description of the opcodes: >> http://azunyanmoe.wordpress.com/2011/03/30/erlang-vm-opcodes/ >> >> and this resource on the file format: >> http://www.erlang.se/~bjorn/beam_file_format.html >> >> But there doesn't seem to be a lot of high level talk about what the opcodes >> do (a la the JVM specification, for example). I know it's not impossible, >> and could always ask the guys at Erjang how they went about it, but thought >> I'd ask here. >> >> Please forgive a newbie question, and thanks in advance >> Jon >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From robert.virding@REDACTED Mon May 7 15:19:56 2012 From: robert.virding@REDACTED (Robert Virding) Date: Mon, 07 May 2012 14:19:56 +0100 (BST) Subject: [erlang-questions] Poor interface of string:to_integer/to_float In-Reply-To: <4c1c5193-4748-4e8c-aaa8-e5d5e5cfcc8a@knuth> Message-ID: <4726f3c5-7bed-4511-a349-985bec92d30a@knuth> The interfaces to the functions string:to_integer/1 and string:to_float/1 is, to put it mildly, very poor. You get the impressions that someone needed special case functions which were then put into the libraries. Would it be possible to change them to something more useful? Shall I write an EEP? Robert From torben.lehoff@REDACTED Mon May 7 15:21:46 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Mon, 07 May 2012 15:21:46 +0200 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> <4FA1CEC8.1060807@gmail.com> Message-ID: <4FA7CC6A.2000004@gmail.com> On 04/05/2012 09:32, Tim Watson wrote: > Is Columbo somehow different from what rebar already does in this space? One major difference is that Colombo tries to adhere to the version numbering scheme of OTP and avoids using the source area for generating releases. What does that mean? It means that after building in ./gproc the compiled version will be installed in ./lib/gproc-0.4.3 This allows a more strict separation of managing dependencies and the actual building of the app you are working on. I tried to get rebar to do the same, but it was too painful for me to get that to work, so I decided to do an experiment of my own to see if I could create something that could help me out. Another reason for doing this was to create a set-up so that I could use sinan as my build tool and since sinan does not deal with dependencies I had to come up with a solutions for my needs. I will try to write a blog post and create a Readme for Colombo so that other can see what I am trying to do more clearly. In my dreams Colombo functionality is rolled into the major tools once we agree on the steps in the process, but that is not really up to me to decide. Cheers, Torben > > > > On 4 May 2012, at 03:06, Tristan Sloughter > > wrote: > >> Ah yes, I briefly looked at colombo when you mentioned it on the >> erlware list. I'll give that a deeper look to see how much of what I >> need is covered by it and possibly simply start extending it. >> >> Tristan >> >> On Thu, May 3, 2012 at 4:52 PM, Torben Hoffmann >> > wrote: >> >> That actually sounds like a good step forward - it almost, but >> only almost makes me sad that I have created my own dependency >> management tool called colombo (alpha vers. is on github/lehoff). >> >> I haven't spent much time on it, but it can fetch apps from git >> repos as well as their dependencies, one can specify a tag or a >> branch to check out, a simple build can be done and you can get a >> version tagged version installed in the subdir lib of the dir >> where you you colombo (vers is taken from the app file or a >> default is choosen). >> >> very much alpha quality, but it shows that small dedicated tools >> aren't that hard to create for the Erlang eco system. >> >> I will continue improving colombo since it helps me solve my own >> problem, but it would be nice to have "real" users as well that >> could help form the functionality of the tool. Or tools since I >> might already have added too much functionality to colombo. >> >> Cheers, >> Torben >> >> Sent from my iPad >> >> On 03/05/2012, at 15.30, Tristan Sloughter >> > > wrote: >> >>> I feel like someone said Travis-CI wasn't the answer... But I am >>> now having the thought it is, at least to make something I'd >>> like to see without me having to do it. >>> >>> The question is would the community be interested in this if >>> I implement it. Here is what I'm talking about: >>> >>> 1) A github hook that informs X that an update to a branch or >>> tag is pushed. >>> 2) This hook does not only ensure tests pass but also that the >>> version numbers are correct. If you tag a project as 0.1.0 and >>> the .app file has {vsn, 0.1.0} it fails >>> 3) Assuming these criteria are met the agner repo for the >>> project is either updated or created. Instead of @master it >>> would use a piece of the git hash for the branch pointer (using >>> a moving target -- a branch name -- as the version of an app >>> doesn't make sense), for a tag it would use the tag name. >>> >>> Now, this is only part of the large solution people want, and I >>> agree with them we should do. >>> >>> But, I think it is a start and something that can be done in >>> small enough amount of effort to be done soon. >>> >>> Tristan >>> >>> On Wed, May 2, 2012 at 7:18 PM, Tim Watson >>> > wrote: >>> >>> On 02/05/2012 21:13, Ciprian Dorin Craciun wrote: >>> >>> On Wed, May 2, 2012 at 11:03 PMv, Tim >>> Watson>> > wrote: >>> >>> Eric Merit and I have had some lengthy discussions >>> about this on the Erlware mailing list and have some >>> ideas that I think are pretty solid. >>> >>> I'm glad to hear this. (I'll give it a look.) >>> >>> Cool thanks - please do leave any feedback you feel is >>> relevant. >>> >>> The thing is though, you don't just need tools - you >>> also need people to package their stuff up using the >>> tools. >>> >>> I agree about this. In fact I think currently there >>> are a lot of >>> tools with diverging solutions. >>> >>> Also I don't think that one "blessed" tool would be >>> the final >>> answer. I would have taken a somehow different road, >>> similar maybe to >>> how Go is going (although they do have the "one" tool): >>> i.e. strict >>> conventions. >>> >>> Yes I agree that '1 tool to rule them all' isn't going to >>> work. Eric and I have discussed building a unix-y pipeline >>> of tools to deal with local/remote package and repository >>> management, dependency resolution, installing and/or >>> generating a code path (or some other structure) for >>> bootstrapping either ERL_LIBS or a call to code:path, etc. >>> >>> The assumption we have is that various parts can be dealt >>> with be different tools providing they respect the APIs, so >>> you can build with rebar or sinan or make or whatever. We >>> are also planning on using reltool (or some >>> alternative/replica of sorts) to deal with the packaging >>> bits, as well as a set of tools for publication, >>> code/package signing and the like. >>> >>> We would also like to support different release types (e.g., >>> embedded vs. non), distinguish between applications and >>> library applications and so on. >>> >>> >>> What do I mean is this: >>> a) it would be nice to have a standardized way to >>> specify "extra" >>> options to compile an Erlang module. (We have the module >>> attributes >>> that we could use.) (the same for C sources); >>> b) we already have a standard project layout; (i.e. >>> `./src`, >>> `./include`, etc.) >>> c) we already have a standard project dependencies >>> (i.e. the `app` file); >>> >>> All we need to do is be consistent in this >>> convention, and then >>> all the various build and packaging systems could adapt. >>> >>> Packaging wise, I don't see how any of this really helps so >>> much, although I completely agree that sticking to this >>> consistent layout is a good thing. >>> >>> Using the app file for dependencies is fine at runtime - I >>> do this in https://github.com/hyperthunk/appstart - but if >>> you're connecting to a (possibly remote) artefact repository >>> and fetching stuff, then you need to specify the >>> publisher/signer (because 2 OTP applications could be >>> written with the same name, and of course with FOSS project >>> forking this happens all the time), the application name, >>> and the version. You can't whack all of that into your .app >>> file without breaking various things. >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Mon May 7 15:25:01 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 7 May 2012 14:25:01 +0100 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: Message-ID: Joe this is exactly what we need, and I hope this gains some traction. A few comments inline. On 7 May 2012, at 08:37, Joe Armstrong wrote: > Hello, > > I've been watching this thread - but not contributing ... > > I have a few questions/thoughts/comments > > 1) NodeJSs NPM looks pretty good to me. I've been toying with the idea > of making a clone > the advantages of cloning seem to be: > - it is very well documented > - it has a decent set of commands > - it produces a reasonable (target) directory structure > > 2) What's wrong today? > > It seems to me that the single biggest problem is the lack of a decent > manifest file. > > NodeJS uses > https://github.com/isaacs/npm/blob/master/package.json > For comparison Google chrome apps use > http://code.google.com/chrome/extensions/manifest.html > We (Erlang) use > http://www.erlang.org/doc/design_principles/applications.html#id73480 > > In a sense once you *defined* the manifest file and the file > system layout, all the rest > follows naturally. (Once you've agreed on the data formats - > which program produces > the files is irrelevant) > > Now the .app files date from a time *before* Erlang was open > source, so there > are no things for licenses, digital signatures and so on. > > So I think a first step is hammer out the content of the manifest. > +1 > The manifest could be simple or very complicated - so there are > a lot of design questions > here. > > 3) Tricky problems > > How do we avoid version nightmares? > > i.e. if A depends upon X.vsn1 and breaks with X.vsn2 > and B depends upon X.vsn2 and breaks with X.vsn1 > how can we run A and B in the same namespace? > > Three answers: > > a) You can't > b) Declare A to be broken and fix it (assuming X.vsn2 is the > latest and best version of > X) > c) do some magic name munging (refactoring) > that allows A and B to run in the same namspace with > X.vsn1 and X.vsn2 > d) run in separate nodes with message passing > e) add some sort of namespaces to erlang > > Comments: > > c) is pie in the sky - I don't see this happening realistically > > d) yes - but the apps need changing > > e) is research > > a) is bad > > b) is the best option - fix A - now why can't the > refactoring guys do this for us :-) > > In practice I'd mail the authors of A and hope they could help > I agree with this. I think that it would also be really useful if I could fork the project, change the dependency and re-package it signed under my packaging id. Eric Merrit and I have talking about this as a kind of third party signing. > 4) Signing > > No we want digital signatures - bla bla bla .... signed code > Yep > > 5) Should packages be unpacked on the target machines > > (I think not - this is abstraction breaking) > > The way rebar is packaged is nice. > Yes but not all stuff can be done like that. If we fix the little problems with .ez files then they're probably ideal for most cases. > 6) The way forward > > - Define a new manifest (say erlang .app files with inspiration from NPM) > - Define the layout of the local repository (ie where things land > locally after > a package has been fetched) > - Make a clone of NPM (or similar) > - Make a converter that sucks code from rebar github repository > and repackages, throwing away the history inn (.git) > You also need to support a notion of two or more publishers for apps win the same name and version, in order to properly support the kind of forking we see in many open source packages. I think I this as the 'publisher' > 7) Binary or source packages? > > - Ummm - tricky > I think binary is better but adds a lot of complexity. Erts versions, projects containing native code, etc. makin this the publishers responsibility means the tool only has to focus on the packaging problem not the build space. Third party signing allows you to support sources which gaven't been published by the originator. On the other hand, being able to only build conforming projects makes edge cases awkward when using source packages and whilst consistency and uniformity are good, the real world Ian's always like that. > 8) Community buy-in > > We need one recognised format that everybody uses. > > I'd actually proposed a NPM clone as the subject of a hackathon at the > next Erlang > User conference (in may) - now (of course) this is not a particularly difficult > program to write (famous last words) - the tricky bit is agreeing on the > manifest and file layout and semantics of updates etc. > > If any of you guys are coming to the Erlang users conference in > Stockholm I'd be happy to > discuss this - or even participate in a hackathon session > > > Cheers > > /Joe > > > > > > > > > On Wed, May 2, 2012 at 8:39 PM, Ciprian Dorin Craciun > wrote: >> (As no-one replied in 4 days, I'll add my opinion...) >> >> On Sat, Apr 28, 2012 at 4:36 PM, Tristan Sloughter >> wrote: >>> I started with this problem as something to simply discuss with the >>> maintainers of certain build and package management projects -- sinan and >>> agner specifically. But it seems to be more broad and cover how all of us >>> who keep apps on github handle versioning and dependencies. >>> >>> The basic issue is app versions within .app files on branches in github, the >>> resultant directory from a agner install and the discrepancies this causes. >>> >>> [...] >>> >>> Am I the only one going crazy with this? >> >> Nop. I've been faced with this problem myself a couple of times. >> (I've chosen to ignore it.) >> >> >>> Does anyone have >>> suggestions/examples of what they do for production projects on teams to >>> ease these annoyances? >> >> For example for my project I have one big Git repository called >> `myproject-repositories` (replace `myproject` with what you need) with >> submodules pointing to the dependencies. >> >> Then inside my own project repositories I have a symlink >> `.repositories` poiting to the "current" snapshot of dependencies. >> >> I also don't use any of rebar, agner, etc., mainly because I've >> hacked something on-top of the ninja build system. >> >> >>> I mostly just create my own packages and repos of dependencies, or package >>> third party deps with the project. >> >> Yup. +1 :) >> >> >> P.S.: I hate Java from the bottom of my heart, but boy-o-boy is >> the Java tooling way better than that of Erlang... I mean what Erlang >> is missing is something similar to NodeJS's NPM, or Java's Maven >> (minus the XML baggage and all useless complexities), or Python >> setup-tools (with virtual environment), etc... >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From tristan.sloughter@REDACTED Mon May 7 15:27:57 2012 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Mon, 7 May 2012 08:27:57 -0500 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: <4FA7CC6A.2000004@gmail.com> References: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> <4FA1CEC8.1060807@gmail.com> <4FA7CC6A.2000004@gmail.com> Message-ID: > > One major difference is that Colombo tries to adhere to the version > numbering scheme of OTP and avoids using the source area for generating > releases. It means that after building in ./gproc the compiled version will be > installed in ./lib/gproc-0.4.3 > This allows a more strict separation of managing dependencies and the > actual building of the app you are working on. > I tried to get rebar to do the same, but it was too painful for me to get > that to work, so I decided to do an experiment of my own to see if I could > create something that could help me out. > Just wanted to say this was an issue that has been recently debated regarding source area for generating releases, http://blog.erlware.org/2012/05/04/sinan-releases-and-being-right/ And that Colombo is sounding definitely like I had already planned on building around agner, so I think I'm going to start out trying to use Colombo for my needs. Tristan -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.james.barry@REDACTED Mon May 7 15:31:04 2012 From: paul.james.barry@REDACTED (Paul Barry) Date: Mon, 7 May 2012 14:31:04 +0100 Subject: [erlang-questions] Which is best? string:concat or ++? Message-ID: Hi folks. This might be a case of a dumb question, but here goes anyway. :-) I have two strings, A and B. Is it better to use: string:concat(A, B) or A ++ B when combining them to create a new string? I'm writing some code that generates a chunk of HTML. I know that using ++ on big lists is regarded as a "no-no", but is it acceptable here? Thanks. Paul. -- Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul.barry@REDACTED Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From tristan.sloughter@REDACTED Mon May 7 15:31:24 2012 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Mon, 7 May 2012 08:31:24 -0500 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: Message-ID: > > > 7) Binary or source packages? > > > > - Ummm - tricky > > > > I think binary is better but adds a lot of complexity. Erts versions, > projects containing native code, etc. makin this the publishers > responsibility means the tool only has to focus on the packaging problem > not the build space. Third party signing allows you to support sources > which gaven't been published by the originator. > I'd be happy with starting with just a source solution, but I think a binary one is definitely needed and can be simplified with TravisCI -- it'll build with multiple Erlang versions. I really wish I could plugin to the hosted TravisCI to have a job run to publish artifacts. I only briefly looked into that, so maybe something there is possible. Tristan -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.joseph.davis@REDACTED Mon May 7 15:37:13 2012 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Mon, 7 May 2012 08:37:13 -0500 Subject: [erlang-questions] Which is best? string:concat or ++? In-Reply-To: References: Message-ID: Quick scan of the source shows that string:concat/2 is exactly the ++ operator. So using ++ will save you a function invocation assuming the Erlang compiler doesn't optimize that away. https://github.com/erlang/otp/blob/maint/lib/stdlib/src/string.erl#L61 On Mon, May 7, 2012 at 8:31 AM, Paul Barry wrote: > Hi folks. > > This might be a case of a dumb question, but here goes anyway. ?:-) > > I have two strings, A and B. ?Is it better to use: > > ? string:concat(A, B) > > or > > ? ?A ++ B > > when combining them to create a new string? ?I'm writing some code > that generates a chunk of HTML. ?I know that using ++ on big lists is > regarded as a "no-no", but is it acceptable here? > > Thanks. > > Paul. > > -- > Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul.barry@REDACTED > Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From sedrik@REDACTED Mon May 7 15:45:19 2012 From: sedrik@REDACTED (Fredrik Andersson) Date: Mon, 7 May 2012 15:45:19 +0200 Subject: [erlang-questions] Which is best? string:concat or ++? In-Reply-To: References: Message-ID: I would recommend also looking at http://www.erlang.org/doc/man/lists.html#concat-1 On Mon, May 7, 2012 at 3:37 PM, Paul Davis wrote: > Quick scan of the source shows that string:concat/2 is exactly the ++ > operator. So using ++ will save you a function invocation assuming the > Erlang compiler doesn't optimize that away. > > https://github.com/erlang/otp/blob/maint/lib/stdlib/src/string.erl#L61 > > On Mon, May 7, 2012 at 8:31 AM, Paul Barry wrote: >> Hi folks. >> >> This might be a case of a dumb question, but here goes anyway. ?:-) >> >> I have two strings, A and B. ?Is it better to use: >> >> ? string:concat(A, B) >> >> or >> >> ? ?A ++ B >> >> when combining them to create a new string? ?I'm writing some code >> that generates a chunk of HTML. ?I know that using ++ on big lists is >> regarded as a "no-no", but is it acceptable here? >> >> Thanks. >> >> Paul. >> >> -- >> Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul.barry@REDACTED >> Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From carlsson.richard@REDACTED Mon May 7 15:59:02 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Mon, 07 May 2012 15:59:02 +0200 Subject: [erlang-questions] Poor interface of string:to_integer/to_float In-Reply-To: <4726f3c5-7bed-4511-a349-985bec92d30a@knuth> References: <4726f3c5-7bed-4511-a349-985bec92d30a@knuth> Message-ID: <4FA7D526.1050801@gmail.com> On 05/07/2012 03:19 PM, Robert Virding wrote: > The interfaces to the functions string:to_integer/1 and > string:to_float/1 is, to put it mildly, very poor. You get the > impressions that someone needed special case functions which were > then put into the libraries. Would it be possible to change them to > something more useful? Shall I write an EEP? Assuming that you intend to preserve backwards compatibility and just make the functions more general, I don't see that an EEP is needed. /Richard From erlang@REDACTED Mon May 7 15:59:51 2012 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 7 May 2012 15:59:51 +0200 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: Message-ID: On Mon, May 7, 2012 at 3:14 PM, Tristan Sloughter wrote: >> I'd actually proposed a NPM clone as the subject of a hackathon at the >> >> next Erlang?User conference (in may) > > > Or a team could take it on for?http://spawnfest.com/ Sorry I got confused - I meant spawnfest (I don't know if there is a hackathon this year) /Joe From jose.valim@REDACTED Mon May 7 16:01:36 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Mon, 7 May 2012 16:01:36 +0200 Subject: [erlang-questions] binary_to_integer/1 and binary_to_float/1 and friends In-Reply-To: References: Message-ID: Can someone from the OTP team let me know if I need an EEP for this or if I should just send patches straight to erlang-patches? Thanks in advance. On Mon, Apr 30, 2012 at 7:47 PM, Jos? Valim wrote: > At the beginning of this year, there was a discussion about how to > implement binary_to_integer/1 in Erlang. > There is a possible implementation for such function which is quite > trivial: > > list_to_integer(binary_to_list(binary)) > > However, it has the trade-off that it can't be used in guards (unless > defined in a macro) and there is an unnecessary conversion cost to list > (although the cost is small since both binary and list are likely small). > > With Riak, Cowboy and other tools pushing towards using binaries instead > of lists for strings, the need for such functions will become more and more > common. > Is there any chance we will see this as built-in functions and therefore > available in guards in R16? > > Thanks, > > * > Jos? Valim > www.plataformatec.com.br > Founder and Lead Developer > * > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.james.barry@REDACTED Mon May 7 16:02:39 2012 From: paul.james.barry@REDACTED (Paul Barry) Date: Mon, 7 May 2012 15:02:39 +0100 Subject: [erlang-questions] Which is best? string:concat or ++? In-Reply-To: References: Message-ID: Thanks for the quick responses, Paul and Fredrik. That's cleared things up for me (as well as given me another way to accomplish what I need). Cheers. :-) On 7 May 2012 14:45, Fredrik Andersson wrote: > I would recommend also looking at > http://www.erlang.org/doc/man/lists.html#concat-1 > > On Mon, May 7, 2012 at 3:37 PM, Paul Davis wrote: >> Quick scan of the source shows that string:concat/2 is exactly the ++ >> operator. So using ++ will save you a function invocation assuming the >> Erlang compiler doesn't optimize that away. >> >> https://github.com/erlang/otp/blob/maint/lib/stdlib/src/string.erl#L61 >> >> On Mon, May 7, 2012 at 8:31 AM, Paul Barry wrote: >>> Hi folks. >>> >>> This might be a case of a dumb question, but here goes anyway. ?:-) >>> >>> I have two strings, A and B. ?Is it better to use: >>> >>> ? string:concat(A, B) >>> >>> or >>> >>> ? ?A ++ B >>> >>> when combining them to create a new string? ?I'm writing some code >>> that generates a chunk of HTML. ?I know that using ++ on big lists is >>> regarded as a "no-no", but is it acceptable here? >>> >>> Thanks. >>> >>> Paul. >>> >>> -- >>> Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul.barry@REDACTED >>> Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -- Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul.barry@REDACTED Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From comptekki@REDACTED Mon May 7 16:12:45 2012 From: comptekki@REDACTED (Wes James) Date: Mon, 7 May 2012 08:12:45 -0600 Subject: [erlang-questions] Which is best? string:concat or ++? In-Reply-To: References: Message-ID: On Mon, May 7, 2012 at 7:31 AM, Paul Barry wrote: > Hi folks. > > This might be a case of a dumb question, but here goes anyway. ?:-) > > I have two strings, A and B. ?Is it better to use: > > ? string:concat(A, B) > > or > > ? ?A ++ B > > when combining them to create a new string? ?I'm writing some code > that generates a chunk of HTML. ?I know that using ++ on big lists is > regarded as a "no-no", but is it acceptable here? > You may want to consider, if possible, to move all your string operations to binary format. For example: A = <<"">>. B = <<"My Title">>. C = <<"hi">>. All = <>. The All part could just be a return from a fun, etc. -wes From mononcqc@REDACTED Mon May 7 16:17:33 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 07 May 2012 10:17:33 -0400 Subject: [erlang-questions] Which is best? string:concat or ++? In-Reply-To: References: Message-ID: <4FA7D97D.8010905@ferd.ca> Given you generate chunks of HTML and that this will undoubtedly be pushed over a socket at some point, what you want to use is IO Lists. They are lists of either bytes (integers from 0 to 255), binaries, or other IO lists. This means that functions that accept IO lists can accept items such as [$H, $e, [$l, <<"lo">>, " "], [[["W","o"], <<"rl">>]] | [<<"d">>]]. When this happens, the Erlang VM will just flatten the list as it needs to do it to obtain the sequence of characters Hello World. Any function from the io module, file module, TCP and UDP sockets will be able to handle them. Some library functions, such as some coming from the unicode module and all of the functions from the re (for regular expressions) module will also handle them, to name a few. So in your case, to join string A and B, no matter whether they're binaries or lists, just do [A,B] and that's your new string. You'll save a lot in terms of rewriting terms and whatnot doing things that way. On 12-05-07 9:31 AM, Paul Barry wrote: > Hi folks. > > This might be a case of a dumb question, but here goes anyway. :-) > > I have two strings, A and B. Is it better to use: > > string:concat(A, B) > > or > > A ++ B > > when combining them to create a new string? I'm writing some code > that generates a chunk of HTML. I know that using ++ on big lists is > regarded as a "no-no", but is it acceptable here? > > Thanks. > > Paul. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.james.barry@REDACTED Mon May 7 16:18:23 2012 From: paul.james.barry@REDACTED (Paul Barry) Date: Mon, 7 May 2012 15:18:23 +0100 Subject: [erlang-questions] Which is best? string:concat or ++? In-Reply-To: References: Message-ID: I take it that doing it that way is "faster" than string manipulation (or is there some other reason for this suggestion)? On 7 May 2012 15:12, Wes James wrote: > On Mon, May 7, 2012 at 7:31 AM, Paul Barry wrote: >> Hi folks. >> >> This might be a case of a dumb question, but here goes anyway. ?:-) >> >> I have two strings, A and B. ?Is it better to use: >> >> ? string:concat(A, B) >> >> or >> >> ? ?A ++ B >> >> when combining them to create a new string? ?I'm writing some code >> that generates a chunk of HTML. ?I know that using ++ on big lists is >> regarded as a "no-no", but is it acceptable here? >> > > > You may want to consider, if possible, to move all your string > operations to binary format. > > For example: > > A = <<"">>. > > B = <<"My Title">>. > > C = <<"hi">>. > > All = <>. > > The All part could just be a return from a fun, etc. > > -wes -- Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul.barry@REDACTED Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From comptekki@REDACTED Mon May 7 16:47:36 2012 From: comptekki@REDACTED (Wes James) Date: Mon, 7 May 2012 08:47:36 -0600 Subject: [erlang-questions] Which is best? string:concat or ++? In-Reply-To: References: Message-ID: On Mon, May 7, 2012 at 8:18 AM, Paul Barry wrote: > I take it that doing it that way is "faster" than string manipulation > (or is there some other reason for this suggestion)? > > On 7 May 2012 15:12, Wes James wrote: >> On Mon, May 7, 2012 at 7:31 AM, Paul Barry wrote: >>> Hi folks. >>> >>> This might be a case of a dumb question, but here goes anyway. ?:-) >>> >>> I have two strings, A and B. ?Is it better to use: >>> >>> ? string:concat(A, B) >>> >>> or >>> >>> ? ?A ++ B >>> >>> when combining them to create a new string? ?I'm writing some code >>> that generates a chunk of HTML. ?I know that using ++ on big lists is >>> regarded as a "no-no", but is it acceptable here? >>> >> >> >> You may want to consider, if possible, to move all your string >> operations to binary format. >> >> For example: >> >> A = <<"">>. >> >> B = <<"My Title">>. >> >> C = <<"hi">>. >> >> All = <>. >> >> The All part could just be a return from a fun, etc. >> >> -wes > > > > -- > Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul.barry@REDACTED > Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From michael.eugene.turner@REDACTED Mon May 7 16:48:14 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Mon, 7 May 2012 23:48:14 +0900 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: <4945B8FA-2780-498F-A510-3D1905AA6CB4@masklinn.net> References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <4945B8FA-2780-498F-A510-3D1905AA6CB4@masklinn.net> Message-ID: > On 7 mai 2012, at 13:47, Joe Armstrong wrote: > >> I think it works like this: >> >> ? ?1) first you don't understand how the X works (X=Beam, JVM, X11, >> ... you name it) >> ? ?2) You struggle - and think - ?google and have a hot bath >> ? ?3) Eureka - bath flows over 3.1) Because you're Archimedes, you have a very bright slave-boy scribe. You explain it to him exultantly ... 3.2) ... then again in puzzlement over *his* puzzlement .... 3.3) ... then yet again in profound irritation over his obtuseness. 3.4) He finally gets it. But then he points out a subtle error in your reasoning, one that had been tripping him up in his understanding. 3.5) You have him flogged for his impertinence. 3.6) He copy-edits and publishes the manuscript under your name. But he also embezzles 10% of your book royalties to repatriate to his war-widowed mother who is still living in the city-state that your city-state conquered, resulting in his slavery. We now return you to our regularly-scheduled historical narrative .... -michael turner From comptekki@REDACTED Mon May 7 16:48:19 2012 From: comptekki@REDACTED (Wes James) Date: Mon, 7 May 2012 08:48:19 -0600 Subject: [erlang-questions] Which is best? string:concat or ++? In-Reply-To: References: Message-ID: On Mon, May 7, 2012 at 8:18 AM, Paul Barry wrote: > I take it that doing it that way is "faster" than string manipulation > (or is there some other reason for this suggestion)? Based on some discussion I've seen on the list in the past, I believe binary is faster. ?In my case, I'm using binaries to construct html chunks as I'm using cowboy, but I think cowboy can also use io lists, like Fred mentioned. -wes From bengt.kleberg@REDACTED Mon May 7 16:49:10 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 7 May 2012 16:49:10 +0200 Subject: [erlang-questions] beginner: Is there an alternative to net_adm:ping/1 before global:sync/0 Message-ID: <1336402150.4750.99.camel@seasc1137> Greetings, Before I do a global:sync/0 I have a net_adm:ping/1 to the node that has the global process. Are there any alternatives to net_adm:ping/1? bengt From mononcqc@REDACTED Mon May 7 16:50:38 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 07 May 2012 10:50:38 -0400 Subject: [erlang-questions] Which is best? string:concat or ++? In-Reply-To: References: Message-ID: <4FA7E13E.30000@ferd.ca> Cowboy does accept IOLists. They're very rarely going to *not* be the fastest data structure to handle the concatenation strings to be output, in my experience. I do recommend them for any and all appending and prepending that needs to be done with web servers, files, etc. On Mon May 7 10:48:19 2012, Wes James wrote: > > On Mon, May 7, 2012 at 8:18 AM, Paul Barry > wrote: >> >> I take it that doing it that way is "faster" than string manipulation >> (or is there some other reason for this suggestion)? > > > Based on some discussion I've seen on the list in the past, I believe > binary is faster. In my case, I'm using binaries to construct html > chunks as I'm using cowboy, but I think cowboy can also use io lists, > like Fred mentioned. > > -wes > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From Tobias.Schlager@REDACTED Mon May 7 17:08:19 2012 From: Tobias.Schlager@REDACTED (Tobias Schlager) Date: Mon, 7 May 2012 15:08:19 +0000 Subject: [erlang-questions] [ANN] maven-erlang-plugin 2.2.0 Message-ID: <12F2115FD1CCEE4294943B2608A18FA335AA24E7@MAIL01.win.lbaum.eu> Hi, the maven-erlang-plugin version 2.2.0 has been released. The release is focused on improvements regarding erlang release and target system creation as well as SNMP support. We are currently moving the project to github along with some mavenified forks of popular erlang open source projects. Links * http://erlang-plugin.sourceforge.net/ * https://github.com/lindenbaum Regards Tobias From mfidelman@REDACTED Mon May 7 17:18:53 2012 From: mfidelman@REDACTED (Miles Fidelman) Date: Mon, 07 May 2012 11:18:53 -0400 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: Message-ID: <4FA7E7DD.6070803@meetinghouse.net> It keep occurring to me that it sure would be nice to have something like a Smalltalk environment for Erlang - essentially tools for inspecting and managing live systems, with the ability to edit, update, and rollback code. -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From bob@REDACTED Mon May 7 17:19:14 2012 From: bob@REDACTED (Bob Ippolito) Date: Mon, 7 May 2012 08:19:14 -0700 Subject: [erlang-questions] Which is best? string:concat or ++? In-Reply-To: <4FA7E13E.30000@ferd.ca> References: <4FA7E13E.30000@ferd.ca> Message-ID: The advantage of binaries is that they take up significantly less memory per character and you can send them to other processes on the same node with no copying. Iolists of binaries are also good to use for IO. On Monday, May 7, 2012, Fred Hebert wrote: > Cowboy does accept IOLists. They're very rarely going to *not* be the > fastest data structure to handle the concatenation strings to be output, in > my experience. I do recommend them for any and all appending and prepending > that needs to be done with web servers, files, etc. > > On Mon May 7 10:48:19 2012, Wes James wrote: > > > On Mon, May 7, 2012 at 8:18 AM, Paul Barrywrote: > > > I take it that doing it that way is "faster" than string manipulation > (or is there some other reason for this suggestion)? > > > > Based on some discussion I've seen on the list in the past, I believe > binary is faster. In my case, I'm using binaries to construct html > chunks as I'm using cowboy, but I think cowboy can also use io lists, > like Fred mentioned. > > -wes > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED 'erlang-questions@REDACTED');> > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peerst@REDACTED Mon May 7 17:27:31 2012 From: peerst@REDACTED (Peer Stritzinger) Date: Mon, 7 May 2012 17:27:31 +0200 Subject: [erlang-questions] BIF or NIF for extending Erlang on OS which can't dynamic link In-Reply-To: References: <20120504081420.GA19861@circlewave.net> Message-ID: On Fri, May 4, 2012 at 8:00 PM, Jared Kofron wrote: > I'm curious what you are working on that you would be porting Erlang to > RTEMS? Its work for a customer, can't tell too much about it yet. Its for a small industrial control component that has the following requirements: * physically small + hardware cost sensitive + support for a wide area of industrial communication protocols This results in choice of a SoC (MPC8309 from Freescale in our case) with only some NAND flash and limited RAM. * Some uses have hard realtime requirements, but many have soft realtime requirements and some dynamic memory use. So we have the hard realtime parts running in a RTEMS thread with higher prio than the Erlang VM's threads. Do I have to elaborate why not to write it completely in C on this list? Erlang is a great choice for most of the stuff we want to do with this platform. Besides RTEMS and Erlang fit quite well, both are built for high reliability (RTEMS is running on plenty of satelites and planetary probes e.g.), RTEMS has very fast thread switching and the amount of code run with higher prio than Erlang can be reduced to a minimum. This should result in higher responsiveness for Erlang since it can basically running on the bare metal with no preemption at all. I have built larger embedded systems where Erlang is running on top of FreeBSD talking to hardware boards running RTEMS via USB. There I found the collaboration of RTEMS (for the hard realtime, ultra low latency stuff) and Erlang (for as much of the rest as possible) very useful. So it was a natural choice to put Erlang and RTEMS on the same CPU if we don't have the room/"price budget" to use separate CPU's. So far everything looks well, there will be more info about this at a later point in time. And the RTEMS + Erlang code/build system etc. will be published (preferably by committing as much as possible back to the respective projects) Cheers, -- Peer Stritzinger From jared.kofron@REDACTED Mon May 7 17:30:05 2012 From: jared.kofron@REDACTED (Jared Kofron) Date: Mon, 7 May 2012 08:30:05 -0700 Subject: [erlang-questions] BIF or NIF for extending Erlang on OS which can't dynamic link In-Reply-To: References: <20120504081420.GA19861@circlewave.net> Message-ID: Very cool, I was just wondering as I use Erlang for a lot of process control type projects. A lot of folks in my field use RTEMS, so I thought there may be some crossover. Looking forward to hearing about your results! JK On Mon, May 7, 2012 at 8:27 AM, Peer Stritzinger wrote: > On Fri, May 4, 2012 at 8:00 PM, Jared Kofron > wrote: > > I'm curious what you are working on that you would be porting Erlang to > > RTEMS? > > Its work for a customer, can't tell too much about it yet. Its for a > small industrial control component that has the following > requirements: > > * physically small + hardware cost sensitive + support for a wide area > of industrial communication protocols > > This results in choice of a SoC (MPC8309 from Freescale in our case) > with only some NAND flash and limited RAM. > > * Some uses have hard realtime requirements, but many have soft > realtime requirements and some dynamic memory use. > > So we have the hard realtime parts running in a RTEMS thread with > higher prio than the Erlang VM's threads. > > Do I have to elaborate why not to write it completely in C on this > list? Erlang is a great choice for most of the stuff we want to do > with this platform. > > Besides RTEMS and Erlang fit quite well, both are built for high > reliability (RTEMS is running on plenty of satelites and planetary > probes e.g.), RTEMS has very fast thread switching and the amount of > code run with higher prio than Erlang can be reduced to a minimum. > This should result in higher responsiveness for Erlang since it can > basically running on the bare metal with no preemption at all. > > I have built larger embedded systems where Erlang is running on top of > FreeBSD talking to hardware boards running RTEMS via USB. There I > found the collaboration of RTEMS (for the hard realtime, ultra low > latency stuff) and Erlang (for as much of the rest as possible) very > useful. > > So it was a natural choice to put Erlang and RTEMS on the same CPU if > we don't have the room/"price budget" to use separate CPU's. > > So far everything looks well, there will be more info about this at a > later point in time. > > And the RTEMS + Erlang code/build system etc. will be published > (preferably by committing as much as possible back to the respective > projects) > > Cheers, > -- Peer Stritzinger > -------------- next part -------------- An HTML attachment was scrubbed... URL: From trevorw@REDACTED Mon May 7 17:36:13 2012 From: trevorw@REDACTED (Trevor Woollacott) Date: Mon, 7 May 2012 17:36:13 +0200 Subject: [erlang-questions] Strange SSL transport_accept behaviour In-Reply-To: References: Message-ID: Hi Ingela, Thank you Regards, Trevor > -----Original Message----- > From: Ingela Andin [mailto:ingela.andin@REDACTED] > Sent: 07 May 2012 04:12 PM > To: Trevor Woollacott > Subject: Re: [erlang-questions] Strange SSL transport_accept behaviour > > Hi! > > I think you are correct, its a bug we will fix it. > > Regards Ingela Erlang/OTP team - Ericsson AB > > 2012/5/4, Trevor Woollacott : > > Hi, > > > > I've noticed strange behaviour when ssl:transport_accept/2 returns > > with a timeout. It appears that certain inet options are reset to > > internal values and are never changed back if accept encounters an > error (such as timeout). > > > > In my case, I had set inet option {packet, http}, and after the > accept > > timed out it had changed to 0 (raw). > > > > For example: > >> ssl:setopts( ListenSocket, [{packet, http}] ). > > ok > >> ssl:transport_accept( ListenSocket, Timeout ). > > {error, timeout} > >> ssl:getopts( ListenSocket, [packet] ). > > {ok, [{packet,0}]} > > > > The problem seems to have been introduced in Erlang R14B. Looking at > > the source code below, we can see that the initial inet options are > > stored in InetValues, and then the inet options are reset to > > internal_inet_values (which includes {packet,0}) and the inet options > > are only set back to InetValues if the accept was successful. > > > > Is this the correct behaviour? I'm thinking that for timeout (and > > possibly certain other errors) the listener socket should be reset > > with the user defined inet options? > > > > transport_accept(#sslsocket{pid = {ListenSocket, #config{cb=CbInfo, > > ssl=SslOpts}}, > > fd = new_ssl}, Timeout) -> > > > > %% The setopt could have been invoked on the listen socket > > %% and options should be inherited. > > EmOptions = emulated_options(), > > {ok, InetValues} = inet:getopts(ListenSocket, EmOptions), > > ok = inet:setopts(ListenSocket, internal_inet_values()), > > {CbModule,_,_, _} = CbInfo, > > case CbModule:accept(ListenSocket, Timeout) of > > {ok, Socket} -> > > ok = inet:setopts(ListenSocket, InetValues), > > {ok, Port} = inet:port(Socket), > > ConnArgs = [server, "localhost", Port, Socket, > > {SslOpts, socket_options(InetValues)}, self(), CbInfo], > > case ssl_connection_sup:start_child(ConnArgs) of > > {ok, Pid} -> > > ssl_connection:socket_control(Socket, Pid, CbModule); > > {error, Reason} -> > > {error, Reason} > > end; > > {error, Reason} -> > > {error, Reason} > > end; > > > > Regards, > > Trevor > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > From paul.james.barry@REDACTED Mon May 7 17:39:09 2012 From: paul.james.barry@REDACTED (Paul Barry) Date: Mon, 7 May 2012 16:39:09 +0100 Subject: [erlang-questions] Which is best? string:concat or ++? In-Reply-To: References: <4FA7E13E.30000@ferd.ca> Message-ID: Thanks for all of that, folks. If there are any other newbies out there wanting to know more about IOlists (which are "missing" from my copies of "Programming Erlang" as well as "Erlang Programming"), here's a couple of links that I found that do a good job of describing same: http://dev.af83.com/2012/01/16/erlang-iolist.html http://prog21.dadgum.com/70.html Regards, Paul. P.S. Before others (and Joe) shout at me, IOlists are mentioned on page 230 of Programming Erlang, but only just). On 7 May 2012 16:19, Bob Ippolito wrote: > The advantage of binaries is that they take up significantly less memory per > character and you can send them to other processes on the same node with no > copying. Iolists of binaries are also good to use for IO. > > > On Monday, May 7, 2012, Fred Hebert wrote: >> >> Cowboy does accept IOLists. They're very rarely going to not be the >> fastest data structure to handle the concatenation strings to be output, in >> my experience. I do recommend them for any and all appending and prepending >> that needs to be done with web servers, files, etc. >> >> On Mon May 7 10:48:19 2012, Wes James wrote: >> >> >> On Mon, May 7, 2012 at 8:18 AM, Paul Barry >> wrote: >> >> >> I take it that doing it that way is "faster" than string manipulation >> (or is there some other reason for this suggestion)? >> >> >> >> Based on some discussion I've seen on the list in the past, I believe >> binary is faster. In my case, I'm using binaries to construct html >> chunks as I'm using cowboy, but I think cowboy can also use io lists, >> like Fred mentioned. >> >> -wes >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul.barry@REDACTED Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From peerst@REDACTED Mon May 7 19:44:15 2012 From: peerst@REDACTED (Peer Stritzinger) Date: Mon, 7 May 2012 19:44:15 +0200 Subject: [erlang-questions] Which is best? string:concat or ++? In-Reply-To: References: <4FA7E13E.30000@ferd.ca> Message-ID: On Mon, May 7, 2012 at 5:39 PM, Paul Barry wrote: > Thanks for all of that, folks. > > If there are any other newbies out there wanting to know more about > IOlists (which are "missing" from my copies of ?"Programming Erlang" > as well as "Erlang Programming"), here's a couple of links that I > found that do a good job of describing same: > > ? ?http://dev.af83.com/2012/01/16/erlang-iolist.html > > ? ?http://prog21.dadgum.com/70.html And the beginning of this chapter (you should read the rest also and buy the book once it is out!) http://learnyousomeerlang.com/buckets-of-sockets Cheers, -- Peer From watson.timothy@REDACTED Mon May 7 20:54:41 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 7 May 2012 19:54:41 +0100 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: Message-ID: This came up on the erlware lists as well. Although both Eric and I were hoping to put something a bit more structured in place before discussing it with the community, the gathering pace behind this thread suggests to me that there's little time for writing things up so here are the links to those discussions: - original idea from Eric: https://groups.google.com/forum/?fromgroups#!topic/erlware-questions/GtFBTQtgeng - overview: https://groups.google.com/forum/?fromgroups#!topic/erlware-questions/omunsj8pfs4 - some of Joe's questions we already visited: https://groups.google.com/forum/?fromgroups#!topic/erlware-questions/ZbRdDAkFQPo - repository design questions: https://groups.google.com/forum/?fromgroups#!topic/erlware-questions/vNHjrvIScGE - erlang/repository namespace issues: https://groups.google.com/forum/?fromgroups#!topic/erlware-questions/cav3oK_D8sw - code signing: https://groups.google.com/forum/?fromgroups#!topic/erlware-questions/1esqRJU11EE I hope they provide some useful inputs to this idea for everyone's benefit. Note that as far as I'm aware, no implementation work has gone into this yet, although I have done some prototyping with git as the fetch/synch mechanism using rather low level commands such as fetch-receive-pack, which I mentioned towards the end of this thread: https://groups.google.com/forum/?fromgroups#!topic/erlware-questions/js06abXa8Mk. Cheers, Tim On 7 May 2012 14:31, Tristan Sloughter wrote: >> > 7) Binary or source packages? >> > >> > ? ?- Ummm - tricky >> > >> >> I think binary is better but adds a lot of complexity. Erts versions, >> projects containing native code, etc. makin this the publishers >> responsibility means the tool only has to focus on the packaging problem not >> the build space. Third party ?signing allows you to support sources which >> gaven't been published by the originator. > > > I'd be happy with starting with just a source solution, but I think a binary > one is definitely needed and can be simplified with TravisCI -- it'll build > with multiple Erlang versions. > > I really wish I could plugin to the hosted TravisCI to have a job run to > publish artifacts. I only briefly looked into that, so maybe something there > is possible. > > Tristan From carlsson.richard@REDACTED Mon May 7 21:04:18 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Mon, 07 May 2012 21:04:18 +0200 Subject: [erlang-questions] beginner: Is there an alternative to net_adm:ping/1 before global:sync/0 In-Reply-To: <1336402150.4750.99.camel@seasc1137> References: <1336402150.4750.99.camel@seasc1137> Message-ID: <4FA81CB2.2040102@gmail.com> On 2012-05-07 16:49, Bengt Kleberg wrote: > Greetings, > > Before I do a global:sync/0 I have a net_adm:ping/1 to the node that has > the global process. Are there any alternatives to net_adm:ping/1? How about net_kernel:connect_node/1? /Richard From carlsson.richard@REDACTED Mon May 7 21:06:58 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Mon, 07 May 2012 21:06:58 +0200 Subject: [erlang-questions] binary_to_integer/1 and binary_to_float/1 and friends In-Reply-To: References: Message-ID: <4FA81D52.3090203@gmail.com> EEPs should only be needed for changes to the language itself, or more far reaching changes to the standard libraries. For a small change like this, just submit it as a patch and see if OTP will take it in. Don't forget that documentation and test cases are needed as well. /Richard On 2012-05-07 16:01, Jos? Valim wrote: > Can someone from the OTP team let me know if I need an EEP for this or > if I should just send patches straight to erlang-patches? Thanks in advance. > > On Mon, Apr 30, 2012 at 7:47 PM, Jos? Valim > wrote: > > At the beginning of this year, there was a discussion about how to > implement binary_to_integer/1 in Erlang. > There is a possible implementation for such function which is quite > trivial: > > list_to_integer(binary_to_list(binary)) > > However, it has the trade-off that it can't be used in guards > (unless defined in a macro) and there is an unnecessary conversion > cost to list (although the cost is small since both binary and list > are likely small). > > With Riak, Cowboy and other tools pushing towards using binaries > instead of lists for strings, the need for such functions will > become more and more common. > Is there any chance we will see this as built-in functions and > therefore available in guards in R16? > > Thanks, > > * > *Jos? Valim* > www.plataformatec.com.br > Founder and Lead Developer > * > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From thomasl_erlang@REDACTED Mon May 7 21:15:14 2012 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 7 May 2012 12:15:14 -0700 (PDT) Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> Message-ID: <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> ----- Original Message ----- > From: Michael Turner > To: Thomas Lindgren > Cc: Jonathan Coveney ; "erlang-questions@REDACTED" > Sent: Monday, May 7, 2012 11:27 AM > Subject: Re: [erlang-questions] Is there a good source for documentation on BEAM? > >& quot;Actually, I don't think such docs are all _that_ crucial -- who > really needs to know, except a small number of VM implementors?" > > Aren't Erlang's chances of greater mindshare improved by making it > easier to become a VM implementor? I doubt very much that Java would > be where it is today had it not been for clear VM specification. > That's not to say that Erlang should follow in all of Java's > footsteps, even if it could. But I have to say I was a boggled to > learn that you can't find out what the VM opcodes mean without reading > the source (and maybe not even then, if the source contains bugs > vis-a-vis some idealized machine model.) Well, we should ask why we need them. There has been a substantial number of non-BEAM Erlang implementations already, so I'm not convinced detailed BEAM docs is the key property* to spread Erlang.? Indeed, requiring detailed docs of?every change of BEAM seems likely to slow innovation down instead. If the motive is education, I think someone interested in compilers and virtual machine architectures would have little trouble with BEAM as such.?In a real sense, BEAM is just a vehicle to express compiler optimizations for a restricted part of ERTS (the sequential execution part, basically).?The interesting choices and optimizations are found by examining the whole implementation.?But again, a tutorial could be useful. Another argument might be that BEAM should be specified in detail in order to be a suitable binary format for distribution,? which is essentially?what the JVM instruction set has become. (The commercial implementations convert it into internal formats and optimize the hell out of those.) My druthers would in that case be to define?something less detailed? and less changeable to serve in this role.? Okay, that's all I can come up with. (* I'd prefer to have the marketing muscle of Sun instead!) Best regards, Thomas From torben.lehoff@REDACTED Mon May 7 21:30:59 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Mon, 07 May 2012 21:30:59 +0200 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> <4FA1CEC8.1060807@gmail.com> <4FA7CC6A.2000004@gmail.com> Message-ID: <4FA822F3.70807@gmail.com> On 07/05/2012 15:27, Tristan Sloughter wrote: > > One major difference is that Colombo tries to adhere to the > version numbering scheme of OTP and avoids using the source area > for generating releases. > > It means that after building in ./gproc the compiled version will > be installed in ./lib/gproc-0.4.3 > This allows a more strict separation of managing dependencies and > the actual building of the app you are working on. > I tried to get rebar to do the same, but it was too painful for me > to get that to work, so I decided to do an experiment of my own to > see if I could create something that could help me out. > > > Just wanted to say this was an issue that has been recently debated > regarding source area for generating releases, > http://blog.erlware.org/2012/05/04/sinan-releases-and-being-right/ > > And that Colombo is sounding definitely like I had already planned on > building around agner, so I think I'm going to start out trying to use > Colombo for my needs. > > Tristan Just report any issues you have with Colombo - it is alpha software, so I do not expect it to be in any way perfect. I am very open to put in new functionality or split Colombo into two tools if that seems better. It might be that my needs are a corner case and the more generally applicable solution is different - in fact it will be the case, so I am ready to get feedback and find ways to improve it. Cheers, Torben -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From fritchie@REDACTED Mon May 7 22:07:56 2012 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Mon, 07 May 2012 15:07:56 -0500 Subject: [erlang-questions] signal arrival time In-Reply-To: Message of "Wed, 02 May 2012 11:52:06 +0200." Message-ID: <96827.1336421276@snookles.snookles.com> Ulf Wiger wrote: uw> Another way to do it is to use DTrace (not yet officially supported) Hrm, perhaps it's "officially experimental"? :-) As of the R15B01 release, using "./configure --enable-vm-probes --with-dynamic-trace=dtrace [... other configure options" will create a VM that's instrumented to be able to fire probes on the following events: probe message__send(char *sender, char *receiver, uint32_t size, int token_label, int token_previous, int token_current); probe message__send__remote(char *sender, char *node_name, char *receiver, uint32_t size, int token_label, int token_previous, int token_current); probe message__queued(char *receiver, uint32_t size, uint32_t queue_len, int token_label, int token_previous, int token_current); probe message__receive(char *receiver, uint32_t size, uint32_t queue_len, int token_label, int token_previous, int token_current); See the comments in erts/emulator/beam/erlang_dtrace.d for more information on what each of those probes do. It's worth noting that none of these probes allow you to peek inside the message to see what it looks like. The message-queued probe is the one that Joszef is looking for. I'm not convinced that all possible queueing scenarios are actually covered by probes: there are SMP-related code paths that are obscured by #ifdef statements and remote-node-messaging-related paths that perhaps I missed entirely. As far as I know, the DTrace patches have only been reviewed for bugginess, *not* thoroughness/correctness. -Scott From jcoveney@REDACTED Mon May 7 22:22:40 2012 From: jcoveney@REDACTED (Jonathan Coveney) Date: Tue, 8 May 2012 05:22:40 +0900 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> Message-ID: I love the thoughtfulness of this listserv. I think a tutorial would be invaluable. I think the sorts of people who get into erlang may be like me... some functional background, a lot of non-functional background, you see erlang and want to get a sense of how the opcodes (and by extension the internals) work. You can trawl the source (and I will), but I definitely think that if someone were up to it, the community would be well served by a tutorial on the implementation details.[1] [1] where the devil is, so they say Thanks for the responses! Jon 2012/5/7 Thomas Lindgren > > > > > ----- Original Message ----- > > From: Michael Turner > > To: Thomas Lindgren > > Cc: Jonathan Coveney ; "erlang-questions@REDACTED" > > > Sent: Monday, May 7, 2012 11:27 AM > > Subject: Re: [erlang-questions] Is there a good source for documentation > on BEAM? > > > >& quot;Actually, I don't think such docs are all _that_ crucial -- who > > really needs to know, except a small number of VM implementors?" > > > > Aren't Erlang's chances of greater mindshare improved by making it > > easier to become a VM implementor? I doubt very much that Java would > > be where it is today had it not been for clear VM specification. > > That's not to say that Erlang should follow in all of Java's > > footsteps, even if it could. But I have to say I was a boggled to > > learn that you can't find out what the VM opcodes mean without reading > > the source (and maybe not even then, if the source contains bugs > > vis-a-vis some idealized machine model.) > > > Well, we should ask why we need them. > > There has been a substantial number of non-BEAM Erlang implementations > already, so I'm > not convinced detailed BEAM docs is the key property* to spread Erlang. > Indeed, requiring detailed docs of every change of BEAM seems likely to > slow innovation down instead. > > If the motive is education, I think someone interested in compilers and > virtual machine architectures > would have little trouble with BEAM as such. In a real sense, BEAM is just > a vehicle to express compiler optimizations for a > restricted part of ERTS (the sequential execution part, basically). The > interesting choices and optimizations are found by examining > the whole implementation. But again, a tutorial could be useful. > > Another argument might be that BEAM should be specified in detail in order > to be a suitable binary format for distribution, > which is essentially what the JVM instruction set has become. (The > commercial implementations convert it into internal > formats and optimize the hell out of those.) My druthers would in that > case be to define something less detailed > and less changeable to serve in this role. > > Okay, that's all I can come up with. > > (* I'd prefer to have the marketing muscle of Sun instead!) > > Best regards, > Thomas > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Mon May 7 22:56:58 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 7 May 2012 21:56:58 +0100 Subject: [erlang-questions] beginner: Is there an alternative to net_adm:ping/1 before global:sync/0 In-Reply-To: <4FA81CB2.2040102@gmail.com> References: <1336402150.4750.99.camel@seasc1137> <4FA81CB2.2040102@gmail.com> Message-ID: On 7 May 2012 20:04, Richard Carlsson wrote: > On 2012-05-07 16:49, Bengt Kleberg wrote: >> >> Greetings, >> >> Before I do a global:sync/0 I have a net_adm:ping/1 to the node that has >> the global process. Are there any alternatives to net_adm:ping/1? > > > How about net_kernel:connect_node/1? > What's wrong with net_adm:ping/1 just out of interest? Are you trying to avoid the long delay before it times out if epmd is not running on that node? If so you could try to gen_tcp connect to epmd on the right IP+port to ensure that it's running before you attempt to you net_kernel:connect_node/1. From ok@REDACTED Mon May 7 23:57:34 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 8 May 2012 09:57:34 +1200 Subject: [erlang-questions] Fwd: Is there a good source for documentation on BEAM? In-Reply-To: References: Message-ID: <79520DF4-2E14-4CA4-8880-0B03E4D6B8DD@cs.otago.ac.nz> On 7/05/2012, at 8:47 PM, Joe Armstrong wrote: > ---------- Forwarded message ---------- > From: Joe Armstrong > Date: Mon, May 7, 2012 at 10:46 AM > Subject: Re: [erlang-questions] Is there a good source for > documentation on BEAM? > To: Jonathan Coveney > > > Hi, > > I did start writing a description but it's not very complete. > > This is on my list of things-to-do-one-day-when-you-get-time > > See http://dl.dropbox.com/u/4764922/beam.pdf > > If there is any interest I could up the priority :-) > > /Joe I would have expected it to be a matter of personal and professional pride that One reason that several of my EEPs and so on never got model implementations is that the BEAM instructions have less than a 10th of the documentation that the WAM instructions had at Quintus. It does seem clear that the Erlang/OTP team are much better programmers than Quintus were, because whenever we forgot to document some detail of an instruction, someone was sure to get it wrong and new builds would start crashing horribly. It's also a reason I've never contributed at a low level to SWI Prolog. Two of the best documented abstract instruction sets I've seen are Icon (there's a whole book about it) and Lua (there's a document someone wrote with nearly a page per instruction; if I could bring myself to _care_ about Lua I could start writing extensions for it tomorrow). Since Hassan A?t-Kaci wrote his little book about the WAM, that counts as well documented too, except of course that I doubt anyone actually follows it all that closely, Quintus certainly departed from it. (And yes of course's there's the Java VM book too; I don't suppose it's a co?ncidence that Tim Lindholm worked at Quintus and times on the WAM.) Looking at that document of yours, Joe, (a) Out of 8 pages, only 5 are actually about the BEAM. (b) We learn only at the end of those 5 pages that there are really TWO different things called BEAM: - the 'high level abstract instructions' generated by the compiler, which is pretty stable, and - the 'internal form' meant for execution, generated by the loader, which 'has changed many times'. No *wonder* I failed to learn about the BEAM by reading the emulator code and trying to match what I saw there against the "instructions" I saw in the .S files. (c) The document begins in the middle. The beginning is the *architecture*. Given the architecture, a lot of the instructions become much easier to understand. (d) The document is very obviously a draft, so there's not much more it's fair to say at the moment. With there being two "BEAM"s, there need to be two manuals. "External BEAM", quite full, for people working on tools that generate, parse, or otherwise manipulate that kind of code. "Internal BEAM", taking an external BEAM background and architectural understanding for granted, that tells people working on the loader, emulator, HiPE, &c what they need to know. Since the Erlang/OTP team and HiPE teams have evidently managed without the "Internal BEAM" document, the "External BEAM" document is the priority. From ok@REDACTED Tue May 8 00:08:23 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 8 May 2012 10:08:23 +1200 Subject: [erlang-questions] Poor interface of string:to_integer/to_float In-Reply-To: <4726f3c5-7bed-4511-a349-985bec92d30a@knuth> References: <4726f3c5-7bed-4511-a349-985bec92d30a@knuth> Message-ID: <9A005A24-D4C1-4FF7-A7B6-1DBDD3E68064@cs.otago.ac.nz> On 8/05/2012, at 1:19 AM, Robert Virding wrote: > The interfaces to the functions string:to_integer/1 and string:to_float/1 is, to put it mildly, very poor. You get the impressions that someone needed special case functions which were then put into the libraries. Would it be possible to change them to something more useful? Shall I write an EEP? Weren't we told that the EEP process wasn't appropriate/needed for small library changes? For myself, I would like something that can read a number with underscore separators. Even Java has finally seen _that_ light! It may be hard to change the interface of existing functions, but a description of the replacements you would like to see would be interesting. From ok@REDACTED Tue May 8 00:25:50 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 8 May 2012 10:25:50 +1200 Subject: [erlang-questions] Which is best? string:concat or ++? In-Reply-To: References: Message-ID: <7826821D-FEEA-4D81-B3C3-6BCE4B5855B0@cs.otago.ac.nz> On 8/05/2012, at 1:31 AM, Paul Barry wrote: > Hi folks. > > This might be a case of a dumb question, but here goes anyway. :-) > > I have two strings, A and B. Is it better to use: > > string:concat(A, B) > > or > > A ++ B > > when combining them to create a new string? Erlang is open source. You can read the source code of library modules, and it's often instructive to do so. In string.erl you find concat(S1, S2) -> S1 ++ S2. There is probably no measurable difference. > I'm writing some code > that generates a chunk of HTML. I know that using ++ on big lists is > regarded as a "no-no", but is it acceptable here? I'm not aware of any guideline that says using ++ on big lists is a no-no. If for some reason you *need* the concatenation of two big lists as a list, ++ is the very best way to do it. The thing that people are warned against is that (((A ++ B) ++ C) ++ D) ++ E is more expensive than A ++ (B ++ (C ++ (D ++ E))) because the former copies A 4 times, B 3 times, C 2 times, while the latter copies them each only once. (This is true in Lisp, Prolog, Erlang, Haskell, Clean, SML, F#, ... or even in C if you do your own one-way linked lists there.) You may be asking the wrong question. The right question is probably "Is it a good idea to generate HTML as a string in any programming language?" to which the answer is "only in a language that does not let you build trees." One way to represent SGML data, including HTML, in Erlang looks like {Element_Name, [{Attribute_Name,Value}...], [Child...]} or <> or "text as a list" in which while generating the tree you never ever have to worry about escaping any data. You write a function that walks over a tree like this, perhaps sending it to a port, or perhaps creating an IO list, and *that* function does whatever escaping is necessary. {p,[],["This safe &so; is !"]} is perfectly safe, as long as your output function knows what it is doing. Also, if you know you are generating HTML, you could have your output function take care of omitting end tags for empty-by-definition elements. You wouldn't believe how easy it is to manipulate SGML data this way until you've tried it. (I _could_ have pointed to xmerl, but that's rather more complicated.) From ok@REDACTED Tue May 8 00:35:38 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 8 May 2012 10:35:38 +1200 Subject: [erlang-questions] Which is best? string:concat or ++? In-Reply-To: References: Message-ID: <2F136790-9DFE-4526-9E19-A7968F04B744@cs.otago.ac.nz> Lists can represent arbitrary Unicode codepoints as single elements. Concatenation A++B copies A but shares B. Suffixes of a list can be shared, taking prefixes requires copying. Lists require at least two full words of memory per codepoint. Binaries can represent bit-level data, or Latin 1 strings, or UTF-8 encoded Unicode. When used to represent Unicode, there is no one-to-one correspondence between characters and bytes, which you often don't need anyway. Concatenating A and B *may* have to copy *both* A and B, but Erlang can be astonishingly clever about binaries. Any slice of a binary can be shared (but will likely be copied if and only if it is *small*). Binaries require one byte of memory per byte (which means up to 3 bytes for a BMP character) plus some fixed overhead. Binaries are a closer analogue to say Java strings than lists are. Concatenation is a thing best avoided for all three (lists, binaries, Java strings). For example, instead of concatenating A++B++C, you might just form a list [A,B,C] --- this is the "iolist" approach that's been mentioned --- or indeed some other kind of tree, and turn it into a single sequence only when you really really need to. In practice, this is only when you cross an interface that demands a string of some other sort. I have done benchmarks in C, Lisp, Java, Smalltalk, and Prolog over the years, and the cost of using strings instead of trees is such as to drive all blood from the face. It is *scary* how bad strings of *any* kind can be, compared with using trees. It is also scary how *dangerous* strings can be, compared with using trees. From ok@REDACTED Tue May 8 00:45:55 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 8 May 2012 10:45:55 +1200 Subject: [erlang-questions] Which is best? string:concat or ++? In-Reply-To: References: <4FA7E13E.30000@ferd.ca> Message-ID: <5F549586-B82F-4B43-BF57-B9CD8C55BFFC@cs.otago.ac.nz> On 8/05/2012, at 3:39 AM, Paul Barry wrote: > > http://dev.af83.com/2012/01/16/erlang-iolist.html Under the heading IOList, the third paragraph gets list concatenation about as wrong as it possibly can. (The last sentence of that paragraph is right, though.) Let's consider a simple "nested list" representation of strings. nstring = integer | [] | [nstring|nstring] integer represents a codepoint [] represents an empty string [A|B] represents (but is not) the concatenation of A and B % Convert an nstring to a plain string. % Do not allocate any space that is not part of the % final result. reify_nstring(NS) -> reify_nstring(NS, []). reify_nstring([A|B], S) -> reify_nstring(A, reify_nstring(B, S)); reify_nstring([], S) -> S; reify_nstring(C, S) when is_integer(C), C >= 0 -> [C|S]. % Perform F(C) for each character C in nstring S. foreach_nstring(F, [A|B]) -> foreach_nstring(F, A), foreach_nstring(F, B); foreach_nstring(F, []) -> ok; foreach_nstring(F, C) when is_integer(C), C >= 0 -> F(C). % Report the effective length of an nstring in codepoints. length_nstring(S) -> length_nstring(S, 0). length_nstring([A|B], N) -> length_nstring(B, length_nstring(A, N)); length_nstring([], N) -> N; length_nstring(C, N) when is_integer(C), C >= 0 -> N + 1. You can generalise this to allow atoms and binaries as well as character codes. The basic idea is to *describe* concatenation without *doing* it, and to give you functions that act *as if* the concatenation had been done. From ok@REDACTED Tue May 8 03:07:13 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 8 May 2012 13:07:13 +1200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> Message-ID: On 8/05/2012, at 7:15 AM, Thomas Lindgren wrote: > There has been a substantial number of non-BEAM Erlang implementations already, so I'm > not convinced detailed BEAM docs is the key property* to spread Erlang. And how many of those non-BEAM implementations still exist? Does GERL? Is E2S still maintained? How much of OTP can it handle? > Indeed, requiring detailed docs of every change of BEAM seems likely to slow innovation down instead. I not only *don't* believe that, I *can't* believe that. Joe has informed us that there are TWO levels of BEAM, one of which has been very stable, and one of which has changed many times. I don't even believe your claim if made about the low level much changed "BEAM", but let's suppose it true for the sake of argument. If the high level of BEAM has remained pretty stable for quite a while, how would documenting it have slowed innovation down? I can *prove* that the absence of documentation has definitely slowed innovation down. There are several of my EEPS where I *would* have provided model implementations had I been able to. Heck, the frames proposal was in more or less its present shape *years* ago. It was only last month when it finally dawned on me that I could make measurements by constructing my *own* micro-BEAM which I *could* understand. > > If the motive is education, I think someone interested in compilers and virtual machine architectures > would have little trouble with BEAM as such. I have an interest in compilers and VMs. I worked professionally on Quintus Prolog and the real WAM (not the one in the papers or A?t-Kaci's book). And trying to figure out the BEAM was such a slog that to be honest, I said to myself "the hell with it, if they don't *WANT* me to understand the BEAM, I'm not going to waste any more of my time trying to penetrate the obscurity". There are three key software engineering lessons: - if it isn't tested, it doesn't work; - if it isn't documented, it doesn't work; - tests are one kind of documentation but not enough. For example, at Quintus, shortly after David Warren left, a compiler bug was reported. The whole compiler had two comments in it. One was a copyright notice, and the other was a commented out bit of code saying "this doesn't work". All variable names were either one letter or one letter and one digit. To fix the bug, I had to document the data structures used by the compiler, including running lots of tests through it to make sure my documentation was correct. Whenever I figured out what a variable meant, I gave it a longer name. Once I understood the data structures, I was able to revise them and make the compiler about 20% faster. That was good, because the bug turned out to be in the one part of the compiler I could never understand. Adding some extra code to check if the bug had happened and deoptimise in that case increased compile time by 10%, but overall the compiler was now faster. I am still quite angry that the information I needed to fix that bug properly was *KNOWN* to the author, who didn't bother writing it down. I didn't even need to know how the thing worked; what I needed to know was what exactly it was supposed to *do*. > In a real sense, BEAM is just a vehicle to express compiler optimizations for a > restricted part of ERTS (the sequential execution part, basically). No, compiler optimisations are expressed in the executable code of the compiler. BEAM lets you express the *results* of such optimisations, which is a different thing. It's just like the Quintus compiler: I could figure out in that case what the *results* were, but the actual process remained obscure. (More precisely, what the 'invariants' were.) The thing is, the compiler module in question was about 2 kSLOC, and it took me two full *weeks* to figure out what the author already *knew*. That was not a good use of my time. I've already spent about 4 full days equivalent trying to figure BEAM out, and with nobody paying me to do that, it's just not worth it, especially because somebody already KNOWS and just can't be bothered TELLING. Yes, I'm shouting. "We don't need it" and "you don't need it" are utterly different propositions, and too many people in too many areas of life fail to realise that. > Another argument might be that BEAM should be specified in detail in order to be a suitable binary format for distribution, > which is essentially what the JVM instruction set has become. I suggested many years ago that Erlang should take a leaf out of Kistler's book (or PhD thesis). The "Juice" system for Oberon compiled source files to abstract syntax trees, then cleverly compressed the ASTs and used them as the binary distribution form. They came in smaller than .class files and had no presuppositions about the target hardware (not even primitive size and alignment if I recall correctly). The cost of decompressing and generating native code was low, to the point where it was faster to dynamically load Juice files than their equivalent of .so/.dll files, and the generated code actually ran faster because the code generator knew more about the environment of the target, including existing code. (I don't know if the Juice runtime did cross-module inlining, but it would have been possible.) From bengt.kleberg@REDACTED Tue May 8 07:27:08 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 8 May 2012 07:27:08 +0200 Subject: [erlang-questions] beginner: Is there an alternative to net_adm:ping/1 before global:sync/0 In-Reply-To: References: <1336402150.4750.99.camel@seasc1137> <4FA81CB2.2040102@gmail.com> Message-ID: <1336454828.5135.8.camel@seasc1137> Greetings, Thanks for all the answers. My interest in replacing net_adm:ping/1 stems from the comment % Connect to resource manager node that accompanies it. If replaced by a connect_node/1 function call the comment can be removed. Also, perhaps somebody would have told me that while using global there is a better way. It is easy to keep doing the same things that worked last millennium, never switching to newer alternatives. bengt On Mon, 2012-05-07 at 22:56 +0200, Tim Watson wrote: > On 7 May 2012 20:04, Richard Carlsson wrote: > > On 2012-05-07 16:49, Bengt Kleberg wrote: > >> > >> Greetings, > >> > >> Before I do a global:sync/0 I have a net_adm:ping/1 to the node that has > >> the global process. Are there any alternatives to net_adm:ping/1? > > > > > > How about net_kernel:connect_node/1? > > > > What's wrong with net_adm:ping/1 just out of interest? Are you trying > to avoid the long delay before it times out if epmd is not running on > that node? If so you could try to gen_tcp connect to epmd on the right > IP+port to ensure that it's running before you attempt to you > net_kernel:connect_node/1. From michael.eugene.turner@REDACTED Tue May 8 07:35:01 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Tue, 8 May 2012 14:35:01 +0900 Subject: [erlang-questions] Fwd: Is there a good source for documentation on BEAM? In-Reply-To: <79520DF4-2E14-4CA4-8880-0B03E4D6B8DD@cs.otago.ac.nz> References: <79520DF4-2E14-4CA4-8880-0B03E4D6B8DD@cs.otago.ac.nz> Message-ID: "(b) We learn only at the end of those 5 pages that there are really TWO different things called BEAM: - the 'high level abstract instructions' generated by the compiler, which is pretty stable, and - the 'internal form' meant for execution, generated by the loader, which 'has changed many times'." Then there's the recommendation I read somewhere (I believe by a HiPE implementor) that it makes more sense to target Core Erlang. So, in a way, there are three levels you could target, two of them plausibly called "BEAM". I got interested in this issue when I noticed that ECLiPSe Prolog *used* to have working multicore parallelism, but then ... not. Wow: one of the better constraint programming packages, and it's not multicore. That seems like an opportunity going to waste. I also noticed that there's a Prolog implementation in Erlang. ECLiPSe provides a lot of power. It seems to be languishing anyway. ECLiPSe-over-BEAM might solve a lot of problems of viability for ECLiPSe, by reducing the workload on its maintainers while potentially increasing its user base. And I'm sure Ericsson could find uses for ECLiPSe in telecom network design and optimization, which (if there were ECLiPSe-over-BEAM) could further increase Erlang's value within the company. Just to give one possible example. -michael turner On Tue, May 8, 2012 at 6:57 AM, Richard O'Keefe wrote: > > On 7/05/2012, at 8:47 PM, Joe Armstrong wrote: > >> ---------- Forwarded message ---------- >> From: Joe Armstrong >> Date: Mon, May 7, 2012 at 10:46 AM >> Subject: Re: [erlang-questions] Is there a good source for >> documentation on BEAM? >> To: Jonathan Coveney >> >> >> Hi, >> >> I did start writing a description but it's not very complete. >> >> This is on my list of things-to-do-one-day-when-you-get-time >> >> ? See http://dl.dropbox.com/u/4764922/beam.pdf >> >> If there is any interest I could up the priority :-) >> >> /Joe > > I would have expected it to be a matter of personal and > professional pride that > One reason that several of my EEPs and so on never got model > implementations is that the BEAM instructions have less than > a 10th of the documentation that the WAM instructions had at > Quintus. ?It does seem clear that the Erlang/OTP team are > much better programmers than Quintus were, because whenever > we forgot to document some detail of an instruction, someone > was sure to get it wrong and new builds would start crashing > horribly. ?It's also a reason I've never contributed at a low > level to SWI Prolog. > > Two of the best documented abstract instruction sets I've seen > are Icon (there's a whole book about it) and Lua (there's a > document someone wrote with nearly a page per instruction; if > I could bring myself to _care_ about Lua I could start writing > extensions for it tomorrow). ?Since Hassan A?t-Kaci wrote his > little book about the WAM, that counts as well documented too, > except of course that I doubt anyone actually follows it all > that closely, Quintus certainly departed from it. ?(And yes of > course's there's the Java VM book too; I don't suppose it's a > co?ncidence that Tim Lindholm worked at Quintus and times on > the WAM.) > > Looking at that document of yours, Joe, > (a) Out of 8 pages, only 5 are actually about the BEAM. > (b) We learn only at the end of those 5 pages that there are > really TWO different things called BEAM: > ? ? ? ?- the 'high level abstract instructions' generated > ? ? ? ? ?by the compiler, which is pretty stable, and > ? ? ? ?- the 'internal form' meant for execution, generated > ? ? ? ? ?by the loader, which 'has changed many times'. > No *wonder* I failed to learn about the BEAM by reading the > emulator code and trying to match what I saw there against > the "instructions" I saw in the .S files. > (c) The document begins in the middle. ?The beginning is > the *architecture*. ?Given the architecture, a lot of the > instructions become much easier to understand. > (d) The document is very obviously a draft, so there's not > much more it's fair to say at the moment. > > With there being two "BEAM"s, there need to be two manuals. > "External BEAM", quite full, for people working on tools that > generate, parse, or otherwise manipulate that kind of code. > "Internal BEAM", taking an external BEAM background and > architectural understanding for granted, that tells people > working on the loader, emulator, HiPE, &c what they need to > know. > > Since the Erlang/OTP team and HiPE teams have evidently > managed without the "Internal BEAM" document, the > "External BEAM" document is the priority. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Tue May 8 08:05:12 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 8 May 2012 18:05:12 +1200 Subject: [erlang-questions] Fwd: Is there a good source for documentation on BEAM? In-Reply-To: References: <79520DF4-2E14-4CA4-8880-0B03E4D6B8DD@cs.otago.ac.nz> Message-ID: On 8/05/2012, at 5:35 PM, Michael Turner wrote: > "(b) We learn only at the end of those 5 pages that there are > really TWO different things called BEAM: > - the 'high level abstract instructions' generated > by the compiler, which is pretty stable, and > - the 'internal form' meant for execution, generated > by the loader, which 'has changed many times'." > > Then there's the recommendation I read somewhere (I believe by a HiPE > implementor) that it makes more sense to target Core Erlang. So, in a > way, there are three levels you could target, two of them plausibly > called "BEAM". If you want to compile an extended syntax into something that is already supported in Core Erlang, no worries. For example, implementing list comprehensions with out-of-line code, as currently done, could be done that way. If you want to compile list comprehensions into inline code, I believe it can be done using high-level BEAM, but you can't do it through Core Erlang. If you want to compile something like frames, you need to - extend HiPE if you want the new feature to go fast *and* - extend low level BEAM *and* - extend high level BEAM *and* - extend Core Erlang *and* - extend source Erlang (probably the easiest step) *and* - extend the AST form (not too hard) *and* - extend the tools that process the AST form *and* - last but very much not least, you need to extend the documentation. The more levels you have the harder it gets, unless the design is very modular. Here I will say that Quintus never really got this as organised as they should have. Adding a new instruction required editing upwards of a dozen files. The nearest I've seen to a well-structured emulator was the Icon one, where the various C files that were used were generated from an annotated master. From huangjl@REDACTED Tue May 8 08:09:02 2012 From: huangjl@REDACTED (huangjl) Date: Tue, 8 May 2012 14:09:02 +0800 Subject: [erlang-questions] =?utf-8?b?562U5aSNOiDnrZTlpI06ICBDb25uZWN0IHRv?= =?utf-8?q?_Node_with_multiple_IPs?= In-Reply-To: References: <002201cd2bff$12bacfa0$38306ee0$@com> <5A8DF162-B90D-455E-B5D9-8433C8BE15DE@gmail.com> <002f01cd2c1e$20d8c2d0$628a4870$@com> Message-ID: <003a01cd2ce1$113c2330$33b46990$@com> I don?t think so, the proxy mode will increase the net delay or system load. ??? ????????? ???: Zheng Zhibin [mailto:witeman.g@REDACTED] ????: 2012?5?7? 15:02 ???: huangjl ??: ??: Re: ??: [erlang-questions] Connect to Node with multiple IPs one domain + reverse proxy, such as nginx? ???? iPhone ? 2012-5-7???2:53?"huangjl" ??? But without DNS, it seems I have to config two domains for two IPs? My purpose is detect which IP is suitable for MachineA to connect to MachineB, but I don?t want to start two nodes for MachineB, is there any way to do this? ???: Zheng Zhibin [mailto:witeman.g@REDACTED] ????: 2012?5?7? 14:08 ???: huangjl ??: ??: Re: [erlang-questions] Connect to Node with multiple IPs use domain instead of ip, then config your hosts table ???? iPhone ? 2012-5-7???11:11?"huangjl" ??? Hi, All I have a question about erlang node connection. There are two machines: MachineA and MachineB, MachineB has two IPs: IP1 and IP2. MachineA want to connect to MachineB using IP, just like net_kernel:connect_node(?test@REDACTED?) or net_kernel:connect_node(?test@REDACTED?). Then , how to start only one test node at MachineB which can receive all messages from test@REDACTED and test@REDACTED _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From madtrick@REDACTED Tue May 8 09:06:36 2012 From: madtrick@REDACTED (Farruco Sanjurjo) Date: Tue, 8 May 2012 09:06:36 +0200 Subject: [erlang-questions] Badarg error Message-ID: Hi, I'm getting a badarg error in this code from_binary(Data = <>, Acc) -> PayloadBytes= case PayloadLen of 126 -> <> = Trailing, 2 + ExtPayloadLen; 127 -> <> = Trailing, 8 + ExtPayloadLen; _ -> %<> PayloadLen end, FrameSize = 16 + (PayloadBytes * 8), <> = Data, from_binary(Rest, [decode_frame(<>) | Acc]); The error points to the line " FrameSize = 16 + (PayloadBytes * 8)". I've been struggling for a while but I can't figure out what's wrong. Any suggestion? -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Tue May 8 09:11:47 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 8 May 2012 08:11:47 +0100 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: Message-ID: <669B4163-8EED-4B9C-A606-86E4DD55E86D@gmail.com> Yes that's a good idea and similar in many ways to what tuncer suggested so +1 on both counts. Nonetheless I am of the opinion that binary packages are simpler for the publication tool author as they push the build responsibility to the library author To configure their project and travis hooks accordingly. Hackage is a good example architecture in many ways though. On 7 May 2012, at 14:31, Tristan Sloughter wrote: > > 7) Binary or source packages? > > > > - Ummm - tricky > > > > I think binary is better but adds a lot of complexity. Erts versions, projects containing native code, etc. makin this the publishers responsibility means the tool only has to focus on the packaging problem not the build space. Third party signing allows you to support sources which gaven't been published by the originator. > > I'd be happy with starting with just a source solution, but I think a binary one is definitely needed and can be simplified with TravisCI -- it'll build with multiple Erlang versions. > > I really wish I could plugin to the hosted TravisCI to have a job run to publish artifacts. I only briefly looked into that, so maybe something there is possible. > > Tristan -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Tue May 8 09:39:45 2012 From: vances@REDACTED (Vance Shipley) Date: Tue, 8 May 2012 13:09:45 +0530 Subject: [erlang-questions] Observer Message-ID: <20120508073943.GD798@aluminum.wavenet.lk> The new observer application is a nice improvement over the old toolbar apps. It uses wxWidgets instead of gs and looks good on my MacBook Pro. In R15B01 load charts were added as well. It would be really nice if there was a "Print" option to export a screen to a graphic file. I'd like to capture the supervision hierarchy in the Applications window for example. Does anyone know of a way to coax a graphic out of wx somehow? While I'm at it I will say that it would also be nice to extend it to add application versions and even environment variables and values. -- -Vance From Tobias.Schlager@REDACTED Tue May 8 09:43:16 2012 From: Tobias.Schlager@REDACTED (Tobias Schlager) Date: Tue, 8 May 2012 07:43:16 +0000 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: <4FA822F3.70807@gmail.com> References: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> <4FA1CEC8.1060807@gmail.com> <4FA7CC6A.2000004@gmail.com> , <4FA822F3.70807@gmail.com> Message-ID: <12F2115FD1CCEE4294943B2608A18FA335AA25F8@MAIL01.win.lbaum.eu> I know that this is not a very popular opinion but to me it sounds like what you're looking for is pretty close to what maven delivers: * binary-based packaging * well-defined dependency management (handling version conflicts) * (open source) repository managers (e.g. sonatype nexus) * group id concept (could be used to avoid clashes when forking github projects) * support for continuous integration (e.g. jenkins) * a manifest file (pom.xml) * built-in support for documentation and reporting * support for code/package signing Of course there are certain drawbacks like: * not written in erlang (its written in java) * requires a certain amount of (static) xml configuration * poor support for building native code * slower than other tools Currently, everyone is hot to re-implement all this in erlang and that's perfectly ok, I just wanted to throw in another point of view. Regards Tobias Von: erlang-questions-bounces@REDACTED [erlang-questions-bounces@REDACTED]" im Auftrag von "Torben Hoffmann [torben.lehoff@REDACTED] Gesendet: Montag, 7. Mai 2012 21:30 An: Tristan Sloughter Cc: erlang-questions@REDACTED Betreff: Re: [erlang-questions] Building, Packaging and Installing On 07/05/2012 15:27, Tristan Sloughter wrote: One major difference is that Colombo tries to adhere to the version numbering scheme of OTP and avoids using the source area for generating releases. It means that after building in ./gproc the compiled version will be installed in ./lib/gproc-0.4.3 This allows a more strict separation of managing dependencies and the actual building of the app you are working on. I tried to get rebar to do the same, but it was too painful for me to get that to work, so I decided to do an experiment of my own to see if I could create something that could help me out. Just wanted to say this was an issue that has been recently debated regarding source area for generating releases, http://blog.erlware.org/2012/05/04/sinan-releases-and-being-right/ And that Colombo is sounding definitely like I had already planned on building around agner, so I think I'm going to start out trying to use Colombo for my needs. Tristan Just report any issues you have with Colombo - it is alpha software, so I do not expect it to be in any way perfect. I am very open to put in new functionality or split Colombo into two tools if that seems better. It might be that my needs are a corner case and the more generally applicable solution is different - in fact it will be the case, so I am ready to get feedback and find ways to improve it. Cheers, Torben -- http://www.linkedin.com/in/torbenhoffmann From carlsson.richard@REDACTED Tue May 8 09:44:20 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 08 May 2012 09:44:20 +0200 Subject: [erlang-questions] Which is best? string:concat or ++? In-Reply-To: <7826821D-FEEA-4D81-B3C3-6BCE4B5855B0@cs.otago.ac.nz> References: <7826821D-FEEA-4D81-B3C3-6BCE4B5855B0@cs.otago.ac.nz> Message-ID: <4FA8CED4.1060208@gmail.com> On 05/08/2012 12:25 AM, Richard O'Keefe wrote: > One way to represent SGML data, including HTML, in Erlang > looks like > {Element_Name, [{Attribute_Name,Value}...], [Child...]} > or <> > or "text as a list" > > in which while generating the tree you never ever have to worry > about escaping any data. You write a function that walks over > a tree like this, perhaps sending it to a port, or perhaps > creating an IO list, and *that* function does whatever escaping > is necessary. > > {p,[],["This safe&so; is!"]} > > is perfectly safe, as long as your output function knows what it > is doing. Also, if you know you are generating HTML, you could > have your output function take care of omitting end tags for > empty-by-definition elements. > > You wouldn't believe how easy it is to manipulate SGML data this > way until you've tried it. > > (I _could_ have pointed to xmerl, but that's rather more complicated.) What you describe is what is called "simple form" in xmerl: http://www.erlang.org/doc/man/xmerl.html#export_simple-3 http://www.erlang.org/doc/apps/xmerl/xmerl_ug.html#id56386 Using the xmerl_html callback module: 1> io:put_chars(xmerl:export_simple([{p,[],["This safe&so; is!"]}], xmerl_html)).

This<is> safe&so; is<this>!

and the xmerl_xml callback module: 2> io:put_chars(xmerl:export_simple([{p,[],["This safe&so; is!"]}], xmerl_xml)).

This<is> safe&so; is<this>!

/Richard Carlsson From watson.timothy@REDACTED Tue May 8 09:49:52 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 8 May 2012 08:49:52 +0100 Subject: [erlang-questions] Observer In-Reply-To: <20120508073943.GD798@aluminum.wavenet.lk> References: <20120508073943.GD798@aluminum.wavenet.lk> Message-ID: <4B3C45E2-41BA-455B-963C-09CC1435BE21@gmail.com> Are you on snow leopard or lion? I've been struggling to get wx to play nice on the former. On 8 May 2012, at 08:39, Vance Shipley wrote: > The new observer application is a nice improvement over the old > toolbar apps. It uses wxWidgets instead of gs and looks good on > my MacBook Pro. In R15B01 load charts were added as well. > > It would be really nice if there was a "Print" option to export a > screen to a graphic file. I'd like to capture the supervision > hierarchy in the Applications window for example. Does anyone know > of a way to coax a graphic out of wx somehow? > > While I'm at it I will say that it would also be nice to extend it > to add application versions and even environment variables and values. > > -- > -Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bob@REDACTED Tue May 8 09:51:17 2012 From: bob@REDACTED (Bob Ippolito) Date: Tue, 8 May 2012 00:51:17 -0700 Subject: [erlang-questions] Badarg error In-Reply-To: References: Message-ID: In the second to last line you are declaring Frame as an integer and in the last line you are using it as a binary. I don't see anything obviously wrong other than that, but harder to read if on a phone :) On Tuesday, May 8, 2012, Farruco Sanjurjo wrote: > Hi, > > I'm getting a badarg error in this code > > from_binary(Data = <>, Acc) -> > PayloadBytes= case PayloadLen of > 126 -> > <> = Trailing, > 2 + ExtPayloadLen; > 127 -> > <> = Trailing, > 8 + ExtPayloadLen; > _ -> > %<> > PayloadLen > end, > FrameSize = 16 + (PayloadBytes * 8), > <> = Data, > from_binary(Rest, [decode_frame(<>) | Acc]); > > The error points to the line " FrameSize = 16 + (PayloadBytes * 8)". > > I've been struggling for a while but I can't figure out what's wrong. Any > suggestion? > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Tue May 8 10:11:56 2012 From: dangud@REDACTED (Dan Gudmundsson) Date: Tue, 8 May 2012 10:11:56 +0200 Subject: [erlang-questions] Observer In-Reply-To: <4B3C45E2-41BA-455B-963C-09CC1435BE21@gmail.com> References: <20120508073943.GD798@aluminum.wavenet.lk> <4B3C45E2-41BA-455B-963C-09CC1435BE21@gmail.com> Message-ID: On Tue, May 8, 2012 at 9:49 AM, Tim Watson wrote: > Are you on snow leopard or lion? I've been struggling to get wx to play nice on the former. Snow leopard should work, it comes with a wxWidgets pre-installed. Some bugs are in pre-installed version, something with the file selector, but most of Observer should work, it is what I have been test with. > > On 8 May 2012, at 08:39, Vance Shipley wrote: > >> The new observer application is a nice improvement over the old >> toolbar apps. ?It uses wxWidgets instead of gs and looks good on >> my MacBook Pro. ?In R15B01 load charts were added as well. Thanks >> It would be really nice if there was a "Print" option to export a >> screen to a graphic file. ?I'd like to capture the supervision >> hierarchy in the Applications window for example. ?Does anyone know >> of a way to coax a graphic out of wx somehow? Printing in wxWidgets, can be easy when it works out of box, and very hard when it doesn't. Do printing work on the mac in wx:demo()? >> While I'm at it I will say that it would also be nice to extend it >> to add application versions and even environment variables and values. Good calls, will see what I can do, as always gui stuff have the least priority here, but I'm poking around when I have time, right now anyway. Patches are always welcome :-) /Dan >> -- >> ? ?-Vance >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vances@REDACTED Tue May 8 10:38:30 2012 From: vances@REDACTED (Vance Shipley) Date: Tue, 8 May 2012 14:08:30 +0530 Subject: [erlang-questions] Observer In-Reply-To: <4B3C45E2-41BA-455B-963C-09CC1435BE21@gmail.com> References: <20120508073943.GD798@aluminum.wavenet.lk> <4B3C45E2-41BA-455B-963C-09CC1435BE21@gmail.com> Message-ID: <20120508083829.GF798@aluminum.wavenet.lk> I recently upgraded to Lion but it was wx was working fine on Snow Leopard for me as well. I use the MacPorts package (wxWidgets @2.8.12). On Tue, May 08, 2012 at 08:49:52AM +0100, Tim Watson wrote: } Are you on snow leopard or lion? I've been struggling to get wx to play } nice on the former. -- -Vance From erlang@REDACTED Tue May 8 10:40:48 2012 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 8 May 2012 10:40:48 +0200 Subject: [erlang-questions] Which is best? string:concat or ++? In-Reply-To: References: Message-ID: Here a dumb answer. "You might not need to concatenate them" - if all you're going to do is output the result later then you don't need to concatenate them. For example X = A ++ B, file:write_file("foo", X) and X = [A,B], %% <--- this thing is called an iolist file:write_file("foo", X) will result in identical content in the file "foo". In this case (where you're just going send the concatenated data to an I/O stream) you don't need to bother concatenating the data. If you are building complex iolists (which is very common) and you need to flatten it, then a good strategy is to build a single iolist (X) and then call erlang:iolist_to_binary(X). Using '++' when you don't need to at all is a common programming error :-) In the cases where you do need it probably both the A and B in A++B should be small. If A is a long string then you should look for a different algorithm. Cheers /Joe On Mon, May 7, 2012 at 3:31 PM, Paul Barry wrote: > Hi folks. > > This might be a case of a dumb question, but here goes anyway. ?:-) > > I have two strings, A and B. ?Is it better to use: > > ? string:concat(A, B) > > or > > ? ?A ++ B > > when combining them to create a new string? ?I'm writing some code > that generates a chunk of HTML. ?I know that using ++ on big lists is > regarded as a "no-no", but is it acceptable here? > > Thanks. > > Paul. > > -- > Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul.barry@REDACTED > Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From watson.timothy@REDACTED Tue May 8 11:49:19 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 8 May 2012 10:49:19 +0100 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: <12F2115FD1CCEE4294943B2608A18FA335AA25F8@MAIL01.win.lbaum.eu> References: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> <4FA1CEC8.1060807@gmail.com> <4FA7CC6A.2000004@gmail.com> <4FA822F3.70807@gmail.com> <12F2115FD1CCEE4294943B2608A18FA335AA25F8@MAIL01.win.lbaum.eu> Message-ID: On 8 May 2012 08:43, Tobias Schlager wrote: > I know that this is not a very popular opinion but to me it sounds like what you're looking for is pretty close to what maven delivers: We used maven and the maven-erlang-plugin for a long time in my previous job - it worked very well. > * binary-based packaging I think for a package or dependency manager, this is a given - otherwise you're reinventing too many other things at once. Unlike maven, I think what we want here is either a package manager or (simpler) a dependency manager - not another build tool. > * well-defined dependency management (handling version conflicts) > * (open source) repository managers (e.g. sonatype nexus) Eric Merrit and I have tabled a design that uses git, so you can host your personal/organisational repository on github and consumers simply add your repository to their local index. > * group id concept (could be used to avoid clashes when forking github projects) Yes we very much need this to support multiple forks of the same application name. > * support for continuous integration (e.g. jenkins) To my mind, this is a corollary theme and not central to this discussion, but yes, clearly people want this. > * a manifest file (pom.xml) Sure, and you and I both know that almost nobody on this list is going to settle for xml. There is a maven extension that supports defining the pom using scala/grooxy instead, but again, I doubt it'll get much love here. > * built-in support for documentation and reporting No - this is the build tool's responsibility and AFAICT that is *not* what we're talking about here. > * support for code/package signing > Yes very much so, especially to get around a problem that maven doesn't actually solve particularly well. When Project-X is available in source only form and not built of packaged in a way that the tool understands, what do we do??? In the maven world, you build it locally and do `mvn deploy:file ...` to put it into your local repository and perhaps push it into a corporate/group nexus instance. But there is *no indication of the original of this file* when you do that, so trust is difficult to establish. What I'd like to see is support for a kind of 'third party signing' where you build + package the missing app yourself, sign it with your own private key and publish it into your own repository, which gets synchronised back onto github. Eventually someone else *might* decide they want to use it, and if they cannot persuade the original author to publish his own binary artefact repository (for whatever reason) then they may choose to either package and sign it themselves, OR to utilize the version that you have packaged and signed, because the know and trust you. I would, for example, consider using a published artefact that has been signed by basho or esl, because I trust them not to publish garbage or malicious code, given that doing so would implicitly damage their corporate reputation. > Of course there are certain drawbacks like: > * not written in erlang (its written in java) I suspect this is why it'll not see much traction in the community at large. > * requires a certain amount of (static) xml configuration As I mentioned earlier, there are alternatives. > * poor support for building native code Actually the C/C++ build plugins for maven are pretty good, but I'm not sure how capable the maven-erlang-plugin is at doing this. Perhaps the author could chime in and let us know? > * slower than other tools > It's about the same speed as most of the make based projects I've seen. Rebar is MUCH faster though! > Currently, everyone is hot to re-implement all this in erlang and that's perfectly ok, I just wanted to throw in another point of view. > Another thing I'd like to point at at this juncture is that there is a difference between a package manager and a dependency manager. A very important point here is that a package manager will allow you to install AND uninstall software, whereas a dependency manager will fetch artefacts and solve dependency graphs in order for you to utilise them, but nothing more. The other thing I want to point out is that with binary artefacts, you need to be able to install them once and use them across multiple (compatible) erts versions, which is something Eric and I have discussed at some length on the erlware-questions mailing list. From rickard@REDACTED Tue May 8 14:00:57 2012 From: rickard@REDACTED (Rickard Green) Date: Tue, 8 May 2012 14:00:57 +0200 Subject: [erlang-questions] About Erlang SMP scheduler In-Reply-To: <0773DEE9-E2B7-440A-814E-33C55A472A7C@gmail.com> References: <0F4D9489-7755-4FD2-8831-E4BEE2A8C8F1@gmail.com> <2FF9F129-C96C-48C2-BFE9-223F591D624B@gmail.com> <0773DEE9-E2B7-440A-814E-33C55A472A7C@gmail.com> Message-ID: The work stealing is there to quickly distribute work between schedulers, but it doesn't balance the load. check_balance() sets up migration paths and migration limits that are used in order to balance the load between schedulers. Without balancing, processes with the same assigned priority can get very different effective priorities. Example: With 4 schedulers and 400 cpu bound processes executing the same code, each scheduler will eventually end up with 100 processes to manage. That is, all processes will have the same access to the cpu. If you disable check_balance(), you may end up with three schedulers only having one process each, and one scheduler with 397 processes. These 397 processes will effectively have a much lower priority than the other three processes. Regards, Rickard Green, Erlang/OTP, Ericsson AB 2012/5/2 Siyao Zheng(???) : > Hi, > > Yes, that's true. But the "try_steal_task()" afterwards is also controlled > by ERTS_SMP macro. It doesn't make any sense to talk about workload balance > outside multi-processor environment. I'm just wondering why "check_balance" > is needed while there exists work stealing(try_steal_task). > > Cheers > > On May 2, 2012, at 12:04 PM, xu yu wrote: > > Hi, > > "check_balance" controlled by "ifdef ERTS_SMP", so... > > > 2012/4/27 "Siyao Zheng(???)" >> >> Hi, >> >> schedule(), in which check_balance() is only called. >> >> In schedule(), when check_balance_reds reaches zero, scheduler decides to >> "check_balance". But later in schedule(), when scheduler finds run queue >> being empty, it will try to steal task from other schedulers (by calling >> try_steal_task()), which is also a load balance mechanism. I'm just >> wondering what the purpose of check_balance(). >> >> Cheers >> Siyao >> >> On Apr 27, 2012, at 9:40 PM, Zabrane Mickael wrote: >> >> > Hi Siyao, >> > >> > Which "check_balance" you've commented out? >> > >> > Here is what I found so far: >> > >> > $ find /otp_src_R15B01 -type f | xargs grep check_balance | grep -v >> > matches >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: int >> > forced_check_balance; >> > >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c:check_balance(ErtsRunQueue >> > *c_rq) >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c:# error >> > check_balance() assumes ERTS_MAX_PROCESS < (1 << 27) >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >> > c_rq->check_balance_reds = INT_MAX; >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >> > c_rq->check_balance_reds = INT_MAX; >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >> > rq->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: * >> > check_balance() is never called in more threads >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: forced = >> > balance_info.forced_check_balance; >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >> > balance_info.forced_check_balance = 0; >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >> > c_rq->check_balance_reds = INT_MAX; >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >> > rq->check_balance_reds = INT_MAX; >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >> > rq->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >> > rq->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >> > balance_info.forced_check_balance = 0; >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >> > (RQ)->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; \ >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >> > balance_info.forced_check_balance = 1; >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >> > ERTS_RUNQ_IX(0)->check_balance_reds = 0; >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: if >> > (rq->check_balance_reds <= 0) >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >> > check_balance(rq); >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.h: int >> > check_balance_reds; >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.h: >> > (RQ)->check_balance_reds -= (REDS); \ >> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.h: >> > (RQ)->check_balance_reds -= (REDS); >> > [...] >> > >> > For OTP Team: is this dangerous ? >> > >> > Regards, >> > Zabrane >> > >> > >> > On Apr 27, 2012, at 1:23 PM, Siyao Zheng(???) wrote: >> > >> >> Hi, >> >> >> >> In SMP version of Erlang, every scheduler periodically call >> >> "check_balance()" to check load balance among all schedulers, then it might >> >> migrate some processes to balance load, or shut down some schedulers with >> >> low load. Does anyone know why scheduler should do this? check_balance() is >> >> quite a big one, and it has to lock every run queue when it checks the run >> >> queue. I think it's quite a big cost here. The work stealing at each >> >> schedule step afterwards actually does work load balance very well. Actually >> >> after I comment out the check_balance() step and run BigBang and Hackbench >> >> benchmarks, they are really faster! >> >> >> >> So, I wonder what is the purpose of check_balance() step here, if it is >> >> related to scalability on many core processors or something, and if there is >> >> any other benchmark I can run to prove its usefulness. >> >> >> >> Cheers! >> >> >> >> Siyao >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From simon@REDACTED Tue May 8 16:30:11 2012 From: simon@REDACTED (Simon MacMullen) Date: Tue, 08 May 2012 15:30:11 +0100 Subject: [erlang-questions] 1PB of arg_reg - wait what? Message-ID: <4FA92DF3.7090000@rabbitmq.com> We've just had a report of a Rabbit broker dying trying to allocate approximately 1PB of memory: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2012-May/019896.html Slogan: std_alloc: Cannot allocate 1124304355803072 bytes of memory (of type "arg_reg"). Which is unfortunate enough in itself - but arg_reg? My understanding of beam internals is not great, but that looks to be something like a stack frame. A 1PB stack frame? As far as I can see "arg_reg" memory is only allocated in one place, around line 3100 of beam_emu.c: https://github.com/erlang/otp/blob/maint/erts/emulator/beam/beam_emu.c#L3157 ...where size is set to c_p->arity * sizeof(c_p->arg_reg[0]). process.arg_reg is an Eterm*, so sizeof(Eterm) *cannot* be huge, so it must be the arity which is ~10^15. We have some excessive arities in RabbitMQ, but not quite to that extent. Does anyone have any idea how this could arise, or can it just be chalked up to cosmic rays? Cheers, Simon -- Simon MacMullen RabbitMQ, VMware From janburse@REDACTED Tue May 8 16:37:49 2012 From: janburse@REDACTED (Jan Burse) Date: Tue, 08 May 2012 16:37:49 +0200 Subject: [erlang-questions] Nice T-shirt; Hewitt expert episode on actors. Message-ID: <4FA92FBD.3070503@fastmail.fm> http://channel9.msdn.com/Shows/Going+Deep/Hewitt-Meijer-and-Szyperski-The-Actor-Model-everything-you-wanted-to-know-but-were-afraid-to-ask We'll begin with these Expert to Expert episodes with a "standing" conversation (participants stand comfortably close to the whiteboard) with computer scientists Carl Hewitt, Visiting Professor at Stanford University, creator of the Planner programming language. From madtrick@REDACTED Tue May 8 16:43:42 2012 From: madtrick@REDACTED (Farruco Sanjurjo) Date: Tue, 8 May 2012 16:43:42 +0200 Subject: [erlang-questions] Badarg error In-Reply-To: References: Message-ID: Oh, I see. Thanks for pointing that : ) On 8 May 2012 09:51, Bob Ippolito wrote: > In the second to last line you are declaring Frame as an integer and in > the last line you are using it as a binary. I don't see anything obviously > wrong other than that, but harder to read if on a phone :) > > On Tuesday, May 8, 2012, Farruco Sanjurjo wrote: > >> Hi, >> >> I'm getting a badarg error in this code >> >> from_binary(Data = <>, Acc) -> >> PayloadBytes= case PayloadLen of >> 126 -> >> <> = Trailing, >> 2 + ExtPayloadLen; >> 127 -> >> <> = Trailing, >> 8 + ExtPayloadLen; >> _ -> >> %<> >> PayloadLen >> end, >> FrameSize = 16 + (PayloadBytes * 8), >> <> = Data, >> from_binary(Rest, [decode_frame(<>) | Acc]); >> >> The error points to the line " FrameSize = 16 + (PayloadBytes * 8)". >> >> I've been struggling for a while but I can't figure out what's wrong. Any >> suggestion? >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Tue May 8 17:28:41 2012 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 8 May 2012 17:28:41 +0200 Subject: [erlang-questions] =?iso-8859-1?q?Erlounge_=D6rebro=2C_9_May?= In-Reply-To: <792A6200-BAF1-4820-8B98-7D1BB3AC316D@feuerlabs.com> References: <792A6200-BAF1-4820-8B98-7D1BB3AC316D@feuerlabs.com> Message-ID: Update for tomorrow's Erlounge ?rebro: We will be at the Bishops Arms [1] at around 19.00, if anyone wants to join us. ;-) BR, Ulf W [1] http://www.bishopsarms.com/Orebro/Presentation On 20 Apr 2012, at 19:07, Ulf Wiger wrote: > The event you've all been waiting for! > > On May 9, we arrange the first ever Erlounge ?rebro. > > (For those of you who don't know where ?rebro is, here's some help: > http://g.co/maps/g748z) > > We haven't decided on exact place yet, but 18.00 seems like a good time > to shoot for. Tony Rogvall has promised to join us, so it will be a truly > cosmopolitan event. :) > > Let me know if you want to join us. > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From lukas@REDACTED Tue May 8 18:35:56 2012 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 8 May 2012 18:35:56 +0200 Subject: [erlang-questions] binary_to_integer/1 and binary_to_float/1 and friends In-Reply-To: References: Message-ID: Hi, There is no need to write an EEP for this and we would appreciate a patch in this area. For consistency the solution should also have functions called binary_to_integer/2, integer_to_binary/1,2 and float_to_binary/1 which behave the same was as their list counterparts. Lukas On Mon, May 7, 2012 at 4:01 PM, Jos? Valim wrote: > Can someone from the OTP team let me know if I need an EEP for this or if I > should just send patches straight to erlang-patches??Thanks in advance. > > > On Mon, Apr 30, 2012 at 7:47 PM, Jos? Valim wrote: >> >> At the beginning of this year, there was a discussion about how to >> implement binary_to_integer/1 in Erlang. >> There is a possible implementation for such function which is quite >> trivial: >> >> ? ? list_to_integer(binary_to_list(binary)) >> >> However, it has the trade-off that it can't be used in guards (unless >> defined in a macro) and there is an unnecessary conversion cost to list >> (although the cost is small since both binary and list are likely small). >> >> With Riak, Cowboy and other tools pushing towards using binaries instead >> of lists for strings, the need for such functions will become more and more >> common. >> Is there any chance we will see this as built-in functions and therefore >> available in guards in R16? >> >> Thanks, >> >> Jos? Valim >> www.plataformatec.com.br >> Founder and Lead Developer >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ericbmerritt@REDACTED Tue May 8 18:37:38 2012 From: ericbmerritt@REDACTED (Eric Merritt) Date: Tue, 8 May 2012 11:37:38 -0500 Subject: [erlang-questions] How to integrate common_test into a continuous integration system? Message-ID: Guys, I am working on R14B03 (hopefully that will change soon) and trying to transition to Common Test. One issue that I have run into is that ct_run does not return a non-zero exit status on failure. That makes it pretty hard to trivially integrate into a continuous integration environment. I can create a wrapper around ct that would do what I need. However, I am pretty sure I am not the only one that has tried to integrate common test into make/ci. So, I would love to hear how you guys go about it. Thanks, Eric From ericbmerritt@REDACTED Tue May 8 18:57:32 2012 From: ericbmerritt@REDACTED (Eric Merritt) Date: Tue, 8 May 2012 11:57:32 -0500 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> <4FA1CEC8.1060807@gmail.com> <4FA7CC6A.2000004@gmail.com> <4FA822F3.70807@gmail.com> <12F2115FD1CCEE4294943B2608A18FA335AA25F8@MAIL01.win.lbaum.eu> Message-ID: Sorry guys , I am late to the party. On Tue, May 8, 2012 at 4:49 AM, Tim Watson wrote: > On 8 May 2012 08:43, Tobias Schlager wrote: >> I know that this is not a very popular opinion but to me it sounds like what you're looking for is pretty close to what maven delivers: > > We used maven and the maven-erlang-plugin for a long time in my > previous job - it worked very well. The problem is always adding the (rather large) non-erlang dependency. Even if the barrier is small in reality, the mental/social barrier is huge. > >> * binary-based packaging > > I think for a package or dependency manager, this is a given - > otherwise you're reinventing too many other things at once. Unlike > maven, I think what we want here is either a package manager or > (simpler) a dependency manager - not another build tool. > >> * well-defined dependency management (handling version conflicts) >> * (open source) repository managers (e.g. sonatype nexus) > > Eric Merrit and I have tabled a design that uses git, so you can host > your personal/organisational repository on github and consumers simply > add your repository to their local index. The goal is to make it trivial for anyone to host their own repo, and let tools that are good at that handle it. We may reinvent some of the wheel but we can make that is minimal as possible. > >> * group id concept (could be used to avoid clashes when forking github projects) > > Yes we very much need this to support multiple forks of the same > application name. an unfortunate reality in the modern open source world. >> * support for continuous integration (e.g. jenkins) > > To my mind, this is a corollary theme and not central to this > discussion, but yes, clearly people want this. if you get building correct, you should have CI correct as well. > >> * a manifest file (pom.xml) > > Sure, and you and I both know that almost nobody on this list is going > to settle for xml. There is a maven extension that supports defining > the pom using scala/grooxy instead, but again, I doubt it'll get much > love here. > >> * built-in support for documentation and reporting > > No - this is the build tool's responsibility and AFAICT that is *not* > what we're talking about here. agreed. > >> * support for code/package signing >> > > Yes very much so, especially to get around a problem that maven > doesn't actually solve particularly well. When Project-X is available > in source only form and not built of packaged in a way that the tool > understands, what do we do??? > > In the maven world, you build it locally and do `mvn deploy:file ...` > to put it into your local repository and perhaps push it into a > corporate/group nexus instance. But there is *no indication of the > original of this file* when you do that, so trust is difficult to > establish. > > What I'd like to see is support for a kind of 'third party signing' > where you build + package the missing app yourself, sign it with your > own private key and publish it into your own repository, which gets > synchronised back onto github. Eventually someone else *might* decide > they want to use it, and if they cannot persuade the original author > to publish his own binary artefact repository (for whatever reason) > then they may choose to either package and sign it themselves, OR to > utilize the version that you have packaged and signed, because the > know and trust you. > > I would, for example, consider using a published artefact that has > been signed by basho or esl, because I trust them not to publish > garbage or malicious code, given that doing so would implicitly damage > their corporate reputation. > >> Of course there are certain drawbacks like: >> * not written in erlang (its written in java) > > I suspect this is why it'll not see much traction in the community at large. > >> * requires a certain amount of (static) xml configuration > > As I mentioned earlier, there are alternatives. > >> * poor support for building native code > > Actually the C/C++ build plugins for maven are pretty good, but I'm > not sure how capable the maven-erlang-plugin is at doing this. Perhaps > the author could chime in and let us know? > >> * slower than other tools >> > > It's about the same speed as most of the make based projects I've > seen. Rebar is MUCH faster though! > >> Currently, everyone is hot to re-implement all this in erlang and that's perfectly ok, I just wanted to throw in another point of view. >> > > Another thing I'd like to point at at this juncture is that there is a > difference between a package manager and a dependency manager. A very > important point here is that a package manager will allow you to > install AND uninstall software, whereas a dependency manager will > fetch artefacts and solve dependency graphs in order for you to > utilise them, but nothing more. > > The other thing I want to point out is that with binary artefacts, you > need to be able to install them once and use them across multiple > (compatible) erts versions, which is something Eric and I have > discussed at some length on the erlware-questions mailing list. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From arjan@REDACTED Tue May 8 20:49:18 2012 From: arjan@REDACTED (Arjan Hakkesteegt) Date: Tue, 08 May 2012 20:49:18 +0200 Subject: [erlang-questions] Multiple embedded Yaws instances Message-ID: <4FA96AAE.50907@hakwerk.com> Hi, I am trying to setup a Common Test mock layer around my SUT (System Under Test). Both the SUT and CT should start an embedded Yaws instance because both should both act as client and as server for SOAP. Client(SUT) -> Server(CT) Server(SUT) <- Client(CT) However, it appears that it is not possible to run more than 1 embedded Yaws within a VM? When starting the second instance I get already_started errors from the supervisor. I tried to modify the ChildSpecs returned by yaws_api:embedded_start_conf/4 to append the Yaws instance Id to the supervisor child Ids, but still get the already_started errors. I tried it like this: [{yaws_log_myid, {yaws_log,start_link,[]}, permanent,5000,worker, [yaws_log]}, {yaws_server_myid, ...}] I've used multiple SConfs within a single embedded Yaws instance before, but I don't think that this is an option here as my SUT and the CT suite are completely separate applications and the SUT should be seen as a blackbox. I guess it could be an option to start the SUT in its own Erlang VM, but how would I start and stop this VM from the Common Test one (from init_per_suite/end_per_suite)? Any solutions to this problem? Or am I missing something obvious? Thanks, Arjan From jcoveney@REDACTED Tue May 8 20:52:19 2012 From: jcoveney@REDACTED (Jonathan Coveney) Date: Tue, 8 May 2012 11:52:19 -0700 Subject: [erlang-questions] Fwd: Is there a good source for documentation on BEAM? In-Reply-To: References: <79520DF4-2E14-4CA4-8880-0B03E4D6B8DD@cs.otago.ac.nz> Message-ID: I think that any time you find yourself justifying not documenting things, it's probably a mistake :) I know it'd be a lot of someone's time (I read the pdf in dropbox and found it very helpful, and think that a document like that that covered more of the opcodes in the context of more complicated functions would be extremely helpful). I agree that the External BEAM is probably the thing to focus on. I think that more documentation around this would be good in many respects... 1. Documenting these sorts of things frames understanding for others, which can lead to more eyes on the implementation, which is always good 2. It'll make it easier for people to write custom VM's, which is only a good thing for Erlang. For the JVM, for example, this is a huge benefit and as Java the language dies, the ecosystem of budding JVM languages will no doubt go strong 3. It'll make it easier for people to compile things to BEAM, which only goes to show that Erlang's underbelly is general purpose and useful for building robust, fault-tolerant software I realize that it'd be a lot of someone's time, but I want to say that it'd probably be a really good endeavor for the community. I think there are a lot of smart people that aren't full time VM or language architects who still can probably do interesting things if they have some guidance on what is going on with BEAM. Jon 2012/5/7 Richard O'Keefe > > On 8/05/2012, at 5:35 PM, Michael Turner wrote: > > > "(b) We learn only at the end of those 5 pages that there are > > really TWO different things called BEAM: > > - the 'high level abstract instructions' generated > > by the compiler, which is pretty stable, and > > - the 'internal form' meant for execution, generated > > by the loader, which 'has changed many times'." > > > > Then there's the recommendation I read somewhere (I believe by a HiPE > > implementor) that it makes more sense to target Core Erlang. So, in a > > way, there are three levels you could target, two of them plausibly > > called "BEAM". > > If you want to compile an extended syntax into something that is > already supported in Core Erlang, no worries. For example, > implementing list comprehensions with out-of-line code, as > currently done, could be done that way. > > If you want to compile list comprehensions into inline code, > I believe it can be done using high-level BEAM, but you can't > do it through Core Erlang. > > If you want to compile something like frames, you need to > - extend HiPE if you want the new feature to go fast *and* > - extend low level BEAM *and* > - extend high level BEAM *and* > - extend Core Erlang *and* > - extend source Erlang (probably the easiest step) *and* > - extend the AST form (not too hard) *and* > - extend the tools that process the AST form *and* > - last but very much not least, you need to > extend the documentation. > > The more levels you have the harder it gets, unless the design > is very modular. Here I will say that Quintus never really > got this as organised as they should have. Adding a new > instruction required editing upwards of a dozen files. > > The nearest I've seen to a well-structured emulator was the > Icon one, where the various C files that were used were generated > from an annotated master. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gianfranco.alongi@REDACTED Tue May 8 20:53:34 2012 From: gianfranco.alongi@REDACTED (Gianfranco Alongi) Date: Tue, 8 May 2012 20:53:34 +0200 Subject: [erlang-questions] How to integrate common_test into a continuous integration system? In-Reply-To: References: Message-ID: Maybe not what you want exactly, but this question was brought up once on Stackoverflow http://stackoverflow.com/questions/6215166/continuous-integration-server-for-erlang-code I think the easiest way if you won't use Lukas wrapper https://github.com/garazdawi/cth_tools/blob/master/src/cth_junit.erl is to roll your own. /G On Tue, May 8, 2012 at 6:37 PM, Eric Merritt wrote: > Guys, > > ?I am working on R14B03 (hopefully that will change soon) and trying > to transition to Common Test. One issue that I have run into is that > ct_run does not return a non-zero exit status on failure. That makes > it pretty hard to trivially integrate into a continuous integration > environment. I can create a wrapper around ct that would do what I > need. However, I am pretty sure I am not the only one that has tried > to integrate common test into make/ci. So, I would love to hear how > you guys go about it. > > Thanks, > Eric > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ericbmerritt@REDACTED Tue May 8 21:07:24 2012 From: ericbmerritt@REDACTED (Eric Merritt) Date: Tue, 8 May 2012 14:07:24 -0500 Subject: [erlang-questions] How to integrate common_test into a continuous integration system? In-Reply-To: References: Message-ID: Gianfranco, Thanks for the response. I was literally getting ready to write something very similar to cth_tools myself. Now I can take that and add to it instead. That's very nice. Eric On Tue, May 8, 2012 at 1:53 PM, Gianfranco Alongi wrote: > Maybe not what you want exactly, but this question was brought up once > on Stackoverflow > > http://stackoverflow.com/questions/6215166/continuous-integration-server-for-erlang-code > > I think the easiest way if you won't use Lukas wrapper > https://github.com/garazdawi/cth_tools/blob/master/src/cth_junit.erl > is to roll your own. > > /G > > On Tue, May 8, 2012 at 6:37 PM, Eric Merritt wrote: >> Guys, >> >> ?I am working on R14B03 (hopefully that will change soon) and trying >> to transition to Common Test. One issue that I have run into is that >> ct_run does not return a non-zero exit status on failure. That makes >> it pretty hard to trivially integrate into a continuous integration >> environment. I can create a wrapper around ct that would do what I >> need. However, I am pretty sure I am not the only one that has tried >> to integrate common test into make/ci. So, I would love to hear how >> you guys go about it. >> >> Thanks, >> Eric >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Tue May 8 21:31:24 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 8 May 2012 23:31:24 +0400 Subject: [erlang-questions] 1PB of arg_reg - wait what? In-Reply-To: <4FA92DF3.7090000@rabbitmq.com> References: <4FA92DF3.7090000@rabbitmq.com> Message-ID: 99% that some error in process with many terms leads to error_logger eating all possible CPU and memory. From august.schwartzwald@REDACTED Tue May 8 21:46:25 2012 From: august.schwartzwald@REDACTED (August Schwartzwald) Date: Tue, 08 May 2012 21:46:25 +0200 Subject: [erlang-questions] Fwd: Is there a good source for documentation on BEAM? In-Reply-To: References: Message-ID: I've also been looking for information about how Erlang works internally for some time now and I found this document very helpful. I'm looking forward to the complete version whenever it is done. If you want comments/feedback on the document I can gladly help with that too. On Mon, 07 May 2012 10:47:26 +0200, Joe Armstrong wrote: > ---------- Forwarded message ---------- > From: Joe Armstrong > Date: Mon, May 7, 2012 at 10:46 AM > Subject: Re: [erlang-questions] Is there a good source for > documentation on BEAM? > To: Jonathan Coveney > > > Hi, > > I did start writing a description but it's not very complete. > > This is on my list of things-to-do-one-day-when-you-get-time > > See http://dl.dropbox.com/u/4764922/beam.pdf > > If there is any interest I could up the priority :-) > > /Joe > > > On Mon, May 7, 2012 at 8:39 AM, Jonathan Coveney > wrote: >> This question seems to come up now and again, and it's surprising to me >> that >> a crucial part of the documentation isn't better documented. Is there a >> reason that it is the case? Is the reason that there is no VM spec to >> give >> the devs the flexibility to change the intermediate layer without >> having to >> worry about backwards compatibility to the degree that Java does? >> >> Thus far I've found a description of the opcodes: >> http://azunyanmoe.wordpress.com/2011/03/30/erlang-vm-opcodes/ >> >> and this resource on the file format: >> http://www.erlang.se/~bjorn/beam_file_format.html >> >> But there doesn't seem to be a lot of high level talk about what the >> opcodes >> do (a la the JVM specification, for example). I know it's not >> impossible, >> and could always ask the guys at Erjang how they went about it, but >> thought >> I'd ask here. >> >> Please forgive a newbie question, and thanks in advance >> Jon >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From fritchie@REDACTED Wed May 9 01:46:37 2012 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Tue, 08 May 2012 18:46:37 -0500 Subject: [erlang-questions] Proposed expansion of Erlang-triggerable DTrace probes Message-ID: <82161.1336520797@snookles.snookles.com> Hi, all. I've sent a proposal for expanding the number of Erlang-triggerable DTrace probes to the erlang-patches mailing list. If you haven't seen that message but have an opinion, please read it over and email me (or the erlang-patches list) with your comments. http://erlang.org/pipermail/erlang-patches/2012-May/002781.html -Scott From ok@REDACTED Wed May 9 01:58:44 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 9 May 2012 11:58:44 +1200 Subject: [erlang-questions] Fwd: Is there a good source for documentation on BEAM? In-Reply-To: References: <79520DF4-2E14-4CA4-8880-0B03E4D6B8DD@cs.otago.ac.nz> Message-ID: <4CEA1F2B-BA47-4E61-AF44-541573228096@cs.otago.ac.nz> Let me illustrate the Icon approach by showing you a fragment of the micro-BEAM I wrote to get the performance numbers in the frames proposal. (The whole thing is fragmentary.) ... @i max src, snd, dst @d dst := max(src, snd) This computes the maximum using the micro-Erlang term ordering. If src and snd are tagged immediate integers the comparison is done inline; the compare() function is called otherwise. @c T = @src; U = @snd; @dst = cmp(T, >, U) ? T : U; @step; @e ... @i check_record src, size, const @d Type test. Fail unless src is tagged as a pointer to a tuple or frame, the first word it points to is size, and the second is the const (which must be an atom, but we don't check that). Used for record matching. @c T = @src; if (!is_tuple(T)) @fail "is_record"; else if (FIELD(T, TUP_TAG, 0) != @size) @fail "is_record"; else if (FIELD(T, TUP_TAG, 1) != @const) @fail "is_record"; else @step; @e ... There is a preprocessor written in AWK that turns this into several C files. One of them is the emulator cases. For the check_record instruction you get #line 75 "frame.master" case CHECK_RECORD: #line 76 "frame.master" T = reg[(int)P[1]]; #line 77 "frame.master" if (!is_tuple(T)) P = failure, operation = "is_record"; else #line 78 "frame.master" if (FIELD(T, TUP_TAG, 0) != 4) P = failure, operation = "is_record"; else #line 79 "frame.master" if (FIELD(T, TUP_TAG, 1) != P[3]) P = failure, operation = "is_record"; else #line 80 "frame.master" P += 4; break; where I've broken the long lines (the preprocessor doesn't). The #line directives are option. @i introduces an instruction; the next line is a template for it saying what the operands are. @d introduces the description for people. @c introduces the code. In it, various built-in @macros are expanded. One advantage of doing it this way is that by using @step to update the PC I *cannot* get the offset wrong; the preprocessor counted the operands and their sizes for me. Similarly, what I write has *no* operand numbers; the preprocessor counted those, and supplies all necessary casts as well. I can shuffle operands around (in @i) without revising the code (in @c), and have. It wouldn't be too hard to write another preprocessor that built some kind of documentation (HTML would probably be easiest) out of this, but since this was an experiment, it didn't seem worth while. Why did I write the preprocessor? Well, to be honest, the first draft didn't use one. I got a bit sick of debugging, and wrote the preprocessor (based on vague memories of Icon) to eliminate a class of errors. It turned out to be _easier_ to develop a documented emulator than an undocumented one. From robert.virding@REDACTED Wed May 9 02:04:05 2012 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 09 May 2012 01:04:05 +0100 (BST) Subject: [erlang-questions] Erlounge in Houston? In-Reply-To: <7fa4790d-d69f-4122-b73f-aac7b5640feb@knuth> Message-ID: No takers for an Erlounge in Houston next week, or the week after? Or both weeks for that matter? Ah well, not much to do about it. I drink alone (in the words of the bard) :-) Robert ----- Original Message ----- > Hi, I will be in Houston, Texas 13-26 May giving a course. Are there > any erlangers in or around Houston who would like to have an > erlounge while I am there. It is a bit of a short notice for a > Factory Lite I'm afraid. > > Also what can you recommend I like go see as a tourist while there? > > Robert > > -- > Robert Virding, Erlang Solutions Ltd. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From jws@REDACTED Wed May 9 02:21:08 2012 From: jws@REDACTED (Jeff Schultz) Date: Wed, 9 May 2012 10:21:08 +1000 Subject: [erlang-questions] Fwd: Is there a good source for documentation on BEAM? In-Reply-To: References: <79520DF4-2E14-4CA4-8880-0B03E4D6B8DD@cs.otago.ac.nz> Message-ID: <20120509002108.GA3417@mulga.csse.unimelb.edu.au> On Tue, May 08, 2012 at 11:52:19AM -0700, Jonathan Coveney wrote: [plain, good sense elided] > I think that more documentation around this would be good in many > respects... > 1. Documenting these sorts of things frames understanding for others, which > can lead to more eyes on the implementation, which is always good > 2. It'll make it easier for people to write custom VM's, which is only a > good thing for Erlang. For the JVM, for example, this is a huge benefit and > as Java the language dies, the ecosystem of budding JVM languages will no > doubt go strong > 3. It'll make it easier for people to compile things to BEAM, which only > goes to show that Erlang's underbelly is general purpose and useful for > building robust, fault-tolerant software And to add one more: 4. Documenting, and publishing, your internals makes them patent-system visible prior art. This protects you from some Johnny-come-lately company, or individual, who patents something you've been using for many years. Source code, even publicly available source code like Erlang, is not generally valid prior art in patent terms. Just because you've been doing, say, memory management, one particular way for a decade, won't block a patent covering exactly that. And yes, it does happen. Jeff Schultz From ericbmerritt@REDACTED Wed May 9 03:45:06 2012 From: ericbmerritt@REDACTED (Eric Merritt) Date: Tue, 8 May 2012 20:45:06 -0500 Subject: [erlang-questions] How to integrate common_test into a continuous integration system? In-Reply-To: References: Message-ID: For future posterity. This is how I solved this problem. ct_run -dir tests ... | awk "/FAILED/{exit 1;}" This should work in most cases. Just a note. The documentation for ct:run_test is wrong at least for R14B04. It *always* returns ok regardless of what the result of the tests are. Thank, Eric On Tue, May 8, 2012 at 2:07 PM, Eric Merritt wrote: > Gianfranco, > > Thanks for the response. I was literally getting ready to write > something very similar to cth_tools myself. Now I can take that and > add to it instead. That's very nice. > > Eric > > On Tue, May 8, 2012 at 1:53 PM, Gianfranco Alongi > wrote: >> Maybe not what you want exactly, but this question was brought up once >> on Stackoverflow >> >> http://stackoverflow.com/questions/6215166/continuous-integration-server-for-erlang-code >> >> I think the easiest way if you won't use Lukas wrapper >> https://github.com/garazdawi/cth_tools/blob/master/src/cth_junit.erl >> is to roll your own. >> >> /G >> >> On Tue, May 8, 2012 at 6:37 PM, Eric Merritt wrote: >>> Guys, >>> >>> ?I am working on R14B03 (hopefully that will change soon) and trying >>> to transition to Common Test. One issue that I have run into is that >>> ct_run does not return a non-zero exit status on failure. That makes >>> it pretty hard to trivially integrate into a continuous integration >>> environment. I can create a wrapper around ct that would do what I >>> need. However, I am pretty sure I am not the only one that has tried >>> to integrate common test into make/ci. So, I would love to hear how >>> you guys go about it. >>> >>> Thanks, >>> Eric >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions From dean_yanqing@REDACTED Wed May 9 04:50:17 2012 From: dean_yanqing@REDACTED (Dean Song) Date: Wed, 9 May 2012 10:50:17 +0800 (CST) Subject: [erlang-questions] Multiple embedded Yaws instances In-Reply-To: <4FA96AAE.50907@hakwerk.com> References: <4FA96AAE.50907@hakwerk.com> Message-ID: <1336531817.36498.YahooMailNeo@web15203.mail.cnb.yahoo.com> Hi Arijan ? ? If want to start two Yaws, it could work start them in?separate?VM rather than one VM ( one in dog@REDACTED, the other in cat@REDACTED). I guess it could be an option to start the SUT in its own Erlang VM, but how would I start and stop this VM from the Common Test one (from init_per_suite/end_per_suite)?? ? You can start SUT through os:cmd function, and I suggest you start SUT outside Common Test and make it running,at least before CT?execute TC.? Stat/stop SUT frequently useless. ________________________________ From: Arjan Hakkesteegt To: erlang-questions@REDACTED Sent: Wednesday, May 9, 2012 2:49 AM Subject: [erlang-questions] Multiple embedded Yaws instances Hi, I am trying to setup a Common Test mock layer around my SUT (System Under Test). Both the SUT and CT should start an embedded Yaws instance because both should both act as client and as server for SOAP. Client(SUT) -> Server(CT) Server(SUT) <- Client(CT) However, it appears that it is not possible to run more than 1 embedded Yaws within a VM? When starting the second instance I get already_started errors from the supervisor. I tried to modify the ChildSpecs returned by yaws_api:embedded_start_conf/4 to append the Yaws instance Id to the supervisor child Ids, but still get the already_started errors. I tried it like this: ? [{yaws_log_myid, ? ? {yaws_log,start_link,[]}, ? ? permanent,5000,worker, ? ? [yaws_log]}, ? ? {yaws_server_myid, ? ? ...}] I've used multiple SConfs within a single embedded Yaws instance before, but I don't think that this is an option here as my SUT and the CT suite are completely separate applications and the SUT should be seen as a blackbox. I guess it could be an option to start the SUT in its own Erlang VM, but how would I start and stop this VM from the Common Test one (from init_per_suite/end_per_suite)? Any solutions to this problem? Or am I missing something obvious? Thanks, Arjan _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From Tobias.Schlager@REDACTED Wed May 9 10:20:17 2012 From: Tobias.Schlager@REDACTED (Tobias Schlager) Date: Wed, 9 May 2012 08:20:17 +0000 Subject: [erlang-questions] Building, Packaging and Installing In-Reply-To: References: <128F7B67-3477-40C3-A85A-67A906C21E83@gmail.com> <4FA1CEC8.1060807@gmail.com> <4FA7CC6A.2000004@gmail.com> <4FA822F3.70807@gmail.com> <12F2115FD1CCEE4294943B2608A18FA335AA25F8@MAIL01.win.lbaum.eu> , Message-ID: <12F2115FD1CCEE4294943B2608A18FA335AA2725@MAIL01.win.lbaum.eu> Of course I know that Maven will never get a wide acceptance in the Erlang community and furthermore I think Erlang deserves its own tool chain (build, dependency/package management tooling) written in Erlang. With my previous post I just wanted to point out what the current tooling can offer *today* (although I could have left out the build-tool-only aspects) and of course future tooling will have to compete against that to convince the community. We didn't rewrite the maven-erlang-plugin because we don't like Rebar. Rebar is a great build tool that supports everything you can compile in the Erlang world. We just found (as the previous discussion also revealed) that its dependency mechanism based on links to SCM repos and pulling the sources wasn't sufficient for us. I think that build/dependency tooling can't really be kept separate from package management. The build and dependency tooling must make use of packages installed in local or remote repositories and also has to make available built artefacts to these repositories. So there must be some kind of support at least in the build tool. I think a symbiosis or combination of Rebar and Sinan (if Sinan is the tool that will evolve in the direction Tim and Eric worked out) could really become the standard Erlang tool chain in this area and I would be the last one refusing to use it. Now some words to Tim's question regarding native code support: The maven-erlang-plugin doesn't deal with native code itself. As Tim pointed out there already are Maven plugins handling native code (afaik pretty complicated ). The user is free to use them. Either in the same project putting the native output manually in a directory of the application or in a separate project specifying a dependency to it in the Erlang project. In the latter case the maven-erlang-plugin would put all non-Erlang dependencies into the target application's priv directory automatically. Alternatively the OS mechanism could be used to install native dependencies on a (target) system, an Erlang application would just presume that the needed artefacts are available. Another solution would be a well-defined, unified production and build environment. In this case native code could be built everywhere and native artefacts could be put under version control. Regards Tobias ________________________________________ Von: Eric Merritt [ericbmerritt@REDACTED] Gesendet: Dienstag, 8. Mai 2012 18:57 An: Tim Watson Cc: Tobias Schlager; erlang-questions@REDACTED Betreff: Re: [erlang-questions] Building, Packaging and Installing Sorry guys , I am late to the party. On Tue, May 8, 2012 at 4:49 AM, Tim Watson wrote: > On 8 May 2012 08:43, Tobias Schlager wrote: >> I know that this is not a very popular opinion but to me it sounds like what you're looking for is pretty close to what maven delivers: > > We used maven and the maven-erlang-plugin for a long time in my > previous job - it worked very well. The problem is always adding the (rather large) non-erlang dependency. Even if the barrier is small in reality, the mental/social barrier is huge. > >> * binary-based packaging > > I think for a package or dependency manager, this is a given - > otherwise you're reinventing too many other things at once. Unlike > maven, I think what we want here is either a package manager or > (simpler) a dependency manager - not another build tool. > >> * well-defined dependency management (handling version conflicts) >> * (open source) repository managers (e.g. sonatype nexus) > > Eric Merrit and I have tabled a design that uses git, so you can host > your personal/organisational repository on github and consumers simply > add your repository to their local index. The goal is to make it trivial for anyone to host their own repo, and let tools that are good at that handle it. We may reinvent some of the wheel but we can make that is minimal as possible. > >> * group id concept (could be used to avoid clashes when forking github projects) > > Yes we very much need this to support multiple forks of the same > application name. an unfortunate reality in the modern open source world. >> * support for continuous integration (e.g. jenkins) > > To my mind, this is a corollary theme and not central to this > discussion, but yes, clearly people want this. if you get building correct, you should have CI correct as well. > >> * a manifest file (pom.xml) > > Sure, and you and I both know that almost nobody on this list is going > to settle for xml. There is a maven extension that supports defining > the pom using scala/grooxy instead, but again, I doubt it'll get much > love here. > >> * built-in support for documentation and reporting > > No - this is the build tool's responsibility and AFAICT that is *not* > what we're talking about here. agreed. > >> * support for code/package signing >> > > Yes very much so, especially to get around a problem that maven > doesn't actually solve particularly well. When Project-X is available > in source only form and not built of packaged in a way that the tool > understands, what do we do??? > > In the maven world, you build it locally and do `mvn deploy:file ...` > to put it into your local repository and perhaps push it into a > corporate/group nexus instance. But there is *no indication of the > original of this file* when you do that, so trust is difficult to > establish. > > What I'd like to see is support for a kind of 'third party signing' > where you build + package the missing app yourself, sign it with your > own private key and publish it into your own repository, which gets > synchronised back onto github. Eventually someone else *might* decide > they want to use it, and if they cannot persuade the original author > to publish his own binary artefact repository (for whatever reason) > then they may choose to either package and sign it themselves, OR to > utilize the version that you have packaged and signed, because the > know and trust you. > > I would, for example, consider using a published artefact that has > been signed by basho or esl, because I trust them not to publish > garbage or malicious code, given that doing so would implicitly damage > their corporate reputation. > >> Of course there are certain drawbacks like: >> * not written in erlang (its written in java) > > I suspect this is why it'll not see much traction in the community at large. > >> * requires a certain amount of (static) xml configuration > > As I mentioned earlier, there are alternatives. > >> * poor support for building native code > > Actually the C/C++ build plugins for maven are pretty good, but I'm > not sure how capable the maven-erlang-plugin is at doing this. Perhaps > the author could chime in and let us know? > >> * slower than other tools >> > > It's about the same speed as most of the make based projects I've > seen. Rebar is MUCH faster though! > >> Currently, everyone is hot to re-implement all this in erlang and that's perfectly ok, I just wanted to throw in another point of view. >> > > Another thing I'd like to point at at this juncture is that there is a > difference between a package manager and a dependency manager. A very > important point here is that a package manager will allow you to > install AND uninstall software, whereas a dependency manager will > fetch artefacts and solve dependency graphs in order for you to > utilise them, but nothing more. > > The other thing I want to point out is that with binary artefacts, you > need to be able to install them once and use them across multiple > (compatible) erts versions, which is something Eric and I have > discussed at some length on the erlware-questions mailing list. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From andy.richards.iit@REDACTED Wed May 9 10:58:09 2012 From: andy.richards.iit@REDACTED (Andy Richards) Date: Wed, 9 May 2012 09:58:09 +0100 Subject: [erlang-questions] Decoupling supervisor:start_child from a web front end Message-ID: <12323ED0-BDF5-41FE-AC47-81E3846212D3@googlemail.com> Hi all, What the most appropriate way of calling supervisor:start_child on a remote node when the supervisor module does not exist locally? I have a scenario where I have a webmachine web front end which needs to spawn a child process via a supervisor on a backend middleware node. My supervisor exists on my target middleware node so I could call from my web tier either... supervisor:start_child({Name, Node}, [list_of_terms]) ...however this duplicates the code in my own target supervisor module start_child function and feels as though I'm coupling my middleware to my front end should my supervisor start_ child function ever change? or I could call... rpc:call(Node, Module, Function, [Args]) ...but this feels a bit clunky and I'm unsure if there is additional overhead with rpc:call compared to the above approach? What's the best approach ? Many thanks, Andy. From andy.richards.iit@REDACTED Wed May 9 11:05:56 2012 From: andy.richards.iit@REDACTED (Andy Richards) Date: Wed, 9 May 2012 10:05:56 +0100 Subject: [erlang-questions] msg mailbox and gen_svr shutdown Message-ID: Hi, I can't seem to see any confirmation in the documentation so was wondering if anyone could confirm if messages are still sent to a supervised gen_svr following a shutdown message? If so how do I cleanly shutdown my gen_svr without loosing messages? I read in the supervisor child spec that a shutdown can be set to infinity which i hoped would allow me to process the msg's in my mailbox but if I do this will my module continue to receive messages from other processes? Is my approach flawed and if so what other ways are there to cleanly shutting down my gen_svr without loosing messages? Many thanks, Andy. From sam@REDACTED Wed May 9 15:44:01 2012 From: sam@REDACTED (Samuel Elliott) Date: Wed, 9 May 2012 14:44:01 +0100 Subject: [erlang-questions] Decoupling supervisor:start_child from a web front end In-Reply-To: <12323ED0-BDF5-41FE-AC47-81E3846212D3@googlemail.com> References: <12323ED0-BDF5-41FE-AC47-81E3846212D3@googlemail.com> Message-ID: So the supervisor callback module already has a method where you can get it to start its children? Add another method that allows you to specify the node/supervisor it should start it off (rather than just the local one). The other option is to register the supervisor globally, and then make your callback module also have a function to start it wherever it is - the advantage of this is for testing you should be able to pull everything onto the same node, to save a complex test harness launch procedure. However, if you have lots of these supervisors, global registration isn't an option. Sam On Wed, May 9, 2012 at 9:58 AM, Andy Richards wrote: > Hi all, > > What the most appropriate way of calling supervisor:start_child on a remote node when the supervisor module does not exist locally? > > I have a scenario where I have a webmachine web front end which needs to spawn a child process via a supervisor on a backend middleware node. My supervisor exists on my target middleware node so I could call from my web tier either... > > supervisor:start_child({Name, Node}, [list_of_terms]) > > ...however this duplicates the code in my own target supervisor module start_child function and feels as though I'm coupling my middleware to my front end should my supervisor start_ child function ever change? or I could call... > > rpc:call(Node, Module, Function, [Args]) > > ...but this feels a bit clunky and I'm unsure if there is additional overhead with rpc:call compared to the above approach? > > What's the best approach ? > > Many thanks, > > Andy. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Samuel Elliott sam@REDACTED http://lenary.co.uk/ +44 (0)7891 993 664 From arnaud74130@REDACTED Wed May 9 15:53:50 2012 From: arnaud74130@REDACTED (Arnaud Garcia) Date: Wed, 9 May 2012 15:53:50 +0200 Subject: [erlang-questions] FSM: How to wait the end of handle_info ? Message-ID: Hello everybody, I have two files, main.erl which control a finite state machine (FSM) with a gen_fsm behavior. The FSM is also able to received incoming data from a socket for certain state (using the handle_info method). The question is how the main program can wait and know the next state after handle_info callback is done by the gen_fsm.... because, according to the rawdata received the system move to another state and main.erl must know this new state to send specifics events.... Maybe more easy to understand with a small illustration main.erl {ok, FSMPid} = myfsm:start_link(...), ok = gen_fsm:sync_send_event(FSMPid, event1) % => will move the FSM to state_5 where we are waiting socket incoming data % these data will be handle by "handle_info({tcp, _Socket, RawData}, state_5, StateData)" % depending on the received socket rawdata, we will change the state of the FSM => How can I wait handle_info has finished ? => How can I get the state ? (for this last question, I think I can just send a send_all_state(FSMPid, current_state) for example and return the StateName program will continue like this: case StateName of state_13 -> send another event .... thanks for help, Arnaud -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Wed May 9 16:12:09 2012 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 09 May 2012 15:12:09 +0100 (BST) Subject: [erlang-questions] FSM: How to wait the end of handle_info ? In-Reply-To: Message-ID: <89313815-f421-4b6a-a965-76803369c79c@knuth> You can use gen_fsm:reply/2 to explicitly send the reply back to the caller of gen_fsm:sync_send_event/2. But note then the State/3 functions must not return a reply value which the behaviour automatically send back to the client, it must just return {next_state,...}. But then the gen_fsm:sync_send_event call will wait! Robert ----- Original Message ----- > Hello everybody, > I have two files, main.erl which control a finite state machine (FSM) > with a gen_fsm behavior. > The FSM is also able to received incoming data from a socket for > certain state (using the handle_info method). > The question is how the main program can wait and know the next state > after handle_info callback is done by the gen_fsm.... > because, according to the rawdata received the system move to another > state and main.erl must know this new state to send specifics > events.... > Maybe more easy to understand with a small illustration > main.erl > {ok, FSMPid} = myfsm:start_link(...), > ok = gen_fsm:sync_send_event(FSMPid, event1) > % => will move the FSM to state_5 where we are waiting socket > incoming data > % these data will be handle by "handle_info({tcp, _Socket, RawData}, > state_5, StateData)" > % depending on the received socket rawdata, we will change the state > of the FSM > => How can I wait handle_info has finished ? > => How can I get the state ? > (for this last question, I think I can just send a > send_all_state(FSMPid, current_state) for example and return the > StateName > program will continue like this: > case StateName of > state_13 -> send another event > .... > thanks for help, > Arnaud > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Wed May 9 16:24:38 2012 From: dmkolesnikov@REDACTED (dmitry kolesnikov) Date: Wed, 9 May 2012 17:24:38 +0300 Subject: [erlang-questions] FSM: How to wait the end of handle_info ? In-Reply-To: References: Message-ID: <-5795479398603410632@unknownmsgid> Hello, You can send message out of handle_info to parent process when state is changed. Parent process shall have a receive handle. You can use gen_event to dispatch notification about a changed state. IMHO, there is something wrong with design if you need to synchronize main routine with async message from socket. It should be asynchronous... Best Regards, Dmitry >-|-|-*> On 9.5.2012, at 16.53, Arnaud Garcia wrote: > Hello everybody, > > I have two files, main.erl which control a finite state machine (FSM) with a gen_fsm behavior. > The FSM is also able to received incoming data from a socket for certain state (using the handle_info method). > > The question is how the main program can wait and know the next state after handle_info callback is done by the gen_fsm.... > because, according to the rawdata received the system move to another state and main.erl must know this new state to send specifics events.... > > Maybe more easy to understand with a small illustration > > main.erl > {ok, FSMPid} = myfsm:start_link(...), > ok = gen_fsm:sync_send_event(FSMPid, event1) > % => will move the FSM to state_5 where we are waiting socket incoming data > % these data will be handle by "handle_info({tcp, _Socket, RawData}, state_5, StateData)" > % depending on the received socket rawdata, we will change the state of the FSM > > > => How can I wait handle_info has finished ? > => How can I get the state ? > (for this last question, I think I can just send a send_all_state(FSMPid, current_state) for example and return the StateName > > program will continue like this: > case StateName of > state_13 -> send another event > .... > > > > > > thanks for help, > > Arnaud > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From hvjunk@REDACTED Wed May 9 17:26:28 2012 From: hvjunk@REDACTED (Hendrik Visage) Date: Wed, 9 May 2012 17:26:28 +0200 Subject: [erlang-questions] trapexit.org In-Reply-To: <-542295322088726156@unknownmsgid> References: <-542295322088726156@unknownmsgid> Message-ID: And they seems to be used for spam advertising too ;( On Tue, May 1, 2012 at 1:39 PM, Andrew Thompson wrote: > Sorry all I know this isn't the right forum but ... does anyone know how to > let these guys know they have a problem??Been like this for a couple of > days. > > phpBB :?Critical Error > Could not connect to the database > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From seglingskungen@REDACTED Wed May 9 18:36:57 2012 From: seglingskungen@REDACTED (Seglings Kungen) Date: Wed, 9 May 2012 18:36:57 +0200 Subject: [erlang-questions] Elang compiler options from within a module? Message-ID: Hi All, I have a simple module where i define a compile directive: -module(test). -compile(debug_info). -export([test/0]). test()-> ok. I compile it with erlc and load it >test:module_info(compile). [{options,[{outdir,"/Users/chris"}]}, {version,"4.8.1"}, {time,{2012,5,9,16,30,44}}, {source,"/Users/chris/test.erl"}] And we can see it has not been debug_compiled......... Where is it defined what compiler options can be used within a a module and what can be don with erlc? Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomasl_erlang@REDACTED Wed May 9 20:03:27 2012 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 9 May 2012 11:03:27 -0700 (PDT) Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> Message-ID: <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> ----- Original Message ----- > From: Richard O'Keefe > To: Thomas Lindgren > Cc: Michael Turner ; "erlang-questions@REDACTED" > Sent: Tuesday, May 8, 2012 3:07 AM > Subject: Re: [erlang-questions] Is there a good source for documentation on BEAM? > > > On 8/05/2012, at 7:15 AM, Thomas Lindgren wrote: >> There has been a substantial number of non-BEAM Erlang implementations > already, so I'm >> not convinced detailed BEAM docs is the key property* to spread Erlang. > > And how many of those non-BEAM implementations still exist? > Does GERL?? Is E2S still maintained?? How much of OTP can it handle? This, to my mind, says more about the (lack of) need for a second source implementation than any inherent problems with learning BEAM.?If you want to try your hand, quite a bit of the complexity is not in handling BEAM as such but in reimplementing ERTS: writing the BIFs, SMP, memory management, etc. >> Indeed, requiring detailed docs of every change of BEAM seems likely to? > slow innovation down instead. > > I not only *don't* believe that, I *can't* believe that. > Joe has informed us that there are TWO levels of BEAM, > one of which has been very stable, and one of which has > changed many times. > > I don't even believe your claim if made about the low level > much changed "BEAM", but let's suppose it true for the sake > of argument.? If the high level of BEAM has remained pretty > stable for quite a while, how would documenting it have > slowed innovation down? Note that BEAM files are not guaranteed to be compatible across releases, and they do change incompatibly every now and then.?(Not very often, to be sure. I recall it happening twice.) Check the mailing list for some discussions. The "sub-BEAM" implementation can change more rapidly, of course. I assume implementors there can do? platform specific things?like inline expanding instructions into native, mapping VM registers to native registers, constructing superinstructions, etc. (I seem to recall all of these being tried at one time or another.) As for what I see would cause a slowdown: the attention of the key hackers would be spent on writing this documentation (and then maintaining it, I assume). Perhaps people will?start depending on documented details? of implementation, explicitly or implicitly.?Major changes would also mean major internal docs rewrites. See below for one option. > ... [pace of innovation, see below on kickstarter for my comment] >>? >> If the motive is education, I think someone interested in compilers and > virtual machine architectures >> would have little trouble with BEAM as such. > > I have an interest in compilers and VMs.? I worked professionally on Quintus > Prolog and the real WAM (not the one in the papers or A?t-Kaci's book).? And > trying to figure out the BEAM was such a slog that to be honest, I said to > myself "the hell with it, if they don't *WANT* me to understand the > BEAM, > I'm not going to waste any more of my time trying to penetrate the > obscurity". At that level of knowledge, I assume the BEAM instruction set in itself is no big hurdle. If you want to learn the internals beyond that, what level of detail are you looking for? >> In a real sense, BEAM is just a vehicle to express compiler optimizations > for a >> restricted part of ERTS (the sequential execution part, basically). > > No, compiler optimisations are expressed in the executable code of the > compiler.? BEAM lets you express the *results* of such optimisations, > which is a different thing.? It's just like the Quintus compiler:? I could > figure out in that case what the *results* were, but the actual process > remained obscure.? (More precisely, what the 'invariants' were.) Here is how I see it: The instruction set of BEAM has been chosen for the purpose of expressing, and then used to express, various optimizations. Consider a simple example: targeting BEAM vs JAM (a stack machine used previously to implement erlang).? In order to optimize register use on JAM, you first have to?translate it to a new intermediate language (and then probably never try to translate it back to JAM), while BEAM?(like its uncle WAM) expresses registers explicitly and so makes such optimizations straightforward. > ... > Yes, I'm shouting.? "We don't need it" and "you don't > need it" are utterly > different propositions, and too many people in too many areas of life fail > to realise that. (To avoid any confusion, let me add that I last worked at Ericsson CSLAB in 1998. So I'm hardly an OTP insider.) So perhaps the right approach is to do a kickstarter to fund?someone writing a deep dive Erlang/OTP internals?book?? Complexity: roughly the level of writing a Linux kernel book,?at a quick guess. Perhaps a bit easier. >> Another argument might be that BEAM should be specified in detail in order > to be a suitable binary format for distribution, >> which is essentially what the JVM instruction set has become. > > I suggested many years ago that Erlang should take a leaf out of Kistler's > book (or PhD thesis).? The "Juice" system for Oberon compiled source > files > to abstract syntax trees, then cleverly compressed the ASTs and used them > as the binary distribution form.? They came in smaller than .class files > and had no presuppositions about the target hardware (not even primitive > size and alignment if I recall correctly).? The cost of decompressing and > generating native code was low, to the point where it was faster to > dynamically load Juice files than their equivalent of .so/.dll files, and > the generated code actually ran faster because the code generator knew > more about the environment of the target, including existing code.? (I > don't know if the Juice runtime did cross-module inlining, but it would > have been possible.) Not a bad idea. Best regards, Thomas From thomasl_erlang@REDACTED Wed May 9 20:07:25 2012 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 9 May 2012 11:07:25 -0700 (PDT) Subject: [erlang-questions] Fwd: Is there a good source for documentation on BEAM? In-Reply-To: <20120509002108.GA3417@mulga.csse.unimelb.edu.au> References: <79520DF4-2E14-4CA4-8880-0B03E4D6B8DD@cs.otago.ac.nz> <20120509002108.GA3417@mulga.csse.unimelb.edu.au> Message-ID: <1336586845.12870.YahooMailNeo@web111401.mail.gq1.yahoo.com> ----- Original Message ----- > From: Jeff Schultz >... > 4.? Documenting, and publishing, your internals makes them > patent-system visible prior art.? This protects you from some > Johnny-come-lately company, or individual, who patents something > you've been using for many years.? Source code, even publicly > available source code like Erlang, is not generally valid prior art in > patent terms.? Just because you've been doing, say, memory management, > one particular way for a decade, won't block a patent covering exactly > that. > > And yes, it does happen. Not to open a can of worms, but perhaps OTP should spend their time patenting their technology instead in that case? I'm sure Ericsson AB would like that, since they then may be able to crush their competitors into a finer pulp. Best regards, Thomas From wvvwwvw@REDACTED Wed May 9 21:00:29 2012 From: wvvwwvw@REDACTED (Andrew Ledvina) Date: Wed, 9 May 2012 12:00:29 -0700 Subject: [erlang-questions] foldl vs foldr and ++ Message-ID: Okay, so I was profiling something and it led me to write this: from_left(X) -> lists:foldl(fun(Y,R) -> R ++ [{Y+1,Y*2}] end, [], X). from_right(X) -> lists:foldr(fun(Y,R) -> [{Y+1,Y*2}] ++ R end, [], X). timeit(N) -> X = lists:seq(1,N), statistics(runtime), statistics(wall_clock), from_left(X), {_,LTime1} = statistics(runtime), {_,LTime2} = statistics(wall_clock), from_right(X), {_,RTime1} = statistics(runtime), {_,RTime2} = statistics(wall_clock), io:format("Left runtime: ~p (~p)~n", [LTime1,LTime2]), io:format("Right runtime: ~p (~p)~n", [RTime1,RTime2]). My understanding is that foldl is preferred over foldr because it is tail recursive. I was also to understand that putting the accumulator on the left-hand-side of the ++ operation is super bad. So, the following results were a bit of a surprise at first: Erlang R14B01 (erts-5.8.2) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.2 (abort with ^G) 1> listtime:timeit(10000). Left runtime: 290 (286) Right runtime: 0 (1) ok 2> listtime:timeit(100000). Left runtime: 30670 (31011) Right runtime: 10 (13) ok The times are in ms. 30 s versus 10 ms. The result of both functions from_left and from_right are identical. Also, I am clearly aware that these are both equivalent to just a more simple map. I think I understand the results after a bit of a go around but is there anyone who has a simple, clear explanation? --Andrew Ledvina -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Wed May 9 21:14:18 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Wed, 9 May 2012 21:14:18 +0200 Subject: [erlang-questions] foldl vs foldr and ++ In-Reply-To: References: Message-ID: > My understanding is that foldl is preferred over foldr because it is tail > recursive. I was also to understand that putting the accumulator on the > left-hand-side of the ++ operation is super bad. > According to Erlang and OTP in Action, the cost of ++ is specified by the left side. ++ copies the items on the left side on top of the right side which explains the performance difference. If the order does not matter, always put the smaller list on the left side. * Jos? Valim www.plataformatec.com.br Founder and Lead Developer * -------------- next part -------------- An HTML attachment was scrubbed... URL: From masklinn@REDACTED Wed May 9 21:20:27 2012 From: masklinn@REDACTED (Masklinn) Date: Wed, 9 May 2012 21:20:27 +0200 Subject: [erlang-questions] foldl vs foldr and ++ In-Reply-To: References: Message-ID: On 2012-05-09, at 21:00 , Andrew Ledvina wrote: > Okay, so I was profiling something and it led me to write this: > > from_left(X) -> > lists:foldl(fun(Y,R) -> R ++ [{Y+1,Y*2}] end, [], X). > > from_right(X) -> > lists:foldr(fun(Y,R) -> [{Y+1,Y*2}] ++ R end, [], X). > > timeit(N) -> > X = lists:seq(1,N), > statistics(runtime), > statistics(wall_clock), > from_left(X), > {_,LTime1} = statistics(runtime), > {_,LTime2} = statistics(wall_clock), > from_right(X), > {_,RTime1} = statistics(runtime), > {_,RTime2} = statistics(wall_clock), > io:format("Left runtime: ~p (~p)~n", [LTime1,LTime2]), > io:format("Right runtime: ~p (~p)~n", [RTime1,RTime2]). > > My understanding is that foldl is preferred over foldr because it is tail > recursive. I was also to understand that putting the accumulator on the > left-hand-side of the ++ operation is super bad. So, the following results > were a bit of a surprise at first: > > Erlang R14B01 (erts-5.8.2) [source] [smp:4:4] [rq:4] [async-threads:0] > [hipe] [kernel-poll:false] > > Eshell V5.8.2 (abort with ^G) > 1> listtime:timeit(10000). > Left runtime: 290 (286) > Right runtime: 0 (1) > ok > 2> listtime:timeit(100000). > Left runtime: 30670 (31011) > Right runtime: 10 (13) > ok > > The times are in ms. 30 s versus 10 ms. The result of both functions > from_left and from_right are identical. Also, I am clearly aware that these > are both equivalent to just a more simple map. I think I understand the > results after a bit of a go around but is there anyone who has a simple, > clear explanation? Well the explanation is exactly in what you wrote: > I was also to understand that putting the accumulator on the > left-hand-side of the ++ operation is super bad. that is indeed the case: in your left fold, the runtime has to copy all of the accumulator to add data to its end, whereas in your right fold it just has to create a new cons and append the old accumulator to it. The cost of cloning thousands of conses at each step is way over any potential saving from a tail-recursive left fold. To see this, replace your from_* functions by these: from_left(0) -> [0]; from_left(N) -> from_left(N-1) ++ [N]. from_right(0) -> [0]; from_right(N) -> [N] ++ from_right(N-1). and you'll get this: 1> listtime:timeit(10000). Left runtime: 390 (401) Right runtime: 0 (0) ok 2> listtime:timeit(100000). Left runtime: 56450 (57424) Right runtime: 10 (18) ok The operation which is costing you is long_list ++ element, because all of long_list has to be copied. Oh, and by the way you could (should?) use [Element | List] instead of [Element] ++ List. PS: just to prove that a tail-recursive fold is nowhere near good enough to compensate for the millions of extra copies, here it is: from_left(0, Acc) -> Acc ++ [0]; from_left(N, Acc) -> from_left(N-1, Acc ++ [N]). 6> listtime:timeit(10000). Left runtime: 310 (316) Right runtime: 0 (1) ok 7> listtime:timeit(100000). Left runtime: 52720 (54166) Right runtime: 10 (17) ok From ok@REDACTED Wed May 9 23:57:46 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 10 May 2012 09:57:46 +1200 Subject: [erlang-questions] foldl vs foldr and ++ In-Reply-To: References: Message-ID: On 10/05/2012, at 7:00 AM, Andrew Ledvina wrote: > Okay, so I was profiling something and it led me to write this: > > from_left(X) -> > lists:foldl(fun(Y,R) -> R ++ [{Y+1,Y*2}] end, [], X). > > from_right(X) -> > lists:foldr(fun(Y,R) -> [{Y+1,Y*2}] ++ R end, [], X). Immediate reaction: from_left/1 is going to be atrociously slow. > My understanding is that foldl is preferred over foldr because it is tail recursive. ***other things being equal*** In this case, they are not. It should be clear that from_right/1 is O(|X|) because it does O(1) work per element of X, while from_left/1 is O(|X|**2) because it does O(|R|) work per element of X, and R grows from empty to 2|X|-2 elements long. Write [{Y+1,Y*2} || Y <- X] and let the compiler worry about the best way to do it. > I was also to understand that putting the accumulator on the left-hand-side of the ++ operation is super bad. So, the following results were a bit of a surprise at first: If you understood that long++short is more expensive than short++long, you should NOT have been surprised by your result. From paulperegud@REDACTED Thu May 10 00:28:43 2012 From: paulperegud@REDACTED (=?UTF-8?Q?Pawe=C5=82_Peregud?=) Date: Thu, 10 May 2012 00:28:43 +0200 Subject: [erlang-questions] foldl vs foldr and ++ In-Reply-To: References: Message-ID: As I see it, cost of being not tail-recursive is O(2n) comparing to O(n) for tail-recursive where n is the number of calls. Cost of from_left in your example is O(n^2), because for every element whole linked list is reconstructed. On May 9, 2012 11:58 PM, "Richard O'Keefe" wrote: > > On 10/05/2012, at 7:00 AM, Andrew Ledvina wrote: > > > Okay, so I was profiling something and it led me to write this: > > > > from_left(X) -> > > lists:foldl(fun(Y,R) -> R ++ [{Y+1,Y*2}] end, [], X). > > > > from_right(X) -> > > lists:foldr(fun(Y,R) -> [{Y+1,Y*2}] ++ R end, [], X). > > Immediate reaction: from_left/1 is going to be atrociously slow. > > > My understanding is that foldl is preferred over foldr because it is > tail recursive. > > ***other things being equal*** > > In this case, they are not. It should be clear that from_right/1 is > O(|X|) because it does O(1) work per element of X, while > from_left/1 is O(|X|**2) because it does O(|R|) work per element of > X, and R grows from empty to 2|X|-2 elements long. > > Write [{Y+1,Y*2} || Y <- X] and let the compiler worry about the > best way to do it. > > > I was also to understand that putting the accumulator on the > left-hand-side of the ++ operation is super bad. So, the following results > were a bit of a surprise at first: > > If you understood that long++short is more expensive than short++long, > you should NOT have been surprised by your result. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wvvwwvw@REDACTED Thu May 10 02:01:16 2012 From: wvvwwvw@REDACTED (Andrew Ledvina) Date: Wed, 9 May 2012 17:01:16 -0700 Subject: [erlang-questions] foldl vs foldr and ++ In-Reply-To: References: Message-ID: <8459F898-B839-45A9-978F-E38A827F699C@gmail.com> Aha I just looked at my own explanation and realized it made no sense. Of course the slow one is slow for exactly the reason I expected. Sorry for the waste, but thank you for the clear complexity analysis. Andrew Ledvina On May 9, 2012, at 2:57 PM, "Richard O'Keefe" wrote: > > On 10/05/2012, at 7:00 AM, Andrew Ledvina wrote: > >> Okay, so I was profiling something and it led me to write this: >> >> from_left(X) -> >> lists:foldl(fun(Y,R) -> R ++ [{Y+1,Y*2}] end, [], X). >> >> from_right(X) -> >> lists:foldr(fun(Y,R) -> [{Y+1,Y*2}] ++ R end, [], X). > > Immediate reaction: from_left/1 is going to be atrociously slow. > >> My understanding is that foldl is preferred over foldr because it is tail recursive. > > ***other things being equal*** > > In this case, they are not. It should be clear that from_right/1 is > O(|X|) because it does O(1) work per element of X, while > from_left/1 is O(|X|**2) because it does O(|R|) work per element of > X, and R grows from empty to 2|X|-2 elements long. > > Write [{Y+1,Y*2} || Y <- X] and let the compiler worry about the > best way to do it. > >> I was also to understand that putting the accumulator on the left-hand-side of the ++ operation is super bad. So, the following results were a bit of a surprise at first: > > If you understood that long++short is more expensive than short++long, > you should NOT have been surprised by your result. > > From ok@REDACTED Thu May 10 08:10:24 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 10 May 2012 18:10:24 +1200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> Message-ID: <7E452DB1-8AB8-44C6-8CC9-5DA99EF8AB22@cs.otago.ac.nz> (1) We were told that BEAM documentation isn't needed because there are other Erlang implementations. (2) I ask whether any of those other implementations ever kept up with The Real Thing. (By the way, as far as I know, none of them ever supported bit syntax, and my recent attempt to install GERL failed miserably.) (3) Suddenly we are told that the abandoning of those other things just *proves* that we don't need BEAM documentation. ? > > >>> Indeed, requiring detailed docs of every change of BEAM seems likely to >> slow innovation down instead. >> [Me] >> I not only *don't* believe that, I *can't* believe that. >> Joe has informed us that there are TWO levels of BEAM, >> one of which has been very stable, and one of which has >> changed many times. >> >> I don't even believe your claim if made about the low level >> much changed "BEAM", but let's suppose it true for the sake >> of argument. If the high level of BEAM has remained pretty >> stable for quite a while, how would documenting it have >> slowed innovation down? > [Thomas Lindgren] > Note that BEAM files are not guaranteed to be compatible across releases, and they do change incompatibly > every now and then. (Not very often, to be sure. I recall it happening twice.) Check the mailing list for some discussions. If BEAM were completely stable, it might be reasonable to expect anyone who cares to figure out BEAM for themselves, once and for all. The more it changes, THE MORE IT NEEDS TO BE DOCUMENTED. I repeat my claim about the fragmentary emulator I wrote: the better the documentation was, the *FASTER* I could write and rewrite it. Having the tables needed for the disassembler (which exists) and the assembler (which doesn't yet, but will) automatically generated from the same file that the emulator switch is generated from means they are consistent *all the time* without me having to check. Switching a numeric operand from scaled (use @size) to unscaled (use @number) or back with the instruction description where the preprocessor can see it means that an incomplete edit (changing some occurrences but not others) will be caught *before the C compiler is run* let alone before run time. > > As for what I see would cause a slowdown: the attention of the key hackers would be spent on writing this > documentation (and then maintaining it, I assume). In my fragmentary emulator, there are roughly equal lines of documentation and code, except that thanks to the preprocessor, the lines of code are simpler and more often correct than they would have been without. *This* hacker, at least, found it took *less* time to write documentation+code than to just write code. It's not hugely detailed documentation, but whether operands are raw or tagged, whether a tag check is *intentionally* omitted, whether some other instruction is expected to set up a context in some register, the name of a related instruction where the details can be found, sometimes what possibly surprising source forms an instruction was meant for. It DOESN'T have to be huge benefit, and I would expect 'key hackers' to be doing it for their *own* sake, never mind anyone else's. I can understand the documentation being stripped out before release; what I'm having trouble with is the idea of it never having existed, and the idea that not having it makes life in some unimaginable way easier for the developers. > Perhaps people will start depending on documented details > of implementation, explicitly or implicitly. Major changes would also mean major internal docs rewrites. Well, yes, BUT a preprocessor that helps you get the original code right will also make it easier to make major changes right. > > At that level of knowledge, I assume the BEAM instruction set in itself is no big hurdle. > If you want to learn the internals beyond that, what level of detail are you looking for? Getting a *rough* idea of the compiler's output is no big deal. Understanding it well enough to generate correct code myself *is*. For one example of the kind of understanding you can get from documentation, I was puzzled because I couldn't see the instructions I knew must be there to nil out X registers that ceased to be live. Joe's little document made it clear that (a) there are a lot more X registers allowed in Erlang that in Quintus Prolog; (b) maintaining them is more expensive than in Quintus Prolog; (c) the nilling instructions I expected don't exist; (d) there is a (temporary) space leak: if register K is live at an allocation point, all registers <= K are assumed to be live. With hindsight, I can now see (most of) that in the .S files. But it really wasn't obvious. One important detail is the layout of Erlang stack frames. From michael.eugene.turner@REDACTED Thu May 10 08:22:38 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 10 May 2012 15:22:38 +0900 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> Message-ID: "As for what I see would cause a slowdown: the attention of the key hackers would be spent on writing this documentation (and then maintaining it, I assume)." Perhaps better: volunteers could document it (on a relatively controlled wiki, for example). Then the "key hackers" could mention any needed corrections. As for maintenance, you say yourself (in a later e-mail) that you can only remember significant changes happening twice. Documenting such infrequent changes doesn't exactly sound like some grinding daily burden for already-overworked Ericsson programmers. If they have to propose these changes in writing anyway (at least in internal e-mail), sounds like most of the documentation work gets done before the changes are made. -michael turner On Thu, May 10, 2012 at 3:03 AM, Thomas Lindgren wrote: > > > > > ----- Original Message ----- >> From: Richard O'Keefe >> To: Thomas Lindgren >> Cc: Michael Turner ; "erlang-questions@REDACTED" >> Sent: Tuesday, May 8, 2012 3:07 AM >> Subject: Re: [erlang-questions] Is there a good source for documentation on BEAM? >> >> >> On 8/05/2012, at 7:15 AM, Thomas Lindgren wrote: >>> ?There has been a substantial number of non-BEAM Erlang implementations >> already, so I'm >>> ?not convinced detailed BEAM docs is the key property* to spread Erlang. >> >> And how many of those non-BEAM implementations still exist? >> Does GERL?? Is E2S still maintained?? How much of OTP can it handle? > > This, to my mind, says more about the (lack of) need for a second source implementation than any inherent > problems with learning BEAM.?If you want to try your hand, quite a bit of the complexity is not in handling BEAM > as such but in reimplementing ERTS: writing the BIFs, SMP, memory management, etc. > >>> ?Indeed, requiring detailed docs of every change of BEAM seems likely to > >> slow innovation down instead. >> >> I not only *don't* believe that, I *can't* believe that. >> Joe has informed us that there are TWO levels of BEAM, >> one of which has been very stable, and one of which has >> changed many times. >> >> I don't even believe your claim if made about the low level >> much changed "BEAM", but let's suppose it true for the sake >> of argument.? If the high level of BEAM has remained pretty >> stable for quite a while, how would documenting it have >> slowed innovation down? > > Note that BEAM files are not guaranteed to be compatible across releases, and they do change incompatibly > every now and then.?(Not very often, to be sure. I recall it happening twice.) Check the mailing list for some discussions. > > The "sub-BEAM" implementation can change more rapidly, of course. I assume implementors there can do > platform specific things?like inline expanding instructions into native, mapping VM registers to native registers, > constructing superinstructions, etc. (I seem to recall all of these being tried at one time or another.) > > As for what I see would cause a slowdown: the attention of the key hackers would be spent on writing this > documentation (and then maintaining it, I assume). Perhaps people will?start depending on documented details > of implementation, explicitly or implicitly.?Major changes would also mean major internal docs rewrites. > > See below for one option. > >> ... [pace of innovation, see below on kickstarter for my comment] >>> > >>> ?If the motive is education, I think someone interested in compilers and >> virtual machine architectures >>> ?would have little trouble with BEAM as such. >> >> I have an interest in compilers and VMs.? I worked professionally on Quintus >> Prolog and the real WAM (not the one in the papers or A?t-Kaci's book).? And >> trying to figure out the BEAM was such a slog that to be honest, I said to >> myself "the hell with it, if they don't *WANT* me to understand the >> BEAM, >> I'm not going to waste any more of my time trying to penetrate the >> obscurity". > > > At that level of knowledge, I assume the BEAM instruction set in itself is no big hurdle. > If you want to learn the internals beyond that, what level of detail are you looking for? > >>> ?In a real sense, BEAM is just a vehicle to express compiler optimizations >> for a >>> ?restricted part of ERTS (the sequential execution part, basically). >> >> No, compiler optimisations are expressed in the executable code of the >> compiler.? BEAM lets you express the *results* of such optimisations, >> which is a different thing.? It's just like the Quintus compiler:? I could >> figure out in that case what the *results* were, but the actual process >> remained obscure.? (More precisely, what the 'invariants' were.) > > > Here is how I see it: The instruction set of BEAM has been chosen for the purpose of expressing, and then used to express, various optimizations. > Consider a simple example: targeting BEAM vs JAM (a stack machine used previously to implement erlang). > In order to optimize register use on JAM, you first have to?translate it to a new intermediate language (and then probably never > try to translate it back to JAM), while BEAM?(like its uncle WAM) expresses registers explicitly and so makes such optimizations straightforward. > >> ... >> Yes, I'm shouting.? "We don't need it" and "you don't >> need it" are utterly >> different propositions, and too many people in too many areas of life fail >> to realise that. > > (To avoid any confusion, let me add that I last worked at Ericsson CSLAB in 1998. So I'm hardly an OTP insider.) > > So perhaps the right approach is to do a kickstarter to fund?someone writing a deep dive Erlang/OTP internals?book? > Complexity: roughly the level of writing a Linux kernel book,?at a quick guess. Perhaps a bit easier. > >>> ?Another argument might be that BEAM should be specified in detail in order >> to be a suitable binary format for distribution, >>> ?which is essentially what the JVM instruction set has become. >> >> I suggested many years ago that Erlang should take a leaf out of Kistler's >> book (or PhD thesis).? The "Juice" system for Oberon compiled source >> files >> to abstract syntax trees, then cleverly compressed the ASTs and used them >> as the binary distribution form.? They came in smaller than .class files >> and had no presuppositions about the target hardware (not even primitive >> size and alignment if I recall correctly).? The cost of decompressing and >> generating native code was low, to the point where it was faster to >> dynamically load Juice files than their equivalent of .so/.dll files, and >> the generated code actually ran faster because the code generator knew >> more about the environment of the target, including existing code.? (I >> don't know if the Juice runtime did cross-module inlining, but it would >> have been possible.) > > > Not a bad idea. > > Best regards, > Thomas > From watson.timothy@REDACTED Thu May 10 09:38:00 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 10 May 2012 08:38:00 +0100 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> Message-ID: <3E707156-15AB-4E77-9832-212F4EB7B081@gmail.com> > > > So perhaps the right approach is to do a kickstarter to fund someone writing a deep dive Erlang/OTP internals book? > Complexity: roughly the level of writing a Linux kernel book, at a quick guess. Perhaps a bit easier. > That would be a vital spot on every erlang programmer's bookshelf for sure. >>> Another argument might be that BEAM should be specified in detail in order >> to be a suitable binary format for distribution, >>> which is essentially what the JVM instruction set has become. >> >> I suggested many years ago that Erlang should take a leaf out of Kistler's >> book (or PhD thesis). The "Juice" system for Oberon compiled source >> files >> to abstract syntax trees, then cleverly compressed the ASTs and used them >> as the binary distribution form. They came in smaller than .class files >> and had no presuppositions about the target hardware (not even primitive >> size and alignment if I recall correctly). The cost of decompressing and >> generating native code was low, to the point where it was faster to >> dynamically load Juice files than their equivalent of .so/.dll files, and >> the generated code actually ran faster because the code generator knew >> more about the environment of the target, including existing code. (I >> don't know if the Juice runtime did cross-module inlining, but it would >> have been possible.) > > > Not a bad idea. > > Best regards, > Thomas > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu May 10 09:59:04 2012 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 10 May 2012 09:59:04 +0200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> Message-ID: What a long discussion ... I have a few comments. I have started writing a 2'nd edition of "Programming Erlang" - I have a dumping ground for potential chapters in books that one-day-some-time-if-i-get-time I might write. One of these in called "beam". A question was asked about how the beam worked - so I thought it's bit silly me sitting on this - I'll post it off since it might help. I don't actually know how the Beam works - I have a vague idea - but fortunately I can just wander down the corridor and ask Bj?rn who does know how it works. I also have a strong aversion to reading code - I like to know how stuff is supposed to work and not reading the code to find out. (I hate reading code - as soon as I read code - I get sidetracked by wondering "why was it written this way" and often get a strong desire to rewrite it - I once wondered how PDF worked and that was a complete disaster - 3 months down the drain and ErlGuten was the result - and all I really wanted to do was figure out why the kerning in an open office slide was manifestly wrong. Any sensible person would have said "don't ask") I know of only two people who have figured out how the beam works *without* asking Bj?rn that was Fredrik Svan and Kresten Krab Thorup and I am deeply impressed that they managed to do this. I asked both Fredrik and Kresten how they did this - they both said "I reverse engineered the code" - they both had good reasons Fredrik made a javascript Erlang in the browser thing, and Kresten made Erjang. Now there are two levels at which one could describe the Beam - level one is the relationship between erlang code and the beam instructions - this is what I described. Actually running erl -S and guesswork gets you pretty far - this what I did - I only had to play my "ask Bj?rn card" a few times (there's some stuff about marking which registers have to be garbaged, which is not guessable). At this level of abstraction we can completely ignore memory management, most of garbage collection, how process stacks and heaps are organised, how multicores are locked etc. To describe the next level - we suddenly jump from a one chapter description to a entire book. This is a book that is tricky to write - I guess no one person knows all the answers. it's also a book that few (I suspect) would read. Fredrik and Krestin didn't have to understand much of the beam memory managment. Fredrik used whatever GC and object representation javascript uses and Krestin used whatever the JVM did - so it wasn't really relevant. Realistically the only thing that might get written is a piffed up version of what I've distributed - but I would be reluctant to include it in the next edition of Programming Erlang, I can use the space for content with wider appeal. I'll try and make a better version of what I've distributed and put it up on the main web site... but this is a low priority task Cheers /Joe On Thu, May 10, 2012 at 8:22 AM, Michael Turner wrote: > "As for what I see would cause a slowdown: the attention of the key > hackers would be spent on writing this > documentation (and then maintaining it, I assume)." > > Perhaps better: volunteers could document it (on a relatively > controlled wiki, for example). Then the "key hackers" could mention > any needed corrections. > > As for maintenance, you say yourself (in a later e-mail) that you can > only remember significant changes happening twice. Documenting such > infrequent changes doesn't exactly sound like some grinding daily > burden for already-overworked Ericsson programmers. If they have to > propose these changes in writing anyway (at least in internal e-mail), > sounds like most of the documentation work gets done before the > changes are made. > > -michael turner > > > On Thu, May 10, 2012 at 3:03 AM, Thomas Lindgren > wrote: >> >> >> >> >> ----- Original Message ----- >>> From: Richard O'Keefe >>> To: Thomas Lindgren >>> Cc: Michael Turner ; "erlang-questions@REDACTED" >>> Sent: Tuesday, May 8, 2012 3:07 AM >>> Subject: Re: [erlang-questions] Is there a good source for documentation on BEAM? >>> >>> >>> On 8/05/2012, at 7:15 AM, Thomas Lindgren wrote: >>>> ?There has been a substantial number of non-BEAM Erlang implementations >>> already, so I'm >>>> ?not convinced detailed BEAM docs is the key property* to spread Erlang. >>> >>> And how many of those non-BEAM implementations still exist? >>> Does GERL?? Is E2S still maintained?? How much of OTP can it handle? >> >> This, to my mind, says more about the (lack of) need for a second source implementation than any inherent >> problems with learning BEAM.?If you want to try your hand, quite a bit of the complexity is not in handling BEAM >> as such but in reimplementing ERTS: writing the BIFs, SMP, memory management, etc. >> >>>> ?Indeed, requiring detailed docs of every change of BEAM seems likely to >> >>> slow innovation down instead. >>> >>> I not only *don't* believe that, I *can't* believe that. >>> Joe has informed us that there are TWO levels of BEAM, >>> one of which has been very stable, and one of which has >>> changed many times. >>> >>> I don't even believe your claim if made about the low level >>> much changed "BEAM", but let's suppose it true for the sake >>> of argument.? If the high level of BEAM has remained pretty >>> stable for quite a while, how would documenting it have >>> slowed innovation down? >> >> Note that BEAM files are not guaranteed to be compatible across releases, and they do change incompatibly >> every now and then.?(Not very often, to be sure. I recall it happening twice.) Check the mailing list for some discussions. >> >> The "sub-BEAM" implementation can change more rapidly, of course. I assume implementors there can do >> platform specific things?like inline expanding instructions into native, mapping VM registers to native registers, >> constructing superinstructions, etc. (I seem to recall all of these being tried at one time or another.) >> >> As for what I see would cause a slowdown: the attention of the key hackers would be spent on writing this >> documentation (and then maintaining it, I assume). Perhaps people will?start depending on documented details >> of implementation, explicitly or implicitly.?Major changes would also mean major internal docs rewrites. >> >> See below for one option. >> >>> ... [pace of innovation, see below on kickstarter for my comment] >>>> >> >>>> ?If the motive is education, I think someone interested in compilers and >>> virtual machine architectures >>>> ?would have little trouble with BEAM as such. >>> >>> I have an interest in compilers and VMs.? I worked professionally on Quintus >>> Prolog and the real WAM (not the one in the papers or A?t-Kaci's book).? And >>> trying to figure out the BEAM was such a slog that to be honest, I said to >>> myself "the hell with it, if they don't *WANT* me to understand the >>> BEAM, >>> I'm not going to waste any more of my time trying to penetrate the >>> obscurity". >> >> >> At that level of knowledge, I assume the BEAM instruction set in itself is no big hurdle. >> If you want to learn the internals beyond that, what level of detail are you looking for? >> >>>> ?In a real sense, BEAM is just a vehicle to express compiler optimizations >>> for a >>>> ?restricted part of ERTS (the sequential execution part, basically). >>> >>> No, compiler optimisations are expressed in the executable code of the >>> compiler.? BEAM lets you express the *results* of such optimisations, >>> which is a different thing.? It's just like the Quintus compiler:? I could >>> figure out in that case what the *results* were, but the actual process >>> remained obscure.? (More precisely, what the 'invariants' were.) >> >> >> Here is how I see it: The instruction set of BEAM has been chosen for the purpose of expressing, and then used to express, various optimizations. >> Consider a simple example: targeting BEAM vs JAM (a stack machine used previously to implement erlang). >> In order to optimize register use on JAM, you first have to?translate it to a new intermediate language (and then probably never >> try to translate it back to JAM), while BEAM?(like its uncle WAM) expresses registers explicitly and so makes such optimizations straightforward. >> >>> ... >>> Yes, I'm shouting.? "We don't need it" and "you don't >>> need it" are utterly >>> different propositions, and too many people in too many areas of life fail >>> to realise that. >> >> (To avoid any confusion, let me add that I last worked at Ericsson CSLAB in 1998. So I'm hardly an OTP insider.) >> >> So perhaps the right approach is to do a kickstarter to fund?someone writing a deep dive Erlang/OTP internals?book? >> Complexity: roughly the level of writing a Linux kernel book,?at a quick guess. Perhaps a bit easier. >> >>>> ?Another argument might be that BEAM should be specified in detail in order >>> to be a suitable binary format for distribution, >>>> ?which is essentially what the JVM instruction set has become. >>> >>> I suggested many years ago that Erlang should take a leaf out of Kistler's >>> book (or PhD thesis).? The "Juice" system for Oberon compiled source >>> files >>> to abstract syntax trees, then cleverly compressed the ASTs and used them >>> as the binary distribution form.? They came in smaller than .class files >>> and had no presuppositions about the target hardware (not even primitive >>> size and alignment if I recall correctly).? The cost of decompressing and >>> generating native code was low, to the point where it was faster to >>> dynamically load Juice files than their equivalent of .so/.dll files, and >>> the generated code actually ran faster because the code generator knew >>> more about the environment of the target, including existing code.? (I >>> don't know if the Juice runtime did cross-module inlining, but it would >>> have been possible.) >> >> >> Not a bad idea. >> >> Best regards, >> Thomas >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From robert.virding@REDACTED Thu May 10 10:26:14 2012 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 10 May 2012 09:26:14 +0100 (BST) Subject: [erlang-questions] foldl vs foldr and ++ In-Reply-To: <8459F898-B839-45A9-978F-E38A827F699C@gmail.com> Message-ID: The documentation is referring to the *implementation* of the foldl/foldr functions, foldl uses a tail-recursive function while foldr cannot do that. The difference in speed comes from what you do inside the fold function. As others have pointed out the efficiency of ++ depends solely on its left argument. Robert ----- Original Message ----- > Aha I just looked at my own explanation and realized it made no > sense. Of course the slow one is slow for exactly the reason I > expected. Sorry for the waste, but thank you for the clear > complexity analysis. > > Andrew Ledvina > > On May 9, 2012, at 2:57 PM, "Richard O'Keefe" > wrote: > > > > > On 10/05/2012, at 7:00 AM, Andrew Ledvina wrote: > > > >> Okay, so I was profiling something and it led me to write this: > >> > >> from_left(X) -> > >> lists:foldl(fun(Y,R) -> R ++ [{Y+1,Y*2}] end, [], X). > >> > >> from_right(X) -> > >> lists:foldr(fun(Y,R) -> [{Y+1,Y*2}] ++ R end, [], X). > > > > Immediate reaction: from_left/1 is going to be atrociously slow. > > > >> My understanding is that foldl is preferred over foldr because it > >> is tail recursive. > > > > ***other things being equal*** > > > > In this case, they are not. It should be clear that from_right/1 > > is > > O(|X|) because it does O(1) work per element of X, while > > from_left/1 is O(|X|**2) because it does O(|R|) work per element of > > X, and R grows from empty to 2|X|-2 elements long. > > > > Write [{Y+1,Y*2} || Y <- X] and let the compiler worry about the > > best way to do it. > > > >> I was also to understand that putting the accumulator on the > >> left-hand-side of the ++ operation is super bad. So, the > >> following results were a bit of a surprise at first: > > > > If you understood that long++short is more expensive than > > short++long, > > you should NOT have been surprised by your result. > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From bgustavsson@REDACTED Thu May 10 10:28:24 2012 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 10 May 2012 10:28:24 +0200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: <7E452DB1-8AB8-44C6-8CC9-5DA99EF8AB22@cs.otago.ac.nz> References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> <7E452DB1-8AB8-44C6-8CC9-5DA99EF8AB22@cs.otago.ac.nz> Message-ID: On Thu, May 10, 2012 at 8:10 AM, Richard O'Keefe wrote: [...] > Joe's little document made it clear that > (a) there are a lot more X registers allowed in Erlang that in > ? ?Quintus Prolog; > (b) maintaining them is more expensive than in Quintus Prolog; > (c) the nilling instructions I expected don't exist; > (d) there is a (temporary) space leak: if register K is live > ? ?at an allocation point, all registers <= K are assumed to > ? ?be live. There is not a temporary space leak in practice, because the compiler will insert an instruction that will clear (set to NIL) each dead register below K before any allocation point. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From vladdu55@REDACTED Thu May 10 10:28:28 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 10 May 2012 10:28:28 +0200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> Message-ID: Hi, On Thu, May 10, 2012 at 9:59 AM, Joe Armstrong wrote: > Now there are two levels at which one could describe the Beam - level one > is the relationship between erlang code and the beam instructions - > this is what I described. > > To describe the next level - we suddenly jump from a one chapter > description to a > entire book. This is a book that is tricky to write - I guess no one > person knows > all the answers. This is not entirely on topic, but close enough (and probably only Bj?rn could know the answer, I have a feeling that reading the code would require many years of study :-)): I have been wondering how much is the runtime tied up to the BEAM instruction set. More precisely, would it be possible (at least conceptually) to separate the scheduler + process manager + message passing infrastructure from the instruction executor? One common thing is the internal data format (because the runtime creates for example exit messages) so there are restrictions imposed by that, but it would be kind of cool to be able to plug in an interpreter for a different instruction set, in case one wants to use something else than Erlang and this other thing can't be compiled to Erlang, Erlang core or BEAM. best regards, Vlad From sverker@REDACTED Thu May 10 10:52:49 2012 From: sverker@REDACTED (Sverker Eriksson) Date: Thu, 10 May 2012 10:52:49 +0200 Subject: [erlang-questions] Questions about ETS locking In-Reply-To: <72940294.1d37c.137348e71e5.Coremail.cloudzen@163.com> References: <2d34a408.e600.1371c4aa309.Coremail.cloudzen@163.com> <4FA7C64B.5090002@erix.ericsson.se> <72940294.1d37c.137348e71e5.Coremail.cloudzen@163.com> Message-ID: <4FAB81E1.40300@erix.ericsson.se> By default, several concurrent readers can access the same table at the same time. Option read_concurrency will make concurrent reading even more faster at the expense of a slower operation when switching between "read mode" and "write mode". http://www.erlang.org/doc/man/ets.html#new_2_read_concurrency /Sverker, Erlang/OTP skyman wrote: > Oh, I see. Thanks a lot. > Another question: is default reading from an ETS table is concurrency? what's the difference between the default reading and option read_concurrency? > Thank you! > > > > > > At 2012-05-07 20:55:39,"Sverker Eriksson" wrote: > >> skyman wrote: >> >>> Hi all, >>> >>> >>> I know when access an ETS table, there are wo locks that need to be taken. >>> 1) A lock to access the meta table, to convert the numeric table identifier >>> to a pointer to the actual table. >>> >>> 2) The lock for the table itself (either a read or write lock). >>> I know in the SMP mode??an ETS table must be locked when access it, but I don't make sure that must an ETS table be locked in the non-SMP mode(only one scheduler)? >>> >>> >> An SMP-enabled emulator will do the locking even when running only one >> scheduler. >> A non-SMP emulator (erl -smp disable) will not do any locking. >> >> >>> Another question is, whether the default ETS table locking is on table-level or on record-level? >>> Thanks in advance! >>> >>> >>> >> The default locking is on table-level, allowing only one update >> operation at a time per table. >> Table option write_concurrency will enable locking on a more fine >> grained level, allowing concurrent update operations. In current >> implementation 16 locks per table is used, which result in a probability >> of 1/16 that two random keys will collide on the same lock. >> >> Option write_concurrency have no effect on ordered_set's. >> >> >> /Sverker, Erlang/OTP Ericsson >> >> >> From hynek@REDACTED Thu May 10 10:53:19 2012 From: hynek@REDACTED (Hynek Vychodil) Date: Thu, 10 May 2012 10:53:19 +0200 Subject: [erlang-questions] foldl vs foldr and ++ In-Reply-To: References: <8459F898-B839-45A9-978F-E38A827F699C@gmail.com> Message-ID: When I see foldr implementation I wonder why it is not simple foldr(F, Acc, L) -> foldl(F, Acc, lists:reverse(L, [])). There should be less memory consumption. lists:reverse is BIF which I believe consume less memory than stack frame per each list element what current foldr implementation do. It should be faster and both accepts only proper lists. Only difference which I can found current implementation consumes memory on stack but above implementation would consume heap and affect GC behavior. Current foldr implementation would make sense in Haskell but I don't understand its reason in Erlang. On Thu, May 10, 2012 at 10:26 AM, Robert Virding wrote: > The documentation is referring to the *implementation* of the foldl/foldr functions, foldl uses a tail-recursive function while foldr cannot do that. The difference in speed comes from what you do inside the fold function. As others have pointed out the efficiency of ++ depends solely on its left argument. > > Robert > > ----- Original Message ----- >> Aha I just looked at my own explanation and realized it made no >> sense. Of course the slow one is slow for exactly the reason I >> expected. Sorry for the waste, but thank you for the clear >> complexity analysis. >> >> Andrew Ledvina >> >> On May 9, 2012, at 2:57 PM, "Richard O'Keefe" >> wrote: >> >> > >> > On 10/05/2012, at 7:00 AM, Andrew Ledvina wrote: >> > >> >> Okay, so I was profiling something and it led me to write this: >> >> >> >> from_left(X) -> >> >> ? lists:foldl(fun(Y,R) -> R ++ [{Y+1,Y*2}] end, [], X). >> >> >> >> from_right(X) -> >> >> ? lists:foldr(fun(Y,R) -> [{Y+1,Y*2}] ++ R end, [], X). >> > >> > Immediate reaction: ?from_left/1 is going to be atrociously slow. >> > >> >> My understanding is that foldl is preferred over foldr because it >> >> is tail recursive. >> > >> > ***other things being equal*** >> > >> > In this case, they are not. ?It should be clear that from_right/1 >> > is >> > O(|X|) because it does O(1) work per element of X, while >> > from_left/1 is O(|X|**2) because it does O(|R|) work per element of >> > X, and R grows from empty to 2|X|-2 elements long. >> > >> > Write [{Y+1,Y*2} || Y <- X] and let the compiler worry about the >> > best way to do it. >> > >> >> I was also to understand that putting the accumulator on the >> >> left-hand-side of the ++ operation is super bad. So, the >> >> following results were a bit of a surprise at first: >> > >> > If you understood that long++short is more expensive than >> > short++long, >> > you should NOT have been surprised by your result. >> > >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Hynek Vychodil Chief Scientists GoodData n?m?st? 28. ??jna 1104/17, 602 00, Brno - ?ern? Pole Office: ? +420 530 50 7704 E-mail: ?hynek@REDACTED Web: ? ? www.gooddata.com From ulf@REDACTED Thu May 10 11:08:13 2012 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 10 May 2012 11:08:13 +0200 Subject: [erlang-questions] foldl vs foldr and ++ In-Reply-To: References: <8459F898-B839-45A9-978F-E38A827F699C@gmail.com> Message-ID: <615394A3-CD21-406F-91F5-3A3C0986A150@feuerlabs.com> Try this benchmark program with, e.g. L = lists:seq(1,100000). -module(lr). -export([l/1, r/1, x/1, foldr/3]). l(L) -> timer:tc(lists, foldl, [fun erlang:'+'/2, 0, L]). r(L) -> timer:tc(lists, foldr, [fun erlang:'+'/2, 0, L]). x(L) -> timer:tc(?MODULE, foldr, [fun erlang:'+'/2, 0, L]). foldr(F, Acc, L) -> lists:foldl(F, Acc, lists:reverse(L, [])). lists:foldl/3 and lists:foldr/3 run a fairly even race, whereas lr:foldr/3 is consistenty slower. BR, Ulf W On 10 May 2012, at 10:53, Hynek Vychodil wrote: > When I see foldr implementation I wonder why it is not simple > > foldr(F, Acc, L) -> foldl(F, Acc, lists:reverse(L, [])). > > There should be less memory consumption. lists:reverse is BIF which I > believe consume less memory than stack frame per each list element > what current foldr implementation do. It should be faster and both > accepts only proper lists. Only difference which I can found current > implementation consumes memory on stack but above implementation would > consume heap and affect GC behavior. Current foldr implementation > would make sense in Haskell but I don't understand its reason in > Erlang. > > On Thu, May 10, 2012 at 10:26 AM, Robert Virding > wrote: >> The documentation is referring to the *implementation* of the foldl/foldr functions, foldl uses a tail-recursive function while foldr cannot do that. The difference in speed comes from what you do inside the fold function. As others have pointed out the efficiency of ++ depends solely on its left argument. >> >> Robert >> >> ----- Original Message ----- >>> Aha I just looked at my own explanation and realized it made no >>> sense. Of course the slow one is slow for exactly the reason I >>> expected. Sorry for the waste, but thank you for the clear >>> complexity analysis. >>> >>> Andrew Ledvina >>> >>> On May 9, 2012, at 2:57 PM, "Richard O'Keefe" >>> wrote: >>> >>>> >>>> On 10/05/2012, at 7:00 AM, Andrew Ledvina wrote: >>>> >>>>> Okay, so I was profiling something and it led me to write this: >>>>> >>>>> from_left(X) -> >>>>> lists:foldl(fun(Y,R) -> R ++ [{Y+1,Y*2}] end, [], X). >>>>> >>>>> from_right(X) -> >>>>> lists:foldr(fun(Y,R) -> [{Y+1,Y*2}] ++ R end, [], X). >>>> >>>> Immediate reaction: from_left/1 is going to be atrociously slow. >>>> >>>>> My understanding is that foldl is preferred over foldr because it >>>>> is tail recursive. >>>> >>>> ***other things being equal*** >>>> >>>> In this case, they are not. It should be clear that from_right/1 >>>> is >>>> O(|X|) because it does O(1) work per element of X, while >>>> from_left/1 is O(|X|**2) because it does O(|R|) work per element of >>>> X, and R grows from empty to 2|X|-2 elements long. >>>> >>>> Write [{Y+1,Y*2} || Y <- X] and let the compiler worry about the >>>> best way to do it. >>>> >>>>> I was also to understand that putting the accumulator on the >>>>> left-hand-side of the ++ operation is super bad. So, the >>>>> following results were a bit of a surprise at first: >>>> >>>> If you understood that long++short is more expensive than >>>> short++long, >>>> you should NOT have been surprised by your result. >>>> >>>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Hynek Vychodil > Chief Scientists > > GoodData > n?m?st? 28. ??jna 1104/17, 602 00, Brno - ?ern? Pole > Office: +420 530 50 7704 > E-mail: hynek@REDACTED > Web: www.gooddata.com > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From olivier.boudeville@REDACTED Thu May 10 11:31:56 2012 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Thu, 10 May 2012 11:31:56 +0200 Subject: [erlang-questions] [ANN] Functional Summer School Message-ID: Hi all, With our colleagues of CEA and INRIA, we are organizing a summer school focusing on "Functional Programming for Parallel and Concurrent Applications". It is to take place from June 11 to June 22, in south of France. A few seats are still available. The implementation language we selected is Haskell, but it may be nevertheless of interest to the Erlang community, as a main purpose is to emphasize on how the functional approach is to improve the way we manage concurrency challenges. Moreover there will be at least a few Erlang developers in the audience, and we will have the pleasure to listen there to a short talk given by Joe Armstrong himself! All details are available on http://www-hpc.cea.fr/SummerSchools2012-CS.htm Best regards, Olivier. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hynek@REDACTED Thu May 10 11:39:29 2012 From: hynek@REDACTED (Hynek Vychodil) Date: Thu, 10 May 2012 11:39:29 +0200 Subject: [erlang-questions] foldl vs foldr and ++ In-Reply-To: <615394A3-CD21-406F-91F5-3A3C0986A150@feuerlabs.com> References: <8459F898-B839-45A9-978F-E38A827F699C@gmail.com> <615394A3-CD21-406F-91F5-3A3C0986A150@feuerlabs.com> Message-ID: You are right. I should realize it myself instead guessing. Compiler has to do something really smart there. Strange thing happen when I copied lists:foldl and lists:foldr implementation inside benchmark module and compile native. It is far (more than twice) slower than BEAM. On Thu, May 10, 2012 at 11:08 AM, Ulf Wiger wrote: > > Try this benchmark program with, e.g. L = lists:seq(1,100000). > > -module(lr). > -export([l/1, r/1, x/1, foldr/3]). > > > l(L) -> > ? ?timer:tc(lists, foldl, [fun erlang:'+'/2, 0, L]). > > r(L) -> > ? ?timer:tc(lists, foldr, [fun erlang:'+'/2, 0, L]). > > x(L) -> > ? ?timer:tc(?MODULE, foldr, [fun erlang:'+'/2, 0, L]). > > foldr(F, Acc, L) -> lists:foldl(F, Acc, lists:reverse(L, [])). > > > lists:foldl/3 and lists:foldr/3 run a fairly even race, whereas lr:foldr/3 is consistenty slower. > > BR, > Ulf W > > On 10 May 2012, at 10:53, Hynek Vychodil wrote: > >> When I see foldr implementation I wonder why it is not simple >> >> foldr(F, Acc, L) -> foldl(F, Acc, lists:reverse(L, [])). >> >> There should be less memory consumption. lists:reverse is BIF which I >> believe consume less memory than stack frame per each list element >> what current foldr implementation do. It should be faster and both >> accepts only proper lists. Only difference which I can found current >> implementation consumes memory on stack but above implementation would >> consume heap and affect GC behavior. Current foldr implementation >> would make sense in Haskell but I don't understand its reason in >> Erlang. >> >> On Thu, May 10, 2012 at 10:26 AM, Robert Virding >> wrote: >>> The documentation is referring to the *implementation* of the foldl/foldr functions, foldl uses a tail-recursive function while foldr cannot do that. The difference in speed comes from what you do inside the fold function. As others have pointed out the efficiency of ++ depends solely on its left argument. >>> >>> Robert >>> >>> ----- Original Message ----- >>>> Aha I just looked at my own explanation and realized it made no >>>> sense. Of course the slow one is slow for exactly the reason I >>>> expected. Sorry for the waste, but thank you for the clear >>>> complexity analysis. >>>> >>>> Andrew Ledvina >>>> >>>> On May 9, 2012, at 2:57 PM, "Richard O'Keefe" >>>> wrote: >>>> >>>>> >>>>> On 10/05/2012, at 7:00 AM, Andrew Ledvina wrote: >>>>> >>>>>> Okay, so I was profiling something and it led me to write this: >>>>>> >>>>>> from_left(X) -> >>>>>> ? lists:foldl(fun(Y,R) -> R ++ [{Y+1,Y*2}] end, [], X). >>>>>> >>>>>> from_right(X) -> >>>>>> ? lists:foldr(fun(Y,R) -> [{Y+1,Y*2}] ++ R end, [], X). >>>>> >>>>> Immediate reaction: ?from_left/1 is going to be atrociously slow. >>>>> >>>>>> My understanding is that foldl is preferred over foldr because it >>>>>> is tail recursive. >>>>> >>>>> ***other things being equal*** >>>>> >>>>> In this case, they are not. ?It should be clear that from_right/1 >>>>> is >>>>> O(|X|) because it does O(1) work per element of X, while >>>>> from_left/1 is O(|X|**2) because it does O(|R|) work per element of >>>>> X, and R grows from empty to 2|X|-2 elements long. >>>>> >>>>> Write [{Y+1,Y*2} || Y <- X] and let the compiler worry about the >>>>> best way to do it. >>>>> >>>>>> I was also to understand that putting the accumulator on the >>>>>> left-hand-side of the ++ operation is super bad. So, the >>>>>> following results were a bit of a surprise at first: >>>>> >>>>> If you understood that long++short is more expensive than >>>>> short++long, >>>>> you should NOT have been surprised by your result. >>>>> >>>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> -- >> Hynek Vychodil >> Chief Scientists >> >> GoodData >> n?m?st? 28. ??jna 1104/17, 602 00, Brno - ?ern? Pole >> Office: ? +420 530 50 7704 >> E-mail: ?hynek@REDACTED >> Web: ? ? www.gooddata.com >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > -- Hynek Vychodil Chief Scientists GoodData n?m?st? 28. ??jna 1104/17, 602 00, Brno - ?ern? Pole Office: ? +420 530 50 7704 E-mail: ?hynek@REDACTED Web: ? ? www.gooddata.com From paulperegud@REDACTED Thu May 10 11:53:13 2012 From: paulperegud@REDACTED (=?UTF-8?Q?Pawe=C5=82_Peregud?=) Date: Thu, 10 May 2012 11:53:13 +0200 Subject: [erlang-questions] msg mailbox and gen_svr shutdown In-Reply-To: References: Message-ID: I was having fun with supervisors yesterday (Cowboy seems to fail to fulfill the promise of not killing request processes after listener removal) and I have an example. I've only investigated the case when supervisor is killed, so YMMV. Example code is attached. You may modify it to check the behavior in your case. Start supervisor tree with exp_sup_sup:start_link(). Execute test with exp_sup_sup:test() and exp_sup_sup:test_simple(). In case of dying supervisor the answer is "no, it does not". When supervisor dies, your process is killed as via link mechanism, so it may leave some unprocessed messages in inbox. To make sure that every delivered message is served, you need to add process_flag(trap_exit, true). Messages that are sent after the moment when supervisor dies are not processed. Best regards, Paul. On May 9, 2012 11:06 AM, "Andy Richards" wrote: > Hi, > > I can't seem to see any confirmation in the documentation so was wondering > if anyone could confirm if messages are still sent to a supervised gen_svr > following a shutdown message? > > If so how do I cleanly shutdown my gen_svr without loosing messages? I > read in the supervisor child spec that a shutdown can be set to infinity > which i hoped would allow me to process the msg's in my mailbox but if I do > this will my module continue to receive messages from other processes? Is > my approach flawed and if so what other ways are there to cleanly shutting > down my gen_svr without loosing messages? > > Many thanks, > > Andy. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: example.zip Type: application/zip Size: 2372 bytes Desc: not available URL: From francesco@REDACTED Thu May 10 12:57:52 2012 From: francesco@REDACTED (Francesco Mazzoli) Date: Thu, 10 May 2012 11:57:52 +0100 Subject: [erlang-questions] `mnesia:dump_tables/1' and the schema table Message-ID: <4FAB9F30.80102@rabbitmq.com> Hi, For some reason, `mnesia:dump_tables/1' refuses to dump the schema table: https://github.com/erlang/otp/blob/maint/lib/mnesia/src/mnesia_schema.erl#L2797 . 1) Why is this the case? 2) This is not documented, see http://www.erlang.org/doc/man/mnesia.html#dump_tables-1 and http://www.erlang.org/doc/apps/mnesia/Mnesia_chap7.html#id77600 Francesco. From francesco@REDACTED Thu May 10 13:09:50 2012 From: francesco@REDACTED (Francesco Mazzoli) Date: Thu, 10 May 2012 12:09:50 +0100 Subject: [erlang-questions] dumping schema on a ram node (was: Re: `mnesia:dump_tables/1' and the schema table) In-Reply-To: <4FAB9F30.80102@rabbitmq.com> References: <4FAB9F30.80102@rabbitmq.com> Message-ID: <4FABA1FE.5070004@rabbitmq.com> OK, I think I have misunderstood the condition for dump_tables to work: "To dump the tables to disc all replicas must have the storage type ram_copies.". In this case then the check for schema makes sense since there must be one disc copy of the schema. Then my question changes to: is it possible to dump the schema of a ram node somewhere so that it will be recovered my mnesia when the node gets restarted? Francesco. On 10/05/12 11:57, Francesco Mazzoli wrote: > Hi, > > For some reason, `mnesia:dump_tables/1' refuses to dump the schema > table: > https://github.com/erlang/otp/blob/maint/lib/mnesia/src/mnesia_schema.erl#L2797 > . > > 1) Why is this the case? > 2) This is not documented, see > http://www.erlang.org/doc/man/mnesia.html#dump_tables-1 and > http://www.erlang.org/doc/apps/mnesia/Mnesia_chap7.html#id77600 > > Francesco. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dangud@REDACTED Thu May 10 13:49:52 2012 From: dangud@REDACTED (Dan Gudmundsson) Date: Thu, 10 May 2012 13:49:52 +0200 Subject: [erlang-questions] dumping schema on a ram node (was: Re: `mnesia:dump_tables/1' and the schema table) In-Reply-To: <4FABA1FE.5070004@rabbitmq.com> References: <4FAB9F30.80102@rabbitmq.com> <4FABA1FE.5070004@rabbitmq.com> Message-ID: If the schema is not on disc, mnesia never reads the disc at all. Thus dump_tables will not have any effect either. Create a backup and restore is your only option. /Dan On Thu, May 10, 2012 at 1:09 PM, Francesco Mazzoli wrote: > OK, I think I have misunderstood the condition for dump_tables to work: "To > dump the tables to disc all replicas must have the storage type > ram_copies.". In this case then the check for schema makes sense since there > must be one disc copy of the schema. > > Then my question changes to: is it possible to dump the schema of a ram node > somewhere so that it will be recovered my mnesia when the node gets > restarted? > > Francesco. > > On 10/05/12 11:57, Francesco Mazzoli wrote: >> >> Hi, >> >> For some reason, `mnesia:dump_tables/1' refuses to dump the schema >> table: >> >> https://github.com/erlang/otp/blob/maint/lib/mnesia/src/mnesia_schema.erl#L2797 >> . >> >> 1) Why is this the case? >> 2) This is not documented, see >> http://www.erlang.org/doc/man/mnesia.html#dump_tables-1 and >> http://www.erlang.org/doc/apps/mnesia/Mnesia_chap7.html#id77600 >> >> Francesco. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mrtndimitrov@REDACTED Thu May 10 14:32:02 2012 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Thu, 10 May 2012 15:32:02 +0300 Subject: [erlang-questions] Questions about ETS locking In-Reply-To: <4FAB81E1.40300@erix.ericsson.se> References: <2d34a408.e600.1371c4aa309.Coremail.cloudzen@163.com> <4FA7C64B.5090002@erix.ericsson.se> <72940294.1d37c.137348e71e5.Coremail.cloudzen@163.com> <4FAB81E1.40300@erix.ericsson.se> Message-ID: <4FABB542.9030305@gmail.com> > By default, several concurrent readers can access the same table at > the same time. > So when in the documentation it says that read_concurrency is by default false, it doesn't mean that reading from an ETS table is not concurrent? Thanks, Martin From francesco@REDACTED Thu May 10 15:05:00 2012 From: francesco@REDACTED (Francesco Mazzoli) Date: Thu, 10 May 2012 14:05:00 +0100 Subject: [erlang-questions] dumping schema on a ram node In-Reply-To: References: <4FAB9F30.80102@rabbitmq.com> <4FABA1FE.5070004@rabbitmq.com> Message-ID: <4FABBCFC.4060307@rabbitmq.com> Well, if I understand correctly dump_tables will write the content of the tables somewhere that will be read when the ram node is restarted, so I was hoping for something similar to exist. Francesco. On 10/05/12 12:49, Dan Gudmundsson wrote: > If the schema is not on disc, mnesia never reads the disc at all. > > Thus dump_tables will not have any effect either. > > Create a backup and restore is your only option. > > /Dan > > On Thu, May 10, 2012 at 1:09 PM, Francesco Mazzoli > wrote: >> OK, I think I have misunderstood the condition for dump_tables to work: "To >> dump the tables to disc all replicas must have the storage type >> ram_copies.". In this case then the check for schema makes sense since there >> must be one disc copy of the schema. >> >> Then my question changes to: is it possible to dump the schema of a ram node >> somewhere so that it will be recovered my mnesia when the node gets >> restarted? >> >> Francesco. >> >> On 10/05/12 11:57, Francesco Mazzoli wrote: >>> >>> Hi, >>> >>> For some reason, `mnesia:dump_tables/1' refuses to dump the schema >>> table: >>> >>> https://github.com/erlang/otp/blob/maint/lib/mnesia/src/mnesia_schema.erl#L2797 >>> . >>> >>> 1) Why is this the case? >>> 2) This is not documented, see >>> http://www.erlang.org/doc/man/mnesia.html#dump_tables-1 and >>> http://www.erlang.org/doc/apps/mnesia/Mnesia_chap7.html#id77600 >>> >>> Francesco. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From sverker@REDACTED Thu May 10 15:09:27 2012 From: sverker@REDACTED (Sverker Eriksson) Date: Thu, 10 May 2012 15:09:27 +0200 Subject: [erlang-questions] Questions about ETS locking In-Reply-To: <4FABB542.9030305@gmail.com> References: <2d34a408.e600.1371c4aa309.Coremail.cloudzen@163.com> <4FA7C64B.5090002@erix.ericsson.se> <72940294.1d37c.137348e71e5.Coremail.cloudzen@163.com> <4FAB81E1.40300@erix.ericsson.se> <4FABB542.9030305@gmail.com> Message-ID: <4FABBE07.5020303@erix.ericsson.se> Martin Dimitrov wrote: >> By default, several concurrent readers can access the same table at >> the same time. >> >> > > So when in the documentation it says that read_concurrency is by default > false, it doesn't mean that reading from an ETS table is not concurrent? > > True. 'write_concurrency' enables concurrent writes. 'read_concurrency' speeds up concurrent reads. Good documentation patches are always welcome... /Sverker From zabrane3@REDACTED Thu May 10 15:13:36 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 10 May 2012 15:13:36 +0200 Subject: [erlang-questions] Questions about ETS locking In-Reply-To: <4FABBE07.5020303@erix.ericsson.se> References: <2d34a408.e600.1371c4aa309.Coremail.cloudzen@163.com> <4FA7C64B.5090002@erix.ericsson.se> <72940294.1d37c.137348e71e5.Coremail.cloudzen@163.com> <4FAB81E1.40300@erix.ericsson.se> <4FABB542.9030305@gmail.com> <4FABBE07.5020303@erix.ericsson.se> Message-ID: If both are specified (write_concurreny + read_concurrency), what one should expect? How ETS will behaves? Regards, Zabrane On May 10, 2012, at 3:09 PM, Sverker Eriksson wrote: > Martin Dimitrov wrote: >>> By default, several concurrent readers can access the same table at >>> the same time. >>> >>> >> >> So when in the documentation it says that read_concurrency is by default >> false, it doesn't mean that reading from an ETS table is not concurrent? >> >> > True. > > 'write_concurrency' enables concurrent writes. > 'read_concurrency' speeds up concurrent reads. > > > Good documentation patches are always welcome... > > /Sverker > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From joaohf@REDACTED Thu May 10 19:24:07 2012 From: joaohf@REDACTED (=?UTF-8?Q?Jo=C3=A3o_Henrique_Freitas?=) Date: Thu, 10 May 2012 14:24:07 -0300 Subject: [erlang-questions] non-erlang SNMP Agents In-Reply-To: References: Message-ID: Hello, I have the some problem. My agents are net-snmp based. And I can't figure out how to change it. I think that a small change in erlang snmp app to register a agent with specific parameter to don't match the {Addr, Port} when Trap could be great. Is it the way? Thanks On Wed, Oct 12, 2011 at 8:19 AM, Daniel Widgren wrote: > Hi, > > I have some questions. > > I have a system where a non-erlang system sends snmp traps depending > on what is going on. > > I have tried to set up a snmp manager on another computer to listen to > these traps. When I do so I get: > > =INFO REPORT==== 12-Oct-2011::10:15:36 === > SNMPM default user callback received handle_agent: > > So SNMPM uses handle_agent instead of handle_trap and is wondering > why? Is it because I use snmp_default_user? > > If so, how should I configure a user to listen to a non-erlang snmp > system. Do I need a agent and is that agent the snmp system sending > traps? > > Is it possible to listen to all traps that comes from a IP without specify > port? > > regards, > Daniel > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- ----------------------------------------------------------- Jo?o Henrique Freitas - joaohf_at_gmail.com Campinas-SP-Brasil BSD051283 LPI 1 http://www.joaohfreitas.eti.br -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthias@REDACTED Thu May 10 20:15:46 2012 From: matthias@REDACTED (Matthias Lang) Date: Thu, 10 May 2012 20:15:46 +0200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: Message-ID: <20120510181546.GA3446@corelatus.se> On Monday, May 07, Jonathan Coveney wrote: > This question seems to come up now and again, and it's surprising to me > that a crucial part of the documentation isn't better documented. Of historical interest only: http://www.cs-lab.org/historical_beam_instruction_set.html Then there's this partial beam emulator in Erlang which gives some insight: https://github.com/tonyrog/beam > always ask the guys at Erjang how they went about it, Please do. AFAICT he's succeeded very well, so his opinion on how much documentation would have helped would be worth hearing. Matt From robert.virding@REDACTED Thu May 10 21:30:14 2012 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 10 May 2012 20:30:14 +0100 (BST) Subject: [erlang-questions] foldl vs foldr and ++ In-Reply-To: Message-ID: <3c4ca91f-0f20-4184-b6f0-f8ac3204ab99@knuth> These say more about it: http://www.erlang.org/doc/efficiency_guide/myths.html#tail_recursive http://www.erlang.org/doc/efficiency_guide/listHandling.html#id63380 >From the horse's mouth so to speak, Robert ----- Original Message ----- > You are right. I should realize it myself instead guessing. Compiler > has to do something really smart there. Strange thing happen when I > copied lists:foldl and lists:foldr implementation inside benchmark > module and compile native. It is far (more than twice) slower than > BEAM. > > On Thu, May 10, 2012 at 11:08 AM, Ulf Wiger > wrote: > > > > Try this benchmark program with, e.g. L = lists:seq(1,100000). > > > > -module(lr). > > -export([l/1, r/1, x/1, foldr/3]). > > > > > > l(L) -> > > ? ?timer:tc(lists, foldl, [fun erlang:'+'/2, 0, L]). > > > > r(L) -> > > ? ?timer:tc(lists, foldr, [fun erlang:'+'/2, 0, L]). > > > > x(L) -> > > ? ?timer:tc(?MODULE, foldr, [fun erlang:'+'/2, 0, L]). > > > > foldr(F, Acc, L) -> lists:foldl(F, Acc, lists:reverse(L, [])). > > > > > > lists:foldl/3 and lists:foldr/3 run a fairly even race, whereas > > lr:foldr/3 is consistenty slower. > > > > BR, > > Ulf W > > > > On 10 May 2012, at 10:53, Hynek Vychodil wrote: > > > >> When I see foldr implementation I wonder why it is not simple > >> > >> foldr(F, Acc, L) -> foldl(F, Acc, lists:reverse(L, [])). > >> > >> There should be less memory consumption. lists:reverse is BIF > >> which I > >> believe consume less memory than stack frame per each list element > >> what current foldr implementation do. It should be faster and both > >> accepts only proper lists. Only difference which I can found > >> current > >> implementation consumes memory on stack but above implementation > >> would > >> consume heap and affect GC behavior. Current foldr implementation > >> would make sense in Haskell but I don't understand its reason in > >> Erlang. > >> > >> On Thu, May 10, 2012 at 10:26 AM, Robert Virding > >> wrote: > >>> The documentation is referring to the *implementation* of the > >>> foldl/foldr functions, foldl uses a tail-recursive function > >>> while foldr cannot do that. The difference in speed comes from > >>> what you do inside the fold function. As others have pointed out > >>> the efficiency of ++ depends solely on its left argument. > >>> > >>> Robert > >>> > >>> ----- Original Message ----- > >>>> Aha I just looked at my own explanation and realized it made no > >>>> sense. Of course the slow one is slow for exactly the reason I > >>>> expected. Sorry for the waste, but thank you for the clear > >>>> complexity analysis. > >>>> > >>>> Andrew Ledvina > >>>> > >>>> On May 9, 2012, at 2:57 PM, "Richard O'Keefe" > >>>> > >>>> wrote: > >>>> > >>>>> > >>>>> On 10/05/2012, at 7:00 AM, Andrew Ledvina wrote: > >>>>> > >>>>>> Okay, so I was profiling something and it led me to write > >>>>>> this: > >>>>>> > >>>>>> from_left(X) -> > >>>>>> ? lists:foldl(fun(Y,R) -> R ++ [{Y+1,Y*2}] end, [], X). > >>>>>> > >>>>>> from_right(X) -> > >>>>>> ? lists:foldr(fun(Y,R) -> [{Y+1,Y*2}] ++ R end, [], X). > >>>>> > >>>>> Immediate reaction: ?from_left/1 is going to be atrociously > >>>>> slow. > >>>>> > >>>>>> My understanding is that foldl is preferred over foldr because > >>>>>> it > >>>>>> is tail recursive. > >>>>> > >>>>> ***other things being equal*** > >>>>> > >>>>> In this case, they are not. ?It should be clear that > >>>>> from_right/1 > >>>>> is > >>>>> O(|X|) because it does O(1) work per element of X, while > >>>>> from_left/1 is O(|X|**2) because it does O(|R|) work per > >>>>> element of > >>>>> X, and R grows from empty to 2|X|-2 elements long. > >>>>> > >>>>> Write [{Y+1,Y*2} || Y <- X] and let the compiler worry about > >>>>> the > >>>>> best way to do it. > >>>>> > >>>>>> I was also to understand that putting the accumulator on the > >>>>>> left-hand-side of the ++ operation is super bad. So, the > >>>>>> following results were a bit of a surprise at first: > >>>>> > >>>>> If you understood that long++short is more expensive than > >>>>> short++long, > >>>>> you should NOT have been surprised by your result. > >>>>> > >>>>> > >>>> _______________________________________________ > >>>> erlang-questions mailing list > >>>> erlang-questions@REDACTED > >>>> http://erlang.org/mailman/listinfo/erlang-questions > >>>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> > >> > >> -- > >> Hynek Vychodil > >> Chief Scientists > >> > >> GoodData > >> n?m?st? 28. ??jna 1104/17, 602 00, Brno - ?ern? Pole > >> Office: ? +420 530 50 7704 > >> E-mail: ?hynek@REDACTED > >> Web: ? ? www.gooddata.com > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > > http://feuerlabs.com > > > > > > > > > > -- > Hynek Vychodil > Chief Scientists > > GoodData > n?m?st? 28. ??jna 1104/17, 602 00, Brno - ?ern? Pole > Office: ? +420 530 50 7704 > E-mail: ?hynek@REDACTED > Web: ? ? www.gooddata.com > From eriksoe@REDACTED Thu May 10 22:05:12 2012 From: eriksoe@REDACTED (=?ISO-8859-1?Q?Erik_S=F8e_S=F8rensen?=) Date: Thu, 10 May 2012 22:05:12 +0200 Subject: [erlang-questions] Fwd: Is there a good source for documentation on BEAM? In-Reply-To: References: Message-ID: I began writing about the instruction set and possibly binary format at some point, but got sidetracked by trying to explain "why isn't this documented, and is that a good or a bad thing?" (I have a bit of text and graphics, though, which perhaps I ought to publish.) One interesting thing came out of that effort, though: I noticed that - there is no tail-call version of the call_fun (function object application) instruction - even so, TCO still works in beam as it should - the reason for this is that a tail-call version exists in the *internal* version of the instruction set, which is introduced by rewriting the sequence "call_fun; deallocate; return" (iirc) - Erjang had, of course, missed this the first time around, so I corrected it once I'd found out. I've been wondering whether this special case was caught in your description of beam... Of course, what I should have been wondering is whether the JS interpreter had got it right :-) That kind of irregularity is probably not unrelated to the unpublishedness of the format. On the other hand, this is one gotcha that affects not only would-be producers of beam code (which is probably the harder part to get right, when you don't know which invariants you have to maintain in the produced code), but it also affects beam consumers (which is in some ways easier, at least when you can ignore everything GC-related as it was the case with Erjang) which have to do the same kind of rewrite. 2012/5/7 Joe Armstrong > ---------- Forwarded message ---------- > From: Joe Armstrong > Date: Mon, May 7, 2012 at 10:46 AM > Subject: Re: [erlang-questions] Is there a good source for > documentation on BEAM? > To: Jonathan Coveney > > > Hi, > > I did start writing a description but it's not very complete. > > This is on my list of things-to-do-one-day-when-you-get-time > > See http://dl.dropbox.com/u/4764922/beam.pdf > > If there is any interest I could up the priority :-) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Thu May 10 22:37:55 2012 From: bob@REDACTED (Bob Ippolito) Date: Thu, 10 May 2012 13:37:55 -0700 Subject: [erlang-questions] msg mailbox and gen_svr shutdown In-Reply-To: References: Message-ID: I had the same problem with Cowboy recently, really ought to file an issue to at least fix the docs for that. For my app I wanted to implement a graceful shutdown that stopped accepting new connections but allowed existing requests to finish. The waiting part I implemented by adding a gproc local property to the workers that I wanted to wait for and then monitoring them. Looks like this: https://gist.github.com/2655724 On Thu, May 10, 2012 at 2:53 AM, Pawe? Peregud wrote: > I was having fun with supervisors yesterday (Cowboy seems to fail to > fulfill the promise of not killing request processes after listener > removal) and I have an example. I've only investigated the case when > supervisor is killed, so YMMV. Example code is attached. You may modify it > to check the behavior in your case. > > Start supervisor tree with exp_sup_sup:start_link(). Execute test with > exp_sup_sup:test() and exp_sup_sup:test_simple(). > > In case of dying supervisor the answer is "no, it does not". > > When supervisor dies, your process is killed as via link mechanism, so it > may leave some unprocessed messages in inbox. To make sure that every > delivered message is served, you need to add process_flag(trap_exit, true). > Messages that are sent after the moment when supervisor dies are not > processed. > > Best regards, > > Paul. > > On May 9, 2012 11:06 AM, "Andy Richards" > wrote: > >> Hi, >> >> I can't seem to see any confirmation in the documentation so was >> wondering if anyone could confirm if messages are still sent to a >> supervised gen_svr following a shutdown message? >> >> If so how do I cleanly shutdown my gen_svr without loosing messages? I >> read in the supervisor child spec that a shutdown can be set to infinity >> which i hoped would allow me to process the msg's in my mailbox but if I do >> this will my module continue to receive messages from other processes? Is >> my approach flawed and if so what other ways are there to cleanly shutting >> down my gen_svr without loosing messages? >> >> Many thanks, >> >> Andy. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From royen@REDACTED Fri May 11 00:01:10 2012 From: royen@REDACTED (Dick Oyen) Date: Thu, 10 May 2012 18:01:10 -0400 Subject: [erlang-questions] How to Upgrade a Windows Service Message-ID: <1E3838671188454484A9D5F6A0C0C9A905031B3A65@EXCH07.vht.virtualhold.com> I am a 2 month Erlang OTP newbie. I am trying to do my first release_handler:install_upgrade. It does not complain, but the upgrade functionality does not seem to take effect. Is there a good instruction on using rebar and release_handler in Windows? I do not have any special requirements (not trying to do distributed Erlang) and I would be joyful to just have an example project that would work. My information sources are the release_handler design principles and man pages, the github/basho/rebar, wiki.erlang-web.org/Rebar, Richard Jones' upgrade tutorial. Following some of these has not resulted in my upgrade working and others seem so oriented toward Unix or to their own sample application. Here is how I am building my release and upgrade: I have two applications and their versions are expressed in three levels, 0.1.2. The name of the application module (qss_app.erl) is not the same as the application (queue_stats_server). Maybe a problem? reltool.config contains {rel, "QSS_SYS", "0.2", [...]}. I open a git bash shell to build the release. For the first (the non-upgrade) release, I call rebar compile and rebar generate. I then copy the QSS_SYS folder from rel on the development machine and make it C:\QSS_SYS on the target machine (which had been empty of Erlang or QSS_SYS). I create a Windows service with the following command : C:\QSS_SYS\releases\0.2\qss -config C:\QSS_SYS\releases\0.2\sys.config -s qss_main" -comment "Queue Statistics Server" qss_main:start() calls application:start('QSS_SYS'). The non-upgrade release works just fine. Next I make an upgrade. I rename rel\QSS_SYS to rel\QSS_SYS_0.2. The only source code change is to write a new version number into the debug log. I change the app release from 0.1.2 to 0.1.3 in queue_stats_server.app.src. I change the release version from 0.2 to 0.3 in reltool.config. I use a git bash shell to build the upgrade. I call the following: rebar compile rebar generate rebar generate-appups previous_release=QSS_SYS_0.1 rebar generate-upgrade previous_release=QSS_SYS_0.1 Everything responds with ok. On the target machine I keep the QSS_SYS Windows service running. I copy the QSS_SYS_0.3.tar.gz to C:\QSS_SYS\Releases. I make sure the path variable points to erl under C:\QSS_SYS. I open an Erlang shell with: werl -boot c:\QSS_SYS\releases\0.2\start_clean I call the following: cd("c:\\qss\\releases"). {ok, Pid} = release_handler:start_link(). release_handler:unpack_release("QSS_SYS_0.2"). release_handler:install_release("0.2"). I think the symptoms have already started because I suspect that I should not have needed to call release_handler:start_link/0. I think there should already be one running with the QSS_SYS windows service. The fact, that I do not know the OTP-appropriate place or way to have it started, makes me think that I have not read the right documentation. I try to run release_handler:make_permanent("0.3"), but it returns {error,{service_update_failed,service_name_missmatch}}. I think that is because nonode@REDACTED = node(). I don't know much about nodes, but I think they are associated with distributed Erlang, which I am not using. It seems too much to expect that someone can see all of my mistakes from this, but I do hope that someone can point me to a reproducible example of going from nothing through a first release, through an upgrade release for an embedded Erlang system as a Windows service. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wmknapik@REDACTED Fri May 11 00:06:41 2012 From: wmknapik@REDACTED (Wojciech Knapik) Date: Fri, 11 May 2012 00:06:41 +0200 Subject: [erlang-questions] foldl vs foldr and ++ In-Reply-To: References: Message-ID: <20120510220641.GA9639@h82-143-167-72-static.e-wro.net.pl> On Wed, May 09, 2012 at 12:00:29PM -0700, Andrew Ledvina wrote: > My understanding is that foldl is preferred over foldr because it is > tail recursive. Yes. But my uderstanding has always been that this preference has little to do with performance. If anything, I'd expect the body-recursive variant to be faster in most languages. It's about memory consumption. WK From ok@REDACTED Fri May 11 00:37:32 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 11 May 2012 10:37:32 +1200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> Message-ID: On 10/05/2012, at 6:22 PM, Michael Turner wrote: > "As for what I see would cause a slowdown: the attention of the key > hackers would be spent on writing this > documentation (and then maintaining it, I assume)." > > Perhaps better: volunteers could document it (on a relatively > controlled wiki, for example). Then the "key hackers" could mention > any needed corrections. This is certainly possible, and it would be a lot better than nothing. The problem is of course that it goes backwards: volunteers can document what *is* there, not what was *meant* to be there, and cannot document why certain things that *aren't* there shouldn't be. Peitho: Here you are, Socrates, your very own orrery. Socrates: But Peitho, where is the description of how it works? Peitho: When you've figured it out, why don't you write that? Socrates: [censored] From ok@REDACTED Fri May 11 00:42:01 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 11 May 2012 10:42:01 +1200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> <7E452DB1-8AB8-44C6-8CC9-5DA99EF8AB22@cs.otago.ac.nz> Message-ID: <9E1A7A5F-4F0C-4F72-9C83-928F14123464@cs.otago.ac.nz> On 10/05/2012, at 8:28 PM, Bj?rn Gustavsson wrote: > On Thu, May 10, 2012 at 8:10 AM, Richard O'Keefe wrote: > [...] >> Joe's little document made it clear that >> (a) there are a lot more X registers allowed in Erlang that in >> Quintus Prolog; >> (b) maintaining them is more expensive than in Quintus Prolog; >> (c) the nilling instructions I expected don't exist; >> (d) there is a (temporary) space leak: if register K is live >> at an allocation point, all registers <= K are assumed to >> be live. > > There is not a temporary space leak in practice, because the compiler > will insert an instruction that will clear (set to NIL) each dead register > below K before any allocation point. And *THAT* is precisely the kind of BEAM documentation that volunteers will not be able to reconstruct without a disproportionate amount of effort: This operand gives the number of the highest live X register, so that if a garbage collection is needed, the collector knows which registers to trace from. If any X registers with smaller numbers are dead at this point, the compiler MUST ensure that they contain immediate values, by nilling them if necessary. From rexxe98@REDACTED Fri May 11 01:15:28 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Thu, 10 May 2012 16:15:28 -0700 Subject: [erlang-questions] Is there a way to test on a module's behaviors? Message-ID: I was wondering if there is a way to test on behaviors of a module. Essentially what I'm looking to do is say, if this module has x behavior, do this. I couldn't find any way to get at the module's behaviors, did I miss something or is this something that cannot be done? Thanks! Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From francesco@REDACTED Fri May 11 01:25:06 2012 From: francesco@REDACTED (Francesco Mazzoli) Date: Fri, 11 May 2012 00:25:06 +0100 Subject: [erlang-questions] Is there a way to test on a module's behaviors? In-Reply-To: References: Message-ID: <4FAC4E52.3000907@rabbitmq.com> You can use `module_info/0', amongst the other things it'll return a list of module attributes, including the behaviors. Francesco. On 11/05/12 00:15, Andrew Berman wrote: > I was wondering if there is a way to test on behaviors of a module. > Essentially what I'm looking to do is say, if this module has x > behavior, do this. I couldn't find any way to get at the module's > behaviors, did I miss something or is this something that cannot be done? > > Thanks! > > Andrew > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From rexxe98@REDACTED Fri May 11 01:53:01 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Thu, 10 May 2012 16:53:01 -0700 Subject: [erlang-questions] Is there a way to test on a module's behaviors? In-Reply-To: <4FAC4E52.3000907@rabbitmq.com> References: <4FAC4E52.3000907@rabbitmq.com> Message-ID: Awesome, thanks! On Thu, May 10, 2012 at 4:25 PM, Francesco Mazzoli wrote: > You can use `module_info/0', amongst the other things it'll return a list > of module attributes, including the behaviors. > > Francesco. > > > On 11/05/12 00:15, Andrew Berman wrote: > >> I was wondering if there is a way to test on behaviors of a module. >> Essentially what I'm looking to do is say, if this module has x >> behavior, do this. I couldn't find any way to get at the module's >> behaviors, did I miss something or is this something that cannot be done? >> >> Thanks! >> >> Andrew >> >> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri May 11 04:02:41 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 11 May 2012 14:02:41 +1200 Subject: [erlang-questions] foldl vs foldr and ++ In-Reply-To: <3c4ca91f-0f20-4184-b6f0-f8ac3204ab99@knuth> References: <3c4ca91f-0f20-4184-b6f0-f8ac3204ab99@knuth> Message-ID: <9D182CD2-CD31-411F-8641-E12E003EFDD7@cs.otago.ac.nz> On 11/05/2012, at 7:30 AM, Robert Virding wrote: > These say more about it: > > http://www.erlang.org/doc/efficiency_guide/myths.html#tail_recursive Four key points from that: the relative costs of tail and body recursion have *changed*, they are *different* on different machines, don't assume but measure, and you should probably do whatever is easiest to get your program working correctly rather than micro-optimising. I was just looking at some student Java code where he'd read a line of code as a String, reversed it so he could process it from right to left, and reversed it again for printing. Since there is no String.reverse() -- yet -- in Java, he had written something like this: StringBuilder b = new StringBuilder(s.length()); for (int i = 0; i < s.length(); i++) b = b.append(s.substring(i, 1).toCharArray()[0]); return b.toString(); This can be written more efficiently, but it's best of all just not to reverse the string in the first place. Keep the code at a level where you can see what's going on well enough to spot the really _big_ efficiency gains. From wmknapik@REDACTED Fri May 11 10:06:59 2012 From: wmknapik@REDACTED (Wojciech Knapik) Date: Fri, 11 May 2012 10:06:59 +0200 Subject: [erlang-questions] foldl vs foldr and ++ In-Reply-To: <9D182CD2-CD31-411F-8641-E12E003EFDD7@cs.otago.ac.nz> References: <3c4ca91f-0f20-4184-b6f0-f8ac3204ab99@knuth> <9D182CD2-CD31-411F-8641-E12E003EFDD7@cs.otago.ac.nz> Message-ID: <20120511080659.GA18274@h82-143-167-72-static.e-wro.net.pl> On Fri, May 11, 2012 at 02:02:41PM +1200, Richard O'Keefe wrote: > This can be written more efficiently, but it's best of all > just not to reverse the string in the first place. Keep > the code at a level where you can see what's going on well > enough to spot the really _big_ efficiency gains. Yeah, the way I see it, you should optimize for clarity in 90% of the cases and for performance when a piece of code is run many times in the course of the program's execution. So in this case: foo, bar, loop( loop(baz)) the efficiency of foo and bar doesn't really matter. Baz on the other hand needs to be relatively efficient. People get way too hung up on speed IMO. I'll take slower, but more readable code, over fast and unreadable any day. And in any "professional" setting it's cheaper to buy a faster machine and keep the code clean if the performance is not satisfactory. Of all the developers out there, those favouring declarative languages should agree... WK From fabrice.nourisson@REDACTED Fri May 11 10:36:34 2012 From: fabrice.nourisson@REDACTED (Fabrice Nourisson) Date: Fri, 11 May 2012 10:36:34 +0200 Subject: [erlang-questions] [ANN] Functional Summer School In-Reply-To: References: Message-ID: <4FACCF92.9070704@gmail.com> Hi Olivier, Have you plan to do a Haskell lounge (or FPLounge or Erlounge or ...) during those 2 weeks ? I live not far the course location. And for once that functional programmers are in the corner I would be interested to meet them. Fabrice -- Fabrice Nourisson ExtremeForge On 10/05/2012 11:31, Olivier BOUDEVILLE wrote: > > Hi all, > > With our colleagues of CEA and INRIA, we are organizing a summer > school focusing on "Functional Programming for Parallel and Concurrent > Applications". > > It is to take place from June 11 to June 22, in south of France. A few > seats are still available. > > The implementation language we selected is Haskell, but it may be > nevertheless of interest to the Erlang community, as a main purpose is > to emphasize on how the functional approach is to improve the way we > manage concurrency challenges. > > Moreover there will be at least a few Erlang developers in the > audience, and we will have the pleasure to listen there to a short > talk given by Joe Armstrong himself! > > All details are available on > http://www-hpc.cea.fr/SummerSchools2012-CS.htm > > Best regards, > > Olivier. > --------------------------- > Olivier Boudeville > > EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France > D?partement SINETICS, groupe ASICS (I2A), bureau B-226 > Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 > 47 65 27 13 > > > Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont > ?tablis ? l'intention exclusive des destinataires et les informations > qui y figurent sont strictement confidentielles. Toute utilisation de > ce Message non conforme ? sa destination, toute diffusion ou toute > publication totale ou partielle, est interdite sauf autorisation expresse. > > Si vous n'?tes pas le destinataire de ce Message, il vous est interdit > de le copier, de le faire suivre, de le divulguer ou d'en utiliser > tout ou partie. Si vous avez re?u ce Message par erreur, merci de le > supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en > garder aucune trace sur quelque support que ce soit. Nous vous > remercions ?galement d'en avertir imm?diatement l'exp?diteur par > retour du message. > > Il est impossible de garantir que les communications par messagerie > ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de > toute erreur ou virus. > ____________________________________________________ > > This message and any attachments (the 'Message') are intended solely > for the addressees. The information contained in this Message is > confidential. Any use of information contained in this Message not in > accord with its purpose, any dissemination or disclosure, either whole > or partial, is prohibited except formal approval. > > If you are not the addressee, you may not copy, forward, disclose or > use any part of it. If you have received this message in error, please > delete it and all copies from your system and notify the sender > immediately by return message. > > E-mail communication cannot be guaranteed to be timely secure, error > or virus-free. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Fabrice Nourisson Twitter : @duckmole -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Fri May 11 11:27:29 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 11 May 2012 11:27:29 +0200 Subject: [erlang-questions] foldl vs foldr and ++ In-Reply-To: <9D182CD2-CD31-411F-8641-E12E003EFDD7@cs.otago.ac.nz> References: <3c4ca91f-0f20-4184-b6f0-f8ac3204ab99@knuth> <9D182CD2-CD31-411F-8641-E12E003EFDD7@cs.otago.ac.nz> Message-ID: <47BF1E72-9119-4E6F-AAD4-D58F3C154380@feuerlabs.com> On 11 May 2012, at 04:02, Richard O'Keefe wrote: > This can be written more efficiently, but it's best of all > just not to reverse the string in the first place. Keep > the code at a level where you can see what's going on well > enough to spot the really _big_ efficiency gains. And this is of course the really important difference between foldr() and foldl(): - in which order do you want to process the list? - if you build a list in the accumulator, in which order do you want it? For most practical purposes, one should assume that they both both do their job efficiently. BR, Ulf Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From Bart.vanDeenen@REDACTED Fri May 11 11:45:23 2012 From: Bart.vanDeenen@REDACTED (Bart van Deenen) Date: Fri, 11 May 2012 11:45:23 +0200 Subject: [erlang-questions] otp application killing all spawned children, also ones not linked? Message-ID: <4FACDFB3.5080701@spilgames.com> Hi all I've set up a simple test-case at https://github.com/bvdeenen/otp_super_nukes_all that shows that an otp application:stop() actually kills *all* spawned processes by its children, even the ones that are *not linked*. The test-case consists of one gen_server (registered as par) spawning a plain erlang process (registered as par_worker) and a gen_server (registered as reg_child), which also spawns a plain erlang process (registered as child_worker). Calling application:stop(test_app) does a normal termination on the 'par' gen_server, but an exit(kill) on all others! Is this nominal behaviour? If so, where is it documented, and can I disable it? I want the processes I spawn from my gen_server (not link), to stay alive when the application terminates. Thanks Bart van Deenen From YurinVV@REDACTED Fri May 11 11:52:01 2012 From: YurinVV@REDACTED (Slava Yurin) Date: Fri, 11 May 2012 16:52:01 +0700 Subject: [erlang-questions] otp application killing all spawned children, also ones not linked? In-Reply-To: <4FACDFB3.5080701@spilgames.com> References: <4FACDFB3.5080701@spilgames.com> Message-ID: <674441336729921@web12e.yandex.ru> An HTML attachment was scrubbed... URL: From YurinVV@REDACTED Fri May 11 12:12:25 2012 From: YurinVV@REDACTED (Slava Yurin) Date: Fri, 11 May 2012 17:12:25 +0700 Subject: [erlang-questions] otp application killing all spawned children, also ones not linked? In-Reply-To: <4FACDFB3.5080701@spilgames.com> References: <4FACDFB3.5080701@spilgames.com> Message-ID: <423491336731145@web16f.yandex.ru> An HTML attachment was scrubbed... URL: From Bart.vanDeenen@REDACTED Fri May 11 12:27:32 2012 From: Bart.vanDeenen@REDACTED (Bart van Deenen) Date: Fri, 11 May 2012 12:27:32 +0200 Subject: [erlang-questions] otp application killing all spawned children, also ones not linked? In-Reply-To: <423491336731145@web16f.yandex.ru> References: <4FACDFB3.5080701@spilgames.com> <423491336731145@web16f.yandex.ru> Message-ID: <4FACE994.8070206@spilgames.com> Thanks!!! I've learned a lot tracking this problem. Bart On 05/11/2012 12:12 PM, Slava Yurin wrote: > Dirty hack for processes don't kill by application:stop/1: > diff --git a/par.erl b/par.erl > index f2232c2..207b5c7 100644 > --- a/par.erl > +++ b/par.erl > @@ -41,6 +41,7 @@ code_change(_OldVsn, State, _Extra) -> {ok, State}. > worker() -> > register(par_worker, self()), > + group_leader(whereis(init), self()), > receive Any -> io:format("~p Parent worker received ~p~n", > [self(), Any]) end. From sverker.eriksson@REDACTED Fri May 11 14:37:46 2012 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Fri, 11 May 2012 14:37:46 +0200 Subject: [erlang-questions] Questions about ETS locking In-Reply-To: References: <2d34a408.e600.1371c4aa309.Coremail.cloudzen@163.com> <4FA7C64B.5090002@erix.ericsson.se> <72940294.1d37c.137348e71e5.Coremail.cloudzen@163.com> <4FAB81E1.40300@erix.ericsson.se> <4FABB542.9030305@gmail.com> <4FABBE07.5020303@erix.ericsson.se> Message-ID: <4FAD081A.1030302@erix.ericsson.se> Zabrane Mickael wrote: > If both are specified (write_concurreny + read_concurrency), what one should expect? > Both. > How ETS will behaves? > > ETS will * use fine grained locking within the table (write_concurrency) at the expense of the overhead from that extra layer of locking. * use read-optimized locks (read_concurrency) at the expense of heavier mode switches for the locks (read <-> write). http://www.erlang.org/doc/man/ets.html#new_2_read_concurrency: "The read_concurrency option can be combined with the write_concurrency option. You typically want to combine these when large concurrent read bursts and large concurrent write bursts are common." /Sverker, Erlang/OTP Ericsson From lists@REDACTED Fri May 11 16:03:14 2012 From: lists@REDACTED (lists@REDACTED) Date: Fri, 11 May 2012 16:03:14 +0200 Subject: [erlang-questions] Directory layout for installing 3rd party Erlang libs Message-ID: <20120511160314.07d16c68@mygoo> Hi all, I'm currently trying to improve the support my favorite Linux distro's (Gentoo) support for Erlang by creating packages for some Erlang-related tools and libraries. At the moment, there are hardly any of them available through our package manager (just Erlang/OTP itself, esdl, tsung and ejabberd). For this reason, there is no convention on the directory layout yet: Where to put the beam files of 3rd party Erlang libs, like getopt, esdl or erlware_commons? There are two (main) possibilities: 1) Put them to the same dir were the libs from Erlang/OTP itself reside --> /usr/lib/erlang/lib/getopt-0.4.3 2) Create an own directory for them and add it to the ERL_LIBS environment variable --> e.g. /usr/lib/erlang-site/getopt-0.4.3 Debian uses the first approach. But I'd prefer the second, because I think it would be better to separate Erlang/OTP and 3rd-party libs, even if they are all handled by a package manager. Other programming languages, for example Python or Perl, use an approach similar to 2) (at least on Gentoo): Python libs go into /usr/lib/python2.7/site-packages, Perl libs use /usr/lib/perl5/site_perl. Does anybody know about drawbacks and other pros/cons of the two approaches? Thanks in advance, aranea From lemenkov@REDACTED Fri May 11 16:24:49 2012 From: lemenkov@REDACTED (Peter Lemenkov) Date: Fri, 11 May 2012 18:24:49 +0400 Subject: [erlang-questions] Directory layout for installing 3rd party Erlang libs In-Reply-To: <20120511160314.07d16c68@mygoo> References: <20120511160314.07d16c68@mygoo> Message-ID: Hello. 2012/5/11 : > There are two (main) possibilities: > 1) Put them to the same dir were the libs from Erlang/OTP itself reside > ? --> /usr/lib/erlang/lib/getopt-0.4.3 > 2) Create an own directory for them and add it to the ERL_LIBS > ? environment variable > ? --> e.g. /usr/lib/erlang-site/getopt-0.4.3 > > Debian uses the first approach. Fedora/EPEL does the same. > think it would be better to separate Erlang/OTP and 3rd-party libs, Why do you think it's better? Could you please elaborate on this? > Does anybody know about drawbacks and other pros/cons of the two > approaches? Increasing complexity is an obvious drawback of the second option. However the former one doesn't handle arch-independent packages in the optimal way. -- With best regards, Peter Lemenkov. From cao.xu@REDACTED Fri May 11 17:58:27 2012 From: cao.xu@REDACTED (cao xu) Date: Fri, 11 May 2012 23:58:27 +0800 Subject: [erlang-questions] Driver load problem on Mac OS Lion Message-ID: <054C72CB-D24C-498D-9879-9FDDDFCF41B9@rytong.com> Hi, I tried to update the OS to Lion, but found drivers those work on 10.6 have problem during the start. The following error prompted and the VM crashed : Erlang R15B (erts-5.9) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9 (abort with ^G) 1> erl_ddll:load_driver("priv", xml_eng_drv). Trace/BPT trap: 5 Has it happened to anyone else? I also found some of the drivers still work, is there something I should fix when migrate linked-in drivers to Mac OS Lion? Any suggestion will be appreciated. BR Xu Cao From christopher.faulet@REDACTED Fri May 11 21:18:19 2012 From: christopher.faulet@REDACTED (Christopher Faulet) Date: Fri, 11 May 2012 21:18:19 +0200 Subject: [erlang-questions] Multiple embedded Yaws instances In-Reply-To: <4FA96AAE.50907@hakwerk.com> References: <4FA96AAE.50907@hakwerk.com> Message-ID: <4FAD65FB.3070105@capflam.org> On 08/05/2012 20:49, Arjan Hakkesteegt wrote: > Hi, > > I am trying to setup a Common Test mock layer around my SUT (System > Under Test). Both the SUT and CT should start an embedded Yaws instance > because both should both act as client and as server for SOAP. > Client(SUT) -> Server(CT) > Server(SUT) <- Client(CT) > > However, it appears that it is not possible to run more than 1 embedded > Yaws within a VM? When starting the second instance I get > already_started errors from the supervisor. I tried to modify the > ChildSpecs returned by yaws_api:embedded_start_conf/4 to append the Yaws > instance Id to the supervisor child Ids, but still get the > already_started errors. I tried it like this: > [{yaws_log_myid, > {yaws_log,start_link,[]}, > permanent,5000,worker, > [yaws_log]}, > {yaws_server_myid, > ...}] > > I've used multiple SConfs within a single embedded Yaws instance before, > but I don't think that this is an option here as my SUT and the CT suite > are completely separate applications and the SUT should be seen as a > blackbox. > > I guess it could be an option to start the SUT in its own Erlang VM, but > how would I start and stop this VM from the Common Test one (from > init_per_suite/end_per_suite)? > > Any solutions to this problem? Or am I missing something obvious? > Hi, Unfortunately, it's not possible to have multiple instances of Yaws on the same VM, there are some named processes. I never used the soap server of Yaws but it is possible to run several services. Each one can be registered using yaws_soap_srv:setup/2. So you can have a service for your client and another for your server. You can find some information on Yaws website: http://yaws.hyber.org/soap_intro.yaws -- Christopher From roberto@REDACTED Fri May 11 22:01:17 2012 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 11 May 2012 13:01:17 -0700 Subject: [erlang-questions] prettypr Message-ID: Dear All, is there a way to use prettypr on a list()? I want to pretty format some source code, all I found is a Trapexit post showing this: F="tst.erl", erl_prettypr:format(erl_recomment:recomment_forms(element(2,epp_dodger:parse_file(F)),erl_comment_scan:file(F))). Unfortunately this is based on parsing a file, and epp_dodger needs an IoDevice to parse code. Is there a way to do something similar but inputting the code as list() instead of parsing a file? r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Fri May 11 22:09:36 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 11 May 2012 22:09:36 +0200 Subject: [erlang-questions] prettypr In-Reply-To: References: Message-ID: You could take a look at how parse_trans does it: https://github.com/esl/parse_trans/blob/master/src/parse_trans_pp.erl#L65 It uses erl_pp instead of erl_prettypr for reasons I have since forgotten (it was something that, at one time, erl_prettypr had problems with, but erl_pp was able to handle). BR, Ulf W On 11 May 2012, at 22:01, Roberto Ostinelli wrote: > Dear All, > > is there a way to use prettypr on a list()? > > I want to pretty format some source code, all I found is a Trapexit post showing this: > > F="tst.erl", > erl_prettypr:format(erl_recomment:recomment_forms(element(2,epp_dodger:parse_file(F)),erl_comment_scan:file(F))). > > Unfortunately this is based on parsing a file, and epp_dodger needs an IoDevice to parse code. > > Is there a way to do something similar but inputting the code as list() instead of parsing a file? > > r. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From straightflush@REDACTED Fri May 11 22:22:34 2012 From: straightflush@REDACTED (AD) Date: Fri, 11 May 2012 16:22:34 -0400 Subject: [erlang-questions] listen to http and non-http tcp connections in cowboy Message-ID: hello, We are looking to implement a flash policy server on our cowboy websocket server. Using websocket-js, the flash swf on the client will communicate over raw TCP socket to get a policy file back. Is there a way to have a cowboy server set up to serve HTTP requests, but to also be able to deal with non-http in the same instance and same port? Thanks -AD -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri May 11 22:28:28 2012 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 11 May 2012 13:28:28 -0700 Subject: [erlang-questions] prettypr In-Reply-To: References: Message-ID: Thank you Ulf. So i first should erl_parse:tokens/1, then erl_parse:parse/form/1 then use what you suggested? On Fri, May 11, 2012 at 1:09 PM, Ulf Wiger wrote: > > You could take a look at how parse_trans does it: > > https://github.com/esl/parse_trans/blob/master/src/parse_trans_pp.erl#L65 > > It uses erl_pp instead of erl_prettypr for reasons I have since forgotten > (it was something that, at one time, erl_prettypr had problems with, but > erl_pp was able to handle). > > BR, > Ulf W > > On 11 May 2012, at 22:01, Roberto Ostinelli wrote: > > Dear All, > > is there a way to use prettypr on a list()? > > I want to pretty format some source code, all I found is a Trapexit post > showing this: > > F="tst.erl", > > erl_prettypr:format(erl_recomment:recomment_forms(element(2,epp_dodger:parse_file(F)),erl_comment_scan:file(F))). > > Unfortunately this is based on parsing a file, and epp_dodger needs an > IoDevice to parse code. > > Is there a way to do something similar but inputting the code as list() > instead of parsing a file? > > r. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Fri May 11 23:03:10 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 11 May 2012 23:03:10 +0200 Subject: [erlang-questions] prettypr In-Reply-To: References: Message-ID: <43D55E53-906B-4F33-BF03-601DE77FA53B@feuerlabs.com> Hmm, so the code is in text form now? E.g. Src = "foo() ->\n foo.\n"? Well, you could do it that way: Eshell V5.9 (abort with ^G) 1> Src = "add(X, Y) ->\n X + Y.\n". "add(X, Y) ->\n X + Y.\n" 2> erl_scan:string(Src). {ok,[{atom,1,add},...],3} 3> Ts = element(2,v(2)). [{atom,1,add}, {'(',1},... {dot,2}] 4> erl_parse:parse_form(Ts). {ok,{function,1,add,2, [{clause,1, [{var,1,'X'},{var,1,'Y'}], [], [{op,2,'+',{var,2,'X'},{var,2,'Y'}}]}]}} 5> io:fwrite("~s~n", [erl_pp:form(element(2,v(4)))]). add(X, Y) -> X + Y. ok ?but it will only work as long as you don't rely on the preprocessor for anything in the code, and only for one form at a time. Full parsing, including preprocessing, from string input is not that easy. I did it in erlhive, but then used a ram_file_io_server (which I've long thought should be part of OTP): http://erlhive.svn.sourceforge.net/viewvc/erlhive/trunk/lib/erlhive/src/erlhive_compile.erl?revision=70&view=markup (line 123) http://erlhive.svn.sourceforge.net/viewvc/erlhive/trunk/lib/erlhive/src/erlhive_ram_file_io_server.erl?revision=57&view=markup BR, Ulf W On 11 May 2012, at 22:28, Roberto Ostinelli wrote: > Thank you Ulf. > > So i first should erl_parse:tokens/1, then erl_parse:parse/form/1 then use what you suggested? > > > On Fri, May 11, 2012 at 1:09 PM, Ulf Wiger wrote: > > You could take a look at how parse_trans does it: > > https://github.com/esl/parse_trans/blob/master/src/parse_trans_pp.erl#L65 > > It uses erl_pp instead of erl_prettypr for reasons I have since forgotten > (it was something that, at one time, erl_prettypr had problems with, but > erl_pp was able to handle). > > BR, > Ulf W > > On 11 May 2012, at 22:01, Roberto Ostinelli wrote: > >> Dear All, >> >> is there a way to use prettypr on a list()? >> >> I want to pretty format some source code, all I found is a Trapexit post showing this: >> >> F="tst.erl", >> erl_prettypr:format(erl_recomment:recomment_forms(element(2,epp_dodger:parse_file(F)),erl_comment_scan:file(F))). >> >> Unfortunately this is based on parsing a file, and epp_dodger needs an IoDevice to parse code. >> >> Is there a way to do something similar but inputting the code as list() instead of parsing a file? >> >> r. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Fri May 11 23:15:18 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 11 May 2012 23:15:18 +0200 Subject: [erlang-questions] prettypr In-Reply-To: References: Message-ID: <69DCD66D-609A-4C59-94AC-DE5CF93B0CCE@feuerlabs.com> If the code you want to pretty-print is generated from other code - and you don't want to create abstract forms by hand (most don't) - I can also recommend parse_trans_codegen: https://github.com/esl/parse_trans/blob/master/doc/parse_trans_codegen.md It's a fairly easy, and less error-prone, way to generate code. If you look e.g. at https://github.com/esl/parse_trans/blob/master/examples/ex_codegen.erl we can try the two first functions: f(Name) -> codegen:gen_function( Name, fun(1,2,3) -> foo; (A,B,C) -> {A,B,C} end). g(Name, V) -> codegen:gen_function( Name, fun(L) -> member({'$var',V}, L) end). Eshell V5.9 (abort with ^G) 1> c(ex_codegen). {ok,ex_codegen} 2> ex_codegen:f(foo). {function,9,foo,3, [{clause,11, [{integer,11,1},{integer,11,2},{integer,11,3}], [], [{atom,12,foo}]}, {clause,13, [{var,13,'A'},{var,13,'B'},{var,13,'C'}], [], [{tuple,14,[{var,14,'A'},{var,14,'B'},{var,14,'C'}]}]}]} 3> ex_codegen:g(is_in, foo). {function,20,is_in,1, [{clause,22, [{var,22,'L'}], [], [{call,23,{atom,23,member},[{atom,23,foo},{var,23,'L'}]}]}]} 4> io:fwrite("~s~n", [erl_pp:form(v(3))]). is_in(L) -> member(foo, L). ?you get the idea. Now, since I actually don't know exactly what you are trying to do, I will quit throwing out suggestions. :) BR, Ulf W On 11 May 2012, at 22:28, Roberto Ostinelli wrote: > Thank you Ulf. > > So i first should erl_parse:tokens/1, then erl_parse:parse/form/1 then use what you suggested? > > > On Fri, May 11, 2012 at 1:09 PM, Ulf Wiger wrote: > > You could take a look at how parse_trans does it: > > https://github.com/esl/parse_trans/blob/master/src/parse_trans_pp.erl#L65 > > It uses erl_pp instead of erl_prettypr for reasons I have since forgotten > (it was something that, at one time, erl_prettypr had problems with, but > erl_pp was able to handle). > > BR, > Ulf W > > On 11 May 2012, at 22:01, Roberto Ostinelli wrote: > >> Dear All, >> >> is there a way to use prettypr on a list()? >> >> I want to pretty format some source code, all I found is a Trapexit post showing this: >> >> F="tst.erl", >> erl_prettypr:format(erl_recomment:recomment_forms(element(2,epp_dodger:parse_file(F)),erl_comment_scan:file(F))). >> >> Unfortunately this is based on parsing a file, and epp_dodger needs an IoDevice to parse code. >> >> Is there a way to do something similar but inputting the code as list() instead of parsing a file? >> >> r. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From royen@REDACTED Fri May 11 23:37:13 2012 From: royen@REDACTED (Dick Oyen) Date: Fri, 11 May 2012 17:37:13 -0400 Subject: [erlang-questions] release_handler:make_permanent/1 and erlsrv lead me to inInvalid node name Message-ID: <1E3838671188454484A9D5F6A0C0C9A905031B3ED9@EXCH07.vht.virtualhold.com> I have a release running as a Windows service named QSS, because I had called erlsrv add QSS -.... It is version 0.1. I install an upgrade and then call release_handler:make_permanent("0.2"). This fails because it tries to find a non-existant Windows service named QSS_0.1. Now I start guessing at what to do about it, so I take everything out of the target machine and reinstall the original release again, but this time I call erlsrv add QSS_0.1 -.... When I go to the Windows service manager and try to start the service, it fails with "Invalid node name: ~p~n",['QSS_0.1@REDACTED']} What can I do to satisfy make_permanent/1? -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric@REDACTED Sat May 12 01:55:52 2012 From: eric@REDACTED (Eric Moritz) Date: Fri, 11 May 2012 19:55:52 -0400 Subject: [erlang-questions] listen to http and non-http tcp connections in cowboy In-Reply-To: References: Message-ID: I haven't tried this but you should be able to run start_listener multiple times for each server. On Fri, May 11, 2012 at 4:22 PM, AD wrote: > hello, > > We are looking to implement a flash policy server on our cowboy websocket > server. Using websocket-js, the flash swf on the client will communicate > over raw TCP socket to get a policy file back. Is there a way to have a > cowboy server set up to serve HTTP requests, but to also be able to deal > with non-http in the same instance and same port? > > Thanks > -AD > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Sat May 12 02:11:08 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Sat, 12 May 2012 02:11:08 +0200 Subject: [erlang-questions] listen to http and non-http tcp connections in cowboy In-Reply-To: References: Message-ID: <4FADAA9C.8070201@ninenines.eu> The way I usually suggest this is to start a listener for your flash_policy_server protocol that would detect what protocol is used from the first few bytes and redirect to cowboy_http_protocol if it's HTTP (calling init, not start_link). You'll probably need to patch cowboy_http_protocol a little (I'm interested in such a patch) or use inet_tcp:unrecv/2 to put the data you read back into the socket buffer so that cowboy_http_protocol can run directly. On 05/11/2012 10:22 PM, AD wrote: > hello, > > We are looking to implement a flash policy server on our cowboy > websocket server. Using websocket-js, the flash swf on the client will > communicate over raw TCP socket to get a policy file back. Is there a > way to have a cowboy server set up to serve HTTP requests, but to also > be able to deal with non-http in the same instance and same port? > > Thanks > -AD > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin Erlang Cowboy Nine Nines From rexxe98@REDACTED Sat May 12 03:33:26 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 11 May 2012 18:33:26 -0700 Subject: [erlang-questions] prettypr In-Reply-To: <69DCD66D-609A-4C59-94AC-DE5CF93B0CCE@feuerlabs.com> References: <69DCD66D-609A-4C59-94AC-DE5CF93B0CCE@feuerlabs.com> Message-ID: What about this? Comments = erl_comment_scan:string(Text), {ok, Form, _} = erl_scan:string(Text), {ok, Tree} = erl_parse:parse_form(Form), {Tree2, _} = erl_recomment:recomment_tree(Tree, Comments), erl_prettypr:format(Tree2). --Andrew On Fri, May 11, 2012 at 2:15 PM, Ulf Wiger wrote: > > If the code you want to pretty-print is generated from other code - and > you don't want to create abstract forms by hand (most don't) - I can also > recommend parse_trans_codegen: > > https://github.com/esl/parse_trans/blob/master/doc/parse_trans_codegen.md > > It's a fairly easy, and less error-prone, way to generate code. > > If you look e.g. at > https://github.com/esl/parse_trans/blob/master/examples/ex_codegen.erl > > we can try the two first functions: > > f(Name) -> > codegen:gen_function( > Name, > fun(1,2,3) -> > foo; > (A,B,C) -> > {A,B,C} > end). > > g(Name, V) -> > codegen:gen_function( > Name, > fun(L) -> > member({'$var',V}, L) > end). > > > Eshell V5.9 (abort with ^G) > 1> c(ex_codegen). > {ok,ex_codegen} > 2> ex_codegen:f(foo). > {function,9,foo,3, > [{clause,11, > [{integer,11,1},{integer,11,2},{integer,11,3}], > [], > [{atom,12,foo}]}, > {clause,13, > [{var,13,'A'},{var,13,'B'},{var,13,'C'}], > [], > [{tuple,14,[{var,14,'A'},{var,14,'B'},{var,14,'C'}]}]}]} > 3> ex_codegen:g(is_in, foo). > {function,20,is_in,1, > [{clause,22, > [{var,22,'L'}], > [], > > [{call,23,{atom,23,member},[{atom,23,foo},{var,23,'L'}]}]}]} > 4> io:fwrite("~s~n", [erl_pp:form(v(3))]). > is_in(L) -> > member(foo, L). > > > ?you get the idea. > > Now, since I actually don't know exactly what you are trying to do, I will > quit throwing out suggestions. :) > > BR, > Ulf W > > On 11 May 2012, at 22:28, Roberto Ostinelli wrote: > > Thank you Ulf. > > So i first should erl_parse:tokens/1, then erl_parse:parse/form/1 then use > what you suggested? > > > On Fri, May 11, 2012 at 1:09 PM, Ulf Wiger wrote: > >> >> You could take a look at how parse_trans does it: >> >> >> https://github.com/esl/parse_trans/blob/master/src/parse_trans_pp.erl#L65 >> >> It uses erl_pp instead of erl_prettypr for reasons I have since forgotten >> (it was something that, at one time, erl_prettypr had problems with, but >> erl_pp was able to handle). >> >> BR, >> Ulf W >> >> On 11 May 2012, at 22:01, Roberto Ostinelli wrote: >> >> Dear All, >> >> is there a way to use prettypr on a list()? >> >> I want to pretty format some source code, all I found is a Trapexit post >> showing this: >> >> F="tst.erl", >> >> erl_prettypr:format(erl_recomment:recomment_forms(element(2,epp_dodger:parse_file(F)),erl_comment_scan:file(F))). >> >> Unfortunately this is based on parsing a file, and epp_dodger needs an >> IoDevice to parse code. >> >> Is there a way to do something similar but inputting the code as list() >> instead of parsing a file? >> >> r. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >> http://feuerlabs.com >> >> >> >> > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cao.xu@REDACTED Sat May 12 04:20:49 2012 From: cao.xu@REDACTED (cao xu) Date: Sat, 12 May 2012 10:20:49 +0800 Subject: [erlang-questions] Driver load problem on Mac OS Lion In-Reply-To: <054C72CB-D24C-498D-9879-9FDDDFCF41B9@rytong.com> References: <054C72CB-D24C-498D-9879-9FDDDFCF41B9@rytong.com> Message-ID: I got the stack info from core file when the VM crashed: (gdb) up #1 0x000000001c0c77c1 in erts_sys_main_thread () at sys.c:2905 2905 res = select(erts_darwin_main_thread_pipe[0] + 1, &readfds, NULL, NULL, NULL); (gdb) up #2 0x000000001bfe5bc4 in erl_start (argc=12, argv=0x7fff6bbcb5a8) at erl_init.c:1483 1483 erts_sys_main_thread(); /* May or may not return! */ (gdb) up #3 0x6c61636f6c2f7273 in ?? () (gdb) and after I turned off the SMP, the driver worked again. Does that mean it have something to do with the SMP support? BR ? 2012-5-11???11:58? cao xu ??? > Hi, > > I tried to update the OS to Lion, but found drivers those work on 10.6 have problem during the start. The following error prompted and the VM crashed : > > Erlang R15B (erts-5.9) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.9 (abort with ^G) > 1> erl_ddll:load_driver("priv", xml_eng_drv). > Trace/BPT trap: 5 > > Has it happened to anyone else? I also found some of the drivers still work, is there something I should fix when migrate linked-in drivers to Mac OS Lion? Any suggestion will be appreciated. > > BR > > Xu Cao > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From barcojie@REDACTED Sat May 12 08:47:36 2012 From: barcojie@REDACTED (Barco You) Date: Sat, 12 May 2012 14:47:36 +0800 Subject: [erlang-questions] [Zotonic] Can not start up zotonic after compilation Message-ID: Dear Zotonic users, After successfully compiling the zotonic project with make command I tried to start up the zotonic, but the erl can not start the application as following: Erlang R16B (erts-5.10) [source] [64-bit] [smp:4:4] [async-threads:0] [kernel-poll:false] *** ERROR: Shell process terminated! (^G to start new job) *** Any one has idea about this ? thank you! Best regards, Barco -------------- next part -------------- An HTML attachment was scrubbed... URL: From arun11299@REDACTED Sat May 12 09:34:40 2012 From: arun11299@REDACTED (Arun Muralidharan) Date: Sat, 12 May 2012 13:04:40 +0530 Subject: [erlang-questions] Writing platform independent Socket based binary parsing code Message-ID: Hi Folks, I have an erlang application which includes parsing of binary data from socket (both TCP and UDP). The binary data is as per some specific protocol, so my options for TCP and UDP sock is as per below: TCP sock : Opts = [binary, {packet, 0}, {reuseaddr, true}, {keepalive, true}, {backlog, 30}, {active, false}], UDP sock : [binary,{active,once},{recbuf,2097152}] Now, when i parse the data I get from socket, I do like this (On UNIX): << Version:4/big-unsigned-integer-unit:8, Length:2/big-unsigned-integer:8, _Rest/binary >> = Bin_event_From_Socket. Now, this would give me a problem when run on LINUX as LINUX is little endian based. So, the option here for me is to convert 'big' to 'little' in the above code. But, Erlang being a VM based language, I must be having some facility to write platform independent code. I searched the NEt but couldnt find much on it. So, the question here is how can i write a platform independent sockaet data parsing code ? Thanks. -Arun -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Sat May 12 10:22:03 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Sat, 12 May 2012 11:22:03 +0300 Subject: [erlang-questions] Writing platform independent Socket based binary parsing code In-Reply-To: References: Message-ID: <0E282F28-E417-4E19-84CA-E45204C6871D@gmail.com> Hello, I'd like to quote one statement here: "Network stacks and communication protocols must also define their endianness. Otherwise, two nodes of different endianness would be unable to communicate." As an example, TCP/IP protocol suite is defined to be a big-endian. So, the parsing of protocol primitive shall not be dependent on CPU architecture. E.g. If you protocol say that version is serialized into big-endian then your parsing code shall be same and it is platform independent. - Dmitry P.S: ;-) http://geekandpoke.typepad.com/geekandpoke/2011/09/simply-explained-1.html On May 12, 2012, at 10:34 AM, Arun Muralidharan wrote: > Hi Folks, > > I have an erlang application which includes parsing of binary data from socket (both TCP and UDP). The binary data is as per some specific protocol, so my options for TCP and UDP sock is as per below: > > > TCP sock : > Opts = [binary, {packet, 0}, {reuseaddr, true}, > {keepalive, true}, {backlog, 30}, {active, false}], > UDP sock : > [binary,{active,once},{recbuf,2097152}] > > Now, when i parse the data I get from socket, I do like this (On UNIX): > > << Version:4/big-unsigned-integer-unit:8, > Length:2/big-unsigned-integer:8, > _Rest/binary >> = Bin_event_From_Socket. > > Now, this would give me a problem when run on LINUX as LINUX is little endian based. So, the option here for me is to convert 'big' to 'little' in the above code. > > But, Erlang being a VM based language, I must be having some facility to write platform independent code. I searched the NEt but couldnt find much on it. > > So, the question here is how can i write a platform independent sockaet data parsing code ? > > Thanks. > > -Arun > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists@REDACTED Sat May 12 11:08:04 2012 From: lists@REDACTED (lists@REDACTED) Date: Sat, 12 May 2012 11:08:04 +0200 Subject: [erlang-questions] Directory layout for installing 3rd party Erlang libs In-Reply-To: References: <20120511160314.07d16c68@mygoo> Message-ID: <20120512110804.23073512@mygoo> Thanks for your opinion. I'm not the first who had this idea of separating 3rd party libraries from the ones a programming language comes with ? for example, Python, Perl or Ruby provide a way to do that (not only on my distro, it's really part of their install scripts). I just don't want to clutter the "official" directorys with stuff which is not part of the main distribution. Of course this adds a bit of complexity, but on the other hand a clear structure is also very helpful sometimes. Regards, aranea PS: Aren't there any other opinions on this matter? From arun11299@REDACTED Sat May 12 11:13:21 2012 From: arun11299@REDACTED (Arun Muralidharan) Date: Sat, 12 May 2012 14:43:21 +0530 Subject: [erlang-questions] Writing platform independent Socket based binary parsing code In-Reply-To: <0E282F28-E417-4E19-84CA-E45204C6871D@gmail.com> References: <0E282F28-E417-4E19-84CA-E45204C6871D@gmail.com> Message-ID: Thanks Dmitry for the quick response. I am aware of the fact that from network we get data in our application in Big Endian format and in c/c++ I would use ntohs/ntohl functions for converting it into a format that the OS supports. But, the same in Erlang is confusing me (no idea why !!). As per your last statement *"So, the parsing of protocol primitive shall not be dependent on CPU architecture. E.g. If you protocol say that version is serialized into big-endian then your parsing code shall be same and it is platform independent. " *do you want to say that, as I am parsing data from network socket I should use "<> = Sock_Data." as the data in network is serialized in big endian format ? Really sorry if I am sounding naive..but somehow i am having trouble understanding this in Erlang. Thanks, Arun On Sat, May 12, 2012 at 1:52 PM, Dmitry Kolesnikov wrote: > Hello, > > I'd like to quote one statement here: "Network stacks and communication > protocols must also define their endianness. Otherwise, two nodes of > different endianness would be unable to communicate." > As an example, TCP/IP protocol suite is defined to be a big-endian. > > So, the parsing of protocol primitive shall not be dependent on CPU > architecture. E.g. If you protocol say that version is serialized into > big-endian then your parsing code shall be same and it is platform > independent. > > - Dmitry > > P.S: ;-) > http://geekandpoke.typepad.com/geekandpoke/2011/09/simply-explained-1.html > > > On May 12, 2012, at 10:34 AM, Arun Muralidharan wrote: > > Hi Folks, > > I have an erlang application which includes parsing of binary data from > socket (both TCP and UDP). The binary data is as per some specific > protocol, so my options for TCP and UDP sock is as per below: > > TCP sock : > Opts = [binary, {packet, 0}, {reuseaddr, true}, > {keepalive, true}, {backlog, 30}, {active, false}], > > UDP sock : > [binary,{active,once},{recbuf,2097152}] > > > Now, when i parse the data I get from socket, I do like this (On UNIX): > > << Version:4/big-unsigned-integer-unit:8, > Length:2/big-unsigned-integer:8, > _Rest/binary >> = Bin_event_From_Socket. > > > Now, this would give me a problem when run on LINUX as LINUX is little > endian based. So, the option here for me is to convert 'big' to 'little' in > the above code. > > But, Erlang being a VM based language, I must be having some facility to > write platform independent code. I searched the NEt but couldnt find much > on it. > So, the question here is how can i write a platform independent sockaet > data parsing code ? > > Thanks. > -Arun > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemenkov@REDACTED Sat May 12 11:27:55 2012 From: lemenkov@REDACTED (Peter Lemenkov) Date: Sat, 12 May 2012 13:27:55 +0400 Subject: [erlang-questions] Directory layout for installing 3rd party Erlang libs In-Reply-To: <20120512110804.23073512@mygoo> References: <20120511160314.07d16c68@mygoo> <20120512110804.23073512@mygoo> Message-ID: 2012/5/12 : > Thanks for your opinion. > > I'm not the first who had this idea of separating 3rd party libraries > from the ones a programming language comes with ? for example, Python, > Perl or Ruby provide a way to do that (not only on my distro, it's > really part of their install scripts). I just don't want to clutter the > "official" directorys with stuff which is not part of the main > distribution. > > Of course this adds a bit of complexity, but on the other hand a clear > structure is also very helpful sometimes. Let me bottom line this for everyone - no technical motivation behind that. If so then this proposal must be rejected. -- With best regards, Peter Lemenkov. From watson.timothy@REDACTED Sat May 12 11:45:29 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 12 May 2012 10:45:29 +0100 Subject: [erlang-questions] Directory layout for installing 3rd party Erlang libs In-Reply-To: References: <20120511160314.07d16c68@mygoo> <20120512110804.23073512@mygoo> Message-ID: <791E493E-0CD9-4D37-B76C-13ADD90C6F1E@gmail.com> Peter, one motivation is to handle installed library apps separately so that multiple otp installations can share them (allowing for potential incompatibility). IIRC a lot of experienced folks have recommend this over installing to lib_dir and there are probably other factors. Not least of these could be permissions - I don't want my installs to belong to the standard user/group, but I do want it to be easy to interact with ERL_LIBS without needing sudo et al. Bottom line is, it depends on what people expect. On 12 May 2012, at 10:27, Peter Lemenkov wrote: > 2012/5/12 : >> Thanks for your opinion. >> >> I'm not the first who had this idea of separating 3rd party libraries >> from the ones a programming language comes with ? for example, Python, >> Perl or Ruby provide a way to do that (not only on my distro, it's >> really part of their install scripts). I just don't want to clutter the >> "official" directorys with stuff which is not part of the main >> distribution. >> >> Of course this adds a bit of complexity, but on the other hand a clear >> structure is also very helpful sometimes. > > Let me bottom line this for everyone - no technical motivation behind > that. If so then this proposal must be rejected. > > -- > With best regards, Peter Lemenkov. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dmkolesnikov@REDACTED Sat May 12 11:50:43 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Sat, 12 May 2012 12:50:43 +0300 Subject: [erlang-questions] Writing platform independent Socket based binary parsing code In-Reply-To: References: <0E282F28-E417-4E19-84CA-E45204C6871D@gmail.com> Message-ID: Hi, There are no needs for ntohs functions in erlang. Binary matching does the job and you should keep in-mind that by default it assumes "network byte order" ~ big-endian So, like you've wrote you parse data through binary match <> = Data "big-unsigned" can be skipped (it is assumed by default) - Dmitry On May 12, 2012, at 12:13 PM, Arun Muralidharan wrote: > Thanks Dmitry for the quick response. > I am aware of the fact that from network we get data in our application in Big Endian format and in c/c++ I would use ntohs/ntohl functions for converting it into a format that the OS supports. > But, the same in Erlang is confusing me (no idea why !!). > As per your last statement "So, the parsing of protocol primitive shall not be dependent on CPU architecture. E.g. If you protocol say that version is serialized into big-endian then your parsing code shall be same and it is platform independent. " do you want to say that, as I am parsing data from network socket I should use "<> = Sock_Data." as the data in network is serialized in big endian format ? > > Really sorry if I am sounding naive..but somehow i am having trouble understanding this in Erlang. > > Thanks, > Arun > > > On Sat, May 12, 2012 at 1:52 PM, Dmitry Kolesnikov wrote: > Hello, > > I'd like to quote one statement here: "Network stacks and communication protocols must also define their endianness. Otherwise, two nodes of different endianness would be unable to communicate." > As an example, TCP/IP protocol suite is defined to be a big-endian. > > So, the parsing of protocol primitive shall not be dependent on CPU architecture. E.g. If you protocol say that version is serialized into big-endian then your parsing code shall be same and it is platform independent. > > - Dmitry > > P.S: ;-) http://geekandpoke.typepad.com/geekandpoke/2011/09/simply-explained-1.html > > > On May 12, 2012, at 10:34 AM, Arun Muralidharan wrote: > >> Hi Folks, >> >> I have an erlang application which includes parsing of binary data from socket (both TCP and UDP). The binary data is as per some specific protocol, so my options for TCP and UDP sock is as per below: >> >> >> TCP sock : >> Opts = [binary, {packet, 0}, {reuseaddr, true}, >> {keepalive, true}, {backlog, 30}, {active, false}], >> UDP sock : >> [binary,{active,once},{recbuf,2097152}] >> >> Now, when i parse the data I get from socket, I do like this (On UNIX): >> >> << Version:4/big-unsigned-integer-unit:8, >> Length:2/big-unsigned-integer:8, >> _Rest/binary >> = Bin_event_From_Socket. >> >> Now, this would give me a problem when run on LINUX as LINUX is little endian based. So, the option here for me is to convert 'big' to 'little' in the above code. >> >> But, Erlang being a VM based language, I must be having some facility to write platform independent code. I searched the NEt but couldnt find much on it. >> >> So, the question here is how can i write a platform independent sockaet data parsing code ? >> >> Thanks. >> >> -Arun >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From arun11299@REDACTED Sat May 12 11:53:27 2012 From: arun11299@REDACTED (Arun Muralidharan) Date: Sat, 12 May 2012 15:23:27 +0530 Subject: [erlang-questions] Writing platform independent Socket based binary parsing code In-Reply-To: References: <0E282F28-E417-4E19-84CA-E45204C6871D@gmail.com> Message-ID: Aha...you have hit the point.. I really wish something like this was explicitly mentioned :) (unless I have missed it). Thanks a lot Dmitry. Regards, Arun On Sat, May 12, 2012 at 3:20 PM, Dmitry Kolesnikov wrote: > Hi, > > There are no needs for ntohs functions in erlang. Binary matching does the > job and you should keep in-mind that by default it assumes "network byte > order" ~ big-endian > So, like you've wrote you parse data through binary match > <> = Data "big-unsigned" can be skipped (it is > assumed by default) > > - Dmitry > > On May 12, 2012, at 12:13 PM, Arun Muralidharan wrote: > > Thanks Dmitry for the quick response. > I am aware of the fact that from network we get data in our application in > Big Endian format and in c/c++ I would use ntohs/ntohl functions for > converting it into a format that the OS supports. > But, the same in Erlang is confusing me (no idea why !!). > As per your last statement *"So, the parsing of protocol primitive shall > not be dependent on CPU architecture. E.g. If you protocol say that version > is serialized into big-endian then your parsing code shall be same and it > is platform independent. " *do you want to say that, as I am parsing > data from network socket I should use "< 4/big-unsigned-integer:8,_Rest/binary>> = Sock_Data." as the data in > network is serialized in big endian format ? > > Really sorry if I am sounding naive..but somehow i am having trouble > understanding this in Erlang. > > Thanks, > Arun > > > On Sat, May 12, 2012 at 1:52 PM, Dmitry Kolesnikov > wrote: > >> Hello, >> >> I'd like to quote one statement here: "Network stacks and communication >> protocols must also define their endianness. Otherwise, two nodes of >> different endianness would be unable to communicate." >> As an example, TCP/IP protocol suite is defined to be a big-endian. >> >> So, the parsing of protocol primitive shall not be dependent on CPU >> architecture. E.g. If you protocol say that version is serialized into >> big-endian then your parsing code shall be same and it is platform >> independent. >> >> - Dmitry >> >> P.S: ;-) >> http://geekandpoke.typepad.com/geekandpoke/2011/09/simply-explained-1.html >> >> >> On May 12, 2012, at 10:34 AM, Arun Muralidharan wrote: >> >> Hi Folks, >> >> I have an erlang application which includes parsing of binary data from >> socket (both TCP and UDP). The binary data is as per some specific >> protocol, so my options for TCP and UDP sock is as per below: >> >> TCP sock : >> Opts = [binary, {packet, 0}, {reuseaddr, true}, >> {keepalive, true}, {backlog, 30}, {active, false}], >> >> UDP sock : >> [binary,{active,once},{recbuf,2097152}] >> >> >> Now, when i parse the data I get from socket, I do like this (On UNIX): >> >> << Version:4/big-unsigned-integer-unit:8, >> Length:2/big-unsigned-integer:8, >> _Rest/binary >> = Bin_event_From_Socket. >> >> >> Now, this would give me a problem when run on LINUX as LINUX is little >> endian based. So, the option here for me is to convert 'big' to 'little' in >> the above code. >> >> But, Erlang being a VM based language, I must be having some facility to >> write platform independent code. I searched the NEt but couldnt find much >> on it. >> So, the question here is how can i write a platform independent sockaet >> data parsing code ? >> >> Thanks. >> -Arun >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cloudzen@REDACTED Sun May 13 12:32:34 2012 From: cloudzen@REDACTED (skyman) Date: Sun, 13 May 2012 18:32:34 +0800 (CST) Subject: [erlang-questions] Is the performance cost of inet:i() big or small? Message-ID: <116fc4e4.9d78.13745c1c016.Coremail.cloudzen@163.com> Hi all, inet:i() can print all socket info statistics, and it is very useful for monitoring the network for the Erlang node. Is the performance cost of inet:i() big or small? Can I use it in the production system? Thanks in advance! -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Sun May 13 15:07:19 2012 From: tony@REDACTED (Tony Rogvall) Date: Sun, 13 May 2012 15:07:19 +0200 Subject: [erlang-questions] windows Message-ID: <111ECCC1-8CC3-4C79-A331-28161A5FFFC2@rogvall.se> Dear erlang-ers. Is there anyone on this list that do develop on windows ? How on earth do you do that ? ;) I am trying to get a driver compiled using rebar. rebar is nearly working on MingW but git is not available for MingW, the reason is somewhat bizarre ... rebar is nearly working on cygwin but there is some path problem that result in "access denied". Right now I reverted to using a Makefile with MingW, that after lots of copy and past finally works. Is rebar supposed to be working with cygwin/mingw ? BTW I will NOT install Visual Studio XYZ :-) Thanks /Tony "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From dang@REDACTED Sun May 13 16:16:41 2012 From: dang@REDACTED (Daniel Goertzen) Date: Sun, 13 May 2012 09:16:41 -0500 Subject: [erlang-questions] windows In-Reply-To: <111ECCC1-8CC3-4C79-A331-28161A5FFFC2@rogvall.se> References: <111ECCC1-8CC3-4C79-A331-28161A5FFFC2@rogvall.se> Message-ID: I am also unable to make rebar work on Windows. Whenever I download something with a rebar script, I have to basically open it in an editor "execute" it by hand and hack everything into place... even for straight forward things with no C compiling like gproc. No fun at all. I settled for cygwin Make, and I was able to build a working NIF module using the compile command indicated in the NIF tutorial. The Makefile is completely Windows specific, but it meets my needs. This of course uses Visual Studio... which is a free download and really not a big deal to install. Dan. On Sun, May 13, 2012 at 8:07 AM, Tony Rogvall wrote: > Dear erlang-ers. > > Is there anyone on this list that do develop on windows ? > How on earth do you do that ? ;) > > I am trying to get a driver compiled using rebar. > rebar is nearly working on MingW but git is not available for MingW, the > reason is somewhat bizarre ... > rebar is nearly working on cygwin but there is some path problem that result > in "access denied". > > Right now I reverted to using a Makefile with MingW, that after lots of copy > and past finally works. > > Is rebar supposed to be working with cygwin/mingw ? > > BTW > I will NOT install Visual Studio XYZ :-) > Thanks > > /Tony > > "Installing applications can lead to corruption over time.?Applications > gradually write over each other's libraries, partial upgrades occur, user > and system errors happen, and minute changes may be unnoticeable and > difficult to fix" > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Daniel Goertzen?|?Senior Software Engineer Office:?828.610.4596?|?Fax:?828.322.5294?|?dang@REDACTED Network Integrity Systems?|?We Bring Security To Light? 1937 Tate Blvd. SE Hickory, North Carolina, USA 28602 Network Integrity Systems? INTERCEPTOR? Optical Network Security System is a Smart-PDS? that ensures superior protection and cost effectiveness of classified networks. ?For more information, visit our website at:www.networkintegritysystems.com. __________________________________________ INTERCEPTOR? Optical Network Security System is made in the USA for the USA. Although not an export controlled item, because of the role it plays in the assurance of the safety and integrity of National Security Information, Network Integrity Systems (NIS) is committed to compliance with the U.S. Export Administration Act. Accordingly, NIS will not ship INTERCEPTOR products to certain foreign government end users without U.S. government approval and will refuse transactions with individuals or entities that have been denied export privileges. From dmitryme@REDACTED Sun May 13 16:28:39 2012 From: dmitryme@REDACTED (Dmitry Melnikov) Date: Sun, 13 May 2012 18:28:39 +0400 Subject: [erlang-questions] windows In-Reply-To: References: <111ECCC1-8CC3-4C79-A331-28161A5FFFC2@rogvall.se> Message-ID: Hi, I am using vs + cmake + rebar. C++ part is buit with cmake + vs and rest erlang code is buit with rebar. All build procedures is made from one build.bat file. Works well. 13.05.2012 18:16 ???????????? "Daniel Goertzen" < dang@REDACTED> ???????: > I am also unable to make rebar work on Windows. Whenever I download > something with a rebar script, I have to basically open it in an > editor "execute" it by hand and hack everything into place... even for > straight forward things with no C compiling like gproc. No fun at > all. > > I settled for cygwin Make, and I was able to build a working NIF > module using the compile command indicated in the NIF tutorial. The > Makefile is completely Windows specific, but it meets my needs. This > of course uses Visual Studio... which is a free download and really > not a big deal to install. > > Dan. > > On Sun, May 13, 2012 at 8:07 AM, Tony Rogvall wrote: > > Dear erlang-ers. > > > > Is there anyone on this list that do develop on windows ? > > How on earth do you do that ? ;) > > > > I am trying to get a driver compiled using rebar. > > rebar is nearly working on MingW but git is not available for MingW, the > > reason is somewhat bizarre ... > > rebar is nearly working on cygwin but there is some path problem that > result > > in "access denied". > > > > Right now I reverted to using a Makefile with MingW, that after lots of > copy > > and past finally works. > > > > Is rebar supposed to be working with cygwin/mingw ? > > > > BTW > > I will NOT install Visual Studio XYZ :-) > > Thanks > > > > /Tony > > > > "Installing applications can lead to corruption over time. Applications > > gradually write over each other's libraries, partial upgrades occur, user > > and system errors happen, and minute changes may be unnoticeable and > > difficult to fix" > > > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > Daniel Goertzen | Senior Software Engineer > Office: 828.610.4596 | Fax: 828.322.5294 | > dang@REDACTED > Network Integrity Systems | We Bring Security To Light? > > 1937 Tate Blvd. SE > > Hickory, North Carolina, USA 28602 > > Network Integrity Systems? INTERCEPTOR? Optical Network Security > System is a Smart-PDS? that ensures superior protection and cost > effectiveness of classified networks. For more information, visit our > website at:www.networkintegritysystems.com. > > > __________________________________________ > INTERCEPTOR? Optical Network Security System is made in the USA for > the USA. Although not an export controlled item, because of the role > it plays in the assurance of the safety and integrity of National > Security Information, Network Integrity Systems (NIS) is committed to > compliance with the U.S. Export Administration Act. Accordingly, NIS > will not ship INTERCEPTOR products to certain foreign government end > users without U.S. government approval and will refuse transactions > with individuals or entities that have been denied export privileges. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Sun May 13 17:51:48 2012 From: n.oxyde@REDACTED (Anthony Ramine) Date: Sun, 13 May 2012 17:51:48 +0200 Subject: [erlang-questions] msg mailbox and gen_svr shutdown In-Reply-To: References: Message-ID: Pawe?, from what I gather your supervisor waits 1 ms before killing its children; I think it's pretty normal that some messages are left unprocessed. Bob, cowboy_requests_sup' shutdown strategy is brutal_kill, is it documented somewhere that Cowboy waits before killing requests' processes? Regards, -- Anthony Ramine Le 10 mai 2012 ? 22:37, Bob Ippolito a ?crit : > I had the same problem with Cowboy recently, really ought to file an issue to at least fix the docs for that. > > For my app I wanted to implement a graceful shutdown that stopped accepting new connections but allowed existing requests to finish. The waiting part I implemented by adding a gproc local property to the workers that I wanted to wait for and then monitoring them. > > Looks like this: > https://gist.github.com/2655724 > > On Thu, May 10, 2012 at 2:53 AM, Pawe? Peregud wrote: > I was having fun with supervisors yesterday (Cowboy seems to fail to fulfill the promise of not killing request processes after listener removal) and I have an example. I've only investigated the case when supervisor is killed, so YMMV. Example code is attached. You may modify it to check the behavior in your case. > > Start supervisor tree with exp_sup_sup:start_link(). Execute test with exp_sup_sup:test() and exp_sup_sup:test_simple(). > > In case of dying supervisor the answer is "no, it does not". > > When supervisor dies, your process is killed as via link mechanism, so it may leave some unprocessed messages in inbox. To make sure that every delivered message is served, you need to add process_flag(trap_exit, true). Messages that are sent after the moment when supervisor dies are not processed. > > Best regards, > > Paul. > > > On May 9, 2012 11:06 AM, "Andy Richards" wrote: > Hi, > > I can't seem to see any confirmation in the documentation so was wondering if anyone could confirm if messages are still sent to a supervised gen_svr following a shutdown message? > > If so how do I cleanly shutdown my gen_svr without loosing messages? I read in the supervisor child spec that a shutdown can be set to infinity which i hoped would allow me to process the msg's in my mailbox but if I do this will my module continue to receive messages from other processes? Is my approach flawed and if so what other ways are there to cleanly shutting down my gen_svr without loosing messages? > > Many thanks, > > Andy. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Sun May 13 19:58:30 2012 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 13 May 2012 10:58:30 -0700 Subject: [erlang-questions] prettypr In-Reply-To: <69DCD66D-609A-4C59-94AC-DE5CF93B0CCE@feuerlabs.com> References: <69DCD66D-609A-4C59-94AC-DE5CF93B0CCE@feuerlabs.com> Message-ID: On Fri, May 11, 2012 at 2:15 PM, Ulf Wiger wrote: > > Now, since I actually don't know exactly what you are trying to do, I will > quit throwing out suggestions. :) > I'm just trying to provide auto-formatting capabilities to a text editor, so I want to pass the full or partial module code in, and get it out 'pretty' formatted :) thank you ulf, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Sun May 13 20:03:36 2012 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 13 May 2012 11:03:36 -0700 Subject: [erlang-questions] prettypr In-Reply-To: References: <69DCD66D-609A-4C59-94AC-DE5CF93B0CCE@feuerlabs.com> Message-ID: On Fri, May 11, 2012 at 6:33 PM, Andrew Berman wrote: > What about this? > > Comments = erl_comment_scan:string(Text), > {ok, Form, _} = erl_scan:string(Text), > {ok, Tree} = erl_parse:parse_form(Form), > {Tree2, _} = erl_recomment:recomment_tree(Tree, Comments), > erl_prettypr:format(Tree2). > hello andrew, this looks interesting, though i cannot find a way to parse a whole mudule: %%%%%%%%%%%%%%%%%%%%%%%%%% -module(pretty). -compile(export_all). start() -> Code = "-module(start). -export([start/0]). start() ->ok. % a comment here.", Comments = erl_comment_scan:string(Code), {ok, Form, _} = erl_scan:string(Code), {ok, Tree} = erl_parse:parse_form(Form), {Tree2, _} = erl_recomment:recomment_tree(Tree, Comments), erl_prettypr:format(Tree2). %%%%%%%%%%%%%%%%%%%%%%%%%% 1> pretty:start(). ** exception error: no match of right hand side value {error,{2,erl_parse,["syntax error before: ","'-'"]}} in function pretty:start/0 (pretty.erl, line 13) any ideas? r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Sun May 13 20:12:18 2012 From: bob@REDACTED (Bob Ippolito) Date: Sun, 13 May 2012 12:12:18 -0600 Subject: [erlang-questions] msg mailbox and gen_svr shutdown In-Reply-To: References: Message-ID: Yes, it is documented that request processes continue after you stop the listener. This is incorrect. On Sunday, May 13, 2012, Anthony Ramine wrote: > Pawe?, from what I gather your supervisor waits 1 ms before killing its > children; I think it's pretty normal that some messages are left > unprocessed. > Bob, cowboy_requests_sup' shutdown strategy is brutal_kill, is it > documented somewhere that Cowboy waits before killing requests' processes? > > Regards, > > -- > Anthony Ramine > > > > > Le 10 mai 2012 ? 22:37, Bob Ippolito a ?crit : > > I had the same problem with Cowboy recently, really ought to file an issue > to at least fix the docs for that. > > For my app I wanted to implement a graceful shutdown that stopped > accepting new connections but allowed existing requests to finish. The > waiting part I implemented by adding a gproc local property to the workers > that I wanted to wait for and then monitoring them. > > Looks like this: > https://gist.github.com/2655724 > > On Thu, May 10, 2012 at 2:53 AM, Pawe? Peregud > > wrote: > >> I was having fun with supervisors yesterday (Cowboy seems to fail to >> fulfill the promise of not killing request processes after listener >> removal) and I have an example. I've only investigated the case when >> supervisor is killed, so YMMV. Example code is attached. You may modify it >> to check the behavior in your case. >> >> Start supervisor tree with exp_sup_sup:start_link(). Execute test with >> exp_sup_sup:test() and exp_sup_sup:test_simple(). >> >> In case of dying supervisor the answer is "no, it does not". >> >> When supervisor dies, your process is killed as via link mechanism, so it >> may leave some unprocessed messages in inbox. To make sure that every >> delivered message is served, you need to add process_flag(trap_exit, true). >> Messages that are sent after the moment when supervisor dies are not >> processed. >> >> Best regards, >> >> Paul. >> >> On May 9, 2012 11:06 AM, "Andy Richards" < >> andy.richards.iit@REDACTED > 'andy.richards.iit@REDACTED');>> wrote: >> >>> Hi, >>> >>> I can't seem to see any confirmation in the documentation so was >>> wondering if anyone could confirm if messages are still sent to a >>> supervised gen_svr following a shutdown message? >>> >>> If so how do I cleanly shutdown my gen_svr without loosing messages? I >>> read in the supervisor child spec that a shutdown can be set to infinity >>> which i hoped would allow me to process the msg's in my mailbox but if I do >>> this will my module continue to receive messages from other processes? Is >>> my approach flawed and if so what other ways are there to cleanly shutting >>> down my gen_svr without loosing messages? >>> >>> Many thanks, >>> >>> Andy. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >> 'erlang-questions@REDACTED');> >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED > 'erlang-questions@REDACTED');> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED 'erlang-questions@REDACTED');> > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Sun May 13 20:14:30 2012 From: n.oxyde@REDACTED (Anthony Ramine) Date: Sun, 13 May 2012 20:14:30 +0200 Subject: [erlang-questions] prettypr In-Reply-To: References: <69DCD66D-609A-4C59-94AC-DE5CF93B0CCE@feuerlabs.com> Message-ID: If Code is a list of forms, split its result at each '.' and call parse_form on that. -- Anthony Ramine Le 13 mai 2012 ? 20:03, Roberto Ostinelli a ?crit : > On Fri, May 11, 2012 at 6:33 PM, Andrew Berman wrote: > What about this? > > Comments = erl_comment_scan:string(Text), > {ok, Form, _} = erl_scan:string(Text), > {ok, Tree} = erl_parse:parse_form(Form), > {Tree2, _} = erl_recomment:recomment_tree(Tree, Comments), > erl_prettypr:format(Tree2). > > hello andrew, > > this looks interesting, though i cannot find a way to parse a whole mudule: > > > %%%%%%%%%%%%%%%%%%%%%%%%%% > -module(pretty). > -compile(export_all). > > start() -> > Code = "-module(start). > -export([start/0]). > > start() ->ok. > > % a comment here.", > Comments = erl_comment_scan:string(Code), > {ok, Form, _} = erl_scan:string(Code), > {ok, Tree} = erl_parse:parse_form(Form), > {Tree2, _} = erl_recomment:recomment_tree(Tree, Comments), > erl_prettypr:format(Tree2). > %%%%%%%%%%%%%%%%%%%%%%%%%% > > > 1> pretty:start(). > ** exception error: no match of right hand side value {error,{2,erl_parse,["syntax error before: ","'-'"]}} > in function pretty:start/0 (pretty.erl, line 13) > > any ideas? > > r. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From gumm@REDACTED Sun May 13 20:56:22 2012 From: gumm@REDACTED (Jesse Gumm) Date: Sun, 13 May 2012 13:56:22 -0500 Subject: [erlang-questions] windows In-Reply-To: <111ECCC1-8CC3-4C79-A331-28161A5FFFC2@rogvall.se> References: <111ECCC1-8CC3-4C79-A331-28161A5FFFC2@rogvall.se> Message-ID: Hi Tony, I spent a lot of time fighting with Windows (Mingw/Msys), Rebar, and Erlang to get them all playing nicely together well to generate a decent windows release for Nitrogen, and it works pretty reliably now. You can use my instructions for compiling Nitrogen as a basis for getting your stuff working: https://github.com/nitrogen/nitrogen/blob/master/rel/overlay/win/README.md -Jesse On Sun, May 13, 2012 at 8:07 AM, Tony Rogvall wrote: > Dear erlang-ers. > > Is there anyone on this list that do develop on windows ? > How on earth do you do that ? ;) > > I am trying to get a driver compiled using rebar. > rebar is nearly working on MingW but git is not available for MingW, the > reason is somewhat bizarre ... > rebar is nearly working on cygwin but there is some path problem that result > in "access denied". > > Right now I reverted to using a Makefile with MingW, that after lots of copy > and past finally works. > > Is rebar supposed to be working with cygwin/mingw ? > > BTW > I will NOT install Visual Studio XYZ :-) > Thanks > > /Tony > > "Installing applications can lead to corruption over time.?Applications > gradually write over each other's libraries, partial upgrades occur, user > and system errors happen, and minute changes may be unnoticeable and > difficult to fix" > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm From tuncer.ayaz@REDACTED Sun May 13 21:47:03 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 13 May 2012 21:47:03 +0200 Subject: [erlang-questions] windows In-Reply-To: <111ECCC1-8CC3-4C79-A331-28161A5FFFC2@rogvall.se> References: <111ECCC1-8CC3-4C79-A331-28161A5FFFC2@rogvall.se> Message-ID: On Sun, May 13, 2012 at 3:07 PM, Tony Rogvall wrote: > Dear erlang-ers. > > Is there anyone on this list that do develop on windows ? All Windows support in rebar is from contributors using rebar on Windows. I remember it's being used for building drivers plus reltool and appup support. > rebar is nearly working on cygwin but there is some path problem > that result in "access denied". Can you file a ticket or mail the rebar list with more info? > Is rebar supposed to be working with cygwin/mingw ? >From what I recall it's being used in cygwin, mingw, and plain Windows with various shells. > BTW > I will NOT install Visual Studio XYZ :-) There is nothing depending on msvc in rebar's native code plugin. All of it is configurable via rebar.config and environment variables. Please see 1. {port_env, []} in rebar.config.sample 2. default_env/0 in rebar_port_compiler.erl From tuncer.ayaz@REDACTED Sun May 13 21:51:09 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sun, 13 May 2012 21:51:09 +0200 Subject: [erlang-questions] windows In-Reply-To: References: <111ECCC1-8CC3-4C79-A331-28161A5FFFC2@rogvall.se> Message-ID: On Sun, May 13, 2012 at 4:16 PM, Daniel Goertzen wrote: > I am also unable to make rebar work on Windows. Whenever I download > something with a rebar script, I have to basically open it in an > editor "execute" it by hand and hack everything into place... even > for straight forward things with no C compiling like gproc. No fun > at all. Is that because Makefiles have rules to run "./rebar CMD"? It's preferable to run it as "rebar" and therefore if required use the rebar.cmd wrapper. From rexxe98@REDACTED Sun May 13 22:00:13 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Sun, 13 May 2012 13:00:13 -0700 Subject: [erlang-questions] prettypr In-Reply-To: References: <69DCD66D-609A-4C59-94AC-DE5CF93B0CCE@feuerlabs.com> Message-ID: Hey Roberto, Try epp_dodger:parse_file(Module), I think that should do it. --Andrew On Sun, May 13, 2012 at 11:03 AM, Roberto Ostinelli wrote: > On Fri, May 11, 2012 at 6:33 PM, Andrew Berman wrote: > >> What about this? >> >> Comments = erl_comment_scan:string(Text), >> {ok, Form, _} = erl_scan:string(Text), >> {ok, Tree} = erl_parse:parse_form(Form), >> {Tree2, _} = erl_recomment:recomment_tree(Tree, Comments), >> erl_prettypr:format(Tree2). >> > > hello andrew, > > this looks interesting, though i cannot find a way to parse a whole mudule: > > > %%%%%%%%%%%%%%%%%%%%%%%%%% > -module(pretty). > -compile(export_all). > > start() -> > Code = "-module(start). > -export([start/0]). > > start() ->ok. > > % a comment here.", > Comments = erl_comment_scan:string(Code), > {ok, Form, _} = erl_scan:string(Code), > {ok, Tree} = erl_parse:parse_form(Form), > {Tree2, _} = erl_recomment:recomment_tree(Tree, Comments), > erl_prettypr:format(Tree2). > %%%%%%%%%%%%%%%%%%%%%%%%%% > > > 1> pretty:start(). > ** exception error: no match of right hand side value > {error,{2,erl_parse,["syntax error before: ","'-'"]}} > in function pretty:start/0 (pretty.erl, line 13) > > any ideas? > > r. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Sun May 13 22:51:37 2012 From: tony@REDACTED (Tony Rogvall) Date: Sun, 13 May 2012 22:51:37 +0200 Subject: [erlang-questions] windows In-Reply-To: References: <111ECCC1-8CC3-4C79-A331-28161A5FFFC2@rogvall.se> Message-ID: On 13 maj 2012, at 20:56, Jesse Gumm wrote: > Hi Tony, > > I spent a lot of time fighting with Windows (Mingw/Msys), Rebar, and > Erlang to get them all playing nicely together well to generate a > decent windows release for Nitrogen, and it works pretty reliably now. > > You can use my instructions for compiling Nitrogen as a basis for > getting your stuff working: > https://github.com/nitrogen/nitrogen/blob/master/rel/overlay/win/README.md > Thanks Installing the (very full installer) msysgit worked (have not tried all fancy features of rebar yet) I also stumbled on that I had written a special spec: {"(win32)","priv/dthread_drv.dll", ["c_src/dthread.c", "c_src/dthread_drv.c"]}, The string "priv/dthread_drv.dll" had the effect that the driver was named ".dll" so I had some problem finding it :-) using "priv/dthread_drv.so" fixed it (strange ?) Now it loads and runs under both mingw (msysgit) and werl /Tony > -Jesse > > On Sun, May 13, 2012 at 8:07 AM, Tony Rogvall wrote: >> Dear erlang-ers. >> >> Is there anyone on this list that do develop on windows ? >> How on earth do you do that ? ;) >> >> I am trying to get a driver compiled using rebar. >> rebar is nearly working on MingW but git is not available for MingW, the >> reason is somewhat bizarre ... >> rebar is nearly working on cygwin but there is some path problem that result >> in "access denied". >> >> Right now I reverted to using a Makefile with MingW, that after lots of copy >> and past finally works. >> >> Is rebar supposed to be working with cygwin/mingw ? >> >> BTW >> I will NOT install Visual Studio XYZ :-) >> Thanks >> >> /Tony >> >> "Installing applications can lead to corruption over time. Applications >> gradually write over each other's libraries, partial upgrades occur, user >> and system errors happen, and minute changes may be unnoticeable and >> difficult to fix" >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Jesse Gumm > Owner, Sigma Star Systems > 414.940.4866 || sigma-star.com || @jessegumm "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From gumm@REDACTED Sun May 13 23:01:43 2012 From: gumm@REDACTED (Jesse Gumm) Date: Sun, 13 May 2012 16:01:43 -0500 Subject: [erlang-questions] windows In-Reply-To: References: <111ECCC1-8CC3-4C79-A331-28161A5FFFC2@rogvall.se> Message-ID: Excellent! Glad that helped! It can become quite cumbersome fighting with MinGW and the multiple version of `make` (mingw32-make.exe, a downloaded version of `make.exe` from the msys repo, etc). It's not clear at all, especially with the names 'netinstaller' and 'fullinstaller', where the 'netinstaller' is the "more full installer" like you mentioned. Also, apparently, if you copy msys's copy of make into a normal mingw installer, it just hangs because there's some DLL dependency. The documentation is scattered, sparse, and online discussions about msys and mingw all seem to be referring to old versions that no longer work as you'd expect. That first time I got all my stuff to compile with msysgit and rebar and generating the erlang release - that was a good day. And Tuncer has been great about accepting patches for Windows rebar and providing insights into rebar workings. -Jesse On Sun, May 13, 2012 at 3:51 PM, Tony Rogvall wrote: > > On 13 maj 2012, at 20:56, Jesse Gumm wrote: > > Hi Tony, > > I spent a lot of time fighting with Windows (Mingw/Msys), Rebar, and > Erlang to get them all playing nicely together well to generate a > decent windows release for Nitrogen, and it works pretty reliably now. > > You can use my instructions for compiling Nitrogen as a basis for > getting your stuff working: > https://github.com/nitrogen/nitrogen/blob/master/rel/overlay/win/README.md > > > Thanks > > Installing the (very full installer) msysgit worked > ?(have not tried all fancy features of rebar yet) > I also stumbled on that I had written a special spec: > > ? ? ?{"(win32)","priv/dthread_drv.dll", > ? ? ? ["c_src/dthread.c", "c_src/dthread_drv.c"]}, > > The string "priv/dthread_drv.dll" had the effect that the driver was named > ".dll" so > I had some problem finding it :-) using?"priv/dthread_drv.so" fixed it > (strange ?) > > Now it loads and runs under both mingw (msysgit) and werl > > /Tony > > > -Jesse > > On Sun, May 13, 2012 at 8:07 AM, Tony Rogvall wrote: > > Dear erlang-ers. > > > Is there anyone on this list that do develop on windows ? > > How on earth do you do that ? ;) > > > I am trying to get a driver compiled using rebar. > > rebar is nearly working on MingW but git is not available for MingW, the > > reason is somewhat bizarre ... > > rebar is nearly working on cygwin but there is some path problem that result > > in "access denied". > > > Right now I reverted to using a Makefile with MingW, that after lots of copy > > and past finally works. > > > Is rebar supposed to be working with cygwin/mingw ? > > > BTW > > I will NOT install Visual Studio XYZ :-) > > Thanks > > > /Tony > > > "Installing applications can lead to corruption over time.?Applications > > gradually write over each other's libraries, partial upgrades occur, user > > and system errors happen, and minute changes may be unnoticeable and > > difficult to fix" > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > -- > Jesse Gumm > Owner, Sigma Star Systems > 414.940.4866 || sigma-star.com || @jessegumm > > > "Installing applications can lead to corruption over time.?Applications > gradually write over each other's libraries, partial upgrades occur, user > and system errors happen, and minute changes may be unnoticeable and > difficult to fix" > > > -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm From ok@REDACTED Sun May 13 23:49:09 2012 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 14 May 2012 09:49:09 +1200 Subject: [erlang-questions] Writing platform independent Socket based binary parsing code In-Reply-To: References: <0E282F28-E417-4E19-84CA-E45204C6871D@gmail.com> Message-ID: <86310B70-F719-41A6-B53F-6B63C6E295C6@cs.otago.ac.nz> If the data are big-endian on the wire, use /big. If the data are little-endian on the wire, use /little. Use the *same* Erlang code to match the wire data whether you are using a big-endian SPARC or a little- endian x86-64. > I am aware of the fact that from network we get data in our application in Big Endian format and in c/c++ I would use ntohs/ntohl functions for converting it into a format that the OS supports. > But, the same in Erlang is confusing me (no idea why !!). In effect, binary matching *is* ntoh... and binary construction *is* hton... From roberto@REDACTED Mon May 14 01:22:47 2012 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 13 May 2012 16:22:47 -0700 Subject: [erlang-questions] prettypr In-Reply-To: References: <69DCD66D-609A-4C59-94AC-DE5CF93B0CCE@feuerlabs.com> Message-ID: Hi Andrew, that is in my original example, but it parses a *file*. r. On Sun, May 13, 2012 at 1:00 PM, Andrew Berman wrote: > Hey Roberto, > > Try epp_dodger:parse_file(Module), I think that should do it. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Mon May 14 02:34:07 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Sun, 13 May 2012 20:34:07 -0400 Subject: [erlang-questions] msg mailbox and gen_svr shutdown In-Reply-To: References: Message-ID: <4FB052FF.9080604@ferd.ca> This is just a guess, but is it possible that this is due to the fact Cowboy is using simple_one_for_one supervision? In Pre R15, if I recall correctly, the shutdown of sofo supervisors was asynchronous. The supervisor would just die and let its children figure out it was gone. Starting with R15, things started being synchronous and the supervisor would wait. A brutal kill that made things work fine before R15 (excluding the issue of the application master killing everything) could start breaking in later versions. Again, this is just a guess, without looking at the source or anything. On Sun May 13 14:12:18 2012, Bob Ippolito wrote: > Yes, it is documented that request processes continue after you stop > the listener. This is incorrect. > > On Sunday, May 13, 2012, Anthony Ramine wrote: > > Pawe?, from what I gather your supervisor waits 1 ms before > killing its children; I think it's pretty normal that some > messages are left unprocessed. > Bob, cowboy_requests_sup' shutdown strategy is brutal_kill, is it > documented somewhere that Cowboy waits before killing requests' > processes? > > Regards, > > -- > Anthony Ramine > > > > > Le 10 mai 2012 ? 22:37, Bob Ippolito a ?crit : > >> I had the same problem with Cowboy recently, really ought to file >> an issue to at least fix the docs for that. >> >> For my app I wanted to implement a graceful shutdown that stopped >> accepting new connections but allowed existing requests to >> finish. The waiting part I implemented by adding a gproc local >> property to the workers that I wanted to wait for and then >> monitoring them. >> >> Looks like this: >> https://gist.github.com/2655724 >> >> On Thu, May 10, 2012 at 2:53 AM, Pawe? Peregud >> > 'paulperegud@REDACTED');>> wrote: >> >> I was having fun with supervisors yesterday (Cowboy seems to >> fail to fulfill the promise of not killing request processes >> after listener removal) and I have an example. I've only >> investigated the case when supervisor is killed, so YMMV. >> Example code is attached. You may modify it to check the >> behavior in your case. >> >> Start supervisor tree with exp_sup_sup:start_link(). Execute >> test with exp_sup_sup:test() and exp_sup_sup:test_simple(). >> >> In case of dying supervisor the answer is "no, it does not". >> >> When supervisor dies, your process is killed as via link >> mechanism, so it may leave some unprocessed messages in >> inbox. To make sure that every delivered message is served, >> you need to add process_flag(trap_exit, true). Messages that >> are sent after the moment when supervisor dies are not processed. >> >> Best regards, >> >> Paul. >> >> >> On May 9, 2012 11:06 AM, "Andy Richards" >> > 'andy.richards.iit@REDACTED');>> wrote: >> >> Hi, >> >> I can't seem to see any confirmation in the documentation >> so was wondering if anyone could confirm if messages are >> still sent to a supervised gen_svr following a shutdown >> message? >> >> If so how do I cleanly shutdown my gen_svr without >> loosing messages? I read in the supervisor child spec >> that a shutdown can be set to infinity which i hoped >> would allow me to process the msg's in my mailbox but if >> I do this will my module continue to receive messages >> from other processes? Is my approach flawed and if so >> what other ways are there to cleanly shutting down my >> gen_svr without loosing messages? >> >> Many thanks, >> >> Andy. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED > 'erlang-questions@REDACTED');> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED > 'erlang-questions@REDACTED');> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED > 'erlang-questions@REDACTED');> >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dmkolesnikov@REDACTED Mon May 14 06:31:24 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Mon, 14 May 2012 07:31:24 +0300 Subject: [erlang-questions] windows In-Reply-To: References: <111ECCC1-8CC3-4C79-A331-28161A5FFFC2@rogvall.se> Message-ID: <52551E89-C08C-4E38-921A-0E4EF043FFD3@gmail.com> Hi, Like Jesse, "I spent a lot of time fighting with Windows..." but then switched to Linux. Everything works out of box, I am not fighting anymore ;-) Just get VirtualBox + Fedora LXDE... BTW, If you are planning to host your results on Linux then probably you should have it at R&D env. - Dmitry On May 13, 2012, at 9:56 PM, Jesse Gumm wrote: > Hi Tony, > > I spent a lot of time fighting with Windows (Mingw/Msys), Rebar, and > Erlang to get them all playing nicely together well to generate a > decent windows release for Nitrogen, and it works pretty reliably now. > > You can use my instructions for compiling Nitrogen as a basis for > getting your stuff working: > https://github.com/nitrogen/nitrogen/blob/master/rel/overlay/win/README.md > > -Jesse > > On Sun, May 13, 2012 at 8:07 AM, Tony Rogvall wrote: >> Dear erlang-ers. >> >> Is there anyone on this list that do develop on windows ? >> How on earth do you do that ? ;) >> >> I am trying to get a driver compiled using rebar. >> rebar is nearly working on MingW but git is not available for MingW, the >> reason is somewhat bizarre ... >> rebar is nearly working on cygwin but there is some path problem that result >> in "access denied". >> >> Right now I reverted to using a Makefile with MingW, that after lots of copy >> and past finally works. >> >> Is rebar supposed to be working with cygwin/mingw ? >> >> BTW >> I will NOT install Visual Studio XYZ :-) >> Thanks >> >> /Tony >> >> "Installing applications can lead to corruption over time. Applications >> gradually write over each other's libraries, partial upgrades occur, user >> and system errors happen, and minute changes may be unnoticeable and >> difficult to fix" >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Jesse Gumm > Owner, Sigma Star Systems > 414.940.4866 || sigma-star.com || @jessegumm > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Mon May 14 07:02:14 2012 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Mon, 14 May 2012 07:02:14 +0200 Subject: [erlang-questions] msg mailbox and gen_svr shutdown In-Reply-To: <4FB052FF.9080604@ferd.ca> References: <4FB052FF.9080604@ferd.ca> Message-ID: <4FB091D6.8000409@ninenines.eu> Right, this is pretty much what I said to Paul in PM. It was in R14B03 that the behavior changed. I apparently have a @todo wrong past that release, and will take a look if I find other things to fix in the docs. Copy pasting my private reply on this: Ultimately if we remove the listener I think we want to stop everything. For "server is overloaded" situations, you can very well use the 'onrequest' hook which can be set or changed dynamically through cowboy:set_protocol_options, in addition to giving it in start_listener. Takes a fun that has a single arg as a Req, returns a Req, and if you replied from within it it doesn't dispatch the request and stops there (you can also force close the connection by setting the Connection header to "close"). And adding this: For graceful shutdowns, well it's highly application dependent. Some apps are just short lived connections, so not accepting requests plus a short delay ought to do it. Some are long lived, which probably requires to send a shutdown message. Some apps may have connections critical enough that you don't want to shutdown them. It's up to the application implementor to devise the strategy to use for stopping. On 05/14/2012 02:34 AM, Fred Hebert wrote: > This is just a guess, but is it possible that this is due to the fact > Cowboy is using simple_one_for_one supervision? In Pre R15, if I recall > correctly, the shutdown of sofo supervisors was asynchronous. The > supervisor would just die and let its children figure out it was gone. > > Starting with R15, things started being synchronous and the supervisor > would wait. A brutal kill that made things work fine before R15 > (excluding the issue of the application master killing everything) could > start breaking in later versions. > > Again, this is just a guess, without looking at the source or anything. > > On Sun May 13 14:12:18 2012, Bob Ippolito wrote: >> Yes, it is documented that request processes continue after you stop >> the listener. This is incorrect. >> >> On Sunday, May 13, 2012, Anthony Ramine wrote: >> >> Pawe?, from what I gather your supervisor waits 1 ms before >> killing its children; I think it's pretty normal that some >> messages are left unprocessed. >> Bob, cowboy_requests_sup' shutdown strategy is brutal_kill, is it >> documented somewhere that Cowboy waits before killing requests' >> processes? >> >> Regards, >> >> -- >> Anthony Ramine >> >> >> >> >> Le 10 mai 2012 ? 22:37, Bob Ippolito a ?crit : >> >>> I had the same problem with Cowboy recently, really ought to file >>> an issue to at least fix the docs for that. >>> >>> For my app I wanted to implement a graceful shutdown that stopped >>> accepting new connections but allowed existing requests to >>> finish. The waiting part I implemented by adding a gproc local >>> property to the workers that I wanted to wait for and then >>> monitoring them. >>> >>> Looks like this: >>> https://gist.github.com/2655724 >>> >>> On Thu, May 10, 2012 at 2:53 AM, Pawe? Peregud >>> >> 'paulperegud@REDACTED');>> wrote: >>> >>> I was having fun with supervisors yesterday (Cowboy seems to >>> fail to fulfill the promise of not killing request processes >>> after listener removal) and I have an example. I've only >>> investigated the case when supervisor is killed, so YMMV. >>> Example code is attached. You may modify it to check the >>> behavior in your case. >>> >>> Start supervisor tree with exp_sup_sup:start_link(). Execute >>> test with exp_sup_sup:test() and exp_sup_sup:test_simple(). >>> >>> In case of dying supervisor the answer is "no, it does not". >>> >>> When supervisor dies, your process is killed as via link >>> mechanism, so it may leave some unprocessed messages in >>> inbox. To make sure that every delivered message is served, >>> you need to add process_flag(trap_exit, true). Messages that >>> are sent after the moment when supervisor dies are not processed. >>> >>> Best regards, >>> >>> Paul. >>> >>> >>> On May 9, 2012 11:06 AM, "Andy Richards" >>> >> 'andy.richards.iit@REDACTED');>> wrote: >>> >>> Hi, >>> >>> I can't seem to see any confirmation in the documentation >>> so was wondering if anyone could confirm if messages are >>> still sent to a supervised gen_svr following a shutdown >>> message? >>> >>> If so how do I cleanly shutdown my gen_svr without >>> loosing messages? I read in the supervisor child spec >>> that a shutdown can be set to infinity which i hoped >>> would allow me to process the msg's in my mailbox but if >>> I do this will my module continue to receive messages >>> from other processes? Is my approach flawed and if so >>> what other ways are there to cleanly shutting down my >>> gen_svr without loosing messages? >>> >>> Many thanks, >>> >>> Andy. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >> 'erlang-questions@REDACTED');> >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >> 'erlang-questions@REDACTED');> >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >> 'erlang-questions@REDACTED');> >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin Erlang Cowboy Nine Nines From rexxe98@REDACTED Mon May 14 09:28:23 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Mon, 14 May 2012 00:28:23 -0700 Subject: [erlang-questions] prettypr In-Reply-To: References: <69DCD66D-609A-4C59-94AC-DE5CF93B0CCE@feuerlabs.com> Message-ID: Ah, ok. Weird, it doesn't work. I'm not really sure then, maybe look at the source for erl_tidy:module or just use erl_tidy? Andrew On Sun, May 13, 2012 at 4:22 PM, Roberto Ostinelli wrote: > Hi Andrew, > > that is in my original example, but it parses a *file*. > > r. > > > On Sun, May 13, 2012 at 1:00 PM, Andrew Berman wrote: > >> Hey Roberto, >> >> Try epp_dodger:parse_file(Module), I think that should do it. >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From clist@REDACTED Mon May 14 09:41:01 2012 From: clist@REDACTED (Angel J. Alvarez Miguel) Date: Mon, 14 May 2012 09:41:01 +0200 Subject: [erlang-questions] erlang float comparison Message-ID: <201205140941.01511.clist@uah.es> Hi guys I need to compare two floats something like 0.99999998... vs 1.000000 and we came across accuracy problems when testing dihedral angles on a molecule... so we read http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm and wanted to implement something like: if (*(int*)&f1 < *(int*)&f2)... when ef1 and f2 are floats... (f1 about 0.9999899.. and f2=1.0 is think we should start with... <> = term_to_binary(1.9999976,[{minor_version,1}]). and let Padding swallow the external format and tag while Myint gets the ieee754 float But MyInt = 4611686017346523993 instead of 1073741822 these conversion should follow ieee754 and being lexicografic ordered but ... term_to_binary(1.99999976,[{minor_version,1}]). <<131,70,63,255,255,255,191,147,83,89>> term_to_binary(1.99999988,[{minor_version,1}]). <<131,70,63,255,255,255,223,201,169,173>> doesnt seem to be the same thatn you spect to see after reading that page... ..What im doing wrong....? ieee754 layout.... +1.99999976 0x3FFFFFFE 1073741822 +1.99999988 0x3FFFFFFF 1073741823 +2.00000000 0x40000000 1073741824 +2.00000024 0x40000001 1073741825 +2.00000048 0x40000002 1073741826 Thanks!.. /Angel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hynek@REDACTED Mon May 14 09:50:57 2012 From: hynek@REDACTED (Hynek Vychodil) Date: Mon, 14 May 2012 09:50:57 +0200 Subject: [erlang-questions] erlang float comparison In-Reply-To: <201205140941.01511.clist@uah.es> References: <201205140941.01511.clist@uah.es> Message-ID: 1> 0.99999998 < 1.000000. true 2> 1.99999976 < 1.99999988. true What's the problem? On Mon, May 14, 2012 at 9:41 AM, Angel J. Alvarez Miguel wrote: > ** > > > Hi guys > > > > I need to compare two floats something like 0.99999998... vs 1.000000 > > and we came across accuracy problems when testing dihedral angles on a > molecule... > > > > so we read > http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm > > and wanted to implement something like: > > > > if (*(int*)&f1 < *(int*)&f2)... > > > > when ef1 and f2 are floats... (f1 about 0.9999899.. and f2=1.0 > > > > is think we should start with... > > <> = term_to_binary(1.9999976,[{minor_version,1}]). > > > > and let Padding swallow the external format and tag while Myint gets the > ieee754 float > > > > But MyInt = 4611686017346523993 instead of 1073741822 > > these conversion should follow ieee754 and being lexicografic ordered > > but ... > > term_to_binary(1.99999976,[{minor_version,1}]). > > <<131,70,63,255,255,255,191,147,83,89>> > > term_to_binary(1.99999988,[{minor_version,1}]). > > <<131,70,63,255,255,255,223,201,169,173>> > > doesnt seem to be the same thatn you spect to see after reading that > page... > > ..What im doing wrong....? > > > ieee754 layout.... > > > +1.99999976 > > 0x3FFFFFFE > > 1073741822 > > +1.99999988 > > 0x3FFFFFFF > > 1073741823 > > +2.00000000 > > 0x40000000 > > 1073741824 > > +2.00000024 > > 0x40000001 > > 1073741825 > > +2.00000048 > > 0x40000002 > > 1073741826 > > Thanks!.. > > /Angel > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Hynek Vychodil Chief Scientists GoodData n?m?st? 28. ??jna 1104/17, 602 00, Brno - ?ern? Pole Office: +420 530 50 7704 E-mail: hynek@REDACTED Web: www.gooddata.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From hynek@REDACTED Mon May 14 09:58:54 2012 From: hynek@REDACTED (Hynek Vychodil) Date: Mon, 14 May 2012 09:58:54 +0200 Subject: [erlang-questions] erlang float comparison In-Reply-To: References: <201205140941.01511.clist@uah.es> Message-ID: Anyway if you are willing shoot yourself in the foot: 1> F = fun(X) when is_float(X) -> <> = <>, V end. #Fun 2> [F(X) || X<-[0.99999998, 1.000000, 1.99999976, 1.99999988]]. [1072693247,1072693248,1073741823,1073741823] On Mon, May 14, 2012 at 9:50 AM, Hynek Vychodil wrote: > 1> 0.99999998 < 1.000000. > true > 2> 1.99999976 < 1.99999988. > true > > What's the problem? > > On Mon, May 14, 2012 at 9:41 AM, Angel J. Alvarez Miguel wrote: > >> ** >> >> >> Hi guys >> >> >> >> I need to compare two floats something like 0.99999998... vs 1.000000 >> >> and we came across accuracy problems when testing dihedral angles on a >> molecule... >> >> >> >> so we read >> http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm >> >> and wanted to implement something like: >> >> >> >> if (*(int*)&f1 < *(int*)&f2)... >> >> >> >> when ef1 and f2 are floats... (f1 about 0.9999899.. and f2=1.0 >> >> >> >> is think we should start with... >> >> <> = term_to_binary(1.9999976,[{minor_version,1}]). >> >> >> >> and let Padding swallow the external format and tag while Myint gets the >> ieee754 float >> >> >> >> But MyInt = 4611686017346523993 instead of 1073741822 >> >> these conversion should follow ieee754 and being lexicografic ordered >> >> but ... >> >> term_to_binary(1.99999976,[{minor_version,1}]). >> >> <<131,70,63,255,255,255,191,147,83,89>> >> >> term_to_binary(1.99999988,[{minor_version,1}]). >> >> <<131,70,63,255,255,255,223,201,169,173>> >> >> doesnt seem to be the same thatn you spect to see after reading that >> page... >> >> ..What im doing wrong....? >> >> >> ieee754 layout.... >> >> >> +1.99999976 >> >> 0x3FFFFFFE >> >> 1073741822 >> >> +1.99999988 >> >> 0x3FFFFFFF >> >> 1073741823 >> >> +2.00000000 >> >> 0x40000000 >> >> 1073741824 >> >> +2.00000024 >> >> 0x40000001 >> >> 1073741825 >> >> +2.00000048 >> >> 0x40000002 >> >> 1073741826 >> >> Thanks!.. >> >> /Angel >> >> >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > Hynek Vychodil > Chief Scientists > > GoodData > n?m?st? 28. ??jna 1104/17, 602 00, Brno - ?ern? Pole > Office: +420 530 50 7704 > E-mail: hynek@REDACTED > Web: www.gooddata.com > -- Hynek Vychodil Chief Scientists GoodData n?m?st? 28. ??jna 1104/17, 602 00, Brno - ?ern? Pole Office: +420 530 50 7704 E-mail: hynek@REDACTED Web: www.gooddata.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From clist@REDACTED Mon May 14 10:47:19 2012 From: clist@REDACTED (Angel J. Alvarez Miguel) Date: Mon, 14 May 2012 10:47:19 +0200 Subject: [erlang-questions] erlang float comparison In-Reply-To: References: <201205140941.01511.clist@uah.es> Message-ID: <201205141047.19368.clist@uah.es> Well we just picked the wrong example that's is the problem sinosuke@REDACTED:~> erl Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] [QuickCheck] [PropEr] [Erl0MQ] Eshell V5.9.1 (abort with ^G) 1> 1.000000000000000002 > 1.0. false On Lunes, 14 de Mayo de 2012 09:50:57 usted escribi?: > 1> 0.99999998 < 1.000000. > true > 2> 1.99999976 < 1.99999988. > true > > What's the problem? > > On Mon, May 14, 2012 at 9:41 AM, Angel J. Alvarez Miguel wrote: > > ** > > > > > > Hi guys > > > > > > > > I need to compare two floats something like 0.99999998... vs 1.000000 > > > > and we came across accuracy problems when testing dihedral angles on a > > molecule... > > > > > > > > so we read > > http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm > > > > and wanted to implement something like: > > > > > > > > if (*(int*)&f1 < *(int*)&f2)... > > > > > > > > when ef1 and f2 are floats... (f1 about 0.9999899.. and f2=1.0 > > > > > > > > is think we should start with... > > > > <> = term_to_binary(1.9999976,[{minor_version,1}]). > > > > and let Padding swallow the external format and tag while Myint gets the > > ieee754 float > > > > > > > > But MyInt = 4611686017346523993 instead of 1073741822 > > > > these conversion should follow ieee754 and being lexicografic ordered > > > > but ... > > > > term_to_binary(1.99999976,[{minor_version,1}]). > > > > <<131,70,63,255,255,255,191,147,83,89>> > > > > term_to_binary(1.99999988,[{minor_version,1}]). > > > > <<131,70,63,255,255,255,223,201,169,173>> > > > > doesnt seem to be the same thatn you spect to see after reading that > > page... > > > > ..What im doing wrong....? > > > > ieee754 layout.... > > > > +1.99999976 > > > > 0x3FFFFFFE > > > > 1073741822 > > > > +1.99999988 > > > > 0x3FFFFFFF > > > > 1073741823 > > > > +2.00000000 > > > > 0x40000000 > > > > 1073741824 > > > > +2.00000024 > > > > 0x40000001 > > > > 1073741825 > > > > +2.00000048 > > > > 0x40000002 > > > > 1073741826 > > > > Thanks!.. > > > > /Angel > > > > > > > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions From hynek@REDACTED Mon May 14 11:18:36 2012 From: hynek@REDACTED (Hynek Vychodil) Date: Mon, 14 May 2012 11:18:36 +0200 Subject: [erlang-questions] erlang float comparison In-Reply-To: <201205141047.19368.clist@uah.es> References: <201205140941.01511.clist@uah.es> <201205141047.19368.clist@uah.es> Message-ID: They are exactly same numbers: 1> <<1.000000000000000002/float>>. <<63,240,0,0,0,0,0,0>> 2> <<1.0/float>>. <<63,240,0,0,0,0,0,0>> 3> 1.000000000000000002. 1.0 Anyway I think I know what you are about. You want work with numbers in some interval as they are same. Something like 1> F = fun(X) when is_float(X) -> <> = <>, <> = <>, <> = <>, {F, T} end. #Fun 2> F(1.0). {1.0,1.0000009536743162} 3> F(0.99999999). {0.9999995231628418,0.9999999999999999} 4> F(1.99999976). {1.9999990463256836,1.9999999999999998} So you would be able make approximate comparison: 5> Comp = fun(X, Y, N) when is_float(X), is_float(Y), is_integer(N), N>16, N<64 -> <> = <>, <> = <>, if XI < YI -> lt; XI =:= YI -> eq; XI > YI -> gt end end. #Fun 6> Comp(1.99999976, 1.99999988, 32). eq 7> Comp(1.99999976, 1.99999988, 48). lt lt On Mon, May 14, 2012 at 10:47 AM, Angel J. Alvarez Miguel wrote: > Well we just picked the wrong example > > > that's is the problem > > sinosuke@REDACTED:~> erl > Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] > [hipe] [kernel-poll:false] > > [QuickCheck] [PropEr] [Erl0MQ] > Eshell V5.9.1 ?(abort with ^G) > 1> 1.000000000000000002 > 1.0. > false > > > On Lunes, 14 de Mayo de 2012 09:50:57 usted escribi?: >> 1> 0.99999998 < 1.000000. >> true >> 2> 1.99999976 < 1.99999988. >> true >> >> What's the problem? >> >> On Mon, May 14, 2012 at 9:41 AM, Angel J. Alvarez Miguel > wrote: >> > ** >> > >> > >> > Hi guys >> > >> > >> > >> > I need to compare two floats something like 0.99999998... vs 1.000000 >> > >> > and we came across accuracy problems when testing dihedral angles on a >> > molecule... >> > >> > >> > >> > so we read >> > http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm >> > >> > and wanted to implement something like: >> > >> > >> > >> > if (*(int*)&f1 < *(int*)&f2)... >> > >> > >> > >> > when ef1 and f2 are floats... (f1 about 0.9999899.. and f2=1.0 >> > >> > >> > >> > is think we should start with... >> > >> > ?<> = term_to_binary(1.9999976,[{minor_version,1}]). >> > >> > and let Padding swallow the external format and tag while Myint gets the >> > ieee754 float >> > >> > >> > >> > But MyInt = 4611686017346523993 instead of 1073741822 >> > >> > these conversion should follow ieee754 and being lexicografic ordered >> > >> > but ... >> > >> > term_to_binary(1.99999976,[{minor_version,1}]). >> > >> > <<131,70,63,255,255,255,191,147,83,89>> >> > >> > term_to_binary(1.99999988,[{minor_version,1}]). >> > >> > <<131,70,63,255,255,255,223,201,169,173>> >> > >> > doesnt seem to be the same thatn you spect to see after reading that >> > page... >> > >> > ?..What im doing wrong....? >> > >> > ieee754 layout.... >> > >> > ? +1.99999976 >> > >> > 0x3FFFFFFE >> > >> > 1073741822 >> > >> > +1.99999988 >> > >> > 0x3FFFFFFF >> > >> > 1073741823 >> > >> > +2.00000000 >> > >> > 0x40000000 >> > >> > 1073741824 >> > >> > +2.00000024 >> > >> > 0x40000001 >> > >> > 1073741825 >> > >> > +2.00000048 >> > >> > 0x40000002 >> > >> > 1073741826 >> > >> > Thanks!.. >> > >> > /Angel >> > >> > >> > >> > >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions -- Hynek Vychodil Chief Scientists GoodData n?m?st? 28. ??jna 1104/17, 602 00, Brno - ?ern? Pole Office: ? +420 530 50 7704 E-mail: ?hynek@REDACTED Web: ? ? www.gooddata.com From Bart.vanDeenen@REDACTED Mon May 14 11:19:03 2012 From: Bart.vanDeenen@REDACTED (Bart van Deenen) Date: Mon, 14 May 2012 11:19:03 +0200 Subject: [erlang-questions] erlang float comparison In-Reply-To: <201205141047.19368.clist@uah.es> References: <201205140941.01511.clist@uah.es> <201205141047.19368.clist@uah.es> Message-ID: <4FB0CE07.9010106@spilgames.com> On 05/14/2012 10:47 AM, Angel J. Alvarez Miguel wrote: > Well we just picked the wrong example > > > that's is the problem > > sinosuke@REDACTED:~> erl > Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] > [hipe] [kernel-poll:false] > > [QuickCheck] [PropEr] [Erl0MQ] > Eshell V5.9.1 (abort with ^G) > 1> 1.000000000000000002> 1.0. > false > 1> 1.000000000000000002 > 1.0. false 2> 1.000000000000000002 == 1.0. true 3> So I don't get the problem. From clist@REDACTED Mon May 14 12:47:54 2012 From: clist@REDACTED (Angel J. Alvarez Miguel) Date: Mon, 14 May 2012 12:47:54 +0200 Subject: [erlang-questions] erlang float comparison In-Reply-To: References: <201205140941.01511.clist@uah.es> <201205141047.19368.clist@uah.es> Message-ID: <201205141247.54686.clist@uah.es> On Lunes, 14 de Mayo de 2012 11:18:36 usted escribi?: > They are exactly same numbers: > > 1> <<1.000000000000000002/float>>. > <<63,240,0,0,0,0,0,0>> > 2> <<1.0/float>>. > <<63,240,0,0,0,0,0,0>> > 3> 1.000000000000000002. > 1.0 > > Anyway I think I know what you are about. You want work with numbers > in some interval as they are same. Something like > > 1> F = fun(X) when is_float(X) -> <> = <>, > <> = <>, <> = <>, > {F, T} end. > #Fun > 2> F(1.0). > {1.0,1.0000009536743162} > 3> F(0.99999999). > {0.9999995231628418,0.9999999999999999} > 4> F(1.99999976). > {1.9999990463256836,1.9999999999999998} > > So you would be able make approximate comparison: > > 5> Comp = fun(X, Y, N) when is_float(X), is_float(Y), is_integer(N), > N>16, N<64 -> <> = <>, <> = > <>, if XI < YI -> lt; XI =:= YI -> eq; XI > YI -> gt end end. > #Fun > 6> Comp(1.99999976, 1.99999988, 32). > > eq > 7> Comp(1.99999976, 1.99999988, 48). > > lt > lt > That's what we need, i must confess that i have not earlier expeience with binaries so needed a bit of thinking just to attack this problem but now im starting to see it ... Thanks Angel > On Mon, May 14, 2012 at 10:47 AM, Angel J. Alvarez Miguel wrote: > > Well we just picked the wrong example > > > > > > that's is the problem > > > > sinosuke@REDACTED:~> erl > > Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] > > [hipe] [kernel-poll:false] > > > > [QuickCheck] [PropEr] [Erl0MQ] > > Eshell V5.9.1 (abort with ^G) > > 1> 1.000000000000000002 > 1.0. > > false > > > > On Lunes, 14 de Mayo de 2012 09:50:57 usted escribi?: > >> 1> 0.99999998 < 1.000000. > >> true > >> 2> 1.99999976 < 1.99999988. > >> true > >> > >> What's the problem? > >> > >> On Mon, May 14, 2012 at 9:41 AM, Angel J. Alvarez Miguel > > > > wrote: > >> > ** > >> > > >> > > >> > Hi guys > >> > > >> > > >> > > >> > I need to compare two floats something like 0.99999998... vs 1.000000 > >> > > >> > and we came across accuracy problems when testing dihedral angles on a > >> > molecule... > >> > > >> > > >> > > >> > so we read > >> > http://www.cygnus-software.com/papers/comparingfloats/comparingfloats. > >> > htm > >> > > >> > and wanted to implement something like: > >> > > >> > > >> > > >> > if (*(int*)&f1 < *(int*)&f2)... > >> > > >> > > >> > > >> > when ef1 and f2 are floats... (f1 about 0.9999899.. and f2=1.0 > >> > > >> > > >> > > >> > is think we should start with... > >> > > >> > <> = > >> > term_to_binary(1.9999976,[{minor_version,1}]). > >> > > >> > and let Padding swallow the external format and tag while Myint gets > >> > the ieee754 float > >> > > >> > > >> > > >> > But MyInt = 4611686017346523993 instead of 1073741822 > >> > > >> > these conversion should follow ieee754 and being lexicografic ordered > >> > > >> > but ... > >> > > >> > term_to_binary(1.99999976,[{minor_version,1}]). > >> > > >> > <<131,70,63,255,255,255,191,147,83,89>> > >> > > >> > term_to_binary(1.99999988,[{minor_version,1}]). > >> > > >> > <<131,70,63,255,255,255,223,201,169,173>> > >> > > >> > doesnt seem to be the same thatn you spect to see after reading that > >> > page... > >> > > >> > ..What im doing wrong....? > >> > > >> > ieee754 layout.... > >> > > >> > +1.99999976 > >> > > >> > 0x3FFFFFFE > >> > > >> > 1073741822 > >> > > >> > +1.99999988 > >> > > >> > 0x3FFFFFFF > >> > > >> > 1073741823 > >> > > >> > +2.00000000 > >> > > >> > 0x40000000 > >> > > >> > 1073741824 > >> > > >> > +2.00000024 > >> > > >> > 0x40000001 > >> > > >> > 1073741825 > >> > > >> > +2.00000048 > >> > > >> > 0x40000002 > >> > > >> > 1073741826 > >> > > >> > Thanks!.. > >> > > >> > /Angel > >> > > >> > > >> > > >> > > >> > > >> > > >> > _______________________________________________ > >> > erlang-questions mailing list > >> > erlang-questions@REDACTED > >> > http://erlang.org/mailman/listinfo/erlang-questions From n.oxyde@REDACTED Mon May 14 15:02:17 2012 From: n.oxyde@REDACTED (Anthony Ramine) Date: Mon, 14 May 2012 15:02:17 +0200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: <4CEA1F2B-BA47-4E61-AF44-541573228096@cs.otago.ac.nz> References: <79520DF4-2E14-4CA4-8880-0B03E4D6B8DD@cs.otago.ac.nz> <4CEA1F2B-BA47-4E61-AF44-541573228096@cs.otago.ac.nz> Message-ID: <84A5A874-89F2-42C2-B746-AFA98601936A@gmail.com> Couldn't some of the bootstrap Perl scripts like beam_makeops and make_tables be rewritten and documented in Erlang? I think it would make things more obvious if they were not obscure Perl scripts without comments. Furthermore it would make Erlang/OTP eat more of its own dog food. The only thing that would need to be changed with regard to the bootstrap itself is that their output would have to be versioned just as the erts/preloaded/ BEAM files. A new command should also be added to otp_build to update them. Some of these generated files are: beam_tr_funcs.h beam_pred_funcs.h beam_hot.h beam_cold.h beam_opcodes.c beam_opcodes.h beam_opcodes.erl beam_opcodes.hrl erl_am.c erl_bif_table.c erl_bif_wrap.c erl_pbifs.c erl_atom_table.h erl_bif_table.h There may be an obvious reason for them not to be generated by Erlang itself but I'm not aware of it. Regards. -- Anthony Ramine Le 9 mai 2012 ? 01:58, Richard O'Keefe a ?crit : > Let me illustrate the Icon approach by showing you a fragment of the > micro-BEAM I wrote to get the performance numbers in the frames proposal. > (The whole thing is fragmentary.) > > ... > @i > max src, snd, dst > @d > dst := max(src, snd) > > This computes the maximum using the micro-Erlang term ordering. > If src and snd are tagged immediate integers the comparison is > done inline; the compare() function is called otherwise. > @c > T = @src; > U = @snd; > @dst = cmp(T, >, U) ? T : U; > @step; > @e > ... > @i > check_record src, size, const > @d > Type test. > > Fail unless src is tagged as a pointer to a tuple or frame, > the first word it points to is size, and the second is the > const (which must be an atom, but we don't check that). > Used for record matching. > @c > T = @src; > if (!is_tuple(T)) @fail "is_record"; else > if (FIELD(T, TUP_TAG, 0) != @size) @fail "is_record"; else > if (FIELD(T, TUP_TAG, 1) != @const) @fail "is_record"; else > @step; > @e > ... > There is a preprocessor written in AWK that turns this into > several C files. One of them is the emulator cases. For > the check_record instruction you get > > #line 75 "frame.master" > case CHECK_RECORD: > #line 76 "frame.master" > T = reg[(int)P[1]]; > #line 77 "frame.master" > if (!is_tuple(T)) > P = failure, operation = "is_record"; else > #line 78 "frame.master" > if (FIELD(T, TUP_TAG, 0) != 4) > P = failure, operation = "is_record"; else > #line 79 "frame.master" > if (FIELD(T, TUP_TAG, 1) != P[3]) > P = failure, operation = "is_record"; else > #line 80 "frame.master" > P += 4; > break; > > where I've broken the long lines (the preprocessor doesn't). > The #line directives are option. > > @i introduces an instruction; the next line is a template > for it saying what the operands are. > @d introduces the description for people. > @c introduces the code. In it, various built-in @macros > are expanded. > > One advantage of doing it this way is that by using > @step to update the PC I *cannot* get the offset wrong; > the preprocessor counted the operands and their sizes > for me. Similarly, what I write has *no* operand numbers; > the preprocessor counted those, and supplies all necessary > casts as well. I can shuffle operands around (in @i) > without revising the code (in @c), and have. > > It wouldn't be too hard to write another preprocessor that > built some kind of documentation (HTML would probably be > easiest) out of this, but since this was an experiment, > it didn't seem worth while. > > Why did I write the preprocessor? > Well, to be honest, the first draft didn't use one. > I got a bit sick of debugging, and wrote the preprocessor > (based on vague memories of Icon) to eliminate a class of > errors. It turned out to be _easier_ to develop a > documented emulator than an undocumented one. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bengt.kleberg@REDACTED Mon May 14 15:08:12 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 14 May 2012 15:08:12 +0200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: <84A5A874-89F2-42C2-B746-AFA98601936A@gmail.com> References: <79520DF4-2E14-4CA4-8880-0B03E4D6B8DD@cs.otago.ac.nz> <4CEA1F2B-BA47-4E61-AF44-541573228096@cs.otago.ac.nz> <84A5A874-89F2-42C2-B746-AFA98601936A@gmail.com> Message-ID: <1337000892.4744.37.camel@seasc1137> Greetings, Perhaps there is a chicken-and-egg problem with requiring Erlang to generate files used to build Erlang? bengt On Mon, 2012-05-14 at 15:02 +0200, Anthony Ramine wrote: > Couldn't some of the bootstrap Perl scripts like beam_makeops and make_tables be > rewritten and documented in Erlang? I think it would make things more obvious if > they were not obscure Perl scripts without comments. Furthermore it would make > Erlang/OTP eat more of its own dog food. > > The only thing that would need to be changed with regard to the bootstrap itself > is that their output would have to be versioned just as the erts/preloaded/ BEAM > files. A new command should also be added to otp_build to update them. > > Some of these generated files are: > > beam_tr_funcs.h > beam_pred_funcs.h > beam_hot.h > beam_cold.h > beam_opcodes.c > beam_opcodes.h > beam_opcodes.erl > beam_opcodes.hrl > erl_am.c > erl_bif_table.c > erl_bif_wrap.c > erl_pbifs.c > erl_atom_table.h > erl_bif_table.h > > There may be an obvious reason for them not to be generated by Erlang itself but > I'm not aware of it. > > Regards. > > -- > Anthony Ramine > > Le 9 mai 2012 ? 01:58, Richard O'Keefe a ?crit : > > > Let me illustrate the Icon approach by showing you a fragment of the > > micro-BEAM I wrote to get the performance numbers in the frames proposal. > > (The whole thing is fragmentary.) > > > > ... > > @i > > max src, snd, dst > > @d > > dst := max(src, snd) > > > > This computes the maximum using the micro-Erlang term ordering. > > If src and snd are tagged immediate integers the comparison is > > done inline; the compare() function is called otherwise. > > @c > > T = @src; > > U = @snd; > > @dst = cmp(T, >, U) ? T : U; > > @step; > > @e > > ... > > @i > > check_record src, size, const > > @d > > Type test. > > > > Fail unless src is tagged as a pointer to a tuple or frame, > > the first word it points to is size, and the second is the > > const (which must be an atom, but we don't check that). > > Used for record matching. > > @c > > T = @src; > > if (!is_tuple(T)) @fail "is_record"; else > > if (FIELD(T, TUP_TAG, 0) != @size) @fail "is_record"; else > > if (FIELD(T, TUP_TAG, 1) != @const) @fail "is_record"; else > > @step; > > @e > > ... > > There is a preprocessor written in AWK that turns this into > > several C files. One of them is the emulator cases. For > > the check_record instruction you get > > > > #line 75 "frame.master" > > case CHECK_RECORD: > > #line 76 "frame.master" > > T = reg[(int)P[1]]; > > #line 77 "frame.master" > > if (!is_tuple(T)) > > P = failure, operation = "is_record"; else > > #line 78 "frame.master" > > if (FIELD(T, TUP_TAG, 0) != 4) > > P = failure, operation = "is_record"; else > > #line 79 "frame.master" > > if (FIELD(T, TUP_TAG, 1) != P[3]) > > P = failure, operation = "is_record"; else > > #line 80 "frame.master" > > P += 4; > > break; > > > > where I've broken the long lines (the preprocessor doesn't). > > The #line directives are option. > > > > @i introduces an instruction; the next line is a template > > for it saying what the operands are. > > @d introduces the description for people. > > @c introduces the code. In it, various built-in @macros > > are expanded. > > > > One advantage of doing it this way is that by using > > @step to update the PC I *cannot* get the offset wrong; > > the preprocessor counted the operands and their sizes > > for me. Similarly, what I write has *no* operand numbers; > > the preprocessor counted those, and supplies all necessary > > casts as well. I can shuffle operands around (in @i) > > without revising the code (in @c), and have. > > > > It wouldn't be too hard to write another preprocessor that > > built some kind of documentation (HTML would probably be > > easiest) out of this, but since this was an experiment, > > it didn't seem worth while. > > > > Why did I write the preprocessor? > > Well, to be honest, the first draft didn't use one. > > I got a bit sick of debugging, and wrote the preprocessor > > (based on vague memories of Icon) to eliminate a class of > > errors. It turned out to be _easier_ to develop a > > documented emulator than an undocumented one. > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From toby@REDACTED Mon May 14 15:11:52 2012 From: toby@REDACTED (Toby Thain) Date: Mon, 14 May 2012 09:11:52 -0400 Subject: [erlang-questions] erlang float comparison In-Reply-To: <201205141047.19368.clist@uah.es> References: <201205140941.01511.clist@uah.es> <201205141047.19368.clist@uah.es> Message-ID: <4FB10498.6000005@telegraphics.com.au> On 14/05/12 4:47 AM, Angel J. Alvarez Miguel wrote: > Well we just picked the wrong example > > > that's is the problem > > sinosuke@REDACTED:~> erl > Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] > [hipe] [kernel-poll:false] > > [QuickCheck] [PropEr] [Erl0MQ] > Eshell V5.9.1 (abort with ^G) > 1> 1.000000000000000002> 1.0. > false > It's not clear whether you need an approximate equality test as suggested by others (tolerance won't help you with > or < or the example above) or whether you need more (or arbitrary) precision. --Toby > > On Lunes, 14 de Mayo de 2012 09:50:57 usted escribi?: >> 1> 0.99999998< 1.000000. >> true >> 2> 1.99999976< 1.99999988. >> true >> >> What's the problem? >> >> On Mon, May 14, 2012 at 9:41 AM, Angel J. Alvarez Miguel > wrote: >>> ** >>> >>> >>> Hi guys >>> >>> >>> >>> I need to compare two floats something like 0.99999998... vs 1.000000 >>> >>> and we came across accuracy problems when testing dihedral angles on a >>> molecule... >>> >>> >>> >>> so we read >>> http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm >>> >>> and wanted to implement something like: >>> >>> >>> >>> if (*(int*)&f1< *(int*)&f2)... >>> >>> >>> >>> when ef1 and f2 are floats... (f1 about 0.9999899.. and f2=1.0 >>> >>> >>> >>> is think we should start with... >>> >>> <> = term_to_binary(1.9999976,[{minor_version,1}]). >>> >>> and let Padding swallow the external format and tag while Myint gets the >>> ieee754 float >>> >>> >>> >>> But MyInt = 4611686017346523993 instead of 1073741822 >>> >>> these conversion should follow ieee754 and being lexicografic ordered >>> >>> but ... >>> >>> term_to_binary(1.99999976,[{minor_version,1}]). >>> >>> <<131,70,63,255,255,255,191,147,83,89>> >>> >>> term_to_binary(1.99999988,[{minor_version,1}]). >>> >>> <<131,70,63,255,255,255,223,201,169,173>> >>> >>> doesnt seem to be the same thatn you spect to see after reading that >>> page... >>> >>> ..What im doing wrong....? >>> >>> ieee754 layout.... >>> >>> +1.99999976 >>> >>> 0x3FFFFFFE >>> >>> 1073741822 >>> >>> +1.99999988 >>> >>> 0x3FFFFFFF >>> >>> 1073741823 >>> >>> +2.00000000 >>> >>> 0x40000000 >>> >>> 1073741824 >>> >>> +2.00000024 >>> >>> 0x40000001 >>> >>> 1073741825 >>> >>> +2.00000048 >>> >>> 0x40000002 >>> >>> 1073741826 >>> >>> Thanks!.. >>> >>> /Angel >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From n.oxyde@REDACTED Mon May 14 15:15:44 2012 From: n.oxyde@REDACTED (Anthony Ramine) Date: Mon, 14 May 2012 15:15:44 +0200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: <1337000892.4744.37.camel@seasc1137> References: <79520DF4-2E14-4CA4-8880-0B03E4D6B8DD@cs.otago.ac.nz> <4CEA1F2B-BA47-4E61-AF44-541573228096@cs.otago.ac.nz> <84A5A874-89F2-42C2-B746-AFA98601936A@gmail.com> <1337000892.4744.37.camel@seasc1137> Message-ID: There is already a chicken-and-egg problem, that's why there are some BEAM files in the Git repository, look in erts/preloaded/ebin. Nothing prevents us from generating these C files from some Erlang code and versioning them in Git too. This way the system would be bootstrapped from the previously generated files in the repository. -- Anthony Ramine Le 14 mai 2012 ? 15:08, Bengt Kleberg a ?crit : > Greetings, > > Perhaps there is a chicken-and-egg problem with requiring Erlang to > generate files used to build Erlang? > > > bengt > > On Mon, 2012-05-14 at 15:02 +0200, Anthony Ramine wrote: >> Couldn't some of the bootstrap Perl scripts like beam_makeops and make_tables be >> rewritten and documented in Erlang? I think it would make things more obvious if >> they were not obscure Perl scripts without comments. Furthermore it would make >> Erlang/OTP eat more of its own dog food. >> >> The only thing that would need to be changed with regard to the bootstrap itself >> is that their output would have to be versioned just as the erts/preloaded/ BEAM >> files. A new command should also be added to otp_build to update them. >> >> Some of these generated files are: >> >> beam_tr_funcs.h >> beam_pred_funcs.h >> beam_hot.h >> beam_cold.h >> beam_opcodes.c >> beam_opcodes.h >> beam_opcodes.erl >> beam_opcodes.hrl >> erl_am.c >> erl_bif_table.c >> erl_bif_wrap.c >> erl_pbifs.c >> erl_atom_table.h >> erl_bif_table.h >> >> There may be an obvious reason for them not to be generated by Erlang itself but >> I'm not aware of it. >> >> Regards. >> >> -- >> Anthony Ramine >> >> Le 9 mai 2012 ? 01:58, Richard O'Keefe a ?crit : >> >>> Let me illustrate the Icon approach by showing you a fragment of the >>> micro-BEAM I wrote to get the performance numbers in the frames proposal. >>> (The whole thing is fragmentary.) >>> >>> ... >>> @i >>> max src, snd, dst >>> @d >>> dst := max(src, snd) >>> >>> This computes the maximum using the micro-Erlang term ordering. >>> If src and snd are tagged immediate integers the comparison is >>> done inline; the compare() function is called otherwise. >>> @c >>> T = @src; >>> U = @snd; >>> @dst = cmp(T, >, U) ? T : U; >>> @step; >>> @e >>> ... >>> @i >>> check_record src, size, const >>> @d >>> Type test. >>> >>> Fail unless src is tagged as a pointer to a tuple or frame, >>> the first word it points to is size, and the second is the >>> const (which must be an atom, but we don't check that). >>> Used for record matching. >>> @c >>> T = @src; >>> if (!is_tuple(T)) @fail "is_record"; else >>> if (FIELD(T, TUP_TAG, 0) != @size) @fail "is_record"; else >>> if (FIELD(T, TUP_TAG, 1) != @const) @fail "is_record"; else >>> @step; >>> @e >>> ... >>> There is a preprocessor written in AWK that turns this into >>> several C files. One of them is the emulator cases. For >>> the check_record instruction you get >>> >>> #line 75 "frame.master" >>> case CHECK_RECORD: >>> #line 76 "frame.master" >>> T = reg[(int)P[1]]; >>> #line 77 "frame.master" >>> if (!is_tuple(T)) >>> P = failure, operation = "is_record"; else >>> #line 78 "frame.master" >>> if (FIELD(T, TUP_TAG, 0) != 4) >>> P = failure, operation = "is_record"; else >>> #line 79 "frame.master" >>> if (FIELD(T, TUP_TAG, 1) != P[3]) >>> P = failure, operation = "is_record"; else >>> #line 80 "frame.master" >>> P += 4; >>> break; >>> >>> where I've broken the long lines (the preprocessor doesn't). >>> The #line directives are option. >>> >>> @i introduces an instruction; the next line is a template >>> for it saying what the operands are. >>> @d introduces the description for people. >>> @c introduces the code. In it, various built-in @macros >>> are expanded. >>> >>> One advantage of doing it this way is that by using >>> @step to update the PC I *cannot* get the offset wrong; >>> the preprocessor counted the operands and their sizes >>> for me. Similarly, what I write has *no* operand numbers; >>> the preprocessor counted those, and supplies all necessary >>> casts as well. I can shuffle operands around (in @i) >>> without revising the code (in @c), and have. >>> >>> It wouldn't be too hard to write another preprocessor that >>> built some kind of documentation (HTML would probably be >>> easiest) out of this, but since this was an experiment, >>> it didn't seem worth while. >>> >>> Why did I write the preprocessor? >>> Well, to be honest, the first draft didn't use one. >>> I got a bit sick of debugging, and wrote the preprocessor >>> (based on vague memories of Icon) to eliminate a class of >>> errors. It turned out to be _easier_ to develop a >>> documented emulator than an undocumented one. >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From clist@REDACTED Mon May 14 15:53:51 2012 From: clist@REDACTED (Angel J. Alvarez Miguel) Date: Mon, 14 May 2012 15:53:51 +0200 Subject: [erlang-questions] erlang float comparison In-Reply-To: <4FB10498.6000005@telegraphics.com.au> References: <201205140941.01511.clist@uah.es> <201205141047.19368.clist@uah.es> <4FB10498.6000005@telegraphics.com.au> Message-ID: <201205141553.52104.clist@uah.es> On Lunes, 14 de Mayo de 2012 15:11:52 usted escribi?: > On 14/05/12 4:47 AM, Angel J. Alvarez Miguel wrote: > > Well we just picked the wrong example > > > > > > that's is the problem > > > > sinosuke@REDACTED:~> erl > > Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] > > [hipe] [kernel-poll:false] > > > > [QuickCheck] [PropEr] [Erl0MQ] > > Eshell V5.9.1 (abort with ^G) > > 1> 1.000000000000000002> 1.0. > > false > > It's not clear whether you need an approximate equality test as > suggested by others (tolerance won't help you with > or < or the example > above) or whether you need more (or arbitrary) precision. Sorry folks, im just a bit puzzled for the strangeness of fp operations I am talking about things like this: 8> 1.0000000000000001 > 1.0. false 9> 1.0000000000000002 > 1.0. true And i wander that there is some method for avoiding such beasts.. Well ill be happy just if i can compare if Something > 1.0 and it just returns false unless Something is really greater than 1.0 (whatever the accuracy is) so i can relay on doing things like if Angle > 1.0 the shutdown_orbital_laser_before_its_too_late(). because we found that: math:acos(0.99999999999999991). 1.4901161193847656e-8 math:acos(0.99999999999999992). 1.4901161193847656e-8 math:acos(0.99999999999999993). 1.4901161193847656e-8 math:acos(0.99999999999999999). 0.0 math:acos(1.0000000000000001). 0.0 math:acos(1.0000000000000002). ** exception error: bad argument in an arithmetic expression in function math:acos/1 called as math:acos(1.0000000000000002) this last case is waht we wanted to avoid, specifically /Angel > > --Toby > > > On Lunes, 14 de Mayo de 2012 09:50:57 usted escribi?: > >> 1> 0.99999998< 1.000000. > >> true > >> 2> 1.99999976< 1.99999988. > >> true > >> > >> What's the problem? > >> > >> On Mon, May 14, 2012 at 9:41 AM, Angel J. Alvarez Miguel > > > > wrote: > >>> ** > >>> > >>> > >>> Hi guys > >>> > >>> > >>> > >>> I need to compare two floats something like 0.99999998... vs 1.000000 > >>> > >>> and we came across accuracy problems when testing dihedral angles on a > >>> molecule... > >>> > >>> > >>> > >>> so we read > >>> http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.h > >>> tm > >>> > >>> and wanted to implement something like: > >>> > >>> > >>> > >>> if (*(int*)&f1< *(int*)&f2)... > >>> > >>> > >>> > >>> when ef1 and f2 are floats... (f1 about 0.9999899.. and f2=1.0 > >>> > >>> > >>> > >>> is think we should start with... > >>> > >>> <> = > >>> term_to_binary(1.9999976,[{minor_version,1}]). > >>> > >>> and let Padding swallow the external format and tag while Myint gets > >>> the ieee754 float > >>> > >>> > >>> > >>> But MyInt = 4611686017346523993 instead of 1073741822 > >>> > >>> these conversion should follow ieee754 and being lexicografic ordered > >>> > >>> but ... > >>> > >>> term_to_binary(1.99999976,[{minor_version,1}]). > >>> > >>> <<131,70,63,255,255,255,191,147,83,89>> > >>> > >>> term_to_binary(1.99999988,[{minor_version,1}]). > >>> > >>> <<131,70,63,255,255,255,223,201,169,173>> > >>> > >>> doesnt seem to be the same thatn you spect to see after reading that > >>> page... > >>> > >>> ..What im doing wrong....? > >>> > >>> ieee754 layout.... > >>> > >>> +1.99999976 > >>> > >>> 0x3FFFFFFE > >>> > >>> 1073741822 > >>> > >>> +1.99999988 > >>> > >>> 0x3FFFFFFF > >>> > >>> 1073741823 > >>> > >>> +2.00000000 > >>> > >>> 0x40000000 > >>> > >>> 1073741824 > >>> > >>> +2.00000024 > >>> > >>> 0x40000001 > >>> > >>> 1073741825 > >>> > >>> +2.00000048 > >>> > >>> 0x40000002 > >>> > >>> 1073741826 > >>> > >>> Thanks!.. > >>> > >>> /Angel > >>> > >>> > >>> > >>> > >>> > >>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions From desired.mta@REDACTED Mon May 14 17:47:42 2012 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Mon, 14 May 2012 16:47:42 +0100 Subject: [erlang-questions] Dialyzer, Callback info about the behaviour is not available In-Reply-To: References: <7D661B20-3D54-4FBC-95EF-EF846C92F48F@gmail.com> <4F057E22.7080509@cs.ntua.gr> Message-ID: <20120514154742.GA12528@localhost> Sorry for raising and old topic, but... On Thu, Jan 05, 2012 at 10:08:43PM +0100, Magnus Klaar wrote: > Hi! > > We created a development branch for adding this information to this, and > other behaviors and came to the same conclusion, we'd have to drop R14 > support, something which we cannot do when we know that there are users who > still haven't upgraded their production environments to R15. It was a > rather negative surprise to see the build fail after having put great care > into making sure the -callback specifications were accurate. ;) Hm, do you still have that development branch so I can check my code with dialyzer for my own sake? > Since we're on the subject, is there currently a good way to add > specifications for optional callback functions? If it causes compile failure on References: <054C72CB-D24C-498D-9879-9FDDDFCF41B9@rytong.com> Message-ID: <0EB0E29F-E5CC-47FD-94A1-858A84855326@rytong.com> Finally, I found a solution from the Portfile of erlang on Macports. After built erlang with LDFLAGS -framework CoreFoundation, these drivers can now be loaded with SMP enabled. ? 2012-5-12???10:20? cao xu ??? > I got the stack info from core file when the VM crashed: > > (gdb) up > #1 0x000000001c0c77c1 in erts_sys_main_thread () at sys.c:2905 > 2905 res = select(erts_darwin_main_thread_pipe[0] + 1, &readfds, NULL, NULL, NULL); > (gdb) up > #2 0x000000001bfe5bc4 in erl_start (argc=12, argv=0x7fff6bbcb5a8) > at erl_init.c:1483 > 1483 erts_sys_main_thread(); /* May or may not return! */ > (gdb) up > #3 0x6c61636f6c2f7273 in ?? () > (gdb) > > > and after I turned off the SMP, the driver worked again. Does that mean it have something to do with the SMP support? > > BR > > ? 2012-5-11???11:58? cao xu ??? > >> Hi, >> >> I tried to update the OS to Lion, but found drivers those work on 10.6 have problem during the start. The following error prompted and the VM crashed : >> >> Erlang R15B (erts-5.9) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] >> >> Eshell V5.9 (abort with ^G) >> 1> erl_ddll:load_driver("priv", xml_eng_drv). >> Trace/BPT trap: 5 >> >> Has it happened to anyone else? I also found some of the drivers still work, is there something I should fix when migrate linked-in drivers to Mac OS Lion? Any suggestion will be appreciated. >> >> BR >> >> Xu Cao >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From wallentin.dahlberg@REDACTED Mon May 14 17:54:37 2012 From: wallentin.dahlberg@REDACTED (=?ISO-8859-1?Q?Bj=F6rn=2DEgil_Dahlberg?=) Date: Mon, 14 May 2012 17:54:37 +0200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: <79520DF4-2E14-4CA4-8880-0B03E4D6B8DD@cs.otago.ac.nz> <4CEA1F2B-BA47-4E61-AF44-541573228096@cs.otago.ac.nz> <84A5A874-89F2-42C2-B746-AFA98601936A@gmail.com> <1337000892.4744.37.camel@seasc1137> Message-ID: It could probably be done. That does not mean it should be done. The preloaded files during development are a pain as it is. Please do not add to it =D // Bj?rn-Egil 2012/5/14 Anthony Ramine > There is already a chicken-and-egg problem, that's why there are some BEAM > files > in the Git repository, look in erts/preloaded/ebin. > > Nothing prevents us from generating these C files from some Erlang code and > versioning them in Git too. This way the system would be bootstrapped from > the > previously generated files in the repository. > > -- > Anthony Ramine > > > > > Le 14 mai 2012 ? 15:08, Bengt Kleberg a ?crit : > > > Greetings, > > > > Perhaps there is a chicken-and-egg problem with requiring Erlang to > > generate files used to build Erlang? > > > > > > bengt > > > > On Mon, 2012-05-14 at 15:02 +0200, Anthony Ramine wrote: > >> Couldn't some of the bootstrap Perl scripts like beam_makeops and > make_tables be > >> rewritten and documented in Erlang? I think it would make things more > obvious if > >> they were not obscure Perl scripts without comments. Furthermore it > would make > >> Erlang/OTP eat more of its own dog food. > >> > >> The only thing that would need to be changed with regard to the > bootstrap itself > >> is that their output would have to be versioned just as the > erts/preloaded/ BEAM > >> files. A new command should also be added to otp_build to update them. > >> > >> Some of these generated files are: > >> > >> beam_tr_funcs.h > >> beam_pred_funcs.h > >> beam_hot.h > >> beam_cold.h > >> beam_opcodes.c > >> beam_opcodes.h > >> beam_opcodes.erl > >> beam_opcodes.hrl > >> erl_am.c > >> erl_bif_table.c > >> erl_bif_wrap.c > >> erl_pbifs.c > >> erl_atom_table.h > >> erl_bif_table.h > >> > >> There may be an obvious reason for them not to be generated by Erlang > itself but > >> I'm not aware of it. > >> > >> Regards. > >> > >> -- > >> Anthony Ramine > >> > >> Le 9 mai 2012 ? 01:58, Richard O'Keefe a ?crit : > >> > >>> Let me illustrate the Icon approach by showing you a fragment of the > >>> micro-BEAM I wrote to get the performance numbers in the frames > proposal. > >>> (The whole thing is fragmentary.) > >>> > >>> ... > >>> @i > >>> max src, snd, dst > >>> @d > >>> dst := max(src, snd) > >>> > >>> This computes the maximum using the micro-Erlang term ordering. > >>> If src and snd are tagged immediate integers the comparison is > >>> done inline; the compare() function is called otherwise. > >>> @c > >>> T = @src; > >>> U = @snd; > >>> @dst = cmp(T, >, U) ? T : U; > >>> @step; > >>> @e > >>> ... > >>> @i > >>> check_record src, size, const > >>> @d > >>> Type test. > >>> > >>> Fail unless src is tagged as a pointer to a tuple or frame, > >>> the first word it points to is size, and the second is the > >>> const (which must be an atom, but we don't check that). > >>> Used for record matching. > >>> @c > >>> T = @src; > >>> if (!is_tuple(T)) @fail "is_record"; else > >>> if (FIELD(T, TUP_TAG, 0) != @size) @fail "is_record"; else > >>> if (FIELD(T, TUP_TAG, 1) != @const) @fail "is_record"; else > >>> @step; > >>> @e > >>> ... > >>> There is a preprocessor written in AWK that turns this into > >>> several C files. One of them is the emulator cases. For > >>> the check_record instruction you get > >>> > >>> #line 75 "frame.master" > >>> case CHECK_RECORD: > >>> #line 76 "frame.master" > >>> T = reg[(int)P[1]]; > >>> #line 77 "frame.master" > >>> if (!is_tuple(T)) > >>> P = failure, operation = "is_record"; else > >>> #line 78 "frame.master" > >>> if (FIELD(T, TUP_TAG, 0) != 4) > >>> P = failure, operation = "is_record"; else > >>> #line 79 "frame.master" > >>> if (FIELD(T, TUP_TAG, 1) != P[3]) > >>> P = failure, operation = "is_record"; else > >>> #line 80 "frame.master" > >>> P += 4; > >>> break; > >>> > >>> where I've broken the long lines (the preprocessor doesn't). > >>> The #line directives are option. > >>> > >>> @i introduces an instruction; the next line is a template > >>> for it saying what the operands are. > >>> @d introduces the description for people. > >>> @c introduces the code. In it, various built-in @macros > >>> are expanded. > >>> > >>> One advantage of doing it this way is that by using > >>> @step to update the PC I *cannot* get the offset wrong; > >>> the preprocessor counted the operands and their sizes > >>> for me. Similarly, what I write has *no* operand numbers; > >>> the preprocessor counted those, and supplies all necessary > >>> casts as well. I can shuffle operands around (in @i) > >>> without revising the code (in @c), and have. > >>> > >>> It wouldn't be too hard to write another preprocessor that > >>> built some kind of documentation (HTML would probably be > >>> easiest) out of this, but since this was an experiment, > >>> it didn't seem worth while. > >>> > >>> Why did I write the preprocessor? > >>> Well, to be honest, the first draft didn't use one. > >>> I got a bit sick of debugging, and wrote the preprocessor > >>> (based on vague memories of Icon) to eliminate a class of > >>> errors. It turned out to be _easier_ to develop a > >>> documented emulator than an undocumented one. > >>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Mon May 14 19:26:11 2012 From: bob@REDACTED (Bob Ippolito) Date: Mon, 14 May 2012 11:26:11 -0600 Subject: [erlang-questions] msg mailbox and gen_svr shutdown In-Reply-To: <4FB091D6.8000409@ninenines.eu> References: <4FB052FF.9080604@ferd.ca> <4FB091D6.8000409@ninenines.eu> Message-ID: I agree that graceful shutdown is very application-specific. However, cowboy doesn't currently facilitate any sort of graceful shutdown unless you read the source code and poke directly at the appropriate supervisors like I did. The application specific stuff is easily done on your own with a process registry or a timeout, we used a combination of gproc and a timeout if things didn't shut down in an acceptable time frame. I would suggest something like cowboy:stop_listener/1, maybe something like cowboy:stop_listening/1 or cowboy:stop_accepting/1. On Sun, May 13, 2012 at 11:02 PM, Lo?c Hoguin wrote: > Right, this is pretty much what I said to Paul in PM. It was in R14B03 > that the behavior changed. I apparently have a @todo wrong past that > release, and will take a look if I find other things to fix in the docs. > > Copy pasting my private reply on this: > > Ultimately if we remove the listener I think we want to stop everything. > > For "server is overloaded" situations, you can very well use the > 'onrequest' hook which can be set or changed dynamically through > cowboy:set_protocol_options, in addition to giving it in start_listener. > Takes a fun that has a single arg as a Req, returns a Req, and if you > replied from within it it doesn't dispatch the request and stops there (you > can also force close the connection by setting the Connection header to > "close"). > > And adding this: > > For graceful shutdowns, well it's highly application dependent. Some apps > are just short lived connections, so not accepting requests plus a short > delay ought to do it. Some are long lived, which probably requires to send > a shutdown message. Some apps may have connections critical enough that you > don't want to shutdown them. It's up to the application implementor to > devise the strategy to use for stopping. > > > On 05/14/2012 02:34 AM, Fred Hebert wrote: > >> This is just a guess, but is it possible that this is due to the fact >> Cowboy is using simple_one_for_one supervision? In Pre R15, if I recall >> correctly, the shutdown of sofo supervisors was asynchronous. The >> supervisor would just die and let its children figure out it was gone. >> >> Starting with R15, things started being synchronous and the supervisor >> would wait. A brutal kill that made things work fine before R15 >> (excluding the issue of the application master killing everything) could >> start breaking in later versions. >> >> Again, this is just a guess, without looking at the source or anything. >> >> On Sun May 13 14:12:18 2012, Bob Ippolito wrote: >> >>> Yes, it is documented that request processes continue after you stop >>> the listener. This is incorrect. >>> >>> On Sunday, May 13, 2012, Anthony Ramine wrote: >>> >>> Pawe?, from what I gather your supervisor waits 1 ms before >>> killing its children; I think it's pretty normal that some >>> messages are left unprocessed. >>> Bob, cowboy_requests_sup' shutdown strategy is brutal_kill, is it >>> documented somewhere that Cowboy waits before killing requests' >>> processes? >>> >>> Regards, >>> >>> -- >>> Anthony Ramine >>> >>> >>> >>> >>> Le 10 mai 2012 ? 22:37, Bob Ippolito a ?crit : >>> >>> I had the same problem with Cowboy recently, really ought to file >>>> an issue to at least fix the docs for that. >>>> >>>> For my app I wanted to implement a graceful shutdown that stopped >>>> accepting new connections but allowed existing requests to >>>> finish. The waiting part I implemented by adding a gproc local >>>> property to the workers that I wanted to wait for and then >>>> monitoring them. >>>> >>>> Looks like this: >>>> https://gist.github.com/**2655724 >>>> >>>> On Thu, May 10, 2012 at 2:53 AM, Pawe? Peregud >>>> >>> 'paulperegud@REDACTED');>> wrote: >>>> >>>> I was having fun with supervisors yesterday (Cowboy seems to >>>> fail to fulfill the promise of not killing request processes >>>> after listener removal) and I have an example. I've only >>>> investigated the case when supervisor is killed, so YMMV. >>>> Example code is attached. You may modify it to check the >>>> behavior in your case. >>>> >>>> Start supervisor tree with exp_sup_sup:start_link(). Execute >>>> test with exp_sup_sup:test() and exp_sup_sup:test_simple(). >>>> >>>> In case of dying supervisor the answer is "no, it does not". >>>> >>>> When supervisor dies, your process is killed as via link >>>> mechanism, so it may leave some unprocessed messages in >>>> inbox. To make sure that every delivered message is served, >>>> you need to add process_flag(trap_exit, true). Messages that >>>> are sent after the moment when supervisor dies are not processed. >>>> >>>> Best regards, >>>> >>>> Paul. >>>> >>>> >>>> On May 9, 2012 11:06 AM, "Andy Richards" >>>> >>> 'andy.richards.iit@REDACTED**com ');>> >>>> wrote: >>>> >>>> Hi, >>>> >>>> I can't seem to see any confirmation in the documentation >>>> so was wondering if anyone could confirm if messages are >>>> still sent to a supervised gen_svr following a shutdown >>>> message? >>>> >>>> If so how do I cleanly shutdown my gen_svr without >>>> loosing messages? I read in the supervisor child spec >>>> that a shutdown can be set to infinity which i hoped >>>> would allow me to process the msg's in my mailbox but if >>>> I do this will my module continue to receive messages >>>> from other processes? Is my approach flawed and if so >>>> what other ways are there to cleanly shutting down my >>>> gen_svr without loosing messages? >>>> >>>> Many thanks, >>>> >>>> Andy. >>>> ______________________________**_________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>> 'erlang-questions@REDACTED')**;> >>>> http://erlang.org/mailman/**listinfo/erlang-questions >>>> >>>> >>>> ______________________________**_________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>> 'erlang-questions@REDACTED')**;> >>>> http://erlang.org/mailman/**listinfo/erlang-questions >>>> >>>> >>>> ______________________________**_________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>> 'erlang-questions@REDACTED')**;> >>>> http://erlang.org/mailman/**listinfo/erlang-questions >>>> >>> >>> >>> >>> ______________________________**_________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/**listinfo/erlang-questions >>> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Mon May 14 20:38:42 2012 From: dangud@REDACTED (Dan Gudmundsson) Date: Mon, 14 May 2012 20:38:42 +0200 Subject: [erlang-questions] Driver load problem on Mac OS Lion In-Reply-To: <0EB0E29F-E5CC-47FD-94A1-858A84855326@rytong.com> References: <054C72CB-D24C-498D-9879-9FDDDFCF41B9@rytong.com> <0EB0E29F-E5CC-47FD-94A1-858A84855326@rytong.com> Message-ID: Nice find. Maybe erlang should by default link against that, can you make a patch? /Dan On Mon, May 14, 2012 at 5:53 PM, cao xu wrote: > Finally, I found a solution from the Portfile of erlang on Macports. After built erlang with LDFLAGS -framework CoreFoundation, these drivers can now be loaded with SMP enabled. > > > ? 2012-5-12???10:20? cao xu ??? > >> I got the stack info from core file when the VM crashed: >> >> (gdb) up >> #1 0x000000001c0c77c1 in erts_sys_main_thread () at sys.c:2905 >> 2905 res = select(erts_darwin_main_thread_pipe[0] + 1, &readfds, NULL, NULL, NULL); >> (gdb) up >> #2 0x000000001bfe5bc4 in erl_start (argc=12, argv=0x7fff6bbcb5a8) >> at erl_init.c:1483 >> 1483 erts_sys_main_thread(); /* May or may not return! */ >> (gdb) up >> #3 0x6c61636f6c2f7273 in ?? () >> (gdb) >> >> >> and after I turned off the SMP, the driver worked again. Does that mean it have something to do with the SMP support? >> >> BR >> >> ? 2012-5-11???11:58? cao xu ??? >> >>> Hi, >>> >>> I tried to update the OS to Lion, but found drivers those work on 10.6 have problem during the start. The following error prompted and the VM crashed : >>> >>> Erlang R15B (erts-5.9) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] >>> >>> Eshell V5.9 (abort with ^G) >>> 1> erl_ddll:load_driver("priv", xml_eng_drv). >>> Trace/BPT trap: 5 >>> >>> Has it happened to anyone else? I also found some of the drivers still work, is there something I should fix when migrate linked-in drivers to Mac OS Lion? Any suggestion will be appreciated. >>> >>> BR >>> >>> Xu Cao >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From tuncer.ayaz@REDACTED Mon May 14 22:30:38 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Mon, 14 May 2012 22:30:38 +0200 Subject: [erlang-questions] windows In-Reply-To: References: <111ECCC1-8CC3-4C79-A331-28161A5FFFC2@rogvall.se> Message-ID: On Sun, May 13, 2012 at 10:44 PM, Tony Rogvall wrote: > The next thing was that I specified: > > {"(win32)","priv/dthread_drv.dll", > ["c_src/dthread.c", "c_src/dthread_drv.c"]}, You don't need that win32 rule. rebar automatically switches the extension on Windows as follows: 1. priv/foo_drv.so -> priv/foo_drv.dll 2. priv/foo -> priv/foo.exe > This leads to the (bug?) that the spec ended up as {spec, drv, > ".dll", ...} the file was linked as .dll, no wonder I could not find > it :-) Thanks for the bug report. Can you confirm it's fixed in the ta-dll branch? From tony@REDACTED Mon May 14 23:04:40 2012 From: tony@REDACTED (Tony Rogvall) Date: Mon, 14 May 2012 23:04:40 +0200 Subject: [erlang-questions] windows In-Reply-To: References: <111ECCC1-8CC3-4C79-A331-28161A5FFFC2@rogvall.se> Message-ID: <3582453A-BC00-47D5-B9AD-2E5AED675033@rogvall.se> On 14 maj 2012, at 22:30, Tuncer Ayaz wrote: > On Sun, May 13, 2012 at 10:44 PM, Tony Rogvall wrote: >> The next thing was that I specified: >> >> {"(win32)","priv/dthread_drv.dll", >> ["c_src/dthread.c", "c_src/dthread_drv.c"]}, > > You don't need that win32 rule. > rebar automatically switches the extension on Windows as follows: > 1. priv/foo_drv.so -> priv/foo_drv.dll > 2. priv/foo -> priv/foo.exe > Yes I noticed ;-) >> This leads to the (bug?) that the spec ended up as {spec, drv, >> ".dll", ...} the file was linked as .dll, no wonder I could not find >> it :-) > > Thanks for the bug report. Can you confirm it's fixed in the ta-dll > branch? Next time I start windows. Unfortunately this week. Thanks /Tony "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue May 15 00:23:07 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 15 May 2012 10:23:07 +1200 Subject: [erlang-questions] erlang float comparison In-Reply-To: <201205140941.01511.clist@uah.es> References: <201205140941.01511.clist@uah.es> Message-ID: <35F323E4-6707-4E22-9F70-BAD5A91803BD@cs.otago.ac.nz> On 14/05/2012, at 7:41 PM, Angel J. Alvarez Miguel wrote: > > Hi guys > > > I need to compare two floats something like 0.99999998... vs 1.000000 > and we came across accuracy problems when testing dihedral angles on a molecule... > > > so we read http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm You did read the bit at the top, didn't you? As the author has said elsewhere, Years ago I wrote an article about how to do epsilon floating-point comparisons by using integer comparisons. That article has been quite popular (it is frequently cited, and the code samples have been used by a number of companies) and this worries me a bit, because the article has some flaws. I?m not going to link to the article because I want to replace it, not send people looking for it. The crucial questions are "what exactly do you want to accomplish, *semantically*, and what is the problem you are *really* trying to solve by doing that?" IEEE formats look simple until you discover that there are, in reality, endianness issues and worse. For my Smalltalk system I had to implement a number of functions that I couldn't rely on being present in C; the code is unpleasantly riddled with machine-dependent #ifdefs. The way things are stored in memory and the way they are stored in registers aren't always related the way you expect. > is think we should start with... > > <> = term_to_binary(1.9999976,[{minor_version,1}]). http://www.erlang.org/doc/man/erlang.html#term_to_binary-2 speaks of "THE 64-bit IEEE format", but of course there is no such animal. The IEEE format is a register format; both floats and an integers have endianness issues, and it looks as though here they do not match. From ok@REDACTED Tue May 15 00:26:02 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 15 May 2012 10:26:02 +1200 Subject: [erlang-questions] erlang float comparison In-Reply-To: <201205141047.19368.clist@uah.es> References: <201205140941.01511.clist@uah.es> <201205141047.19368.clist@uah.es> Message-ID: <5D09443F-17DF-4B17-A10B-3ED5C10CCC6E@cs.otago.ac.nz> On 14/05/2012, at 8:47 PM, Angel J. Alvarez Miguel wrote: > 1.000000000000000002 > 1.0. That's because in 64-bit arithmetic, 1.000000000000000002 *is* 1.0. From ok@REDACTED Tue May 15 00:38:56 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 15 May 2012 10:38:56 +1200 Subject: [erlang-questions] erlang float comparison In-Reply-To: <201205141553.52104.clist@uah.es> References: <201205140941.01511.clist@uah.es> <201205141047.19368.clist@uah.es> <4FB10498.6000005@telegraphics.com.au> <201205141553.52104.clist@uah.es> Message-ID: <8FCB9E29-2B55-4D82-8957-0AF108178091@cs.otago.ac.nz> On 15/05/2012, at 1:53 AM, Angel J. Alvarez Miguel wrote: > > Sorry folks, im just a bit puzzled for the strangeness of fp operations > > I am talking about things like this: > > 8> 1.0000000000000001 > 1.0. > false > 9> 1.0000000000000002 > 1.0. > true > > And i wander that there is some method for avoiding such beasts.. There is nothing in the least strange about those results. Floating point numbers are not mathematical real numbers. They have a bounded, indeed a fixed, precision. Single-precision IEEE floats have about 7 digits; double-precision ones have about 16 digits. Considering only strictly positive numbers, and being rather vaguer about the details than I'd like, in order to keep this simple, double precision can distinguish x from y only when |x-y|/max(|x|,|y|) is greater than about 10**-16. When you enter a decimal number like 1.0000000000000001 it is rounded to the nearest number that can be represented as a float, and that happens to be 1.0. > > Well ill be happy just if i can compare if Something > 1.0 and it just returns > false unless Something is really greater than 1.0 (whatever the accuracy is) There is not the least problem here with > . The problem is that the Something you *type* (or try to calculate) is not the Something' you *get*; the very best you have any right to hope for is that Something' is Something rounded to the nearest representable number. For any single + - * / or sqrt, that's what you get. For anything more complicated, Good Luck (you're going to need it)! Oh yeah, one other thing: decimal->binary conversion and binary-> decimal conversion are quite hard to get right. I believe Scheme was the first programming language to require that the results be as good as possible. C does not. In particular, you should not be surprised if scanf(printf(x)) != x; since other languages often use the C libraries for binary<->decimal conversion, you can expect similar problems in any language that does not explicitly guarantee best-possible binary<->decimal conversion (and life being what it is, don't be too surprised to find problems in implementations of languages that _do_ make such guarantees; implementors may be unaware of the difficulties). You know, I have to wonder what the people who fund the development of autonomous weapons think they are playing at, sometimes... From toby@REDACTED Tue May 15 02:34:04 2012 From: toby@REDACTED (Toby Thain) Date: Mon, 14 May 2012 20:34:04 -0400 Subject: [erlang-questions] erlang float comparison In-Reply-To: <201205141553.52104.clist@uah.es> References: <201205140941.01511.clist@uah.es> <201205141047.19368.clist@uah.es> <4FB10498.6000005@telegraphics.com.au> <201205141553.52104.clist@uah.es> Message-ID: <4FB1A47C.3040604@telegraphics.com.au> On 14/05/12 9:53 AM, Angel J. Alvarez Miguel wrote: > On Lunes, 14 de Mayo de 2012 15:11:52 usted escribi?: >> On 14/05/12 4:47 AM, Angel J. Alvarez Miguel wrote: >>> Well we just picked the wrong example >>> >>> >>> that's is the problem >>> >>> sinosuke@REDACTED:~> erl >>> Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] >>> [hipe] [kernel-poll:false] >>> >>> [QuickCheck] [PropEr] [Erl0MQ] >>> Eshell V5.9.1 (abort with ^G) >>> 1> 1.000000000000000002> 1.0. >>> false >> >> It's not clear whether you need an approximate equality test as >> suggested by others (tolerance won't help you with> or< or the example >> above) or whether you need more (or arbitrary) precision. > > Sorry folks, im just a bit puzzled for the strangeness of fp operations > > I am talking about things like this: > > 8> 1.0000000000000001> 1.0. > false > 9> 1.0000000000000002> 1.0. > true As others have pointed out, this is not strange at all. These are the same values, once precision has been lost by converting the decimal string to floating point (which has limited precision, see e.g.[1]). > > And i wander that there is some method for avoiding such beasts.. > > Well ill be happy just if i can compare if Something> 1.0 and it just returns > false unless Something is really greater than 1.0 (whatever the accuracy is) > so i can relay on doing things like > > if Angle> 1.0 the shutdown_orbital_laser_before_its_too_late(). > To PROPERLY make these comparisons, you need more precision (e.g. BigDecimal, or fixed point/rational which would be simple with Erlang's bignum integers). There are many other issues with floating point, but to deal with the above < and > comparisons, you simply need sufficient precision. --Toby [1] http://en.wikipedia.org/wiki/IEEE_754-1985 > because we found that: > math:acos(0.99999999999999991). > 1.4901161193847656e-8 > > math:acos(0.99999999999999992). > 1.4901161193847656e-8 > > math:acos(0.99999999999999993). > 1.4901161193847656e-8 > > math:acos(0.99999999999999999). > 0.0 > > math:acos(1.0000000000000001). > 0.0 > > math:acos(1.0000000000000002). > ** exception error: bad argument in an arithmetic expression > in function math:acos/1 > called as math:acos(1.0000000000000002) > > this last case is waht we wanted to avoid, specifically > > /Angel > >> >> --Toby >> >>> On Lunes, 14 de Mayo de 2012 09:50:57 usted escribi?: >>>> 1> 0.99999998< 1.000000. >>>> true >>>> 2> 1.99999976< 1.99999988. >>>> true >>>> >>>> What's the problem? >>>> >>>> On Mon, May 14, 2012 at 9:41 AM, Angel J. Alvarez Miguel >>> >>> wrote: >>>>> ** >>>>> >>>>> >>>>> Hi guys >>>>> >>>>> >>>>> >>>>> I need to compare two floats something like 0.99999998... vs 1.000000 >>>>> >>>>> and we came across accuracy problems when testing dihedral angles on a >>>>> molecule... >>>>> >>>>> >>>>> >>>>> so we read >>>>> http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.h >>>>> tm >>>>> >>>>> and wanted to implement something like: >>>>> >>>>> >>>>> >>>>> if (*(int*)&f1< *(int*)&f2)... >>>>> >>>>> >>>>> >>>>> when ef1 and f2 are floats... (f1 about 0.9999899.. and f2=1.0 >>>>> >>>>> >>>>> >>>>> is think we should start with... >>>>> >>>>> <> = >>>>> term_to_binary(1.9999976,[{minor_version,1}]). >>>>> >>>>> and let Padding swallow the external format and tag while Myint gets >>>>> the ieee754 float >>>>> >>>>> >>>>> >>>>> But MyInt = 4611686017346523993 instead of 1073741822 >>>>> >>>>> these conversion should follow ieee754 and being lexicografic ordered >>>>> >>>>> but ... >>>>> >>>>> term_to_binary(1.99999976,[{minor_version,1}]). >>>>> >>>>> <<131,70,63,255,255,255,191,147,83,89>> >>>>> >>>>> term_to_binary(1.99999988,[{minor_version,1}]). >>>>> >>>>> <<131,70,63,255,255,255,223,201,169,173>> >>>>> >>>>> doesnt seem to be the same thatn you spect to see after reading that >>>>> page... >>>>> >>>>> ..What im doing wrong....? >>>>> >>>>> ieee754 layout.... >>>>> >>>>> +1.99999976 >>>>> >>>>> 0x3FFFFFFE >>>>> >>>>> 1073741822 >>>>> >>>>> +1.99999988 >>>>> >>>>> 0x3FFFFFFF >>>>> >>>>> 1073741823 >>>>> >>>>> +2.00000000 >>>>> >>>>> 0x40000000 >>>>> >>>>> 1073741824 >>>>> >>>>> +2.00000024 >>>>> >>>>> 0x40000001 >>>>> >>>>> 1073741825 >>>>> >>>>> +2.00000048 >>>>> >>>>> 0x40000002 >>>>> >>>>> 1073741826 >>>>> >>>>> Thanks!.. >>>>> >>>>> /Angel >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > From essen@REDACTED Tue May 15 03:31:48 2012 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 15 May 2012 03:31:48 +0200 Subject: [erlang-questions] msg mailbox and gen_svr shutdown In-Reply-To: References: <4FB052FF.9080604@ferd.ca> <4FB091D6.8000409@ninenines.eu> Message-ID: <4FB1B204.9030006@ninenines.eu> This will be possible later on by reducing the number of acceptors to 0. This should be added sometimes this summer after the acceptor split happens in Cowboy. On 05/14/2012 07:26 PM, Bob Ippolito wrote: > I agree that graceful shutdown is very application-specific. However, > cowboy doesn't currently facilitate any sort of graceful shutdown unless > you read the source code and poke directly at the appropriate > supervisors like I did. The application specific stuff is easily done on > your own with a process registry or a timeout, we used a combination of > gproc and a timeout if things didn't shut down in an acceptable time frame. > > I would suggest something like cowboy:stop_listener/1, maybe something > like cowboy:stop_listening/1 or cowboy:stop_accepting/1. > > On Sun, May 13, 2012 at 11:02 PM, Lo?c Hoguin > wrote: > > Right, this is pretty much what I said to Paul in PM. It was in > R14B03 that the behavior changed. I apparently have a @todo wrong > past that release, and will take a look if I find other things to > fix in the docs. > > Copy pasting my private reply on this: > > Ultimately if we remove the listener I think we want to stop everything. > > For "server is overloaded" situations, you can very well use the > 'onrequest' hook which can be set or changed dynamically through > cowboy:set_protocol_options, in addition to giving it in > start_listener. Takes a fun that has a single arg as a Req, returns > a Req, and if you replied from within it it doesn't dispatch the > request and stops there (you can also force close the connection by > setting the Connection header to "close"). > > And adding this: > > For graceful shutdowns, well it's highly application dependent. Some > apps are just short lived connections, so not accepting requests > plus a short delay ought to do it. Some are long lived, which > probably requires to send a shutdown message. Some apps may have > connections critical enough that you don't want to shutdown them. > It's up to the application implementor to devise the strategy to use > for stopping. > > > On 05/14/2012 02:34 AM, Fred Hebert wrote: > > This is just a guess, but is it possible that this is due to the > fact > Cowboy is using simple_one_for_one supervision? In Pre R15, if I > recall > correctly, the shutdown of sofo supervisors was asynchronous. The > supervisor would just die and let its children figure out it was > gone. > > Starting with R15, things started being synchronous and the > supervisor > would wait. A brutal kill that made things work fine before R15 > (excluding the issue of the application master killing > everything) could > start breaking in later versions. > > Again, this is just a guess, without looking at the source or > anything. > > On Sun May 13 14:12:18 2012, Bob Ippolito wrote: > > Yes, it is documented that request processes continue after > you stop > the listener. This is incorrect. > > On Sunday, May 13, 2012, Anthony Ramine wrote: > > Pawe?, from what I gather your supervisor waits 1 ms before > killing its children; I think it's pretty normal that some > messages are left unprocessed. > Bob, cowboy_requests_sup' shutdown strategy is brutal_kill, > is it > documented somewhere that Cowboy waits before killing requests' > processes? > > Regards, > > -- > Anthony Ramine > > > > > Le 10 mai 2012 ? 22:37, Bob Ippolito a ?crit : > > I had the same problem with Cowboy recently, really > ought to file > an issue to at least fix the docs for that. > > For my app I wanted to implement a graceful shutdown > that stopped > accepting new connections but allowed existing requests to > finish. The waiting part I implemented by adding a gproc > local > property to the workers that I wanted to wait for and then > monitoring them. > > Looks like this: > https://gist.github.com/__2655724 > > > On Thu, May 10, 2012 at 2:53 AM, Pawe? Peregud > > 'paulperegud@REDACTED > ');>> wrote: > > I was having fun with supervisors yesterday (Cowboy seems to > fail to fulfill the promise of not killing request processes > after listener removal) and I have an example. I've only > investigated the case when supervisor is killed, so YMMV. > Example code is attached. You may modify it to check the > behavior in your case. > > Start supervisor tree with exp_sup_sup:start_link(). Execute > test with exp_sup_sup:test() and exp_sup_sup:test_simple(). > > In case of dying supervisor the answer is "no, it does not". > > When supervisor dies, your process is killed as via link > mechanism, so it may leave some unprocessed messages in > inbox. To make sure that every delivered message is served, > you need to add process_flag(trap_exit, true). Messages that > are sent after the moment when supervisor dies are not > processed. > > Best regards, > > Paul. > > > On May 9, 2012 11:06 AM, "Andy Richards" > > 'andy.richards.iit@REDACTED > ');>> wrote: > > Hi, > > I can't seem to see any confirmation in the documentation > so was wondering if anyone could confirm if messages are > still sent to a supervised gen_svr following a shutdown > message? > > If so how do I cleanly shutdown my gen_svr without > loosing messages? I read in the supervisor child spec > that a shutdown can be set to infinity which i hoped > would allow me to process the msg's in my mailbox but if > I do this will my module continue to receive messages > from other processes? Is my approach flawed and if so > what other ways are there to cleanly shutting down my > gen_svr without loosing messages? > > Many thanks, > > Andy. > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > 'cvml', > 'erlang-questions@REDACTED > ')__;> > http://erlang.org/mailman/__listinfo/erlang-questions > > > > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > 'cvml', > 'erlang-questions@REDACTED > ')__;> > http://erlang.org/mailman/__listinfo/erlang-questions > > > > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > 'cvml', > 'erlang-questions@REDACTED > ')__;> > http://erlang.org/mailman/__listinfo/erlang-questions > > > > > > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/__listinfo/erlang-questions > > > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/__listinfo/erlang-questions > > > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > > -- Lo?c Hoguin Erlang Cowboy Nine Nines From bob@REDACTED Tue May 15 03:34:25 2012 From: bob@REDACTED (Bob Ippolito) Date: Mon, 14 May 2012 19:34:25 -0600 Subject: [erlang-questions] msg mailbox and gen_svr shutdown In-Reply-To: <4FB1B204.9030006@ninenines.eu> References: <4FB052FF.9080604@ferd.ca> <4FB091D6.8000409@ninenines.eu> <4FB1B204.9030006@ninenines.eu> Message-ID: I don't think that is sufficient, you will need to stop listening as well so the next OS process can take over. On Monday, May 14, 2012, Lo?c Hoguin wrote: > This will be possible later on by reducing the number of acceptors to 0. > This should be added sometimes this summer after the acceptor split happens > in Cowboy. > > On 05/14/2012 07:26 PM, Bob Ippolito wrote: > >> I agree that graceful shutdown is very application-specific. However, >> cowboy doesn't currently facilitate any sort of graceful shutdown unless >> you read the source code and poke directly at the appropriate >> supervisors like I did. The application specific stuff is easily done on >> your own with a process registry or a timeout, we used a combination of >> gproc and a timeout if things didn't shut down in an acceptable time >> frame. >> >> I would suggest something like cowboy:stop_listener/1, maybe something >> like cowboy:stop_listening/1 or cowboy:stop_accepting/1. >> >> On Sun, May 13, 2012 at 11:02 PM, Lo?c Hoguin > > wrote: >> >> Right, this is pretty much what I said to Paul in PM. It was in >> R14B03 that the behavior changed. I apparently have a @todo wrong >> past that release, and will take a look if I find other things to >> fix in the docs. >> >> Copy pasting my private reply on this: >> >> Ultimately if we remove the listener I think we want to stop >> everything. >> >> For "server is overloaded" situations, you can very well use the >> 'onrequest' hook which can be set or changed dynamically through >> cowboy:set_protocol_options, in addition to giving it in >> start_listener. Takes a fun that has a single arg as a Req, returns >> a Req, and if you replied from within it it doesn't dispatch the >> request and stops there (you can also force close the connection by >> setting the Connection header to "close"). >> >> And adding this: >> >> For graceful shutdowns, well it's highly application dependent. Some >> apps are just short lived connections, so not accepting requests >> plus a short delay ought to do it. Some are long lived, which >> probably requires to send a shutdown message. Some apps may have >> connections critical enough that you don't want to shutdown them. >> It's up to the application implementor to devise the strategy to use >> for stopping. >> >> >> On 05/14/2012 02:34 AM, Fred Hebert wrote: >> >> This is just a guess, but is it possible that this is due to the >> fact >> Cowboy is using simple_one_for_one supervision? In Pre R15, if I >> recall >> correctly, the shutdown of sofo supervisors was asynchronous. The >> supervisor would just die and let its children figure out it was >> gone. >> >> Starting with R15, things started being synchronous and the >> supervisor >> would wait. A brutal kill that made things work fine before R15 >> (excluding the issue of the application master killing >> everything) could >> start breaking in later versions. >> >> Again, this is just a guess, without looking at the source or >> anything. >> >> On Sun May 13 14:12:18 2012, Bob Ippolito wrote: >> >> Yes, it is documented that request processes continue after >> you stop >> the listener. This is incorrect. >> >> On Sunday, May 13, 2012, Anthony Ramine wrote: >> >> Pawe?, from what I gather your supervisor waits 1 ms before >> killing its children; I think it's pretty normal that some >> messages are left unprocessed. >> Bob, cowboy_requests_sup' shutdown strategy is brutal_kill, >> is it >> documented somewhere that Cowboy waits before killing requests' >> processes? >> >> Regards, >> >> -- >> Anthony Ramine >> >> >> >> >> Le 10 mai 2012 ? 22:37, Bob Ippolito a ?crit : >> >> I had the same problem with Cowboy recently, really >> ought to file >> an issue to at least fix the docs for that. >> >> For my app I wanted to implement a graceful shutdown >> that stopped >> accepting new connections but allowed existing requests to >> finish. The waiting part I implemented by adding a gproc >> local >> property to the workers that I wanted to wait for and then >> monitoring them. >> >> Looks like this: >> https://gist.github.com/__**2655724 >> >> > >> >> On Thu, May 10, 2012 at 2:53 AM, Pawe? Peregud >> >> > 'paulperegud@REDACTED >> ');>> wrote: >> >> I was having fun with supervisors yesterday (Cowboy seems >> to >> fail to fulfill the promise of not killing request >> processes >> after listener removal) and I have an example. I've only >> investigated the case when supervisor is killed, so YMMV. >> Example code is attached. You may modify it to check the >> behavior in your case. >> >> Start supervisor tree with exp_sup_sup:start_link(). >> Execute >> test with exp_sup_sup:test() and exp_sup_sup:test_simple(). >> >> In case of dying supervisor the answer is "no, it does >> not". >> >> When supervisor dies, your process is killed as via link >> mechanism, so it may leave some unprocessed messages in >> inbox. To make sure that every delivered message is served, >> you need to add process_flag(trap_exit, true). Messages >> that >> are sent after the moment when supervisor dies are not >> processed. >> >> Best regards, >> >> Paul. >> >> >> On May 9, 2012 11:06 AM, "Andy Richards" >> > >> > 'andy.richards.iit@REDACTED**__com >> ');>> wrote: >> >> Hi, >> >> I can't seem to see any confirmation in the documentation >> so was wondering if anyone could confirm if messages are >> still sent to a supervised gen_svr following a shutdown >> message? >> >> If so how do I cleanly shutdown my gen_svr without >> loosing messages? I read in the supervisor child spec >> that a shutdown can be set to infinity which i hoped >> would allow me to process the msg's in my mailbox but if >> I do this will my module continue to receive messages >> from other processes? Is my approach flawed and if so >> what other ways are there to cleanly shutting down my >> gen_svr without loosing messages? >> >> Many thanks, >> >> Andy. >> ______________________________**___________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > 'cvml', >> 'erlang-questions@REDACTED >> ')__;> >> http://erlang.org/mailman/__**listinfo/erlang-questions >> >> > >> >> >> ______________________________**___________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > 'cvml', >> 'erlang-questions@REDACTED >> ')__;> >> http://erlang.org/mailman/__**listinfo/erlang-questions >> >> > >> >> >> ______________________________**___________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > 'cvml', >> 'erlang-questions@REDACTED >> ')__;> >> http://erlang.org/mailman/__**listinfo/erlang-questions >> >> > >> >> >> >> >> ______________________________**___________________ >> erlang-questions mailing list >> erlang-questions@REDACTED > erlang-questions@REDACTED> >> http://erlang.org/mailman/__**listinfo/erlang-questions >> >> > >> >> ______________________________**___________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/__**listinfo/erlang-questions >> >> > >> >> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> >> >> > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue May 15 04:45:29 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 15 May 2012 14:45:29 +1200 Subject: [erlang-questions] erlang float comparison In-Reply-To: <4FB1A47C.3040604@telegraphics.com.au> References: <201205140941.01511.clist@uah.es> <201205141047.19368.clist@uah.es> <4FB10498.6000005@telegraphics.com.au> <201205141553.52104.clist@uah.es> <4FB1A47C.3040604@telegraphics.com.au> Message-ID: <28C239C2-8DEA-447B-BDA9-7FBAB451B8C4@cs.otago.ac.nz> The original message said something about comparing dihedral angles in molecules. I find myself wondering what the actual problem is that the OP is trying to solve, and how much difference between two angles is *physically* reasonable. In one document I just found on the web, the H-O-H bond angle is given as 107.5 degrees in one place and 104.5 degrees in another. Another document calls it 105 degrees. My 1st-year chemistry days are long past, but web pages I've checked today seem not to quote any bond angles to better than 1 part in 1000 precision. And since angles are constrained to [0,one circle), it looks to me very much as if we have a case for representing bond angles and dihedral angles as *fixed point* numbers, or at any rate for rounding them to multiples of say (one circle)/65536 and comparing the angles as integers. From essen@REDACTED Tue May 15 05:15:54 2012 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 15 May 2012 05:15:54 +0200 Subject: [erlang-questions] msg mailbox and gen_svr shutdown In-Reply-To: References: <4FB052FF.9080604@ferd.ca> <4FB091D6.8000409@ninenines.eu> <4FB1B204.9030006@ninenines.eu> Message-ID: <4FB1CA6A.8080002@ninenines.eu> You stop listening after you're done gracefully stopping your currently running processes. For HTTP that could be something like: Set acceptors to 0. Optionally set 'onrequest' hook to reply with a 503 (for keepalives). Gracefully stop your processes. Stop the listener. Or do you need another process to take over immediately? Because in that case you usually don't need to use the same listening port, you can just change your firewall/lb rules for the port redirection from 80->P1 to 80->P2. On 05/15/2012 03:34 AM, Bob Ippolito wrote: > I don't think that is sufficient, you will need to stop listening as > well so the next OS process can take over. > > On Monday, May 14, 2012, Lo?c Hoguin wrote: > > This will be possible later on by reducing the number of acceptors > to 0. This should be added sometimes this summer after the acceptor > split happens in Cowboy. > > On 05/14/2012 07:26 PM, Bob Ippolito wrote: > > I agree that graceful shutdown is very application-specific. > However, > cowboy doesn't currently facilitate any sort of graceful > shutdown unless > you read the source code and poke directly at the appropriate > supervisors like I did. The application specific stuff is easily > done on > your own with a process registry or a timeout, we used a > combination of > gproc and a timeout if things didn't shut down in an acceptable > time frame. > > I would suggest something like cowboy:stop_listener/1, maybe > something > like cowboy:stop_listening/1 or cowboy:stop_accepting/1. > > On Sun, May 13, 2012 at 11:02 PM, Lo?c Hoguin > wrote: > > Right, this is pretty much what I said to Paul in PM. It was in > R14B03 that the behavior changed. I apparently have a @todo > wrong > past that release, and will take a look if I find other > things to > fix in the docs. > > Copy pasting my private reply on this: > > Ultimately if we remove the listener I think we want to stop > everything. > > For "server is overloaded" situations, you can very well use the > 'onrequest' hook which can be set or changed dynamically through > cowboy:set_protocol_options, in addition to giving it in > start_listener. Takes a fun that has a single arg as a Req, > returns > a Req, and if you replied from within it it doesn't dispatch the > request and stops there (you can also force close the > connection by > setting the Connection header to "close"). > > And adding this: > > For graceful shutdowns, well it's highly application > dependent. Some > apps are just short lived connections, so not accepting requests > plus a short delay ought to do it. Some are long lived, which > probably requires to send a shutdown message. Some apps may have > connections critical enough that you don't want to shutdown > them. > It's up to the application implementor to devise the > strategy to use > for stopping. > > > On 05/14/2012 02:34 AM, Fred Hebert wrote: > > This is just a guess, but is it possible that this is > due to the > fact > Cowboy is using simple_one_for_one supervision? In Pre > R15, if I > recall > correctly, the shutdown of sofo supervisors was > asynchronous. The > supervisor would just die and let its children figure > out it was > gone. > > Starting with R15, things started being synchronous and the > supervisor > would wait. A brutal kill that made things work fine > before R15 > (excluding the issue of the application master killing > everything) could > start breaking in later versions. > > Again, this is just a guess, without looking at the > source or > anything. > > On Sun May 13 14:12:18 2012, Bob Ippolito wrote: > > Yes, it is documented that request processes > continue after > you stop > the listener. This is incorrect. > > On Sunday, May 13, 2012, Anthony Ramine wrote: > > Pawe?, from what I gather your supervisor waits 1 ms > before > killing its children; I think it's pretty normal > that some > messages are left unprocessed. > Bob, cowboy_requests_sup' shutdown strategy is > brutal_kill, > is it > documented somewhere that Cowboy waits before > killing requests' > processes? > > Regards, > > -- > Anthony Ramine > > > > > Le 10 mai 2012 ? 22:37, Bob Ippolito a ?crit : > > I had the same problem with Cowboy recently, really > ought to file > an issue to at least fix the docs for that. > > For my app I wanted to implement a graceful shutdown > that stopped > accepting new connections but allowed existing > requests to > finish. The waiting part I implemented by adding > a gproc > local > property to the workers that I wanted to wait > for and then > monitoring them. > > Looks like this: > https://gist.github.com/____2655724 > > > > > On Thu, May 10, 2012 at 2:53 AM, Pawe? Peregud > > 'paulperegud@REDACTED > ');>> wrote: > > I was having fun with supervisors yesterday > (Cowboy seems to > fail to fulfill the promise of not killing > request processes > after listener removal) and I have an example. > I've only > investigated the case when supervisor is killed, > so YMMV. > Example code is attached. You may modify it to > check the > behavior in your case. > > Start supervisor tree with > exp_sup_sup:start_link(). Execute > test with exp_sup_sup:test() and > exp_sup_sup:test_simple(). > > In case of dying supervisor the answer is "no, > it does not". > > When supervisor dies, your process is killed as > via link > mechanism, so it may leave some unprocessed > messages in > inbox. To make sure that every delivered message > is served, > you need to add process_flag(trap_exit, true). > Messages that > are sent after the moment when supervisor dies > are not > processed. > > Best regards, > > Paul. > > > On May 9, 2012 11:06 AM, "Andy Richards" > > 'andy.richards.iit@REDACTED > ');>> wrote: > > Hi, > > I can't seem to see any confirmation in the > documentation > so was wondering if anyone could confirm if > messages are > still sent to a supervised gen_svr following a > shutdown > message? > > If so how do I cleanly shutdown my gen_svr without > loosing messages? I read in the supervisor child > spec > that a shutdown can be set to infinity which i hoped > would allow me to process the msg's in my > mailbox but if > I do this will my module continue to receive > messages > from other processes? Is my approach flawed and > if so > what other ways are there to cleanly shutting > down my > gen_svr without loosing messages? > > Many thanks, > > Andy. > ___________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > 'cvml', > 'erlang-questions@REDACTED > ')__;> > http://erlang.org/mailman/____listinfo/erlang-questions > > > > > > ___________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > 'cvml', > 'erlang-questions@REDACTED > ')__;> > http://erlang.org/mailman/____listinfo/erlang-questions > > > > > > ___________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > 'cvml', > 'erlang-questions@REDACTED > ')__;> > http://erlang.org/mailman/____listinfo/erlang-questions > > > > > > > > ___________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/____listinfo/erlang-questions > > > > > ___________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/____listinfo/erlang-questions > > > > > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > > > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > -- Lo?c Hoguin Erlang Cowboy Nine Nines From chandrashekhar.mullaparthi@REDACTED Tue May 15 06:06:32 2012 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Tue, 15 May 2012 05:06:32 +0100 Subject: [erlang-questions] SCTP multihoming Message-ID: Hi all, I'm having trouble starting a server with multihoming support. The call: 16> (<0.804.0>) call gen_sctp:open([binary, {active,once}, {reuseaddr,true}, {port,3868}, {ip,{10,249,8,4}}, {ip,{10,249,8,12}}]) crashes with the report below. Anyone has a quick fix? I've traced it down to this: (<0.2364.0>) call erlang:port_control(#Port<0.5560>,62,[[1],[[15,28],10,249,8,12],[[15,28],10,249,8,4]]) (<0.2364.0>) returned from erlang:port_control/3 -> [0,101,105,110,118,97,108] (which is einval basically) Any idea why this would be? I'm running this on Solaris 10, with R15B. cheers Chandru 22> rb:show(4). CRASH REPORT <0.866.0> 2012-05-15 04:44:30 =============================================================================== Crashing process initial_call {diameter_sctp,init,['Argument__1']} pid <0.866.0> registered_name [] error_info {error,badarg, [{gen_sctp,open, [[binary, {active,once}, {reuseaddr,true}, {port,3868}, {ip,{10,249,8,4}}, {ip,{10,249,8,12}}]], [{file,"gen_sctp.erl"},{line,126}]}, {diameter_sctp,open,3, [{file,"transport/diameter_sctp.erl"}, {line,227}]}, {diameter_sctp,i,1, [{file,"transport/diameter_sctp.erl"}, {line,147}]}, {diameter_sctp,init,1, [{file,"transport/diameter_sctp.erl"}, {line,141}]}, {proc_lib,init_p_do_apply,3, [{file,"proc_lib.erl"},{line,227}]}]} ancestors [diameter_sctp_listener_sup,diameter_transport_sup,diameter_sup, <0.779.0>] messages [] links [<0.803.0>] dictionary [] trap_exit false status running heap_size 75025 stack_size 24 reductions 736 -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Tue May 15 06:42:04 2012 From: bob@REDACTED (Bob Ippolito) Date: Mon, 14 May 2012 22:42:04 -0600 Subject: [erlang-questions] msg mailbox and gen_svr shutdown In-Reply-To: <4FB1CA6A.8080002@ninenines.eu> References: <4FB052FF.9080604@ferd.ca> <4FB091D6.8000409@ninenines.eu> <4FB1B204.9030006@ninenines.eu> <4FB1CA6A.8080002@ninenines.eu> Message-ID: Yes, I want the other process to immediately take over. Requiring a load balancer change isn't an elegant solution, I don't understand why you're trying to avoid adding this useful functionality. On Monday, May 14, 2012, Lo?c Hoguin wrote: > You stop listening after you're done gracefully stopping your currently > running processes. > > For HTTP that could be something like: > > Set acceptors to 0. > Optionally set 'onrequest' hook to reply with a 503 (for keepalives). > Gracefully stop your processes. > Stop the listener. > > Or do you need another process to take over immediately? Because in that > case you usually don't need to use the same listening port, you can just > change your firewall/lb rules for the port redirection from 80->P1 to > 80->P2. > > On 05/15/2012 03:34 AM, Bob Ippolito wrote: > >> I don't think that is sufficient, you will need to stop listening as >> well so the next OS process can take over. >> >> On Monday, May 14, 2012, Lo?c Hoguin wrote: >> >> This will be possible later on by reducing the number of acceptors >> to 0. This should be added sometimes this summer after the acceptor >> split happens in Cowboy. >> >> On 05/14/2012 07:26 PM, Bob Ippolito wrote: >> >> I agree that graceful shutdown is very application-specific. >> However, >> cowboy doesn't currently facilitate any sort of graceful >> shutdown unless >> you read the source code and poke directly at the appropriate >> supervisors like I did. The application specific stuff is easily >> done on >> your own with a process registry or a timeout, we used a >> combination of >> gproc and a timeout if things didn't shut down in an acceptable >> time frame. >> >> I would suggest something like cowboy:stop_listener/1, maybe >> something >> like cowboy:stop_listening/1 or cowboy:stop_accepting/1. >> >> On Sun, May 13, 2012 at 11:02 PM, Lo?c Hoguin > > wrote: >> >> Right, this is pretty much what I said to Paul in PM. It was in >> R14B03 that the behavior changed. I apparently have a @todo >> wrong >> past that release, and will take a look if I find other >> things to >> fix in the docs. >> >> Copy pasting my private reply on this: >> >> Ultimately if we remove the listener I think we want to stop >> everything. >> >> For "server is overloaded" situations, you can very well use >> the >> 'onrequest' hook which can be set or changed dynamically through >> cowboy:set_protocol_options, in addition to giving it in >> start_listener. Takes a fun that has a single arg as a Req, >> returns >> a Req, and if you replied from within it it doesn't dispatch >> the >> request and stops there (you can also force close the >> connection by >> setting the Connection header to "close"). >> >> And adding this: >> >> For graceful shutdowns, well it's highly application >> dependent. Some >> apps are just short lived connections, so not accepting >> requests >> plus a short delay ought to do it. Some are long lived, which >> probably requires to send a shutdown message. Some apps may >> have >> connections critical enough that you don't want to shutdown >> them. >> It's up to the application implementor to devise the >> strategy to use >> for stopping. >> >> >> On 05/14/2012 02:34 AM, Fred Hebert wrote: >> >> This is just a guess, but is it possible that this is >> due to the >> fact >> Cowboy is using simple_one_for_one supervision? In Pre >> R15, if I >> recall >> correctly, the shutdown of sofo supervisors was >> asynchronous. The >> supervisor would just die and let its children figure >> out it was >> gone. >> >> Starting with R15, things started being synchronous and the >> supervisor >> would wait. A brutal kill that made things work fine >> before R15 >> (excluding the issue of the application master killing >> everything) could >> start breaking in later versions. >> >> Again, this is just a guess, without looking at the >> source or >> anything. >> >> On Sun May 13 14:12:18 2012, Bob Ippolito wrote: >> >> Yes, it is documented that request processes >> continue after >> you stop >> the listener. This is incorrect. >> >> On Sunday, May 13, 2012, Anthony Ramine wrote: >> >> Pawe?, from what I gather your supervisor waits 1 ms >> before >> killing its children; I think it's pretty normal >> that some >> messages are left unprocessed. >> Bob, cowboy_requests_sup' shutdown strategy is >> brutal_kill, >> is it >> documented somewhere that Cowboy waits before >> killing requests' >> processes? >> >> Regards, >> >> -- >> Anthony Ramine >> >> >> >> >> Le 10 mai 2012 ? 22:37, Bob Ippolito a ?crit : >> https://gist.github.com/____**2655724 >> >> > >> >> >> >> >> >> On Thu, May 10, 2012 at 2:53 AM, Pawe? Peregud >> >> > 'paulperegud@REDACTED >> ');>> wrote: >> >> I was having fun with supervisors yesterday >> (Cowboy seems to >> fail to fulfill the promise of not killing >> request processes >> after listener removal) and I have an example. >> I've only >> investigated the case when supervisor is killed, >> so YMMV. >> Example code is attached. You may modify it to >> check the >> behavior in your case. >> >> Start supervisor tree with >> exp_sup_sup:start_link(). Execute >> test with exp_sup_sup:test() and >> exp_sup_sup:test_simple(). >> >> In case of dying supervisor the answer is "no, >> it does not". >> >> When supervisor dies, your process is killed as >> via link >> mechanism, so it may leave some unprocessed >> messages in >> inbox. To make sure that every delivered message >> is served, >> you need to add process_flag(trap_exit, true). >> Messages that >> are sent after the moment when supervisor dies >> are not >> processed. >> >> Best regards, >> >> Paul. >> >> >> On May 9, 2012 11:06 AM, "Andy Richards" >> > >> > 'andy.richards.iit@REDACTED**____com >> ');>> wrote: >> >> Hi, >> >> I can't seem to see any confirmation in the >> documentation >> so was wondering if anyone could confirm if >> messages are >> still sent to a supervised gen_svr following a >> shutdown >> message? >> >> If so how do I cleanly shutdown my gen_svr without >> loosing messages? I read in the supervisor child >> spec >> that a shutdown can be set to infinity which i >> hoped >> would allow me to process the msg's in my >> mailbox but if >> I do this will my module continue to receive >> messages >> from other processes? Is my approach flawed and >> if so >> what other ways are there to cleanly shutting >> down my >> gen_svr without loosing messages? >> >> Many thanks, >> >> Andy. >> ______________________________** >> _____________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > 'cvml', >> 'erlang-questions@REDACTED >> ')__;> >> http://erlang.org/mailman/____**listinfo/erlang-questions >> >> > >> >> >> >> >> >> >> ______________________________** >> _____________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > 'cvml', >> 'erlang-questions@REDACTED >> ')__;> >> http://erlang.org/mailman/____**listinfo/erlang-questions >> >> > >> >> >> >> >> >> >> ______________________________** >> _____________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > 'cvml', >> 'erlang-questions@REDACTED >> ')__;> >> http://erlang.org/mailman/____**listinfo/erlang-questions >> >> > >> >> >> >> >> >> >> >> >> ______________________________**_____________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/____**listinfo/erlang-questions >> >> > >> >> >> >> >> >> ______________________________**_____________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/____**listinfo/erlang-questions >> >> > >> >> >> >> >> >> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> >> >> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> >> > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue May 15 07:05:29 2012 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 15 May 2012 07:05:29 +0200 Subject: [erlang-questions] msg mailbox and gen_svr shutdown In-Reply-To: References: <4FB052FF.9080604@ferd.ca> <4FB091D6.8000409@ninenines.eu> <4FB1B204.9030006@ninenines.eu> <4FB1CA6A.8080002@ninenines.eu> Message-ID: <4FB1E419.9000000@ninenines.eu> As with all proposed features I first need to understand what it's used for and why, and whether it'll be useful to other people. What you are doing just sounds very weird to me and it's the first time I hear of a need for something like this so at this point I'm neither trying to avoid anything nor considering it useful functionality, just wanting to understand why. On 05/15/2012 06:42 AM, Bob Ippolito wrote: > Yes, I want the other process to immediately take over. Requiring a load > balancer change isn't an elegant solution, I don't understand why you're > trying to avoid adding this useful functionality. > > On Monday, May 14, 2012, Lo?c Hoguin wrote: > > You stop listening after you're done gracefully stopping your > currently running processes. > > For HTTP that could be something like: > > Set acceptors to 0. > Optionally set 'onrequest' hook to reply with a 503 (for keepalives). > Gracefully stop your processes. > Stop the listener. > > Or do you need another process to take over immediately? Because in > that case you usually don't need to use the same listening port, you > can just change your firewall/lb rules for the port redirection from > 80->P1 to 80->P2. > > On 05/15/2012 03:34 AM, Bob Ippolito wrote: > > I don't think that is sufficient, you will need to stop listening as > well so the next OS process can take over. > > On Monday, May 14, 2012, Lo?c Hoguin wrote: > > This will be possible later on by reducing the number of > acceptors > to 0. This should be added sometimes this summer after the > acceptor > split happens in Cowboy. > > On 05/14/2012 07:26 PM, Bob Ippolito wrote: > > I agree that graceful shutdown is very application-specific. > However, > cowboy doesn't currently facilitate any sort of graceful > shutdown unless > you read the source code and poke directly at the > appropriate > supervisors like I did. The application specific stuff > is easily > done on > your own with a process registry or a timeout, we used a > combination of > gproc and a timeout if things didn't shut down in an > acceptable > time frame. > > I would suggest something like cowboy:stop_listener/1, maybe > something > like cowboy:stop_listening/1 or cowboy:stop_accepting/1. > > On Sun, May 13, 2012 at 11:02 PM, Lo?c Hoguin > > wrote: > > Right, this is pretty much what I said to Paul in > PM. It was in > R14B03 that the behavior changed. I apparently have > a @todo > wrong > past that release, and will take a look if I find other > things to > fix in the docs. > > Copy pasting my private reply on this: > > Ultimately if we remove the listener I think we want > to stop > everything. > > For "server is overloaded" situations, you can very > well use the > 'onrequest' hook which can be set or changed dynamically through > cowboy:set_protocol_options, in addition to giving it in > start_listener. Takes a fun that has a single arg as > a Req, > returns > a Req, and if you replied from within it it doesn't > dispatch the > request and stops there (you can also force close the > connection by > setting the Connection header to "close"). > > And adding this: > > For graceful shutdowns, well it's highly application > dependent. Some > apps are just short lived connections, so not > accepting requests > plus a short delay ought to do it. Some are long > lived, which > probably requires to send a shutdown message. Some > apps may have > connections critical enough that you don't want to > shutdown > them. > It's up to the application implementor to devise the > strategy to use > for stopping. > > > On 05/14/2012 02:34 AM, Fred Hebert wrote: > > This is just a guess, but is it possible that > this is > due to the > fact > Cowboy is using simple_one_for_one supervision? > In Pre > R15, if I > recall > correctly, the shutdown of sofo supervisors was > asynchronous. The > supervisor would just die and let its children > figure > out it was > gone. > > Starting with R15, things started being > synchronous and the > supervisor > would wait. A brutal kill that made things work fine > before R15 > (excluding the issue of the application master > killing > everything) could > start breaking in later versions. > > Again, this is just a guess, without looking at the > source or > anything. > > On Sun May 13 14:12:18 2012, Bob Ippolito wrote: > > Yes, it is documented that request processes > continue after > you stop > the listener. This is incorrect. > > On Sunday, May 13, 2012, Anthony Ramine wrote: > > Pawe?, from what I gather your supervisor > waits 1 ms > before > killing its children; I think it's pretty normal > that some > messages are left unprocessed. > Bob, cowboy_requests_sup' shutdown strategy is > brutal_kill, > is it > documented somewhere that Cowboy waits before > killing requests' > processes? > > Regards, > > -- > Anthony Ramine > > > > > Le 10 mai 2012 ? 22:37, Bob Ippolito a ?crit : > https://gist.github.com/______2655724 > > > > > >> > > On Thu, May 10, 2012 at 2:53 AM, Pawe? > Peregud > > 'paulperegud@REDACTED > ');>> wrote: > > I was having fun with supervisors yesterday > (Cowboy seems to > fail to fulfill the promise of not killing > request processes > after listener removal) and I have an > example. > I've only > investigated the case when supervisor is > killed, > so YMMV. > Example code is attached. You may modify > it to > check the > behavior in your case. > > Start supervisor tree with > exp_sup_sup:start_link(). Execute > test with exp_sup_sup:test() and > exp_sup_sup:test_simple(). > > In case of dying supervisor the answer > is "no, > it does not". > > When supervisor dies, your process is > killed as > via link > mechanism, so it may leave some unprocessed > messages in > inbox. To make sure that every delivered > message > is served, > you need to add process_flag(trap_exit, > true). > Messages that > are sent after the moment when > supervisor dies > are not > processed. > > Best regards, > > Paul. > > > On May 9, 2012 11:06 AM, "Andy Richards" > > 'andy.richards.iit@REDACTED > ');>> wrote: > > Hi, > > I can't seem to see any confirmation in the > documentation > so was wondering if anyone could confirm if > messages are > still sent to a supervised gen_svr > following a > shutdown > message? > > If so how do I cleanly shutdown my > gen_svr without > loosing messages? I read in the > supervisor child > spec > that a shutdown can be set to infinity > which i hoped > would allow me to process the msg's in my > mailbox but if > I do this will my module continue to receive > messages > from other processes? Is my approach > flawed and > if so > what other ways are there to cleanly > shutting > down my > gen_svr without loosing messages? > > Many thanks, > > Andy. > > _____________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > 'cvml', > 'erlang-questions@REDACTED > ')__;> > http://erlang.org/mailman/______listinfo/erlang-questions > > > > > >> > > > > _____________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > 'cvml', > 'erlang-questions@REDACTED > ')__;> > http://erlang.org/mailman/______listinfo/erlang-questions > > > > > >> > > > > _____________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > 'cvml', > 'erlang-questions@REDACTED > ')__;> > http://erlang.org/mailman/______listinfo/erlang-questions > > > > > >> > > > > > > _____________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/______listinfo/erlang-questions > > > > > >> > > > _____________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/______listinfo/erlang-questions > > > > > >> > > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > > > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > -- Lo?c Hoguin Erlang Cowboy Nine Nines From bob@REDACTED Tue May 15 07:35:49 2012 From: bob@REDACTED (Bob Ippolito) Date: Mon, 14 May 2012 23:35:49 -0600 Subject: [erlang-questions] msg mailbox and gen_svr shutdown In-Reply-To: <4FB1E419.9000000@ninenines.eu> References: <4FB052FF.9080604@ferd.ca> <4FB091D6.8000409@ninenines.eu> <4FB1B204.9030006@ninenines.eu> <4FB1CA6A.8080002@ninenines.eu> <4FB1E419.9000000@ninenines.eu> Message-ID: The idea is to start a new version of the service and do a quick hand-off of listening from old to new (possibly even switching back). This is particularly useful for code upgrades that aren't suitable for hot code loading. We're doing continuous integration this way, as hot code loading requires too much work and/or planning for us at this stage. It's currently an uncommon need but it would be a lot more common if it was well documented and easier to do. As far as load balancers go, we're not using them. For our use case we don't need them, so why add the complexity and/or cost? There is port forwarding done by the OS kernel but I want to avoid requiring some setuid binary to do the privilege escalation that would be necessary to change ports on the fly. Nginx does upgrades in a similar way. http://wiki.nginx.org/CommandLine#Upgrading_To_a_New_Binary_On_The_Fly On Monday, May 14, 2012, Lo?c Hoguin wrote: > As with all proposed features I first need to understand what it's used > for and why, and whether it'll be useful to other people. What you are > doing just sounds very weird to me and it's the first time I hear of a need > for something like this so at this point I'm neither trying to avoid > anything nor considering it useful functionality, just wanting to > understand why. > > On 05/15/2012 06:42 AM, Bob Ippolito wrote: > >> Yes, I want the other process to immediately take over. Requiring a load >> balancer change isn't an elegant solution, I don't understand why you're >> trying to avoid adding this useful functionality. >> >> On Monday, May 14, 2012, Lo?c Hoguin wrote: >> >> You stop listening after you're done gracefully stopping your >> currently running processes. >> >> For HTTP that could be something like: >> >> Set acceptors to 0. >> Optionally set 'onrequest' hook to reply with a 503 (for keepalives). >> Gracefully stop your processes. >> Stop the listener. >> >> Or do you need another process to take over immediately? Because in >> that case you usually don't need to use the same listening port, you >> can just change your firewall/lb rules for the port redirection from >> 80->P1 to 80->P2. >> >> On 05/15/2012 03:34 AM, Bob Ippolito wrote: >> >> I don't think that is sufficient, you will need to stop listening >> as >> well so the next OS process can take over. >> >> On Monday, May 14, 2012, Lo?c Hoguin wrote: >> >> This will be possible later on by reducing the number of >> acceptors >> to 0. This should be added sometimes this summer after the >> acceptor >> split happens in Cowboy. >> >> On 05/14/2012 07:26 PM, Bob Ippolito wrote: >> >> I agree that graceful shutdown is very >> application-specific. >> However, >> cowboy doesn't currently facilitate any sort of graceful >> shutdown unless >> you read the source code and poke directly at the >> appropriate >> supervisors like I did. The application specific stuff >> is easily >> done on >> your own with a process registry or a timeout, we used a >> combination of >> gproc and a timeout if things didn't shut down in an >> acceptable >> time frame. >> >> I would suggest something like cowboy:stop_listener/1, >> maybe >> something >> like cowboy:stop_listening/1 or cowboy:stop_accepting/1. >> >> On Sun, May 13, 2012 at 11:02 PM, Lo?c Hoguin >> > > wrote: >> >> Right, this is pretty much what I said to Paul in >> PM. It was in >> R14B03 that the behavior changed. I apparently have >> a @todo >> wrong >> past that release, and will take a look if I find other >> things to >> fix in the docs. >> >> Copy pasting my private reply on this: >> >> Ultimately if we remove the listener I think we want >> to stop >> everything. >> >> For "server is overloaded" situations, you can very >> well use the >> 'onrequest' hook which can be set or changed dynamically through >> cowboy:set_protocol_options, in addition to giving it >> in >> start_listener. Takes a fun that has a single arg as >> a Req, >> returns >> a Req, and if you replied from within it it doesn't >> dispatch the >> request and stops there (you can also force close the >> connection by >> setting the Connection header to "close"). >> >> And adding this: >> >> For graceful shutdowns, well it's highly application >> dependent. Some >> apps are just short lived connections, so not >> accepting requests >> plus a short delay ought to do it. Some are long >> lived, which >> probably requires to send a shutdown message. Some >> apps may have >> connections critical enough that you don't want to >> shutdown >> them. >> It's up to the application implementor to devise the >> strategy to use >> for stopping. >> >> >> On 05/14/2012 02:34 AM, Fred Hebert wrote: >> >> This is just a guess, but is it possible that >> this is >> due to the >> fact >> Cowboy is using simple_one_for_one supervision? >> In Pre >> R15, if I >> recall >> correctly, the shutdown of sofo supervisors was >> asynchronous. The >> supervisor would just di >> https://gist.github.com/______**2655724 >> >> > >> >> >> >> >> >> >> > >> >> >> >>> >> >> On Thu, May 10, 2012 at 2:53 AM, Pawe? >> Peregud >> >> > 'paulperegud@REDACTED >> ');>> wrote: >> >> I was having fun with supervisors yesterday >> (Cowboy seems to >> fail to fulfill the promise of not killing >> request processes >> after listener removal) and I have an >> example. >> I've only >> investigated the case when supervisor is >> killed, >> so YMMV. >> Example code is attached. You may modify >> it to >> check the >> behavior in your case. >> >> Start supervisor tree with >> exp_sup_sup:start_link(). Execute >> test with exp_sup_sup:test() and >> exp_sup_sup:test_simple(). >> >> In case of dying supervisor the answer >> is "no, >> it does not". >> >> When supervisor dies, your process is >> killed as >> via link >> mechanism, so it may leave some unprocessed >> messages in >> inbox. To make sure that every delivered >> message >> is served, >> you need to add process_flag(trap_exit, >> true). >> Messages that >> are sent after the moment when >> supervisor dies >> are not >> processed. >> >> Best regards, >> >> Paul. >> >> >> On May 9, 2012 11:06 AM, "Andy Richards" >> > >> > 'andy.richards.iit@REDACTED**______com >> ');>> wrote: >> >> Hi, >> >> I can't seem to see any confirmation in the >> documentation >> so was wondering if anyone could confirm if >> messages are >> still sent to a supervised gen_svr >> following a >> shutdown >> message? >> >> If so how do I cleanly shutdown my >> gen_svr without >> loosing messages? I read in the >> supervisor child >> spec >> that a shutdown can be set to infinity >> which i hoped >> would allow me to process the msg's in my >> mailbox but if >> I do this will my module continue to >> receive >> messages >> from other processes? Is my approach >> flawed and >> if so >> what other ways are there to cleanly >> shutting >> down my >> gen_svr without loosing messages? >> >> Many thanks, >> >> Andy. >> >> ______________________________**_______________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > 'cvml', >> 'erlang-questions@REDACTED >> ')__;> >> http://erlang.org/mailman/____**__listinfo/erlang-questions >> >> > >> >> >> >> >> >> >> > >> >> >> >>> >> >> >> >> ______________________________**_______________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > 'cvml', >> 'erlang-questions@REDACTED >> ')__;> >> http://erlang.org/mailman/____**__listinfo/erlang-questions >> >> > >> >> >> >> >> >> >> > >> >> >> >>> >> >> >> >> ______________________________**_______________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > 'cvml', >> 'erlang-questions@REDACTED >> ')__;> >> http://erlang.org/mailman/____**__listinfo/erlang-questions >> >> > >> >> >> >> >> >> >> > >> >> >> >>> >> >> >> >> >> >> ______________________________**_______________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/____**__listinfo/erlang-questions >> >> > >> >> >> >> >> >> >> > >> >> >> >>> >> >> >> ______________________________**_______________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/____**__listinfo/erlang-questions >> >> > >> >> >> >> >> >> >> > >> >> >> >>> >> >> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> >> >> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> >> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> >> > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Tue May 15 08:36:09 2012 From: garazdawi@REDACTED (Lukas Larsson) Date: Tue, 15 May 2012 08:36:09 +0200 Subject: [erlang-questions] SCTP multihoming In-Reply-To: References: Message-ID: I assume you have looked at this[1] thread on the erlang patches mailinglist? [1] http://erlang.org/pipermail/erlang-patches/2011-December/002559.html Sent from me On 15 maj 2012, at 06:06, Chandru wrote: > Hi all, > > I'm having trouble starting a server with multihoming support. > > The call: > 16> (<0.804.0>) call gen_sctp:open([binary, > {active,once}, > {reuseaddr,true}, > {port,3868}, > {ip,{10,249,8,4}}, > {ip,{10,249,8,12}}]) > > crashes with the report below. Anyone has a quick fix? I've traced it down to this: > > (<0.2364.0>) call erlang:port_control(#Port<0.5560>,62,[[1],[[15,28],10,249,8,12],[[15,28],10,249,8,4]]) > (<0.2364.0>) returned from erlang:port_control/3 -> [0,101,105,110,118,97,108] (which is einval basically) > > Any idea why this would be? I'm running this on Solaris 10, with R15B. > > cheers > Chandru > > 22> rb:show(4). > > CRASH REPORT <0.866.0> 2012-05-15 04:44:30 > =============================================================================== > Crashing process > initial_call {diameter_sctp,init,['Argument__1']} > pid <0.866.0> > registered_name [] > error_info > {error,badarg, > [{gen_sctp,open, > [[binary, > {active,once}, > {reuseaddr,true}, > {port,3868}, > {ip,{10,249,8,4}}, > {ip,{10,249,8,12}}]], > [{file,"gen_sctp.erl"},{line,126}]}, > {diameter_sctp,open,3, > [{file,"transport/diameter_sctp.erl"}, > {line,227}]}, > {diameter_sctp,i,1, > [{file,"transport/diameter_sctp.erl"}, > {line,147}]}, > {diameter_sctp,init,1, > [{file,"transport/diameter_sctp.erl"}, > {line,141}]}, > {proc_lib,init_p_do_apply,3, > [{file,"proc_lib.erl"},{line,227}]}]} > ancestors > [diameter_sctp_listener_sup,diameter_transport_sup,diameter_sup, > <0.779.0>] > messages [] > links [<0.803.0>] > dictionary [] > trap_exit false > status running > heap_size 75025 > stack_size 24 > reductions 736 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From hm@REDACTED Tue May 15 10:26:31 2012 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Tue, 15 May 2012 10:26:31 +0200 Subject: [erlang-questions] `mnesia:dump_tables/1' and the schema table In-Reply-To: <4FAB9F30.80102@rabbitmq.com> References: <4FAB9F30.80102@rabbitmq.com> Message-ID: On Thu, May 10, 2012 at 12:57 PM, Francesco Mazzoli wrote: > Hi, > > For some reason, `mnesia:dump_tables/1' refuses to dump the schema table: > https://github.com/erlang/otp/blob/maint/lib/mnesia/src/mnesia_schema.erl#L2797 > . > > 1) Why is this the case? > 2) This is not documented, see > http://www.erlang.org/doc/man/mnesia.html#dump_tables-1 and > http://www.erlang.org/doc/apps/mnesia/Mnesia_chap7.html#id77600 This is a rather obscure function that should be avoided. The function is not intended to be used for the schema table. It ought to be documented. /H?kan From clist@REDACTED Tue May 15 11:04:40 2012 From: clist@REDACTED (Angel J. Alvarez Miguel) Date: Tue, 15 May 2012 11:04:40 +0200 Subject: [erlang-questions] erlang float comparison In-Reply-To: <201205140941.01511.clist@uah.es> References: <201205140941.01511.clist@uah.es> Message-ID: <201205151104.40442.clist@uah.es> Hi Guys Thanks for all the comments and all those details. after reading the articles, that Richard pointed out, and think a bit about what we need i endended with two solutions that for the most part allow us to get it fixed for the moment: The problem can be stated as: Result = case Myfloat > 1.0 of true -> 1.0; false MyFloat end ... ... math:acos(Result). after that we came across some values that fail to pass the test > 1.0 but are seen as greater to the math:acos function rendering a bad arithmetic expression error so finally we manage to do Result = case trunc(MyFloat) < 1 of true -> MyFloat; false -> 1.0 end ... ... math:acos(Result). that unless anyone see some hidden mosnter that will bite us the... i think it solves our problem. "...My 1st-year chemistry days are long past, but web pages I've checked today seem not to quote any bond angles to better than 1 part in 1000 precision. And since angles are constrained to [0,one circle), it looks to me very much as if we have a case for representing bond angles and dihedral angles as fixed point numbers, or at any rate for rounding them to multiples of say (one circle)/65536 and comparing the angles as integers." Well take a look at this approach, math code is almost finished as my cooworker ported it from an earlier python tool he had developed wher he managed to fix this in a diferent fashion. So i was working on the OTP part while we found this "feature" but provided that we have enough time to check and surely we will refactor some code to be carried ogen_fsm,'s maybe... /Angel On Lunes, 14 de Mayo de 2012 09:41:01 Angel J. Alvarez Miguel escribi?: > Hi guys > > > > I need to compare two floats something like 0.99999998... vs 1.000000 > and we came across accuracy problems when testing dihedral angles on a > molecule... > > > > so we read > http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm > and wanted to implement something like: > > > > if (*(int*)&f1 < *(int*)&f2)... > > > > when ef1 and f2 are floats... (f1 about 0.9999899.. and f2=1.0 > > > > is think we should start with... > > <> = term_to_binary(1.9999976,[{minor_version,1}]). > > > > and let Padding swallow the external format and tag while Myint gets the > ieee754 float > > > > But MyInt = 4611686017346523993 instead of 1073741822 > > these conversion should follow ieee754 and being lexicografic ordered > but ... > > term_to_binary(1.99999976,[{minor_version,1}]). > <<131,70,63,255,255,255,191,147,83,89>> > > term_to_binary(1.99999988,[{minor_version,1}]). > <<131,70,63,255,255,255,223,201,169,173>> > > doesnt seem to be the same thatn you spect to see after reading that > page... > > > ..What im doing wrong....? > > > ieee754 layout.... > > > +1.99999976 > 0x3FFFFFFE > 1073741822 > +1.99999988 > 0x3FFFFFFF > 1073741823 > +2.00000000 > 0x40000000 > 1073741824 > +2.00000024 > 0x40000001 > 1073741825 > +2.00000048 > 0x40000002 > 1073741826 > > Thanks!.. > > /Angel -- - From hynek@REDACTED Tue May 15 11:38:01 2012 From: hynek@REDACTED (Hynek Vychodil) Date: Tue, 15 May 2012 11:38:01 +0200 Subject: [erlang-questions] erlang float comparison In-Reply-To: <201205151104.40442.clist@uah.es> References: <201205140941.01511.clist@uah.es> <201205151104.40442.clist@uah.es> Message-ID: I think this code should work as well Result = if Myfloat < 1.0 -> Myfloat; true -> 1.0 end, but little bit more efficient. On Tue, May 15, 2012 at 11:04 AM, Angel J. Alvarez Miguel wrote: > Hi Guys > > Thanks for all the comments and all those details. after reading the articles, > that Richard pointed out, and think a bit about what we need i endended with two > solutions that for the most part allow us to get it fixed for the moment: > > The problem can be stated as: > > Result = case Myfloat > 1.0 of > ? ? ? ?true -> 1.0; > ? ? ? ?false MyFloat > ? ? ? ?end > ... > ... > math:acos(Result). > > after that ?we came across some values that fail to pass the test > 1.0 but are seen > as greater to the math:acos function rendering a bad arithmetic expression error so finally we manage to do > > Result ?= case trunc(MyFloat) < 1 of > ? ? ? ?true -> MyFloat; > ? ? ? ?false -> 1.0 > ? ? ? ?end > > ... > ... > math:acos(Result). > > that unless anyone see some hidden mosnter that will bite us the... i think it solves our problem. > > > "...My 1st-year chemistry days are long past, but web pages I've checked > today seem not to quote any bond angles to better than 1 part in 1000 > precision. ?And since angles are constrained to [0,one circle), it > looks to me very much as if we have a case for representing bond angles > and dihedral angles as fixed point numbers, or at any rate for > rounding them to multiples of say (one circle)/65536 and comparing > the angles as integers." > > Well take a look at this approach, math code is almost finished as my cooworker ported it from > an earlier python tool he had developed wher he managed to fix this in a diferent fashion. > So i was working on the OTP part while we found this "feature" but provided that we have enough > time to check and surely we will refactor some code to be carried ogen_fsm,'s maybe... > > > > /Angel > > On Lunes, 14 de Mayo de 2012 09:41:01 Angel J. Alvarez Miguel escribi?: >> Hi guys >> >> >> >> I need to compare two floats something like 0.99999998... vs 1.000000 >> and we came across accuracy problems when testing dihedral angles on a >> molecule... >> >> >> >> so we read >> http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm >> and wanted to implement something like: >> >> >> >> if (*(int*)&f1 < *(int*)&f2)... >> >> >> >> when ef1 and f2 are floats... (f1 about 0.9999899.. and f2=1.0 >> >> >> >> is think we should start with... >> >> <> = term_to_binary(1.9999976,[{minor_version,1}]). >> >> >> >> and let Padding swallow the external format and tag while Myint gets the >> ieee754 float >> >> >> >> But MyInt = 4611686017346523993 instead of 1073741822 >> >> these conversion should follow ieee754 and being lexicografic ordered >> but ... >> >> term_to_binary(1.99999976,[{minor_version,1}]). >> <<131,70,63,255,255,255,191,147,83,89>> >> >> term_to_binary(1.99999988,[{minor_version,1}]). >> <<131,70,63,255,255,255,223,201,169,173>> >> >> doesnt seem to be the same thatn you spect to see after reading that >> page... >> >> >> ..What im doing wrong....? >> >> >> ieee754 layout.... >> >> >> +1.99999976 >> 0x3FFFFFFE >> 1073741822 >> +1.99999988 >> 0x3FFFFFFF >> 1073741823 >> +2.00000000 >> 0x40000000 >> 1073741824 >> +2.00000024 >> 0x40000001 >> 1073741825 >> +2.00000048 >> 0x40000002 >> 1073741826 >> >> Thanks!.. >> >> /Angel > > -- > - > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Hynek Vychodil Chief Scientists GoodData n?m?st? 28. ??jna 1104/17, 602 00, Brno - ?ern? Pole Office: ? +420 530 50 7704 E-mail: ?hynek@REDACTED Web: ? ? www.gooddata.com From arg.0.0@REDACTED Tue May 15 11:56:52 2012 From: arg.0.0@REDACTED (=?koi8-r?B?8sXTz9cg6czY0Q==?=) Date: Tue, 15 May 2012 13:56:52 +0400 Subject: [erlang-questions] Erlang on ppc405 Message-ID: <721421337075812@web8h.yandex.ru> Hello all, just compiled and unsuccessfully trying to run erlang on pcc405 configure options: /configure --prefix=/home/devel/otp/ppc/R15B01 \ --host=powerpc-linux --build=i686-linux \ --enable-threads --enable-kernel-poll \ --disable-smp-support --disable-hipe \ --without-ssl \ erl_xcomp_sysroot=/home/devel/oe/dm/dm7020/build/tmp/cross/ppc405/bin erlexec on target machine loads CPU to 98% and nothing happens.... after 30 minutes CPU load still ~98 % by erlexec Any suggestions? From olopierpa@REDACTED Tue May 15 12:02:51 2012 From: olopierpa@REDACTED (Pierpaolo Bernardi) Date: Tue, 15 May 2012 12:02:51 +0200 Subject: [erlang-questions] erlang float comparison In-Reply-To: <201205151104.40442.clist@uah.es> References: <201205140941.01511.clist@uah.es> <201205151104.40442.clist@uah.es> Message-ID: On Tue, May 15, 2012 at 11:04 AM, Angel J. Alvarez Miguel wrote: > The problem can be stated as: > > Result = case Myfloat > 1.0 of > ? ? ? ?true -> 1.0; > ? ? ? ?false MyFloat > ? ? ? ?end > ... > ... > math:acos(Result). > > after that ?we came across some values that fail to pass the test > 1.0 but are seen > as greater to the math:acos function rendering a bad arithmetic expression error If true, this indicates a serious bug. You should report such cases. I am somewhat skeptical that such a gross bug would lie undetected so far in erlang, so please supply an actual reproducible example. Cheers, P. From YurinVV@REDACTED Tue May 15 12:03:55 2012 From: YurinVV@REDACTED (Slava Yurin) Date: Tue, 15 May 2012 17:03:55 +0700 Subject: [erlang-questions] Erlang on ppc405 In-Reply-To: <721421337075812@web8h.yandex.ru> References: <721421337075812@web8h.yandex.ru> Message-ID: <688361337076235@web27g.yandex.ru> An HTML attachment was scrubbed... URL: From olopierpa@REDACTED Tue May 15 12:06:04 2012 From: olopierpa@REDACTED (Pierpaolo Bernardi) Date: Tue, 15 May 2012 12:06:04 +0200 Subject: [erlang-questions] erlang float comparison In-Reply-To: References: <201205140941.01511.clist@uah.es> <201205151104.40442.clist@uah.es> Message-ID: On Tue, May 15, 2012 at 12:02 PM, Pierpaolo Bernardi wrote: > On Tue, May 15, 2012 at 11:04 AM, Angel J. Alvarez Miguel wrote: > >> The problem can be stated as: >> >> Result = case Myfloat > 1.0 of >> ? ? ? ?true -> 1.0; >> ? ? ? ?false MyFloat >> ? ? ? ?end >> ... >> ... >> math:acos(Result). >> >> after that ?we came across some values that fail to pass the test > 1.0 but are seen >> as greater to the math:acos function rendering a bad arithmetic expression error > > If true, this indicates a serious bug. ?You should report such cases. Assuming is not MyFloat < -1, that is. Cheers P. From arg.0.0@REDACTED Tue May 15 12:36:30 2012 From: arg.0.0@REDACTED (=?koi8-r?B?8sXTz9cg6czY0Q==?=) Date: Tue, 15 May 2012 14:36:30 +0400 Subject: [erlang-questions] Erlang on ppc405 Message-ID: <13321337078190@web2g.yandex.ru> In my case threads support enabled and I need to use +A erl option. rebar generated release with etc/vm.args ## Enable kernel poll and a few async threads +K true +A 3 ## Tweak GC to run more often -env ERL_FULLSWEEP_AFTER 10 -connect_all false -hidden -shutdown_time 5000 comment +K and +A does not help. > > Try check http://erlang.org/pipermail/erlang-questions/2012-March/065480.html > > > > 15.05.2012, 16:56, "????? ????" : > > > >> > Hello all, > >> > > >> > just compiled and unsuccessfully trying to run erlang on pcc405 > >> > > >> > configure options: > >> > > >> > /configure --prefix=/home/devel/otp/ppc/R15B01 \ > >> > --host=powerpc-linux --build=i686-linux \ > >> > --enable-threads --enable-kernel-poll \ > >> > --disable-smp-support --disable-hipe \ > >> > --without-ssl \ > >> > erl_xcomp_sysroot=/home/devel/oe/dm/dm7020/build/tmp/cross/ppc405/bin > >> > > >> > erlexec on target machine loads CPU to 98% and nothing happens.... after 30 minutes CPU load still ~98 % by erlexec > >> > > >> > Any suggestions? > >> > _______________________________________________ > >> > erlang-questions mailing list > >> > erlang-questions@REDACTED > >> > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > > ? ???????? ???? -- From essen@REDACTED Tue May 15 13:19:11 2012 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 15 May 2012 13:19:11 +0200 Subject: [erlang-questions] msg mailbox and gen_svr shutdown In-Reply-To: References: <4FB052FF.9080604@ferd.ca> <4FB091D6.8000409@ninenines.eu> <4FB1B204.9030006@ninenines.eu> <4FB1CA6A.8080002@ninenines.eu> <4FB1E419.9000000@ninenines.eu> Message-ID: <4FB23BAF.1020201@ninenines.eu> I personally wouldn't want this to become common, because even if you restart immediately, you would lose a few incoming connections (or a lot depending on the popularity of your service). If I understand correctly you go as far as restarting the whole node? This isn't like nginx, since nginx doesn't stop listening at any point. You could actually get something closer to nginx by doing this: * set acceptors to 0 (incoming connections get queued) * gracefully shutdown the running connections (application dependent) * load the new code * optionally restart parts of your supervision tree * set acceptors back to N You can add an optional set_protocol_options somewhere in there if you need to upgrade the dispatch list (maybe after loading the new code you could always run set_protocol_options(my_project:get_dispatch_stuff) or something, which would always get the last options for your project). The number of incoming connections queued is also application dependent (and will be upgradable later on so you can increase temporarily). This would allow you to do what you want without any extra feature from what is currently planned, and without losing any incoming connections. All this is easily scriptable of course. If that still doesn't fit your needs I'd suggest opening a ticket to see what others think about it. On 05/15/2012 07:35 AM, Bob Ippolito wrote: > The idea is to start a new version of the service and do a quick > hand-off of listening from old to new (possibly even switching back). > This is particularly useful for code upgrades that aren't suitable for > hot code loading. We're doing continuous integration this way, as hot > code loading requires too much work and/or planning for us at this > stage. It's currently an uncommon need but it would be a lot more common > if it was well documented and easier to do. > > As far as load balancers go, we're not using them. For our use case we > don't need them, so why add the complexity and/or cost? There is port > forwarding done by the OS kernel but I want to avoid requiring some > setuid binary to do the privilege escalation that would be necessary to > change ports on the fly. > > Nginx does upgrades in a similar way. > http://wiki.nginx.org/CommandLine#Upgrading_To_a_New_Binary_On_The_Fly > > On Monday, May 14, 2012, Lo?c Hoguin wrote: > > As with all proposed features I first need to understand what it's > used for and why, and whether it'll be useful to other people. What > you are doing just sounds very weird to me and it's the first time I > hear of a need for something like this so at this point I'm neither > trying to avoid anything nor considering it useful functionality, > just wanting to understand why. > > On 05/15/2012 06:42 AM, Bob Ippolito wrote: > > Yes, I want the other process to immediately take over. > Requiring a load > balancer change isn't an elegant solution, I don't understand > why you're > trying to avoid adding this useful functionality. > > On Monday, May 14, 2012, Lo?c Hoguin wrote: > > You stop listening after you're done gracefully stopping your > currently running processes. > > For HTTP that could be something like: > > Set acceptors to 0. > Optionally set 'onrequest' hook to reply with a 503 (for > keepalives). > Gracefully stop your processes. > Stop the listener. > > Or do you need another process to take over immediately? > Because in > that case you usually don't need to use the same listening > port, you > can just change your firewall/lb rules for the port > redirection from > 80->P1 to 80->P2. > > On 05/15/2012 03:34 AM, Bob Ippolito wrote: > > I don't think that is sufficient, you will need to stop > listening as > well so the next OS process can take over. > > On Monday, May 14, 2012, Lo?c Hoguin wrote: > > This will be possible later on by reducing the number of > acceptors > to 0. This should be added sometimes this summer > after the > acceptor > split happens in Cowboy. > > On 05/14/2012 07:26 PM, Bob Ippolito wrote: > > I agree that graceful shutdown is very > application-specific. > However, > cowboy doesn't currently facilitate any sort of > graceful > shutdown unless > you read the source code and poke directly at the > appropriate > supervisors like I did. The application specific > stuff > is easily > done on > your own with a process registry or a timeout, > we used a > combination of > gproc and a timeout if things didn't shut down in an > acceptable > time frame. > > I would suggest something like > cowboy:stop_listener/1, maybe > something > like cowboy:stop_listening/1 or > cowboy:stop_accepting/1. > > On Sun, May 13, 2012 at 11:02 PM, Lo?c Hoguin > > wrote: > > Right, this is pretty much what I said to > Paul in > PM. It was in > R14B03 that the behavior changed. I > apparently have > a @todo > wrong > past that release, and will take a look if I > find other > things to > fix in the docs. > > Copy pasting my private reply on this: > > Ultimately if we remove the listener I think > we want > to stop > everything. > > For "server is overloaded" situations, you > can very > well use the > 'onrequest' hook which can be set or changed dynamically through > cowboy:set_protocol_options, in addition to > giving it in > start_listener. Takes a fun that has a > single arg as > a Req, > returns > a Req, and if you replied from within it it > doesn't > dispatch the > request and stops there (you can also force > close the > connection by > setting the Connection header to "close"). > > And adding this: > > For graceful shutdowns, well it's highly > application > dependent. Some > apps are just short lived connections, so not > accepting requests > plus a short delay ought to do it. Some are long > lived, which > probably requires to send a shutdown > message. Some > apps may have > connections critical enough that you don't > want to > shutdown > them. > It's up to the application implementor to > devise the > strategy to use > for stopping. > > > On 05/14/2012 02:34 AM, Fred Hebert wrote: > > This is just a guess, but is it possible > that > this is > due to the > fact > Cowboy is using simple_one_for_one > supervision? > In Pre > R15, if I > recall > correctly, the shutdown of sofo > supervisors was > asynchronous. The > supervisor would just di > https://gist.github.com/________2655724 > > > > > >> > > > > > >>> > > On Thu, May 10, 2012 at 2:53 AM, > Pawe? > Peregud > > 'paulperegud@REDACTED > ');>> wrote: > > I was having fun with > supervisors yesterday > (Cowboy seems to > fail to fulfill the promise of > not killing > request processes > after listener removal) and I > have an > example. > I've only > investigated the case when > supervisor is > killed, > so YMMV. > Example code is attached. You > may modify > it to > check the > behavior in your case. > > Start supervisor tree with > exp_sup_sup:start_link(). Execute > test with exp_sup_sup:test() and > exp_sup_sup:test_simple(). > > In case of dying supervisor the > answer > is "no, > it does not". > > When supervisor dies, your > process is > killed as > via link > mechanism, so it may leave some > unprocessed > messages in > inbox. To make sure that every > delivered > message > is served, > you need to add > process_flag(trap_exit, > true). > Messages that > are sent after the moment when > supervisor dies > are not > processed. > > Best regards, > > Paul. > > > On May 9, 2012 11:06 AM, "Andy > Richards" > > 'andy.richards.iit@REDACTED > ');>> wrote: > > Hi, > > I can't seem to see any > confirmation in the > documentation > so was wondering if anyone could > confirm if > messages are > still sent to a supervised gen_svr > following a > shutdown > message? > > If so how do I cleanly shutdown my > gen_svr without > loosing messages? I read in the > supervisor child > spec > that a shutdown can be set to > infinity > which i hoped > would allow me to process the > msg's in my > mailbox but if > I do this will my module > continue to receive > messages > from other processes? Is my approach > flawed and > if so > what other ways are there to cleanly > shutting > down my > gen_svr without loosing messages? > > Many thanks, > > Andy. > > _______________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > 'cvml', > 'erlang-questions@REDACTED > ')__;> > http://erlang.org/mailman/________listinfo/erlang-questions > > > > > >> > > > > > >>> > > > > _______________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > 'cvml', > 'erlang-questions@REDACTED > ')__;> > http://erlang.org/mailman/________listinfo/erlang-questions > > > > > >> > > > > > >>> > > > > _______________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > 'cvml', > 'erlang-questions@REDACTED > ')__;> > http://erlang.org/mailman/________listinfo/erlang-questions > > > > > >> > > > > > >>> > > > > > > _______________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/________listinfo/erlang-questions > > > > > >> > > > > > >>> > > > _______________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/________listinfo/erlang-questions > > > > > >> > > > > > >>> > > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > > > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > -- Lo?c Hoguin Erlang Cowboy Nine Nines From clist@REDACTED Tue May 15 13:26:26 2012 From: clist@REDACTED (Angel J. Alvarez Miguel) Date: Tue, 15 May 2012 13:26:26 +0200 Subject: [erlang-questions] erlang float comparison In-Reply-To: References: <201205140941.01511.clist@uah.es> <201205151104.40442.clist@uah.es> Message-ID: <201205151326.27249.clist@uah.es> On Martes, 15 de Mayo de 2012 12:02:51 Pierpaolo Bernardi escribi?: > On Tue, May 15, 2012 at 11:04 AM, Angel J. Alvarez Miguel wrote: > > The problem can be stated as: > > > > Result = case Myfloat > 1.0 of > > true -> 1.0; > > false MyFloat > > end > > ... > > ... > > math:acos(Result). > > > > after that we came across some values that fail to pass the test > 1.0 > > but are seen as greater to the math:acos function rendering a bad > > arithmetic expression error > > If true, this indicates a serious bug. You should report such cases. > > I am somewhat skeptical that such a gross bug would lie undetected so > far in erlang, so please supply an actual reproducible example. > > Cheers, > P. I agree , please let me recheck the whole thing from the very begining. It starts smelling a bit, like i messed something up... From chandrashekhar.mullaparthi@REDACTED Tue May 15 14:29:06 2012 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Tue, 15 May 2012 13:29:06 +0100 Subject: [erlang-questions] SCTP multihoming In-Reply-To: References: Message-ID: No, I didn't . Thanks for the pointer. Though the original poster says it worked for IPv4 for him. It doesn't work for me even for IPv4 addresses. I'll try the patch out and see. Chandru On May 15, 2012 7:35 AM, "Lukas Larsson" wrote: > I assume you have looked at this[1] thread on the erlang patches > mailinglist? > > [1] http://erlang.org/pipermail/erlang-patches/2011-December/002559.html > > Sent from me > > On 15 maj 2012, at 06:06, Chandru > wrote: > > > Hi all, > > > > I'm having trouble starting a server with multihoming support. > > > > The call: > > 16> (<0.804.0>) call gen_sctp:open([binary, > > {active,once}, > > {reuseaddr,true}, > > {port,3868}, > > {ip,{10,249,8,4}}, > > {ip,{10,249,8,12}}]) > > > > crashes with the report below. Anyone has a quick fix? I've traced it > down to this: > > > > (<0.2364.0>) call > erlang:port_control(#Port<0.5560>,62,[[1],[[15,28],10,249,8,12],[[15,28],10,249,8,4]]) > > (<0.2364.0>) returned from erlang:port_control/3 -> > [0,101,105,110,118,97,108] (which is einval basically) > > > > Any idea why this would be? I'm running this on Solaris 10, with R15B. > > > > cheers > > Chandru > > > > 22> rb:show(4). > > > > CRASH REPORT <0.866.0> 2012-05-15 > 04:44:30 > > > =============================================================================== > > Crashing process > > initial_call > {diameter_sctp,init,['Argument__1']} > > pid > <0.866.0> > > registered_name > [] > > error_info > > {error,badarg, > > [{gen_sctp,open, > > [[binary, > > {active,once}, > > {reuseaddr,true}, > > {port,3868}, > > {ip,{10,249,8,4}}, > > {ip,{10,249,8,12}}]], > > [{file,"gen_sctp.erl"},{line,126}]}, > > {diameter_sctp,open,3, > > [{file,"transport/diameter_sctp.erl"}, > > {line,227}]}, > > {diameter_sctp,i,1, > > [{file,"transport/diameter_sctp.erl"}, > > {line,147}]}, > > {diameter_sctp,init,1, > > [{file,"transport/diameter_sctp.erl"}, > > {line,141}]}, > > {proc_lib,init_p_do_apply,3, > > [{file,"proc_lib.erl"},{line,227}]}]} > > ancestors > > [diameter_sctp_listener_sup,diameter_transport_sup,diameter_sup, > > <0.779.0>] > > messages > [] > > links > [<0.803.0>] > > dictionary > [] > > trap_exit > false > > status > running > > heap_size > 75025 > > stack_size > 24 > > reductions > 736 > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tomas.abrahamsson@REDACTED Tue May 15 15:20:18 2012 From: tomas.abrahamsson@REDACTED (Tomas Abrahamsson) Date: Tue, 15 May 2012 15:20:18 +0200 Subject: [erlang-questions] SCTP multihoming In-Reply-To: References: Message-ID: Hi, On Linux, it worked on IPv4, but not IPv6 for me. When I wrote the original patch I hadn't tried Solaris (or FreeBSD) So I found additional issues on Solaris and FreeBSD: even IPv4 SCTP multihoming didn't work there. Instead of trying out the patch in my original post, I'd recommend trying out the https://github.com/tomas-abrahamsson/otp/compare/sctp-multihoming (referenced later in the thead: http://erlang.org/pipermail/erlang-patches/2012-May/002788.html) BRs Tomas On Tue, May 15, 2012 at 2:29 PM, Chandru wrote: > No, I didn't . Thanks? for the pointer. Though the original poster says it > worked for IPv4 for him. It doesn't work for me even for IPv4 addresses. > I'll try the patch out and see. > > Chandru > > On May 15, 2012 7:35 AM, "Lukas Larsson" wrote: >> >> I assume you have looked at this[1] thread on the erlang patches >> mailinglist? >> >> [1] http://erlang.org/pipermail/erlang-patches/2011-December/002559.html >> >> Sent from me >> >> On 15 maj 2012, at 06:06, Chandru >> wrote: >> >> > Hi all, >> > >> > I'm having trouble starting a server with multihoming support. >> > >> > The call: >> > 16> (<0.804.0>) call gen_sctp:open([binary, >> > ?{active,once}, >> > ?{reuseaddr,true}, >> > ?{port,3868}, >> > ?{ip,{10,249,8,4}}, >> > ?{ip,{10,249,8,12}}]) >> > >> > crashes with the report below. Anyone has a quick fix? I've traced it >> > down to this: >> > >> > (<0.2364.0>) call >> > erlang:port_control(#Port<0.5560>,62,[[1],[[15,28],10,249,8,12],[[15,28],10,249,8,4]]) >> > (<0.2364.0>) returned from erlang:port_control/3 -> >> > [0,101,105,110,118,97,108] (which is einval basically) >> > >> > Any idea why this would be? I'm running this on Solaris 10, with R15B. >> > >> > cheers >> > Chandru >> > >> > 22> rb:show(4). >> > >> > CRASH REPORT ?<0.866.0> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 2012-05-15 >> > 04:44:30 >> > >> > =============================================================================== >> > Crashing process >> > ? ?initial_call >> > ?{diameter_sctp,init,['Argument__1']} >> > ? ?pid >> > ?<0.866.0> >> > ? ?registered_name >> > ? ? [] >> > ? ?error_info >> > ? ? ? ? ?{error,badarg, >> > ? ? ? ? ? ? ? ?[{gen_sctp,open, >> > ? ? ? ? ? ? ? ? ? ? ? ? ? [[binary, >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {active,once}, >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {reuseaddr,true}, >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {port,3868}, >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {ip,{10,249,8,4}}, >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {ip,{10,249,8,12}}]], >> > ? ? ? ? ? ? ? ? ? ? ? ? ? [{file,"gen_sctp.erl"},{line,126}]}, >> > ? ? ? ? ? ? ? ? {diameter_sctp,open,3, >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[{file,"transport/diameter_sctp.erl"}, >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {line,227}]}, >> > ? ? ? ? ? ? ? ? {diameter_sctp,i,1, >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[{file,"transport/diameter_sctp.erl"}, >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {line,147}]}, >> > ? ? ? ? ? ? ? ? {diameter_sctp,init,1, >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[{file,"transport/diameter_sctp.erl"}, >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {line,141}]}, >> > ? ? ? ? ? ? ? ? {proc_lib,init_p_do_apply,3, >> > ? ? ? ? ? ? ? ? ? ? ? ? ? [{file,"proc_lib.erl"},{line,227}]}]} >> > ? ?ancestors >> > >> > ?[diameter_sctp_listener_sup,diameter_transport_sup,diameter_sup, >> > ? ? ? ? ?<0.779.0>] >> > ? ?messages >> > ? ? ?[] >> > ? ?links >> > ?[<0.803.0>] >> > ? ?dictionary >> > ? ? ?[] >> > ? ?trap_exit >> > ?false >> > ? ?status >> > running >> > ? ?heap_size >> > ?75025 >> > ? ?stack_size >> > ? ? ?24 >> > ? ?reductions >> > ? ? 736 >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From bob@REDACTED Tue May 15 17:20:29 2012 From: bob@REDACTED (Bob Ippolito) Date: Tue, 15 May 2012 09:20:29 -0600 Subject: [erlang-questions] msg mailbox and gen_svr shutdown In-Reply-To: <4FB23BAF.1020201@ninenines.eu> References: <4FB052FF.9080604@ferd.ca> <4FB091D6.8000409@ninenines.eu> <4FB1B204.9030006@ninenines.eu> <4FB1CA6A.8080002@ninenines.eu> <4FB1E419.9000000@ninenines.eu> <4FB23BAF.1020201@ninenines.eu> Message-ID: This isn't acceptable for me, because graceful shutdown can take minutes. Not listening for connections is ok for me because the clients connect to multiple servers and choose whichever seems fastest. This method of handoff reduces the downtime for a given machine to msec. Having a machine down for minutes would still work, but it's not desirable to have rolling deploys take hours instead of minutes. Any solution that requires hot code loading is also not going to work when you upgrade the VM. The only way to do better is to be able to pass sockets from one OS process to another (new NIFs), or open a hole to allow the app to modify pf settings. For me, it's sufficient that if a socket is accepted then it'll be handed appropriately and in a timely manner, so I prefer the current (cross-platform and pure erlang) solution. On Tuesday, May 15, 2012, Lo?c Hoguin wrote: > I personally wouldn't want this to become common, because even if you > restart immediately, you would lose a few incoming connections (or a lot > depending on the popularity of your service). If I understand correctly you > go as far as restarting the whole node? > > This isn't like nginx, since nginx doesn't stop listening at any point. > You could actually get something closer to nginx by doing this: > > * set acceptors to 0 (incoming connections get queued) > * gracefully shutdown the running connections (application dependent) > * load the new code > * optionally restart parts of your supervision tree > * set acceptors back to N > > You can add an optional set_protocol_options somewhere in there if you > need to upgrade the dispatch list (maybe after loading the new code you > could always run set_protocol_options(my_**project:get_dispatch_stuff) or > something, which would always get the last options for your project). > > The number of incoming connections queued is also application dependent > (and will be upgradable later on so you can increase temporarily). > > This would allow you to do what you want without any extra feature from > what is currently planned, and without losing any incoming connections. > > All this is easily scriptable of course. > > If that still doesn't fit your needs I'd suggest opening a ticket to see > what others think about it. > > On 05/15/2012 07:35 AM, Bob Ippolito wrote: > >> The idea is to start a new version of the service and do a quick >> hand-off of listening from old to new (possibly even switching back). >> This is particularly useful for code upgrades that aren't suitable for >> hot code loading. We're doing continuous integration this way, as hot >> code loading requires too much work and/or planning for us at this >> stage. It's currently an uncommon need but it would be a lot more common >> if it was well documented and easier to do. >> >> As far as load balancers go, we're not using them. For our use case we >> don't need them, so why add the complexity and/or cost? There is port >> forwarding done by the OS kernel but I want to avoid requiring some >> setuid binary to do the privilege escalation that would be necessary to >> change ports on the fly. >> >> Nginx does upgrades in a similar way. >> http://wiki.nginx.org/**CommandLine#Upgrading_To_a_** >> New_Binary_On_The_Fly >> >> On Monday, May 14, 2012, Lo?c Hoguin wrote: >> >> As with all proposed features I first need to understand what it's >> used for and why, and whether it'll be useful to other people. What >> you are doing just sounds very weird to me and it's the first time I >> hear of a need for something like this so at this point I'm neither >> trying to avoid anything nor considering it useful functionality, >> just wanting to understand why. >> >> On 05/15/2012 06:42 AM, Bob Ippolito wrote: >> >> Yes, I want the other process to immediately take over. >> Requiring a load >> balancer change isn't an elegant solution, I don't understand >> why you're >> trying to avoid adding this useful functionality. >> >> On Monday, May 14, 2012, Lo?c Hoguin wrote: >> >> You stop listening after you're done gracefully stopping your >> currently running processes. >> >> For HTTP that could be something like: >> >> Set acceptors to 0. >> Optionally set 'onrequest' hook to reply with a 503 (for >> keepalives). >> Gracefully stop your processes. >> Stop the listener. >> >> Or do you need another process to take over immediately? >> Because in >> that case you usually don't need to use the same listening >> port, you >> can just change your firewall/lb rules for the port >> redirection from >> 80->P1 to 80->P2. >> >> On 05/15/2012 03:34 AM, Bob Ippolito wrote: >> >> I don't think that is sufficient, you will need to stop >> listening as >> well so the next OS process can take over. >> >> On Monday, May 14, 2012, Lo?c Hoguin wrote: >> >> This will be possible later on by reducing the number >> of >> acceptors >> to 0. This should be added sometimes this summer >> after the >> acceptor >> split happens in Cowboy. >> >> On 05/14/2012 07:26 PM, Bob Ippolito wrote: >> >> I agree that graceful shutdown is very >> application-specific. >> However, >> cowboy doesn't currently facilitate any sort of >> graceful >> shutdown unless >> you read the source code and poke directly at the >> appropriate >> supervisors like I did. The application specific >> stuff >> is easily >> done on >> your own with a process registry or a timeout, >> we used a >> combination of >> gproc and a timeout if things didn't shut down in >> an >> acceptable >> time frame. >> >> I would suggest something like >> cowboy:stop_listener/1, maybe >> something >> like cowboy:stop_listening/1 or >> cowboy:stop_accepting/1. >> >> On Sun, May 13, 2012 at 11:02 PM, Lo?c Hoguin >> > > wrote: >> >> Right, this is pretty much what I said to >> Paul in >> PM. It was in >> R14B03 that the behavior changed. I >> apparently have >> a @todo >> wrong >> past that release, and will take a look if I >> find other >> things to >> https://gist.github.com/______**__2655724 >> >> > >> >> >> >> >> >> >> > >> >> >> >>> >> >> >> > >> >> >> >> >> >> >> > >> >> >> >>>> >> >> On Thu, May 10, 2012 at 2:53 AM, >> Pawe? >> Peregud >> >> > 'paulperegud@REDACTED >> ');>> wrote: >> >> I was having fun with >> supervisors yesterday >> (Cowboy seems to >> fail to fulfill the promise of >> not killing >> request processes >> after listener removal) and I >> have an >> example. >> I've only >> investigated the case when >> supervisor is >> killed, >> so YMMV. >> Example code is attached. You >> may modify >> it to >> check the >> behavior in your case. >> >> Start supervisor tree with >> exp_sup_sup:start_link(). Execute >> test with exp_sup_sup:test() and >> exp_sup_sup:test_simple(). >> >> In case of dying supervisor the >> answer >> is "no, >> it does not". >> >> When supervisor dies, your >> process is >> killed as >> via link >> mechanism, so it may leave some >> unprocessed >> messages in >> inbox. To make sure that every >> delivered >> message >> is served, >> you need to add >> process_flag(trap_exit, >> true). >> Messages that >> are sent after the moment when >> supervisor dies >> are not >> processed. >> >> Best regards, >> >> Paul. >> >> >> On May 9, 2012 11:06 AM, "Andy >> Richards" >> > >> > 'andy.richards.iit@REDACTED**________com >> ');>> wrote: >> >> Hi, >> >> I can't seem to see any >> confirmation in the >> documentation >> so was wondering if anyone could >> confirm if >> messages are >> still sent to a supervised gen_svr >> following a >> shutdown >> message? >> >> If so how do I cleanly shutdown my >> gen_svr without >> loosing messages? I read in the >> supervisor child >> spec >> that a shutdown can be set to >> infinity >> which i hoped >> would allow me to process the >> msg's in my >> mailbox but if >> I do this will my module >> continue to receive >> messages >> from other processes? Is my >> approach >> flawed and >> if so >> what other ways are there to >> cleanly >> shutting >> down my >> gen_svr without loosing messages? >> >> Many thanks, >> >> Andy. >> >> ______________________________** >> _________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > 'cvml', >> 'erlang-questions@REDACTED >> ')__;> >> http://erlang.org/mailman/____**____listinfo/erlang-questions >> >> > >> >> >> >> >> >> >> > >> >> >> >>> >> >> >> > >> >> >> >> >> >> >> > >> >> >> >>>> >> >> >> >> ______________________________** >> _________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > 'cvml', >> 'erlang-questions@REDACTED >> ')__;> >> http://erlang.org/mailman/____**____listinfo/erlang-questions >> >> > >> >> >> >> >> >> >> > >> >> >> >>> >> >> >> > >> >> >> >> >> >> >> > >> >> >> >>>> >> >> >> >> ______________________________** >> _________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Tue May 15 17:39:29 2012 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Tue, 15 May 2012 16:39:29 +0100 Subject: [erlang-questions] SCTP multihoming In-Reply-To: References: Message-ID: Thank you Tomas . That sounds promising. I'll try and report back. Chandru On May 15, 2012 2:20 PM, "Tomas Abrahamsson" wrote: > Hi, > > On Linux, it worked on IPv4, but not IPv6 for me. > When I wrote the original patch I hadn't tried Solaris (or FreeBSD) > So I found additional issues on Solaris and FreeBSD: even IPv4 > SCTP multihoming didn't work there. Instead of trying out the > patch in my original post, I'd recommend trying out the > https://github.com/tomas-abrahamsson/otp/compare/sctp-multihoming > (referenced later in the thead: > http://erlang.org/pipermail/erlang-patches/2012-May/002788.html) > > BRs > Tomas > > On Tue, May 15, 2012 at 2:29 PM, Chandru > wrote: > > No, I didn't . Thanks for the pointer. Though the original poster says > it > > worked for IPv4 for him. It doesn't work for me even for IPv4 addresses. > > I'll try the patch out and see. > > > > Chandru > > > > On May 15, 2012 7:35 AM, "Lukas Larsson" wrote: > >> > >> I assume you have looked at this[1] thread on the erlang patches > >> mailinglist? > >> > >> [1] > http://erlang.org/pipermail/erlang-patches/2011-December/002559.html > >> > >> Sent from me > >> > >> On 15 maj 2012, at 06:06, Chandru > > >> wrote: > >> > >> > Hi all, > >> > > >> > I'm having trouble starting a server with multihoming support. > >> > > >> > The call: > >> > 16> (<0.804.0>) call gen_sctp:open([binary, > >> > {active,once}, > >> > {reuseaddr,true}, > >> > {port,3868}, > >> > {ip,{10,249,8,4}}, > >> > {ip,{10,249,8,12}}]) > >> > > >> > crashes with the report below. Anyone has a quick fix? I've traced it > >> > down to this: > >> > > >> > (<0.2364.0>) call > >> > > erlang:port_control(#Port<0.5560>,62,[[1],[[15,28],10,249,8,12],[[15,28],10,249,8,4]]) > >> > (<0.2364.0>) returned from erlang:port_control/3 -> > >> > [0,101,105,110,118,97,108] (which is einval basically) > >> > > >> > Any idea why this would be? I'm running this on Solaris 10, with R15B. > >> > > >> > cheers > >> > Chandru > >> > > >> > 22> rb:show(4). > >> > > >> > CRASH REPORT <0.866.0> 2012-05-15 > >> > 04:44:30 > >> > > >> > > =============================================================================== > >> > Crashing process > >> > initial_call > >> > {diameter_sctp,init,['Argument__1']} > >> > pid > >> > <0.866.0> > >> > registered_name > >> > [] > >> > error_info > >> > {error,badarg, > >> > [{gen_sctp,open, > >> > [[binary, > >> > {active,once}, > >> > {reuseaddr,true}, > >> > {port,3868}, > >> > {ip,{10,249,8,4}}, > >> > {ip,{10,249,8,12}}]], > >> > [{file,"gen_sctp.erl"},{line,126}]}, > >> > {diameter_sctp,open,3, > >> > [{file,"transport/diameter_sctp.erl"}, > >> > {line,227}]}, > >> > {diameter_sctp,i,1, > >> > [{file,"transport/diameter_sctp.erl"}, > >> > {line,147}]}, > >> > {diameter_sctp,init,1, > >> > [{file,"transport/diameter_sctp.erl"}, > >> > {line,141}]}, > >> > {proc_lib,init_p_do_apply,3, > >> > [{file,"proc_lib.erl"},{line,227}]}]} > >> > ancestors > >> > > >> > [diameter_sctp_listener_sup,diameter_transport_sup,diameter_sup, > >> > <0.779.0>] > >> > messages > >> > [] > >> > links > >> > [<0.803.0>] > >> > dictionary > >> > [] > >> > trap_exit > >> > false > >> > status > >> > running > >> > heap_size > >> > 75025 > >> > stack_size > >> > 24 > >> > reductions > >> > 736 > >> > _______________________________________________ > >> > erlang-questions mailing list > >> > erlang-questions@REDACTED > >> > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue May 15 18:40:06 2012 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 15 May 2012 18:40:06 +0200 Subject: [erlang-questions] msg mailbox and gen_svr shutdown In-Reply-To: References: <4FB052FF.9080604@ferd.ca> <4FB091D6.8000409@ninenines.eu> <4FB1B204.9030006@ninenines.eu> <4FB1CA6A.8080002@ninenines.eu> <4FB1E419.9000000@ninenines.eu> <4FB23BAF.1020201@ninenines.eu> Message-ID: <4FB286E6.4090709@ninenines.eu> Okay, thanks for the details, I have a clearer view about it now. I think I can include something that could help you. Today we have: * start_listener * stop_listener What we could also have could be something like: * suspend_listener * resume_listener What this basically would do is normally stop part of the listener's supervision tree, that is cowboy_acceptors_sup, or start it again when we want to resume. Stopping that part would have the effect of killing the listening socket and the acceptors. Considering your gist you know that already. However you are also killing cowboy_listener, which shouldn't really considering protocols can call it (but that's more a consideration for me, it probably isn't in your case). This all fits properly with the patch I'm working on for ranch (split off acceptor code that Cowboy will use past 0.6). On restarting this supervisor, a new listening socket gets opened, the acceptors query the listener gen_server for up to date protocol options and everything runs as before (with my patch, anyway). Done like this I can certainly include it, being able to suspend and resume listening without killing currently running connections could definitely be useful for many things (the key point being to be able to resume). Especially when manipulating a release in production when you're behind a lb. Gracefully stopping connection processes, either custom protocols or HTTP handlers, falls into the application's side however. So this interests me eventually, but only after the Ranch switch + changes currently cooking because otherwise you could resume to an older version of the protocol options. Thanks for the suggestion! On 05/15/2012 05:20 PM, Bob Ippolito wrote: > This isn't acceptable for me, because graceful shutdown can take > minutes. Not listening for connections is ok for me because the clients > connect to multiple servers and choose whichever seems fastest. This > method of handoff reduces the downtime for a given machine to msec. > Having a machine down for minutes would still work, but it's not > desirable to have rolling deploys take hours instead of minutes. Any > solution that requires hot code loading is also not going to work when > you upgrade the VM. > > The only way to do better is to be able to pass sockets from one OS > process to another (new NIFs), or open a hole to allow the app to modify > pf settings. For me, it's sufficient that if a socket is accepted then > it'll be handed appropriately and in a timely manner, so I prefer the > current (cross-platform and pure erlang) solution. > > On Tuesday, May 15, 2012, Lo?c Hoguin wrote: > > I personally wouldn't want this to become common, because even if > you restart immediately, you would lose a few incoming connections > (or a lot depending on the popularity of your service). If I > understand correctly you go as far as restarting the whole node? > > This isn't like nginx, since nginx doesn't stop listening at any > point. You could actually get something closer to nginx by doing this: > > * set acceptors to 0 (incoming connections get queued) > * gracefully shutdown the running connections (application dependent) > * load the new code > * optionally restart parts of your supervision tree > * set acceptors back to N > > You can add an optional set_protocol_options somewhere in there if > you need to upgrade the dispatch list (maybe after loading the new > code you could always run > set_protocol_options(my___project:get_dispatch_stuff) or something, > which would always get the last options for your project). > > The number of incoming connections queued is also application > dependent (and will be upgradable later on so you can increase > temporarily). > > This would allow you to do what you want without any extra feature > from what is currently planned, and without losing any incoming > connections. > > All this is easily scriptable of course. > > If that still doesn't fit your needs I'd suggest opening a ticket to > see what others think about it. > > On 05/15/2012 07:35 AM, Bob Ippolito wrote: > > The idea is to start a new version of the service and do a quick > hand-off of listening from old to new (possibly even switching > back). > This is particularly useful for code upgrades that aren't > suitable for > hot code loading. We're doing continuous integration this way, > as hot > code loading requires too much work and/or planning for us at this > stage. It's currently an uncommon need but it would be a lot > more common > if it was well documented and easier to do. > > As far as load balancers go, we're not using them. For our use > case we > don't need them, so why add the complexity and/or cost? There is > port > forwarding done by the OS kernel but I want to avoid requiring some > setuid binary to do the privilege escalation that would be > necessary to > change ports on the fly. > > Nginx does upgrades in a similar way. > http://wiki.nginx.org/__CommandLine#Upgrading_To_a___New_Binary_On_The_Fly > > > On Monday, May 14, 2012, Lo?c Hoguin wrote: > > As with all proposed features I first need to understand > what it's > used for and why, and whether it'll be useful to other > people. What > you are doing just sounds very weird to me and it's the > first time I > hear of a need for something like this so at this point I'm > neither > trying to avoid anything nor considering it useful > functionality, > just wanting to understand why. > > On 05/15/2012 06:42 AM, Bob Ippolito wrote: > > Yes, I want the other process to immediately take over. > Requiring a load > balancer change isn't an elegant solution, I don't > understand > why you're > trying to avoid adding this useful functionality. > > On Monday, May 14, 2012, Lo?c Hoguin wrote: > > You stop listening after you're done gracefully > stopping your > currently running processes. > > For HTTP that could be something like: > > Set acceptors to 0. > Optionally set 'onrequest' hook to reply with a 503 (for > keepalives). > Gracefully stop your processes. > Stop the listener. > > Or do you need another process to take over immediately? > Because in > that case you usually don't need to use the same > listening > port, you > can just change your firewall/lb rules for the port > redirection from > 80->P1 to 80->P2. > > On 05/15/2012 03:34 AM, Bob Ippolito wrote: > > I don't think that is sufficient, you will need > to stop > listening as > well so the next OS process can take over. > > On Monday, May 14, 2012, Lo?c Hoguin wrote: > > This will be possible later on by reducing > the number of > acceptors > to 0. This should be added sometimes this summer > after the > acceptor > split happens in Cowboy. > > On 05/14/2012 07:26 PM, Bob Ippolito wrote: > > I agree that graceful shutdown is very > application-specific. > However, > cowboy doesn't currently facilitate any > sort of > graceful > shutdown unless > you read the source code and poke > directly at the > appropriate > supervisors like I did. The application > specific > stuff > is easily > done on > your own with a process registry or a > timeout, > we used a > combination of > gproc and a timeout if things didn't > shut down in an > acceptable > time frame. > > I would suggest something like > cowboy:stop_listener/1, maybe > something > like cowboy:stop_listening/1 or > cowboy:stop_accepting/1. > > On Sun, May 13, 2012 at 11:02 PM, Lo?c > Hoguin > > wrote: > > Right, this is pretty much what I > said to > Paul in > PM. It was in > R14B03 that the behavior changed. I > apparently have > a @todo > wrong > past that release, and will take a > look if I > find other > things to > https://gist.github.com/__________2655724 > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>> > > On Thu, May 10, 2012 at > 2:53 AM, > Pawe? > Peregud > > 'paulperegud@REDACTED > ');>> wrote: > > I was having fun with > supervisors yesterday > (Cowboy seems to > fail to fulfill the > promise of > not killing > request processes > after listener removal) > and I > have an > example. > I've only > investigated the case when > supervisor is > killed, > so YMMV. > Example code is > attached. You > may modify > it to > check the > behavior in your case. > > Start supervisor tree with > exp_sup_sup:start_link(). Execute > test with > exp_sup_sup:test() and > exp_sup_sup:test_simple(). > > In case of dying > supervisor the > answer > is "no, > it does not". > > When supervisor dies, your > process is > killed as > via link > mechanism, so it may > leave some > unprocessed > messages in > inbox. To make sure that > every > delivered > message > is served, > you need to add > process_flag(trap_exit, > true). > Messages that > are sent after the > moment when > supervisor dies > are not > processed. > > Best regards, > > Paul. > > > On May 9, 2012 11:06 AM, > "Andy > Richards" > > 'andy.richards.iit@REDACTED > ');>> wrote: > > Hi, > > I can't seem to see any > confirmation in the > documentation > so was wondering if > anyone could > confirm if > messages are > still sent to a > supervised gen_svr > following a > shutdown > message? > > If so how do I cleanly > shutdown my > gen_svr without > loosing messages? I read > in the > supervisor child > spec > that a shutdown can be > set to > infinity > which i hoped > would allow me to > process the > msg's in my > mailbox but if > I do this will my module > continue to receive > messages > from other processes? Is > my approach > flawed and > if so > what other ways are > there to cleanly > shutting > down my > gen_svr without loosing > messages? > > Many thanks, > > Andy. > > > _________________________________________________________ > erlang-questions mailing > list > erlang-questions@REDACTED > 'cvml', > 'erlang-questions@REDACTED > ')__;> > http://erlang.org/mailman/__________listinfo/erlang-questions > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>> > > > > > _________________________________________________________ > erlang-questions mailing > list > erlang-questions@REDACTED > 'cvml', > 'erlang-questions@REDACTED > ')__;> > http://erlang.org/mailman/__________listinfo/erlang-questions > > > > > >> > > > > > >>> > > > > > >> > > > > > >>>> > > > > > _________________________________________________________ > erlang-questions mailing > list > erlang-questions@REDACTED > -- Lo?c Hoguin Erlang Cowboy Nine Nines From emeka_1978@REDACTED Tue May 15 19:03:47 2012 From: emeka_1978@REDACTED (eigenfunction) Date: Tue, 15 May 2012 10:03:47 -0700 (PDT) Subject: [erlang-questions] windows In-Reply-To: <3582453A-BC00-47D5-B9AD-2E5AED675033@rogvall.se> References: <111ECCC1-8CC3-4C79-A331-28161A5FFFC2@rogvall.se> <3582453A-BC00-47D5-B9AD-2E5AED675033@rogvall.se> Message-ID: <7bbdcba6-5bff-4d66-bf62-f68ca50ae3c0@dg7g2000vbb.googlegroups.com> I develop on windows+cygwin and Linux in about 50/50. That's one reason why rebar was a no go for me. On the other hand, sinan works perfectly in both environment for me. Most of the times, i use the same code base shared between windows and linux with the same sinan and i just switch in and out of the virtual machine to test in both environments. Just my 2 cents. On May 14, 11:04?pm, Tony Rogvall wrote: > On 14 maj 2012, at 22:30, Tuncer Ayaz wrote: > > > On Sun, May 13, 2012 at 10:44 PM, Tony Rogvall wrote: > >> The next thing was that I specified: > > >> ? ? {"(win32)","priv/dthread_drv.dll", > >> ? ? ?["c_src/dthread.c", "c_src/dthread_drv.c"]}, > > > You don't need that win32 rule. > > rebar automatically switches the extension on Windows as follows: > > 1. priv/foo_drv.so -> priv/foo_drv.dll > > 2. priv/foo -> priv/foo.exe > > Yes I noticed ;-) > > >> This leads to the (bug?) that the spec ended up as {spec, drv, > >> ".dll", ...} the file was linked as .dll, no wonder I could not find > >> it :-) > > > Thanks for the bug report. Can you confirm it's fixed in the ta-dll > > branch? > > Next time I start windows. Unfortunately this week. > > Thanks > > /Tony > > "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" > > _______________________________________________ > erlang-questions mailing list > erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions From comptekki@REDACTED Tue May 15 20:03:50 2012 From: comptekki@REDACTED (Wes James) Date: Tue, 15 May 2012 12:03:50 -0600 Subject: [erlang-questions] secure use of cookies in an erlang application Message-ID: I'm using cowboy for an application and I'm setting a simple cookie with an expire to log users out after a certain time. Using a simple cookie seams like it is not secure, though. As someone could read the code and set the cookie in their browser and then get access to the site, right? Are there any examples of securely using cookies in cowboy or some other erlang app/framework that shows how cookies are used? I guess some random time based cookie might work better. Thanks, Wes -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue May 15 20:10:50 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 15 May 2012 20:10:50 +0200 Subject: [erlang-questions] secure use of cookies in an erlang application In-Reply-To: References: Message-ID: <4FB29C2A.8020001@ninenines.eu> On login, make the server generate an UUID, associate that UUID with the logged in user, and set it as the cookie you will use to identify the user. Then just compare that cookie with your list of logged in users to find who it is. It should already be a good start. On 05/15/2012 08:03 PM, Wes James wrote: > I'm using cowboy for an application and I'm setting a simple cookie with > an expire to log users out after a certain time. Using a simple cookie > seams like it is not secure, though. As someone could read the code and > set the cookie in their browser and then get access to the site, right? > Are there any examples of securely using cookies in cowboy or some > other erlang app/framework that shows how cookies are used? I guess > some random time based cookie might work better. > > Thanks, > > Wes > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin Erlang Cowboy Nine Nines From comptekki@REDACTED Tue May 15 20:13:25 2012 From: comptekki@REDACTED (Wes James) Date: Tue, 15 May 2012 12:13:25 -0600 Subject: [erlang-questions] secure use of cookies in an erlang application In-Reply-To: <4FB29C2A.8020001@ninenines.eu> References: <4FB29C2A.8020001@ninenines.eu> Message-ID: Ok - thanks for the quick response! Wes On Tue, May 15, 2012 at 12:10 PM, Lo?c Hoguin wrote: > On login, make the server generate an UUID, associate that UUID with the > logged in user, and set it as the cookie you will use to identify the user. > Then just compare that cookie with your list of logged in users to find who > it is. > > It should already be a good start. > > > On 05/15/2012 08:03 PM, Wes James wrote: > >> I'm using cowboy for an application and I'm setting a simple cookie with >> an expire to log users out after a certain time. Using a simple cookie >> seams like it is not secure, though. As someone could read the code and >> set the cookie in their browser and then get access to the site, right? >> Are there any examples of securely using cookies in cowboy or some >> other erlang app/framework that shows how cookies are used? I guess >> some random time based cookie might work better. >> >> Thanks, >> >> Wes >> >> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.rutkowski@REDACTED Tue May 15 20:19:50 2012 From: adam.rutkowski@REDACTED (Adam Rutkowski) Date: Tue, 15 May 2012 20:19:50 +0200 Subject: [erlang-questions] secure use of cookies in an erlang application In-Reply-To: References: Message-ID: <59195F00-3978-4D48-90ED-19C37535FE0F@jtendo.com> On May 15, 2012, at 8:03 PM, Wes James wrote: > I'm using cowboy for an application and I'm setting a simple cookie with an expire to log users out after a certain time. Using a simple cookie seams like it is not secure, though. As someone could read the code and set the cookie in their browser and then get access to the site, right? Are there any examples of securely using cookies in cowboy or some other erlang app/framework that shows how cookies are used? I guess some random time based cookie might work better. One simple approach is to make an expiring process that generates a random binary per user session. Use its value to encyrpt the cookie, encode it somehow so it's browser/cowboy-friendly (I remember I had some trouble using base64 with cowboy cookie module, which led me to a hexstring eventually), then verify it using the onrequest hook. Whatever you choose to do, pass it through SSL and use secure cookies (https://github.com/essen/cowboy/blob/master/src/cowboy_cookies.erl#L28). Hope that helps. -- AR From bob@REDACTED Tue May 15 20:26:32 2012 From: bob@REDACTED (Bob Ippolito) Date: Tue, 15 May 2012 12:26:32 -0600 Subject: [erlang-questions] secure use of cookies in an erlang application In-Reply-To: References: <4FB29C2A.8020001@ninenines.eu> Message-ID: You should also make sure to only use https for cookies [1] and to make them invisible to JavaScript [2]. This of course makes it a bit more complicated because you'll need a SSL certificate, but that's basically necessary for session security. cowboy_cookies:cookie( ?COOKIENAME, SessionId, [{http_only, true}, {secure, true}]). [1] HTTP makes it too easy for an observer to steal your session. {secure, true} forces the cookie to only be visible on HTTPS. [2] JavaScript accessible cookies can be stolen via CSRF attacks (given some other constraints, but it's better to be safe). {secure, true} makes the cookie show up as an empty string from JavaScript. On Tue, May 15, 2012 at 12:13 PM, Wes James wrote: > Ok - thanks for the quick response! > > Wes > > On Tue, May 15, 2012 at 12:10 PM, Lo?c Hoguin wrote: > >> On login, make the server generate an UUID, associate that UUID with the >> logged in user, and set it as the cookie you will use to identify the user. >> Then just compare that cookie with your list of logged in users to find who >> it is. >> >> It should already be a good start. >> >> >> On 05/15/2012 08:03 PM, Wes James wrote: >> >>> I'm using cowboy for an application and I'm setting a simple cookie with >>> an expire to log users out after a certain time. Using a simple cookie >>> seams like it is not secure, though. As someone could read the code and >>> set the cookie in their browser and then get access to the site, right? >>> Are there any examples of securely using cookies in cowboy or some >>> other erlang app/framework that shows how cookies are used? I guess >>> some random time based cookie might work better. >>> >>> Thanks, >>> >>> Wes >>> >>> >>> ______________________________**_________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/**listinfo/erlang-questions >>> >> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Tue May 15 20:28:38 2012 From: bob@REDACTED (Bob Ippolito) Date: Tue, 15 May 2012 12:28:38 -0600 Subject: [erlang-questions] secure use of cookies in an erlang application In-Reply-To: <59195F00-3978-4D48-90ED-19C37535FE0F@jtendo.com> References: <59195F00-3978-4D48-90ED-19C37535FE0F@jtendo.com> Message-ID: On Tue, May 15, 2012 at 12:19 PM, Adam Rutkowski wrote: > On May 15, 2012, at 8:03 PM, Wes James wrote: > > > I'm using cowboy for an application and I'm setting a simple cookie with > an expire to log users out after a certain time. Using a simple cookie > seams like it is not secure, though. As someone could read the code and > set the cookie in their browser and then get access to the site, right? > Are there any examples of securely using cookies in cowboy or some other > erlang app/framework that shows how cookies are used? I guess some random > time based cookie might work better. > > One simple approach is to make an expiring process that generates a random > binary per user session. Use its value to encyrpt the cookie, encode it > somehow so it's browser/cowboy-friendly (I remember I had some trouble > using base64 with cowboy cookie module, which led me to a hexstring > eventually), then verify it using the onrequest hook. > Whatever you choose to do, pass it through SSL and use secure cookies ( > https://github.com/essen/cowboy/blob/master/src/cowboy_cookies.erl#L28). > The base64 codec uses some characters that aren't safe in URLs or cookies, so you'll have to modify it slightly. http://en.wikipedia.org/wiki/Base64#URL_applications -bob -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue May 15 21:44:52 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 15 May 2012 23:44:52 +0400 Subject: [erlang-questions] secure use of cookies in an erlang application In-Reply-To: References: <59195F00-3978-4D48-90ED-19C37535FE0F@jtendo.com> Message-ID: Ruby on Rails implements a wonderful idea: you pack your session as a JSON: {user_id : 15, ip : "23.45.67.89", expire_at : 1234567780}, then you take base64 from it, then you concat base64 with secret key and take sha1 from it. Cookie = base64(Session) ++ "--" ++ sha1(base64(Session) ++ SecretKey) later on each request you validate that session is signed properly, check IP address, timeout and use your really stateless session. From dmercer@REDACTED Tue May 15 22:48:24 2012 From: dmercer@REDACTED (David Mercer) Date: Tue, 15 May 2012 15:48:24 -0500 Subject: [erlang-questions] How to debug "Kernel pid terminated" Message-ID: <014c01cd32dc$1351bc10$39f53430$@com> I have a distributed application that I run on a couple of nodes. I have had various problems where one node spontaneously decides another node is not available and starts up its own instance of the application, but this one is a first for me: One of my failover nodes exited after printing the following messages: =ERROR REPORT==== 14-May-2012::19:43:24 === ** Generic server dist_ac terminating ** Last message in was {internal_restart_appl,cron} ** When Server state == {state, [{appl,cron, {failover,cron_main@REDACTED}, 5000, [cron_main@REDACTED, {cron_failover@REDACTED,cron_failover@REDACTED}], [{cron_failover@REDACTED,true}]}], [],[], [cron_failover@REDACTED], [cron], [],[],[],[],[]} ** Reason for termination == ** {{case_clause, {'EXIT', {timeout, {gen_server,call, [application_controller,which_applications]}}}}, [{dist_ac,restart_appl,2,[{file,"dist_ac.erl"},{line,952}]}, {dist_ac,handle_info,2,[{file,"dist_ac.erl"},{line,697}]}, {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,597}]}, {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]} =ERROR REPORT==== 14-May-2012::19:43:24 === server: clickon_backup_server error: enoent path: <<"\\\\ftp-corp2\\SFTP-MW\\70350\\Upload\\837">> {error_logger,{{2012,5,14},{19,43,25}},std_info,[{application,kernel},{exite d,shutdown},{type,permanent}]} {"Kernel pid terminated",application_controller,"{application_terminated,kernel,shutdown} "} Kernel pid terminated (application_controller) ({application_terminated,kernel,shutdown}) Abnormal termination I am guessing this node (cron_failover@REDACTED) somehow lost contact with the main node (cron_main@REDACTED) on the same host. I am not sure, however, why this would cause the whole Erlang node to crash. How would I go about debugging this? (1) What circumstances caused this node to lose contact with the other node on the same host? (2) What can I do to gracefully handle this situation? Here's my thought process so far, which doesn't really answer any of my questions: 1. The error message seems to point me to the case statement on line 952 of dist_ac.erl (restart_appl/2). This is a call to start_appl/3, which expects either {ok, _, _} or {error, _}, but not {'EXIT', .}, which is what it received. 2. Looking at start_appl/3, I doubt it is the keysearch which is throwing the EXIT, so I'm going to assume that it is the call to start_distributed/6. 3. I can continue down this rabbit hole, but I'm not sure how it will answer either of my questions. Can someone who perhaps knows the workings of distributed applications better than I please give me a few pointers? Please advise. Thank-you. David Mercer -------------- next part -------------- An HTML attachment was scrubbed... URL: From emeka_1978@REDACTED Tue May 15 23:59:05 2012 From: emeka_1978@REDACTED (eigenfunction) Date: Tue, 15 May 2012 14:59:05 -0700 (PDT) Subject: [erlang-questions] user home.dir Message-ID: <2dce056d-723a-42e8-98b4-a1008e3f8ba4@w24g2000vby.googlegroups.com> Hi guys, How do i retrieve the user home directory in erlang? I mean the os- independant version of os:getenv("HOME") or os:getenv("HOMEPATH") . Is there such thing? From ok@REDACTED Wed May 16 00:43:40 2012 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 16 May 2012 10:43:40 +1200 Subject: [erlang-questions] erlang float comparison In-Reply-To: <201205151104.40442.clist@uah.es> References: <201205140941.01511.clist@uah.es> <201205151104.40442.clist@uah.es> Message-ID: <7984E468-2463-49AD-A734-3C0E5032E10D@cs.otago.ac.nz> On 15/05/2012, at 9:04 PM, Angel J. Alvarez Miguel wrote: > > The problem can be stated as: > > Result = case Myfloat > 1.0 of > true -> 1.0; > false MyFloat > end > ... > ... > math:acos(Result). > > after that we came across some values that fail to pass the test > 1.0 but are seen > as greater to the math:acos function rendering a bad arithmetic expression error That is pretty astonishing, actually. The obvious question is *what are these values*? I've just written a test loop searching for such values and so far haven't found any. Is there any difference between emulated and native code? Why aren't you worried about Myfloat < -1.0? By the way, this seems a rather strange way to write Result = if Myfloat > 1.0 -> 1.0 ; true -> Myfloat end Do you know what trig library your copy of Erlang was linked with? Here's the relevant code from fdlibm: hx = __HI(x); ix = hx & 0x7fffffff; if (ix >= 0x3ff00000) { /* |x| >= 1 */ if (((ix - 0x3ff00000) | __LO(x)) == 0) { /* |x|==1 */ if (hx > 0) return 0.0; /* acos(1) = 0 */ else return pi + 2.0 * pio2_lo; /* acos(-1)= pi */ } return (x - x) / (x - x); /* acos(|x|>1) is NaN */ } There's no way _that_ could claim an IEEE double was greater than 1.0 if it wasn't. From mononcqc@REDACTED Wed May 16 02:21:21 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 15 May 2012 20:21:21 -0400 Subject: [erlang-questions] secure use of cookies in an erlang application In-Reply-To: References: <4FB29C2A.8020001@ninenines.eu> Message-ID: <4FB2F301.9050907@ferd.ca> CSRF and Javascript cookies are unrelated. I could build a CSRF attack using an img tag or submitting a form, and never touching it with Javascript. The only way to truly protect against CSRF is to use session tokens (referrer checking doesn't work if you expect requests from flash and/or SSL). Store the session token in the form for any request that can change data and match it with the cookie. If either the cookie or token do not match, reject the request. The token should be protected thanks to the Same-Origin Policy implemented by browsers. As far as I know, making the cookie http only has no impact on this, but is rather protection when it comes to XSS accessing cookie data, not CSRF. On 12-05-15 2:26 PM, Bob Ippolito wrote: > You should also make sure to only use https for cookies [1] and to > make them invisible to JavaScript [2]. This of course makes it a bit > more complicated because you'll need a SSL certificate, but that's > basically necessary for session security. > > cowboy_cookies:cookie( > ?COOKIENAME, > SessionId, > [{http_only, true}, > {secure, true}]). > > [1] HTTP makes it too easy for an observer to steal your session. > {secure, true} forces the cookie to only be visible on HTTPS. > [2] JavaScript accessible cookies can be stolen via CSRF attacks > (given some other constraints, but it's better to be safe). {secure, > true} makes the cookie show up as an empty string from JavaScript. > > On Tue, May 15, 2012 at 12:13 PM, Wes James > wrote: > > Ok - thanks for the quick response! > > Wes > > On Tue, May 15, 2012 at 12:10 PM, Lo?c Hoguin > wrote: > > On login, make the server generate an UUID, associate that > UUID with the logged in user, and set it as the cookie you > will use to identify the user. Then just compare that cookie > with your list of logged in users to find who it is. > > It should already be a good start. > > > On 05/15/2012 08:03 PM, Wes James wrote: > > I'm using cowboy for an application and I'm setting a > simple cookie with > an expire to log users out after a certain time. Using a > simple cookie > seams like it is not secure, though. As someone could > read the code and > set the cookie in their browser and then get access to the > site, right? > Are there any examples of securely using cookies in > cowboy or some > other erlang app/framework that shows how cookies are > used? I guess > some random time based cookie might work better. > > Thanks, > > Wes > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Lod(c Hoguin > Erlang Cowboy > Nine Nines > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Wed May 16 02:23:57 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 15 May 2012 20:23:57 -0400 Subject: [erlang-questions] user home.dir In-Reply-To: <2dce056d-723a-42e8-98b4-a1008e3f8ba4@w24g2000vby.googlegroups.com> References: <2dce056d-723a-42e8-98b4-a1008e3f8ba4@w24g2000vby.googlegroups.com> Message-ID: <4FB2F39D.4080801@ferd.ca> Use: {ok, [[Home]]} = init:get_argument(home) and Home will be the user's home directory. On Tue May 15 17:59:05 2012, eigenfunction wrote: > > Hi guys, > How do i retrieve the user home directory in erlang? I mean the os- > independant version of os:getenv("HOME") or os:getenv("HOMEPATH") . > > Is there such thing? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bob@REDACTED Wed May 16 02:39:10 2012 From: bob@REDACTED (Bob Ippolito) Date: Wed, 16 May 2012 00:39:10 +0000 Subject: [erlang-questions] secure use of cookies in an erlang application In-Reply-To: <4FB2F301.9050907@ferd.ca> References: <4FB29C2A.8020001@ninenines.eu> <4FB2F301.9050907@ferd.ca> Message-ID: You're right, I meant XSS but typed CSRF. Thanks :) On Tuesday, May 15, 2012, Fred Hebert wrote: > CSRF and Javascript cookies are unrelated. > > I could build a CSRF attack using an img tag or submitting a form, and > never touching it with Javascript. The only way to truly protect against > CSRF is to use session tokens (referrer checking doesn't work if you expect > requests from flash and/or SSL). > > Store the session token in the form for any request that can change data > and match it with the cookie. If either the cookie or token do not match, > reject the request. The token should be protected thanks to the Same-Origin > Policy implemented by browsers. > > As far as I know, making the cookie http only has no impact on this, but > is rather protection when it comes to XSS accessing cookie data, not CSRF. > > On 12-05-15 2:26 PM, Bob Ippolito wrote: > > You should also make sure to only use https for cookies [1] and to make > them invisible to JavaScript [2]. This of course makes it a bit more > complicated because you'll need a SSL certificate, but that's basically > necessary for session security. > > cowboy_cookies:cookie( > ?COOKIENAME, > SessionId, > [{http_only, true}, > {secure, true}]). > > [1] HTTP makes it too easy for an observer to steal your session. > {secure, true} forces the cookie to only be visible on HTTPS. > [2] JavaScript accessible cookies can be stolen via CSRF attacks (given > some other constraints, but it's better to be safe). {secure, true} makes > the cookie show up as an empty string from JavaScript. > > On Tue, May 15, 2012 at 12:13 PM, Wes James wrote: > > Ok - thanks for the quick response! > > Wes > > On Tue, May 15, 2012 at 12:10 PM, Lo?c Hoguin wrote: > > On login, make the server generate an UUID, associate that UUID with the > logged in user, and set it as the cookie you will use to identify the user. > Then just compare that cookie with your list of logged in users to find who > it is. > > It should already be a good start. > > > On 05/15/2012 08:03 PM, Wes James wrote: > > I'm using cowboy for an application and I'm setting a simple cookie with > an expire to log users out after a certain time. Using a simple cookie > seams like it is not secure, though. As someone could read the code and > set the cookie in their browser and then get access to the site, right? > Are there any examples of securely using cookies in cowboy or some > other erlang app/framework that shows how cookies are used? I guess > some random time based cookie might work better. > > Thanks, > > Wes > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > > > > ____________ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matti.oinas@REDACTED Wed May 16 05:56:44 2012 From: matti.oinas@REDACTED (Matti Oinas) Date: Wed, 16 May 2012 06:56:44 +0300 Subject: [erlang-questions] secure use of cookies in an erlang application In-Reply-To: References: <59195F00-3978-4D48-90ED-19C37535FE0F@jtendo.com> Message-ID: I would recommend reading the paper http://www.cs.umass.edu/~kevinfu/papers/webauth_tr.pdf It is a good description about the method used by Rails and it also presents couple not so good implementations of web authentication. This stackoverflow post is great and contains links to good sources of information including the one mentioned above. http://stackoverflow.com/questions/549/the-definitive-guide-to-forms-based-website-authentication?answertab=votes#tab-top On Tue, May 15, 2012 at 10:44 PM, Max Lapshin wrote: > Ruby on Rails implements a wonderful idea: > > you pack your session as a JSON: ?{user_id : 15, ip : "23.45.67.89", > expire_at : 1234567780}, > then you take base64 from it, > then you concat base64 with secret key and take sha1 from it. > > Cookie = base64(Session) ++ "--" ++ sha1(base64(Session) ++ SecretKey) > > later on each request you validate that session is signed properly, > check IP address, timeout and use your really stateless session. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From gleber.p@REDACTED Wed May 16 10:43:57 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Wed, 16 May 2012 10:43:57 +0200 Subject: [erlang-questions] os:cmd/1 with exit status aka erlexec or eunit_lib:command/1 or ? Message-ID: Hello list. Is there any rebar-compatible easy to use lib for executing shell commands with ability to get it's stdout (stderr would be nice to have separately if possible) and it's exit status? erlexec seems to be old and non-maintained, but does all this. eunit_lib:command/1 does some of this, but it's part of eunit which is rarely a part of production Erlang setup. Any pointers? Best, Gleb From g.a.c.rijnders@REDACTED Wed May 16 10:49:48 2012 From: g.a.c.rijnders@REDACTED (Michel Rijnders) Date: Wed, 16 May 2012 10:49:48 +0200 Subject: [erlang-questions] String versus variable in binary literal Message-ID: Hi, Confused newbie here. Can someone please explain the following: 1> <<"foo">>. <<"foo">> 2> Foo = "foo". "foo" 3> <>. ** exception error: bad argument What's the error in the last case? Best, Michel -- My other car is a cdr. From carlsson.richard@REDACTED Wed May 16 10:56:33 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 16 May 2012 10:56:33 +0200 Subject: [erlang-questions] String versus variable in binary literal In-Reply-To: References: Message-ID: <4FB36BC1.8020903@gmail.com> On 05/16/2012 10:49 AM, Michel Rijnders wrote: > Hi, > > Confused newbie here. Can someone please explain the following: > > 1> <<"foo">>. > <<"foo">> > 2> Foo = "foo". > "foo" > 3> <>. > ** exception error: bad argument > > What's the error in the last case? > > Best, > Michel The bit syntax doesn't (currently) support encoding strings that are not constant literals. This is something that should be fixed, IMO. /Richard From ivan@REDACTED Wed May 16 11:00:28 2012 From: ivan@REDACTED (Ivan Uemlianin) Date: Wed, 16 May 2012 10:00:28 +0100 Subject: [erlang-questions] String versus variable in binary literal In-Reply-To: References: Message-ID: <4FB36CAC.2090505@llaisdy.com> On 16/05/2012 09:49, Michel Rijnders wrote: > Hi, > > Confused newbie here. Can someone please explain the following: > > 1> <<"foo">>. > <<"foo">> > 2> Foo = "foo". > "foo" > 3> <>. > ** exception error: bad argument > > What's the error in the last case? > > Best, > Michel You can use list_to_binary/1 for the same effect: 1> <<"foo">>. <<"foo">> 2> Foo = "foo". "foo" 3> list_to_binary(Foo). <<"foo">> Ivan -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin "hilaritas excessum habere nequit" (Spinoza, Ethica, IV, XLII) ============================================================ From hobson42@REDACTED Wed May 16 11:15:06 2012 From: hobson42@REDACTED (Ian) Date: Wed, 16 May 2012 10:15:06 +0100 Subject: [erlang-questions] String versus variable in binary literal In-Reply-To: References: Message-ID: <4FB3701A.6090702@gmail.com> On 16/05/2012 09:49, Michel Rijnders wrote: > Hi, > > Confused newbie here. Can someone please explain the following: > > 1> <<"foo">>. > <<"foo">> > 2> Foo = "foo". > "foo" > 3> <>. > ** exception error: bad argument > > What's the error in the last case? > > Best, > Michel Hi Michel, As a slightly less confused (I hope) newbie.... <<"foo">> is a binary - and stored as 3 consecutive bytes. "foo" is a list of characters, stored as the character "f" (in a word) and an address (also a word) pointing to the list "oo" stored in a similar manner. Yes, that does take 8 bytes of storage per character on 32 bit systems, and 16 bytes on 64 bit systems. Erlang was designed for telephony, and not text processing! You have to trigger the conversion yourself, using list_to_binary(IoList) -> binary() - see http://www.erlang.org/doc/man/erlang.html Note that the parameter is an IoList - you can give list_to_binary a comma separated list of lists and it will concatenate them for you. This is more efficient than converting things as you go. Regards Ian From erlang@REDACTED Wed May 16 12:29:35 2012 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 16 May 2012 12:29:35 +0200 Subject: [erlang-questions] String versus variable in binary literal In-Reply-To: <4FB36BC1.8020903@gmail.com> References: <4FB36BC1.8020903@gmail.com> Message-ID: On Wed, May 16, 2012 at 10:56 AM, Richard Carlsson wrote: > On 05/16/2012 10:49 AM, Michel Rijnders wrote: >> >> Hi, >> >> Confused newbie here. Can someone please explain the following: >> >> 1> ?<<"foo">>. >> <<"foo">> >> 2> ?Foo = "foo". >> "foo" >> 3> ?<>. >> ** exception error: bad argument >> >> What's the error in the last case? >> >> Best, >> Michel > > > The bit syntax doesn't (currently) support encoding strings that are not > constant literals. This is something that should be fixed, IMO. It's a bug (or should be a bug) - try this for size: > <<1223232321111,3476824682351,18368119>>. <<"Wow">> Isn't that beautiful :-) /Joe /Joe > > ? /Richard > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From kostis@REDACTED Wed May 16 12:44:17 2012 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 16 May 2012 12:44:17 +0200 Subject: [erlang-questions] String versus variable in binary literal In-Reply-To: References: <4FB36BC1.8020903@gmail.com> Message-ID: <4FB38501.6050406@cs.ntua.gr> On 05/16/2012 12:29 PM, Joe Armstrong wrote: > On Wed, May 16, 2012 at 10:56 AM, Richard Carlsson > wrote: >> On 05/16/2012 10:49 AM, Michel Rijnders wrote: >>> >>> Hi, >>> >>> Confused newbie here. Can someone please explain the following: >>> >>> 1> <<"foo">>. >>> <<"foo">> >>> 2> Foo = "foo". >>> "foo" >>> 3> <>. >>> ** exception error: bad argument >>> >>> What's the error in the last case? >> >> >> The bit syntax doesn't (currently) support encoding strings that are not >> constant literals. This is something that should be fixed, IMO. > > It's a bug (or should be a bug) - try this for size: > >> <<1223232321111,3476824682351,18368119>>. > <<"Wow">> > > Isn't that beautiful :-) Beautiful or not, please let's not mix two completely different issues that are here. There is no string constant in Joe's example. There is a binary with three segments. And for better or worse, the default binary segment when one does not specify the type and size of a segment is a *byte* (type=integer, size=8). This is clearly stated in the language reference manual in a very visible green box: When constructing binaries, if the size N of an integer segment is too small to contain the given integer, the most significant bits of the integer will be silently discarded and only the N least significant bits will be put into the binary. See: http://erlang.org/doc/reference_manual/expressions.html#bit_syntax Kostis From mononcqc@REDACTED Wed May 16 13:46:26 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 16 May 2012 07:46:26 -0400 Subject: [erlang-questions] String versus variable in binary literal In-Reply-To: <4FB36CAC.2090505@llaisdy.com> References: <4FB36CAC.2090505@llaisdy.com> Message-ID: <4FB39392.8040703@ferd.ca> When dealing with strings of text, please do not use list_to_binary or binary_to_list, as they only handle lists of bytes (0..255). What you might want instead is unicode:characters_to_binary(Chars, InEncoding, OutEncoding), where Chars can be either: * an iolist (list + binaries of bytes) * a list of codepoints as integers (for Unicode) * a list encoded as latin-1 * binaries encoded as latin-1, utf-8, utf-16 or utf-32 And InEncoding and OutEncoding can be any of: utf8 (also aliased as unicode), utf16, utf32, and latin1. It will generally be safer (especially if you want to deal with Unicode) to use this function instead of list_to_binary, as it should crash whenever a given character is > 255, which might happen quite a lot outside of ASCII. On 12-05-16 5:00 AM, Ivan Uemlianin wrote: > On 16/05/2012 09:49, Michel Rijnders wrote: >> Hi, >> >> Confused newbie here. Can someone please explain the following: >> >> 1> <<"foo">>. >> <<"foo">> >> 2> Foo = "foo". >> "foo" >> 3> <>. >> ** exception error: bad argument >> >> What's the error in the last case? >> >> Best, >> Michel > > You can use list_to_binary/1 for the same effect: > > 1> <<"foo">>. > <<"foo">> > 2> Foo = "foo". > "foo" > 3> list_to_binary(Foo). > <<"foo">> > > Ivan > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Wed May 16 14:01:00 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 16 May 2012 14:01:00 +0200 Subject: [erlang-questions] String versus variable in binary literal In-Reply-To: References: <4FB36BC1.8020903@gmail.com> Message-ID: <4FB396FC.7040904@gmail.com> On 05/16/2012 12:29 PM, Joe Armstrong wrote: > On Wed, May 16, 2012 at 10:56 AM, Richard Carlsson > wrote: >> The bit syntax doesn't (currently) support encoding strings that are not >> constant literals. This is something that should be fixed, IMO. > > It's a bug (or should be a bug) - try this for size: > >> <<1223232321111,3476824682351,18368119>>. > <<"Wow">> > > Isn't that beautiful :-) Well, yes, but nothing strange about it. The default size for integers is byte, and numbers get truncated to fit the desired size: 1> <<1223232321111:8,3476824682351:8,18368119:8>>. <<"Wow">> 2> <<1223232321111:16,3476824682351,18368119>>. <<"?Wow">> 3> <<1223232321111:32,3476824682351:32,18368119:32>>. <<206,83,246,87,130,230,111,111,1,24,70,119>> What tends to surprise people is that the default field type is integer, even if the given value is a constant of some other type, but if you add the correct type specifier it works: 1> << <<1,2>> >>. ** exception error: bad argument 2> << <<1,2>>/binary >>. <<1,2>> 3> << 3.14 >>. ** exception error: bad argument 4> << 3.14/float >>. <<64,9,30,184,81,235,133,31>> in fact, <<"abc">> works just because it's considered to be a special notation for a number of integers: << $a, $b, $c >> = <<"abc">>. But you're not allowed to write it as << [$a,$b,$c] >>, even though [$a,$b,$c] = "abc". Nowadays there is some extra syntax for UTF- encodings: 1> <<"???"/utf8>>. <<"??????">> 2> <<"???"/utf16>>. <<0,229,0,228,0,246>> 3> <<"???"/utf32>>. <<0,0,0,229,0,0,0,228,0,0,0,246>> However, <> doesn't work, if String is a variable. This could be fixed. But since <> is interpreted as expecting String to be an integer, there is still no way to easily insert a normal Latin-1 string dynamically in a binary, even if <> is made to work. I would suggest the addition of type specifiers 'latin1' and 'ascii' for this purpose, where 'latin1' would accept only character codes 0-255 (no truncation), and 'ascii' would only accept codes 0-127 (good for ensuring that http headers and similar things are 7-bit clean). While you're at it, String should be allowed to be a chardata() just as in the unicode module, not just flat lists of chars. Oh, and atoms should be allowed in chardata() and iolist(). I think that's about it. /Richard From clist@REDACTED Wed May 16 15:20:09 2012 From: clist@REDACTED (Angel J. Alvarez Miguel) Date: Wed, 16 May 2012 15:20:09 +0200 Subject: [erlang-questions] erlang float comparison In-Reply-To: <7984E468-2463-49AD-A734-3C0E5032E10D@cs.otago.ac.nz> References: <201205140941.01511.clist@uah.es> <201205151104.40442.clist@uah.es> <7984E468-2463-49AD-A734-3C0E5032E10D@cs.otago.ac.nz> Message-ID: <201205161520.10091.clist@uah.es> Hi While presenting this "case" these days we also have changed some other code so (right now) we are.. ah... unable to reproduce such scenario properly, im rechecking the whole thing (i said we dont have any versioning system?) Also, asking my workmate about this, results also in some vague memories about acos failing on a number around 1.0-e8 still we can detail exactly what the number was.. d'oh So we stated to log all runs after this fiasco in order to honor with real data our (future) claims... a few quick checks confirm that you all are right while by all means i must be wrong... so I must state that there isnt erlang fault nor ieee floats... ...in the end is probably my fault, we messed up something, i agree it is rather dificult to find such values passing the test but failing the acos fun. But after all and being unsure about floating point behaviour (its probably the first time I try to work on such numerical thing after many years of boring sysadmin shell scripting...) that i wrote this on the list.... Sorry for bothering you all, nevertheless, such coverage is one of the things i really appreciate, every time the list machinery starts growling, we (me) learn a lot of pretty interesting things.. /Angel On Mi?rcoles, 16 de Mayo de 2012 00:43:40 usted escribi?: > On 15/05/2012, at 9:04 PM, Angel J. Alvarez Miguel wrote: > > The problem can be stated as: > > > > Result = case Myfloat > 1.0 of > > > > true -> 1.0; > > false MyFloat > > end > > > > ... > > ... > > math:acos(Result). > > > > after that we came across some values that fail to pass the test > 1.0 > > but are seen as greater to the math:acos function rendering a bad > > arithmetic expression error > > That is pretty astonishing, actually. > The obvious question is *what are these values*? > I've just written a test loop searching for such > values and so far haven't found any. > > Is there any difference between emulated and native code? > Why aren't you worried about Myfloat < -1.0? > > By the way, this seems a rather strange way to write > > Result = if Myfloat > 1.0 -> 1.0 ; true -> Myfloat end > > Do you know what trig library your copy of Erlang was linked > with? Here's the relevant code from fdlibm: > > hx = __HI(x); > ix = hx & 0x7fffffff; > if (ix >= 0x3ff00000) { /* |x| >= 1 */ > if (((ix - 0x3ff00000) | __LO(x)) == 0) { /* |x|==1 */ > if (hx > 0) > return 0.0; /* acos(1) = 0 */ > else > return pi + 2.0 * pio2_lo; /* acos(-1)= pi */ > } > return (x - x) / (x - x); /* acos(|x|>1) is NaN */ > } > > There's no way _that_ could claim an IEEE double was greater > than 1.0 if it wasn't. -- - From comptekki@REDACTED Wed May 16 16:11:06 2012 From: comptekki@REDACTED (Wes James) Date: Wed, 16 May 2012 08:11:06 -0600 Subject: [erlang-questions] secure use of cookies in an erlang application In-Reply-To: References: Message-ID: Thanks everyone for your responses! Wes -------------- next part -------------- An HTML attachment was scrubbed... URL: From francesco@REDACTED Wed May 16 16:36:23 2012 From: francesco@REDACTED (Francesco Cesarini) Date: Wed, 16 May 2012 15:36:23 +0100 Subject: [erlang-questions] Help Sponsor Students at the 2012 Erlang User Conference Message-ID: <4FB3BB67.5000301@erlang-solutions.com> Hi All, As you all probably know, the 2012 Erlang User Conference is happening May 29th - 31st: http://www.erlang-factory.com/conference/ErlangUserConference2012 It is the biggest lineup we've ever had in Stockholm, with three tracks over two days and a third day filled with tutorials. As always, students interested in Erlang get to attend free of charge. This is possible thanks to the conference sponsors, whose fees go to cover their admission. Sponsors this year include Klarna, Basho, Campanja, Ericsson, Erlang Solutions, Machine Zone, Quviq, ShoreTel, Sjoland & Thysselius and Tail-f. They are not enough! We have budgeted for 50 student places but are quickly running out. Interest from all over Europe is huge. If you are looking for Erlang developers, interns, thesis students, or want to help grow the Erlang programmers of tomorrow, this is a concrete way for you or your company can help. In exchange for sponsoring, on top of the various levels of exposure, we can do an email shoot out to all of the students on your behalf, delegate bag inserts, giveaways or more. More info on the sponsorship is here: http://www.erlang-factory.com/conference/ErlangUserConference2012/sponsorship_opportunities If you have any questions, let me know. Thanks! Francesco -- Erlang Solutions Ltd. http://www.erlang-solutions.com From kostis@REDACTED Wed May 16 17:32:45 2012 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 16 May 2012 17:32:45 +0200 Subject: [erlang-questions] Help Sponsor Students at the 2012 Erlang User Conference In-Reply-To: <4FB3BB67.5000301@erlang-solutions.com> References: <4FB3BB67.5000301@erlang-solutions.com> Message-ID: <4FB3C89D.6050306@cs.ntua.gr> On 05/16/2012 04:36 PM, Francesco Cesarini wrote: > Hi All, > > As you all probably know, the 2012 Erlang User Conference is happening > May 29th - 31st: > > http://www.erlang-factory.com/conference/ErlangUserConference2012 > > It is the biggest lineup we've ever had in Stockholm, with three tracks > over two days and a third day filled with tutorials. As always, students > interested in Erlang get to attend free of charge. This is possible > thanks to the conference sponsors, whose fees go to cover their > admission. Sponsors this year include Klarna, Basho, Campanja, Ericsson, > Erlang Solutions, Machine Zone, Quviq, ShoreTel, Sjoland & Thysselius > and Tail-f. They are not enough! > > We have budgeted for 50 student places but are quickly running out. > Interest from all over Europe is huge. If you are looking for Erlang > developers, interns, thesis students, or want to help grow the Erlang > programmers of tomorrow, this is a concrete way for you or your company > can help. In exchange for sponsoring, on top of the various levels of > exposure, we can do an email shoot out to all of the students on your > behalf, delegate bag inserts, giveaways or more. More info on the > sponsorship is here: > > http://www.erlang-factory.com/conference/ErlangUserConference2012/sponsorship_opportunities Besides seconding the mail of Francesco that helping students attend the EUC event is a great thing to do, I would like to also solicit speakers for a "birds of a feather" 45 min session on the use of Erlang in education and research, scheduled for the end of Day 1. You do not need to prepare anything fancy. One to three slides describing your experiences, views, plans, opinions, etc. on the subject will do. Send me a private mail if you would like to be part of the presenters. Kostis From olivier.boudeville@REDACTED Wed May 16 17:56:06 2012 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Wed, 16 May 2012 17:56:06 +0200 Subject: [erlang-questions] [ANN] Functional Summer School In-Reply-To: <4FACCF92.9070704@gmail.com> Message-ID: Hello Fabrice, That's definitively a good idea. I propose that we set up a functional lounge (Erlang, Haskell, ... birds of a feather flock together) at the location of the summer school (Ch?teau de Cadarache), on the evening of Monday, June 18 (most other evenings are already filled with seminars and other events). I expect this to be a very cosy lounge, to which hopefully many functional enthousiasts could come! (please send me off-list a short message if you plan to join, so that we can properly dimension the appetizers!) Best regards, Olivier. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 fabrice.nourisson@REDACTED Envoy? par : erlang-questions-bounces@REDACTED 11/05/2012 10:36 A erlang-questions@REDACTED cc Objet Re: [erlang-questions] [ANN] Functional Summer School Hi Olivier, Have you plan to do a Haskell lounge (or FPLounge or Erlounge or ...) during those 2 weeks ? I live not far the course location. And for once that functional programmers are in the corner I would be interested to meet them. Fabrice -- Fabrice Nourisson ExtremeForge On 10/05/2012 11:31, Olivier BOUDEVILLE wrote: Hi all, With our colleagues of CEA and INRIA, we are organizing a summer school focusing on "Functional Programming for Parallel and Concurrent Applications". It is to take place from June 11 to June 22, in south of France. A few seats are still available. The implementation language we selected is Haskell, but it may be nevertheless of interest to the Erlang community, as a main purpose is to emphasize on how the functional approach is to improve the way we manage concurrency challenges. Moreover there will be at least a few Erlang developers in the audience, and we will have the pleasure to listen there to a short talk given by Joe Armstrong himself! All details are available on http://www-hpc.cea.fr/SummerSchools2012-CS.htm Best regards, Olivier. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -- Fabrice Nourisson Twitter : @duckmole_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mastah_killah@REDACTED Wed May 16 18:06:20 2012 From: mastah_killah@REDACTED (mastah_killah@REDACTED) Date: Wed, 16 May 2012 18:06:20 +0200 (CEST) Subject: [erlang-questions] c-node: Link error in Visual C++ Message-ID: An HTML attachment was scrubbed... URL: From dmercer@REDACTED Wed May 16 19:00:46 2012 From: dmercer@REDACTED (David Mercer) Date: Wed, 16 May 2012 12:00:46 -0500 Subject: [erlang-questions] How to debug "Kernel pid terminated" In-Reply-To: <014c01cd32dc$1351bc10$39f53430$@com> References: <014c01cd32dc$1351bc10$39f53430$@com> Message-ID: <019f01cd3385$70e7aa70$52b6ff50$@com> As a follow-up question, since I had a problem again overnight where the failover took over for the main, even though the main was still running: Are Erlang distributed applications not intended to be run on multiple nodes on the same host? Anyone have any success doing this in production? I can get it to work, it just doesn't seem to work long-term. I guess I don't often see any posts on this list about the built-in distributed application functionality of Erlang/OTP. Does anyone actually use it, or am I behind the times and I should be using some sort of custom system developed by the RabbitMQ folks or something? Just wondering, because it makes a really good demo when I show people; it just doesn't seem to be working for me long-term. Cheers, DBM From: David Mercer [mailto:dmercer@REDACTED] Sent: Tuesday, May 15, 2012 3:48 PM To: erlang-questions@REDACTED Subject: How to debug "Kernel pid terminated" I have a distributed application that I run on a couple of nodes. I have had various problems where one node spontaneously decides another node is not available and starts up its own instance of the application, but this one is a first for me: One of my failover nodes exited after printing the following messages: =ERROR REPORT==== 14-May-2012::19:43:24 === ** Generic server dist_ac terminating ** Last message in was {internal_restart_appl,cron} ** When Server state == {state, [{appl,cron, {failover,cron_main@REDACTED}, 5000, [cron_main@REDACTED, {cron_failover@REDACTED,cron_failover@REDACTED}], [{cron_failover@REDACTED,true}]}], [],[], [cron_failover@REDACTED], [cron], [],[],[],[],[]} ** Reason for termination == ** {{case_clause, {'EXIT', {timeout, {gen_server,call, [application_controller,which_applications]}}}}, [{dist_ac,restart_appl,2,[{file,"dist_ac.erl"},{line,952}]}, {dist_ac,handle_info,2,[{file,"dist_ac.erl"},{line,697}]}, {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,597}]}, {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]} =ERROR REPORT==== 14-May-2012::19:43:24 === server: clickon_backup_server error: enoent path: <<"\\\\ftp-corp2\\SFTP-MW\\70350\\Upload\\837 ">> {error_logger,{{2012,5,14},{19,43,25}},std_info,[{application,kernel},{exite d,shutdown},{type,permanent}]} {"Kernel pid terminated",application_controller,"{application_terminated,kernel,shutdown} "} Kernel pid terminated (application_controller) ({application_terminated,kernel,shutdown}) Abnormal termination I am guessing this node (cron_failover@REDACTED) somehow lost contact with the main node (cron_main@REDACTED) on the same host. I am not sure, however, why this would cause the whole Erlang node to crash. How would I go about debugging this? (1) What circumstances caused this node to lose contact with the other node on the same host? (2) What can I do to gracefully handle this situation? Here's my thought process so far, which doesn't really answer any of my questions: 1. The error message seems to point me to the case statement on line 952 of dist_ac.erl (restart_appl/2). This is a call to start_appl/3, which expects either {ok, _, _} or {error, _}, but not {'EXIT', .}, which is what it received. 2. Looking at start_appl/3, I doubt it is the keysearch which is throwing the EXIT, so I'm going to assume that it is the call to start_distributed/6. 3. I can continue down this rabbit hole, but I'm not sure how it will answer either of my questions. Can someone who perhaps knows the workings of distributed applications better than I please give me a few pointers? Please advise. Thank-you. David Mercer -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbothma@REDACTED Wed May 16 20:39:37 2012 From: jbothma@REDACTED (JD Bothma) Date: Wed, 16 May 2012 20:39:37 +0200 Subject: [erlang-questions] How to debug "Kernel pid terminated" In-Reply-To: <019f01cd3385$70e7aa70$52b6ff50$@com> References: <014c01cd32dc$1351bc10$39f53430$@com> <019f01cd3385$70e7aa70$52b6ff50$@com> Message-ID: Hi I don't really have professional experience with Erlang yet, but for a university project we looked into many levels of redundancy and fault tolerance with Erlang. I also felt that too little of the intentions of fault tolerance tools were explicitly documented, but the feeling I got was that there are many cases where one might want to run two nodes on one machine, when that isn't really useful. These are all my opinion and they're not battle-proven, but I've given this a bit of thought on a big project :) The idea I have is that, for redundancy, you can run two nodes on different machines, and perhaps failover a distributed application that way. Two nodes on the same machine only helps if the node crashes. That can happen, but I think that's a more serious issue that should be found and fixed theoretically rather than trying to do failover. A second node on the machine might be affected anyway, since the only VM crash I've seen has been when it ran out of memory - the other nodes on the machine didn't have memory either. We ended up having to do some explicit garbage collection, although the mistake was using a process pool rather than spawning a process for each job. Instead the processes in the application should be supervised properly. Since your supervisors should only have one task - supervision - it should be very easy to ensure they are "perfect". If they crash, the supervisor above them can restart the subtree, etc. The top supervisor should never need to crash since its job should be the most simple - as long as the top supervisor is there, the application stays up and functional units will be restarted if they crash. As long as you don't crash the VM, the VM and the application should stay up. If the application _does_ crash, you can set the release to shutdown when a critical application goes down, and the next layer can kick in: the hot standby VM on another machine, or a restarted VM using heartbeat, or perhaps your hot standby on the same machine. If you have a hot standby machine+vm+distributed application, that can then kick in. As to the nodes getting confused about whether the other is online - I have no idea. I guess an intermittent connection is a big problem for a distributed application, but I think we ended up not using distributed applications - I have no experience of them. Good luck! JD On 16 May 2012 19:00, David Mercer wrote: > As a follow-up question, since I had a problem again overnight where the > failover took over for the main, even though the main was still running: Are > Erlang distributed applications not intended to be run on multiple nodes on > the same host? > > > > Anyone have any success doing this in production?? I can get it to work, it > just doesn?t seem to work long-term. > > > > I guess I don?t often see any posts on this list about the built-in > distributed application functionality of Erlang/OTP.? Does anyone actually > use it, or am I behind the times and I should be using some sort of custom > system developed by the RabbitMQ folks or something?? Just wondering, > because it makes a really good demo when I show people; it just doesn?t seem > to be working for me long-term. > > > > Cheers, > > > DBM > > > > From: David Mercer [mailto:dmercer@REDACTED] > Sent: Tuesday, May 15, 2012 3:48 PM > To: erlang-questions@REDACTED > Subject: How to debug "Kernel pid terminated" > > > > I have a distributed application that I run on a couple of nodes.? I have > had various problems where one node spontaneously decides another node is > not available and starts up its own instance of the application, but this > one is a first for me: One of my failover nodes exited after printing the > following messages: > > > > =ERROR REPORT==== 14-May-2012::19:43:24 === > > ** Generic server dist_ac terminating > > ** Last message in was {internal_restart_appl,cron} > > ** When Server state == {state, > > ??????????????????????????? [{appl,cron, > > ???????????????????????????????? {failover,cron_main@REDACTED}, > > ???????????????????????????????? 5000, > > ???????????????????????????????? [cron_main@REDACTED, > > > {cron_failover@REDACTED,cron_failover@REDACTED}], > > ???????????????????????????????? [{cron_failover@REDACTED,true}]}], > > ??????????????????????????? [],[], > > ?????????????????????? ?????[cron_failover@REDACTED], > > ??????????????????????????? [cron], > > ??????????????????????????? [],[],[],[],[]} > > ** Reason for termination == > > ** {{case_clause, > > ??????? {'EXIT', > > ??????????? {timeout, > > ??????????????? {gen_server,call, > > ??????????????????? [application_controller,which_applications]}}}}, > > ??? [{dist_ac,restart_appl,2,[{file,"dist_ac.erl"},{line,952}]}, > > ???? {dist_ac,handle_info,2,[{file,"dist_ac.erl"},{line,697}]}, > > ???? {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,597}]}, > > ???? {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]} > > > > =ERROR REPORT==== 14-May-2012::19:43:24 === > > ??? server: clickon_backup_server > > ??? error: enoent > > ??? path: <<"\\\\ftp-corp2\\SFTP-MW\\70350\\Upload\\837">> > > {error_logger,{{2012,5,14},{19,43,25}},std_info,[{application,kernel},{exited,shutdown},{type,permanent}]} > > {"Kernel pid > terminated",application_controller,"{application_terminated,kernel,shutdown}"} > > Kernel pid terminated (application_controller) > ({application_terminated,kernel,shutdown}) > > > > > > Abnormal termination > > > > I am guessing this node (cron_failover@REDACTED) somehow lost contact with the > main node (cron_main@REDACTED) on the same host.? I am not sure, however, why > this would cause the whole Erlang node to crash.? How would I go about > debugging this?? (1) What circumstances caused this node to lose contact > with the other node on the same host?? (2) What can I do to gracefully > handle this situation? > > > > Here?s my thought process so far, which doesn?t really answer any of my > questions: > > > > 1.?????? The error message seems to point me to the case statement on line > 952 of dist_ac.erl (restart_appl/2).? This is a call to start_appl/3, which > expects either {ok, _, _} or {error, _}, but not {'EXIT', ?}, which is what > it received. > > > > 2.?????? Looking at start_appl/3, I doubt it is the keysearch which is > throwing the EXIT, so I?m going to assume that it is the call to > start_distributed/6. > > > > 3.?????? I can continue down this rabbit hole, but I?m not sure how it will > answer either of my questions. > > > > Can someone who perhaps knows the workings of distributed applications > better than I please give me a few pointers?? Please advise.? Thank-you. > > > > David Mercer > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From tuncer.ayaz@REDACTED Wed May 16 21:02:55 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Wed, 16 May 2012 21:02:55 +0200 Subject: [erlang-questions] windows In-Reply-To: <3582453A-BC00-47D5-B9AD-2E5AED675033@rogvall.se> References: <111ECCC1-8CC3-4C79-A331-28161A5FFFC2@rogvall.se> <3582453A-BC00-47D5-B9AD-2E5AED675033@rogvall.se> Message-ID: On Mon, May 14, 2012 at 11:04 PM, Tony Rogvall wrote: > > On 14 maj 2012, at 22:30, Tuncer Ayaz wrote: > > On Sun, May 13, 2012 at 10:44 PM, Tony Rogvall wrote: > > The next thing was that I specified: > > > ????{"(win32)","priv/dthread_drv.dll", > > ?????["c_src/dthread.c", "c_src/dthread_drv.c"]}, > > > You don't need that win32 rule. > rebar automatically switches the extension on Windows as follows: > 1. priv/foo_drv.so -> priv/foo_drv.dll > 2. priv/foo -> priv/foo.exe > > Yes I noticed ;-) > > This leads to the (bug?) that the spec ended up as {spec, drv, > > ".dll", ...} the file was linked as .dll, no wonder I could not find > > it :-) > > > Thanks for the bug report. Can you confirm it's fixed in the ta-dll > branch? > > > Next time I start windows. Unfortunately this week. Merged the fix to master. From dmercer@REDACTED Wed May 16 21:43:06 2012 From: dmercer@REDACTED (David Mercer) Date: Wed, 16 May 2012 14:43:06 -0500 Subject: [erlang-questions] How to debug "Kernel pid terminated" In-Reply-To: References: <014c01cd32dc$1351bc10$39f53430$@com> <019f01cd3385$70e7aa70$52b6ff50$@com> Message-ID: <01bb01cd339c$1f0098e0$5d01caa0$@com> On May 16, JD Bothma wrote: > Two nodes on the same machine only helps if the node > crashes. That can happen, but I think that's a more serious issue that > should be found and fixed theoretically rather than trying to do > failover. As we saw here, one node crashed, the other didn't. I am trying to figure out how to diagnose this crash. My use case for multiple nodes on the same host is to assist in upgrades. Rather than planning an upgrade release, it's much easier just to crash the node and restart it. However, when we crash it, we would like to have the failover take over. In this case, it's OK for the failover to be on the same host. We do also have a third failover on a different host in case the main host goes down. Over time, a few low-priority jobs have also been added to the failover's role, so we can't just turn it off. We could migrate them over to the main and then shut off the failover for good, but my question now is whether anyone sees any reason why application distribution shouldn't work in this case. Thanks. Cheers, DBM From ok@REDACTED Thu May 17 00:12:37 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 17 May 2012 10:12:37 +1200 Subject: [erlang-questions] String versus variable in binary literal In-Reply-To: <4FB3701A.6090702@gmail.com> References: <4FB3701A.6090702@gmail.com> Message-ID: <318BAC76-C73F-4E2A-9534-743727F232AC@cs.otago.ac.nz> The real answer is that <<"foo">> is ad hoc special purpose syntax; there is (as yet) no provision for building a binary using any old string, only for string *literals*. This hack went in basically to provide a more-or-less convenient way to write string-as-binary literals. Yes, it's confusing, and yes, it would be nice if any list-of-character-code expressions were allowed. From ok@REDACTED Thu May 17 00:52:17 2012 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 17 May 2012 10:52:17 +1200 Subject: [erlang-questions] erlang float comparison In-Reply-To: <201205161520.10091.clist@uah.es> References: <201205140941.01511.clist@uah.es> <201205151104.40442.clist@uah.es> <7984E468-2463-49AD-A734-3C0E5032E10D@cs.otago.ac.nz> <201205161520.10091.clist@uah.es> Message-ID: <09BC5770-BA89-4EBC-A25D-1A6FBC9F6B21@cs.otago.ac.nz> On 17/05/2012, at 1:20 AM, Angel J. Alvarez Miguel wrote: > Also, asking my workmate about this, results also in some vague memories about acos > failing on a number around 1.0-e8 still we can detail exactly what the number was.. d'oh This should be a very easy case for acos. If epsilon is a small number, then acos(epsilon) approx= pi/2 - epsilon. On the other hand, since pi is not machine-representable, this might well *look* like a hard case. One of the features I love about Ada is that the trig functions have an optional argument saying how much a full circle is, so that you can express angles like pi/2 precisely as "0.25 of a full circle. Solaris has a full set of {sin,cos,tan,asin,acos,atan,atan2,sincos}pi() functions with an implicit multiplication or division as appropriate by an infinitely precise pi, which is nearly as good. When it comes to inverse trig functions, I am leery of anything but two-argument arc-tangent. I beg you to examine every use of math:acos/1, math:asin/1, and math:atan/1 and very carefully verify that it is OK in that particular case to get only half the circle back and that the half you get is in fact the half you want and to DOCUMENT in the code what you expect and to test that you were right. I think it's fair to say that floating point is not only weirder than we imagine, it's weirder than most of us *can* imagine. From steven.charles.davis@REDACTED Thu May 17 02:43:40 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 16 May 2012 19:43:40 -0500 Subject: [erlang-questions] secure use of cookies in an erlang application Message-ID: <6DC11414-81D5-45B6-B55E-99E0D4E35AAF@gmail.com> This is also quite interesting on the topic. http://www.cse.msu.edu/~alexliu/publications/Cookie/cookie.pdf From steven.charles.davis@REDACTED Thu May 17 03:02:52 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 16 May 2012 20:02:52 -0500 Subject: [erlang-questions] String versus variable in binary literal Message-ID: <6CF183D2-19FE-4567-8F58-09A6502278A8@gmail.com> Confusing, sure. But not impenetrable... How is "foo" more typed than <<"foo">>? Meaning: [ 102,111,111 ] <<102,111,111>> Isn't the problem really the idea of a "string"? When and how - without proper definition of the context - are you supposed to know when a sequence of integers (in either list or binary format) is actually text, and indeed what particular flavor of text? The confusion is magnified by the fact that the implicit context assumption of a participant decoder implementation hides the underlying assumption of the encoder. /s From steven.charles.davis@REDACTED Thu May 17 03:13:42 2012 From: steven.charles.davis@REDACTED (Steve Davis) Date: Wed, 16 May 2012 20:13:42 -0500 Subject: [erlang-questions] String versus variable in binary literal In-Reply-To: <6CF183D2-19FE-4567-8F58-09A6502278A8@gmail.com> References: <6CF183D2-19FE-4567-8F58-09A6502278A8@gmail.com> Message-ID: <561EF695-CF35-4067-B883-329B111EC3EA@gmail.com> However, should you jump on this idea thinking that "therefore erlang needs stronger typing" then IMHO you have lost sight of the underlying issue of the binary transmission of text; and I would propose that Joe's UBF tagging has a far more interesting remediation! /s On May 16, 2012, at 8:02 PM, Steve Davis wrote: > Confusing, sure. But not impenetrable... > > How is "foo" more typed than <<"foo">>? > > Meaning: > [ 102,111,111 ] > <<102,111,111>> > > Isn't the problem really the idea of a "string"? > > When and how - without proper definition of the context - are you supposed to know when a sequence of integers (in either list or binary format) is actually text, and indeed what particular flavor of text? > > The confusion is magnified by the fact that the implicit context assumption of a participant decoder implementation hides the underlying assumption of the encoder. > > /s From ulf@REDACTED Thu May 17 12:42:44 2012 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 17 May 2012 12:42:44 +0200 Subject: [erlang-questions] String versus variable in binary literal In-Reply-To: <318BAC76-C73F-4E2A-9534-743727F232AC@cs.otago.ac.nz> References: <4FB3701A.6090702@gmail.com> <318BAC76-C73F-4E2A-9534-743727F232AC@cs.otago.ac.nz> Message-ID: <0FC27B60-93E4-40F4-92BC-C0AE33DE537E@feuerlabs.com> If we're discussing ways to extend the bit syntax, here is one gripe of mine: We all try to use binaries as much as possible in ets tables, not least since there is a huge difference in space requirements on 64-bit between binaries and lists. But lists have one great advantage when it comes to match specifications and the select() function: You can do a prefix match on the key, e.g. ets:select(Tab, [{ {"foo" ++ '_', '_'}, [], ['$_']} }]) ?and on an ordered_set table, this will be executed efficiently, with a cost that is roughly proportional to the number of matching elements. I was going to write that the only way to do something similar with binary keys is to bind the key to a variable (forcing a full table scan) and then doing the test in a guard, but (realizing that I'd never actually done that and consulting the docs), it seems to be even worse: you can't do it at all! (Unless I missed something, or the docs fail to describe some relevant guard function). In order to avoid scanning the whole table, you instead have to resort to next-ing your way through the table. This has roughly similar complexity to the select() above, but is slower, and requires more code. Something like: pfx_match(T, P) when is_binary(P) -> lists:concat([ets:lookup(T, P) | pfx_match_(ets:next(T, P), T, P)]). pfx_match_(K, T, P) when is_binary(K) -> Sz = byte_size(P), case K of <> -> [ets:lookup(T, K) | pfx_match_(ets:next(T, K), T, P)]; _ -> [] end; pfx_match_(_, _, _) -> []. So basically, we're talking about two extensions: (1) allowing something similar to "foo" ++ '_' in bit syntax, which is probably a very bad idea, and/or (2) extending the match specifications, possibly with even more bizzare syntax, to at the very least allow som basic inspection of binaries. Obviously, if the condition can be expressed in a match_spec guard, the select operation could figure out that the first part of the key is bound. I see two problems with this: - No one, not even Patrik Nyblom who wrote it, seems to want to mess with the match spec compiler and evaluator - There is obviously a tradeoff between performance and expressivity, since the compilation is done on the fly. All things considered, I guess this is quite a challenge. :) BR, Ulf W On 17 May 2012, at 00:12, Richard O'Keefe wrote: > The real answer is that <<"foo">> is ad hoc special purpose syntax; > there is (as yet) no provision for building a binary using any old > string, only for string *literals*. > > This hack went in basically to provide a more-or-less convenient > way to write string-as-binary literals. > > Yes, it's confusing, and yes, it would be nice if any > list-of-character-code expressions were allowed. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From dafgar@REDACTED Thu May 17 17:23:01 2012 From: dafgar@REDACTED (Sergei Fomin) Date: Thu, 17 May 2012 19:23:01 +0400 Subject: [erlang-questions] inet:peername behaviour Message-ID: <4FB517D5.7060604@yandex-team.ru> Hello. I've come to the same problem as described in "Peername of a closed TCP socket" (http://erlang.2086793.n4.nabble.com/Peername-of-a-closed-TCP-socket-td2114838.html). There is a gen_tcp socket which I need some statistics from. Therefore exit_on_close socket option is turned off. When the other end closes connection the right way, both inet:getstat and inet:peername return usable information. But in case RST is received, inet:getstat still shows statistics while inet:peername produces {error,enotconn}. That seems not quite logical to me. -- Sergei Fomin From avinash@REDACTED Thu May 17 17:30:14 2012 From: avinash@REDACTED (Avinash Dhumane) Date: Thu, 17 May 2012 21:00:14 +0530 Subject: [erlang-questions] Naming the function Message-ID: I am new to Erlang world, and after arriving here I realised that I was in imperative world of programming. So, a basic question: how should the name of my function sound? Should the name sound as if I am declaring a "value" or an "action"? If latter (i.e. action) then would it not be imperative? I tend to see "programming a system" as declaring the causal and structural relations among the values (i.e. forms) in the system; so, I am beginning to have trouble with naming functions imperatively, yet at the same time Erlang-library functions are named imperatively. So, I really need help now. Thanks Avinash From shahrdad1@REDACTED Thu May 17 18:19:42 2012 From: shahrdad1@REDACTED (Shahrdad Shadab) Date: Thu, 17 May 2012 12:19:42 -0400 Subject: [erlang-questions] implementing Forex Trader in Erlang Message-ID: Hi Erlangers I have to implement a foreign exchange trader software for a financial firm. They need fast response and high availability. They also need parallel real time feed from multiple brokers. As I did my preliminary investigations,It seems most of the tools on the field are implemented in Microsoft technologies like .net or com/OLE. I believe (although I might be wrong) the broker servers are DDE servers. I decided to use Erlang/otp. Does anyone know if something like this has done before in Erlang. Also I appreciate any guidance on how to get feed from trader servers. Thanks a lot Best regards Shahrdad -- Software Architect & Computer Scientist -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Fri May 18 00:04:05 2012 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 17 May 2012 15:04:05 -0700 Subject: [erlang-questions] implementing Forex Trader in Erlang In-Reply-To: References: Message-ID: <4FB575D5.2020702@gmail.com> Here is the architecture of something similar: http://www.connamara.com/solutions-hftframework.html On 05/17/2012 09:19 AM, Shahrdad Shadab wrote: > Hi Erlangers > > I have to implement a foreign exchange trader software for a financial firm. > They need fast response and high availability. They also need parallel real time feed from multiple brokers. > As I did my preliminary investigations,It seems most of the tools on the field are implemented in Microsoft technologies like .net or com/OLE. > I believe (although I might be wrong) the broker servers are DDE servers. > I decided to use Erlang/otp. Does anyone know if something like this has done before in Erlang. > Also I appreciate any guidance on how to get feed from trader servers. > > Thanks a lot > Best regards > Shahrdad > > > -- > Software Architect & Computer Scientist > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Fri May 18 02:14:48 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Thu, 17 May 2012 17:14:48 -0700 Subject: [erlang-questions] gproc and gen_event Message-ID: Hello, I am trying to set up a pub-sub type situation where I have a function called which does some stuff and at the end broadcasts a message. I would like for multiple handlers to be able to receive that message and do something with it. I'm currently using gen_event, but have been reading about gproc as well. Can anyone explain why they'd use one over the other for pub/sub? Thanks, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From thpr@REDACTED Fri May 18 02:21:45 2012 From: thpr@REDACTED (Tom Parker) Date: Thu, 17 May 2012 17:21:45 -0700 (PDT) Subject: [erlang-questions] Question/Alternative on Frames Proposal [Warning: Long] In-Reply-To: <4FB575D5.2020702@gmail.com> References: <4FB575D5.2020702@gmail.com> Message-ID: <1337300505.68472.YahooMailNeo@web130106.mail.mud.yahoo.com> Richard, Impressive proposal on Frames, in the explained issues, benefits, and especially the background for the choices.? I've read through it and I'm in agreement that Erlang should plug the issue with updates to records and being able to pass formatted name-keyed information between systems.? Having said that... and I'm certainly not trying to paint any bike sheds here (nor have I read pre-2012 conversations)... but it leaves me wondering one thing... but first some background on the question/alternative. (Bear with me for a moment, while I diverge into some Java analogies).? The Java folks have been very careful recently to distinguish two things: The Java language and the Java platform.? We have a similar situation here, and if you consider the platform composed of the OTP and Virtual Machine (BEAM for example) as two separate entities (like Java has the JDK and VM), we should think about the impact to all 3 of those items in a proposal.? Another thought: Java made huge distortions around injecting types onto lists (using generics) and having to make the compromise that those types never make it into the class files.? This was done - perhaps among other reasons - to maintain compatibility from old code to the large quantity of class libraries available for Java.? I'm not saying we should compromise on capabilities - especially reliability (not sure that is necessary), but we should consider the impact to the libraries (and cost to rewrite) on any changes. Where I think we would vehemently agree: I expect Erlang to be robust.? That's why I'm even here.? An issue with records where you can compile two files (using the same .hrl) and end up with a result that (a) compiles, (b) doesn't produce an error and (c) produces the wrong answer... is a serious issue.? It distinctly shows the shortcomings of records.?? However, in the 3 pillars (Language, Libraries, VM), while all need enhancement, this is primarily a limitation *of BEAM format and the virtual machine* (because this problem would occur EVEN IF we compiled another language to BEAM and used tuples).? As you've noted, the VM needs to comprehend the mapping from keys (Atoms) to values.? I believe you have defined the VM changes in a way that is robust and requires little further discussion.? That leaves how to communicate that information to the VM.? You've defined some BEAM file changes (new commands, and if I'm reading correctly, using the Literal Table to store frames), which doesn't have a major impact on BEAM file structure (but mainly relies on frame_to_binary, etc.).? I believe that works as defined as well. So the next question then becomes what of the Erlang language.? Your proposal covers a new syntax that uses <{ ... }> to identify a frame.? You end up using ~ since that is a character unused in other parts of Erlang, so that you have a new operator to avoid ambiguity. But here is where I get stuck.? Why not use the record format?? ... (Bear with me, I believe the issues in that reuse can be addressed.) Issue #1 with the ~ syntax) Today's Erlang developer, if using records, has already provided a form of that information.? It already has a key->value mapping.? One could easily assert the preprocessor was forced to destroy information for lack of an ability for the compiler and BEAM format to handle the information.? BUT, we know that the record format is NOT ambiguous in ability to parse.? (It is a bit verbose, however).? We cannot force records to be formal as a native format (since backwards compatibility forces tuple comparisons to pass), but that doesn't mean we can't leverage the syntax... Issue #2 with the ~ syntax) The format proposed for extraction from a frame, with a tilde, is ~name(Frame).? (okay, a little bike shed paint :) ) This reverses the standard ordering of items from Major to minor.? For functions, we have module:function/arity, and module is more important than function.? Same goes for the arguments (less important than the function name).? The ~field fun with a target of the frame reverses that relationship, ~minor(Major), which may be uncomfortable, and there is an alternative (use the existing record format).? We are all already familiar with the record format being a key:value mapping, and that's what a frame would look like: P = #point{x=4,y=5}. or as part of the frame proposal, if you prefer the anonymous definition vs. a declared format, we can use (from erlson): P = #{x = 4, y = 5}. The big question: Q: How do I know this #point is a frame and not a record? A: Well, we have 2 ways of knowing: (1) A language requirement would be imposed that #{ is always an "anonymous frame" since it is not a valid record. (2) The namespace defined for -record(ATOM, {...}) and -frame(ATOM,{...}) would be shared (compiler - well, technically the preprocessor - would halt with an error if a record and a frame with the same ATOM identifier were included in the module to be compiled.)? Since the preprocessor destroys records, the only #atom{ the compiler would ever see would be frames - it is never ambiguous to the compiler. Comments on your questions from section 4 of your document in comparing solutions: Q1: Frames answer changes to Yes, this actually forces an increase in intelligence of the preprocessor. Q4: This would alleviate the impact on Dialyzer as it already understands record formats (should make it a less challenging change) Q9: If the frame is a declared frame, then the frames answer is enhanced to be able to catch this issue at compile time, otherwise as before Q10: If the frame is a declared frame, then the frames answer is enhanced to be able to catch this issue at compile time, otherwise as before Q11. Frames now take on the unnecessary recompile situation of records (if the record is a declared frame) Q20: I believe this would change to make detection easier, but am not totally sure. I believe all of the other answers stay the same since this is not changing the BEAM or VM structure of your proposal. Additional Q&A: Q: So this forces declaration of a frame if I want an atom between # and { A: That is my thought.? It would look like: -frame(point, {x, y}). If a frame is to be used throughout a file, the declaration helps to avoid capitalization and other typo errors in the name of the frame.? This DOES give us the "overhead" (and the benefits of forcing a recordkeeping and naming) of .hrl files IF we are using defined frames, but since the frame information is passed into the VM, we do not inherit the "compiled at different times so I used the preprocessor to trick myself" problems of records.? This also provides the language (the libraries, really) the ability to publish standard definitions of frames that could be reliably re-used.? Without that publication, errors would be found at runtime.? With publication, some can be found at compile time.? Consider: get_area(#rectangle{x1 = 4, y1 = 4, x2 = 6, y2 = 8}) ...could be caught at compile if there was (in an hrl) -frame(rectangle, {x, y, w, h}). The ~frame proposal does not have a declaration and would catch the error at run time. Q: Can this use defaults? A: Sure.? Unlike the <{}> syntax which has no ability to declare, this form allows (but does not require) a declaration.? This would mean providing defaults, which could get exponentially large in the number of required functions, would not be required.? The preprocessor could continue to handle that capability of inserting the defaults into the correct locations before the compiler has to take over. Q: How does extraction change? A: Extraction with the record-style format ... and is major to minor: Frame#format.member Q: What about the erlson ambiguity with "."? A: This does not have that issue.? Since it is using a more verbose syntax, it is not "x.y" and "x.y".? The "x.y" atom is "x.y" and the "Y member of Frame X" would be: x#point.y Q: Should this change the stance on extension vs. update? A: It could, and perhaps only for defined frames.? Since the definition at the top of the file would exist it would decrease chance of typographical errors (but admittedly not eliminate them across files).? An alternative here that would require one additional byte of storage (or another type of frame perhaps) would be to have a "strict" definition on a frame.? So the definition might be: -frame(point, {x, y}, [strict]) This would prohibit any frame constructed in that module from being extended (runtime check).? Not sure if that is valuable, but should be considered. Another possibility is to have a file-by-file setting: -compile(strict_frames)???? (The option to default to strict and have a allow_extend_frames is also possible) My concern with losing the ability to extend has to do with extending the use of frames.? Consider a #rectangle{x, y, w, h}.? If it was extendable, we could add "color" and continue to hand the #rectangle frame into a "get_area" function.? Otherwise, we have to decompose either another frame or a tuple to extract the rectangle field in order to pass the rectangle into the get_area function.? Additionally, if we have a transform(x, y, #rectangle) function, an extendable frame would allow just 2 field updates in a constructed frame, while the non-extended version would require the result of the transform be repackaged into a new frame or tuple (increases code size & execution at the expense of the security of extension vs. update).? With declarations and compiler flags, it seems we can get the best of both worlds.? (Although you refer to some optimizations from not allowing updates that I don't think I quite followed) NOTE: Extension might have a side effect of requiring additional virtual machine instructions.? Richard's proposal only allows the "create_frame" to define the tuple of allowable Atoms, while any extension would require a "merge atom list" function that could cause additional VM instructions? Q: What happens if I have: -frame(point, {x, y}) in one file and -frame(point, {x, y, z}) in another? A: The first module will happily accept frames from the second file.? This is because Richard's definition that the pattern matching is loose and I agree with that logic.? The second may or may not fail depending on whether z is ever dereferenced.? If it is, usage would fail at runtime just as with the ~field(frame) syntax or pattern matches with ~. Q: What happens if I have: -frame(chromosome, {x, y}) in one file and -frame(chromosome, {w, z}) in another file? A: It depends on the answer to extension vs. update.? If extension is prohibited, then either (a) Dialyzer should catch this for you or (b) Attempts to modify or pattern match the frame from the other module will fail at runtime just like it would in Richard's ~frame proposal. If extension is allowed, then the extension in a given module is based on the definition IN THAT MODULE.? So things would happily go along unless you dereference a key that does not exist (which in this example is likely, but in an {a, b, c}, {b, c, d} scenario might be avoided. Q: This breaks erlson A: Yes.? I believe that is an acceptable side effect given the benefits. This provides one more additional advantage: The library, aka the rest of OTP The tilde frame proposal basically causes a rewrite of a section of OTP (I'm unsure of the total side effect size) to use frames.? Every function might need a change if it were to use frames instead of records/tuples.? In the case of using the record syntax, all that would be required to change is the -record changed to -frame (unless someone used the tuple form in a match or construction somewhere! - Dialyzer should probably be taught to catch that if it does not already) Even better, a few compiler options could even help out: 1) -record_as_frame would have the system automatically assume -record should be interpreted a -frame.? Unsafe? perhaps, but it provides a quick way of testing for compatibility with frames vs. records in a non-destructive way. 2) -fatrecords.? Think of this as the same as the compatibility that Apple provided in their transitions between architectures.? This could automatically produce extra functions that would emulate BOTH frames and records.? For example: -record(rectangle, {x, y, w, h}). get_area(#rectangle{w = W, h = H}) -> ?? W * H. would pass through the preprocessor (when provided -fatrecords) and become: -frame(rectangle, {x, y, w, h}). % From the -record use, as collapsed by the pre-processor get_area({rectangle, _, _, W, H}) -> ?? W * H; % from the -frame use get_area(#rectangle{w = W, h = H}) -> ?? W * H. This provides - at the expense of a bulkier BEAM file and a bit more pattern matching at runtime - the ability to do "fatrecords" in a library (like OTP) and gradually move code that leverages those libraries from records to frames, not forcing a full all-at-once rewrite.? This provides a smooth transition for legacy code, while providing benefits of the security of frames to those that would like it carried through the standard (and other) libraries. Thoughts? TP. -- Tom Parker thpr@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From straightflush@REDACTED Fri May 18 04:25:41 2012 From: straightflush@REDACTED (AD) Date: Thu, 17 May 2012 22:25:41 -0400 Subject: [erlang-questions] gproc and gen_event In-Reply-To: References: Message-ID: Andrew, I posted about something similar on my blog http://blog.dberg.org/2012/04/using-gproc-and-cowboy-to-pass-messages.html. I cant speak to gen_event personally, I havent used it that much, but I think its more of a use case for logging errors and such centrally versus a messaging system but I could be off here. -A On Thu, May 17, 2012 at 8:14 PM, Andrew Berman wrote: > Hello, > > I am trying to set up a pub-sub type situation where I have a function > called which does some stuff and at the end broadcasts a message. I would > like for multiple handlers to be able to receive that message and do > something with it. I'm currently using gen_event, but have been reading > about gproc as well. Can anyone explain why they'd use one over the other > for pub/sub? > > Thanks, > > Andrew > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From richardprideauxevans@REDACTED Fri May 18 04:36:05 2012 From: richardprideauxevans@REDACTED (Richard Evans) Date: Fri, 18 May 2012 03:36:05 +0100 Subject: [erlang-questions] Deterministic playback of a simulation? Message-ID: I have a multiplayer simulation game running in erlang, involving multiple characters making decisions etc. Each game instance uses a few (currently, 4) erlang processes running concurrently, communicating with an AI simulator written in C. Everything is working well and I am enjoying using erlang. But there is one thing I am finding tricky. For QA purposes, I want to be able to record the exact sequence of function-calls, so I can play the game back exactly, deterministically. My current approach is this: I have a global ets table storing a list of function-calls. Then, whenever I was calling a function I want to record - instead of calling the function directly - I call a procedure which adds the function to the list of function-calls, and then calls it: call_and_store(Fun, Args) -> add_to_script(Fun, Args); apply(lobby, Fun, Args). add_to_script(Fun, Args) -> Info = ets:info(script), {size, Size} = lists:keyfind(size, 1, Info), io:format("Inserting {~p,~p,~p}~n", [Size,Fun,Args]), ets:insert(script, {Size, Fun, Args}). This approach does not seem to work well because of erlang's pre-emptive scheduling. If two concurrent processes both invoke call_and_store, then it is possible (and seems to actually be happening) that erlang's pre-emptive scheduler may stop processing one instance of call_and_store between execution of add_to_script and execution of apply. If this happens, the order of execution and the order of recorded function-calls will diverge, we don't have an accurate recording of the set of function-calls, and we won't be able to deterministically playback. So my question is: is there a way to prevent the scheduler from yielding during a block of code? Some way to insist the block is called as one unit, like this: call_and_store(Fun, Args) -> !!!prevent_yielding, add_to_script(Fun, Args); apply(lobby, Fun, Args), !!!allow_yielding. My guess is: no. This does not seem very erlangy. Alternatively, is there a different approach which can produce the desired list of function-calls? My current approach seems to go against the erlang grain, involving writing to a shared global table. Is there a better, more erlangy, way of doing this? thanks, ~Richard From ok@REDACTED Fri May 18 05:49:31 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 18 May 2012 15:49:31 +1200 Subject: [erlang-questions] Naming the function In-Reply-To: References: Message-ID: On 18/05/2012, at 3:30 AM, Avinash Dhumane wrote: > I am new to Erlang world, and after arriving here I realised that I was in imperative world of programming. > > So, a basic question: how should the name of my function sound? > > Should the name sound as if I am declaring a "value" or an "action"? If latter (i.e. action) then would it not be imperative? Surely the rules for Erlang are the same as those for any programming language, as spelled out by Ledgard and Tauer? - a subprogram which tests whether something is true or false should be named using an adjective, adjectival phrase, "is_...", "has_...", and so on - a subprogram which computes a pure function should be named using a noun or noun phrase - a subprogram which is executed for its effects should be named using an imperative verb phrase > I tend to see "programming a system" as declaring the causal and structural relations among the values (i.e. forms) in the system; so, I am beginning to have trouble with naming functions imperatively, yet at the same time Erlang-library functions are named imperatively. I don't believe that anyone has ever argued that _all_ of the Erlang library is a model of good style (although much of it is), only that it is useful. There are fairly strict rules about how you ought to name Eiffel subprograms. There are rules about Java, some of them stinking bad. There are no official rules about Erlang naming other than "tell the truth and try not to confuse people". > From ok@REDACTED Fri May 18 06:38:42 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 18 May 2012 16:38:42 +1200 Subject: [erlang-questions] Question/Alternative on Frames Proposal [Warning: Long] In-Reply-To: <1337300505.68472.YahooMailNeo@web130106.mail.mud.yahoo.com> References: <4FB575D5.2020702@gmail.com> <1337300505.68472.YahooMailNeo@web130106.mail.mud.yahoo.com> Message-ID: <07EB1790-80A4-4956-88B9-6CA27067414E@cs.otago.ac.nz> On 18/05/2012, at 12:21 PM, Tom Parker wrote: > > But here is where I get stuck. > > Why not use the record format? ... (Bear with me, I believe the issues in that reuse can be addressed.) BECAUSE THE RECORD FORMAT ALREADY EXISTS! It's that simple. Back when Eiffel was undergoing major changes, I proposed something I called the "diarthognathus principle". When you want to evolve from supporting technique X to supporting technique Y, you had *better* evolve through a stage where you support *both* techniques. And that means that there has to be a period when frames have begun to work and record syntax still works *EXACTLY* the way it does now. Frames and records are not inter-operable (records are just a stylised use of tuples, while frames are a new completely distinct type). > Issue #1 with the ~ syntax) Today's Erlang developer, if using records, has already provided a form of that information. It already has a key->value mapping. And it must continue to work *as is* without breaking anything. But there is one thing more. Suppose you go around mechanically translating #id0{id1=E1, ..., idn=En} to which is a fairly straightforward task. What happens? The code immediately stops working. Because the two forms of construction DO NOT HAVE THE SAME SEMANTICS. A frame construction says "here is ALL the information about this frame." A record construction says "here is SOME information about this record, pick up anything I've left out from the -record declaration." Because these things have different semantics they MUST have different syntax. > > Issue #2 with the ~ syntax) The format proposed for extraction from a frame, with a tilde, is ~name(Frame). (okay, a little bike shed paint :) ) This reverses the standard ordering of items from Major to minor. I am tempted to use a very rude word here. THERE IS NO STANDARD ORDER!!! It may still be true that there are more lines of COBOL than any other language. In COBOL, it's of . In Algol 68, it's of . predecessor count OF node -:= 1 seems to me a perfectly natural way to decrement the number of predecessors of some node. For a functional language, note that a field selector is semantically just an overloaded function. In the Pop family of languages, record selectors are just functions (well, technically doublets) and are called exactly like functions. In Lisp, selectors are called exactly like functions. In ML-family languages, if f is a field, #f is the function that extracts that field (and yes, that's where I got the syntax). Yes, in Smalltalk the analogue of field selection is "object field", but that's how Smalltalk writes unary functions: "-42 abs". The syntax is neither novel nor unnatural unless you take an extremely parochial view of programming languages. > We are all already familiar with the record format being a key:value mapping, and that's what a frame would look like: > P = #point{x=4,y=5}. > or as part of the frame proposal, if you prefer the anonymous definition vs. a declared format, we can use (from erlson): > P = #{x = 4, y = 5}. It's not a matter of preference, not having declarations is what frames are ABOUT. > The big question: > Q: How do I know this #point is a frame and not a record? > A: Well, we have 2 ways of knowing: > (1) A language requirement would be imposed that #{ is always an "anonymous frame" since it is not a valid record. > (2) The namespace defined for -record(ATOM, {...}) and -frame(ATOM,{...}) would be shared (compiler - well, technically the preprocessor - would halt with an error if a record and a frame with the same ATOM identifier were included in the module to be compiled.) Since the preprocessor destroys records, the only #atom{ the compiler would ever see would be frames - it is never ambiguous to the compiler. No no no no NO! There can be *type* specifications that declare some type to be implemented as a frame of a particular shape, but there are no frame declarations as such. NOT having them is one of the major design goals! > > Comments on your questions from section 4 of your document in comparing solutions: > Q1: Frames answer changes to Yes, this actually forces an increase in intelligence of the preprocessor. That is utterly unacceptable. The frame proposal *EXISTS* in order to have something vaguely recordy that makes NO use of the preprocessor whatsoever. > Q4: This would alleviate the impact on Dialyzer as it already understands record formats (should make it a less challenging change) No, it wouldn't. I keep coming back to the same key point: frames and records have DIFFERENT SEMANTICS. Syntactic processing is not a challenge. The challenge in extending the Dialyzer to handle frames is in handling the SEMANTICS. > Q9: If the frame is a declared frame, then the frames answer is enhanced to be able to catch this issue at compile time, otherwise as before Frame declarations do not exist. By intent, there is no such animal as "a declared frame". Adding frame declarations is like adding huge paddle wheels to a hot air balloon and still expecting it to fly. One of the things that frames are meant for is to communicate structured values from one context to another context WITHOUT there being any declaration shared by both. > Q10: If the frame is a declared frame, then the frames answer is enhanced to be able to catch this issue at compile time, otherwise as before It is an important feature of the design that there are no frame declarations and there is NO preprocessor involvement in the processing of frames whatsoever. > Q11. Frames now take on the unnecessary recompile situation of records (if the record is a declared frame) Isn't that a clue that there is something wrong with the idea of -frame declarations? > Q20: I believe this would change to make detection easier, but am not totally sure. If you want records, you know where to find them. Records are *not* going to go away. They have their virtues and their vices. Frames are *different* virtues and vices. Trying to make one look like the other is about as kind to programmers as trying to make numbers and strings look the same. It is seductively attractive, but causes more trouble than it's worth in practice. (Speaking as someone who maintains an AWK implementation and tried to write an AWK compiler.) > > Additional Q&A: > Q: So this forces declaration of a frame if I want an atom between # and { > A: That is my thought. It would look like: > -frame(point, {x, y}). But the frames proposal already allows . Without any declaration. We know what is the right way to catch type errors and typos. That's what the Dialyzer does, and does very well. Speaking as someone who has tried to maintain other people's code, if I see #point{x = 1, y = 2} I want to *KNOW* that that is a record, not anything else at all. I don't want it to maybe be a record or maybe something else, possibly depending on a file that isn't open in my editor. You are asking for a "feature" that I regard as active malevolence towards the programmer. > Q: How does extraction change? > A: Extraction with the record-style format ... and is major to minor: > Frame#format.member Having to repeat the record name is one of the most hated aspects of the present -record syntax. I am having no success understanding why anyone would *WANT* to imitate that abomination. As for major to minor, I have never perceived anything major or minor in field access. Some languages put the field first, some put it second. BIG DEAL. Languages are different. Get over it. They don't all have to look like Java. I skipped the rest of the message. The frame proposal exists precisely in order to have something that can do the same job that records do but in a DIFFERENT way. Because the semantics is different, programmers need to know what they are dealing with, so it is an incredibly bad idea to use the same syntax for both. It doesn't matter so very much what the syntax _is_ as long as it is clearly _different_. Frames "fit" into Erlang data structures in a way that records never did; _all_ the information that exists about a frame is right there in it without consulting any declaration. In particular, changing a declaration in one field cannot change the meaning of a frame construction in another. Adding -frame declarations isn't "raising" the proposal but "razing" it. From Barys_Ilyushonak@REDACTED Fri May 18 08:05:09 2012 From: Barys_Ilyushonak@REDACTED (Ilyushonak Barys) Date: Fri, 18 May 2012 06:05:09 +0000 Subject: [erlang-questions] implementing Forex Trader in Erlang In-Reply-To: <4FB575D5.2020702@gmail.com> References: <4FB575D5.2020702@gmail.com> Message-ID: It looks likes that this guys use Erlang only for messaging middleware. The server components, which connected to exchanges, are implemented in c++. From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Michael Truog Sent: Friday, May 18, 2012 2:05 AM To: Shahrdad Shadab Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] implementing Forex Trader in Erlang Here is the architecture of something similar: http://www.connamara.com/solutions-hftframework.html On 05/17/2012 09:19 AM, Shahrdad Shadab wrote: Hi Erlangers I have to implement a foreign exchange trader software for a financial firm. They need fast response and high availability. They also need parallel real time feed from multiple brokers. As I did my preliminary investigations,It seems most of the tools on the field are implemented in Microsoft technologies like .net or com/OLE. I believe (although I might be wrong) the broker servers are DDE servers. I decided to use Erlang/otp. Does anyone know if something like this has done before in Erlang. Also I appreciate any guidance on how to get feed from trader servers. Thanks a lot Best regards Shahrdad -- Software Architect & Computer Scientist _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________________ The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia. If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp -------------- next part -------------- An HTML attachment was scrubbed... URL: From quiquepaz@REDACTED Fri May 18 08:45:14 2012 From: quiquepaz@REDACTED (Enrique Paz) Date: Fri, 18 May 2012 08:45:14 +0200 Subject: [erlang-questions] Deterministic playback of a simulation? In-Reply-To: References: Message-ID: If I get it right, your issue is that the logged function calls might be in the wrong order. Couldn't you add a timestamp to the call info you are logging? erlang:now() or os:timestamp/0 (faster) return a timestamp up to microseconds, then the only action left to do would be ordering your logs before using them. 2012/5/18 Richard Evans > I have a multiplayer simulation game running in erlang, involving > multiple characters making decisions etc. Each game instance uses a > few (currently, 4) erlang processes running concurrently, > communicating with an AI simulator written in C. > > Everything is working well and I am enjoying using erlang. > > But there is one thing I am finding tricky. > > For QA purposes, I want to be able to record the exact sequence of > function-calls, so I can play the game back exactly, > deterministically. > > My current approach is this: I have a global ets table storing a list > of function-calls. Then, whenever I was calling a function I want to > record - instead of calling the function directly - I call a procedure > which adds the function to the list of function-calls, and then calls > it: > > call_and_store(Fun, Args) -> > add_to_script(Fun, Args); > apply(lobby, Fun, Args). > > add_to_script(Fun, Args) -> > Info = ets:info(script), > {size, Size} = lists:keyfind(size, 1, Info), > io:format("Inserting {~p,~p,~p}~n", [Size,Fun,Args]), > ets:insert(script, {Size, Fun, Args}). > > This approach does not seem to work well because of erlang's > pre-emptive scheduling. If two concurrent processes both invoke > call_and_store, then it is possible (and seems to actually be > happening) that erlang's pre-emptive scheduler may stop processing one > instance of call_and_store between execution of add_to_script and > execution of apply. If this happens, the order of execution and the > order of recorded function-calls will diverge, we don't have an > accurate recording of the set of function-calls, and we won't be able > to deterministically playback. > > So my question is: is there a way to prevent the scheduler from > yielding during a block of code? Some way to insist the block is > called as one unit, like this: > > call_and_store(Fun, Args) -> > !!!prevent_yielding, > add_to_script(Fun, Args); > apply(lobby, Fun, Args), > !!!allow_yielding. > > My guess is: no. This does not seem very erlangy. > > Alternatively, is there a different approach which can produce the > desired list of function-calls? My current approach seems to go > against the erlang grain, involving writing to a shared global table. > Is there a better, more erlangy, way of doing this? > > > > thanks, > ~Richard > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- quique -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Fri May 18 10:49:25 2012 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Fri, 18 May 2012 10:49:25 +0200 Subject: [erlang-questions] Deterministic playback of a simulation? In-Reply-To: References: Message-ID: 2012/5/18 Richard Evans : [...] > For QA purposes, I want to be able to record the exact sequence of > function-calls, so I can play the game back exactly, > deterministically. You could try to trace on the function calls with timestamps. From curtis_ovid_poe@REDACTED Fri May 18 11:00:24 2012 From: curtis_ovid_poe@REDACTED (Ovid) Date: Fri, 18 May 2012 02:00:24 -0700 (PDT) Subject: [erlang-questions] Erlang suitability Message-ID: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> Hi there, We've a system that run across 75 servers and needs to be highly performant, fault-tolerant, scalable and shares persistent data across all 75 servers. We're investigating Erlang/Mnesia (which we don't know) because it sounds tailor-made for our situation. We are not using Erlang for our first implementation, but are instead hacking together a solution from known technologies including Perl, MySQL and Redis. We're considering Erlang for our future work. We have two primary needs: Each box can bid on an auction and potentially spend a tiny amount of money and each of the 75 boxes will receive notifications of a small amount of money spent if they win the auction (the auction notification will probably not be sent to the box bidding in the auction). Use case 1: If the *total* of all of those small amounts exceeds a daily cap or an all-time cap, all 75 boxes must immediately stop spending bidding in auctions. It seems that each box can run a separate Erlang process and write out "winning bid" information to an Mnesia database and all boxes can read the total amount spent from that to determine if it should stop bidding. This seems trivial to set up. Use case 2: we periodically need to reauthenticate to the auction system. We MUST NOT have all 75 boxes trying to reauthenticate at the same time because we will be locked out of the system if we do this. Having a central box handling reauthentication is a single point of failure that we would like to avoid, but we don't know what design pattern Erlang would use to ensure that only one of the 75 Erlang instances would attempt to reauthenticate at any one time (all 75 boxes can share the same authentication token). Use case 1 is clearly perfect for Erlang. Use case 2 is less clear to us. We're going to be spending a fair amount of time hacking together a non-Erlang solution, but the benefit is using known technologies that don't depend on the "one guy who knows the system". At full scale, we anticipate billions of auctions per day. ? Does anyone care to comment about things we should look at? Particularly, is use case 2 not something appropriate for Erlang? A tiny amount of sample code would be lovely (note: I am somewhat comforable with Prolog, so Erlang looks fairly straightforward for me, aside from understanding the message passing). Cheers, Ovid -- Live and work overseas - http://www.overseas-exile.com/ Buy the book - http://www.oreilly.com/catalog/perlhks/ Tech blog - http://blogs.perl.org/users/ovid/ Twitter - http://twitter.com/OvidPerl/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From hynek@REDACTED Fri May 18 11:04:26 2012 From: hynek@REDACTED (Hynek Vychodil) Date: Fri, 18 May 2012 11:04:26 +0200 Subject: [erlang-questions] Deterministic playback of a simulation? In-Reply-To: References: Message-ID: Hi. What about using erlang tracing facility. It hase some nice features. There will be zero inpact in production code. There will be zero inpact in production environment if you turn it on only in testing environment and also there is "automatic" ordering of events. I think it is best fit for purpose. Hynek Vychodil On Fri, May 18, 2012 at 4:36 AM, Richard Evans wrote: > I have a multiplayer simulation game running in erlang, involving > multiple characters making decisions etc. Each game instance uses a > few (currently, 4) erlang processes running concurrently, > communicating with an AI simulator written in C. > > Everything is working well and I am enjoying using erlang. > > But there is one thing I am finding tricky. > > For QA purposes, I want to be able to record the exact sequence of > function-calls, so I can play the game back exactly, > deterministically. > > My current approach is this: I have a global ets table storing a list > of function-calls. Then, whenever I was calling a function I want to > record - instead of calling the function directly - I call a procedure > which adds the function to the list of function-calls, and then calls > it: > > call_and_store(Fun, Args) -> > ? ?add_to_script(Fun, Args); > ? ?apply(lobby, Fun, Args). > > add_to_script(Fun, Args) -> > ? ?Info = ets:info(script), > ? ?{size, Size} = lists:keyfind(size, 1, Info), > ? ?io:format("Inserting {~p,~p,~p}~n", [Size,Fun,Args]), > ? ?ets:insert(script, {Size, Fun, Args}). > > This approach does not seem to work well because of erlang's > pre-emptive scheduling. If two concurrent processes both invoke > call_and_store, then it is possible (and seems to actually be > happening) that erlang's pre-emptive scheduler may stop processing one > instance of call_and_store between execution of add_to_script and > execution of apply. If this happens, the order of execution and the > order of recorded function-calls will diverge, we don't have an > accurate recording of the set of function-calls, and we won't be able > to deterministically playback. > > So my question is: is there a way to prevent the scheduler from > yielding during a block of code? Some way to insist the block is > called as one unit, like this: > > call_and_store(Fun, Args) -> > ? ? ? ?!!!prevent_yielding, > ? ?add_to_script(Fun, Args); > ? ?apply(lobby, Fun, Args), > ? ? ? ?!!!allow_yielding. > > My guess is: no. This does not seem very erlangy. > > Alternatively, is there a different approach which can produce the > desired list of function-calls? My current approach seems to go > against the erlang grain, involving writing to a shared global table. > Is there a better, more erlangy, way of doing this? > > > > thanks, > ~Richard > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vasily@REDACTED Fri May 18 11:34:30 2012 From: vasily@REDACTED (Vasily Sulatskov) Date: Fri, 18 May 2012 11:34:30 +0200 Subject: [erlang-questions] Several gen_leader questions Message-ID: Hi, I am using gen_leader from https://github.com/abecciu/gen_leader_revival and I want to use it with a fixed list of candidate nodes - as far as I understand that's the easiest way. I tried several variations of starting gen_leader but none was satisfactory. I start it as part of a supervision tree, so all examples are taken from gen_supervisor modules init/1 functions: Here's what I tried so far: init(_Args) -> %% The same value is used on all machines in the cluster Leader_nodes = ['foobar@REDACTED', 'foobar@REDACTED', 'foobar@REDACTED'], Home = os:getenv("HOME"), Gen_leader_config = {gen_leader_module, {gen_leader_module, start_link, [Leader_nodes, [{vardir, Home}]]}, permanent, 2000, worker, [gen_leader_module]}, {ok, {{one_for_one, 10000, 1}, [Gen_leader_config]}}. If I do this, then nodes specified in Leader_nodes work just fine, they all participate in elections, leaders are elected properly, they are able to do gen_leader:leader_call() to the actual leader etc. The problem is that on all other nodes (which are not specified in Leader_nodes) gen_leader is not started at all. Gen_leader checks if the node it's running on is one of "candidate nodes" or "worker nodes" and if that's not the case - it simply doesn't start. All further attempts at gen_leader:leader_call from that node fail. I tried to run every node in the cluster except for "candidate nodes" as a "worker node", so I changed supervisor to something like: init(_Args) -> %% The same value is used on all machines in the cluster Leader_nodes = ['foobar@REDACTED', 'foobar@REDACTED', 'foobar@REDACTED'], Workers = case lists:member(node(), Leader_nodes) of true -> []; false -> [node()] end, Home = os:getenv("HOME"), {ok, {{one_for_one, 10000, 1}, [{scheduler, {scheduler, start_link, [Leader_nodes, [{vardir, Home}, {workers, Workers}]]}, permanent, 2000, worker, [scheduler_leader]}]}}. As far as I understand, when gen_leader runs in a worker configuration, it doesn't participate in elections, but still keeps track of where an actual leader is running, so gen_leader:leader_call is still possible. This setup kind of works, but it seems that gen_leader process on "worker" nodes constantly grows in memory usage, past several Gb at least, eventually crashing the whole VM. Am I running gen_leader correctly? What is the correct way of running gen_leader with a fixed set of "candidate" nodes and that every other node is aware of where a leader is running, so that gen_leader:leader_call() is possible? Which version of gen_leader is recommended to use? This one https://github.com/abecciu/gen_leader_revival? Or maybe the version from gproc? By the way can someone explain what's the difference between them? And I have another, most likely unrelated, issue with gen_leader. On one deployment, sometimes I find a cluster in a state with two leaders - most of the nodes think that the leader is one node, but some other node thinks that the leader is on the other node. I am not sure if the other leader is the node that diverges from the consensus - I don't have a cluster in this state right now to check. It seems to happen after a gen_leader process crashes somehow (some internal work, not related to gen_leader magic). The other thing that I think might be important here, is that gen_leader process in that setup can get stuck in handle_leader_call for quite some long time. Can it cause problems with leader elections? Should gen_leader processes not block in handle_whatever functions and always be able to handle election callback? Thanks in advance. -- Vasily Sulatskov From rapsey@REDACTED Fri May 18 11:40:36 2012 From: rapsey@REDACTED (Rapsey) Date: Fri, 18 May 2012 11:40:36 +0200 Subject: [erlang-questions] Erlang suitability In-Reply-To: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> References: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> Message-ID: 75 machines in a fully connected distributed erlang network is a bit much but it should work. use case 1: Why do you need to store to mnesia and all machines read from it? If you have 75 servers erlang connected, you can just send a simple erlang message to all of them. use case 2: In a fully connected erlang network you always know which machines are offline/online. Every machine has a name. If you simply sort all online machines by node name, that is the node you pick to do authentication. No real synchronization required and it will work even if that machine is lost. Or you can hardcode a sequence of machines which ones do the authenticating. First in list that is online is the one that does it. Sergej On Fri, May 18, 2012 at 11:00 AM, Ovid wrote: > Hi there, > > We've a system that run across 75 servers and needs to be highly > performant, fault-tolerant, scalable and shares persistent data across all > 75 servers. We're investigating Erlang/Mnesia (which we don't know) because > it sounds tailor-made for our situation. > > We are not using Erlang for our first implementation, but are instead > hacking together a solution from known technologies including Perl, MySQL > and Redis. We're considering Erlang for our future work. > > We have two primary needs: Each box can bid on an auction and potentially > spend a tiny amount of money and each of the 75 boxes will receive > notifications of a small amount of money spent if they win the auction (the > auction notification will probably not be sent to the box bidding in the > auction). > > Use case 1: If the *total* of all of those small amounts exceeds a daily > cap or an all-time cap, all 75 boxes must immediately stop spending bidding > in auctions. It seems that each box can run a separate Erlang process and > write out "winning bid" information to an Mnesia database and all boxes can > read the total amount spent from that to determine if it should stop > bidding. > > This seems trivial to set up. > > Use case 2: we periodically need to reauthenticate to the auction system. > We MUST NOT have all 75 boxes trying to reauthenticate at the same time > because we will be locked out of the system if we do this. Having a central > box handling reauthentication is a single point of failure that we would > like to avoid, but we don't know what design pattern Erlang would use to > ensure that only one of the 75 Erlang instances would attempt to > reauthenticate at any one time (all 75 boxes can share the same > authentication token). > > Use case 1 is clearly perfect for Erlang. Use case 2 is less clear to us. > We're going to be spending a fair amount of time hacking together a > non-Erlang solution, but the benefit is using known technologies that don't > depend on the "one guy who knows the system". > > At full scale, we anticipate billions of auctions per day. > > Does anyone care to comment about things we should look at? Particularly, > is use case 2 not something appropriate for Erlang? A tiny amount of sample > code would be lovely (note: I am somewhat comforable with Prolog, so Erlang > looks fairly straightforward for me, aside from understanding the message > passing). > > Cheers, > Ovid > -- > Live and work overseas - http://www.overseas-exile.com/ > Buy the book - http://www.oreilly.com/catalog/perlhks/ > Tech blog - http://blogs.perl.org/users/ovid/ > Twitter - http://twitter.com/OvidPerl/ > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From curtis_ovid_poe@REDACTED Fri May 18 11:49:02 2012 From: curtis_ovid_poe@REDACTED (Ovid) Date: Fri, 18 May 2012 02:49:02 -0700 (PDT) Subject: [erlang-questions] Erlang suitability In-Reply-To: References: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> Message-ID: <1337334542.93482.YahooMailNeo@web162103.mail.bf1.yahoo.com> Hi Sergej, Thanks for the response. It sounds like use case 2 is easier than I thought. As for Mnesia and use case 1, our thought was twofold: 1. We want ACID compliance. We need to ensure that were all 75 boxes to mysteriously crash, we could bring them back up and not worry about data integrity. 2. Our storage needs are likely to become more complex in the future. The daily and total capping amounts will eventually be broken down into a variety of different categories, along with synchronization of which machines bid on a request and if those individual machines won a request (something that we currently cannot do without offline aggregation of this data). ? When you say "75 machines in a fully connected distributed erlang network is a bit much but it should work."?could you elaborate on that? It makes me a bit nervous to hear, particularly since we have no Erlang knowledge and just assumed that it was suited for this. Are we thinking about this the wrong way? Cheers, Ovid -- Live and work overseas - http://www.overseas-exile.com/ Buy the book - http://www.oreilly.com/catalog/perlhks/ Tech blog - http://blogs.perl.org/users/ovid/ Twitter - http://twitter.com/OvidPerl/ >________________________________ > From: Rapsey >To: Ovid >Cc: "erlang-questions@REDACTED" >Sent: Friday, 18 May 2012, 11:40 >Subject: Re: [erlang-questions] Erlang suitability > > >75 machines in a fully connected distributed erlang network is a bit much but it should work. >use case 1: Why do you need to store to mnesia and all machines read from it? If you have 75 servers erlang connected, you can just send a simple erlang message to all of them. >use case 2: In a fully connected erlang network you always know which machines are offline/online. Every machine has a name. ?If you simply sort all online machines by node name, that is the node you pick to do authentication. No real synchronization required and it will work even if that machine is lost. Or you can hardcode a sequence of machines which ones do the authenticating. First in list that is online is the one that does it. > > > >Sergej > > >On Fri, May 18, 2012 at 11:00 AM, Ovid wrote: > >Hi there, >> >> >>We've a system that run across 75 servers and needs to be highly performant, fault-tolerant, scalable and shares persistent data across all 75 servers. We're investigating Erlang/Mnesia (which we don't know) because it sounds tailor-made for our situation. >> >> >>We are not using Erlang for our first implementation, but are instead hacking together a solution from known technologies including Perl, MySQL and Redis. We're considering Erlang for our future work. >> >> >>We have two primary needs: Each box can bid on an auction and potentially spend a tiny amount of money and each of the 75 boxes will receive notifications of a small amount of money spent if they win the auction (the auction notification will probably not be sent to the box bidding in the auction). >> >> >>Use case 1: If the *total* of all of those small amounts exceeds a daily cap or an all-time cap, all 75 boxes must immediately stop spending bidding in auctions. It seems that each box can run a separate Erlang process and write out "winning bid" information to an Mnesia database and all boxes can read the total amount spent from that to determine if it should stop bidding. >> >> >>This seems trivial to set up. >> >> >>Use case 2: we periodically need to reauthenticate to the auction system. We MUST NOT have all 75 boxes trying to reauthenticate at the same time because we will be locked out of the system if we do this. Having a central box handling reauthentication is a single point of failure that we would like to avoid, but we don't know what design pattern Erlang would use to ensure that only one of the 75 Erlang instances would attempt to reauthenticate at any one time (all 75 boxes can share the same authentication token). >> >> >>Use case 1 is clearly perfect for Erlang. Use case 2 is less clear to us. We're going to be spending a fair amount of time hacking together a non-Erlang solution, but the benefit is using known technologies that don't depend on the "one guy who knows the system". >> >> >>At full scale, we anticipate billions of auctions per day. >>? >>Does anyone care to comment about things we should look at? Particularly, is use case 2 not something appropriate for Erlang? A tiny amount of sample code would be lovely (note: I am somewhat comforable with Prolog, so Erlang looks fairly straightforward for me, aside from understanding the message passing). >> >> >>Cheers, >>Ovid >>-- >>Live and work overseas - http://www.overseas-exile.com/ >>Buy the book - http://www.oreilly.com/catalog/perlhks/ >>Tech blog - http://blogs.perl.org/users/ovid/ >>Twitter - http://twitter.com/OvidPerl/ >>_______________________________________________ >>erlang-questions mailing list >>erlang-questions@REDACTED >>http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Fri May 18 12:27:51 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Fri, 18 May 2012 12:27:51 +0200 Subject: [erlang-questions] Erlang suitability In-Reply-To: <1337334542.93482.YahooMailNeo@web162103.mail.bf1.yahoo.com> References: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> <1337334542.93482.YahooMailNeo@web162103.mail.bf1.yahoo.com> Message-ID: On Fri, May 18, 2012 at 11:49 AM, Ovid wrote: > 1. We want ACID compliance. We need to ensure that were all 75 boxes to > mysteriously crash, we could bring them back up and not worry about data > integrity. This is mostly guaranteed by Mnesia, but it is sensitive to start/stop order. I.e. the last node to stop has to be the first node to start (in most cases this is not necessary, but it is the case if whole cluster crashed). Please correct me if I'm wrong. > 2. Our storage needs are likely to become more complex in the future. The > daily and total capping amounts will eventually be broken down into a > variety of different categories, along with synchronization of which > machines bid on a request and if those individual machines won a request > (something that we currently cannot do without offline aggregation of this > data). Not sure if mnesia supports such "sub clusters". Although it probably run transactions only over nodes on which a involved table is located, isn't it? > When you say "75 machines in a fully connected distributed erlang network is > a bit much but it should work."?could you elaborate on that? It makes me a > bit nervous to hear, particularly since we have no Erlang knowledge and just > assumed that it was suited for this. Are we thinking about this the wrong > way? Mnesia was tailored for 2-3-4-5 nodes at max, since it was designed to be a storage for storing configuration and short lived data over pairs of boxes, where one of the boxes is an failover node. From rapsey@REDACTED Fri May 18 12:29:08 2012 From: rapsey@REDACTED (Rapsey) Date: Fri, 18 May 2012 12:29:08 +0200 Subject: [erlang-questions] Erlang suitability In-Reply-To: <1337334542.93482.YahooMailNeo@web162103.mail.bf1.yahoo.com> References: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> <1337334542.93482.YahooMailNeo@web162103.mail.bf1.yahoo.com> Message-ID: Distributed erlang http://www.erlang.org/doc/reference_manual/distributed.html I actually don't have a lot of experience in running a distributed erlang installation, but guys in this discussion do: https://groups.google.com/forum/#!topic/erlang-programming/YUEPsDtGbso But from the information you have provided erlang should definitely be the best tool for the job. Since you lack erlang experience, maybe you should contact the erlang solution guys: http://www.erlang-solutions.com/ Sergej On Fri, May 18, 2012 at 11:49 AM, Ovid wrote: > Hi Sergej, > > Thanks for the response. It sounds like use case 2 is easier than I > thought. > > As for Mnesia and use case 1, our thought was twofold: > > 1. We want ACID compliance. We need to ensure that were all 75 boxes to > mysteriously crash, we could bring them back up and not worry about data > integrity. > 2. Our storage needs are likely to become more complex in the future. The > daily and total capping amounts will eventually be broken down into a > variety of different categories, along with synchronization of which > machines bid on a request and if those individual machines won a request > (something that we currently cannot do without offline aggregation of this > data). > > When you say "75 machines in a fully connected distributed erlang network > is a bit much but it should work." could you elaborate on that? It makes > me a bit nervous to hear, particularly since we have no Erlang knowledge > and just assumed that it was suited for this. Are we thinking about this > the wrong way? > > Cheers, > Ovid > -- > Live and work overseas - http://www.overseas-exile.com/ > Buy the book - http://www.oreilly.com/catalog/perlhks/ > Tech blog - http://blogs.perl.org/users/ovid/ > Twitter - http://twitter.com/OvidPerl/ > > ------------------------------ > *From:* Rapsey > *To:* Ovid > *Cc:* "erlang-questions@REDACTED" > *Sent:* Friday, 18 May 2012, 11:40 > *Subject:* Re: [erlang-questions] Erlang suitability > > 75 machines in a fully connected distributed erlang network is a bit much > but it should work. > use case 1: Why do you need to store to mnesia and all machines read from > it? If you have 75 servers erlang connected, you can just send a simple > erlang message to all of them. > use case 2: In a fully connected erlang network you always know which > machines are offline/online. Every machine has a name. If you simply sort > all online machines by node name, that is the node you pick to do > authentication. No real synchronization required and it will work even if > that machine is lost. Or you can hardcode a sequence of machines which ones > do the authenticating. First in list that is online is the one that does it. > > > Sergej > > On Fri, May 18, 2012 at 11:00 AM, Ovid wrote: > > Hi there, > > We've a system that run across 75 servers and needs to be highly > performant, fault-tolerant, scalable and shares persistent data across all > 75 servers. We're investigating Erlang/Mnesia (which we don't know) because > it sounds tailor-made for our situation. > > We are not using Erlang for our first implementation, but are instead > hacking together a solution from known technologies including Perl, MySQL > and Redis. We're considering Erlang for our future work. > > We have two primary needs: Each box can bid on an auction and potentially > spend a tiny amount of money and each of the 75 boxes will receive > notifications of a small amount of money spent if they win the auction (the > auction notification will probably not be sent to the box bidding in the > auction). > > Use case 1: If the *total* of all of those small amounts exceeds a daily > cap or an all-time cap, all 75 boxes must immediately stop spending bidding > in auctions. It seems that each box can run a separate Erlang process and > write out "winning bid" information to an Mnesia database and all boxes can > read the total amount spent from that to determine if it should stop > bidding. > > This seems trivial to set up. > > Use case 2: we periodically need to reauthenticate to the auction system. > We MUST NOT have all 75 boxes trying to reauthenticate at the same time > because we will be locked out of the system if we do this. Having a central > box handling reauthentication is a single point of failure that we would > like to avoid, but we don't know what design pattern Erlang would use to > ensure that only one of the 75 Erlang instances would attempt to > reauthenticate at any one time (all 75 boxes can share the same > authentication token). > > Use case 1 is clearly perfect for Erlang. Use case 2 is less clear to us. > We're going to be spending a fair amount of time hacking together a > non-Erlang solution, but the benefit is using known technologies that don't > depend on the "one guy who knows the system". > > At full scale, we anticipate billions of auctions per day. > > Does anyone care to comment about things we should look at? Particularly, > is use case 2 not something appropriate for Erlang? A tiny amount of sample > code would be lovely (note: I am somewhat comforable with Prolog, so Erlang > looks fairly straightforward for me, aside from understanding the message > passing). > > Cheers, > Ovid > -- > Live and work overseas - http://www.overseas-exile.com/ > Buy the book - http://www.oreilly.com/catalog/perlhks/ > Tech blog - http://blogs.perl.org/users/ovid/ > Twitter - http://twitter.com/OvidPerl/ > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Fri May 18 13:22:06 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 18 May 2012 13:22:06 +0200 Subject: [erlang-questions] Erlang suitability In-Reply-To: References: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> <1337334542.93482.YahooMailNeo@web162103.mail.bf1.yahoo.com> Message-ID: <5E99D92D-2A0D-468C-A1E2-10F5DE654155@feuerlabs.com> On 18 May 2012, at 12:27, Gleb Peregud wrote: > This is mostly guaranteed by Mnesia, but it is sensitive to start/stop > order. I.e. the last node to stop has to be the first node to start > (in most cases this is not necessary, but it is the case if whole > cluster crashed). Please correct me if I'm wrong. It doesn't have to be the first to start, but the other nodes will wait for it to appear. That is, all nodes that share replicas with that node, since they cannot know the status of that replica until it comes back on-line. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From andrey@REDACTED Fri May 18 13:25:35 2012 From: andrey@REDACTED (Andrey Tsirulev) Date: Fri, 18 May 2012 15:25:35 +0400 Subject: [erlang-questions] Scale testing server with Erlang bots In-Reply-To: References: Message-ID: <4FB631AF.9000402@artplant.no> Hi, The simple question is: in common test how can I run the same test N times in parallel (not sequentially)? More verbosely - here is my problem. I need to stress-test my client-server application (a multiplayer game) by launching a large number of bots from several machines (launching N bots at once to start with). Bots are implemented in Erlang and use custom TCP protocol to connect to the server. I don't need to collect any metrics on test system side, I can learn everything I need by monitoring the server. The possible solutions are: 1) Common test I'm new to it and still cannot find out how to run the test N times in parallel (on one node) - when using repeat N it waits for the previous test run to end before starting the next one. If this can be solved, common test seems to suit me. 2) Tsung When using tsung I have to write my own plugin - and it seems that I'm very limited in what I can do with it, while my bots are pretty complex in their behaviour, so I'll have to shrink it. Another downside is that I'll have to write and support separate bot code instead of the code used in functionality (not stress) tests. Finally, I feel very uncomfortable with tsung xml configuration - I prefer to setup bot behaviour in code. On the other hand I don't feel I can benefit from tsung's stronger sides, so I come to conclusion it's not the right tool for my job. 3) Write it myself - most likely reinventing the wheel. Is there any solution I miss? Thanks, Andrey From ulf@REDACTED Fri May 18 13:38:05 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 18 May 2012 13:38:05 +0200 Subject: [erlang-questions] Deterministic playback of a simulation? In-Reply-To: References: Message-ID: On 18 May 2012, at 04:36, Richard Evans wrote: > So my question is: is there a way to prevent the scheduler from > yielding during a block of code? Some way to insist the block is > called as one unit, like this: > > call_and_store(Fun, Args) -> > !!!prevent_yielding, > add_to_script(Fun, Args); > apply(lobby, Fun, Args), > !!!allow_yielding. > > My guess is: no. This does not seem very erlangy. No, not at that level. But you could use e.g. global:trans({call_and_store, self()}, fun() -> call_and_store(Fun, Args) end). See erl -man global You can specify on how many nodes you want to acquire a lock - default is all connected nodes, but you can set it e.g. to [node()] if that suits you better. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From mononcqc@REDACTED Fri May 18 14:02:22 2012 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 18 May 2012 08:02:22 -0400 Subject: [erlang-questions] Erlang suitability In-Reply-To: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> References: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> Message-ID: <4FB63A4E.70408@ferd.ca> Answers inline. On 12-05-18 5:00 AM, Ovid wrote: > Hi there, > > We've a system that run across 75 servers and needs to be highly > performant, fault-tolerant, scalable and shares persistent data across > all 75 servers. We're investigating Erlang/Mnesia (which we don't > know) because it sounds tailor-made for our situation. As mentioned earlier in this thread, 75 servers is a bit much, but people have done it before. > > We are not using Erlang for our first implementation, but are instead > hacking together a solution from known technologies including Perl, > MySQL and Redis. We're considering Erlang for our future work. > > We have two primary needs: Each box can bid on an auction and > potentially spend a tiny amount of money and each of the 75 boxes will > receive notifications of a small amount of money spent if they win the > auction (the auction notification will probably not be sent to the box > bidding in the auction). > > Use case 1: If the *total* of all of those small amounts exceeds a > daily cap or an all-time cap, all 75 boxes must immediately stop > spending bidding in auctions. It seems that each box can run a > separate Erlang process and write out "winning bid" information to an > Mnesia database and all boxes can read the total amount spent from > that to determine if it should stop bidding. > > This seems trivial to set up. It isn't trivial. You have think about what happens when a box is seen as crashing. How strongly consistent do you want things to be? There is always a risk that a box didn't crash, but was cut off in a netsplit. You might get divergences in budget that will be hard to explain. There is also a definite timing issue depending on how your data is being observed. For example, you ask permission to bid on an item, but you do not get instant feedback; by the time you sent maybe 5-10 bids, the cap is finally reached and broken at once because the delay to the other network made you keep on bidding without a final result. How much tolerance do you have for this? You mentioned in another post that "We need to ensure that were all 75 boxes to mysteriously crash, we could bring them back up and not worry about data integrity.", Possibly, but what about 1 node only? What about 5? What about 30 or 35? What if they crash and you missed winning bids because you went out after bidding but before getting your notifications back (if that is possible by the bidding rules of whatever exchange you're dealing with). The most solid synchronous database setup might not give you the guarantees you expect in the first place. > > Use case 2: we periodically need to reauthenticate to the auction > system. We MUST NOT have all 75 boxes trying to reauthenticate at the > same time because we will be locked out of the system if we do this. > Having a central box handling reauthentication is a single point of > failure that we would like to avoid, but we don't know what design > pattern Erlang would use to ensure that only one of the 75 Erlang > instances would attempt to reauthenticate at any one time (all 75 > boxes can share the same authentication token). That depends on: 1. how many times you can try to re-authenticate before being blocked, 2. how close together they have to be. Central points of failures are definitely something to avoid. Leader election across 75 boxes might not be the funnest thing in the world either. I could see a scheme where you use some distributed cached value that can say "I am currently being logged" that can time out at some point, visible to all readers. When you read that timeout value from each box (possibly from an OTP Application that only handles auth), each reading of that value adds or subtracts a random number to the timeout. This is to try and avoid a cluster-wide synchronization on the timeout value, and instead have them happen at different times. You could add an "I'm updating" flag related to that value and that could give you good probabilities that only a fraction of all the nodes attempt an authentication at any point in time close to the timeout value. Again, this would depend on how often your authentication needs to be done, and to what frequency you're allowed to do it. If it's too tight, you might need a central server or node that takes care of it, with one or two fail-overs to add some reliability. Note you will still have to care about netsplits ruining your day with this whole scheme. -- I had nothing to add on the rest of the mail, so cut if off. Hope this helps, Fred. -------------- next part -------------- An HTML attachment was scrubbed... URL: From curtis_ovid_poe@REDACTED Fri May 18 14:29:16 2012 From: curtis_ovid_poe@REDACTED (Ovid) Date: Fri, 18 May 2012 05:29:16 -0700 (PDT) Subject: [erlang-questions] Erlang suitability In-Reply-To: <4FB63A4E.70408@ferd.ca> References: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> <4FB63A4E.70408@ferd.ca> Message-ID: <1337344156.19879.YahooMailNeo@web162102.mail.bf1.yahoo.com> Netsplits. Damn. I forgot to put my thinking CAP on. In this case, a netsplit would be disastrous unless we fell back to a central data store such as Redis. At that point, Erlang doesn't look like a solution at all. On top of that, you're the second person to point out that a complete graph with 75 nodes is problematic. Now that I think about it, I guess I can see why. It now sounds like an Erlang solution is not the quick win we thought it might be (quelle surprise!).? ? Thanks to everyone for all of your answers. Cheers, Curtis -- Live and work overseas - http://www.overseas-exile.com/ Buy the book - http://www.oreilly.com/catalog/perlhks/ Tech blog - http://blogs.perl.org/users/ovid/ Twitter - http://twitter.com/OvidPerl/ >________________________________ > From: Fred Hebert >To: Ovid >Cc: "erlang-questions@REDACTED" >Sent: Friday, 18 May 2012, 14:02 >Subject: Re: [erlang-questions] Erlang suitability > > >Answers inline. > >On 12-05-18 5:00 AM, Ovid wrote: >Hi there, >> >> >>We've a system that run across 75 servers and needs to be highly performant, fault-tolerant, scalable and shares persistent data across all 75 servers. We're investigating Erlang/Mnesia (which we don't know) because it sounds tailor-made for our situation. As mentioned earlier in this thread, 75 servers is a bit much, but people have done it before. > > >> >>We are not using Erlang for our first implementation, but are instead hacking together a solution from known technologies including Perl, MySQL and Redis. We're considering Erlang for our future work. >> >> >>We have two primary needs: Each box can bid on an auction and potentially spend a tiny amount of money and each of the 75 boxes will receive notifications of a small amount of money spent if they win the auction (the auction notification will probably not be sent to the box bidding in the auction). >> >> >>Use case 1: If the *total* of all of those small amounts exceeds a daily cap or an all-time cap, all 75 boxes must immediately stop spending bidding in auctions. It seems that each box can run a separate Erlang process and write out "winning bid" information to an Mnesia database and all boxes can read the total amount spent from that to determine if it should stop bidding. >> >> >>This seems trivial to set up. It isn't trivial. You have think about what happens when a box is seen as crashing. How strongly consistent do you want things to be? There is always a risk that a box didn't crash, but was cut off in a netsplit. You might get divergences in budget that will be hard to explain. > >There is also a definite timing issue depending on how your data is being observed. For example, you ask permission to bid on an item, but you do not get instant feedback; by the time you sent maybe 5-10 bids, the cap is finally reached and broken at once because the delay to the other network made you keep on bidding without a final result. How much tolerance do you have for this? > >You mentioned in another post that "We need to ensure that were all 75 boxes to mysteriously crash, we could bring them back up and not worry about data integrity.", Possibly, but what about 1 node only? What about 5? What about 30 or 35? What if they crash and you missed winning bids because you went out after bidding but before getting your notifications back (if that is possible by the bidding rules of whatever exchange you're dealing with). > >The most solid synchronous database setup might not give you the guarantees you expect in the first place. > > >> >>Use case 2: we periodically need to reauthenticate to the auction system. We MUST NOT have all 75 boxes trying to reauthenticate at the same time because we will be locked out of the system if we do this. Having a central box handling reauthentication is a single point of failure that we would like to avoid, but we don't know what design pattern Erlang would use to ensure that only one of the 75 Erlang instances would attempt to reauthenticate at any one time (all 75 boxes can share the same authentication token). That depends on: 1. how many times you can try to re-authenticate before being blocked, 2. how close together they have to be. > >Central points of failures are definitely something to avoid. Leader election across 75 boxes might not be the funnest thing in the world either. I could see a scheme where you use some distributed cached value that can say "I am currently being logged" that can time out at some point, visible to all readers. When you read that timeout value from each box (possibly from an OTP Application that only handles auth), each reading of that value adds or subtracts a random number to the timeout. This is to try and avoid a cluster-wide synchronization on the timeout value, and instead have them happen at different times. You could add an "I'm updating" flag related to that value and that could give you good probabilities that only a fraction of all the nodes attempt an authentication at any point in time close to the timeout value. > >Again, this would depend on how often your authentication needs to be done, and to what frequency you're allowed to do it. > >If it's too tight, you might need a central server or node that takes care of it, with one or two fail-overs to add some reliability. > >Note you will still have to care about netsplits ruining your day with this whole scheme. > >-- I had nothing to add on the rest of the mail, so cut if off. > >Hope this helps, >Fred. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From emeka_1978@REDACTED Fri May 18 16:01:34 2012 From: emeka_1978@REDACTED (eigenfunction) Date: Fri, 18 May 2012 07:01:34 -0700 (PDT) Subject: [erlang-questions] secure use of cookies in an erlang application In-Reply-To: <6DC11414-81D5-45B6-B55E-99E0D4E35AAF@gmail.com> References: <6DC11414-81D5-45B6-B55E-99E0D4E35AAF@gmail.com> Message-ID: <76a257ea-ead2-4550-9d1d-c8f6baa6183a@l5g2000vbo.googlegroups.com> Stateless applications are just too hard to secure. I wished there was a stateful webframework written in erlang, something like java- webobjects or scala-lift where you can just write your application and go to sleep without worrying about security. The first time i saw erlang-web and its component based approach, i thought they got it. I checked their wiki page but they did not mention security anywhere so i had to look somewhere else. On May 17, 2:43?am, Steve Davis wrote: > This is also quite interesting on the topic. > > http://www.cse.msu.edu/~alexliu/publications/Cookie/cookie.pdf > > _______________________________________________ > erlang-questions mailing list > erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions From lenartlad@REDACTED Fri May 18 16:05:22 2012 From: lenartlad@REDACTED (Ladislav Lenart) Date: Fri, 18 May 2012 16:05:22 +0200 Subject: [erlang-questions] Erlang suitability In-Reply-To: <1337344156.19879.YahooMailNeo@web162102.mail.bf1.yahoo.com> References: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> <4FB63A4E.70408@ferd.ca> <1337344156.19879.YahooMailNeo@web162102.mail.bf1.yahoo.com> Message-ID: <4FB65722.6040802@volny.cz> Hello. I am by no means an Erlang expert, but... Netsplits are an inherent part of a distributed solution. Erlang or not, you will have to deal with them one way or the other. And even if some 3rd party framework takes care of them for you, they are still there. Erlang shines here, because it gives you the tools you need to build a solution tailored to your specific problem. I don't know of any other language that has builtin language constructs and libraries that deal with beasts such as netsplits and HW / SW failures. So for me, Erlang would fit nicely to your problem description. Regarding 75 nodes in a cluster... One objection was against native erlang distribution protocol (fully connected mesh of nodes), not against Erlang language + platform itself. You can fairly easily build your own comunication layer on top of TCP/IP, term_to_binary/1 and binary_to_term/1. For example you can divide 75 nodes to fully connected islands with one node responsible for communication with outside world. I believe "hidden nodes" should be of value here. I know that not a long ago, someone posted on this very list that their app runs on ~700 nodes (sorry I dont' remember more) so 75 nodes is certainly doable. Another problem is mnesia, which as some pointed to you, was not built to run on this number of nodes. Nevertheless there are other possibilities, for example riak. See basho.com for more. HTH, Ladislav Lenart On 18.5.2012 14:29, Ovid wrote: > Netsplits. Damn. I forgot to put my thinking CAP on. In this case, a netsplit > would be disastrous unless we fell back to a central data store such as Redis. > At that point, Erlang doesn't look like a solution at all. > > On top of that, you're the second person to point out that a complete graph with > 75 nodes is problematic. Now that I think about it, I guess I can see why. It > now sounds like an Erlang solution is not the quick win we thought it might be > (quelle surprise!). > > Thanks to everyone for all of your answers. > > Cheers, > Curtis > -- > Live and work overseas - http://www.overseas-exile.com/ > Buy the book - http://www.oreilly.com/catalog/perlhks/ > Tech blog - http://blogs.perl.org/users/ovid/ > Twitter - http://twitter.com/OvidPerl/ > > -------------------------------------------------------------------------------- > *From:* Fred Hebert > *To:* Ovid > *Cc:* "erlang-questions@REDACTED" > *Sent:* Friday, 18 May 2012, 14:02 > *Subject:* Re: [erlang-questions] Erlang suitability > > Answers inline. > > On 12-05-18 5:00 AM, Ovid wrote: >> Hi there, >> >> We've a system that run across 75 servers and needs to be highly >> performant, fault-tolerant, scalable and shares persistent data across all >> 75 servers. We're investigating Erlang/Mnesia (which we don't know) >> because it sounds tailor-made for our situation. > As mentioned earlier in this thread, 75 servers is a bit much, but people > have done it before. >> >> We are not using Erlang for our first implementation, but are instead >> hacking together a solution from known technologies including Perl, MySQL >> and Redis. We're considering Erlang for our future work. >> >> We have two primary needs: Each box can bid on an auction and potentially >> spend a tiny amount of money and each of the 75 boxes will receive >> notifications of a small amount of money spent if they win the auction >> (the auction notification will probably not be sent to the box bidding in >> the auction). >> >> Use case 1: If the *total* of all of those small amounts exceeds a daily >> cap or an all-time cap, all 75 boxes must immediately stop spending >> bidding in auctions. It seems that each box can run a separate Erlang >> process and write out "winning bid" information to an Mnesia database and >> all boxes can read the total amount spent from that to determine if it >> should stop bidding. >> >> This seems trivial to set up. > It isn't trivial. You have think about what happens when a box is seen as > crashing. How strongly consistent do you want things to be? There is always > a risk that a box didn't crash, but was cut off in a netsplit. You might get > divergences in budget that will be hard to explain. > > There is also a definite timing issue depending on how your data is being > observed. For example, you ask permission to bid on an item, but you do not > get instant feedback; by the time you sent maybe 5-10 bids, the cap is > finally reached and broken at once because the delay to the other network > made you keep on bidding without a final result. How much tolerance do you > have for this? > > You mentioned in another post that "We need to ensure that were all 75 boxes > to mysteriously crash, we could bring them back up and not worry about data > integrity.", Possibly, but what about 1 node only? What about 5? What about > 30 or 35? What if they crash and you missed winning bids because you went > out after bidding but before getting your notifications back (if that is > possible by the bidding rules of whatever exchange you're dealing with). > > The most solid synchronous database setup might not give you the guarantees > you expect in the first place. >> >> Use case 2: we periodically need to reauthenticate to the auction system. >> We MUST NOT have all 75 boxes trying to reauthenticate at the same time >> because we will be locked out of the system if we do this. Having a >> central box handling reauthentication is a single point of failure that we >> would like to avoid, but we don't know what design pattern Erlang would >> use to ensure that only one of the 75 Erlang instances would attempt to >> reauthenticate at any one time (all 75 boxes can share the same >> authentication token). > That depends on: 1. how many times you can try to re-authenticate before > being blocked, 2. how close together they have to be. > > Central points of failures are definitely something to avoid. Leader > election across 75 boxes might not be the funnest thing in the world either. > I could see a scheme where you use some distributed cached value that can > say "I am currently being logged" that can time out at some point, visible > to all readers. When you read that timeout value from each box (possibly > from an OTP Application that only handles auth), each reading of that value > adds or subtracts a random number to the timeout. This is to try and avoid a > cluster-wide synchronization on the timeout value, and instead have them > happen at different times. You could add an "I'm updating" flag related to > that value and that could give you good probabilities that only a fraction > of all the nodes attempt an authentication at any point in time close to the > timeout value. > > Again, this would depend on how often your authentication needs to be done, > and to what frequency you're allowed to do it. > > If it's too tight, you might need a central server or node that takes care > of it, with one or two fail-overs to add some reliability. > > Note you will still have to care about netsplits ruining your day with this > whole scheme. > > -- I had nothing to add on the rest of the mail, so cut if off. > > Hope this helps, > Fred. > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From g@REDACTED Fri May 18 17:52:04 2012 From: g@REDACTED (Garrett Smith) Date: Fri, 18 May 2012 10:52:04 -0500 Subject: [erlang-questions] Erlang suitability In-Reply-To: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> References: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> Message-ID: Hi Ovid, Thanks for the very interesting use case! On Fri, May 18, 2012 at 4:00 AM, Ovid wrote: > Hi there, > > We've a system that run across 75 servers and needs to be highly performant, > fault-tolerant, scalable and shares persistent data across all 75 servers. > We're investigating Erlang/Mnesia (which we don't know) because it sounds > tailor-made for our situation. The previous comments about 75 being "a bit much" are warranted. In distributed mode, Erlang establishes a "fully connected mesh" where each node tries to maintain connections to every other node. This N(N-1) scheme adds up quickly and caps the practical limits of these clusters. You can partition the clusters to limit their size, but you're moving into a more complex topology. It's probably worth looking at though since you get so much out of distributed Erlang for this problem. > We are not using Erlang for our first implementation, but are instead > hacking together a solution from known technologies including Perl, MySQL > and Redis. We're considering Erlang for our future work. > > We have two primary needs: Each box can bid on an auction and potentially > spend a tiny amount of money and each of the 75 boxes will receive > notifications of a small amount of money spent if they win the auction (the > auction notification will probably not be sent to the box bidding in the > auction). > > Use case 1: If the *total* of all of those small amounts exceeds a daily cap > or an all-time cap, all 75 boxes must immediately stop spending bidding in > auctions. It seems that each box can run a separate Erlang process and write > out "winning bid" information to an Mnesia database and all boxes can read > the total amount spent from that to determine if it should stop bidding. You might also look at Riak for this. Also, something like doozerd. Or a consensus algorithm like Paxos, if you don't mind the extra work in building smarter clients. > This seems trivial to set up. > > Use case 2: we periodically need to reauthenticate to the auction system. We > MUST NOT have all 75 boxes trying to reauthenticate at the same time because > we will be locked out of the system if we do this. Having a central box > handling reauthentication is a single point of failure that we would like to > avoid, but we don't know what design pattern Erlang would use to ensure that > only one of the 75 Erlang instances would attempt to reauthenticate at any > one time (all 75 boxes can share the same authentication token). This could also be framed as a consensus problem -- you'd elect a node to reauthenticate among a set of candidates. > Use case 1 is clearly perfect for Erlang. Use case 2 is less clear to us. > We're going to be spending a fair amount of time hacking together a > non-Erlang solution, but the benefit is using known technologies that don't > depend on the "one guy who knows the system". This is a great approach! One tech you might look at to glue these pieces together is 0MQ. This would let you use whatever languages you'd want, hacking something that just works, then replace pieces as needed without breaking your network protocols. You'll have a learning curve using 0MQ, but this is because you'll be learning core messaging patterns, not so much 0MQ itself, which is very simple. I think all of that learning will be indispensable for your team as you move along. > At full scale, we anticipate billions of auctions per day. > > Does anyone care to comment about things we should look at? Particularly, is > use case 2 not something appropriate for Erlang? A tiny amount of sample > code would be lovely (note: I am somewhat comforable with Prolog, so Erlang > looks fairly straightforward for me, aside from understanding the message > passing). If you haven't already, take a look at this excellent document from Google: http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/archive/chubby-osdi06.pdf Distributed algorithms are a PITA (for me anyway) and the Chubby lock service is a great "cheat" that simplifies a lot of the harder problems. Doozer is one of the few implementations of something Chubby-like that I'm aware of. > Cheers, > Ovid Best of luck! Garrett From erlang@REDACTED Fri May 18 18:00:27 2012 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 18 May 2012 18:00:27 +0200 Subject: [erlang-questions] Erlang suitability In-Reply-To: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> References: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> Message-ID: It's a bit difficult to answer your questions since you don't say much about time. Is the total time of an auction a millisecond or a day (or longer) - I have no idea and don't like to guess. What are the timing requirements for an individual bid in an auction? How many bids before an auction is closed? Re authentication - how often is periodically? - every hour / every day / every second not knowing makes answering your questions very diffcult. If re authentication is every 75 minutes - then you could authenticate machine one on the first minute, machine 2 on the second minute and so on.... Erlang/mensia is not magic you still need to think out your algorithms in detail first ... Cheers /Joe On Fri, May 18, 2012 at 11:00 AM, Ovid wrote: > Hi there, > > We've a system that run across 75 servers and needs to be highly performant, > fault-tolerant, scalable and shares persistent data across all 75 servers. > We're investigating Erlang/Mnesia (which we don't know) because it sounds > tailor-made for our situation. > > We are not using Erlang for our first implementation, but are instead > hacking together a solution from known technologies including Perl, MySQL > and Redis. We're considering Erlang for our future work. > > We have two primary needs: Each box can bid on an auction and potentially > spend a tiny amount of money and each of the 75 boxes will receive > notifications of a small amount of money spent if they win the auction (the > auction notification will probably not be sent to the box bidding in the > auction). > > Use case 1: If the *total* of all of those small amounts exceeds a daily cap > or an all-time cap, all 75 boxes must immediately stop spending bidding in > auctions. It seems that each box can run a separate Erlang process and write > out "winning bid" information to an Mnesia database and all boxes can read > the total amount spent from that to determine if it should stop bidding. > > This seems trivial to set up. > > Use case 2: we periodically need to reauthenticate to the auction system. We > MUST NOT have all 75 boxes trying to reauthenticate at the same time because > we will be locked out of the system if we do this. Having a central box > handling reauthentication is a single point of failure that we would like to > avoid, but we don't know what design pattern Erlang would use to ensure that > only one of the 75 Erlang instances would attempt to reauthenticate at any > one time (all 75 boxes can share the same authentication token). > > Use case 1 is clearly perfect for Erlang. Use case 2 is less clear to us. > We're going to be spending a fair amount of time hacking together a > non-Erlang solution, but the benefit is using known technologies that don't > depend on the "one guy who knows the system". > > At full scale, we anticipate billions of auctions per day. > > Does anyone care to comment about things we should look at? Particularly, is > use case 2 not something appropriate for Erlang? A tiny amount of sample > code would be lovely (note: I am somewhat comforable with Prolog, so Erlang > looks fairly straightforward for me, aside from understanding the message > passing). > > Cheers, > Ovid > -- > Live and work overseas - http://www.overseas-exile.com/ > Buy the book - http://www.oreilly.com/catalog/perlhks/ > Tech blog - http://blogs.perl.org/users/ovid/ > Twitter - http://twitter.com/OvidPerl/ > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ok@REDACTED Fri May 18 18:17:45 2012 From: ok@REDACTED (ok@REDACTED) Date: Sat, 19 May 2012 04:17:45 +1200 Subject: [erlang-questions] Erlang suitability In-Reply-To: References: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> Message-ID: <6bdccf288097f802d1da8c14f8961c2f.squirrel@chasm.otago.ac.nz> > On Fri, May 18, 2012 at 4:00 AM, Ovid wrote: >> Use case 2: we periodically need to reauthenticate to the auction >> system. We >> MUST NOT have all 75 boxes trying to reauthenticate at the same time >> because >> we will be locked out of the system if we do this. Having a central box >> handling reauthentication is a single point of failure that we would >> like to >> avoid, but we don't know what design pattern Erlang would use to ensure >> that >> only one of the 75 Erlang instances would attempt to reauthenticate at >> any >> one time (all 75 boxes can share the same authentication token). How *often* is "periodically? What would happen if each of your boxes just chose times at random? If you have to re-authenticate once an hour, each box could be given two 24-second windows to use (time division multiplexing, as it were). You can probably keep the clocks sufficiently synchronised to make that work. Could you use a scheme where each node holds a token and uses gproc:give_away/2 to pass it to the next node? From g@REDACTED Fri May 18 18:24:37 2012 From: g@REDACTED (Garrett Smith) Date: Fri, 18 May 2012 11:24:37 -0500 Subject: [erlang-questions] Erlang suitability In-Reply-To: <6bdccf288097f802d1da8c14f8961c2f.squirrel@chasm.otago.ac.nz> References: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> <6bdccf288097f802d1da8c14f8961c2f.squirrel@chasm.otago.ac.nz> Message-ID: On Fri, May 18, 2012 at 11:17 AM, wrote: > >> On Fri, May 18, 2012 at 4:00 AM, Ovid wrote: >>> Use case 2: we periodically need to reauthenticate to the auction >>> system. -snip- > How *often* is "periodically? > What would happen if each of your boxes just chose times at random? > If you have to re-authenticate once an hour, each box could be given > two 24-second windows to use (time division multiplexing, as it were). > You can probably keep the clocks sufficiently synchronised to make that > work. > > Could you use a scheme where each node holds a token and uses > gproc:give_away/2 to pass it to the next node? Hah! I forgot about gproc's leader election features. I know that was "in process" for a while -- can anyone share recent experience with it? Garrett From richardprideauxevans@REDACTED Fri May 18 18:30:00 2012 From: richardprideauxevans@REDACTED (Richard Evans) Date: Fri, 18 May 2012 17:30:00 +0100 Subject: [erlang-questions] Deterministic playback of a simulation? In-Reply-To: References: Message-ID: Thanks guys for all your helpful responses :) On Fri, May 18, 2012 at 12:38 PM, Ulf Wiger wrote: > > On 18 May 2012, at 04:36, Richard Evans wrote: > >> So my question is: is there a way to prevent the scheduler from >> yielding during a block of code? Some way to insist the block is >> called as one unit, like this: >> >> call_and_store(Fun, Args) -> >> ? ? ? !!!prevent_yielding, >> ? ?add_to_script(Fun, Args); >> ? ?apply(lobby, Fun, Args), >> ? ? ? !!!allow_yielding. >> >> My guess is: no. This does not seem very erlangy. > > No, not at that level. > > But you could use e.g. > > ?global:trans({call_and_store, self()}, fun() -> call_and_store(Fun, Args) end). > > See erl -man global > > You can specify on how many nodes you want to acquire a lock - default is all connected nodes, but you can set it e.g. to [node()] if that suits you better. > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > From ulf@REDACTED Fri May 18 18:39:28 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 18 May 2012 18:39:28 +0200 Subject: [erlang-questions] Erlang suitability In-Reply-To: References: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> <6bdccf288097f802d1da8c14f8961c2f.squirrel@chasm.otago.ac.nz> Message-ID: On 18 May 2012, at 18:24, Garrett Smith wrote: >> Could you use a scheme where each node holds a token and uses >> gproc:give_away/2 to pass it to the next node? > > Hah! I forgot about gproc's leader election features. > > I know that was "in process" for a while -- can anyone share recent > experience with it? Well, it is about as good as the latest and greatest gen_leader. ;-) Gproc relies on gen_leader, and simply passes on the start options to it. Some versions of gen_leader (I believe you have the fork that's currently 'leading the pack' in the github network graph ;-) However, gproc uses full replication, so gproc:give_away/2 will involve all 75 nodes - not just the two. BR, Ulf W PS Since gen_leader hasn't had a strategy for resolving netsplits (i.e. merging data and resolving inconsistencies), gproc has no support for this either. Does any of the latest gen_leader versions have something that gproc can use? Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From ok@REDACTED Fri May 18 18:42:24 2012 From: ok@REDACTED (ok@REDACTED) Date: Sat, 19 May 2012 04:42:24 +1200 Subject: [erlang-questions] Question/Alternative on Frames Proposal [Warning: Long] In-Reply-To: <1337300505.68472.YahooMailNeo@web130106.mail.mud.yahoo.com> References: <4FB575D5.2020702@gmail.com> <1337300505.68472.YahooMailNeo@web130106.mail.mud.yahoo.com> Message-ID: <1338fcafd15281751fb6491bff9b6109.squirrel@chasm.otago.ac.nz> If I trimmed the frames proposal down to just say 10 pages, presenting the design goals (and non goals), the basic semantics of the data structure, how it can be implemented in a BEAM-like architecture, and some measurements from my mini-BEAM, is that something that might be of interest for the next Erlang conference? From thomas.burdick@REDACTED Fri May 18 18:43:47 2012 From: thomas.burdick@REDACTED (Tom Burdick) Date: Fri, 18 May 2012 11:43:47 -0500 Subject: [erlang-questions] Ruby and open_port Message-ID: Hello, I'm trying to use ruby as an erlang port using erlectricity which nicely uses nouse_stdio for the port (fd's 3 and 4). In Ruby 1.9.3 fd 3 at least is now reserved by the ruby vm. I don't want to use stdio if I can help it. Is there a nice way to setup a custom pipe using open_port? I see it can take {fd, In, Out} but I wasn't sure how I might set that up where it spawns and sets up a different pipe pair. Any help would be appreciated! Tom From thomas.burdick@REDACTED Fri May 18 22:13:38 2012 From: thomas.burdick@REDACTED (Tom Burdick) Date: Fri, 18 May 2012 15:13:38 -0500 Subject: [erlang-questions] Carriage Return Message-ID: So after hunting around in otp source code I found this little gem, erts/emulator/sys/unix/sys.c: erts_printf_add_cr_to_stdout = 1; erts/emulator/sys/unix/sys.c: erts_printf_add_cr_to_stderr = 1; Is there any particular reason this is left to this day? It makes working with ports on linux rather annoying when they print out their stack traces and other things. I can always build a custom erl for myself without this but would that break other things? Thanks! Tom From mjtruog@REDACTED Fri May 18 22:34:32 2012 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 18 May 2012 13:34:32 -0700 Subject: [erlang-questions] Ruby and open_port In-Reply-To: References: Message-ID: <4FB6B258.50206@gmail.com> In CloudI (http://cloudi.org) that is done for you, so you can still receive stdout/stderr within the CloudI log. Within the CloudI code, it is using an erlang port that does the pipe handling and erlectricity is probably not yet doing this. The port code which is used in CloudI is part of GEPD (https://github.com/okeuday/generic-erlang-port--driver-) which makes it a bit simpler if you are using standard types for your port (and if you don't care about the dependency on Boost). On 05/18/2012 09:43 AM, Tom Burdick wrote: > Hello, > > I'm trying to use ruby as an erlang port using erlectricity which > nicely uses nouse_stdio for the port (fd's 3 and 4). > > In Ruby 1.9.3 fd 3 at least is now reserved by the ruby vm. I don't > want to use stdio if I can help it. Is there a nice way to setup a > custom pipe using open_port? I see it can take {fd, In, Out} but I > wasn't sure how I might set that up where it spawns and sets up a > different pipe pair. > > Any help would be appreciated! > > Tom > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From g@REDACTED Fri May 18 22:57:00 2012 From: g@REDACTED (Garrett Smith) Date: Fri, 18 May 2012 15:57:00 -0500 Subject: [erlang-questions] Question/Alternative on Frames Proposal [Warning: Long] In-Reply-To: <1338fcafd15281751fb6491bff9b6109.squirrel@chasm.otago.ac.nz> References: <4FB575D5.2020702@gmail.com> <1337300505.68472.YahooMailNeo@web130106.mail.mud.yahoo.com> <1338fcafd15281751fb6491bff9b6109.squirrel@chasm.otago.ac.nz> Message-ID: On Fri, May 18, 2012 at 11:42 AM, wrote: > If I trimmed the frames proposal down to just say 10 pages, > presenting the design goals (and non goals), the basic semantics > of the data structure, how it can be implemented in a BEAM-like > architecture, and some measurements from my mini-BEAM, is that > something that might be of interest for the next Erlang conference? I apologize, this is somewhat off topic and not an answer to your question... But does anyone know where the Frames proposal stands vis-a-vis the OTP team's work on hashes? My reference point is Kenneth's presentation last year: http://www.erlang-factory.com/upload/presentations/468/EUC_Hashes2011.pdf Garrett From g@REDACTED Fri May 18 23:30:20 2012 From: g@REDACTED (Garrett Smith) Date: Fri, 18 May 2012 16:30:20 -0500 Subject: [erlang-questions] gproc and gen_event In-Reply-To: References: Message-ID: On Thu, May 17, 2012 at 7:14 PM, Andrew Berman wrote: > Hello, > > I am trying to set up a pub-sub type situation where I have a function > called which does some stuff and at the end broadcasts a message. ?I would > like for multiple handlers to be able to receive that message and do > something with it. ?I'm currently using gen_event, but have been reading > about gproc as well. ?Can anyone explain why they'd use one over the other > for pub/sub? gen_event is useful when you need to support "pluggable" functionality and you can't tolerate the overhead of Erlang message passing. gen_event runs all handlers in a single process. If one of those subscribers misbehaves, it's unceremoniously removed. To give you an idea: Eshell V5.8.5 (abort with ^G) 1> error_logger:info_msg("say hello"). =INFO REPORT==== 18-May-2012::16:13:25 === say hellook 2> error_logger:info_msg([{"say", "goodbye"}]). ok 3> error_logger:info_msg("anyone?"). ok error_logger uses gen_event -- tty support is a handler. Bugs, as the one above, result in silent changes to your program! While you can use add_handler_sup to get notification of a handler delete, I find the whole facility easy to get wrong and so I avoid it. But it's not hard to create a publisher in yourself. e2 [1] has something you can steal from, or just use: https://github.com/gar1t/e2/blob/master/src/e2_publisher.erl Here's a silly example of a "broker" that decorates published messages with some tag: https://github.com/gar1t/e2/blob/master/examples/pubsub/src/broker.erl The advantage of pubsub using Erlang messages over gen_event is that you can implement more natural supervision -- and you're not going to get silent, mysterious breakage. The price is that you're sending Erlang messages. If you can't afford that (e.g. perhaps high throughput logging) gen_event is better. Garrett [1] http://e2project.org/ From milindparikh@REDACTED Fri May 18 23:38:57 2012 From: milindparikh@REDACTED (Milind Parikh) Date: Fri, 18 May 2012 14:38:57 -0700 Subject: [erlang-questions] Gproc starting issue Message-ID: application:start(sasl). application:start(gproc) fails with log attached below. I have 0. Downloaded the latest copy of gproc (from github) 1. Downloaded the latest copy of rebar (from github) 2. copied the latest rebar into gproc 3. ./rebar get-deps in gproc 4. ./rebar compile in gproc. and am trying to follow http://rustyklophaus.com/articles/20090916-GProcErlangGlobalProcessRegistry.html I must be missing something trivial. Any help would be appreciated. Thanks Milind +++++++++++++++ LOG ++++++++++++++++++++++++++++ (a@REDACTED)3> application:start(gproc). =SUPERVISOR REPORT==== 18-May-2012::14:12:25 === Supervisor: {local,gproc_sup} Context: start_error Reason: {'EXIT', {badarg, [{ets,new, [gproc, [ordered_set,public,named_table, {write_concurrency,true}, {read_concurrency,true}]]}, {gproc,create_tabs,0}, {gproc,start_link,0}, {supervisor,do_start_child,2}, {supervisor,start_children,3}, {supervisor,init_children,2}, {gen_server,init_it,6}, {proc_lib,init_p_do_apply,3}]}} Offender: [{pid,undefined}, {name,gproc}, {mfa,{gproc,start_link,[]}}, {restart_type,permanent}, {shutdown,2000}, {child_type,worker}] =INFO REPORT==== 18-May-2012::14:12:25 === application: gproc exited: {shutdown,{gproc_app,start,[normal,[]]}} type: temporary {error,{shutdown,{gproc_app,start,[normal,[]]}}} -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Fri May 18 23:47:56 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 18 May 2012 23:47:56 +0200 Subject: [erlang-questions] Gproc starting issue In-Reply-To: References: Message-ID: <9D331027-4669-4A36-917D-1541434133A0@feuerlabs.com> What were commands (1) and (2)? uwair:0518 uwiger$ git clone git://github.com/esl/gproc.git Cloning into gproc... remote: Counting objects: 1042, done. remote: Compressing objects: 100% (293/293), done. remote: Total 1042 (delta 754), reused 995 (delta 715) Receiving objects: 100% (1042/1042), 1.03 MiB | 324 KiB/s, done. Resolving deltas: 100% (754/754), done. uwair:0518 uwiger$ cd gproc uwair:gproc uwiger$ rebar get-deps compile ==> gproc (get-deps) ? ==> gproc (compile) Compiled src/gproc_sup.erl Compiled src/gproc_ps.erl Compiled src/gproc_init.erl Compiled src/gproc_monitor.erl Compiled src/gproc_info.erl Compiled src/gproc_app.erl Compiled src/gproc_lib.erl Compiled src/gproc_dist.erl Compiled src/gproc.erl uwair:gproc uwiger$ unset ERL_LIBS uwair:gproc uwiger$ erl -pa ebin Erlang R15B (erts-5.9) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9 (abort with ^G) 1> application:start(gproc). ok 2> gproc:reg({n,l,foo}). true 3> gproc:where({n,l,foo}). <0.31.0> A possibility for the badarg exception would be that the table already exists. What version of Erlang/OTP are you using? I believe the read_concurrency & write_concurrency options were added to ETS in R14B. BR, Ulf W On 18 May 2012, at 23:38, Milind Parikh wrote: > application:start(sasl). > application:start(gproc) fails with log attached below. > > I have > > 0. Downloaded the latest copy of gproc (from github) > 1. Downloaded the latest copy of rebar (from github) > 2. copied the latest rebar into gproc > 3. ./rebar get-deps in gproc > 4. ./rebar compile in gproc. > > and am trying to follow http://rustyklophaus.com/articles/20090916-GProcErlangGlobalProcessRegistry.html > > I must be missing something trivial. Any help would be appreciated. > > Thanks > Milind > > > > +++++++++++++++ LOG ++++++++++++++++++++++++++++ > > (a@REDACTED)3> application:start(gproc). > > =SUPERVISOR REPORT==== 18-May-2012::14:12:25 === > Supervisor: {local,gproc_sup} > Context: start_error > Reason: {'EXIT', > {badarg, > [{ets,new, > [gproc, > [ordered_set,public,named_table, > {write_concurrency,true}, > {read_concurrency,true}]]}, > {gproc,create_tabs,0}, > {gproc,start_link,0}, > {supervisor,do_start_child,2}, > {supervisor,start_children,3}, > {supervisor,init_children,2}, > {gen_server,init_it,6}, > {proc_lib,init_p_do_apply,3}]}} > Offender: [{pid,undefined}, > {name,gproc}, > {mfa,{gproc,start_link,[]}}, > {restart_type,permanent}, > {shutdown,2000}, > {child_type,worker}] > > > =INFO REPORT==== 18-May-2012::14:12:25 === > application: gproc > exited: {shutdown,{gproc_app,start,[normal,[]]}} > type: temporary > {error,{shutdown,{gproc_app,start,[normal,[]]}}} > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Sat May 19 00:02:57 2012 From: g@REDACTED (Garrett Smith) Date: Fri, 18 May 2012 17:02:57 -0500 Subject: [erlang-questions] Advice on meta programming problem Message-ID: I want to register a module that implements a callback interface. I don't want to use Erlang processes for this case -- otherwise a custom OTP behavior would suffice I want to use the module through an abstract layer like this: foo:do_something() I want to register a particular implementation of "foo" in application config like this: [{my_app, [{registered_foo, my_foo}]}]. my_foo would export do_something/0, which would be called by foo:do_someting/0. There are no registered processes here. I know I can use parameterized modules: -module(foo, [M]). -export([do_something/0]). do_something() -> M:do_something(). So easy! This does require some "factory" support to setup the parameterized module from application config. Not a big deal though, at all. Then there's my irrational, emotional hang-up of using parameterized modules, which is a more serious problem. I'm tempted to assemble a module at runtime using the standard metadata conventions in Erlang (code, compile etc. modules). foo would look like this: -module(foo). -export([do_something/0]). do_something() -> foo_impl:do_something(). foo_impl would not exist at compile time -- it would be generated at runtime using the application config. This is a very simple example -- more complex scenarios could exist, such as multiple implementation modules providing functionality. I'm curious to hear general reactions on this. Obviously parameterized modules fit -- but I'd like to explore some other approaches as well. Garrett From rexxe98@REDACTED Sat May 19 01:38:33 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 18 May 2012 16:38:33 -0700 Subject: [erlang-questions] gproc and gen_event In-Reply-To: References: Message-ID: Ok, cool, thanks Garrett. Do you have any experience using gproc for pub/sub? Nice email address, btw :-) Andrew On Fri, May 18, 2012 at 2:30 PM, Garrett Smith wrote: > On Thu, May 17, 2012 at 7:14 PM, Andrew Berman wrote: > > Hello, > > > > I am trying to set up a pub-sub type situation where I have a function > > called which does some stuff and at the end broadcasts a message. I > would > > like for multiple handlers to be able to receive that message and do > > something with it. I'm currently using gen_event, but have been reading > > about gproc as well. Can anyone explain why they'd use one over the > other > > for pub/sub? > > gen_event is useful when you need to support "pluggable" functionality > and you can't tolerate the overhead of Erlang message passing. > > gen_event runs all handlers in a single process. If one of those > subscribers misbehaves, it's unceremoniously removed. > > To give you an idea: > > Eshell V5.8.5 (abort with ^G) > 1> error_logger:info_msg("say hello"). > > =INFO REPORT==== 18-May-2012::16:13:25 === > say hellook > 2> error_logger:info_msg([{"say", "goodbye"}]). > ok > 3> error_logger:info_msg("anyone?"). > ok > > error_logger uses gen_event -- tty support is a handler. Bugs, as the > one above, result in silent changes to your program! > > While you can use add_handler_sup to get notification of a handler > delete, I find the whole facility easy to get wrong and so I avoid it. > > But it's not hard to create a publisher in yourself. e2 [1] has > something you can steal from, or just use: > > https://github.com/gar1t/e2/blob/master/src/e2_publisher.erl > > Here's a silly example of a "broker" that decorates published messages > with some tag: > > https://github.com/gar1t/e2/blob/master/examples/pubsub/src/broker.erl > > The advantage of pubsub using Erlang messages over gen_event is that > you can implement more natural supervision -- and you're not going to > get silent, mysterious breakage. The price is that you're sending > Erlang messages. If you can't afford that (e.g. perhaps high > throughput logging) gen_event is better. > > Garrett > > [1] http://e2project.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sat May 19 02:52:13 2012 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 18 May 2012 17:52:13 -0700 Subject: [erlang-questions] gproc and gen_event In-Reply-To: References: Message-ID: <4FB6EEBD.6070309@gmail.com> Hi Andrew, gen_event is an OTP behavior for responding to events, like logging, database drivers, etc. (http://www.erlang.org/doc/design_principles/events.html). The only different part is the supervisor wants "dynamic" in the child spec (http://www.erlang.org/doc/design_principles/sup_princ.html#id71666). gproc is a way of having centralized storage among Erlang nodes. However, if you read here http://erlang.org/pipermail/erlang-questions/2010-April/051030.html , you will discover: "As stated before, gen_leader is a bit clumsy in environments where nodes are dynamically added and removed. Changing the participants of a gen_leader cluster is fairly difficult. The gen_leader_revival project tries to merge a number of existing modifications to gen_leader to make it more flexible." (i.e., where gen_leader is the guts behind gproc). So, if you don't have netsplits, then gproc should be fine, but I believe mnesia fits into that category as well. pg2 is one you have ignored, which replicates process groups among the collected nodes, storing them in ETS. I chose an approach based on pg2, but without ETS in CloudI (http://cloudi.org), here: https://github.com/okeuday/CloudI/blob/master/src/lib/cloudi/src/list_pg.erl . This makes more sense with CloudI focusing on the AP part of the CAP theorem, whereas gproc is more focused on CA. I find the approach in CloudI better, since it avoids global state and that is the main point of using Erlang (with its Actor model). However, I am very biased, since I wrote it. Regards, Michael On 05/17/2012 05:14 PM, Andrew Berman wrote: > Hello, > > I am trying to set up a pub-sub type situation where I have a function called which does some stuff and at the end broadcasts a message. I would like for multiple handlers to be able to receive that message and do something with it. I'm currently using gen_event, but have been reading about gproc as well. Can anyone explain why they'd use one over the other for pub/sub? > > Thanks, > > Andrew > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.eugene.turner@REDACTED Sat May 19 09:24:17 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sat, 19 May 2012 16:24:17 +0900 Subject: [erlang-questions] Erlang suitability In-Reply-To: <4FB63A4E.70408@ferd.ca> References: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> <4FB63A4E.70408@ferd.ca> Message-ID: "As mentioned earlier in this thread, 75 servers is a bit much, but people have done it before." What I haven't seen mentioned so far: depending on your application, reimplementing it in Erlang might mean that you no longer need nearly as many as 75 servers. -michael turner On Fri, May 18, 2012 at 9:02 PM, Fred Hebert wrote: > Answers inline. > > > On 12-05-18 5:00 AM, Ovid wrote: > > Hi there, > > We've a system that run across 75 servers and needs to be highly performant, > fault-tolerant, scalable and shares persistent data across all 75 servers. > We're investigating Erlang/Mnesia (which we don't know) because it sounds > tailor-made for our situation. > > As mentioned earlier in this thread, 75 servers is a bit much, but people > have done it before. > > > We are not using Erlang for our first implementation, but are instead > hacking together a solution from known technologies including Perl, MySQL > and Redis. We're considering Erlang for our future work. > > We have two primary needs: Each box can bid on an auction and potentially > spend a tiny amount of money and each of the 75 boxes will receive > notifications of a small amount of money spent if they win the auction (the > auction notification will probably not be sent to the box bidding in the > auction). > > Use case 1: If the *total* of all of those small amounts exceeds a daily cap > or an all-time cap, all 75 boxes must immediately stop spending bidding in > auctions. It seems that each box can run a separate Erlang process and write > out "winning bid" information to an Mnesia database and all boxes can read > the total amount spent from that to determine if it should stop bidding. > > This seems trivial to set up. > > It isn't trivial. You have think about what happens when a box is seen as > crashing. How strongly consistent do you want things to be? There is always > a risk that a box didn't crash, but was cut off in a netsplit. You might get > divergences in budget that will be hard to explain. > > There is also a definite timing issue depending on how your data is being > observed. For example, you ask permission to bid on an item, but you do not > get instant feedback; by the time you sent maybe 5-10 bids, the cap is > finally reached and broken at once because the delay to the other network > made you keep on bidding without a final result. How much tolerance do you > have for this? > > You mentioned in another post that "We need to ensure that were all 75 boxes > to mysteriously crash, we could bring them back up and not worry about data > integrity.", Possibly, but what about 1 node only? What about 5? What about > 30 or 35? What if they crash and you missed winning bids because you went > out after bidding but before getting your notifications back (if that is > possible by the bidding rules of whatever exchange you're dealing with). > > The most solid synchronous database setup might not give you the guarantees > you expect in the first place. > > > Use case 2: we periodically need to reauthenticate to the auction system. We > MUST NOT have all 75 boxes trying to reauthenticate at the same time because > we will be locked out of the system if we do this. Having a central box > handling reauthentication is a single point of failure that we would like to > avoid, but we don't know what design pattern Erlang would use to ensure that > only one of the 75 Erlang instances would attempt to reauthenticate at any > one time (all 75 boxes can share the same authentication token). > > That depends on: 1. how many times you can try to re-authenticate before > being blocked, 2. how close together they have to be. > > Central points of failures are definitely something to avoid. Leader > election across 75 boxes might not be the funnest thing in the world either. > I could see a scheme where you use some distributed cached value that can > say "I am currently being logged" that can time out at some point, visible > to all readers. When you read that timeout value from each box (possibly > from an OTP Application that only handles auth), each reading of that value > adds or subtracts a random number to the timeout. This is to try and avoid a > cluster-wide synchronization on the timeout value, and instead have them > happen at different times. You could add an "I'm updating" flag related to > that value and that could give you good probabilities that only a fraction > of all the nodes attempt an authentication at any point in time close to the > timeout value. > > Again, this would depend on how often your authentication needs to be done, > and to what frequency you're allowed to do it. > > If it's too tight, you might need a central server or node that takes care > of it, with one or two fail-overs to add some reliability. > > Note you will still have to care about netsplits ruining your day with this > whole scheme. > > -- I had nothing to add on the rest of the mail, so cut if off. > > Hope this helps, > Fred. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From milindparikh@REDACTED Sat May 19 09:39:32 2012 From: milindparikh@REDACTED (Milind Parikh) Date: Sat, 19 May 2012 00:39:32 -0700 Subject: [erlang-questions] Gproc starting issue In-Reply-To: <9D331027-4669-4A36-917D-1541434133A0@feuerlabs.com> References: <9D331027-4669-4A36-917D-1541434133A0@feuerlabs.com> Message-ID: Ulf Thanks ... The issue was my version of Erlang (R13) instead of R15B (which is required). Had to recompile from source on Ubuntu 10.04 LTS... but it's all working now. Regards Milind On Fri, May 18, 2012 at 2:47 PM, Ulf Wiger wrote: > > What were commands (1) and (2)? > > uwair:0518 uwiger$ git clone git://github.com/esl/gproc.git > Cloning into gproc... > remote: Counting objects: 1042, done. > remote: Compressing objects: 100% (293/293), done. > remote: Total 1042 (delta 754), reused 995 (delta 715) > Receiving objects: 100% (1042/1042), 1.03 MiB | 324 KiB/s, done. > Resolving deltas: 100% (754/754), done. > uwair:0518 uwiger$ cd gproc > uwair:gproc uwiger$ rebar get-deps compile > ==> gproc (get-deps) > ? > ==> gproc (compile) > Compiled src/gproc_sup.erl > Compiled src/gproc_ps.erl > Compiled src/gproc_init.erl > Compiled src/gproc_monitor.erl > Compiled src/gproc_info.erl > Compiled src/gproc_app.erl > Compiled src/gproc_lib.erl > Compiled src/gproc_dist.erl > Compiled src/gproc.erl > uwair:gproc uwiger$ unset ERL_LIBS > uwair:gproc uwiger$ erl -pa ebin > Erlang R15B (erts-5.9) [source] [64-bit] [smp:4:4] [async-threads:0] > [hipe] [kernel-poll:false] > > Eshell V5.9 (abort with ^G) > 1> application:start(gproc). > ok > 2> gproc:reg({n,l,foo}). > true > 3> gproc:where({n,l,foo}). > <0.31.0> > > A possibility for the badarg exception would be that the table already > exists. > > What version of Erlang/OTP are you using? > I believe the read_concurrency & write_concurrency options were added to > ETS in R14B. > > BR, > Ulf W > > On 18 May 2012, at 23:38, Milind Parikh wrote: > > application:start(sasl). > application:start(gproc) fails with log attached below. > > I have > > 0. Downloaded the latest copy of gproc (from github) > 1. Downloaded the latest copy of rebar (from github) > 2. copied the latest rebar into gproc > 3. ./rebar get-deps in gproc > 4. ./rebar compile in gproc. > > and am trying to follow > http://rustyklophaus.com/articles/20090916-GProcErlangGlobalProcessRegistry.html > > I must be missing something trivial. Any help would be appreciated. > > Thanks > Milind > > > > +++++++++++++++ LOG ++++++++++++++++++++++++++++ > > (a@REDACTED)3> application:start(gproc). > > =SUPERVISOR REPORT==== 18-May-2012::14:12:25 === > Supervisor: {local,gproc_sup} > Context: start_error > Reason: {'EXIT', > {badarg, > [{ets,new, > [gproc, > [ordered_set,public,named_table, > {write_concurrency,true}, > {read_concurrency,true}]]}, > {gproc,create_tabs,0}, > {gproc,start_link,0}, > {supervisor,do_start_child,2}, > {supervisor,start_children,3}, > {supervisor,init_children,2}, > {gen_server,init_it,6}, > {proc_lib,init_p_do_apply,3}]}} > Offender: [{pid,undefined}, > {name,gproc}, > {mfa,{gproc,start_link,[]}}, > {restart_type,permanent}, > {shutdown,2000}, > {child_type,worker}] > > > =INFO REPORT==== 18-May-2012::14:12:25 === > application: gproc > exited: {shutdown,{gproc_app,start,[normal,[]]}} > type: temporary > {error,{shutdown,{gproc_app,start,[normal,[]]}}} > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Sat May 19 11:40:34 2012 From: ulf@REDACTED (Ulf Wiger) Date: Sat, 19 May 2012 11:40:34 +0200 Subject: [erlang-questions] gproc and gen_event In-Reply-To: References: Message-ID: <460CFB0C-BF96-4BD6-8135-FE2497FF971B@feuerlabs.com> On 19 May 2012, at 01:38, Andrew Berman wrote: > Ok, cool, thanks Garrett. > > Do you have any experience using gproc for pub/sub? Gproc does do pub/sub fairly well, but if I may, the main purpose of gproc initially was to serve as a form of "property index" for processes. It had dawned on me that complex signaling applications tended to end up with a lot of code that basically provided different sorts of mappings to allow you to locate the right processes. Having so many different mappings complicated debugging, and obscured many aspects of the architecture. When we started using (the predecessor of) gproc in some Ericsson projects, there was significant code reduction. The nice thing was that the developers picked it up themselves; there wasn't really any central decision to go with it. One of the things I like about gproc is that it allows you to publish the 'interaction features' of a process: registered aliases say much about what services a process can perform, and properties illustrate its dependencies on other processes. The registry can be queried with select operations or QLC, or simply with gproc:i() from the shell. In short, there are many fine ways of doing pub/sub, but if you use gproc consistently for this sort of thing, and similar things, it can help clarify runtime aspects of your system, making it more transparent and easier to debug. This also explains why gproc relies on a central registry (which is a challenge in the global case). The registry itself is much of the point. Of course, for a distributed pub/sub, you don't really need to turn on the distributed parts of gproc. Running a local gproc instance on each node, you can publish like so: publish(Event, Data) -> rpc:eval_everywhere( gproc, send, [{p, l, {?MODULE, Event}}, {?MODULE, Event, Data}]). BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Sat May 19 15:57:40 2012 From: garazdawi@REDACTED (Lukas Larsson) Date: Sat, 19 May 2012 15:57:40 +0200 Subject: [erlang-questions] Advice on meta programming problem In-Reply-To: References: Message-ID: If you are ok with using a "factory" to create a param module, you could just as well use it to create a first argument to pass to the function. i.e. -module(foo). create_context() -> application:get_env(myapp,registered_foo). do_something(Context) -> Context:do_something(). unless of course you have a reason for not wanting the parameter to be explicitly passed? Lukas On Sat, May 19, 2012 at 12:02 AM, Garrett Smith wrote: > I want to register a module that implements a callback interface. I > don't want to use Erlang processes for this case -- otherwise a custom > OTP behavior would suffice > > I want to use the module through an abstract layer like this: > > ? ?foo:do_something() > > I want to register a particular implementation of "foo" in application > config like this: > > ? ?[{my_app, [{registered_foo, my_foo}]}]. > > my_foo would export do_something/0, which would be called by foo:do_someting/0. > > There are no registered processes here. > > I know I can use parameterized modules: > > ? ?-module(foo, [M]). > > ? ?-export([do_something/0]). > > ? ?do_something() -> M:do_something(). > > So easy! > > This does require some "factory" support to setup the parameterized > module from application config. Not a big deal though, at all. > > Then there's my irrational, emotional hang-up of using parameterized > modules, which is a more serious problem. > > I'm tempted to assemble a module at runtime using the standard > metadata conventions in Erlang (code, compile etc. modules). > > foo would look like this: > > ? ?-module(foo). > > ? ?-export([do_something/0]). > > ? ?do_something() -> foo_impl:do_something(). > > foo_impl would not exist at compile time -- it would be generated at > runtime using the application config. > > This is a very simple example -- more complex scenarios could exist, > such as multiple implementation modules providing functionality. > > I'm curious to hear general reactions on this. Obviously parameterized > modules fit -- but I'd like to explore some other approaches as well. > > Garrett > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From andre@REDACTED Sat May 19 16:53:53 2012 From: andre@REDACTED (=?ISO-8859-1?Q?Andr=E9_Graf?=) Date: Sat, 19 May 2012 16:53:53 +0200 Subject: [erlang-questions] Advice on meta programming problem In-Reply-To: References: Message-ID: Can't you use something similar to the mochiglobals module for that? I know people at Basho use it within their Lager logging framework for managing different log levels. I used a similar approach for enabling and disabling a mocked port driver. PS: Garrett.. missed the "reply to all" button On 19 May 2012 15:57, Lukas Larsson wrote: > If you are ok with using a "factory" to create a param module, you > could just as well use it to create a first argument to pass to the > function. i.e. > > -module(foo). > > create_context() -> > ?application:get_env(myapp,registered_foo). > > do_something(Context) -> > ?Context:do_something(). > > unless of course you have a reason for not wanting the parameter to be > explicitly passed? > > Lukas > > On Sat, May 19, 2012 at 12:02 AM, Garrett Smith wrote: >> I want to register a module that implements a callback interface. I >> don't want to use Erlang processes for this case -- otherwise a custom >> OTP behavior would suffice >> >> I want to use the module through an abstract layer like this: >> >> ? ?foo:do_something() >> >> I want to register a particular implementation of "foo" in application >> config like this: >> >> ? ?[{my_app, [{registered_foo, my_foo}]}]. >> >> my_foo would export do_something/0, which would be called by foo:do_someting/0. >> >> There are no registered processes here. >> >> I know I can use parameterized modules: >> >> ? ?-module(foo, [M]). >> >> ? ?-export([do_something/0]). >> >> ? ?do_something() -> M:do_something(). >> >> So easy! >> >> This does require some "factory" support to setup the parameterized >> module from application config. Not a big deal though, at all. >> >> Then there's my irrational, emotional hang-up of using parameterized >> modules, which is a more serious problem. >> >> I'm tempted to assemble a module at runtime using the standard >> metadata conventions in Erlang (code, compile etc. modules). >> >> foo would look like this: >> >> ? ?-module(foo). >> >> ? ?-export([do_something/0]). >> >> ? ?do_something() -> foo_impl:do_something(). >> >> foo_impl would not exist at compile time -- it would be generated at >> runtime using the application config. >> >> This is a very simple example -- more complex scenarios could exist, >> such as multiple implementation modules providing functionality. >> >> I'm curious to hear general reactions on this. Obviously parameterized >> modules fit -- but I'd like to explore some other approaches as well. >> >> Garrett >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From kostis@REDACTED Sat May 19 17:26:01 2012 From: kostis@REDACTED (Kostis Sagonas) Date: Sat, 19 May 2012 17:26:01 +0200 Subject: [erlang-questions] Robustness problems when using records [WAS: Re: Question/Alternative on Frames Proposal [Warning: Long]] In-Reply-To: <1337300505.68472.YahooMailNeo@web130106.mail.mud.yahoo.com> References: <4FB575D5.2020702@gmail.com> <1337300505.68472.YahooMailNeo@web130106.mail.mud.yahoo.com> Message-ID: <4FB7BB89.8050506@cs.ntua.gr> On 05/18/2012 02:21 AM, Tom Parker wrote: > .... > > Where I think we would vehemently agree: I expect Erlang to be robust. > That's why I'm even here. An issue with records where you can compile > two files (using the same .hrl) and end up with a result that (a) > compiles, (b) doesn't produce an error and (c) produces the wrong > answer... is a serious issue. It distinctly shows the shortcomings of > records. Not really related to the frame discussion, but I would like to point out that the above statement is wrong. The situation you describe does not show shortcomings of records; instead it shows shortcomings of programming with .hrl files and without an appropriate 'make'-like utility to track dependencies between files. Records are not without problems, but IMO this is not one of them. This is a problem of using .hrl files and choosing to program in a way which is not disciplined and thus error prone. There is a very simple way of avoiding this problem that IMO is a very nice way of programming: Use records as abstract data types and have them in a single module that exports appropriate getters and setters for manipulating fields of the record. If, for whatever reason, you do not like this way of using records and want the ability to perform pattern matching and field extraction in more than one module as you do today with records, better make sure you use an appropriate Makefile (or equivalent) for compiling your application. It's not the fault of record syntax if you do not use such a mechanism! (Makefiles are technology of the 70's after all...) Kostis From mattevans123@REDACTED Sat May 19 20:32:34 2012 From: mattevans123@REDACTED (Matthew Evans) Date: Sat, 19 May 2012 14:32:34 -0400 Subject: [erlang-questions] Robustness problems when using records [WAS: Re: Question/Alternative on Frames Proposal [Warning: Long]] In-Reply-To: <4FB7BB89.8050506@cs.ntua.gr> References: , <4FB575D5.2020702@gmail.com>, <1337300505.68472.YahooMailNeo@web130106.mail.mud.yahoo.com>, <4FB7BB89.8050506@cs.ntua.gr> Message-ID: Agreed, We are making use of Ulf's exprecs module for this precise reason. Matt > Date: Sat, 19 May 2012 17:26:01 +0200 > From: kostis@REDACTED > To: erlang-questions@REDACTED > Subject: [erlang-questions] Robustness problems when using records [WAS: Re: Question/Alternative on Frames Proposal [Warning: Long]] > > On 05/18/2012 02:21 AM, Tom Parker wrote: > > .... > > > > Where I think we would vehemently agree: I expect Erlang to be robust. > > That's why I'm even here. An issue with records where you can compile > > two files (using the same .hrl) and end up with a result that (a) > > compiles, (b) doesn't produce an error and (c) produces the wrong > > answer... is a serious issue. It distinctly shows the shortcomings of > > records. > > Not really related to the frame discussion, but I would like to point > out that the above statement is wrong. The situation you describe does > not show shortcomings of records; instead it shows shortcomings of > programming with .hrl files and without an appropriate 'make'-like > utility to track dependencies between files. > > Records are not without problems, but IMO this is not one of them. This > is a problem of using .hrl files and choosing to program in a way which > is not disciplined and thus error prone. There is a very simple way of > avoiding this problem that IMO is a very nice way of programming: Use > records as abstract data types and have them in a single module that > exports appropriate getters and setters for manipulating fields of the > record. If, for whatever reason, you do not like this way of using > records and want the ability to perform pattern matching and field > extraction in more than one module as you do today with records, better > make sure you use an appropriate Makefile (or equivalent) for compiling > your application. It's not the fault of record syntax if you do not use > such a mechanism! (Makefiles are technology of the 70's after all...) > > Kostis > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Sat May 19 21:00:17 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Sat, 19 May 2012 12:00:17 -0700 Subject: [erlang-questions] gproc and gen_event In-Reply-To: <460CFB0C-BF96-4BD6-8135-FE2497FF971B@feuerlabs.com> References: <460CFB0C-BF96-4BD6-8135-FE2497FF971B@feuerlabs.com> Message-ID: Awesome, thanks for the explanation Ulf! I'm going to give it a shot. --Andrew On Sat, May 19, 2012 at 2:40 AM, Ulf Wiger wrote: > > On 19 May 2012, at 01:38, Andrew Berman wrote: > > Ok, cool, thanks Garrett. > > Do you have any experience using gproc for pub/sub? > > > Gproc does do pub/sub fairly well, but if I may, the main purpose of > gproc initially was to serve as a form of "property index" for processes. > > It had dawned on me that complex signaling applications tended to > end up with a lot of code that basically provided different sorts of > mappings to allow you to locate the right processes. Having so many > different mappings complicated debugging, and obscured many > aspects of the architecture. > > When we started using (the predecessor of) gproc in some Ericsson > projects, there was significant code reduction. The nice thing was that > the developers picked it up themselves; there wasn't really any central > decision to go with it. > > One of the things I like about gproc is that it allows you to publish the > 'interaction features' of a process: registered aliases say much about > what services a process can perform, and properties illustrate its > dependencies on other processes. The registry can be queried with > select operations or QLC, or simply with gproc:i() from the shell. > > In short, there are many fine ways of doing pub/sub, but if you use > gproc consistently for this sort of thing, and similar things, it can help > clarify runtime aspects of your system, making it more transparent and > easier to debug. > > This also explains why gproc relies on a central registry (which is a > challenge in the global case). The registry itself is much of the point. > > Of course, for a distributed pub/sub, you don't really need to turn on > the distributed parts of gproc. Running a local gproc instance on each > node, you can publish like so: > > publish(Event, Data) -> > rpc:eval_everywhere( > gproc, send, > [{p, l, {?MODULE, Event}}, {?MODULE, Event, Data}]). > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Sat May 19 21:00:44 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Sat, 19 May 2012 12:00:44 -0700 Subject: [erlang-questions] gproc and gen_event In-Reply-To: References: Message-ID: Thanks for this! On Thu, May 17, 2012 at 7:25 PM, AD wrote: > Andrew, > > I posted about something similar on my blog > http://blog.dberg.org/2012/04/using-gproc-and-cowboy-to-pass-messages.html. > I cant speak to gen_event personally, I havent used it that much, but I > think its more of a use case for logging errors and such centrally versus a > messaging system but I could be off here. > > -A > > On Thu, May 17, 2012 at 8:14 PM, Andrew Berman wrote: > >> Hello, >> >> I am trying to set up a pub-sub type situation where I have a function >> called which does some stuff and at the end broadcasts a message. I would >> like for multiple handlers to be able to receive that message and do >> something with it. I'm currently using gen_event, but have been reading >> about gproc as well. Can anyone explain why they'd use one over the other >> for pub/sub? >> >> Thanks, >> >> Andrew >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel@REDACTED Sat May 19 21:20:14 2012 From: daniel@REDACTED (Daniel Schutte) Date: Sat, 19 May 2012 21:20:14 +0200 Subject: [erlang-questions] Common Test Long Nodenames Message-ID: <13797453.d9MUj6Fv7S@greyowl.thebluegroup.co.za> I have been unsuccessfull in trying to run ct_run to execute tests against another node (mercury@REDACTED) - initiated with a long node name. I keep on getting the following message: =ERROR REPORT==== 19-May-2012::16:22:39 === ** System NOT running to use fully qualified hostnames ** ** Hostname greyowl.thebluegroup.co.za is illegal ** Node communication works fine for all other nodes and I notice ct_run starts up only with a short nodename, when I execute ct_run and it displays the helpfile. Hosts file checked and fine, and both short and long hostnames configured and can be ping-ed. I did try to add the tuples described in test_server into my config file but that had no effect that I could see. Any suggestions on how I can resolve it, will be appreciated. -- DANIEL SCHUTTE | CHAIRMAN | T +27 11 475 1639 | C +27 82 808 5490 | F +27 86 583 7467 | E daniel@REDACTED | W www.thebluegroup.co.za | Skype: danielschutte | BBM: 2792FD09 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: tbg_mail.png Type: image/png Size: 28495 bytes Desc: not available URL: From rexxe98@REDACTED Sat May 19 21:38:04 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Sat, 19 May 2012 12:38:04 -0700 Subject: [erlang-questions] Declarative Security in Erlang Message-ID: Hey all, I'm trying to mimic (as much as I can) what JEE 6 does with security on domain objects. Essentially what I'd like to do is create an annotation using Tim's awesome annotation code ( https://github.com/hyperthunk/annotations) and test on a user's roles. The one issue I'm wrestling with is how to get the user into the annotation. The obvious way is to have every function I put the annotation on take in a user record and then loop through the arguments of the function looking for the user record. That way doesn't seem very elegant to me, though. I really want to just say something like User = get_current_user(...). Has anyone tackled this sort of issue or have any advice on how to handle it in an Erlang safe manner? Thanks, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From thpr@REDACTED Sat May 19 22:29:39 2012 From: thpr@REDACTED (Tom Parker) Date: Sat, 19 May 2012 13:29:39 -0700 (PDT) Subject: [erlang-questions] Robustness problems when using records [WAS: Re: Question/Alternative on Frames Proposal [Warning: Long]] In-Reply-To: <4FB7BB89.8050506@cs.ntua.gr> References: <4FB575D5.2020702@gmail.com> <1337300505.68472.YahooMailNeo@web130106.mail.mud.yahoo.com> <4FB7BB89.8050506@cs.ntua.gr> Message-ID: <1337459379.94372.YahooMailNeo@web130106.mail.mud.yahoo.com> Hmm.? Given Richard's reference to "version skew" I had figured this issue had long since been clearly articulated.? Perhaps that is not a good assumption on my part. Let us use the following assumptions: (1) Erlang intends to deliver a robust system.? For purposes of this discussion, robust is defined to mean that a wrong answer (ignoring a fault in the programmer's logic) must not be produced.? Returning errors is acceptable, but a miscalculation is not.(2) Erlang is a programming language used to build massively scalable soft real-time systems with requirements on high availability. (quoted from erlang.org) (3) A data structure with a name->value relationship is useful for software developers, has material value when shared across different modules, and should be robust when shared across different modules. (4) Halting - for an undefined period of time - a set of virtual machines to perform an update violates either high availability or soft-real time, depending on your point of view. (5) The solution needs to be practical, and not require an ISO9000-grade manual of steps to make sure you maintain robustness. If you do not agree with the assumptions, might as well stop now.? The entire discussion around whether my statement is right or wrong turns on the 5 assumptions above (and actually you can throw out either 4 or 5, but not both) Given the above, I believe records force you to choose to break one of the following items: (1) Robustness (2) Soft-Real Time (3) High Availability (4) Practicality I do not believe this can be solved by Make files.? In section 4 of his document (one can specifically refer to question 7), Richard treats the issue with records as one of compilation (What must be done to C? Recompile C).? However, just like "things that are not tested don't work", code which is compiled and not deployed doesn't exist.? Therefore the problem (for records) is not simply one of compilation, but one of compilation *and concurrent deployment of the new version of C with the new version of P*. Therefore, I continue to assert my statement below is correct as written.? Records are the issue, and it is the destruction of the name information before it reaches the VM that is the issue.? Here is the counter-example to why a Make-like functionality is not sufficient to meet the assumptions above for the Erlang language/platform when records are the method used for the name->value relationship: (1) Define a record, let's call it (shorthand, not real syntax) #rectangle{x, y, w, h}, place this into geom.hrl (2) Compile geomlibrary and geomconsumer referencing geom.hrl (3) Start distributed application with geomlibrary on machine#1 and geomconsumer on machine#2 (4) Modify #rectangle in geom.hrl to be {w, h, x, y} (5) Recompile, and Makefile technology ensures both geomlibrary and geomconsumer are rebuilt. (6) Deploy new BEAM files to a running (HA) application Let us start with the assumption that deployment of geomlibrary and geomconsumer are independent.? There is a window of time in which the first module (whichever it is) is deployed and the second module is not.? This window of time, due to potential network and CPU latencies on the machines, is not strictly definable.? In the window of time between deployment, there is a robustness issue as the wrong answer will be produced (an area function would calculate x*y during that period of time regardless of whether P or C were deployed first).? Therefore, we have a known robustness vulnerability if deployment is independent.? Therefore, deployment must be concurrent. One option is to shut down the application and restart.? This violates the high availability requirement. The other option is to make the deployment of multiple BEAM files to a running application a unit of work.? In order to do that, you have to halt any use of all of the modules involved in that unit of work across all of the machines potentially running those modules, and wait for acknowledgement back from all machines before any are allowed to proceed.? If any timeout, you must not deploy the change to any of the machines.? This introduces a number of issues involved with guarantees of robustness across machines and "soft real time" response times... and has the ability to halt the application up to the defined timeout period, which may need to be material for a large scale application.? If you extend the problem to consider machines that are communicating over network connections and NOT sharing a cookie, this unit of work consideration becomes a nearly intractable problem of coordination and trust across potentially disparate organizations... in having to define, coordinate and trust a "shared unit of work". So records fail for at least one of the following reasons: (1) You have failed the robustness requirement by not doing a unit of work deployment and therefore have a window of time in which wrong answers (but no error) can be produced. (2) You have failed the high availability requirement by forcing a full restart of the application. (3) You have failed the soft-real-time requirement due to unit of work deployment of BEAM files across a distributed application (4) You have failed practicality of deployment of BEAM files across a distributed application due to coordinating a "shared unit of work" across untrusted systems. This choice of which of the important aspects of Erlang must be sacrificed is one of the fundamental reasons why frames or some equivalent proposal to have a name-> value relationship _in the VM_ is required in order to maintain the ability to build robust, massively scalable soft real-time systems with requirements on high availability.? Frames, or some equivalent, ensure the name locations make it to the VM and the deployment of recompiled modules can be independent and robust. --Tom -- Tom Parker thpr@REDACTED ________________________________ From: Kostis Sagonas To: erlang-questions@REDACTED Sent: Saturday, May 19, 2012 11:26 AM Subject: [erlang-questions] Robustness problems when using records [WAS: Re: Question/Alternative on Frames Proposal [Warning: Long]] On 05/18/2012 02:21 AM, Tom Parker wrote: > .... > > Where I think we would vehemently agree: I expect Erlang to be robust. > That's why I'm even here. An issue with records where you can compile > two files (using the same .hrl) and end up with a result that (a) > compiles, (b) doesn't produce an error and (c) produces the wrong > answer... is a serious issue. It distinctly shows the shortcomings of > records. Not really related to the frame discussion, but I would like to point out that the above statement is wrong. The situation you describe does not show shortcomings of records; instead it shows shortcomings of programming with .hrl files and without an appropriate 'make'-like utility to track dependencies between files. Records are not without problems, but IMO this is not one of them. This is a problem of using .hrl files and choosing to program in a way which is not disciplined and thus error prone. There is a very simple way of avoiding this problem that IMO is a very nice way of programming: Use records as abstract data types and have them in a single module that exports appropriate getters and setters for manipulating fields of the record. If, for whatever reason, you do not like this way of using records and want the ability to perform pattern matching and field extraction in more than one module as you do today with records, better make sure you use an appropriate Makefile (or equivalent) for compiling your application. It's not the fault of record syntax if you do not use such a mechanism! (Makefiles are technology of the 70's after all...) Kostis -------------- next part -------------- An HTML attachment was scrubbed... URL: From serge@REDACTED Sun May 20 04:21:38 2012 From: serge@REDACTED (Serge Aleynikov) Date: Sat, 19 May 2012 22:21:38 -0400 Subject: [erlang-questions] c-node: Link error in Visual C++ In-Reply-To: References: Message-ID: <4FB85532.6020206@aleynikov.org> Looks like you forgot to wrap the C code in the extern "C" section: extern "C" { #include "erl_interface.h" #include "ei.h" } and therefore the C++ compiler does name mangling and can't find relevant entries. On 5/16/2012 12:06 PM, mastah_killah@REDACTED wrote: > Hello, > > I'm trying to compile the c-node example on Windows. > I have the following: > > #include > #include > #include > > > #include "erl_interface.h" > #include "ei.h" > > And I have included ei.lib,erl_interface.lib, and ws2_32.lib. > > However I'm getting the following error: > > InstantMission.obj : error LNK2001: unresolved external symbol @erl_init@REDACTED > InstantMission.obj : error LNK2001: unresolved external symbol @erl_send@REDACTED > InstantMission.obj : error LNK2001: unresolved external symbol > @erl_connect@REDACTED > InstantMission.obj : error LNK2001: unresolved external symbol > @erl_receive_msg@REDACTED > InstantMission.obj : error LNK2001: unresolved external symbol > @erl_free_term@REDACTED > InstantMission.obj : error LNK2001: unresolved external symbol > @erl_connect_init@REDACTED > InstantMission.obj : error LNK2001: unresolved external symbol > @erl_element@REDACTED > ..\..\Debug\missions\\maseInstant.dll : fatal error LNK1120: 7 > unresolved externals > > Thanks in advance. > > > > Ihr WEB.DE Postfach immer dabei: die kostenlose WEB.DE Mail App f?r > iPhone und Android. > *https://produkte.web.de/freemail_mobile_startseite/* > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ulf@REDACTED Sun May 20 10:07:36 2012 From: ulf@REDACTED (Ulf Wiger) Date: Sun, 20 May 2012 10:07:36 +0200 Subject: [erlang-questions] Robustness problems when using records [WAS: Re: Question/Alternative on Frames Proposal [Warning: Long]] In-Reply-To: <1337459379.94372.YahooMailNeo@web130106.mail.mud.yahoo.com> References: <4FB575D5.2020702@gmail.com> <1337300505.68472.YahooMailNeo@web130106.mail.mud.yahoo.com> <4FB7BB89.8050506@cs.ntua.gr> <1337459379.94372.YahooMailNeo@web130106.mail.mud.yahoo.com> Message-ID: On 19 May 2012, at 22:29, Tom Parker wrote: > Therefore, I continue to assert my statement below is correct as written. Records are the issue, and it is the destruction of the name information before it reaches the VM that is the issue. Actually, I agree with Kostis: the biggest problem is the reliance on .hrl files and the preprocessor for sharing code between modules. You can put a local function definition in a .hrl file as well, and it has been done too. Changes to a record usually trigger badmatch or function_clause exceptions if not all modules are recompiled and replaced in a coordinated way. This is a problem, for sure, but if you make subtle semantic changes to a function in a .hrl file, or to a macro (which can expand to pretty much anything), then *anything* can happen - including a large set of extremely subtle errors that are hard to achieve with records. So putting a function definition in a .hrl file can cause some really subtle problems. This obviously doesn't mean that functions are fundamentally flawed, but yet an example that reliance on preprocessors is a dangerous thing. The worst thing you can do with records - and it's indeed a bad thing - is to rename attributes and use the same positions for entirely different data (no, worst would be the *same* data type but with a different interpretation). Luckily any half-decent programmer will understand enough not to do this. Richard O'Keefe has many times stated that a key goal is to remove the reliance on the pre-processor. If you think about it, Erlang has no facility for sharing structured data types between modules, except for code - as Kostis mentioned - and pattern matching on the primitive elements of the structure. You can provide accessor functions that are exported from the same place where the data structure is managed, or you can use data structures that are as shallow as to be easy to inspect through pattern-matching. If this is not enough, you can pass along property lists, which are simple, yet flexible enough to convey just about anything - however cannot be quickly inspected with pattern-matching. So people trade safety for speed. ?or rely on convention, which actually works very well. Remember that records are little more than syntactic sugar. They do not introduce a data type that would offer anything more than the above. You could get a similar effect by introducing a series of home-cooked macros, expanding to operations and patterns on a tuple structure, and then let multiple modules rely on them through a .hrl file. I even have some old code lying around from the time before records. It had a lot of code like this: person(name, {person,X,_,_}) -> X; person(age, {person,_,X,_}) -> X; person(address, {person,_,_,X}) -> X. person(name, X, {person,A,B,C}) -> {person,X,B,C}; person(age, X, {person,A,B,C}) -> {person,A,X,C}; person(addr, X, {person,A,B,C}) -> {person,A,B,X}. A bit fiddly to create, but easy to verify through visual inspection. Of course, you could put that in a .hrl file and use as a 'shrared data type'. It would suffer from roughly the same problems as records. The obvious limitations with records were the subject of animated discussion even before they were added to the language. You wouldn't recall this unless you were a member of a pretty small group of people debating this on a mailing list inside Ericsson, long before the Open Source release. Even then it was agreed that the proper way to do it was to introduce a new data type. As my memory serves, the problem with that was that JAM didn't have any more type tags to use. A redesign of the memory management structure in the VM would be required in order to add new data types. In light of this, records were seen as a cost-efficient way to solve a common problem of copy-paste programming. Later on, I believe it was the HiPE team that proposed a new tagging scheme and wrote some working code to demonstrate. By the time it was integrated into Erlang, records had become ubiquitous, and could not be easily replaced. If my recollection is faulty, the old-timers can correct me. I recall this playing out right after I joined Ericsson in 1996. (The new tagging scheme was introduced in R7B in 2000, and is described in Mikael Petterson's PhD thesis: http://www.it.uu.se/research/publications/reports/2000-029/) One can argue that it's taken a long time to correct this issue, and add a structured data type with all the goodness of records and none of their drawbacks. The reasons: - Erlang, even taking this into account, is a darn good language for writing complex, robust and scalable systems. - Convention actually works pretty well in practice. - The OTP team wanted to get it right the second time. - Backwards compatibility is considered extremely important in the Erlang world (exactly because we don't want to break the complex, robust and scalable systems already written in Erlang) BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From emeka_1978@REDACTED Sun May 20 14:21:09 2012 From: emeka_1978@REDACTED (eigenfunction) Date: Sun, 20 May 2012 05:21:09 -0700 (PDT) Subject: [erlang-questions] Robustness problems when using records [WAS: Re: Question/Alternative on Frames Proposal [Warning: Long]] In-Reply-To: <4FB7BB89.8050506@cs.ntua.gr> References: <4FB575D5.2020702@gmail.com> <1337300505.68472.YahooMailNeo@web130106.mail.mud.yahoo.com> <4FB7BB89.8050506@cs.ntua.gr> Message-ID: > There is a very simple way of > avoiding this problem that IMO is a very nice way of programming: Use > records as abstract data types and have them in a single module that > exports appropriate getters and setters for manipulating fields of the > record. If, for whatever reason, you do not like this way of using > records and want the ability to perform pattern matching and field > extraction in more than one module as you do today with records, better > make sure you use an appropriate Makefile (or equivalent) for compiling > your application. It's not the fault of record syntax if you do not use > such a mechanism! (Makefiles are technology of the 70's after all...) Would you care to elaborate with a simple example in the context of a gen_server? I definitely feel guilty of what you are describing. From emeka_1978@REDACTED Sun May 20 14:27:34 2012 From: emeka_1978@REDACTED (eigenfunction) Date: Sun, 20 May 2012 05:27:34 -0700 (PDT) Subject: [erlang-questions] os:cmd Message-ID: <24c03709-d210-4d25-b83f-9ce0ce76f2f8@w13g2000vbc.googlegroups.com> I am using rsync through os:cmd and rely on the console ouput from the rsync command to decide whether or not the operation was successulf or not, specially in the case where a file is missing on the remote end. What i found out is that the return value of os:cmd does not always equal the output of the rsync command run in the console. Therefore my question: Is it the normal behaviour? From zeno490@REDACTED Sun May 20 16:24:38 2012 From: zeno490@REDACTED (Nicholas Frechette) Date: Sun, 20 May 2012 10:24:38 -0400 Subject: [erlang-questions] implementing Forex Trader in Erlang In-Reply-To: References: <4FB575D5.2020702@gmail.com> Message-ID: Hi, At home, I built a system to do just that. Erlang worked great to build the prototype and I was able to quickly iterate and get something stable running. In fact, it has been running for 2 years now without downtime. I also fell in love with erlang during this time. Unfortunately, it also became clear at some point that it would have a hard time to scale beyond a certain size. Things like modifying large datasets in memory (order book management, historical data management) very quickly are not strong suits of erlang. You can probably scale to a professional suitable level with the proper time/resources however for me, it wasn't exactly worth it. I already have my GUI application in C# and seeing how I build high availability applications with tight cpu/memory constraints every day at work in C++, C# proved a better choice for me even if only because I know when the need arises, I'll be able to optimize it easily to a level I'm comfortable with. In erlang, I would have had to resort to using drivers written in C++ for the critical bits or entire processes built in C++. C# will also allow me to leverage my functional prog. skills with F# if the need arises :) Basically, that was it for me. I worked in erlang 1+ year building it, I had a great time doing it but ultimately changed my mind despite the fact it'll take me 1 year to rewrite the erlang bits in C#. You could easily build everything you need in erlang, there is no doubt there (and fairly quickly too). Whether you can reach cpu budgets and memory budgets you set yourself will largely depend on how motivated you are with erlang and how much you are willing to hybridize your system (either with drivers or other processes). Using it only for message processing seems reasonable but a bit strange if your solution is in a different language. In C# for example, NServiceBus would probably be an equally good solution. I'm sure java must have a stable solution for messaging as well. Rolling out your own isn't all that hard either, the Disruptor guys sorta did it. That's my story and my 2 cents. Note that it wasn't a professional project, only a hobby. Cheers, Nicholas On Fri, May 18, 2012 at 2:05 AM, Ilyushonak Barys wrote: > It looks likes that this guys use Erlang ?only for messaging middleware. The > server components, which connected to exchanges, are implemented in c++. > > > > From: erlang-questions-bounces@REDACTED > [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Michael Truog > Sent: Friday, May 18, 2012 2:05 AM > To: Shahrdad Shadab > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] implementing Forex Trader in Erlang > > > > Here is the architecture of something similar: > http://www.connamara.com/solutions-hftframework.html > > On 05/17/2012 09:19 AM, Shahrdad Shadab wrote: > > Hi Erlangers > > ?I have to implement a foreign exchange trader software for a financial > firm. > They need fast response and high availability. They also need parallel real > time feed from multiple brokers. > As I did my preliminary investigations,It seems most of the tools on the > field are implemented in Microsoft technologies like .net or com/OLE. > I believe (although I might be wrong) the broker servers are DDE servers. > I decided to use Erlang/otp. Does anyone know if something like this has > done before in Erlang. > Also I appreciate any guidance on how to get feed from trader servers. > > Thanks a lot > Best regards > Shahrdad > > > -- > Software Architect & Computer Scientist > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________________ > > > > The information contained in this message may be privileged and conf > idential and protected from disclosure. If you are not the original intended > recipient, you are hereby notified that any review, retransmission, > dissemination, or other use of, or taking of any action in reliance upon, > this information is prohibited. If you have received this communication in > error, please notify the sender immediately by replying to this message and > delete it from your computer. Thank you for your cooperation. Troika Dialog, > Russia. > > If you need assistance please contact our Contact Center (+7495) 258 0500 or > go to www.troika.ru/eng/Contacts/system.wbp > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From hukl@REDACTED Sun May 20 16:27:51 2012 From: hukl@REDACTED (John-Paul Bader) Date: Sun, 20 May 2012 16:27:51 +0200 Subject: [erlang-questions] Weird Client SSL Behavior / Performance Message-ID: <4FB8FF67.6070108@berlin.ccc.de> Hey, recently we had to implement a service that would send about 50 - 500 https requests per second to a 3rd party API that has an average latency of about 400-800ms. Therefor we had to open a lot of parallel connections to the destination host. First we have used lhttpc but after some time we observed really weird behavior. While lhttpc would use some kind of connection pooling, it was constantly leaking processes that were stuck in prim_inet:recv. Netstat showed a continously growing number of connections stuck in WAIT or TIME_WAIT for port 443. After some time all sockets on the machine were used and no communication was possible anymore. After trying out various settings for lhttpc without any improvement on the situation we added ibrowse instead. ibrowse behaved much nicer as it was not leaking processes and also did not accumulate the WAIT or TIME_WAIT connections that showed up before in netstat. We told it to open 300 connections and that was the exact amount it opened. Now this went fine like it should for about 15 or 20 minutes before _no_ traffic was going through that 300 connections. The VM accumulated load and basically was not communicating with the 3rd party API. Interestingly enough netstat still showed that all 300 connections were established. We started to investigate on the true cause for this behavior. At some point we set up a local nginx with ssl and tried it there to rule out the 3rd party API. The behavior was the same. Then we tried to use http instead of https and boom - that went super smooth just like it should. 300 connections, minimal load on the VM, no leaking processes and it kept running for more than 15 minutes. Then we switched back to the real API and back to https and tried to figure out what part in the VM was holding us back but we could not really find a solution. We only saw that a lot of message were accumulating in the outer gen_fsm:loop of ssl after those 15 minutes. In the end, after multiple days of investigating and experiments, we set up stunnel on the same machine in client mode that was connecting to the 3rd party API. This way our erlang service just talked plain http with stunnel. This performs extremely well for multiple days now. This is somehow unsatisfactory though. First of all I'd really like to know why the problem occurs in the first place and why it creates so much load in general. Stunnel deals with the same load with 1/3 or even less than that of what the erlang vm needs. For now at least I'm a little bit underwhelmed of erlangs ssl stability and performance (I was bitten by the R14B03 bug as well). Secondly, lhttpc socket handling is not really that great. I'm sure it works fine for http or in low load https scenarios but in our case, leaking processes, using up all sockets and by that stopping the service all together was super bad. Is there anybody with similar observations or maybe even solutions? ~ John From daniel@REDACTED Sun May 20 18:35:59 2012 From: daniel@REDACTED (Daniel Schutte) Date: Sun, 20 May 2012 18:35:59 +0200 Subject: [erlang-questions] ct_run and long nodenames Message-ID: <21929758.z8UqELVuaX@greyowl.thebluegroup.co.za> Hi, (sorry if this is a repost - my previous message looks like it got lost) I have been unsuccessfull in trying to run ct_run to execute tests against another node (mercury@REDACTED) - initiated with a long node name. I keep on getting the following message: =ERROR REPORT==== 19-May-2012::16:22:39 === ** System NOT running to use fully qualified hostnames ** ** Hostname greyowl.thebluegroup.co.za is illegal ** Node communication works fine for all other nodes and I notice ct_run starts up only with a short nodename, when I execute ct_run and it displays the helpfile. Hosts file checked and fine, and both short and long hostnames configured and can be ping-ed. I did try to add the tuples described in test_server into my config file but that had no effect that I could see. Any suggestions on how I can resolve it, will be appreciated. -- DANIEL SCHUTTE | CHAIRMAN | T +27 11 475 1639 | C +27 82 808 5490 | F +27 86 583 7467 | E daniel@REDACTED | W www.thebluegroup.co.za | Skype: danielschutte | BBM: 2792FD09 -------------- next part -------------- An HTML attachment was scrubbed... URL: From thpr@REDACTED Sun May 20 19:13:42 2012 From: thpr@REDACTED (Tom Parker) Date: Sun, 20 May 2012 10:13:42 -0700 (PDT) Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> Message-ID: <1337534022.48017.YahooMailNeo@web130103.mail.mud.yahoo.com> Not exactly on a Wiki, but since this seems to be a popular topic and the little project I'm working has moved on from analyzing BEAM files, I typed up what I have found in the hope that this might be useful to the next person attempting to decode/reverse engineer the files.? Lots of TODOs and things I didn't need to figure out for what I was doing, but hopefully moves the ball forward. http://dl.dropbox.com/u/80510194/doc.pdf and the LaTeX source in case anyone wants to continue or branch this to a wiki: http://dl.dropbox.com/u/80510194/doc.tex Thanks. TP. -- Tom Parker thpr@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon May 21 00:42:56 2012 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 21 May 2012 10:42:56 +1200 Subject: [erlang-questions] Question/Alternative on Frames Proposal [Warning: Long] In-Reply-To: References: <4FB575D5.2020702@gmail.com> <1337300505.68472.YahooMailNeo@web130106.mail.mud.yahoo.com> <1338fcafd15281751fb6491bff9b6109.squirrel@chasm.otago.ac.nz> Message-ID: On 19/05/2012, at 8:57 AM, Garrett Smith wrote: > > I apologize, this is somewhat off topic and not an answer to your question... > > But does anyone know where the Frames proposal stands vis-a-vis the > OTP team's work on hashes? I'm not sure what you are asking. The paper that you mentioned made it quite clear that they were NOT experimenting with frames or anything in the same area of design space. From chandrashekhar.mullaparthi@REDACTED Mon May 21 01:29:55 2012 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Mon, 21 May 2012 00:29:55 +0100 Subject: [erlang-questions] Weird Client SSL Behavior / Performance In-Reply-To: <4FB8FF67.6070108@berlin.ccc.de> References: <4FB8FF67.6070108@berlin.ccc.de> Message-ID: Hi John, On 20 May 2012 15:27, John-Paul Bader wrote: > After trying out various settings for lhttpc without any improvement on > the situation we added ibrowse instead. ibrowse behaved much nicer as it > was not leaking processes and also did not accumulate the WAIT or TIME_WAIT > connections that showed up before in netstat. We told it to open 300 > connections and that was the exact amount it opened. > Glad you find it useful! > This is somehow unsatisfactory though. > > First of all I'd really like to know why the problem occurs in the first > place and why it creates so much load in general. Stunnel deals with the > same load with 1/3 or even less than that of what the erlang vm needs. For > now at least I'm a little bit underwhelmed of erlangs ssl stability and > performance (I was bitten by the R14B03 bug as well). > > I don't have a solution to your problem, but my experience with the SSL implementation has been the same. I use stunnel wherever I need SSL support, and for services which handle lots of traffic, I use an SSL accelerator box to front the erlang nodes. cheers Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From mastah_killah@REDACTED Mon May 21 04:00:42 2012 From: mastah_killah@REDACTED (mastah killah) Date: Mon, 21 May 2012 04:00:42 +0200 Subject: [erlang-questions] c-node: Link error in Visual C++ In-Reply-To: <4FB85532.6020206@aleynikov.org> References: <4FB85532.6020206@aleynikov.org> Message-ID: <4FB9A1CA.6020709@web.de> Hello, this does not work. Already tried. Same error. Using VS 2008 btw. On 20.05.2012 04:21, Serge Aleynikov wrote: > Looks like you forgot to wrap the C code in the extern "C" section: > > extern "C" { > #include "erl_interface.h" > #include "ei.h" > } > > and therefore the C++ compiler does name mangling and can't find > relevant entries. > > > On 5/16/2012 12:06 PM, mastah_killah@REDACTED wrote: >> Hello, >> >> I'm trying to compile the c-node example on Windows. >> I have the following: >> >> #include >> #include >> #include >> >> >> #include "erl_interface.h" >> #include "ei.h" >> >> And I have included ei.lib,erl_interface.lib, and ws2_32.lib. >> >> However I'm getting the following error: >> >> InstantMission.obj : error LNK2001: unresolved external symbol @erl_init@REDACTED >> InstantMission.obj : error LNK2001: unresolved external symbol @erl_send@REDACTED >> InstantMission.obj : error LNK2001: unresolved external symbol >> @erl_connect@REDACTED >> InstantMission.obj : error LNK2001: unresolved external symbol >> @erl_receive_msg@REDACTED >> InstantMission.obj : error LNK2001: unresolved external symbol >> @erl_free_term@REDACTED >> InstantMission.obj : error LNK2001: unresolved external symbol >> @erl_connect_init@REDACTED >> InstantMission.obj : error LNK2001: unresolved external symbol >> @erl_element@REDACTED >> ..\..\Debug\missions\\maseInstant.dll : fatal error LNK1120: 7 >> unresolved externals >> >> Thanks in advance. >> >> >> >> Ihr WEB.DE Postfach immer dabei: die kostenlose WEB.DE Mail App f?r >> iPhone und Android. >> *https://produkte.web.de/freemail_mobile_startseite/* >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From eriksoe@REDACTED Mon May 21 04:16:43 2012 From: eriksoe@REDACTED (=?ISO-8859-1?Q?Erik_S=F8e_S=F8rensen?=) Date: Mon, 21 May 2012 04:16:43 +0200 Subject: [erlang-questions] os:cmd In-Reply-To: <24c03709-d210-4d25-b83f-9ce0ce76f2f8@w13g2000vbc.googlegroups.com> References: <24c03709-d210-4d25-b83f-9ce0ce76f2f8@w13g2000vbc.googlegroups.com> Message-ID: The behaviour and output of rsync, and indeed any command, may depend on the environment - for instance the LANG and locale variable - and sometimes also on whether the output goes directly to the console or not (see isatty(3)). Some C library functions behave differently depending on the locale, which can be a bit surprising: $ /usr/bin/printf "%f\n" "123.45" # In the da_DK locale, decimal point is "," 123,450000 $ LANG=C /usr/bin/printf "%f\n" "123.45" 123.450000 $ type printf printf is a shell builtin $ printf "%f\n" "123.45" # Uses the locale (da_DK) for parsing the number bash: printf: 123.45: invalid number 0,000000 $ LANG=C printf "%f\n" "123.45" # This ought to work, but doesn't bash: printf: 123.45: invalid number 0,000000 $ export LANG=C ; printf "%f\n" "123.45" # This works 123.450000 Sorting strings is another thing that may behave differently depending on the locale. 2012/5/20 eigenfunction > I am using rsync through os:cmd and rely on the console ouput from the > rsync command to decide whether or not the operation was successulf or > not, specially in the case where a file is missing on the remote end. > What i found out is that the return value of os:cmd does not always > equal the output of the rsync command run in the console. > Therefore my question: Is it the normal behaviour? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mastah_killah@REDACTED Mon May 21 06:51:07 2012 From: mastah_killah@REDACTED (mastah killah) Date: Mon, 21 May 2012 06:51:07 +0200 Subject: [erlang-questions] c-node: Link error in Visual C++ In-Reply-To: <4FB9A1CA.6020709@web.de> References: <4FB85532.6020206@aleynikov.org> <4FB9A1CA.6020709@web.de> Message-ID: <4FB9C9BB.4080403@web.de> Why, when I use dumpbin on the library, does it not show exported symbols?: dumpbin.exe /exports ei.lib Microsoft (R) COFF/PE Dumper Version 9.00.30729.01 Copyright (C) Microsoft Corporation. All rights reserved. Dump of file ei.lib File Type: LIBRARY Summary 15B .bss 178B .data 1A5CC4 .debug$S 3180 .debug$T 22E0 .drectve 20 .rdata D73A .text dumpbin.exe /exports erl_interface.lib Microsoft (R) COFF/PE Dumper Version 9.00.30729.01 Copyright (C) Microsoft Corporation. All rights reserved. Dump of file erl_interface.lib File Type: LIBRARY Summary 618 .bss 320 .data 4DFD4 .debug$S 7BC .debug$T 545 .drectve 77C3 .text On 21.05.2012 04:00, mastah killah wrote: > Hello, > > this does not work. Already tried. Same error. > Using VS 2008 btw. > > > > > > On 20.05.2012 04:21, Serge Aleynikov wrote: >> Looks like you forgot to wrap the C code in the extern "C" section: >> >> extern "C" { >> #include "erl_interface.h" >> #include "ei.h" >> } >> >> and therefore the C++ compiler does name mangling and can't find >> relevant entries. >> >> >> On 5/16/2012 12:06 PM, mastah_killah@REDACTED wrote: >>> Hello, >>> >>> I'm trying to compile the c-node example on Windows. >>> I have the following: >>> >>> #include >>> #include >>> #include >>> >>> >>> #include "erl_interface.h" >>> #include "ei.h" >>> >>> And I have included ei.lib,erl_interface.lib, and ws2_32.lib. >>> >>> However I'm getting the following error: >>> >>> InstantMission.obj : error LNK2001: unresolved external symbol >>> @erl_init@REDACTED >>> InstantMission.obj : error LNK2001: unresolved external symbol >>> @erl_send@REDACTED >>> InstantMission.obj : error LNK2001: unresolved external symbol >>> @erl_connect@REDACTED >>> InstantMission.obj : error LNK2001: unresolved external symbol >>> @erl_receive_msg@REDACTED >>> InstantMission.obj : error LNK2001: unresolved external symbol >>> @erl_free_term@REDACTED >>> InstantMission.obj : error LNK2001: unresolved external symbol >>> @erl_connect_init@REDACTED >>> InstantMission.obj : error LNK2001: unresolved external symbol >>> @erl_element@REDACTED >>> ..\..\Debug\missions\\maseInstant.dll : fatal error LNK1120: 7 >>> unresolved externals >>> >>> Thanks in advance. >>> >>> >>> >>> Ihr WEB.DE Postfach immer dabei: die kostenlose WEB.DE Mail App f?r >>> iPhone und Android. >>> *https://produkte.web.de/freemail_mobile_startseite/* >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From arun11299@REDACTED Mon May 21 09:31:02 2012 From: arun11299@REDACTED (Arun Muralidharan) Date: Mon, 21 May 2012 13:01:02 +0530 Subject: [erlang-questions] Creating backup of ram based Mnesia on Disc Message-ID: Hi, Currently I am creating the table in following steps: mnesia:start(), mnesia:create_schema([node()]), mnesia:create_table(guiding_data, [ {attributes, record_info(fields, guiding_data)}, {ram_copies,[node()]} ]), Now, all the data I load (huge amounts) will reside in RAM as expected. But what I would like here is to have a backup of all the data residing in RAM to be also present on disk, so that in case i stop and start Mnesia again, it should load the data from the disk to memory, instead of loading the data again into memory explicitly by me. What I can do here is : 1) mnesia:change_table_copy_type(schema,node(),disc_copies), Now, this would create a disc copy, but all my queries to mnesia would access disk. right ? This is what i observed and didnt want. 2) mnesia:add_table_copy(table, node(), disc_copies) This didnt have any effect. Didnt create any disc copy. So, what would be the correct way to 1) Load data to RAM as well as backup in disc. 2) Queries/ Multiple loads should access/update RAM/disc resp. 3) On mnesia restart data should be loaded from Disc automatically. Is there any Mnesia configuration for above ? Thanks Arun -------------- next part -------------- An HTML attachment was scrubbed... URL: From janburse@REDACTED Mon May 21 10:53:18 2012 From: janburse@REDACTED (Jan Burse) Date: Mon, 21 May 2012 10:53:18 +0200 Subject: [erlang-questions] Benchmarks (Was: Question/Alternative on Frames Proposal) In-Reply-To: References: <4FB575D5.2020702@gmail.com> <1337300505.68472.YahooMailNeo@web130106.mail.mud.yahoo.com> <1338fcafd15281751fb6491bff9b6109.squirrel@chasm.otago.ac.nz> Message-ID: <4FBA027E.1080703@fastmail.fm> Richard O'Keefe schrieb: > > On 19/05/2012, at 8:57 AM, Garrett Smith wrote: >> >> I apologize, this is somewhat off topic and not an answer to your question... >> >> But does anyone know where the Frames proposal stands vis-a-vis the >> OTP team's work on hashes? > > I'm not sure what you are asking. The paper that you mentioned made it > quite clear that they were NOT experimenting with frames or anything in > the same area of design space. > Hi Richard, If I remember well, your proposal mentions some benchmarks with monomorphic caching. You also explained the SmallTalk history of caching methods. So basically I suppose someone is doing the following: if (cache == null || cache.base != base) cache = lookup(base,key); /* costly */ return cache; Since last week I have been experimenting with polymorphic caching. And it seems that is practically same speed as monomorphic caching, but has the additional advantage of not trashing the cache. Trashing of monomorphic cache happens when the base is not constant for a call site. When trashing happens time spent goes drastically up. I observed for some test case of mine: No Trashing: 1400 ms Trashing: 2200 ms So I did a very simple polymorphic cache consisting of a a single linked list. Its adding new entries at the end of the list if it didn't find an entry with matching base. Its not that bad: Monomorphic: 1400 ms Polymoprhic: 1450 ms I am not using some locking. Its just overwriting the the last link with a new entry, so concurrently it might do double work sporadically, but the result should never be wrong. But it requires GC of the eventually lost record. Something done for Erlang records? Bye From hm@REDACTED Mon May 21 12:20:36 2012 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Mon, 21 May 2012 12:20:36 +0200 Subject: [erlang-questions] Creating backup of ram based Mnesia on Disc In-Reply-To: References: Message-ID: On Mon, May 21, 2012 at 9:31 AM, Arun Muralidharan wrote: > Hi, > > Currently I am creating the table in following steps: > > mnesia:start(), > mnesia:create_schema([node()]), > > mnesia:create_table(guiding_data, > ? ? ? ? ? ? ? ? ? ? ? ? ?[ > ? ? ? ? ? ? ? ? ? ? ? ? ? ?{attributes, record_info(fields, guiding_data)}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ?{ram_copies,[node()]} > ? ? ? ? ? ? ? ? ? ? ? ? ?]), > > Now, all the data I load (huge amounts) will reside in RAM as expected. > But what I would like here is to have a backup of all the data residing > in RAM to be also present on disk, so that in case i stop and start Mnesia > again, > it should load the data from the disk to memory, instead of loading the data > again into memory explicitly by me. > > What I can do here is : > 1) mnesia:change_table_copy_type(schema,node(),disc_copies), > ? ?Now, this would create a disc copy, but all my queries to mnesia would > ? ?access disk. right ? This is what i observed and didnt want. No, for disc_copies, read access is only performed in RAM. > 2) mnesia:add_table_copy(table, node(), disc_copies) > ? ?This didnt have any effect. Didnt create any disc copy. No, it is not allowed. (Did you really check the return value?) You can only have one replica per node. Its storage type is EITHER ram_copies, disc_copies OR disc_only_copies. disc_copies implies storage both in RAM AND on disk. > So, what would be the correct way to > 1) Load data to RAM as well as backup in disc. > 2) Queries/ Multiple loads should access/update RAM/disc resp. > 3) On mnesia restart data should be loaded from Disc automatically. You will get this behavior if you use the disc_copies storage type. /H?kan From g@REDACTED Mon May 21 14:15:40 2012 From: g@REDACTED (Garrett Smith) Date: Mon, 21 May 2012 07:15:40 -0500 Subject: [erlang-questions] Question/Alternative on Frames Proposal [Warning: Long] In-Reply-To: References: <4FB575D5.2020702@gmail.com> <1337300505.68472.YahooMailNeo@web130106.mail.mud.yahoo.com> <1338fcafd15281751fb6491bff9b6109.squirrel@chasm.otago.ac.nz> Message-ID: On Sun, May 20, 2012 at 5:42 PM, Richard O'Keefe wrote: > > On 19/05/2012, at 8:57 AM, Garrett Smith wrote: >> >> I apologize, this is somewhat off topic and not an answer to your question... >> >> But does anyone know where the Frames proposal stands vis-a-vis the >> OTP team's work on hashes? > > I'm not sure what you are asking. ?The paper that you mentioned made it > quite clear that they were NOT experimenting with frames or anything in > the same area of design space. Richard, I'm not seeing what you're seeing. That's a PDF of presentation slides from 2011 User Conf-- it lists "frames" in the title and as a input to their process. I read the document as a status update on their work, listing some performance results from various data structures. The "Conclusion and way forward" slide seems open ended. In any event, is not relevant to any discussion here what the OTP team is actually working on? Is this is a blip of interest in a decade old proposal that has no shot of getting into the language? Garrett From kenneth.lundin@REDACTED Mon May 21 14:54:19 2012 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Mon, 21 May 2012 14:54:19 +0200 Subject: [erlang-questions] Question/Alternative on Frames Proposal [Warning: Long] In-Reply-To: References: <4FB575D5.2020702@gmail.com> <1337300505.68472.YahooMailNeo@web130106.mail.mud.yahoo.com> <1338fcafd15281751fb6491bff9b6109.squirrel@chasm.otago.ac.nz> Message-ID: On Mon, May 21, 2012 at 2:15 PM, Garrett Smith wrote: > On Sun, May 20, 2012 at 5:42 PM, Richard O'Keefe > wrote: > > > > On 19/05/2012, at 8:57 AM, Garrett Smith wrote: > >> > >> I apologize, this is somewhat off topic and not an answer to your > question... > >> > >> But does anyone know where the Frames proposal stands vis-a-vis the > >> OTP team's work on hashes? > > > > I'm not sure what you are asking. The paper that you mentioned made it > > quite clear that they were NOT experimenting with frames or anything in > > the same area of design space. > > Richard, I'm not seeing what you're seeing. > > That's a PDF of presentation slides from 2011 User Conf-- it lists > "frames" in the title and as a input to their process. > > I read the document as a status update on their work, listing some > performance results from various data structures. > > The "Conclusion and way forward" slide seems open ended. > > In any event, is not relevant to any discussion here what the OTP team > is actually working on? Is this is a blip of interest in a decade old > proposal that has no shot of getting into the language? > > Garrett > The conclusion from my talk at EUC November 2011 was like this: > Conclusions and way forward > ? New better records > ? optionally named, possibly declared > ? few named fields > ? many instances > ? ------- and > -------------------------------------------------------------------------- > ? Hashmaps > ? many ?keys? > ? few instances > ? are two different things (but similar) > ? we want both > ? HAMT looks really promising for Hash Maps > ? but it might be hard to combine with one single representation > ? We have to decide what to address first > ? Probably some experimental implementation released during 2012 > I think this clearly indicates that we intended to work on both "better records" and "hashes". Now I can as a fact tell you that we are addressing both. We are prototyping an implementation of "hashes", or whatever we will call them, and we are also prototyping a reference implementation of the frames proposal. We think that the frames proposal is interesting and think we need an implementation so we can evaluate performance and compare it with records. After the evaluation we will come to a decision and will also give more details about possible changes/restrictions/additions we want to make compared with the current frames proposal. What we find most interesting with the frames proposal is the well thought semantics. /Kenneth , Erlang/OTP, Ericsson _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenneth.lundin@REDACTED Mon May 21 15:27:52 2012 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Mon, 21 May 2012 15:27:52 +0200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: <1337534022.48017.YahooMailNeo@web130103.mail.mud.yahoo.com> References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> <1337534022.48017.YahooMailNeo@web130103.mail.mud.yahoo.com> Message-ID: The reason for not having documentation of the BEAM instructions is just a matter of prioritization nothing else. Given a limited amount of resources we have not focused much on the internal design documentation, since we have enough designers knowing this. I don't think a documentation of these things is so far away now (but no promise). I also want to comment regarding stability and compatibility of the .beam file format and thus instruction set. We always keep backwards compatibility with 2 major releases. This means that for example an R15B based system can load and run .beam files produced with an R13B based system. In practice this means 3-4 years of compatibility, The other way around, i.e loading .beam files produced with R15B on an older system is not supported. /Kenneth Erlang/OTP, Ericsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Mon May 21 16:45:00 2012 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 21 May 2012 16:45:00 +0200 Subject: [erlang-questions] Weird Client SSL Behavior / Performance In-Reply-To: <4FB8FF67.6070108@berlin.ccc.de> References: <4FB8FF67.6070108@berlin.ccc.de> Message-ID: Hi! We are currently working on eliminating some bottlenecks in ssl. If you want to beta-test I can drop you a mail privatly in a couple of days with a git-branch. Regards Ingela Erlang/OTP team Ericsson AB 2012/5/20, John-Paul Bader : > Hey, > > > recently we had to implement a service that would send about 50 - 500 > https requests per second to a 3rd party API that has an average latency > of about 400-800ms. > > Therefor we had to open a lot of parallel connections to the destination > host. First we have used lhttpc but after some time we observed really > weird behavior. While lhttpc would use some kind of connection pooling, > it was constantly leaking processes that were stuck in prim_inet:recv. > Netstat showed a continously growing number of connections stuck in WAIT > or TIME_WAIT for port 443. After some time all sockets on the machine > were used and no communication was possible anymore. > > After trying out various settings for lhttpc without any improvement on > the situation we added ibrowse instead. ibrowse behaved much nicer as it > was not leaking processes and also did not accumulate the WAIT or > TIME_WAIT connections that showed up before in netstat. We told it to > open 300 connections and that was the exact amount it opened. > > Now this went fine like it should for about 15 or 20 minutes before _no_ > traffic was going through that 300 connections. The VM accumulated load > and basically was not communicating with the 3rd party API. > Interestingly enough netstat still showed that all 300 connections were > established. > > We started to investigate on the true cause for this behavior. At some > point we set up a local nginx with ssl and tried it there to rule out > the 3rd party API. The behavior was the same. > > Then we tried to use http instead of https and boom - that went super > smooth just like it should. 300 connections, minimal load on the VM, no > leaking processes and it kept running for more than 15 minutes. > > Then we switched back to the real API and back to https and tried to > figure out what part in the VM was holding us back but we could not > really find a solution. We only saw that a lot of message were > accumulating in the outer gen_fsm:loop of ssl after those 15 minutes. > > In the end, after multiple days of investigating and experiments, we set > up stunnel on the same machine in client mode that was connecting to the > 3rd party API. This way our erlang service just talked plain http with > stunnel. This performs extremely well for multiple days now. > > This is somehow unsatisfactory though. > > First of all I'd really like to know why the problem occurs in the first > place and why it creates so much load in general. Stunnel deals with the > same load with 1/3 or even less than that of what the erlang vm needs. > For now at least I'm a little bit underwhelmed of erlangs ssl stability > and performance (I was bitten by the R14B03 bug as well). > > Secondly, lhttpc socket handling is not really that great. I'm sure it > works fine for http or in low load https scenarios but in our case, > leaking processes, using up all sockets and by that stopping the service > all together was super bad. > > Is there anybody with similar observations or maybe even solutions? > > > ~ John > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From hukl@REDACTED Mon May 21 16:51:10 2012 From: hukl@REDACTED (John-Paul Bader) Date: Mon, 21 May 2012 16:51:10 +0200 Subject: [erlang-questions] Weird Client SSL Behavior / Performance In-Reply-To: References: <4FB8FF67.6070108@berlin.ccc.de> Message-ID: <4FBA565E.4080405@berlin.ccc.de> Hey, if those bottlenecks are related to the client issues which I have described I would be interested. ~ John Ingela Andin wrote: > Hi! > > We are currently working on eliminating some bottlenecks in ssl. If > you want to beta-test > I can drop you a mail privatly in a couple of days with a git-branch. > > Regards Ingela Erlang/OTP team Ericsson AB > > 2012/5/20, John-Paul Bader: >> Hey, >> >> >> recently we had to implement a service that would send about 50 - 500 >> https requests per second to a 3rd party API that has an average latency >> of about 400-800ms. >> >> Therefor we had to open a lot of parallel connections to the destination >> host. First we have used lhttpc but after some time we observed really >> weird behavior. While lhttpc would use some kind of connection pooling, >> it was constantly leaking processes that were stuck in prim_inet:recv. >> Netstat showed a continously growing number of connections stuck in WAIT >> or TIME_WAIT for port 443. After some time all sockets on the machine >> were used and no communication was possible anymore. >> >> After trying out various settings for lhttpc without any improvement on >> the situation we added ibrowse instead. ibrowse behaved much nicer as it >> was not leaking processes and also did not accumulate the WAIT or >> TIME_WAIT connections that showed up before in netstat. We told it to >> open 300 connections and that was the exact amount it opened. >> >> Now this went fine like it should for about 15 or 20 minutes before _no_ >> traffic was going through that 300 connections. The VM accumulated load >> and basically was not communicating with the 3rd party API. >> Interestingly enough netstat still showed that all 300 connections were >> established. >> >> We started to investigate on the true cause for this behavior. At some >> point we set up a local nginx with ssl and tried it there to rule out >> the 3rd party API. The behavior was the same. >> >> Then we tried to use http instead of https and boom - that went super >> smooth just like it should. 300 connections, minimal load on the VM, no >> leaking processes and it kept running for more than 15 minutes. >> >> Then we switched back to the real API and back to https and tried to >> figure out what part in the VM was holding us back but we could not >> really find a solution. We only saw that a lot of message were >> accumulating in the outer gen_fsm:loop of ssl after those 15 minutes. >> >> In the end, after multiple days of investigating and experiments, we set >> up stunnel on the same machine in client mode that was connecting to the >> 3rd party API. This way our erlang service just talked plain http with >> stunnel. This performs extremely well for multiple days now. >> >> This is somehow unsatisfactory though. >> >> First of all I'd really like to know why the problem occurs in the first >> place and why it creates so much load in general. Stunnel deals with the >> same load with 1/3 or even less than that of what the erlang vm needs. >> For now at least I'm a little bit underwhelmed of erlangs ssl stability >> and performance (I was bitten by the R14B03 bug as well). >> >> Secondly, lhttpc socket handling is not really that great. I'm sure it >> works fine for http or in low load https scenarios but in our case, >> leaking processes, using up all sockets and by that stopping the service >> all together was super bad. >> >> Is there anybody with similar observations or maybe even solutions? >> >> >> ~ John >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From dmercer@REDACTED Mon May 21 16:57:11 2012 From: dmercer@REDACTED (David Mercer) Date: Mon, 21 May 2012 09:57:11 -0500 Subject: [erlang-questions] How to debug "Kernel pid terminated" In-Reply-To: <6EA80B3A-6752-43F3-BBD6-999F542201BD@me.com> References: <014c01cd32dc$1351bc10$39f53430$@com> <6EA80B3A-6752-43F3-BBD6-999F542201BD@me.com> Message-ID: <009201cd3762$01451640$03cf42c0$@com> On Monday, May 21, 2012, Rudolph van Graan wrote: > I had a quick look at your crash below. If I have to guess, you are calling > an external system in a gen_server's init function and this function is > causing a crash. It seems as though this init is part of a supervision tree. > My recommendation is that you move any code like this out of the inits that > can crash the whole tree. I would probably have implemented a different > process that managed the call to the external application and make it robust > so that it can't crash when this condition occurs. Thanks for the hint. I'm calling Richard Carlsson's file_monitor module in init, so I'll delve into the source code there to see what it is doing. Cheers, DBM From richardprideauxevans@REDACTED Mon May 21 18:02:30 2012 From: richardprideauxevans@REDACTED (Richard Evans) Date: Mon, 21 May 2012 17:02:30 +0100 Subject: [erlang-questions] Simple question: how to read a trace using trace_client Message-ID: I am trying to trace some function-calls, redirect the trace to a file, and then, later, read that file to see the messages. To trace, I am doing: dbg:start(). % to redirect to a file "trace_output" PortFun = dbg:trace_port(file,{"/tmp/trace_output", wrap,atom_to_list(node()),10000000,20}). dbg:tracer(port, PortFun). dbg:tp(lobby, handle_client_message, 2, []). dbg:p(all, c). % now I execute the program for a certain period of time and watch the trace_output file build up % finishing the trace dbg:stop_clear(). This successfully produces a binary file. The longer I run the program, the larger it gets. This seems right. The bit that I don't get is how to read the binary file. I tried using trace_client: A=dbg:trace_client(file, "/tmp/trace_output"). This returns me the PID of the trace_client process. But I don't know what to do with this PID! What I want to see is the list of function-calls in the file. But all I have is the PID. What do I do now? yours puzzledly, Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Mon May 21 17:40:21 2012 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 21 May 2012 17:40:21 +0200 Subject: [erlang-questions] Erlang suitability In-Reply-To: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> References: <1337331624.44767.YahooMailNeo@web162101.mail.bf1.yahoo.com> Message-ID: <4FBA61E5.2090005@erlang-solutions.com> On 5/18/12 11:00 AM, Ovid wrote: > > Use case 1: If the *total* of all of those small amounts exceeds a > daily cap or an all-time cap, all 75 boxes must immediately stop > spending bidding in auctions. It seems that each box can run a > separate Erlang process and write out "winning bid" information to an > Mnesia database and all boxes can read the total amount spent from > that to determine if it should stop bidding. > Have you considered using the opposite direction? A bidder takes a lease on a part of the possible cap. If there is a lot of remaining money you can pick out a fairly large lease and as you get closer to the spending limit you can decrease the amount of extra money you can get for bids. For instance, you may know there is $100 in the pool and a bidder needs to do a bid. Hence it allocates $5 to itself and can now roam on those $5 as it sees fit. If the leaser crashes, you have a monitor on it, so you will get notified and can recover money it did not spend. You will still need some database solution that is running on multiple nodes to battle a single-point-of-failure. But this idea works even if this is the case. The advantage is that this scales a lot better. A bidder now knows for how much it is allowed to bid and can then operate independently on a synchronization point of "How much more am I allowed to spend?" > This seems trivial to set up. It isn't. But Erlang could perhaps lend you some tools to make this work. > Use case 2: we periodically need to reauthenticate to the auction > system. We MUST NOT have all 75 boxes trying to reauthenticate at the > same time because we will be locked out of the system if we do this. > Having a central box handling reauthentication is a single point of > failure that we would like to avoid, but we don't know what design > pattern Erlang would use to ensure that only one of the 75 Erlang > instances would attempt to reauthenticate at any one time (all 75 > boxes can share the same authentication token). > Your problem is that of a hypothesis: waiting-requires-locking. That is, if you need to wait on others, you need to synchronize who is doing things in what order - and that requires you to block on a single point. This in turn makes it hard to avoid the single-point-of-failure. If you know that you may have up to K simultaneous authentications running it becomes easier to handle because then you have some leverage in how much synchronization that is needed. There is no really good solution though. A problem here is the split-brain scenario, where your network gets disconnected, but individual nodes are still operating and can authenticate. In that case, you might have double auths if you pick the lowest possible node in a list. What you should really do is to use *risk* as a deciding factor. You must evaluate the risk of something happening to the impact. It is, for instance, more likely that a node is lost than the network connectivity is in a split brain where you can still authenticate. Hence you decide to take that risk probably - knowing that certain split-brain scenarios cannot be handled by the solution. If there is anything I wish to tell people about distributed programming it is that it is a fuzzy logic. On a single machine you are *not* safe since it can die. On multiple machines you have an error rate and different types of errors. What is important is that you control the error rate rather than let it flow by itself. You will almost never hit a situation where 100% stability can be guaranteed if you also need speed. So it becomes a question of risk management and trade-offs. -- Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen, DK -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Mon May 21 21:43:18 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Mon, 21 May 2012 12:43:18 -0700 Subject: [erlang-questions] Declarative Security in Erlang In-Reply-To: References: Message-ID: Does anyone have any thoughts on this? I was thinking about just starting a new process every time I access the API and then store the current user in there. Any thoughts? Thanks again! On Sat, May 19, 2012 at 12:38 PM, Andrew Berman wrote: > Hey all, > > I'm trying to mimic (as much as I can) what JEE 6 does with security on > domain objects. Essentially what I'd like to do is create an annotation > using Tim's awesome annotation code ( > https://github.com/hyperthunk/annotations) and test on a user's roles. > The one issue I'm wrestling with is how to get the user into the > annotation. The obvious way is to have every function I put the annotation > on take in a user record and then loop through the arguments of the > function looking for the user record. That way doesn't seem very elegant > to me, though. I really want to just say something like User = > get_current_user(...). Has anyone tackled this sort of issue or have any > advice on how to handle it in an Erlang safe manner? > > Thanks, > > Andrew > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Mon May 21 21:59:57 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 21 May 2012 20:59:57 +0100 Subject: [erlang-questions] Declarative Security in Erlang In-Reply-To: References: Message-ID: <4FBA9EBD.2030904@gmail.com> On 21/05/2012 20:43, Andrew Berman wrote: > Does anyone have any thoughts on this? I was thinking about just > starting a new process every time I access the API and then store the > current user in there. Any thoughts? > There are lots of ways to do this, so I guess it just depends. Passing the user about in your various functions might seem bothersome, but it is 'pure' - and that has merit. If you're holding on to a user in either a request or session like object, then putting the security context information into the request/session data structure seems like a perfectly sane approach to me. Even though in J2SE you'd make a static call such as SecurityContextHolder.getCurrentUser you can still find this information tucked away in context data, for example, in web apps and the like. Interestingly enough, those static calls are often resolved under the covers but reading a thread local variable, so having a process per activity/request and pushing the user data into the process dictionary might not be the wrong thing to do after all. Just bare in mind that you can't make *safe* use of this unless you're absolutely certain whichever code executes get_current_user is definitely running in the same process. So you have two choices here really: around_advice(#annotation{data={mode, pdict}}, M, F, Inputs) -> case get(current_user) of undefined-> handle_restricted(M, F, Inputs); #user{}-> annotation:call_advised(M, F, Inputs) end. Or you could you ets/mnesia/gen_server or whatever instead of the process dictionary - these are *safer* in that they're less likely to surprise you if accessed from some 'other' process than the one you expected. Another approach would be to store the index (1 bound) of the user record in the argument set for the annotation, like so: around_advice(#annotation{data={user_record_idx, Idx}}, M, F, Inputs) -> User = lists:nth(Idx, Inputs), case check(User) of restricted-> handle_restricted(M, F, Inputs); ok-> annotation:call_advised(M, F, Inputs) end. Of course you don't have to use 'around_advice' - if you just need 'before_advice' to prevent unauthorised access, then that will work fine too. HTH - and don't forgot to submit any issues/bugs you come across! :) Cheers, Tim > Thanks again! > > On Sat, May 19, 2012 at 12:38 PM, Andrew Berman > wrote: > > Hey all, > > I'm trying to mimic (as much as I can) what JEE 6 does with > security on domain objects. Essentially what I'd like to do is > create an annotation using Tim's awesome annotation code > (https://github.com/hyperthunk/annotations) and test on a user's > roles. The one issue I'm wrestling with is how to get the user > into the annotation. The obvious way is to have every function I > put the annotation on take in a user record and then loop through > the arguments of the function looking for the user record. That > way doesn't seem very elegant to me, though. I really want to > just say something like User = get_current_user(...). Has anyone > tackled this sort of issue or have any advice on how to handle it > in an Erlang safe manner? > > Thanks, > > Andrew > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Mon May 21 22:04:00 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 21 May 2012 21:04:00 +0100 Subject: [erlang-questions] Declarative Security in Erlang In-Reply-To: <4FBA9EBD.2030904@gmail.com> References: <4FBA9EBD.2030904@gmail.com> Message-ID: <4FBA9FB0.7020503@gmail.com> Holy crap the indentation on that post got screwed somehow. Allow me to tidy up a bit..... around_advice(#annotation{data={mode, pdict}}, M, F, Inputs) -> case get(current_user) of undefined -> handle_restricted(M, F, Inputs); #user{} -> annotation:call_advised(M, F, Inputs) end. around_advice(#annotation{data={user_record_idx, Idx}}, M, F, Inputs) -> User = lists:nth(Idx, Inputs), case check(User) of restricted -> handle_restricted(M, F, Inputs); ok -> annotation:call_advised(M, F, Inputs) end. From rexxe98@REDACTED Mon May 21 22:22:03 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Mon, 21 May 2012 13:22:03 -0700 Subject: [erlang-questions] Declarative Security in Erlang In-Reply-To: <4FBA9FB0.7020503@gmail.com> References: <4FBA9EBD.2030904@gmail.com> <4FBA9FB0.7020503@gmail.com> Message-ID: Cool, thanks Tim. I am using a before_advice since it is only security. I figured that your two suggestions are the best way to do it. As you said, passing around the user is the purest way, so I'll probably just go with that method. I like your suggestion of passing in the index of the user record in the inputs. I was planning on using a gen_server and every time a function is accessed it creates a new process which stores the user. I feel like this could get messy though. Thanks again for the annotations project, I love it! --Andrew On Mon, May 21, 2012 at 1:04 PM, Tim Watson wrote: > Holy crap the indentation on that post got screwed somehow. Allow me to > tidy up a bit..... > > > around_advice(#annotation{**data={mode, pdict}}, M, F, Inputs) -> > case get(current_user) of > undefined -> > handle_restricted(M, F, Inputs); > #user{} -> > annotation:call_advised(M, F, Inputs) > end. > > around_advice(#annotation{**data={user_record_idx, Idx}}, M, F, Inputs) -> > User = lists:nth(Idx, Inputs), > case check(User) of > restricted -> > handle_restricted(M, F, Inputs); > ok -> > annotation:call_advised(M, F, Inputs) > end. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Mon May 21 22:47:58 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 21 May 2012 21:47:58 +0100 Subject: [erlang-questions] Declarative Security in Erlang In-Reply-To: References: <4FBA9EBD.2030904@gmail.com> <4FBA9FB0.7020503@gmail.com> Message-ID: <4FBAA9FE.2010801@gmail.com> On 21/05/2012 21:22, Andrew Berman wrote: > Cool, thanks Tim. I am using a before_advice since it is only > security. I figured that your two suggestions are the best way to do > it. As you said, passing around the user is the purest way, so I'll > probably just go with that method. I like your suggestion of passing > in the index of the user record in the inputs. > > I was planning on using a gen_server and every time a function is > accessed it creates a new process which stores the user. I feel like > this could get messy though. > > Thanks again for the annotations project, I love it! > Cool, glad you're enjoying it! So on the production code, this would look something like: -requires_authorisation({user_record_index, 1}). read_user_info(User, InfoType) -> db:fetch(user_info, User, InfoType). That seems reasonable to me. If all the functions in a module place the user argument in the same position, it ought to be possible to annotate a module so that *all* exported functions are annotated - for something like this it'd make life easier to just say: %%% -module(user_db). -include_lib("annotations/include/annotation.hrl"). -requires_authorisation([{scope, user}, {arg_index, 1}]). %%% In theory, that should work OOTB today, although I'm not 100% about the module level stuff as I've mainly focused on testing (and using) the function level annotations. > --Andrew > > On Mon, May 21, 2012 at 1:04 PM, Tim Watson > wrote: > > Holy crap the indentation on that post got screwed somehow. Allow > me to tidy up a bit..... > > > around_advice(#annotation{data={mode, pdict}}, M, F, Inputs) -> > case get(current_user) of > undefined -> > handle_restricted(M, F, Inputs); > #user{} -> > annotation:call_advised(M, F, Inputs) > end. > > around_advice(#annotation{data={user_record_idx, Idx}}, M, F, > Inputs) -> > User = lists:nth(Idx, Inputs), > case check(User) of > restricted -> > handle_restricted(M, F, Inputs); > ok -> > annotation:call_advised(M, F, Inputs) > end. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue May 22 00:42:53 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 22 May 2012 10:42:53 +1200 Subject: [erlang-questions] Question/Alternative on Frames Proposal [Warning: Long] In-Reply-To: References: <4FB575D5.2020702@gmail.com> <1337300505.68472.YahooMailNeo@web130106.mail.mud.yahoo.com> <1338fcafd15281751fb6491bff9b6109.squirrel@chasm.otago.ac.nz> Message-ID: On 22/05/2012, at 12:15 AM, Garrett Smith wrote: >> >> I'm not sure what you are asking. The paper that you mentioned made it >> quite clear that they were NOT experimenting with frames or anything in >> the same area of design space. > > Richard, I'm not seeing what you're seeing. > > That's a PDF of presentation slides from 2011 User Conf-- it lists > "frames" in the title and as a input to their process. We're talking about http://www.erlang-factory.com/upload/presentations/468/EUC_Hashes2011.pdf and that's all I have to go on. Slide 0: title Slide 1: there's always room for improvements RECORDS Hash maps Slide 2: outline of records (few named fields, many instances, matchable) Slide 3: not a distinct data type, not suitable for many fields Slide 4: outline of hash maps (proplists, dict, gb_trees, gb_sets) Slide 5: what's wrong with proplists Slide 6: what's wrong with dict Slide 7: Requirements/Goals ***FOR HASHMAPS*** And it is quite clear from content slide 7 that they are talking about something that can do the job of dict and gb_tree; suitability for type checking is NOT listed. More precisely, content slide 7 gives the impression of a vague desire for something that's a floor polish AND a desert topping, a bulk hashed container AND a possible replacement for records. Slide 8: mentions me, Joe Armstrong, Anton Lavrik, Phil Bagwell. Slide 9: Erlson syntax. Slide 10: >>> We have implemented prototypes and performed measurements using the Vlists and HAMT datastructures internally in the Erlang VM <<< It is very very clear from this that the experiments concerned something suitable for *LARGE* data structures. Slide 11: Diagram of records as tuples. Slide 12: Diagram of VLists / vhash lists. Slide 13: Diagram of Hash Array Mapped Tries. Popcount is one of those things that *ought* to be fast but that often get left out of actual chips. POPCNT wasn't on PCs until Core i7, I believe. It's in the SPARC V9 architecture, but is not in all SPARC V9 _hardware_, like the machine on my desk. Slide 14: Diagram of hash table with exception list Slide 15: Another such diagram This appears to be an adaptation of a 1980s technique for O(1) array update; the latest version is compact and in place while old versions have a list of deltas. Slide 16: Graph of time in microseconds against number of elements inserted for dict, gb_trees, vlist_hash, and hamt (up to ~ 30) Slide 17: Another such (up to ~ 65 000) Slide 18: Graph of lookup time (up to ~ 30) Slide 19: Another such (up to ~ 65 000) Slide 20: Graph of memory (up to ~ 30 elements) These data structures take from 4 to 8 words per element. Frames, in contrast, take 1 to 2 words. Slide 21: Another such graph (up to ~ 65 000 elements) Slide 22: Table of requirements by data structures. Frames are conspicuous by their absence. Slide 23: "new better records" "and Hashmaps" "ARE TWO DIFFERENT THINGS." I agree with this distinction and made it a decade ago. This talk was about Hashmaps, NOT about anything credible as new better records. It will be really good for Erlang to have a better implementation of hashmaps. This was good work. It owes nothing to me; I'd never even heard of Hash Array Mapped Tries before. I look forward to seeing something coming of it. But there is NOTHING reported in that presentation that evaluates alternatives for "new better records". > In any event, is not relevant to any discussion here what the OTP team > is actually working on? Is this is a blip of interest in a decade old > proposal that has no shot of getting into the language? What's there is a solid examination of "can we provide better support for functionally pure persistent dictionaries than what we've already got" that briefly drags a "new better records" red herring over the track at the beginning and then drops it. It expresses the _wish_ that one data structure might be usable for both purposes, and I suppose you could argue that slide 20 does *implicitly* demonstrate that the current set of candidates for better dictionaries are *NOT* credible candidates for better records. To be fair, there's no reason why a single logical data structure has to have a *uniform* representation at run time (as I was saying to a data base class yesterday). One could use something like frames up to a certain size and something else above that size. one data structure _might From erlangsiri@REDACTED Tue May 22 09:14:23 2012 From: erlangsiri@REDACTED (Siri Hansen) Date: Tue, 22 May 2012 09:14:23 +0200 Subject: [erlang-questions] Simple question: how to read a trace using trace_client In-Reply-To: References: Message-ID: Hi Richard! You need to specify the complete WrapFileSpec in the call to dbg:trace_client, like this: dbg:trace_client(file,{"/tmp/trace_output", wrap,atom_to_list(node()),10000000,20}). Regards /siri 2012/5/21 Richard Evans > I am trying to trace some function-calls, redirect the trace to a file, > and then, later, read that file to see the messages. > > To trace, I am doing: > > dbg:start(). > > % to redirect to a file "trace_output" > PortFun = dbg:trace_port(file,{"/tmp/trace_output", > wrap,atom_to_list(node()),10000000,20}). > dbg:tracer(port, PortFun). > > dbg:tp(lobby, handle_client_message, 2, []). > dbg:p(all, c). > > % now I execute the program for a certain period of time and watch the > trace_output file build up > > % finishing the trace > dbg:stop_clear(). > > > > This successfully produces a binary file. The longer I run the program, > the larger it gets. This seems right. > > The bit that I don't get is how to read the binary file. > > I tried using trace_client: > > A=dbg:trace_client(file, "/tmp/trace_output"). > > > This returns me the PID of the trace_client process. > > But I don't know what to do with this PID! > > What I want to see is the list of function-calls in the file. But all I > have is the PID. > > What do I do now? > > yours puzzledly, > Richard > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue May 22 09:34:23 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 22 May 2012 09:34:23 +0200 Subject: [erlang-questions] [ANN] Sheriff 0.4.0 Message-ID: <4FBB417F.4020003@ninenines.eu> Hello, Sheriff 0.4.0 is now available. Sheriff is a parse transform for type based validation. What changed: * Added a few rarely used types. * Added a more convenient sheriff:check(Var, "list(mytype())") notation. * Fix compilation and type check issues with records. README now includes documentation about the new notation and records type checking. You can find it at https://github.com/extend/sheriff As always I'm interested in any feedback you may have, good or bad. Feel free to drop me a mail about the project! Thanks! -- Lo?c Hoguin Erlang Cowboy Nine Nines From zerthurd@REDACTED Tue May 22 13:37:33 2012 From: zerthurd@REDACTED (Maxim Treskin) Date: Tue, 22 May 2012 18:37:33 +0700 Subject: [erlang-questions] [ANN] Sheriff 0.4.0 In-Reply-To: <4FBB417F.4020003@ninenines.eu> References: <4FBB417F.4020003@ninenines.eu> Message-ID: Thank you Lo?c! Sheriff is very cool tool. I think I can use it in number of projects. On 22 May 2012 14:34, Lo?c Hoguin wrote: > Hello, > > Sheriff 0.4.0 is now available. > > Sheriff is a parse transform for type based validation. > > What changed: > > * Added a few rarely used types. > * Added a more convenient sheriff:check(Var, "list(mytype())") notation. > * Fix compilation and type check issues with records. > > README now includes documentation about the new notation and records type > checking. > > You can find it at https://github.com/extend/**sheriff > > As always I'm interested in any feedback you may have, good or bad. Feel > free to drop me a mail about the project! > > Thanks! > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -- Maxim Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From hm@REDACTED Tue May 22 14:37:09 2012 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Tue, 22 May 2012 14:37:09 +0200 Subject: [erlang-questions] [ANN] Lux - LUcid eXpect scripting Message-ID: I just want to announce that Tail-f has released a very useful test tool as Open Source. It is called Lux (LUcid eXpect scripting) and provides scripting with an Expect-style execution of commands in a powerful but simple fashion. Lux is written in Erlang, but is a general-purpose test tool and its usage is not limited to only test Erlang programs. With Lux it is possible to - simplify automated testing - control interactive programs by sending textual input to them and using regular expressions to ensure that their output matches the expectations - perform detailed post mortem analysis of test suite results - interactively debug and trace single test cases - get editor support for editing scripts by using the Emacs mode You can find the source here: https://github.com/hawk/lux and the on-line documentation here: https://github.com/hawk/lux/blob/master/doc/lux.md /H?kan --- H?kan Mattsson, Tail-f Systems (www.tail-f.com) From witeman.g@REDACTED Tue May 22 15:03:33 2012 From: witeman.g@REDACTED (Zheng Zhibin) Date: Tue, 22 May 2012 21:03:33 +0800 Subject: [erlang-questions] [ANN] Lux - LUcid eXpect scripting In-Reply-To: References: Message-ID: <8E746114-06AF-4040-8A1B-0994D7ADC7C1@gmail.com> Awesome! /Witeman ? 2012-5-22???8:37?H?kan Mattsson ??? > I just want to announce that Tail-f has released > a very useful test tool as Open Source. > > It is called Lux (LUcid eXpect scripting) and > provides scripting with an Expect-style execution > of commands in a powerful but simple fashion. Lux > is written in Erlang, but is a general-purpose > test tool and its usage is not limited to only > test Erlang programs. > > With Lux it is possible to > > - simplify automated testing > - control interactive programs by sending textual > input to them and using regular expressions to > ensure that their output matches the expectations > - perform detailed post mortem analysis of test suite results > - interactively debug and trace single test cases > - get editor support for editing scripts by using the Emacs mode > > You can find the source here: > > https://github.com/hawk/lux > > and the on-line documentation here: > > https://github.com/hawk/lux/blob/master/doc/lux.md > > /H?kan > --- > H?kan Mattsson, Tail-f Systems (www.tail-f.com) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jesper.louis.andersen@REDACTED Tue May 22 16:10:44 2012 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 22 May 2012 16:10:44 +0200 Subject: [erlang-questions] Is the performance cost of inet:i() big or small? In-Reply-To: <116fc4e4.9d78.13745c1c016.Coremail.cloudzen@163.com> References: <116fc4e4.9d78.13745c1c016.Coremail.cloudzen@163.com> Message-ID: <4FBB9E64.2080004@erlang-solutions.com> On 5/13/12 12:32 PM, skyman wrote: > Hi all, > inet:i() can print all socket info statistics, and it is very useful > for monitoring the network for the Erlang node. Is the performance > cost of inet:i() big or small? Can I use it in the production system? > I learned a trick during the Erlang Factory 2012 by Patrik Nyblom: When in doubt, read the source code of the virtual machine. inet:i() makes a call to erlang:ports() and then afterwards does some work to format the result of this nicely. The erlang:ports/0 call is a BIF, so you can go to the BIF-table and look up that its name in the C source is ports_0. Going to this function and reading about the blocking behaviour tells you that you can only have one such snapshot call running at a time and it only blocks concurrency on ports one port at a time (It has to, it bumps a snapshot counter in the port struct). This would suggest that the performance cost is such that it won't block the whole system when it is run (which is good), but it does have a cost if you call all the time. More interestingly, the call also allocates quite the amount of data. So you may end up with a spike in memory usage when you make the call and the result is then dependent on how many ports you have. My guess is that it won't be highly problematic to use in production unless you put it into a tight loop. If you are in doubt, you should measure. -- Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen, DK -------------- next part -------------- An HTML attachment was scrubbed... URL: From james@REDACTED Tue May 22 19:27:32 2012 From: james@REDACTED (James Aimonetti) Date: Tue, 22 May 2012 10:27:32 -0700 Subject: [erlang-questions] lists:concat with non-string sublists Message-ID: <4FBBCC84.5090901@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 According to the docs[1], lists:concat/1 converts the elements in the list to their string representation. The spec for Thing being atom() | integer() | float() | string(). However, in the actual implementation in lists.erl, the is_list/1 guard is used in thing_to_list/1, not a more specific check that the list is a string(). So, according to the docs, I would expect: > lists:concat([[{foo, 123}], [{bar, 234}]]). to fail. It doesn't, and returns [{foo, 123}, {bar,234}]. Reading the implementation of concat/1 its easy to see why this works. My question is, should the docs be updated to allow Thing to be a list of any term so Dialyzer won't issue an error when using sub-lists with tuples (in my case, the result of an ets:match/2), or should thing_to_list/1 do more inspection of a list element? Thanks, James [1] http://www.erlang.org/doc/man/lists.html#concat-1 - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJPu8yEAAoJENc77s1OYoGgk2sH/1T+ZafjSyZXdLLCs0vJcyJe rK/HAVBoKG+8u/t2PxNul3J6abgEnFsad33BA51xMvFfNnIDBifGKT0HDaXaXXBg gzCgaKMRQWajMHuGLHxWWhFWXFseg8ZK8WoYOkRewwoC3leTF7C8KfL68cHCLKw7 oMI+46OkjKog1tCuMVmuXXK8LD+z/mKbDhAB7saeLcXwcWJFduTlvmqyvV4wwaIV v9HKDpy21Em/qyICgwO+WaP4CjDrZmTcwAfimJzI06gJOp31m06EOk3aAIW+q1ja GP5zXVCkmZ3/C/63yO7b6m6AXQqi56yOww7c20Im3KtRGjjplWUyUwoJpzMJsNc= =eioK -----END PGP SIGNATURE----- From kostis@REDACTED Tue May 22 19:55:39 2012 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 22 May 2012 19:55:39 +0200 Subject: [erlang-questions] lists:concat with non-string sublists In-Reply-To: <4FBBCC84.5090901@2600hz.com> References: <4FBBCC84.5090901@2600hz.com> Message-ID: <4FBBD31B.8050204@cs.ntua.gr> On 05/22/2012 07:27 PM, James Aimonetti wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > According to the docs[1], lists:concat/1 converts the elements in the > list to their string representation. The spec for Thing being atom() | > integer() | float() | string(). > > However, in the actual implementation in lists.erl, the is_list/1 > guard is used in thing_to_list/1, not a more specific check that the > list is a string(). > > So, according to the docs, I would expect: > >> lists:concat([[{foo, 123}], [{bar, 234}]]). > > to fail. It doesn't, and returns [{foo, 123}, {bar,234}]. Reading the > implementation of concat/1 its easy to see why this works. > > My question is, should the docs be updated to allow Thing to be a list > of any term so Dialyzer won't issue an error when using sub-lists with > tuples (in my case, the result of an ets:match/2), or should > thing_to_list/1 do more inspection of a list element? You are reading the specs in the wrong way, I am afraid. Specs are contracts: they describe what guarantee the current implementation of a function provides to its callers. The implementation may allow for more than that, but the spec is there to tell you that if you break its premises (the types in its arguments) you really have no guarantee that your code will work without errors the next OTP version. Typical example of this is something like lists:append/2: -spec append(list(), list()) -> list(). % or something more involved but the implementation may allow for calls of the form: Eshell V5.9.2 (abort with ^G) 1> lists:append([], 3.14). 3.14 but you probably would not want to change its spec to allow term() for its second argument and return, would you? Kostis From g@REDACTED Tue May 22 20:39:23 2012 From: g@REDACTED (Garrett Smith) Date: Tue, 22 May 2012 13:39:23 -0500 Subject: [erlang-questions] mochiglobal vs ets vs registered process Message-ID: This is a followup on a recent question I had about "meta programming" in Erlang. The gist of the post was on a best practice for extending functionality in Erlang without using registered processes. This is a short hand of the sort of use case: -module(foo). do_something(Value) -> foo_impl:lookup(do_something)(Value). It led me to experiment with some options for storing/retrieving the "pluggable" functionality: - mochiglobal - ets - registered process running under proc_lib The source code for the tests is pasted at the end of this email. The test was to simply read a "global" value 1M times, after having first initialized the facility. Times are of the "run" portion only and do not include the "init" time. Summary: - ets was the fastest, baseline of 1x - mochiglobal was 1.8x slower than ets - registered process was 7.8x slower than ets The registered process call (slowest) was, on average, ~2 microseconds per call slower than ets (fastest). Note that's *micro* seconds. My conclusion: Provided the process doesn't block unnecessarily, using a process to lookup "pluggable" functionality (e.g. an implementation module) is perfectly good for almost any application. IMO it doesn't warrant the use of ets. This may not apply to systems under heavy load, where message passing overhead could grow non-linearly. A number of tests could be run in parallel to get a better feel for this. mochiglobal's use of dynamically generated Erlang modules to serve "global" values is novel, but it doesn't appear warranted on performance grounds alone. The raw test results: 26> timer:tc(tests, run_ets, []). {307550,ok} 27> timer:tc(tests, run_ets, []). {302596,ok} 28> timer:tc(tests, run_ets, []). {313182,ok} 29> timer:tc(tests, run_ets, []). {300729,ok} 30> timer:tc(tests, run_mochiglobal, []). {576356,ok} 31> timer:tc(tests, run_mochiglobal, []). {567248,ok} 32> timer:tc(tests, run_mochiglobal, []). {541281,ok} 33> timer:tc(tests, run_mochiglobal, []). {574178,ok} 34> timer:tc(tests, run_server, []). {2358793,ok} 35> timer:tc(tests, run_server, []). {2425085,ok} 36> timer:tc(tests, run_server, []). {2394177,ok} 37> timer:tc(tests, run_server, []). {2383787,ok} The test code: -module(tests). -export([init_mochiglobal/0, run_mochiglobal/0, init_ets/0, run_ets/0, init_server/0, run_server/0]). -define(KEY, mykey). -define(VALUE, "My Value"). -define(READS, 1000000). init_mochiglobal() -> mochiglobal:put(?KEY, ?VALUE). run_mochiglobal() -> run_mochiglobal(?READS). run_mochiglobal(0) -> ok; run_mochiglobal(N) when N > 0 -> ?VALUE = mochiglobal:get(mykey), run_mochiglobal(N - 1). init_ets() -> ets:new(tests_ets, [named_table]), ets:insert(tests_ets, {?KEY, ?VALUE}). run_ets() -> run_ets(?READS). run_ets(0) -> ok; run_ets(N) when N > 0 -> [{_, ?VALUE}] = ets:lookup(tests_ets, ?KEY), run_ets(N - 1). init_server() -> Server = proc_lib:spawn(fun() -> server_loop(?VALUE) end), register(server, Server). server_loop(Value) -> receive {get, ?KEY, Src} -> Src ! {ok, Value}, server_loop(Value); _ -> server_loop(Value) end. server_get(Key) -> server ! {get, Key, self()}, receive {ok, Value} -> Value after 3000 -> error(timeout) end. run_server() -> run_server(?READS). run_server(0) -> ok; run_server(N) when N > 0 -> ?VALUE = server_get(?KEY), run_server(N - 1). From james@REDACTED Tue May 22 20:46:28 2012 From: james@REDACTED (James Aimonetti) Date: Tue, 22 May 2012 11:46:28 -0700 Subject: [erlang-questions] lists:concat with non-string sublists In-Reply-To: <4FBBD31B.8050204@cs.ntua.gr> References: <4FBBCC84.5090901@2600hz.com> <4FBBD31B.8050204@cs.ntua.gr> Message-ID: <4FBBDF04.4090007@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 05/22/2012 10:55 AM, Kostis Sagonas wrote: > On 05/22/2012 07:27 PM, James Aimonetti wrote: >> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 >> >> According to the docs[1], lists:concat/1 converts the elements in >> the list to their string representation. The spec for Thing being >> atom() | integer() | float() | string(). >> >> However, in the actual implementation in lists.erl, the >> is_list/1 guard is used in thing_to_list/1, not a more specific >> check that the list is a string(). >> >> So, according to the docs, I would expect: >> >>> lists:concat([[{foo, 123}], [{bar, 234}]]). >> >> to fail. It doesn't, and returns [{foo, 123}, {bar,234}]. Reading >> the implementation of concat/1 its easy to see why this works. >> >> My question is, should the docs be updated to allow Thing to be a >> list of any term so Dialyzer won't issue an error when using >> sub-lists with tuples (in my case, the result of an ets:match/2), >> or should thing_to_list/1 do more inspection of a list element? > > You are reading the specs in the wrong way, I am afraid. Specs > are contracts: they describe what guarantee the current > implementation of a function provides to its callers. The > implementation may allow for more than that, but the spec is there > to tell you that if you break its premises (the types in its > arguments) you really have no guarantee that your code will work > without errors the next OTP version. > > Typical example of this is something like lists:append/2: > > -spec append(list(), list()) -> list(). % or something more > involved > > but the implementation may allow for calls of the form: > > Eshell V5.9.2 (abort with ^G) 1> lists:append([], 3.14). 3.14 > > but you probably would not want to change its spec to allow term() > for its second argument and return, would you? > > Kostis > > Indeed not, wrt to the lists:append/2 example. I guess the safer course of action, in my case, is to implement my own concat/1 for my list of lists of tuples, to ensure that lists:concat/1 doesn't change down the road to enforce the spec more strictly? - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJPu98EAAoJENc77s1OYoGgiKUH/j7+LBvVCI+P04opU0AINEoB eLdqNNKp0thBZVS+0ohZ9y02z3ZXL0jRcQ9PTCIgX1xUv215RXL4XJclUPo0eMG1 5gGal09MqpzXb+PKx6F3hJkigPeH3bD/2PXuGNEXnn728KNW+mnM/wjc7PWmY9dX IXJrPCSsRypx2DFr/S/LXuGW3QOkET84fhFpVomZM3ntQC0KIsU3+AyeHnlI3HRH S2G30OymPmITcNAEWq3BlT10bcDoE9/geWmB668MFh4LbIkf2svHaKTvY7hS6DOO 5O7dFNECDbMRkZzmIoY6n71aK8syKEumQDdW0e4BJrPUoP3jgb7bTzgE/is2xK0= =OkDG -----END PGP SIGNATURE----- From ponton@REDACTED Tue May 22 20:55:03 2012 From: ponton@REDACTED (Tomasz Maciejewski) Date: Tue, 22 May 2012 20:55:03 +0200 Subject: [erlang-questions] mochiglobal vs ets vs registered process In-Reply-To: References: Message-ID: Dnia 22-05-2012 o 20:39:23 Garrett Smith napisa?(a): > The test was to simply read a "global" value 1M times, after having > first initialized the facility. Times are of the "run" portion only > and do not include the "init" time. I see your test reading 1M times *the same* value. I'd rather see the test when you write and read 1M random keys/values. The single value could be cached spoiling the test. -- Tomasz Maciejewski From g@REDACTED Tue May 22 21:02:20 2012 From: g@REDACTED (Garrett Smith) Date: Tue, 22 May 2012 14:02:20 -0500 Subject: [erlang-questions] mochiglobal vs ets vs registered process In-Reply-To: References: Message-ID: Hi Tomasz, On Tue, May 22, 2012 at 1:55 PM, Tomasz Maciejewski wrote: > Dnia 22-05-2012 o 20:39:23 Garrett Smith napisa?(a): > >> The test was to simply read a "global" value 1M times, after having >> first initialized the facility. Times are of the "run" portion only >> and do not include the "init" time. > > I see your test reading 1M times *the same* value. I'd rather see the test > when you write and read 1M random keys/values. The single value could be > cached spoiling the test. Whatever caches are in play here, I *want* to test them. This is not a data problem, it's an extensibility problem. This use would only ever be reading a small number of values. Garrett From bob@REDACTED Tue May 22 21:06:54 2012 From: bob@REDACTED (Bob Ippolito) Date: Tue, 22 May 2012 12:06:54 -0700 Subject: [erlang-questions] mochiglobal vs ets vs registered process In-Reply-To: References: Message-ID: On Tue, May 22, 2012 at 11:55 AM, Tomasz Maciejewski wrote: > Dnia 22-05-2012 o 20:39:23 Garrett Smith napisa?(a): > > > The test was to simply read a "global" value 1M times, after having >> first initialized the facility. Times are of the "run" portion only >> and do not include the "init" time. >> > > I see your test reading 1M times *the same* value. I'd rather see the test > when you write and read 1M random keys/values. The single value could be > cached spoiling the test. mochiglobal is unsuitable for writes at runtime, at least prior to R16, because loading new code blocks the whole VM (possibly for *seconds*). The (only) occasion where you'll see mochiglobal work better than ets is when you have a very large data structure that you need to read at runtime with high process concurrency. The goal is to share memory and avoid garbage collector churn. In the tests the given value is so small AND you're only reading it from one process, so of course that there's no benefit. -bob -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Tue May 22 21:15:32 2012 From: g@REDACTED (Garrett Smith) Date: Tue, 22 May 2012 14:15:32 -0500 Subject: [erlang-questions] mochiglobal vs ets vs registered process In-Reply-To: References: Message-ID: On Tue, May 22, 2012 at 2:06 PM, Bob Ippolito wrote: > > On Tue, May 22, 2012 at 11:55 AM, Tomasz Maciejewski > wrote: >> >> Dnia 22-05-2012 o 20:39:23 Garrett Smith napisa?(a): >> >>> The test was to simply read a "global" value 1M times, after having >>> first initialized the facility. Times are of the "run" portion only >>> and do not include the "init" time. >> >> I see your test reading 1M times *the same* value. I'd rather see the test >> when you write and read 1M random keys/values. The single value could be >> cached spoiling the test. > > mochiglobal is unsuitable for writes at runtime, at least prior to R16, > because loading new code blocks the whole VM (possibly for *seconds*). The > (only) occasion where you'll see mochiglobal work better than ets is when > you have a very large data structure that you need to read at runtime with > high process concurrency. The goal is to share memory and avoid garbage > collector churn. In the tests the given value is so small AND you're only > reading it from one process, so of course that there's no benefit. This is good to know. I included mochiglobal in the tests because it was suggested in the prior thread as a possible approach. In this use case, the values are quite small. Garrett From nem@REDACTED Tue May 22 22:39:10 2012 From: nem@REDACTED (Geoff Cant) Date: Tue, 22 May 2012 13:39:10 -0700 Subject: [erlang-questions] Dialyzer and ets:fun2ms Message-ID: <38176269-3A72-48FB-96F4-47F8AC19EB76@erlang.geek.nz> Hi all, I have hit this problem with my code and dialyzer a few times now and wonder what everyone else does to avoid it. I have record definitions with type specifications as follows: > -module(rec_example). > > -export([find_by_colour/1]). > > -include_lib("stdlib/include/ms_transform.hrl"). > > -record(myrecord, {field = 0 :: integer(), > roses = red :: 'red' | 'blue'}). > I then add some functions that use ets matchspecs or a fun2ms transform and have to fill in fields with dummy values as required by the API (e.g. '_' and '$1'): > find_by_colour(Colour) when is_atom(Colour) -> > ets:lookup(myrecords, ets:fun2ms(fun (#myrecord{roses=C}) > when C =:= Colour -> > object() > end)). Dialyzer complains that this function can't succeed: > rec_example.erl:10: Function find_by_colour/1 has no local return > rec_example.erl:11: Record construction #myrecord{field::'_',roses::'$1'} violates the declared type of field field::integer() and roses::'blue' | 'red' To me this seems like reasonable code to want to write, but dialyzer hates it. The two (terrible) solutions I've found to avoid the dialyzer warnings are to add the dummy values to the record type spec: > -record(myrecord, {field = 0 :: integer() | '_', > roses = red :: 'red' | 'blue' | '$1'}). Or to hide the record construction from dialyzer by constructing the matchspec using functions. First run fun2ms manually to get: > [{#myrecord{field = '_',roses = '$1'}, > [{'=:=','$1',Colour}], > ['$_']}] Which still complains, so you then do something far worse: > [{list_to_tuple([myrecord, '_', '$1']), > [{'=:=','$1',Colour}], > ['$_']}] Clearly none of these options are good - what's a better way to have record type specs and matchspecs and no dialyzer warnings? Or is there something I'm missing that replaces my use of matchspecs and avoids the problem? Cheers, -- Geoff Cant From jeffm@REDACTED Wed May 23 02:04:56 2012 From: jeffm@REDACTED (jm) Date: Wed, 23 May 2012 10:04:56 +1000 Subject: [erlang-questions] [ANN] Lux - LUcid eXpect scripting In-Reply-To: <8E746114-06AF-4040-8A1B-0994D7ADC7C1@gmail.com> References: <8E746114-06AF-4040-8A1B-0994D7ADC7C1@gmail.com> Message-ID: <4FBC29A8.4080005@ghostgun.com> Definitely. I was just looking for something like this the other day. Jeff. On 22/05/12 11:03 PM, Zheng Zhibin wrote: > Awesome! > > /Witeman > > ? 2012-5-22???8:37?H?kan Mattsson ??? > >> I just want to announce that Tail-f has released >> a very useful test tool as Open Source. From lenartlad@REDACTED Wed May 23 10:37:26 2012 From: lenartlad@REDACTED (Ladislav Lenart) Date: Wed, 23 May 2012 10:37:26 +0200 Subject: [erlang-questions] lists:concat with non-string sublists In-Reply-To: <4FBBDF04.4090007@2600hz.com> References: <4FBBCC84.5090901@2600hz.com> <4FBBD31B.8050204@cs.ntua.gr> <4FBBDF04.4090007@2600hz.com> Message-ID: <4FBCA1C6.6030805@volny.cz> Hello. On 22.5.2012 20:46, James Aimonetti wrote: >>> [snip] >>> So, according to the docs, I would expect: >>> >>>> lists:concat([[{foo, 123}], [{bar, 234}]]). >>> >>> to fail. It doesn't, and returns [{foo, 123}, {bar,234}]. Reading >>> the implementation of concat/1 its easy to see why this works. >>> >> [snip] >> >> Typical example of this is something like lists:append/2: >> >> -spec append(list(), list()) -> list(). % or something more >> involved >> >> but the implementation may allow for calls of the form: >> >> Eshell V5.9.2 (abort with ^G) 1> lists:append([], 3.14). 3.14 >> >> but you probably would not want to change its spec to allow term() >> for its second argument and return, would you? > > Indeed not, wrt to the lists:append/2 example. I guess the safer > course of action, in my case, is to implement my own concat/1 for my > list of lists of tuples, to ensure that lists:concat/1 doesn't change > down the road to enforce the spec more strictly? What about lists:append/1? That should be suitable for your use case, or am I missing something? Ladislav Lenart From peerst@REDACTED Wed May 23 11:51:57 2012 From: peerst@REDACTED (Peer Stritzinger) Date: Wed, 23 May 2012 11:51:57 +0200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> <1337534022.48017.YahooMailNeo@web130103.mail.mud.yahoo.com> Message-ID: The sparse documentation of Erlang internals can be seen as a opportunity also. I'm not sure if this needs to be done by the OTP team at Ericsson. The first opportunity is for a 1-3 day tutorial about Erlang internals during the Erlang conferences (probably as part of Erlang University). I would book a training like this as soon as it is offered alongside a conference I can attend (Are you listening Erlang Solutions ?) The second opportunity is for a book author. Perfect would be a book about Erlang internals like TCP/IP Illustrated Vol 2. The Implementation (http://www.amazon.com/TCP-IP-Illustrated-Vol-Implementation/dp/020163354X) where basically every line of code of the BSD TCP/IP implementation is explained. Probably a more realistic goal would be something along the lines of the Design and Implementation of the *BSD Operating System e.g.: (http://www.amazon.com/Implementation-Operating-paperback-Addison-Wesley-Systems/dp/0132317923, http://www.amazon.com/Design-Implementation-FreeBSD-Operating-System/dp/0201702452) I think the persons who are teaching the tutorial/writing the book need not necessarily be members of the OTP team but should rather be great teachers/writers. The would need good access to the people that have the internals knowledge however. With this we can have Erlang internals knowledge (I'm not only talking about beam) more spread and the OTP team can concentrate on making their great new releases. Cheers Peer Stritzinger > enough designers knowing this. > > I don't think a documentation of these things is so far away now (but no > promise). > > I also want to comment regarding stability and compatibility of the .beam > file format and thus instruction set. > We always keep backwards compatibility with 2 major releases. This means > that for example an R15B based system > can load and run .beam files produced with an R13B based system. In practice > this means 3-4 years of compatibility, > The other way around, i.e loading .beam files produced with R15B on an older > system is not supported. > > /Kenneth Erlang/OTP, Ericsson > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From essen@REDACTED Wed May 23 15:15:20 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 23 May 2012 15:15:20 +0200 Subject: [erlang-questions] [ANN] Cowboy 0.6.0 Message-ID: <4FBCE2E8.6010208@ninenines.eu> Hello, Long awaited Cowboy 0.6.0 is finally available. Cowboy is a small, fast, modular HTTP server and socket acceptor pool. The 0.6.0 version is the last to include the socket acceptor pool directly in the project. Future versions will simply depend on the ranch project which is the socket acceptor pool code split from Cowboy. Many additions have been made, along with a few bugfixes. The test suite has been completely reworked. There is still no user guides. Read more here: * https://github.com/extend/cowboy/blob/master/CHANGELOG.md Effort is ongoing to add user guides to both Ranch and Cowboy projects, starting with Ranch. Examples will be added to both projects this summer. * https://gist.github.com/2397647 * https://gist.github.com/2600398 Cowboy can be found at https://github.com/extend/cowboy Next version will switch the acceptor code to Ranch. The dependence on the erlang:decode_packet/3 BIF will be removed. Methods and header names will be uniformized as binaries, there won't be a mix of atoms and binary names anymore. Various changes will be made to REST to improve its semantics with regards to full and partial updates. A new API to generate dispatch tables easily will be added. This version includes an experimental and entirely undocumented HTTP client in cowboy_client. It uses binaries. We do not recommend using it at this point. We hope to add transfer encodings and websocket support in the next version and finally document it for 0.8.0. As always I'm interested in any feedback you may have, good or bad. Feel free to drop me a mail about the project! If you need features added, please open a ticket and describe *what you want to do with it*, chances are that you can already do it and you just don't know it. Thanks. -- Lo?c Hoguin Erlang Cowboy Nine Nines From gumm@REDACTED Wed May 23 16:40:51 2012 From: gumm@REDACTED (Jesse Gumm) Date: Wed, 23 May 2012 09:40:51 -0500 Subject: [erlang-questions] [ANN] Cowboy 0.6.0 In-Reply-To: <4FBCE2E8.6010208@ninenines.eu> References: <4FBCE2E8.6010208@ninenines.eu> Message-ID: Congrats Loic! -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm On May 23, 2012 8:15 AM, "Lo?c Hoguin" wrote: > Hello, > > Long awaited Cowboy 0.6.0 is finally available. > > Cowboy is a small, fast, modular HTTP server and socket acceptor pool. The > 0.6.0 version is the last to include the socket acceptor pool directly in > the project. Future versions will simply depend on the ranch project which > is the socket acceptor pool code split from Cowboy. > > Many additions have been made, along with a few bugfixes. The test suite > has been completely reworked. There is still no user guides. Read more here: > * https://github.com/extend/**cowboy/blob/master/CHANGELOG.**md > > Effort is ongoing to add user guides to both Ranch and Cowboy projects, > starting with Ranch. Examples will be added to both projects this summer. > * https://gist.github.com/**2397647 > * https://gist.github.com/**2600398 > > Cowboy can be found at https://github.com/extend/**cowboy > > Next version will switch the acceptor code to Ranch. The dependence on the > erlang:decode_packet/3 BIF will be removed. Methods and header names will > be uniformized as binaries, there won't be a mix of atoms and binary names > anymore. Various changes will be made to REST to improve its semantics with > regards to full and partial updates. A new API to generate dispatch tables > easily will be added. > > This version includes an experimental and entirely undocumented HTTP > client in cowboy_client. It uses binaries. We do not recommend using it at > this point. We hope to add transfer encodings and websocket support in the > next version and finally document it for 0.8.0. > > As always I'm interested in any feedback you may have, good or bad. Feel > free to drop me a mail about the project! > > If you need features added, please open a ticket and describe *what you > want to do with it*, chances are that you can already do it and you just > don't know it. > > Thanks. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andre@REDACTED Wed May 23 16:48:57 2012 From: andre@REDACTED (=?ISO-8859-1?Q?Andr=E9_Graf?=) Date: Wed, 23 May 2012 16:48:57 +0200 Subject: [erlang-questions] [ANN] Cowboy 0.6.0 In-Reply-To: <4FBCE2E8.6010208@ninenines.eu> References: <4FBCE2E8.6010208@ninenines.eu> Message-ID: Congratz! just upgraded from 0.5.*, worked like a charm! Cheers Andr? On 23 May 2012 15:15, Lo?c Hoguin wrote: > Hello, > > Long awaited Cowboy 0.6.0 is finally available. > > Cowboy is a small, fast, modular HTTP server and socket acceptor pool. The > 0.6.0 version is the last to include the socket acceptor pool directly in > the project. Future versions will simply depend on the ranch project which > is the socket acceptor pool code split from Cowboy. > > Many additions have been made, along with a few bugfixes. The test suite has > been completely reworked. There is still no user guides. Read more here: > ?* https://github.com/extend/cowboy/blob/master/CHANGELOG.md > > Effort is ongoing to add user guides to both Ranch and Cowboy projects, > starting with Ranch. Examples will be added to both projects this summer. > ?* https://gist.github.com/2397647 > ?* https://gist.github.com/2600398 > > Cowboy can be found at https://github.com/extend/cowboy > > Next version will switch the acceptor code to Ranch. The dependence on the > erlang:decode_packet/3 BIF will be removed. Methods and header names will be > uniformized as binaries, there won't be a mix of atoms and binary names > anymore. Various changes will be made to REST to improve its semantics with > regards to full and partial updates. A new API to generate dispatch tables > easily will be added. > > This version includes an experimental and entirely undocumented HTTP client > in cowboy_client. It uses binaries. We do not recommend using it at this > point. We hope to add transfer encodings and websocket support in the next > version and finally document it for 0.8.0. > > As always I'm interested in any feedback you may have, good or bad. Feel > free to drop me a mail about the project! > > If you need features added, please open a ticket and describe *what you want > to do with it*, chances are that you can already do it and you just don't > know it. > > Thanks. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From hukl@REDACTED Wed May 23 17:16:53 2012 From: hukl@REDACTED (John-Paul Bader) Date: Wed, 23 May 2012 17:16:53 +0200 Subject: [erlang-questions] [ANN] Cowboy 0.6.0 In-Reply-To: <4FBCE2E8.6010208@ninenines.eu> References: <4FBCE2E8.6010208@ninenines.eu> Message-ID: <4FBCFF65.3070400@berlin.ccc.de> Quick question: why do you move away from decode_packet and what will you use instead of it? I saw it is being used in a number of projects and I'm curious what your motivation / goal is. ~ John Lo?c Hoguin wrote: > Hello, > > Long awaited Cowboy 0.6.0 is finally available. > > Cowboy is a small, fast, modular HTTP server and socket acceptor pool. > The 0.6.0 version is the last to include the socket acceptor pool > directly in the project. Future versions will simply depend on the ranch > project which is the socket acceptor pool code split from Cowboy. > > Many additions have been made, along with a few bugfixes. The test suite > has been completely reworked. There is still no user guides. Read more > here: > * https://github.com/extend/cowboy/blob/master/CHANGELOG.md > > Effort is ongoing to add user guides to both Ranch and Cowboy projects, > starting with Ranch. Examples will be added to both projects this summer. > * https://gist.github.com/2397647 > * https://gist.github.com/2600398 > > Cowboy can be found at https://github.com/extend/cowboy > > Next version will switch the acceptor code to Ranch. The dependence on > the erlang:decode_packet/3 BIF will be removed. Methods and header names > will be uniformized as binaries, there won't be a mix of atoms and > binary names anymore. Various changes will be made to REST to improve > its semantics with regards to full and partial updates. A new API to > generate dispatch tables easily will be added. > > This version includes an experimental and entirely undocumented HTTP > client in cowboy_client. It uses binaries. We do not recommend using it > at this point. We hope to add transfer encodings and websocket support > in the next version and finally document it for 0.8.0. > > As always I'm interested in any feedback you may have, good or bad. Feel > free to drop me a mail about the project! > > If you need features added, please open a ticket and describe *what you > want to do with it*, chances are that you can already do it and you just > don't know it. > > Thanks. > From essen@REDACTED Wed May 23 18:54:12 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 23 May 2012 18:54:12 +0200 Subject: [erlang-questions] [ANN] Cowboy 0.6.0 In-Reply-To: <4FBCFF65.3070400@berlin.ccc.de> References: <4FBCE2E8.6010208@ninenines.eu> <4FBCFF65.3070400@berlin.ccc.de> Message-ID: <4FBD1634.40203@ninenines.eu> To get rid of the atom+binaries mixed together, a limitation on case sensitivity of header names for header names > 22 characters, get a more straightforward HTTP protocol code, maybe others I forgot. Most of the current inconsistencies in request data happens because of decode_packet. decode_packet can do a lot of things, and we use so few of them it's not giving us much for our money anymore. I'm also interested in seeing the performance differences of a full Erlang HTTP parsing stack vs one that uses decode_packet. This will be investigated this summer by my students. On 05/23/2012 05:16 PM, John-Paul Bader wrote: > Quick question: > > why do you move away from decode_packet and what will you use instead of > it? > > I saw it is being used in a number of projects and I'm curious what your > motivation / goal is. > > ~ John > > Lo?c Hoguin wrote: >> Hello, >> >> Long awaited Cowboy 0.6.0 is finally available. >> >> Cowboy is a small, fast, modular HTTP server and socket acceptor pool. >> The 0.6.0 version is the last to include the socket acceptor pool >> directly in the project. Future versions will simply depend on the ranch >> project which is the socket acceptor pool code split from Cowboy. >> >> Many additions have been made, along with a few bugfixes. The test suite >> has been completely reworked. There is still no user guides. Read more >> here: >> * https://github.com/extend/cowboy/blob/master/CHANGELOG.md >> >> Effort is ongoing to add user guides to both Ranch and Cowboy projects, >> starting with Ranch. Examples will be added to both projects this summer. >> * https://gist.github.com/2397647 >> * https://gist.github.com/2600398 >> >> Cowboy can be found at https://github.com/extend/cowboy >> >> Next version will switch the acceptor code to Ranch. The dependence on >> the erlang:decode_packet/3 BIF will be removed. Methods and header names >> will be uniformized as binaries, there won't be a mix of atoms and >> binary names anymore. Various changes will be made to REST to improve >> its semantics with regards to full and partial updates. A new API to >> generate dispatch tables easily will be added. >> >> This version includes an experimental and entirely undocumented HTTP >> client in cowboy_client. It uses binaries. We do not recommend using it >> at this point. We hope to add transfer encodings and websocket support >> in the next version and finally document it for 0.8.0. >> >> As always I'm interested in any feedback you may have, good or bad. Feel >> free to drop me a mail about the project! >> >> If you need features added, please open a ticket and describe *what you >> want to do with it*, chances are that you can already do it and you just >> don't know it. >> >> Thanks. >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Lo?c Hoguin Erlang Cowboy Nine Nines From james@REDACTED Wed May 23 19:20:51 2012 From: james@REDACTED (James Aimonetti) Date: Wed, 23 May 2012 10:20:51 -0700 Subject: [erlang-questions] lists:concat with non-string sublists In-Reply-To: <4FBCA1C6.6030805@volny.cz> References: <4FBBCC84.5090901@2600hz.com> <4FBBD31B.8050204@cs.ntua.gr> <4FBBDF04.4090007@2600hz.com> <4FBCA1C6.6030805@volny.cz> Message-ID: <4FBD1C73.8030607@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 05/23/2012 01:37 AM, Ladislav Lenart wrote: > Hello. > > On 22.5.2012 20:46, James Aimonetti wrote: >>>> [snip] So, according to the docs, I would expect: >>>> >>>>> lists:concat([[{foo, 123}], [{bar, 234}]]). >>>> >>>> to fail. It doesn't, and returns [{foo, 123}, {bar,234}]. >>>> Reading the implementation of concat/1 its easy to see why >>>> this works. >>>> >>> [snip] >>> >>> Typical example of this is something like lists:append/2: >>> >>> -spec append(list(), list()) -> list(). % or something more >>> involved >>> >>> but the implementation may allow for calls of the form: >>> >>> Eshell V5.9.2 (abort with ^G) 1> lists:append([], 3.14). 3.14 >>> >>> but you probably would not want to change its spec to allow >>> term() for its second argument and return, would you? >> >> Indeed not, wrt to the lists:append/2 example. I guess the safer >> course of action, in my case, is to implement my own concat/1 for >> my list of lists of tuples, to ensure that lists:concat/1 doesn't >> change down the road to enforce the spec more strictly? > > What about lists:append/1? That should be suitable for your use > case, or am I missing something? > > > Ladislav Lenart > > > lists:append/1 is certainly more suitable. I was running Dialyzer on code I hadn't written and was surprised that lists:concat/1 didn't enforce the spec more tightly and error on the input I gave it. - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJPvRxzAAoJENc77s1OYoGgu7sH/3gIT1/7ES8tXR8sb+lnhKfC rBhWyUHV2AnJlYpxZpI/IYMtFurvM9tb1lbVdmU4Ufyz2i0T3MttRWjkcnJoaqIe IYdbD3FJqpbkVgMzxCb/MLVerzkdR/9aMZtUNL9EKJrEebPb/pMg7HQ/EMScN0Ff 1UlyPK/D8eM45f3S8Ia6L9ZMHMSdDRLy5PcV7NB//He+b+uqxEfNM0u0BLcVP6IX HUGb87AhPOkAz9Vbe7fNME3FWdOGWeecx/Xae0LjgKfdVDutVVvUpo6BZDJLBuFS 4O7+RqKP6GSnf8e9UNtOBGNV03HpVCJgk3a4ZAQhl/OH1sa1eO5Z6Ent8sfAzyw= =pVeM -----END PGP SIGNATURE----- From zabrane3@REDACTED Wed May 23 20:50:02 2012 From: zabrane3@REDACTED (Zabrane Mikael) Date: Wed, 23 May 2012 20:50:02 +0200 Subject: [erlang-questions] [ANN] Cowboy 0.6.0 In-Reply-To: <4FBD1634.40203@ninenines.eu> References: <4FBCE2E8.6010208@ninenines.eu> <4FBCFF65.3070400@berlin.ccc.de> <4FBD1634.40203@ninenines.eu> Message-ID: Hi Lo?c, Congrat from the new Cowboy release. > I'm also interested in seeing the performance differences of a full Erlang > HTTP parsing stack vs one that uses decode_packet. This will be investigated > this summer by my students. What are you expecting from such a comparison? a. Speed? "decode_packet" is coded in C. b. ... -- Regards Zabrane From mattevans123@REDACTED Wed May 23 23:29:12 2012 From: mattevans123@REDACTED (Matthew Evans) Date: Wed, 23 May 2012 17:29:12 -0400 Subject: [erlang-questions] Distributed Erlang Message-ID: Hi, I am wanting to create a distributed mesh of Erlang nodes (all nodes will be configured with dist_auto_connect set to false). There could potentially be lots of nodes in the mesh, so to prevent the mesh from getting too big I want to have the ability to disconnect a node. Is this possible? Something like: net_kernel:disconnect_node('node@REDACTED'). Thanks Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From mattevans123@REDACTED Thu May 24 00:19:45 2012 From: mattevans123@REDACTED (Matthew Evans) Date: Wed, 23 May 2012 18:19:45 -0400 Subject: [erlang-questions] Distributed Erlang In-Reply-To: References: Message-ID: I guess I can use the undocumented function net_kernel:disconnect/1 From: mattevans123@REDACTED To: erlang-questions@REDACTED Date: Wed, 23 May 2012 17:29:12 -0400 Subject: [erlang-questions] Distributed Erlang Hi, I am wanting to create a distributed mesh of Erlang nodes (all nodes will be configured with dist_auto_connect set to false). There could potentially be lots of nodes in the mesh, so to prevent the mesh from getting too big I want to have the ability to disconnect a node. Is this possible? Something like: net_kernel:disconnect_node('node@REDACTED'). Thanks Matt _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Thu May 24 00:23:50 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 24 May 2012 00:23:50 +0200 Subject: [erlang-questions] [ANN] Cowboy 0.6.0 - Tutorial at EUC2012 In-Reply-To: <4FBCE2E8.6010208@ninenines.eu> References: <4FBCE2E8.6010208@ninenines.eu> Message-ID: <4FBD6376.6040901@ninenines.eu> Sorry to pollute a little but I forgot to mention it. There will be a 3 hours Cowboy tutorial at the Erlang User Conference in Stockholm next week. We will investigate how to build a web chat application step by step. Should be a good first experience if you are new to Cowboy. Hope to see you there! Thanks and sorry again for the noise. On 05/23/2012 03:15 PM, Lo?c Hoguin wrote: > Hello, > > Long awaited Cowboy 0.6.0 is finally available. > > Cowboy is a small, fast, modular HTTP server and socket acceptor pool. > The 0.6.0 version is the last to include the socket acceptor pool > directly in the project. Future versions will simply depend on the ranch > project which is the socket acceptor pool code split from Cowboy. > > Many additions have been made, along with a few bugfixes. The test suite > has been completely reworked. There is still no user guides. Read more > here: > * https://github.com/extend/cowboy/blob/master/CHANGELOG.md > > Effort is ongoing to add user guides to both Ranch and Cowboy projects, > starting with Ranch. Examples will be added to both projects this summer. > * https://gist.github.com/2397647 > * https://gist.github.com/2600398 > > Cowboy can be found at https://github.com/extend/cowboy > > Next version will switch the acceptor code to Ranch. The dependence on > the erlang:decode_packet/3 BIF will be removed. Methods and header names > will be uniformized as binaries, there won't be a mix of atoms and > binary names anymore. Various changes will be made to REST to improve > its semantics with regards to full and partial updates. A new API to > generate dispatch tables easily will be added. > > This version includes an experimental and entirely undocumented HTTP > client in cowboy_client. It uses binaries. We do not recommend using it > at this point. We hope to add transfer encodings and websocket support > in the next version and finally document it for 0.8.0. > > As always I'm interested in any feedback you may have, good or bad. Feel > free to drop me a mail about the project! > > If you need features added, please open a ticket and describe *what you > want to do with it*, chances are that you can already do it and you just > don't know it. > > Thanks. > -- Lo?c Hoguin Erlang Cowboy Nine Nines From daniel.goertzen@REDACTED Thu May 24 05:31:21 2012 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Wed, 23 May 2012 22:31:21 -0500 Subject: [erlang-questions] [ANN] yaml loader Message-ID: I'm pleased to announce an application for loading YAML files into Erlang. This implementation supports: - Detailed errors on yaml load failures (line, column, reason) - Anchors and aliases - Merge tags - The tag !atom for explicitely tagging values as atoms. - An implicit_atoms mode to interpret values that look atom-ish as atoms. - Customizable schemas via callback modules. - Loading only, but perhaps it will also emit YAML in the future. This application embeds the C yaml parser "libyaml" which is compiled as a NIF. Details at https://github.com/goertzenator/yaml For those unfamiliar with YAML, it is similar to JSON with a strong focus on human readability and writeability. If you need to type in lots of structured data, YAML is a decent choice. Dan. From paperless@REDACTED Thu May 24 05:38:11 2012 From: paperless@REDACTED (Tim McNamara) Date: Thu, 24 May 2012 15:38:11 +1200 Subject: [erlang-questions] [ANN] yaml loader In-Reply-To: References: Message-ID: This is really great. Having an Erlang implementation is excellent. Strongly suggest changing the project name from "yaml" though. YAML does have a few (IMO) advantages over JSON apart from readability: - comment syntax - named entities - supports streaming more easily* * Although it looks like yaml:load_file/1 loads everything into memory. On 24 May 2012 15:31, Daniel Goertzen wrote: > I'm pleased to announce an application for loading YAML files into > Erlang. This implementation supports: > > - Detailed errors on yaml load failures (line, column, reason) > - Anchors and aliases > - Merge tags > - The tag !atom for explicitely tagging values as atoms. > - An implicit_atoms mode to interpret values that look atom-ish as atoms. > - Customizable schemas via callback modules. > - Loading only, but perhaps it will also emit YAML in the future. > > This application embeds the C yaml parser "libyaml" which is compiled as a NIF. > > Details at https://github.com/goertzenator/yaml > > For those unfamiliar with YAML, it is similar to JSON with a strong > focus on human readability and writeability. ?If you need to type in > lots of structured data, YAML is a decent choice. > > > Dan. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From carlsson.richard@REDACTED Thu May 24 05:54:29 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 24 May 2012 05:54:29 +0200 Subject: [erlang-questions] Distributed Erlang In-Reply-To: References: Message-ID: <4FBDB0F5.4020007@gmail.com> Looking at the source code, disconnect/1 is simply implemented as the dual to connect_node/1, whereas connect/1 (also undocumented) is doing some extra stuff on top of what connect_node/1 does. I suggest submitting a patch that implements net_kernel:disconnect_node/1 as doing exactly what disconnect/1 does today, and add it to the documentation. This allows disconnect/1 to stay undocumented so it can be tweaked for whatever internal purposes OTP are using it for in the future without affecting user code. /Richard On 05/24/2012 12:19 AM, Matthew Evans wrote: > I guess I can use the undocumented function net_kernel:disconnect/1 > > > > ------------------------------------------------------------------------ > From: mattevans123@REDACTED > To: erlang-questions@REDACTED > Date: Wed, 23 May 2012 17:29:12 -0400 > Subject: [erlang-questions] Distributed Erlang > > Hi, > > I am wanting to create a distributed mesh of Erlang nodes (all nodes > will be configured with dist_auto_connect set to false). There could > potentially be lots of nodes in the mesh, so to prevent the mesh from > getting too big I want to have the ability to disconnect a node. Is this > possible? > > Something like: net_kernel:disconnect_node('node@REDACTED'). > > Thanks > > Matt > > _______________________________________________ erlang-questions mailing > list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From boris.muehmer@REDACTED Thu May 24 06:20:06 2012 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Thu, 24 May 2012 06:20:06 +0200 Subject: [erlang-questions] [ANN] Cowboy 0.6.0 - Tutorial at EUC2012 In-Reply-To: <4FBD6376.6040901@ninenines.eu> References: <4FBCE2E8.6010208@ninenines.eu> <4FBD6376.6040901@ninenines.eu> Message-ID: 2012/5/24 Lo?c Hoguin : > There will be a 3 hours Cowboy tutorial at the Erlang User Conference in > Stockholm next week. We will investigate how to build a web chat application > step by step. Should be a good first experience if you are new to Cowboy. > Hope to see you there! Will there be a recording of this (and maybe other) talk(s)? It would be great for those who can't attend the Erlang User Conference to watch it (and maybe others). - boris From francesco@REDACTED Thu May 24 07:12:00 2012 From: francesco@REDACTED (Francesco Cesarini) Date: Thu, 24 May 2012 06:12:00 +0100 Subject: [erlang-questions] [ANN] Cowboy 0.6.0 - Tutorial at EUC2012 In-Reply-To: References: <4FBCE2E8.6010208@ninenines.eu> <4FBD6376.6040901@ninenines.eu> Message-ID: <73dddc24-c395-4ff8-9920-210d70ab4d26@email.android.com> We are recording the talks, but not the tutorials. Regards, Francesco "Boris M?hmer" wrote: 2012/5/24 Lo?c Hoguin : > There will be a 3 hours Cowboy tutorial at the Erlang User Conference in > Stockholm next week. We will investigate how to build a web chat application > step by step. Should be a good first experience if you are new to Cowboy. > Hope to see you there! Will there be a recording of this (and maybe other) talk(s)? It would be great for those who can't attend the Erlang User Conference to watch it (and maybe others). - boris _____________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Thu May 24 10:43:23 2012 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 24 May 2012 10:43:23 +0200 Subject: [erlang-questions] Weird Client SSL Behavior / Performance In-Reply-To: <4FB8FF67.6070108@berlin.ccc.de> References: <4FB8FF67.6070108@berlin.ccc.de> Message-ID: Hi! I feel I would like to make some more comments on this, see below. 2012/5/20, John-Paul Bader : > First of all I'd really like to know why the problem occurs in the first > place and why it creates so much load in general. Stunnel deals with the > same load with 1/3 or even less than that of what the erlang vm needs. We are also interested in that. > For now at least I'm a little bit underwhelmed of erlangs ssl stability > and performance (I was bitten by the R14B03 bug as well). The so called R14B03 bug was of course unfortunate, but it was not a take down the whole system bug. It did not effect existing connections. It would of course have a memory growing effect for long lived systems which could be needed to be worked around. Preferably patched with the R14B04 ssl-version. The bug made noise in the logs, so the bug was discovered early, that was good so it could be fixed quickly. We currently have 213 automated test cases for ssl and now we also have a test case to make sure the R14B03 bug will not happen again. It will happen that bugs slip through and there is no such thing as the perfect test suite. As an example when openssl released there 1.0 release I remember that a problem occurred that we could not establish connections to openssl because of a bug in their code, so maybe we should be underwhelmed with openssl too... We hade a legacy implementation of ssl that did not work well enough. We decided to make a new one. Our first focus has been to make it work and be complete enough to replace the old one. In R15 it has replaced the old one. Now when it is that mature we can take it to the next level. And we are constantly working to improve and provide more functionality. An all input is valuable. Regards Ingela Erlang/OTP team - Ericsson AB From vladdu55@REDACTED Thu May 24 10:52:46 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 24 May 2012 10:52:46 +0200 Subject: [erlang-questions] Getting rid of the preprocessor Message-ID: Hi, There have been discussions now and then about the preprocessor and how it would be better to get rid of it. As someone that has to parse and handle source files before they are preprocessed, I am very much affected by it -- with macros that can be anything, an raw Erlang file can look like pretty much anything. Even if the macros are in practice mostly well-behaved, using them as alternatives to inlining code creates problem when debugging, for example, because you can't step through a macro invocation. One of the main reasons that was presented for not being able to replace the preprocessor with something that works at the lexical level is that records would need to be first made first-class citizens (as frames or whatever). I don't understand why that is -- the preprocessor doesn't do anything with records, their expansion is done later, by the compiler. Could someone comment on that, please? Is this issue a "definitely no-no" for OTP, or would it be considered if we can come up with a solution that doesn't break anything? best regards, Vlad From overminddl1@REDACTED Thu May 24 11:42:18 2012 From: overminddl1@REDACTED (OvermindDL1) Date: Thu, 24 May 2012 03:42:18 -0600 Subject: [erlang-questions] secure use of cookies in an erlang application In-Reply-To: <76a257ea-ead2-4550-9d1d-c8f6baa6183a@l5g2000vbo.googlegroups.com> References: <6DC11414-81D5-45B6-B55E-99E0D4E35AAF@gmail.com> <76a257ea-ead2-4550-9d1d-c8f6baa6183a@l5g2000vbo.googlegroups.com> Message-ID: On Fri, May 18, 2012 at 8:01 AM, eigenfunction wrote: > Stateless applications are just too hard to secure. I wished there was > a stateful webframework written in erlang, something like java- > webobjects or scala-lift where you can just write your application and > go to sleep without worrying about security. The first time i saw > erlang-web and its component based approach, i thought they got it. I > checked their wiki page but they did not mention security anywhere so > i had to look somewhere else. > > On May 17, 2:43 am, Steve Davis > wrote: > > This is also quite interesting on the topic. > > > > http://www.cse.msu.edu/~alexliu/publications/Cookie/cookie.pdf > For note, but in the past I had loved using Wt C++ Web Toolkit, completely stateful, high emphasis on security, fully javascript driven but with a full functional fallback when javascript is disabled. I have always wanted something like it in Erlang. Nitrogen gets kind of close, but still missing the vast part of it, like all-inclusive state. -------------- next part -------------- An HTML attachment was scrubbed... URL: From S.J.Thompson@REDACTED Thu May 24 13:10:26 2012 From: S.J.Thompson@REDACTED (Simon Thompson) Date: Thu, 24 May 2012 12:10:26 +0100 Subject: [erlang-questions] IFL 2012 - Call for Papers Message-ID: ------------------------------------------------------------------------------- CALL FOR PAPERS 24th Symposium on Implementation and Application of Functional Languages (IFL 2012) University of Oxford, UK, August 30-September 1, 2012 http://www.cs.ox.ac.uk/conferences/IFL2012/ This year IFL will be hosted by the University of Oxford, within the idyllic setting of the dreaming spires and picturesque colleges, which have been the home to academic endeavour and research for over nine centuries. The symposium will be held between 30 August and 1 September, 2012. Scope ----- The goal of the IFL symposia is to bring together researchers actively engaged in the implementation and application of functional and function-based programming languages. IFL 2012 will be a venue for researchers to present and discuss new ideas and concepts, work in progress, and publication-ripe results related to the implementation and application of functional languages and function-based programming. Following the IFL tradition, IFL 2012 will use a post-symposium review process to produce formal proceedings which will be published by Springer Verlag in the Lecture Notes in Computer Science series. All participants in IFL 2012 are invited to submit either a draft paper or an extended abstract describing work to be presented at the symposium. Work submitted to IFL must not be submitted simultaneously to other venues; submissions must adhere to ACM SIGPLAN's republication policy: http://www.sigplan.org/republicationpolicy.htm. The submissions will be screened by the program committee chair to make sure they are within the scope of IFL, and will appear in the draft proceedings distributed at the symposium. Submissions appearing in the draft proceedings are not peer-reviewed publications. After the symposium, authors will be given the opportunity to incorporate the feedback from discussions at the symposium and will be invited to submit a revised full article for the formal review process. From the revised submissions, the program committee will select papers for the formal proceedings considering their correctness, novelty, originality, relevance, significance, and clarity. Topics ------ IFL welcomes submissions describing practical and theoretical work as well as submissions describing applications and tools. If you are not sure that your work is appropriate for IFL 2012, please contact the PC chair at ralf.hinze@REDACTED Topics of interest include, but are not limited to: language concepts type checking contracts compilation techniques staged compilation run-time function specialization run-time code generation partial evaluation (abstract) interpretation generic programming automatic program generation array processing concurrent/parallel programming concurrent/parallel program execution functional programming and embedded systems functional programming and web applications functional programming and security novel memory management techniques run-time profiling and performance measurements debugging and tracing virtual/abstract machine architectures validation and verification of functional programs tools and programming techniques (industrial) applications of functional programming Submission details ------------------ Presentation submission deadline: July 30th, 2012 Notification of acceptance: August 1st, 2012 Early registration deadline: August 10th, 2012 IFL 2012 Symposium: August 30-September 1, 2012 Submission for (post) review process: November 30th, 2012 Notification Accept/Reject: February 4th, 2013 Camera ready version: March 18th, 2013 Prospective authors are encouraged to submit papers or extended abstracts to be published in the draft proceedings and to present them at the symposium. All contributions must be written in English, conform to the Springer-Verlag LNCS series format and not exceed 16 pages. The draft proceedings will appear as a technical report of the Department of Computer Science of the University of Oxford. (We are more liberal with the draft proceedings, where longer papers or SIGPLAN 2 column 12 page papers are acceptable. For other formats please contact the chair at ralf.hinze@REDACTED For consideration for the final proceedings, only the 16 page LNCS format will be accepted.) Papers are to be submitted via the conference's EasyChair submission page: https://www.easychair.org/conferences/?conf=ifl2012 Peter Landin Prize ------------------ The Peter Landin Prize is awarded to the best paper presented at the symposium every year. The honoured article is selected by the program committee based on the submissions received for the formal review process. The prize carries a cash award equivalent to 150 Euros. Programme committee ------------------- Edwin Brady, University of St. Andrews, UK Andrew Butterfield, University of Dublin, Ireland Matthew Flatt, University of Utah, US Andy Gill, University of Kansas, US Stephan Herhut, Intel Labs, Santa Clara, US Ralf Hinze (Chair), University of Oxford, UK Zhenjiang Hu, National Institute of Informatics, Japan Patrik Jansson, Chalmers University of Technology, Sweden Mauro Jaskelioff, Universidad Nacional de Rosario, Argentina Gabriele Keller, University of New South Wales, Australia Simon Marlow, Microsoft Research, UK Pablo Nogueira, Technical University of Madrid, Spain Bruno Oliveira, Seoul National University, Korea Jos? Nuno Oliveira, University of Minho, Portugal Rinus Plasmeijer, Radboud University Nijmegen, Netherlands Tom Schrijvers, Ghent University, Belgium Tim Sheard, Portland State University, US Wouter Swierstra, University of Utrecht, Netherlands Peter Thiemann, University of Freiburg, Germany Simon Thompson, University of Kent, UK Steve Zdancewic, University of Pennsylvania, US Simon Thompson | Professor of Logic and Computation School of Computing | University of Kent | Canterbury, CT2 7NF, UK s.j.thompson@REDACTED | M +44 7986 085754 | W www.cs.kent.ac.uk/~sjt From bourinov@REDACTED Thu May 24 13:44:28 2012 From: bourinov@REDACTED (Max Bourinov) Date: Thu, 24 May 2012 15:44:28 +0400 Subject: [erlang-questions] Getting rid of the preprocessor In-Reply-To: References: Message-ID: Hi Vlad, I have relatively big OTP system. I use macros. I don't have any problems you described. What I do wrong? I don't debug my code because I think debugging is bad. I do unit tests and ct tests. If after those tests there is a need for debug - my tests are bad. I do better tests. Will this approach work for you too? Best regards, Max On Thu, May 24, 2012 at 12:52 PM, Vlad Dumitrescu wrote: > Hi, > > There have been discussions now and then about the preprocessor and > how it would be better to get rid of it. As someone that has to parse > and handle source files before they are preprocessed, I am very much > affected by it -- with macros that can be anything, an raw Erlang file > can look like pretty much anything. Even if the macros are in practice > mostly well-behaved, using them as alternatives to inlining code > creates problem when debugging, for example, because you can't step > through a macro invocation. > > One of the main reasons that was presented for not being able to > replace the preprocessor with something that works at the lexical > level is that records would need to be first made first-class citizens > (as frames or whatever). I don't understand why that is -- the > preprocessor doesn't do anything with records, their expansion is done > later, by the compiler. Could someone comment on that, please? > > Is this issue a "definitely no-no" for OTP, or would it be considered > if we can come up with a solution that doesn't break anything? > > > > best regards, > Vlad > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Thu May 24 14:02:32 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 24 May 2012 14:02:32 +0200 Subject: [erlang-questions] Getting rid of the preprocessor In-Reply-To: References: Message-ID: Hi Max, On Thu, May 24, 2012 at 1:44 PM, Max Bourinov wrote: > I have relatively big OTP system. I use macros. I don't have any problems > you described. What I do wrong? If you have no problems, then you do nothing wrong! :-) > I don't debug my code because I think debugging is bad. I do unit tests and > ct tests. If after those tests there is a need for debug - my tests are bad. > I do better tests. Will this approach work for you too? Yes, I know about that. But there is a debugger and if someone wants to use it, it should be as useful as possible. I have problems with macros because I am working on erlide [*] which like any IDE that uses the source files and needs also to be as useful as possible by providing editing help. Source files with non-restricted macros can't be parsed with normal parsers. Even putting restrictions on the macro values so that they are well-behaved requires a context-dependent parsing. As one example, how would you parse the following line? ?HELLO(world) The answer is that it depends. With -define(HELLO, hello), it is a function call to hello(world). With -define(HELLO(Arg), whatever), it is whatever the macro value is. This could be disambiguated by requiring all macro references to be followed by parentheses, like function calls, but that would be massively not backwards-compatible. [*] Eclipse based IDE, http://erlide.org regards, Vlad From bourinov@REDACTED Thu May 24 14:38:35 2012 From: bourinov@REDACTED (Max Bourinov) Date: Thu, 24 May 2012 16:38:35 +0400 Subject: [erlang-questions] Getting rid of the preprocessor In-Reply-To: References: Message-ID: Hi Vlad, You are right, it is nearly impossible to properly work with macros in IDE. I never tried erlide. Does it give any coding performance boost? I use sublime2 and emacs. Most of Erlangers uses Emacs (as far as I know). About your example: You can agree within your team which macros you will use and which not. For example: 1. No complicated code snippets in macros - this is good for code simplicity. 2. All atoms that are flying between modules must be in macros. Or even better - use records for that. Simple rules - simple code. Macros are good. They are cool. Use right tool and you have no problems. Best regards, Max On Thu, May 24, 2012 at 4:02 PM, Vlad Dumitrescu wrote: > Hi Max, > > On Thu, May 24, 2012 at 1:44 PM, Max Bourinov wrote: > > I have relatively big OTP system. I use macros. I don't have any problems > > you described. What I do wrong? > > If you have no problems, then you do nothing wrong! :-) > > > I don't debug my code because I think debugging is bad. I do unit tests > and > > ct tests. If after those tests there is a need for debug - my tests are > bad. > > I do better tests. Will this approach work for you too? > > Yes, I know about that. But there is a debugger and if someone wants > to use it, it should be as useful as possible. > > I have problems with macros because I am working on erlide [*] which > like any IDE that uses the source files and needs also to be as useful > as possible by providing editing help. Source files with > non-restricted macros can't be parsed with normal parsers. Even > putting restrictions on the macro values so that they are well-behaved > requires a context-dependent parsing. > > As one example, how would you parse the following line? > ?HELLO(world) > > The answer is that it depends. > With -define(HELLO, hello), it is a function call to hello(world). > With -define(HELLO(Arg), whatever), it is whatever the macro value is. > > This could be disambiguated by requiring all macro references to be > followed by parentheses, like function calls, but that would be > massively not backwards-compatible. > > > > [*] Eclipse based IDE, http://erlide.org > > regards, > Vlad > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Thu May 24 14:54:38 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 24 May 2012 14:54:38 +0200 Subject: [erlang-questions] Getting rid of the preprocessor In-Reply-To: References: Message-ID: On Thu, May 24, 2012 at 2:38 PM, Max Bourinov wrote: > You are right, it is nearly impossible to properly work with macros in > IDE.?I never tried erlide. Does it give any coding performance boost??I use > sublime2 and emacs. Most of Erlangers uses Emacs (as far as I know). Well, there's a whole bunch of them at Ericsson use erlide, plus others that I only have fragmentary information. Of course, I believe that an IDE helps the development process, but I know that it is a controversial question. Some people like it, some don't. > About your example: You can agree within your team which macros you will use > and which not. For example: > 1. No complicated code snippets in macros - this is good for code > simplicity. > 2. All atoms that are flying between modules must be in macros. Or even > better - use records for that. > Simple rules - simple code. Macros are good. They are cool. Use right tool > and you have no problems. Yes, that would be good enough for me -- but as a generic tool provider, I can't force people to follow any rules. There is also the issue of legacy code, that nobody will touch as long as it works. Anyway, the simple usage you describe above doesn't require a preprocessor, these could be handled inside the language with some additions to the compiler. This would enforce the cleanliness and I don't think anybody would feel sorry about that. For the dirty work, the preprocessor could still be there, if needed. regards, Vlad From bourinov@REDACTED Thu May 24 15:11:32 2012 From: bourinov@REDACTED (Max Bourinov) Date: Thu, 24 May 2012 17:11:32 +0400 Subject: [erlang-questions] Getting rid of the preprocessor In-Reply-To: References: Message-ID: Ah I see :-) The "bad" key-word here is "generic tool provider". Maybe good README.markdown file will help somehow? It always helps when code cannot solve everything. Best regards, Max On Thu, May 24, 2012 at 4:54 PM, Vlad Dumitrescu wrote: > On Thu, May 24, 2012 at 2:38 PM, Max Bourinov wrote: > > You are right, it is nearly impossible to properly work with macros in > > IDE. I never tried erlide. Does it give any coding performance boost? I > use > > sublime2 and emacs. Most of Erlangers uses Emacs (as far as I know). > > Well, there's a whole bunch of them at Ericsson use erlide, plus > others that I only have fragmentary information. Of course, I believe > that an IDE helps the development process, but I know that it is a > controversial question. Some people like it, some don't. > > > About your example: You can agree within your team which macros you will > use > > and which not. For example: > > 1. No complicated code snippets in macros - this is good for code > > simplicity. > > 2. All atoms that are flying between modules must be in macros. Or even > > better - use records for that. > > Simple rules - simple code. Macros are good. They are cool. Use right > tool > > and you have no problems. > > Yes, that would be good enough for me -- but as a generic tool > provider, I can't force people to follow any rules. There is also the > issue of legacy code, that nobody will touch as long as it works. > > Anyway, the simple usage you describe above doesn't require a > preprocessor, these could be handled inside the language with some > additions to the compiler. This would enforce the cleanliness and I > don't think anybody would feel sorry about that. For the dirty work, > the preprocessor could still be there, if needed. > > regards, > Vlad > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dang@REDACTED Thu May 24 15:42:58 2012 From: dang@REDACTED (Daniel Goertzen) Date: Thu, 24 May 2012 08:42:58 -0500 Subject: [erlang-questions] [ANN] yaml loader In-Reply-To: References: Message-ID: You're right, that's a terrible name for this project. I wanted the main module to be "yaml" but that doesn't mean the application or project has to be called that. This project is now at: https://github.com/goertzenator/goyaml Apologies for those who already watched or forked. Regarding streaming, libyaml forced me to go with an all-at-once approach. Erlang native processes would help overcome this, as would a parser written in Erlang. Dan. On Wed, May 23, 2012 at 10:38 PM, Tim McNamara wrote: > This is really great. Having an Erlang implementation is excellent. > Strongly suggest changing the project name from "yaml" though. > > YAML does have a few (IMO) advantages over JSON apart from readability: > > ?- comment syntax > ?- named entities > ?- supports streaming more easily* > > > * Although it looks like yaml:load_file/1 loads everything into memory. > > On 24 May 2012 15:31, Daniel Goertzen wrote: >> I'm pleased to announce an application for loading YAML files into >> Erlang. This implementation supports: >> >> - Detailed errors on yaml load failures (line, column, reason) >> - Anchors and aliases >> - Merge tags >> - The tag !atom for explicitely tagging values as atoms. >> - An implicit_atoms mode to interpret values that look atom-ish as atoms. >> - Customizable schemas via callback modules. >> - Loading only, but perhaps it will also emit YAML in the future. >> >> This application embeds the C yaml parser "libyaml" which is compiled as a NIF. >> >> Details at https://github.com/goertzenator/yaml >> >> For those unfamiliar with YAML, it is similar to JSON with a strong >> focus on human readability and writeability. ?If you need to type in >> lots of structured data, YAML is a decent choice. >> >> >> Dan. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Daniel Goertzen?|?Senior Software Engineer Office:?828.610.4596?|?Fax:?828.322.5294?|?dang@REDACTED Network Integrity Systems?|?We Bring Security To Light? 1937 Tate Blvd. SE Hickory, North Carolina, USA 28602 Network Integrity Systems? INTERCEPTOR? Optical Network Security System is a Smart-PDS? that ensures superior protection and cost effectiveness of classified networks. ?For more information, visit our website at:www.networkintegritysystems.com. __________________________________________ INTERCEPTOR? Optical Network Security System is made in the USA for the USA. Although not an export controlled item, because of the role it plays in the assurance of the safety and integrity of National Security Information, Network Integrity Systems (NIS) is committed to compliance with the U.S. Export Administration Act. Accordingly, NIS will not ship INTERCEPTOR products to certain foreign government end users without U.S. government approval and will refuse transactions with individuals or entities that have been denied export privileges. From gleber.p@REDACTED Thu May 24 15:46:27 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Thu, 24 May 2012 15:46:27 +0200 Subject: [erlang-questions] [ANN] yaml loader In-Reply-To: References: Message-ID: On Thu, May 24, 2012 at 3:42 PM, Daniel Goertzen wrote: > You're right, that's a terrible name for this project. ?This project is now at: > > https://github.com/goertzenator/goyaml I'm afraid people using Go programming language will be stumbling upon your project now :) From dmercer@REDACTED Thu May 24 18:23:49 2012 From: dmercer@REDACTED (David Mercer) Date: Thu, 24 May 2012 11:23:49 -0500 Subject: [erlang-questions] What causes nodes to become disconnected/reconnected? Message-ID: <020101cd39c9$9b3f0260$d1bd0720$@com> (Yes, I am still working on my issue with a distributed application that keeps losing its connection to the other node.) It is not a problem only when I have 2 nodes on the same host. I was running the nodes on different hosts last night, and this morning the failover node had lost its connection to the main and so had started its own instance of the application. Calling nodes() on the failover returned []. Then I started a new node on the same host as the main (to see if it would restore the connections), and, yes, it did. After starting the third node, nodes() now on the failover node returns a list of two nodes, the two on the main host. However, the application on the failover node did not shut down, and so it is still running on both the main and failover nodes. To summarize: 1. Distributed application running on a node on host A ("main@REDACTED"), failover on node on host B ("failover@REDACTED"). 2. At some point, failover@REDACTED becomes disconnected from main@REDACTED, and the application starts on failover@REDACTED Now there are two instances of the application running. 3. From a network point of view, there is still (or again) a valid network connection between hosts A and B. I can't say for sure if some network/firewall/other issue caused a temporary disconnect, but I can say that by the time I got in this morning, when the application was running on both nodes, there was a firm network connection between the two hosts. 4. Calling nodes() on failover@REDACTED returns []. 5. A new (failover) node was started on host A ("failover@REDACTED"). It does not start the application (which is correct, since it is already running, albeit on both other nodes instead of just one). 6. Calling nodes() on failover@REDACTED now returns [failover@REDACTED,main@REDACTED]. 7. The application, however, is still running on failover@REDACTED, despite the fact that nodes/0 reports a connection to main@REDACTED I don't need someone to diagnose this for me. If someone could just educate me a little on how the connections work, how net splits are detected and nodes disconnected, etc., I might be able to take it from there. Anyone know enough and have the time to type out a little blurb? I can read source code, but having a little background knowledge would help put it into context for me. Thank-you! Cheers, David -------------- next part -------------- An HTML attachment was scrubbed... URL: From alisdairsullivan@REDACTED Thu May 24 22:42:11 2012 From: alisdairsullivan@REDACTED (alisdair sullivan) Date: Thu, 24 May 2012 13:42:11 -0700 Subject: [erlang-questions] Getting rid of the preprocessor In-Reply-To: References: Message-ID: <97A7FD042FDD4A95AD3439363DC4F826@yahoo.ca> you could compile using 'P' or 'E' to get a source listing after preprocessing, compile that and debug from that source. not optimal, but better than not having source at all -- alisdair sullivan Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Thursday, 24 May, 2012 at 6:11 AM, Max Bourinov wrote: > Ah I see :-) > > The "bad" key-word here is "generic tool provider". > > Maybe good README.markdown file will help somehow? It always helps when code cannot solve everything. > > Best regards, > Max > > > > > On Thu, May 24, 2012 at 4:54 PM, Vlad Dumitrescu wrote: > > On Thu, May 24, 2012 at 2:38 PM, Max Bourinov wrote: > > > You are right, it is nearly impossible to properly work with macros in > > > IDE. I never tried erlide. Does it give any coding performance boost? I use > > > sublime2 and emacs. Most of Erlangers uses Emacs (as far as I know). > > > > Well, there's a whole bunch of them at Ericsson use erlide, plus > > others that I only have fragmentary information. Of course, I believe > > that an IDE helps the development process, but I know that it is a > > controversial question. Some people like it, some don't. > > > > > About your example: You can agree within your team which macros you will use > > > and which not. For example: > > > 1. No complicated code snippets in macros - this is good for code > > > simplicity. > > > 2. All atoms that are flying between modules must be in macros. Or even > > > better - use records for that. > > > Simple rules - simple code. Macros are good. They are cool. Use right tool > > > and you have no problems. > > > > Yes, that would be good enough for me -- but as a generic tool > > provider, I can't force people to follow any rules. There is also the > > issue of legacy code, that nobody will touch as long as it works. > > > > Anyway, the simple usage you describe above doesn't require a > > preprocessor, these could be handled inside the language with some > > additions to the compiler. This would enforce the cleanliness and I > > don't think anybody would feel sorry about that. For the dirty work, > > the preprocessor could still be there, if needed. > > > > regards, > > Vlad > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Thu May 24 22:43:20 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Thu, 24 May 2012 13:43:20 -0700 Subject: [erlang-questions] Record validation Message-ID: I am looking at building something resembling Java's JSR-303 Bean Validation. I'm currently building annotations using https://github.com/hyperthunk/annotations which can go on functions only. I was wondering if there is an event or some way of validating on the actual record itself. Is there some event published when something is changed or set on a record? Just curious if there is some kind of pre or post-hook I can use on records themselves which would let me validate the entire record any time it is changed or created. Any thoughts on if this is possible in any way? Thanks, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Thu May 24 23:09:18 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 24 May 2012 23:09:18 +0200 Subject: [erlang-questions] Getting rid of the preprocessor In-Reply-To: <97A7FD042FDD4A95AD3439363DC4F826@yahoo.ca> References: <97A7FD042FDD4A95AD3439363DC4F826@yahoo.ca> Message-ID: Hi, On Thu, May 24, 2012 at 10:42 PM, alisdair sullivan wrote: > you could compile using 'P' or 'E' to get a source listing after > preprocessing, compile that and debug from that source. not optimal, but > better than not having source at all Yes, that is an option that helps when debugging. It's not useful for keeping track of all the places a function is called from, for example, so that one can rename it. It doesn't make parsing the original source easier, either. best regards, Vlad From ok@REDACTED Fri May 25 00:16:33 2012 From: ok@REDACTED (Richard O'Keefe) Date: Fri, 25 May 2012 10:16:33 +1200 Subject: [erlang-questions] Getting rid of the preprocessor In-Reply-To: References: Message-ID: <390E1F41-608F-4FE3-B4B4-3B7685BD0D58@cs.otago.ac.nz> On 25/05/2012, at 12:02 AM, Vlad Dumitrescu wrote: > As one example, how would you parse the following line? > ?HELLO(world) I am no friend of the preprocessor. But this at least is no new problem: C has exactly the same problem, and presumably the C and C++ support in Eclipse already does something with it. (If the editor manual is bigger than the listing of my editor, I don't use it. Eclipse massively fails this test, so I don't know much about it.) I presume it goes something like this: if it looks like a variable, display it as one; if it looks like a function call, display it as one; if the display is wrong, it's the programmer's fault for defining such a stupid macro. There's one possible extra you _might_ want to consider. I have C macros that would benefit from it. That is %%erlide%% begin_like %%erlide%% end_like so that for example you could use %%erlide%% begin_like SPAWN %%erlide%% end_like END -define(SPAWN, spawn(fun () ->). -define(END, end)). ... Pid = SPAWN Ping!pong, Pong!ping END, ... and have it coloured/styled/indented appropriately. From corticalcomputer@REDACTED Fri May 25 02:19:23 2012 From: corticalcomputer@REDACTED (G.S.) Date: Thu, 24 May 2012 20:19:23 -0400 Subject: [erlang-questions] ODE solver. Message-ID: Hello, Is there an ordinery differential equation solver in Erlang? Has anyone tried writing one? or is familiar with the performance of one? -Gene -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.eugene.turner@REDACTED Fri May 25 07:26:56 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Fri, 25 May 2012 14:26:56 +0900 Subject: [erlang-questions] [ANN] yaml loader In-Reply-To: References: Message-ID: Then ... "yamerl"? -michael turner On Thu, May 24, 2012 at 10:46 PM, Gleb Peregud wrote: > On Thu, May 24, 2012 at 3:42 PM, Daniel Goertzen > wrote: >> You're right, that's a terrible name for this project. ?This project is now at: >> >> https://github.com/goertzenator/goyaml > > I'm afraid people using Go programming language will be stumbling upon > your project now :) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bourinov@REDACTED Fri May 25 08:41:47 2012 From: bourinov@REDACTED (Max Bourinov) Date: Fri, 25 May 2012 10:41:47 +0400 Subject: [erlang-questions] [ANN] yaml loader In-Reply-To: References: Message-ID: <-5512473052633725442@unknownmsgid> Eyaml or erlyaml? Sent from my iPhone On 25.05.2012, at 9:27, Michael Turner wrote: > Then ... "yamerl"? > > -michael turner > > On Thu, May 24, 2012 at 10:46 PM, Gleb Peregud wrote: >> On Thu, May 24, 2012 at 3:42 PM, Daniel Goertzen >> wrote: >>> You're right, that's a terrible name for this project. This project is now at: >>> >>> https://github.com/goertzenator/goyaml >> >> I'm afraid people using Go programming language will be stumbling upon >> your project now :) >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vladdu55@REDACTED Fri May 25 09:12:05 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 25 May 2012 09:12:05 +0200 Subject: [erlang-questions] Getting rid of the preprocessor In-Reply-To: <390E1F41-608F-4FE3-B4B4-3B7685BD0D58@cs.otago.ac.nz> References: <390E1F41-608F-4FE3-B4B4-3B7685BD0D58@cs.otago.ac.nz> Message-ID: Hi Richard, My real question got buried in the discussion about my particular use case: previously when the preprocessor and its demise were discussed, one main reason for not being able to do anything about it at the moment was that first we need to do something about the records. But the preprocessor doesn't do anything to records, they are processed by the compiler. To quote you from a recent thread: "Abstract patterns and frames are all part of a long-time project to make the preprocessor unnecessary". Could you please enlighten me as to how the preprocessor is involved here? On Fri, May 25, 2012 at 12:16 AM, Richard O'Keefe wrote: > On 25/05/2012, at 12:02 AM, Vlad Dumitrescu wrote: >> As one example, how would you parse the following line? >> ? ??HELLO(world) > I am no friend of the preprocessor. > But this at least is no new problem: ?C has exactly the same > problem, and presumably the C and C++ support in Eclipse > already does something with it. Yes, it's not new and yes, the C/C++ support does something with it. The problem at hand is that we don't have the kind of resources that were put into the C support by the likes of Intel, IBM, Texas Instruments and, yes, even Ericsson. There are many tens of man-years invested in that project... > I presume it goes something like this: > ? ? ? ?if it looks like a variable, display it as one; > ? ? ? ?if it looks like a function call, display it as one; > ? ? ? ?if the display is wrong, it's the programmer's fault > ? ? ? ?for defining such a stupid macro. The displaying of fancy colors in the editor is not the important issue. An IDE needs to know as much detail as possible about the code, to support navigation and refactorings, to be able to suggest meaningful completions and offer reasonable ways to fix any issues it finds. This isn't possible if one can't understand (=parse) the code properly. > ? ? ? ?if the display is wrong, it's the programmer's fault > ? ? ? ?for defining such a stupid macro. Unfortunately, this argument doesn't really work for legacy systems, where the development environment has to work with code that won't be changed unless it's broken at runtime. For macros that expand to well-formed expressions, we can treat them as function calls, that's the easy part. There are other uses where the parser needs to be prepared for all kinds of weirdness. The most pervasive example is the '?line' macro used with common_test (this is now no longer necessary, but see above about legacy), but I have to handle for example macros that expand to: - whole function clauses, in the middle of regular clauses - guard tests including the 'when' keyword The parser grammar is basically describing this nice language, except that any grammar construct or combination thereof may be represented by a macro call... In any case, the problems related to smart editors are not the only ones, there were many people complaining for not so many years ago and you wrote the oh-so-old-that-Google-can't-find-an-electronic-copy-if-one-even-exists paper "Delenda est preprocessor" even before that (1998?). I'm taking the liberty of quoting you here: "... the preprocessor is violently at odds with everything else in the language. One of the papers I wrote for SERC had the title "Delenda Est Preprocessor". There really isn't anything that can be done with the processor that could not be done better without it. In particular, one of the major things about Erlang is the module system combined with hot loading, but the preprocessor subverts the module system and causes dependencies between source units that are not and cannot be tracked by the run time system." [http://erlang.org/pipermail/erlang-questions/2006-March/019614.html] best regards, Vlad From tilman.holschuh@REDACTED Fri May 25 09:23:32 2012 From: tilman.holschuh@REDACTED (Tilman Holschuh) Date: Fri, 25 May 2012 00:23:32 -0700 Subject: [erlang-questions] [ANN] yaml loader In-Reply-To: <-5512473052633725442@unknownmsgid> References: <-5512473052633725442@unknownmsgid> Message-ID: <56ACBC2E-B257-420D-8C7A-39D3B42CAA82@gmail.com> yamler On 2012-05-24, at 11:41 PM, Max Bourinov wrote: > Eyaml or erlyaml? > > Sent from my iPhone > > On 25.05.2012, at 9:27, Michael Turner wrote: > >> Then ... "yamerl"? >> >> -michael turner >> >> On Thu, May 24, 2012 at 10:46 PM, Gleb Peregud wrote: >>> On Thu, May 24, 2012 at 3:42 PM, Daniel Goertzen >>> wrote: >>>> You're right, that's a terrible name for this project. This project is now at: >>>> >>>> https://github.com/goertzenator/goyaml >>> >>> I'm afraid people using Go programming language will be stumbling upon >>> your project now :) >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From andre@REDACTED Fri May 25 09:27:46 2012 From: andre@REDACTED (=?ISO-8859-1?Q?Andr=E9_Graf?=) Date: Fri, 25 May 2012 09:27:46 +0200 Subject: [erlang-questions] [ANN] yaml loader In-Reply-To: <56ACBC2E-B257-420D-8C7A-39D3B42CAA82@gmail.com> References: <-5512473052633725442@unknownmsgid> <56ACBC2E-B257-420D-8C7A-39D3B42CAA82@gmail.com> Message-ID: +1 for yamler On 25 May 2012 09:23, Tilman Holschuh wrote: > yamler > > On 2012-05-24, at 11:41 PM, Max Bourinov wrote: > >> Eyaml or erlyaml? >> >> Sent from my iPhone >> >> On 25.05.2012, at 9:27, Michael Turner wrote: >> >>> Then ... "yamerl"? >>> >>> -michael turner >>> >>> On Thu, May 24, 2012 at 10:46 PM, Gleb Peregud wrote: >>>> On Thu, May 24, 2012 at 3:42 PM, Daniel Goertzen >>>> wrote: >>>>> You're right, that's a terrible name for this project. ?This project is now at: >>>>> >>>>> https://github.com/goertzenator/goyaml >>>> >>>> I'm afraid people using Go programming language will be stumbling upon >>>> your project now :) >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ulf@REDACTED Fri May 25 09:31:38 2012 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 25 May 2012 09:31:38 +0200 Subject: [erlang-questions] Getting rid of the preprocessor In-Reply-To: References: <390E1F41-608F-4FE3-B4B4-3B7685BD0D58@cs.otago.ac.nz> Message-ID: On 25 May 2012, at 09:12, Vlad Dumitrescu wrote: > The displaying of fancy colors in the editor is not the important > issue. An IDE needs to know as much detail as possible about the code, > to support navigation and refactorings, to be able to suggest > meaningful completions and offer reasonable ways to fix any issues it > finds. This isn't possible if one can't understand (=parse) the code > properly. Yes - many years ago, I wrote a source code browser for code stored in ClearCase, allowing you to select a "config spec" and get a consistent view of all the code. To make it attractive to our erlang programmers, I put a lot of effort into doing syntax highlighting and cross-linking of erlang code. You could click on a function head and get a listing of all call points, for example. I struggled with what to do with macros. I finally decided to treat macros and records as similar to functions, and cross-linked them too. Thus, clicking on a macro invocation would take you to the definition of the macro, and from a macro definition, you could find all the invocations - same with records. Personally, I found this very useful, and did put in some work to try to make it VCS-agnostic (it did handle CVS repositories, actually). In the end, though, I gave up trying to maintain it, since I really didn't have the time to do so. I also found that after a series of unfinished refactorings to make the code more maintainable, it was more or less *impossible* to maintain. :) Still, compared to ErlIDE/Eclipse, this tool had pretty shallow ambitions as regards actually understanding what the code did. But just having that level of cross-linking was a great benefit. Somewhere, there is a cost-benefit threshold, where reaching for more sophistication simply doesn't pay off. (At the scale of AXD 301 - 1.5 million lines of code and lots of branches in the repos - I felt I had exceeded that threshold when my cross-reference database exhausted the inodes on the SUN server it was running on). BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From Barys_Ilyushonak@REDACTED Fri May 25 10:29:41 2012 From: Barys_Ilyushonak@REDACTED (Ilyushonak Barys) Date: Fri, 25 May 2012 08:29:41 +0000 Subject: [erlang-questions] amqp 0.10 erlang client Message-ID: Hi there, I'm looking for the possibility to work with amqp 0.10 messaging broker in Erlang. As I see, rabbitmq client supports only 0.9.1 version. If you have an experience or thoughts in this case, could you please share it? Regards, Boris _______________________________________________________ The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia. If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp From vladdu55@REDACTED Fri May 25 11:23:46 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 25 May 2012 11:23:46 +0200 Subject: [erlang-questions] Getting rid of the preprocessor In-Reply-To: References: <390E1F41-608F-4FE3-B4B4-3B7685BD0D58@cs.otago.ac.nz> Message-ID: On Fri, May 25, 2012 at 9:31 AM, Ulf Wiger wrote: > > On 25 May 2012, at 09:12, Vlad Dumitrescu wrote: >> The displaying of fancy colors in the editor is not the important >> issue. An IDE needs to know as much detail as possible about the code, > > Still, compared to ErlIDE/Eclipse, this tool had pretty shallow > ambitions as regards actually understanding what the code did. > But just having that level of cross-linking was a great benefit. > Somewhere, there is a cost-benefit threshold, where reaching for > more sophistication simply doesn't pay off. Hi Ulf, I think one issue with using Eclipse is that many of the users are coming from using Eclipse with Java or C, where the code cross-referencing works fully. If we have apparently the same functionality, but it doesn't really finds everything for one reason or another, then people are getting confused. They could even hack happily ahead, unaware of the problems they left behind. There is not much one can do about that, but try to cover as much ground as possible and put a warning in the docs (should anyone read them). regards, Vlad From martynasp@REDACTED Fri May 25 11:24:07 2012 From: martynasp@REDACTED (Martynas Pumputis) Date: Fri, 25 May 2012 11:24:07 +0200 Subject: [erlang-questions] What causes nodes to become disconnected/reconnected? In-Reply-To: <020101cd39c9$9b3f0260$d1bd0720$@com> References: <020101cd39c9$9b3f0260$d1bd0720$@com> Message-ID: Erlang doesn't detect net splits by itself. You could start looking at net_kernel:set_net_ticktime/2 (try to increase this value if your node is suffering from high load/traffic) and kernel/src/dist_utils.erl to get a grasp how erlang handles node connections. Martynas On Thu, May 24, 2012 at 6:23 PM, David Mercer wrote: > (Yes, I am still working on my issue with a distributed application that > keeps losing its connection to the other node.)**** > > ** ** > > It is not a problem only when I have 2 nodes on the *same* host. I was > running the nodes on *different* hosts last night, and this morning the > failover node had lost its connection to the main and so had started its > own instance of the application. Calling nodes() on the failover > returned [].**** > > ** ** > > Then I started a new node on the same host as the main (to see if it would > restore the connections), and, yes, it did. After starting the third node, nodes() > now on the failover node returns a list of two nodes, the two on the main > host. However, the application on the failover node did not shut down, and > so it is still running on both the main and failover nodes.**** > > ** ** > > To summarize:**** > > ** ** > > **1. **Distributed application running on a node on host A (?main@REDACTED?), > failover on node on host B (?failover@REDACTED?).**** > > ** ** > > **2. **At some point, *failover@REDACTED* becomes disconnected from *main@REDACTED > *, and the application starts on *failover@REDACTED*. Now there are two > instances of the application running.**** > > ** ** > > **3. **From a network point of view, there is still (or again) a > valid network connection between hosts A and B. I can?t say for sure if > some network/firewall/other issue caused a temporary disconnect, but I can > say that by the time I got in this morning, when the application was > running on both nodes, there was a firm network connection between the two > hosts.**** > > ** ** > > **4. **Calling nodes() on *failover@REDACTED* returns [].**** > > ** ** > > **5. **A new (failover) node was started on host A (?failover@REDACTED?). > It does not start the application (which is correct, since it is already > running, albeit on *both* other nodes instead of just one).**** > > ** ** > > **6. **Calling nodes() on *failover@REDACTED* now returns [failover@REDACTED > ,main@REDACTED].**** > > ** ** > > **7. **The application, however, is still running on *failover@REDACTED*, > despite the fact that nodes/0 reports a connection to *main@REDACTED*.**** > > ** ** > > I don?t need someone to diagnose this for me. If someone could just > educate me a little on how the connections work, how net splits are > detected and nodes disconnected, etc., I might be able to take it from > there. Anyone know enough and have the time to type out a little blurb? I > can read source code, but having a little background knowledge would help > put it into context for me.**** > > ** ** > > Thank-you!**** > > ** ** > > Cheers,**** > > ** ** > > David**** > > ** ** > > ** ** > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jozsef.berces@REDACTED Fri May 25 11:30:03 2012 From: jozsef.berces@REDACTED (=?iso-8859-1?Q?J=F3zsef_B=E9rces?=) Date: Fri, 25 May 2012 11:30:03 +0200 Subject: [erlang-questions] What causes nodes to become disconnected/reconnected? In-Reply-To: References: <020101cd39c9$9b3f0260$d1bd0720$@com> Message-ID: <3717CEF51B134A4F8073627839A9DF412744B3B02D@ESESSCMS0353.eemea.ericsson.se> It is somewhat offtopic as I do not believe David experenced this, but I have similar disconnection when Windows goes to sleep. After the wake-up, the nodes do not see each other. Does anyone have any idea how to fix this? Thanks, Jozsef ________________________________ From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Martynas Pumputis Sent: Friday, May 25, 2012 16:24 To: David Mercer Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] What causes nodes to become disconnected/reconnected? Erlang doesn't detect net splits by itself. You could start looking at net_kernel:set_net_ticktime/2 (try to increase this value if your node is suffering from high load/traffic) and kernel/src/dist_utils.erl to get a grasp how erlang handles node connections. Martynas On Thu, May 24, 2012 at 6:23 PM, David Mercer > wrote: (Yes, I am still working on my issue with a distributed application that keeps losing its connection to the other node.) It is not a problem only when I have 2 nodes on the same host. I was running the nodes on different hosts last night, and this morning the failover node had lost its connection to the main and so had started its own instance of the application. Calling nodes() on the failover returned []. Then I started a new node on the same host as the main (to see if it would restore the connections), and, yes, it did. After starting the third node, nodes() now on the failover node returns a list of two nodes, the two on the main host. However, the application on the failover node did not shut down, and so it is still running on both the main and failover nodes. To summarize: 1. Distributed application running on a node on host A ("main@REDACTED"), failover on node on host B ("failover@REDACTED"). 2. At some point, failover@REDACTED becomes disconnected from main@REDACTED, and the application starts on failover@REDACTED Now there are two instances of the application running. 3. From a network point of view, there is still (or again) a valid network connection between hosts A and B. I can't say for sure if some network/firewall/other issue caused a temporary disconnect, but I can say that by the time I got in this morning, when the application was running on both nodes, there was a firm network connection between the two hosts. 4. Calling nodes() on failover@REDACTED returns []. 5. A new (failover) node was started on host A ("failover@REDACTED"). It does not start the application (which is correct, since it is already running, albeit on both other nodes instead of just one). 6. Calling nodes() on failover@REDACTED now returns [failover@REDACTED,main@REDACTED]. 7. The application, however, is still running on failover@REDACTED, despite the fact that nodes/0 reports a connection to main@REDACTED I don't need someone to diagnose this for me. If someone could just educate me a little on how the connections work, how net splits are detected and nodes disconnected, etc., I might be able to take it from there. Anyone know enough and have the time to type out a little blurb? I can read source code, but having a little background knowledge would help put it into context for me. Thank-you! Cheers, David _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Fri May 25 13:03:23 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 25 May 2012 13:03:23 +0200 Subject: [erlang-questions] Getting rid of the preprocessor In-Reply-To: References: <390E1F41-608F-4FE3-B4B4-3B7685BD0D58@cs.otago.ac.nz> Message-ID: On Fri, May 25, 2012 at 9:12 AM, Vlad Dumitrescu wrote: > My real question got buried in the discussion about my particular use > case: previously when the preprocessor and its demise were discussed, > one main reason for not being able to do anything about it at the > moment was that first we need to do something about the records. But > the preprocessor doesn't do anything to records, they are processed by > the compiler. To quote you from a recent thread: "Abstract patterns > and frames are all part of a long-time project to make the > preprocessor unnecessary". Could you please enlighten me as to how the > preprocessor is involved here? I think I can answer that myself now: the preprocessor is needed because most of the times the record definitions are in hrl files, that are included in the module source. I don't see any other reasons, are there any? I missed that because including files and ifdef-ing sections of code aren't problems for me and are useful, so I kind of automatically assumed that a restricted preprocessor that performs these tasks will still be available. The predefined macros should probably be handled by the preprocessor too. best regards, Vlad From vladdu55@REDACTED Fri May 25 13:57:43 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 25 May 2012 13:57:43 +0200 Subject: [erlang-questions] missing options in compiler docs Message-ID: Hi, The compiler docs are not listing all the accepted options: inline and inline_size are not there. They are only described at the end, makes them easy to miss... I can fix that, but maybe there is a reason for it being so? (Like the reason for not including parse_transform might be that it is not a recommended nor supported activity?) best regards, Vlad From zabrane3@REDACTED Fri May 25 14:03:40 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Fri, 25 May 2012 14:03:40 +0200 Subject: [erlang-questions] [ANN] yaml loader In-Reply-To: References: <-5512473052633725442@unknownmsgid> <56ACBC2E-B257-420D-8C7A-39D3B42CAA82@gmail.com> Message-ID: +1 yamler Regards, Zabrane From daniel.goertzen@REDACTED Fri May 25 15:26:14 2012 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Fri, 25 May 2012 08:26:14 -0500 Subject: [erlang-questions] [ANN] yaml loader In-Reply-To: References: <-5512473052633725442@unknownmsgid> <56ACBC2E-B257-420D-8C7A-39D3B42CAA82@gmail.com> Message-ID: Lol, I love it! It is done... https://github.com/goertzenator/yamler Thanks for the input, Dan. On Fri, May 25, 2012 at 7:03 AM, Zabrane Mickael wrote: > +1 yamler > > > Regards, > Zabrane > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dmercer@REDACTED Fri May 25 15:39:16 2012 From: dmercer@REDACTED (David Mercer) Date: Fri, 25 May 2012 08:39:16 -0500 Subject: [erlang-questions] What causes nodes to become disconnected/reconnected? In-Reply-To: <3717CEF51B134A4F8073627839A9DF412744B3B02D@ESESSCMS0353.eemea.ericsson.se> References: <020101cd39c9$9b3f0260$d1bd0720$@com> <3717CEF51B134A4F8073627839A9DF412744B3B02D@ESESSCMS0353.eemea.ericsson.se> Message-ID: <026d01cd3a7b$c80b3ae0$5821b0a0$@com> These might be related, though. You are right, my case does not occur due to the host going to sleep, but both my hosts are Windows, and perhaps something is happening ? whether it be with the network, firewall, virus protection, or whatever ? that is causing the nodes to get disconnected, and they do not properly reconnect when the temporary issue is resolved. It might be the same thing in the Windows port of Erlang that prevents them from reconnecting after sleep that is preventing my nodes from reconnecting after something else. Cheers, DBM From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of J?zsef B?rces Sent: Friday, May 25, 2012 4:30 AM To: erlang-questions@REDACTED Subject: Re: [erlang-questions] What causes nodes to become disconnected/reconnected? It is somewhat offtopic as I do not believe David experenced this, but I have similar disconnection when Windows goes to sleep. After the wake-up, the nodes do not see each other. Does anyone have any idea how to fix this? Thanks, Jozsef _____ From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Martynas Pumputis Sent: Friday, May 25, 2012 16:24 To: David Mercer Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] What causes nodes to become disconnected/reconnected? Erlang doesn't detect net splits by itself. You could start looking at net_kernel:set_net_ticktime/2 (try to increase this value if your node is suffering from high load/traffic) and kernel/src/dist_utils.erl to get a grasp how erlang handles node connections. Martynas On Thu, May 24, 2012 at 6:23 PM, David Mercer wrote: (Yes, I am still working on my issue with a distributed application that keeps losing its connection to the other node.) It is not a problem only when I have 2 nodes on the same host. I was running the nodes on different hosts last night, and this morning the failover node had lost its connection to the main and so had started its own instance of the application. Calling nodes() on the failover returned []. Then I started a new node on the same host as the main (to see if it would restore the connections), and, yes, it did. After starting the third node, nodes() now on the failover node returns a list of two nodes, the two on the main host. However, the application on the failover node did not shut down, and so it is still running on both the main and failover nodes. To summarize: 1. Distributed application running on a node on host A (?main@REDACTED?), failover on node on host B (?failover@REDACTED?). 2. At some point, failover@REDACTED becomes disconnected from main@REDACTED, and the application starts on failover@REDACTED Now there are two instances of the application running. 3. From a network point of view, there is still (or again) a valid network connection between hosts A and B. I can?t say for sure if some network/firewall/other issue caused a temporary disconnect, but I can say that by the time I got in this morning, when the application was running on both nodes, there was a firm network connection between the two hosts. 4. Calling nodes() on failover@REDACTED returns []. 5. A new (failover) node was started on host A (?failover@REDACTED?). It does not start the application (which is correct, since it is already running, albeit on both other nodes instead of just one). 6. Calling nodes() on failover@REDACTED now returns [failover@REDACTED,main@REDACTED]. 7. The application, however, is still running on failover@REDACTED, despite the fact that nodes/0 reports a connection to main@REDACTED I don?t need someone to diagnose this for me. If someone could just educate me a little on how the connections work, how net splits are detected and nodes disconnected, etc., I might be able to take it from there. Anyone know enough and have the time to type out a little blurb? I can read source code, but having a little background knowledge would help put it into context for me. Thank-you! Cheers, David _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From freeakk@REDACTED Fri May 25 16:29:44 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Fri, 25 May 2012 18:29:44 +0400 Subject: [erlang-questions] Record validation In-Reply-To: References: Message-ID: There are no triggers for records. Call validation functions manually when you want to check data. -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.eugene.turner@REDACTED Fri May 25 17:18:06 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sat, 26 May 2012 00:18:06 +0900 Subject: [erlang-questions] Getting rid of the preprocessor In-Reply-To: References: <390E1F41-608F-4FE3-B4B4-3B7685BD0D58@cs.otago.ac.nz> Message-ID: > ... the oh-so-old-that-Google-can't-find-an-electronic-copy-if-one-even-exists > paper "Delenda est preprocessor" even before that (1998?). Is it this? Richard O'Keefe. Abstract patterns for Erlang. Fourth International Erlang/OTP User Conference. In context, the citations I've seen seem to indicate this paper. Perhaps someone would be so kind as to scan it and put it online? -michael turner On Fri, May 25, 2012 at 4:12 PM, Vlad Dumitrescu wrote: > Hi Richard, > > My real question got buried in the discussion about my particular use > case: previously when the preprocessor and its demise were discussed, > one main reason for not being able to do anything about it at the > moment was that first we need to do something about the records. But > the preprocessor doesn't do anything to records, they are processed by > the compiler. To quote you from a recent thread: "Abstract patterns > and frames are all part of a long-time project to make the > preprocessor unnecessary". Could you please enlighten me as to how the > preprocessor is involved here? > > On Fri, May 25, 2012 at 12:16 AM, Richard O'Keefe wrote: >> On 25/05/2012, at 12:02 AM, Vlad Dumitrescu wrote: >>> As one example, how would you parse the following line? >>> ? ??HELLO(world) >> I am no friend of the preprocessor. >> But this at least is no new problem: ?C has exactly the same >> problem, and presumably the C and C++ support in Eclipse >> already does something with it. > > Yes, it's not new and yes, the C/C++ support does something with it. > The problem at hand is that we don't have the kind of resources that > were put into the C support by the likes of Intel, IBM, Texas > Instruments and, yes, even Ericsson. There are many tens of man-years > invested in that project... > >> I presume it goes something like this: >> ? ? ? ?if it looks like a variable, display it as one; >> ? ? ? ?if it looks like a function call, display it as one; >> ? ? ? ?if the display is wrong, it's the programmer's fault >> ? ? ? ?for defining such a stupid macro. > > The displaying of fancy colors in the editor is not the important > issue. An IDE needs to know as much detail as possible about the code, > to support navigation and refactorings, to be able to suggest > meaningful completions and offer reasonable ways to fix any issues it > finds. This isn't possible if one can't understand (=parse) the code > properly. > >> ? ? ? ?if the display is wrong, it's the programmer's fault >> ? ? ? ?for defining such a stupid macro. > > Unfortunately, this argument doesn't really work for legacy systems, > where the development environment has to work with code that won't be > changed unless it's broken at runtime. > > For macros that expand to well-formed expressions, we can treat them > as function calls, that's the easy part. There are other uses where > the parser needs to be prepared for all kinds of weirdness. The most > pervasive example is the '?line' macro used with common_test (this is > now no longer necessary, but see above about legacy), but I have to > handle for example macros that expand to: > - whole function clauses, in the middle of regular clauses > - guard tests including the 'when' keyword > The parser grammar is basically describing this nice language, except > that any grammar construct or combination thereof may be represented > by a macro call... > > In any case, the problems related to smart editors are not the only > ones, there were many people complaining for not so many years ago and > you wrote the oh-so-old-that-Google-can't-find-an-electronic-copy-if-one-even-exists > paper "Delenda est preprocessor" even before that (1998?). > > I'm taking the liberty of quoting you here: > "... the preprocessor is violently > at odds with everything else in the language. ?One of the papers I > wrote for SERC had the title "Delenda Est Preprocessor". ?There > really isn't anything that can be done with the processor that > could not be done better without it. ?In particular, one of the > major things about Erlang is the module system combined with hot > loading, but the preprocessor subverts the module system and > causes dependencies between source units that are not and cannot > be tracked by the run time system." > [http://erlang.org/pipermail/erlang-questions/2006-March/019614.html] > > best regards, > Vlad > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From michael.eugene.turner@REDACTED Fri May 25 17:45:05 2012 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sat, 26 May 2012 00:45:05 +0900 Subject: [erlang-questions] Getting rid of the preprocessor In-Reply-To: References: <390E1F41-608F-4FE3-B4B4-3B7685BD0D58@cs.otago.ac.nz> Message-ID: "For macros that expand to well-formed expressions, we can treat them as function calls, that's the easy part." That would be nice. After an exchange with ROK (mostly offline) about how to achieve a pattern-directed invocation style more like that of Smalltalk, I found myself experimenting with ungainly syntax that included using macros in arguments, to wrap the prepositional phrases. The result is too embarrassing to show here. If the macro invocations were instead inlined function calls, it would be slightly less embarrassing. -michael turner On Fri, May 25, 2012 at 4:12 PM, Vlad Dumitrescu wrote: > Hi Richard, > > My real question got buried in the discussion about my particular use > case: previously when the preprocessor and its demise were discussed, > one main reason for not being able to do anything about it at the > moment was that first we need to do something about the records. But > the preprocessor doesn't do anything to records, they are processed by > the compiler. To quote you from a recent thread: "Abstract patterns > and frames are all part of a long-time project to make the > preprocessor unnecessary". Could you please enlighten me as to how the > preprocessor is involved here? > > On Fri, May 25, 2012 at 12:16 AM, Richard O'Keefe wrote: >> On 25/05/2012, at 12:02 AM, Vlad Dumitrescu wrote: >>> As one example, how would you parse the following line? >>> ? ??HELLO(world) >> I am no friend of the preprocessor. >> But this at least is no new problem: ?C has exactly the same >> problem, and presumably the C and C++ support in Eclipse >> already does something with it. > > Yes, it's not new and yes, the C/C++ support does something with it. > The problem at hand is that we don't have the kind of resources that > were put into the C support by the likes of Intel, IBM, Texas > Instruments and, yes, even Ericsson. There are many tens of man-years > invested in that project... > >> I presume it goes something like this: >> ? ? ? ?if it looks like a variable, display it as one; >> ? ? ? ?if it looks like a function call, display it as one; >> ? ? ? ?if the display is wrong, it's the programmer's fault >> ? ? ? ?for defining such a stupid macro. > > The displaying of fancy colors in the editor is not the important > issue. An IDE needs to know as much detail as possible about the code, > to support navigation and refactorings, to be able to suggest > meaningful completions and offer reasonable ways to fix any issues it > finds. This isn't possible if one can't understand (=parse) the code > properly. > >> ? ? ? ?if the display is wrong, it's the programmer's fault >> ? ? ? ?for defining such a stupid macro. > > Unfortunately, this argument doesn't really work for legacy systems, > where the development environment has to work with code that won't be > changed unless it's broken at runtime. > > For macros that expand to well-formed expressions, we can treat them > as function calls, that's the easy part. There are other uses where > the parser needs to be prepared for all kinds of weirdness. The most > pervasive example is the '?line' macro used with common_test (this is > now no longer necessary, but see above about legacy), but I have to > handle for example macros that expand to: > - whole function clauses, in the middle of regular clauses > - guard tests including the 'when' keyword > The parser grammar is basically describing this nice language, except > that any grammar construct or combination thereof may be represented > by a macro call... > > In any case, the problems related to smart editors are not the only > ones, there were many people complaining for not so many years ago and > you wrote the oh-so-old-that-Google-can't-find-an-electronic-copy-if-one-even-exists > paper "Delenda est preprocessor" even before that (1998?). > > I'm taking the liberty of quoting you here: > "... the preprocessor is violently > at odds with everything else in the language. ?One of the papers I > wrote for SERC had the title "Delenda Est Preprocessor". ?There > really isn't anything that can be done with the processor that > could not be done better without it. ?In particular, one of the > major things about Erlang is the module system combined with hot > loading, but the preprocessor subverts the module system and > causes dependencies between source units that are not and cannot > be tracked by the run time system." > [http://erlang.org/pipermail/erlang-questions/2006-March/019614.html] > > best regards, > Vlad > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jose.valim@REDACTED Fri May 25 17:57:22 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Fri, 25 May 2012 17:57:22 +0200 Subject: [erlang-questions] [ANN] Elixir v0.5.0 released Message-ID: Hello everyone, Today we have officially released Elixir. Elixir is a programming language built on top of the Erlang. It is homoiconic, allowing meta-programming via macros, it also supports polymorphism via protocols (similar to Clojure's), dynamic records and a module reference mechanism. Elixir aims to stay compatible with Erlang, so it shares the same bytecode and data types. This means you can invoke Erlang code from Elixir (and vice-versa) without any conversion or performance hit. This allows a developer to mix the expressiveness of Elixir with the robustness and performance of Erlang. The getting started guide is here: http://elixir-lang.org/getting_started/1.html We also have a fast crash course for Erlang developers: http://elixir-lang.org/crash-course.html If anyone is interested in the release background, here is the announcement: http://elixir-lang.org/blog/2012/05/25/elixir-v0-5-0-released/ If anyone would like to discuss Elixir or give feedback, feel free to join #elixir-lang on irc.freenode.com Thanks! * Jos? Valim www.plataformatec.com.br Founder and Lead Developer * -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri May 25 18:15:48 2012 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 25 May 2012 20:15:48 +0400 Subject: [erlang-questions] [ANN] Elixir v0.5.0 released In-Reply-To: References: Message-ID: Is it possible to use elixir to emulate objects on top of gen_server with their "instance variables" which are just fields of record? From jose.valim@REDACTED Fri May 25 18:19:55 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Fri, 25 May 2012 18:19:55 +0200 Subject: [erlang-questions] [ANN] Elixir v0.5.0 released In-Reply-To: References: Message-ID: > > Is it possible to use elixir to emulate objects on top of gen_server > with their "instance variables" which are just fields of record? > Elixir does not have the concept of objects. So you have to do the same as Erlang. You need to create a record and access/manipulate the records fields. There is a difference that Elixir records are not a preprocessor feature, so access/manipulation is less verbose. -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri May 25 18:45:37 2012 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Fri, 25 May 2012 18:45:37 +0200 Subject: [erlang-questions] EUC 2012 Hackaton Message-ID: <4FBFB731.8060103@ninenines.eu> Just a heads up, there *will* be a hackaton this year, on the 3rd day, at the same location as the tutorial. I do not have all the details about the room yet but you can get some info from this page: http://www.erlang-factory.com/conference/ErlangUserConference2012/hackathon Feel free to ping me to say you will attend, although that is by no means mandatory. See you there! -- Lo?c Hoguin Erlang Cowboy Nine Nines From jose.valim@REDACTED Fri May 25 20:12:44 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Fri, 25 May 2012 20:12:44 +0200 Subject: [erlang-questions] [ANN] Elixir v0.5.0 released In-Reply-To: References: Message-ID: Actually, a better answer to this question would be: it should be easy for you to create a macro that will generate the appropriate handle_call or handle_cast functions based on a record. Here is a spike (I haven't tested it, but you will have an idea): https://gist.github.com/fb133f91db77a9938d57 Feel free to ping me by e-mail if you want to go further into this. :) * Jos? Valim www.plataformatec.com.br Founder and Lead Developer * On Fri, May 25, 2012 at 6:19 PM, Jos? Valim wrote: > Is it possible to use elixir to emulate objects on top of gen_server >> with their "instance variables" which are just fields of record? >> > > Elixir does not have the concept of objects. So you have to do the same as > Erlang. You need to create a record and access/manipulate the records > fields. There is a difference that Elixir records are not a preprocessor > feature, so access/manipulation is less verbose. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Fri May 25 21:20:40 2012 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 25 May 2012 12:20:40 -0700 Subject: [erlang-questions] Record validation In-Reply-To: References: Message-ID: Ok, thanks Michael! On Fri, May 25, 2012 at 7:29 AM, Michael Uvarov wrote: > There are no triggers for records. Call validation functions manually when > you want to check data. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Fri May 25 21:30:04 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Fri, 25 May 2012 22:30:04 +0300 Subject: [erlang-questions] gb_trees missing function!? Message-ID: <63E687EC-6E5F-4F36-9BE6-3355DE11B24C@gmail.com> Hello, gb_trees is pretty efficient in many use cases but I am starting to miss couple of features: * range(From, To, Tree) -> [{Key, Val}] to retrieve multiple keys * iterator(Key, Tree) -> Iter start iterator from Key of course lists:dropwhile(fun(X) -> X < Key end, gb_trees:to_list(Tree)) makes trick but I think it is not very efficient... What is your opinion? - Dmitry From forrie@REDACTED Fri May 25 21:39:49 2012 From: forrie@REDACTED (Forrest Aldrich) Date: Fri, 25 May 2012 15:39:49 -0400 Subject: [erlang-questions] Erlang compile/install vs. RPMs, issues Message-ID: <4FBFE005.1090204@gmail.com> We are using RabbitMQ for a new production system, which requires Erlang and I've run into a few issues getting things to install properly. First of all, I found the EPEL repositories to be an issue in that the 32-bit RPMs are more current, whereas the 64-bit RPMs don't really exist. When installing from one of them onto a 64-bit platform, I ran into a pesky dependency on wxGTK (which we'll never, ever use). So I compiled from source... Compiling from source also requires that you "touch" lib/wx/SKIP to disable that feature (which isn't that done with ./configure?). "make clean" happily removes those SKIP files. Ultimately, I wasn't able to figure out how to get it to install into the filesystem properly, so that RabbitMQ is happy. The RPMs do a lot of nitty gritty work, such as creating directory structures, creating users and permissions. I was able to get the 32-bit lot to work fine, but we require a 64-bit architecture. My question is ultimately about either where to obtain an official (if that exists) Source RPM for the latest R15B01 version that is suitable for RHEL 5.x, or how to ./configure from the source to mimic the installation processes that the RPMs normally do. I see other Source RPMs on Rpmbone, but some refer to dependencies on RHEL 6.x and updated libc, which conflicts with our environment. If someone can point me in the right direction, I would appreciate it. I've spent about 3 working days fussing over this :-) Forrest -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Fri May 25 21:59:29 2012 From: garazdawi@REDACTED (Lukas Larsson) Date: Fri, 25 May 2012 21:59:29 +0200 Subject: [erlang-questions] Erlang compile/install vs. RPMs, issues In-Reply-To: <4FBFE005.1090204@gmail.com> References: <4FBFE005.1090204@gmail.com> Message-ID: <81428056-E51F-4337-A1AE-9F511BDB0F43@gmail.com> Hi, Have you tried the rpm here: http://www.erlang-solutions.com/section/132/download-erlang-otp It is not RHEL specific but it might be worth a try. Lukas Sent from me On 25 maj 2012, at 21:39, Forrest Aldrich wrote: > We are using RabbitMQ for a new production system, which requires Erlang and I've run into a few issues getting things to install properly. > > First of all, I found the EPEL repositories to be an issue in that the 32-bit RPMs are more current, whereas the 64-bit RPMs don't really exist. When installing from one of them onto a 64-bit platform, I ran into a pesky dependency on wxGTK (which we'll never, ever use). So I compiled from source... > > Compiling from source also requires that you "touch" lib/wx/SKIP to disable that feature (which isn't that done with ./configure?). "make clean" happily removes those SKIP files. Ultimately, I wasn't able to figure out how to get it to install into the filesystem properly, so that RabbitMQ is happy. > > The RPMs do a lot of nitty gritty work, such as creating directory structures, creating users and permissions. I was able to get the 32-bit lot to work fine, but we require a 64-bit architecture. > > My question is ultimately about either where to obtain an official (if that exists) Source RPM for the latest R15B01 version that is suitable for RHEL 5.x, or how to ./configure from the source to mimic the installation processes that the RPMs normally do. I see other Source RPMs on Rpmbone, but some refer to dependencies on RHEL 6.x and updated libc, which conflicts with our environment. > > If someone can point me in the right direction, I would appreciate it. I've spent about 3 working days fussing over this :-) > > > > Forrest > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From forrie@REDACTED Fri May 25 22:32:48 2012 From: forrie@REDACTED (Forrest Aldrich) Date: Fri, 25 May 2012 16:32:48 -0400 Subject: [erlang-questions] Erlang compile/install vs. RPMs, issues In-Reply-To: <81428056-E51F-4337-A1AE-9F511BDB0F43@gmail.com> References: <4FBFE005.1090204@gmail.com> <81428056-E51F-4337-A1AE-9F511BDB0F43@gmail.com> Message-ID: <4FBFEC70.7070806@gmail.com> On 5/25/12 3:59 PM, Lukas Larsson wrote: > Hi, > > Have you tried the rpm here: http://www.erlang-solutions.com/section/132/download-erlang-otp > > It is not RHEL specific but it might be worth a try. > > Lukas > Thank you, Lukas. I was unaware of this distribution. I am building a new 64-bit CentOS VM to test this out on. I will let you know. Thanks! Forrest -------------- next part -------------- An HTML attachment was scrubbed... URL: From forrie@REDACTED Fri May 25 22:49:31 2012 From: forrie@REDACTED (Forrest Aldrich) Date: Fri, 25 May 2012 16:49:31 -0400 Subject: [erlang-questions] Erlang compile/install vs. RPMs, issues In-Reply-To: <81428056-E51F-4337-A1AE-9F511BDB0F43@gmail.com> References: <4FBFE005.1090204@gmail.com> <81428056-E51F-4337-A1AE-9F511BDB0F43@gmail.com> Message-ID: <4FBFF05B.7080104@gmail.com> On 5/25/12 3:59 PM, Lukas Larsson wrote: > Hi, > > Have you tried the rpm here: http://www.erlang-solutions.com/section/132/download-erlang-otp > > It is not RHEL specific but it might be worth a try. > > Lukas Lukas, I'm running into the same problem as before, with a 64-bit system and dependencies. Here is the output: http://pastebin.com/k9Cgrn0z We don't/won't even use wxGTK. Do you have a Source RPM available, maybe I/we could edit that out and rebuild it from source? Thanks, Forrest -------------- next part -------------- An HTML attachment was scrubbed... URL: From forrie@REDACTED Fri May 25 23:31:04 2012 From: forrie@REDACTED (Forrest Aldrich) Date: Fri, 25 May 2012 17:31:04 -0400 Subject: [erlang-questions] Default install prefix Message-ID: <4FBFFA18.4080700@gmail.com> My configure flags are pretty straightforward: ./configure --prefix=/usr --enable-m64-build --without-javac --with-odbc=/usr WHY does erlang insist on doing this for the installation? /usr/lib/erlang/bin/./erl No other program is going to find this in a reasonable $PATH setting. The ./configure --prefix=/usr should assume /usr as the root and bin/ underneath that, just like any other program. Am I missing something here. Forrest -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Fri May 25 23:59:01 2012 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Sat, 26 May 2012 00:59:01 +0300 Subject: [erlang-questions] Default install prefix In-Reply-To: <4FBFFA18.4080700@gmail.com> References: <4FBFFA18.4080700@gmail.com> Message-ID: Hello, you should have /usr/bin and it should contains a symbolic links to ../lib/erlang/bin/* I do not know what was the actual design decision to put binaries under ../lib/erlang/bin but I am suspecting it facilitates "release" creation. The release contains a full VM binaries and erts it is much more easy to tar gzip ../lib/erlang/bin instead of filtering /usr/bin for relevant files. - Dmitry On May 26, 2012, at 12:31 AM, Forrest Aldrich wrote: > My configure flags are pretty straightforward: > > ./configure --prefix=/usr --enable-m64-build --without-javac --with-odbc=/usr > > WHY does erlang insist on doing this for the installation? > > /usr/lib/erlang/bin/./erl > > No other program is going to find this in a reasonable $PATH setting. > > The ./configure --prefix=/usr should assume /usr as the root and bin/ underneath that, just like any other program. Am I missing something here. > > > > Forrest > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From overminddl1@REDACTED Sat May 26 03:17:11 2012 From: overminddl1@REDACTED (OvermindDL1) Date: Fri, 25 May 2012 19:17:11 -0600 Subject: [erlang-questions] [ANN] Elixir v0.5.0 released In-Reply-To: References: Message-ID: On Fri, May 25, 2012 at 9:57 AM, Jos? Valim wrote: > Hello everyone, > > Today we have officially released Elixir. > > Elixir is a programming language built on top of the Erlang. It is > homoiconic, allowing meta-programming via macros, it also supports > polymorphism via protocols (similar to Clojure's), dynamic records and a > module reference mechanism. > > Elixir aims to stay compatible with Erlang, so it shares the same bytecode > and data types. This means you can invoke Erlang code from Elixir (and > vice-versa) without any conversion or performance hit. This allows a > developer to mix the expressiveness of Elixir with the robustness and > performance of Erlang. > > The getting started guide is here: > > http://elixir-lang.org/getting_started/1.html > > We also have a fast crash course for Erlang developers: > > http://elixir-lang.org/crash-course.html > > If anyone is interested in the release background, here is the announcement: > > http://elixir-lang.org/blog/2012/05/25/elixir-v0-5-0-released/ > > If anyone would like to discuss Elixir or give feedback, feel free to join > #elixir-lang on irc.freenode.com I like the looks of this. How is its rebar integration? Any chance of a rebar module to let rebar transparently manage elixir files like it does erlang files? From freeakk@REDACTED Sat May 26 07:03:08 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Sat, 26 May 2012 09:03:08 +0400 Subject: [erlang-questions] Record validation In-Reply-To: References: Message-ID: The nice thing about moving validation into a function is that you can have few functions for different cases. For example, *valid_manager(#user{}) -> boolean(). valid_user(#user{}) -> boolean(). * Best regards, Uvarov Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Sat May 26 08:35:28 2012 From: bourinov@REDACTED (Max Bourinov) Date: Sat, 26 May 2012 10:35:28 +0400 Subject: [erlang-questions] Erlang compile/install vs. RPMs, issues In-Reply-To: <4FBFF05B.7080104@gmail.com> References: <4FBFE005.1090204@gmail.com> <81428056-E51F-4337-A1AE-9F511BDB0F43@gmail.com> <4FBFF05B.7080104@gmail.com> Message-ID: <-1629906112113031790@unknownmsgid> Hi Forrest, $ sudo apt-get install make $ sudo apt-get install gcc $ sudo apt-get build-dep erlang $ sudo apt-get install tk # Now you can install kerl and erlang *https://github.com/spawngrid/kerl* * * *This is from our production servers.* * * * *Sent from my iPhone On 26.05.2012, at 0:50, Forrest Aldrich wrote: On 5/25/12 3:59 PM, Lukas Larsson wrote: Hi, Have you tried the rpm here: http://www.erlang-solutions.com/section/132/download-erlang-otp It is not RHEL specific but it might be worth a try. Lukas Lukas, I'm running into the same problem as before, with a 64-bit system and dependencies. Here is the output: http://pastebin.com/k9Cgrn0z We don't/won't even use wxGTK. Do you have a Source RPM available, maybe I/we could edit that out and rebuild it from source? Thanks, Forrest _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Sat May 26 11:06:51 2012 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sat, 26 May 2012 11:06:51 +0200 Subject: [erlang-questions] managing OS processes Message-ID: I'm trying to find a way to manage some os processes launched from Erlang. For example I need to relaunch them when the external process die for an unknown reason or send an HUP signal or just stop the external process. For now I'm using erlang ports to open an external process and detect when it die. I also send signals using the `os:cmd` and and always send a SIGKILL signal using the `os:cmd`before closing the port. But is there a better way to do that? Maybe a module somewhere ? I saw erlexec [1] but it doesn't seem to be maintained since a while. - beno?t [1] http://code.google.com/p/erlexec/ From jose.valim@REDACTED Sat May 26 12:29:00 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Sat, 26 May 2012 12:29:00 +0200 Subject: [erlang-questions] [ANN] Elixir v0.5.0 released In-Reply-To: References: Message-ID: We will be focusing on rebar integration and proper Erlang releases in the upcoming weeks! -- Sent from my iPhone -------------- next part -------------- An HTML attachment was scrubbed... URL: From gleber.p@REDACTED Sat May 26 14:12:36 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Sat, 26 May 2012 14:12:36 +0200 Subject: [erlang-questions] managing OS processes In-Reply-To: References: Message-ID: On Sat, May 26, 2012 at 11:06 AM, Benoit Chesneau wrote: > I'm trying to find a way to manage some os processes launched from > Erlang. For example I need to relaunch them when the external process > die for an unknown reason or send an HUP signal or just stop the > external process. Hello Benoit Looks like there are two of us, who needs a better mechanism to manage OS processes from Erlang. I need it for a CI server which I'm working on, so my requirements are not as high as yours, but still os:cmd doesn't work for me and ports are missing one feature which I need. Maybe we could reanimate erlexec together? Cheers, Gleb From avinash@REDACTED Sat May 26 18:13:13 2012 From: avinash@REDACTED (Avinash Dhumane) Date: Sat, 26 May 2012 21:43:13 +0530 Subject: [erlang-questions] Yaws and httpc Message-ID: I have a web client written using "httpc" module, and I have Yaws web server with all default configurations except for "listen_backlog", which is set to 10000 (ten thousand). I am not sure how to program error-recovery for a test-client application that uses "httpc" module when httpc:request/2 encounters an error as follows: >>>> Error in process <0.17159.0> with exit value: >>>> {{badmatch,{error,socket_closed_remotely}},[{qyuki_test,'-work/2-fun-0-',6,[{file,"qyuki_test.erl"},{line,34}]},{lists,map,2,[{file,"lists.erl"},{line,1173}]},{qyuki_test,work,2,[{file,"qyuki_test.erl"},{line,31}]}]} I presumed I would close the the dynamically created profile [*1] using inets:stop/2, and restart it with the same profile-name using inets:start/2, and submit the http request again using httpc:request/2. But, then, it encounters this error: >>>> Error in process <0.11323.0> with exit value: >>>> {{case_clause,{error,{failed_connect,[{to_address,{"192.168.1.4",80}},{inet,[inet],etimedout}]}}},[{qyuki_test,web_tran,6,[{file,"qyuki_test.erl"},{line,40}]},{qyuki_test,work,2,[{file,"qyuki_test.erl"},{line,31}]}]} Basically, the test-client application spawns 10000 (ten thousand) processes [*2], each creating its own profile with max_sessions = 1 and max_keep_alive_length = 1 [*3], and submitting the transactions synchronously in a loop with an internal of 20 seconds between each submission. The first submission for all processes go through successfully; but, from second submission onwards, some of the client processes start receiving "socket_closed_remotely" errors as shown above. Please advise on how to recover from this error and resume the transactions on the client side [*4] I tried altering Yaws configuration parameter keepalive_timeout from 30000 to 120000. I also tried keeping an internal (60 seconds) between stopping and restarting the client profile upon encountering the socket failures. But that didn't alter the error behaviours seen at the client side [*5] My assumption is that Yaws should not close the socket because the client takes care of not keeping the connection idle for more than 20 seconds. Please advise. Thanks Avinash [*1] The test-application needs to simulate many "web browser" clients, so I suppose I keep a separate profile for each client process. [*2] Both Erlang web client and Yaws (1.92) web server are running on R15B01 (erts-5.9.1) and on 32-bit Ubuntu Server 12, and on separate machines. ERL_MAX_PORTS, ERL_MAX_ETS_TABLES and "ulimit -n" are set to 64K on both client & server machines. [*3] My assumption is that "httpc" inets service should not open more than one connection to web server since test-client is submitting the transactions synchronously. But, I am seeing that the number of connections to the web server grow more than 16000 even when there are only 10000 "httpc" services running in the inets container. [*4] Yaws logs show no abnormal activities when "httpc" services on the client receive sockets failures, except for information on reaching the maximum number of files in cache. [*5] I have raised the number of processes to 500000 using +P only on the client side ERTS. From forrie@REDACTED Sat May 26 20:27:02 2012 From: forrie@REDACTED (Forrest Aldrich) Date: Sat, 26 May 2012 14:27:02 -0400 Subject: [erlang-questions] Erlang compile/install vs. RPMs, issues In-Reply-To: <-1629906112113031790@unknownmsgid> References: <4FBFE005.1090204@gmail.com> <81428056-E51F-4337-A1AE-9F511BDB0F43@gmail.com> <4FBFF05B.7080104@gmail.com> <-1629906112113031790@unknownmsgid> Message-ID: <4FC12076.7050308@gmail.com> On 5/26/12 2:35 AM, Max Bourinov wrote: > Hi Forrest, > > $ sudo apt-get install make > $ sudo apt-get install gcc > $ sudo apt-get build-dep erlang > $ sudo apt-get install tk > # Now you can install kerl and erlang > > *https://github.com/spawngrid/kerl* > * > * > *This is from our production servers.* > ** Max, We are using Redhat, so apt-get doesn't apply. I was finally able to get this working. There was a misunderstanding on my part about a couple of things, namely how erlang installs itself through "make install". Typically, I will install a product in --prefix=/usr/local/product and then point my $PATH to it. In this case, that didn't work so well, and I forgot to keep notes. I configured it without wxWIDGETS, java and something else -- compiled and installed it. Erlang creates symbolic links into /usr/bin for "erl" etc. At this point, RabbitMQ was happy... and I was happy (lol). I use an old, unmaintained program called "make uninstall" that records the writes in a "make install" so I can easily remove this for an update. I am curious, tho, if there are patches in the aforementioned erlang-solutions.com RPM. But it was for RH6, we're on RH5. Thanks again for everyone's input, I do appreciate it. It helped me work through this. Forrest -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Sat May 26 22:38:17 2012 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Sat, 26 May 2012 23:38:17 +0300 Subject: [erlang-questions] pg2 vs gproc? Message-ID: <20120526203817.GA10058@localhost> Hi, For term() to pid() mapping I've always used gproc. For the same thing my coworker always used pg2. I am making a comparison now. There are some features in gproc which are not in pg2: * Await for registration * Per-process properties * Aggregate counters * QLC Whereas pg2 "natively" supports mapping one key to several processes (which can be easily achieved with QLC in gproc). Any more differences between them? Thanks, Motiejus From watson.timothy@REDACTED Sat May 26 22:52:32 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 26 May 2012 21:52:32 +0100 Subject: [erlang-questions] managing OS processes In-Reply-To: References: Message-ID: I'm also working on very similar requirements in https://github.com/nebularis/systest/blob/master/src/systest_cli.erl and like you two, I have to rely on a combination of open_port with its constituent events and os:cmd/1 in order to SIGHUP/SIGKILL which feels rather 'hackish' to me. I've also noticed that it makes consumer code rather timing dependent, which is frustrating. I would be willing to participate in either (a) improving the built in port functionality or (b) working on erlexec Although C I'm fairly comfortable with, but C++ (which erlexec appears to be written in) isn't really my bag. I do have to ask though, is erlexec in need of much additional work? I also have 'client' modules for the OTP slave and ssh modules and don't want to invest a mountain of time in replacing open_port, but a little TLC to bring erlexec up to scratch would be time well spent if it would give a more consistent interface. In particular I have some issues with open_port based code with now such as 1. cli clients my code launches can be 'detached' (usually erlang nodes run with -detached or scripts that launch erlang nodes using `sh -C` or setsid/nohup and the like), which makes the port handling code both complex and messy. 2. some client code I'm testing will execute another (different) script/command to shut down the node and the listening port doesn't always get the exit status properly 3. signal handling is a mess So erlexec does sound promising, but IMO it would be *better* if the built in open_port functionality could be extending to deal with these situations properly. Cheers, Tim On 26 May 2012 13:12, Gleb Peregud wrote: > On Sat, May 26, 2012 at 11:06 AM, Benoit Chesneau wrote: >> I'm trying to find a way to manage some os processes launched from >> Erlang. For example I need to relaunch them when the external process >> die for an unknown reason or send an HUP signal or just stop the >> external process. > > Hello Benoit > > Looks like there are two of us, who needs a better mechanism to manage > OS processes from Erlang. I need it for a CI server which I'm working > on, so my requirements are not as high as yours, but still os:cmd > doesn't work for me and ports are missing one feature which I need. > > Maybe we could reanimate erlexec together? > > Cheers, > Gleb > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From anthonym@REDACTED Sun May 27 00:00:48 2012 From: anthonym@REDACTED (Anthony Molinaro) Date: Sat, 26 May 2012 15:00:48 -0700 Subject: [erlang-questions] managing OS processes In-Reply-To: References: Message-ID: <20120526220048.GA66695@alumni.caltech.edu> I've found a need for this sort of thing as well. +1 for getting something into OTP itself. I'm sure the real issue will be with Windows, although I believe there was some code in couchdb for doing some control of windows processes, and maybe erlexec already supports Windows? Anyway, I can't say I'd do much other than testing, but just wanted to voice my support. -Anthony On Sat, May 26, 2012 at 09:52:32PM +0100, Tim Watson wrote: > I'm also working on very similar requirements in > https://github.com/nebularis/systest/blob/master/src/systest_cli.erl > and like you two, I have to rely on a combination of open_port with > its constituent events and os:cmd/1 in order to SIGHUP/SIGKILL which > feels rather 'hackish' to me. I've also noticed that it makes consumer > code rather timing dependent, which is frustrating. > > I would be willing to participate in either > > (a) improving the built in port functionality or > (b) working on erlexec > > Although C I'm fairly comfortable with, but C++ (which erlexec appears > to be written in) isn't really my bag. I do have to ask though, is > erlexec in need of much additional work? I also have 'client' modules > for the OTP slave and ssh modules and don't want to invest a mountain > of time in replacing open_port, but a little TLC to bring erlexec up > to scratch would be time well spent if it would give a more consistent > interface. In particular I have some issues with open_port based code > with now such as > > 1. cli clients my code launches can be 'detached' (usually erlang > nodes run with -detached or scripts that launch erlang nodes using `sh > -C` or setsid/nohup and the like), which makes the port handling code > both complex and messy. > 2. some client code I'm testing will execute another (different) > script/command to shut down the node and the listening port doesn't > always get the exit status properly > 3. signal handling is a mess > > So erlexec does sound promising, but IMO it would be *better* if the > built in open_port functionality could be extending to deal with these > situations properly. > > Cheers, > Tim > > On 26 May 2012 13:12, Gleb Peregud wrote: > > On Sat, May 26, 2012 at 11:06 AM, Benoit Chesneau wrote: > >> I'm trying to find a way to manage some os processes launched from > >> Erlang. For example I need to relaunch them when the external process > >> die for an unknown reason or send an HUP signal or just stop the > >> external process. > > > > Hello Benoit > > > > Looks like there are two of us, who needs a better mechanism to manage > > OS processes from Erlang. I need it for a CI server which I'm working > > on, so my requirements are not as high as yours, but still os:cmd > > doesn't work for me and ports are missing one feature which I need. > > > > Maybe we could reanimate erlexec together? > > > > Cheers, > > Gleb > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- ------------------------------------------------------------------------ Anthony Molinaro From watson.timothy@REDACTED Sun May 27 00:45:28 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 26 May 2012 23:45:28 +0100 Subject: [erlang-questions] managing OS processes In-Reply-To: <20120526220048.GA66695@alumni.caltech.edu> References: <20120526220048.GA66695@alumni.caltech.edu> Message-ID: On 26 May 2012 23:00, Anthony Molinaro wrote: > I've found a need for this sort of thing as well. ?+1 for getting something > into OTP itself. ? I'm sure the real issue will be with Windows, although > I believe there was some code in couchdb for doing some control of windows > processes, and maybe erlexec already supports Windows? ?Anyway, I can't > say I'd do much other than testing, but just wanted to voice my support. > So I really do prefer the idea of patching the OTP mechanism, but I think it's worth those of us who're consider it discussing (a) what is missing that we'd like to have and (b) the general approach, with the OTP team before doing this. I do not want to spend a month building some patch that will just be rejected - I'd rather know up front if it's likely to be accepted and what kind of approach would be reasonable. That is the flip side of this - it would be better to have the feature in OTP proper, but it introduces risk to change the emulator and having *normal* port process handling erlexec is nice and safe. But not so clean. From zabrane3@REDACTED Sun May 27 00:48:22 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Sun, 27 May 2012 00:48:22 +0200 Subject: [erlang-questions] managing OS processes In-Reply-To: <20120526220048.GA66695@alumni.caltech.edu> References: <20120526220048.GA66695@alumni.caltech.edu> Message-ID: <0373D27A-3AF4-438E-ADD0-F2365A989617@gmail.com> Quick look to "erlexec" code shows that's only support Linux for now (with dependency on libpcap). Any revival work should extend "erlexec" to all other OSes when Erlang is expected to run (ex. Windows, OSX, FreeBSD ...) and try to minimize external dependencies. +1 I would be willing to help for a complete rewrite of "erlexec" (as Tim, I'm more C than a C++ guy). Regards, Zabrane On May 27, 2012, at 12:00 AM, Anthony Molinaro wrote: > I've found a need for this sort of thing as well. +1 for getting something > into OTP itself. I'm sure the real issue will be with Windows, although > I believe there was some code in couchdb for doing some control of windows > processes, and maybe erlexec already supports Windows? Anyway, I can't > say I'd do much other than testing, but just wanted to voice my support. > > -Anthony > > On Sat, May 26, 2012 at 09:52:32PM +0100, Tim Watson wrote: >> I'm also working on very similar requirements in >> https://github.com/nebularis/systest/blob/master/src/systest_cli.erl >> and like you two, I have to rely on a combination of open_port with >> its constituent events and os:cmd/1 in order to SIGHUP/SIGKILL which >> feels rather 'hackish' to me. I've also noticed that it makes consumer >> code rather timing dependent, which is frustrating. >> >> I would be willing to participate in either >> >> (a) improving the built in port functionality or >> (b) working on erlexec >> >> Although C I'm fairly comfortable with, but C++ (which erlexec appears >> to be written in) isn't really my bag. I do have to ask though, is >> erlexec in need of much additional work? I also have 'client' modules >> for the OTP slave and ssh modules and don't want to invest a mountain >> of time in replacing open_port, but a little TLC to bring erlexec up >> to scratch would be time well spent if it would give a more consistent >> interface. In particular I have some issues with open_port based code >> with now such as >> >> 1. cli clients my code launches can be 'detached' (usually erlang >> nodes run with -detached or scripts that launch erlang nodes using `sh >> -C` or setsid/nohup and the like), which makes the port handling code >> both complex and messy. >> 2. some client code I'm testing will execute another (different) >> script/command to shut down the node and the listening port doesn't >> always get the exit status properly >> 3. signal handling is a mess >> >> So erlexec does sound promising, but IMO it would be *better* if the >> built in open_port functionality could be extending to deal with these >> situations properly. >> >> Cheers, >> Tim >> >> On 26 May 2012 13:12, Gleb Peregud wrote: >>> On Sat, May 26, 2012 at 11:06 AM, Benoit Chesneau wrote: >>>> I'm trying to find a way to manage some os processes launched from >>>> Erlang. For example I need to relaunch them when the external process >>>> die for an unknown reason or send an HUP signal or just stop the >>>> external process. >>> >>> Hello Benoit >>> >>> Looks like there are two of us, who needs a better mechanism to manage >>> OS processes from Erlang. I need it for a CI server which I'm working >>> on, so my requirements are not as high as yours, but still os:cmd >>> doesn't work for me and ports are missing one feature which I need. >>> >>> Maybe we could reanimate erlexec together? >>> >>> Cheers, >>> Gleb >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -- > ------------------------------------------------------------------------ > Anthony Molinaro > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From gleber.p@REDACTED Sun May 27 11:44:58 2012 From: gleber.p@REDACTED (Gleb Peregud) Date: Sun, 27 May 2012 11:44:58 +0200 Subject: [erlang-questions] managing OS processes In-Reply-To: <0373D27A-3AF4-438E-ADD0-F2365A989617@gmail.com> References: <20120526220048.GA66695@alumni.caltech.edu> <0373D27A-3AF4-438E-ADD0-F2365A989617@gmail.com> Message-ID: On Sun, May 27, 2012 at 12:48 AM, Zabrane Mickael wrote: > Quick look to "erlexec" code shows that's only support Linux for now (with dependency on libpcap). > Any revival work should extend "erlexec" to all other OSes when Erlang is expected to run (ex. Windows, OSX, FreeBSD ...) and try to minimize external dependencies. > > +1 > > I would be willing to help for a complete rewrite of "erlexec" (as Tim, I'm more C than a C++ guy). Yes. In a long run improving open_port would be much better. The questions are - how to make it cross platform? Would OTP team accept such patches? Would OTP team accept not fully cross platform patch? Features which I feel are missing from oper_port: - receiving stderr and stdout separately - sending POSIX signals - setting startup ENV - setting startup working directory - setting OS process priority - monitoring of ports Anything else? From watson.timothy@REDACTED Sun May 27 13:25:48 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sun, 27 May 2012 12:25:48 +0100 Subject: [erlang-questions] managing OS processes In-Reply-To: References: <20120526220048.GA66695@alumni.caltech.edu> <0373D27A-3AF4-438E-ADD0-F2365A989617@gmail.com> Message-ID: <7CE35CA8-C56B-43E3-8F97-B259166A327D@gmail.com> On 27 May 2012, at 10:44, Gleb Peregud wrote: > On Sun, May 27, 2012 at 12:48 AM, Zabrane Mickael wrote: >> Quick look to "erlexec" code shows that's only support Linux for now (with dependency on libpcap). >> Any revival work should extend "erlexec" to all other OSes when Erlang is expected to run (ex. Windows, OSX, FreeBSD ...) and try to minimize external dependencies. >> >> +1 >> >> I would be willing to help for a complete rewrite of "erlexec" (as Tim, I'm more C than a C++ guy). > > Yes. In a long run improving open_port would be much better. > > The questions are - how to make it cross platform? Would OTP team > accept such patches? Would OTP team accept not fully cross platform > patch? > > Features which I feel are missing from oper_port: > - receiving stderr and stdout separately > - sending POSIX signals > - setting startup ENV You can do this with open port already. > - setting startup working directory You can do this with open port already. > - setting OS process priority > - monitoring of ports Yes the messages between the port and the controlling process could be richer and more robust. > > Anything else? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From msegalis@REDACTED Sun May 27 14:20:27 2012 From: msegalis@REDACTED (Morgan Segalis) Date: Sun, 27 May 2012 14:20:27 +0200 Subject: [erlang-questions] OpenSSL 1.0.X : SSL_MODE_RELEASE_BUFFERS Message-ID: <6B933F39-A381-4067-9483-714F971AB778@gmail.com> Hi everyone, Since a month or so, I'm looking for ways to reduce SSL memory footprint, in vain? - I have read something about SSL_MODE_RELEASE_BUFFERS which can help you save some memory for idles connections (around 45Ko). Since most of client connected are idle for a long time, I was thinking that would be a way to go. I'm wondering if Erlang R15B can work with openssl's version 1.0.X, and if so, how to active this option ? Thanks, Morgan. From watson.timothy@REDACTED Sun May 27 15:03:15 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sun, 27 May 2012 14:03:15 +0100 Subject: [erlang-questions] managing OS processes In-Reply-To: <7CE35CA8-C56B-43E3-8F97-B259166A327D@gmail.com> References: <20120526220048.GA66695@alumni.caltech.edu> <0373D27A-3AF4-438E-ADD0-F2365A989617@gmail.com> <7CE35CA8-C56B-43E3-8F97-B259166A327D@gmail.com> Message-ID: Guys - I think there is code in https://github.com/hawk/lux that addresses some of these issues..... On 27 May 2012 12:25, Tim Watson wrote: > > On 27 May 2012, at 10:44, Gleb Peregud wrote: > >> On Sun, May 27, 2012 at 12:48 AM, Zabrane Mickael wrote: >>> Quick look to "erlexec" code shows that's only support Linux for now (with dependency on libpcap). >>> Any revival work should extend "erlexec" to all other OSes when Erlang is expected to run (ex. Windows, OSX, FreeBSD ...) and try to minimize external dependencies. >>> >>> +1 >>> >>> I would be willing to help for a complete rewrite of "erlexec" (as Tim, I'm more C than a C++ guy). >> >> Yes. In a long run improving open_port would be much better. >> >> The questions are - how to make it cross platform? Would OTP team >> accept such patches? Would OTP team accept not fully cross platform >> patch? >> >> Features which I feel are missing from oper_port: >> - receiving stderr and stdout separately >> - sending POSIX signals >> - setting startup ENV > > You can do this with open port already. > >> - setting startup working directory > > You can do this with open port already. > >> - setting OS process priority >> - monitoring of ports > > Yes the messages between the port and the controlling process could be richer and more robust. > >> >> Anything else? >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From freeakk@REDACTED Sun May 27 16:28:52 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Sun, 27 May 2012 18:28:52 +0400 Subject: [erlang-questions] managing OS processes In-Reply-To: References: <20120526220048.GA66695@alumni.caltech.edu> <0373D27A-3AF4-438E-ADD0-F2365A989617@gmail.com> <7CE35CA8-C56B-43E3-8F97-B259166A327D@gmail.com> Message-ID: > - sending POSIX signals Is there an example of this? From antoine.koener@REDACTED Sun May 27 16:39:44 2012 From: antoine.koener@REDACTED (Antoine Koener) Date: Sun, 27 May 2012 16:39:44 +0200 Subject: [erlang-questions] OpenSSL 1.0.X : SSL_MODE_RELEASE_BUFFERS In-Reply-To: <6B933F39-A381-4067-9483-714F971AB778@gmail.com> References: <6B933F39-A381-4067-9483-714F971AB778@gmail.com> Message-ID: <1516726595418293131@unknownmsgid> Hi, Afaik OpenSSL is only used in ssl for the crypto module. So buffers may not be relates with your issue... -- Was on the go On 27 mai 2012, at 14:20, Morgan Segalis wrote: > Hi everyone, > > Since a month or so, I'm looking for ways to reduce SSL memory footprint, in vain? > > - I have read something about SSL_MODE_RELEASE_BUFFERS which can help you save some memory for idles connections (around 45Ko). > Since most of client connected are idle for a long time, I was thinking that would be a way to go. > > I'm wondering if Erlang R15B can work with openssl's version 1.0.X, and if so, how to active this option ? > > Thanks, > > Morgan. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From watson.timothy@REDACTED Sun May 27 18:25:10 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Sun, 27 May 2012 17:25:10 +0100 Subject: [erlang-questions] managing OS processes In-Reply-To: References: <20120526220048.GA66695@alumni.caltech.edu> <0373D27A-3AF4-438E-ADD0-F2365A989617@gmail.com> <7CE35CA8-C56B-43E3-8F97-B259166A327D@gmail.com> Message-ID: No you can't do this with the current open_port implementation - you can do os:cmd("kill -" ++ Signal ++ " " ++ OsPid) but that's about it. I think in the code for lux there may be some signal handling, and there appears to be support for it in erlexec. On 27 May 2012 15:28, Michael Uvarov wrote: >> - sending POSIX signals > Is there an example of this? From msegalis@REDACTED Sun May 27 19:49:52 2012 From: msegalis@REDACTED (Morgan Segalis) Date: Sun, 27 May 2012 19:49:52 +0200 Subject: [erlang-questions] OpenSSL 1.0.X : SSL_MODE_RELEASE_BUFFERS In-Reply-To: <1516726595418293131@unknownmsgid> References: <6B933F39-A381-4067-9483-714F971AB778@gmail.com> <1516726595418293131@unknownmsgid> Message-ID: <76BE3907-96F5-46B8-9630-1D7C21564811@gmail.com> Hi Antoine, I guess I have been mislead by ejabberd which can activate the option when available, but after investigating, it seems that they have done connection handling through a C-node using openssl. Then from where can come this "huge" memory footprint, is there a way in Erlang to check from where memory request are coming ? Because when I check with "process_info(spawn(Fun), memory)." Here's what I get : before gc: {memory,13376} after gc: {memory,2108} for each client? 2108 is that word size, or bits ? How can I get word size ? My system is 64bits, so word size is also ? Thanks, Morgan. Le 27 mai 2012 ? 16:39, Antoine Koener a ?crit : > Hi, Afaik > OpenSSL is only used in ssl for the crypto module. So buffers may not > be relates with your issue... > > -- > Was on the go > > On 27 mai 2012, at 14:20, Morgan Segalis wrote: > >> Hi everyone, >> >> Since a month or so, I'm looking for ways to reduce SSL memory footprint, in vain? >> >> - I have read something about SSL_MODE_RELEASE_BUFFERS which can help you save some memory for idles connections (around 45Ko). >> Since most of client connected are idle for a long time, I was thinking that would be a way to go. >> >> I'm wondering if Erlang R15B can work with openssl's version 1.0.X, and if so, how to active this option ? >> >> Thanks, >> >> Morgan. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From msegalis@REDACTED Sun May 27 23:16:39 2012 From: msegalis@REDACTED (Morgan Segalis) Date: Sun, 27 May 2012 23:16:39 +0200 Subject: [erlang-questions] OpenSSL 1.0.X : SSL_MODE_RELEASE_BUFFERS In-Reply-To: <76BE3907-96F5-46B8-9630-1D7C21564811@gmail.com> References: <6B933F39-A381-4067-9483-714F971AB778@gmail.com> <1516726595418293131@unknownmsgid> <76BE3907-96F5-46B8-9630-1D7C21564811@gmail.com> Message-ID: After checking out, "process_info(spawn(Fun), memory)." return the value in bits? so each process should take 2KB, this is where I do not understand where all this memory is taken? Le 27 mai 2012 ? 19:49, Morgan Segalis a ?crit : > Hi Antoine, > > I guess I have been mislead by ejabberd which can activate the option when available, but after investigating, it seems that they have done connection handling through a C-node using openssl. > > Then from where can come this "huge" memory footprint, is there a way in Erlang to check from where memory request are coming ? > Because when I check with "process_info(spawn(Fun), memory)." > > Here's what I get : > > before gc: {memory,13376} > after gc: {memory,2108} > > for each client? > 2108 is that word size, or bits ? How can I get word size ? My system is 64bits, so word size is also ? > > Thanks, > > Morgan. > > Le 27 mai 2012 ? 16:39, Antoine Koener a ?crit : > >> Hi, Afaik >> OpenSSL is only used in ssl for the crypto module. So buffers may not >> be relates with your issue... >> >> -- >> Was on the go >> >> On 27 mai 2012, at 14:20, Morgan Segalis wrote: >> >>> Hi everyone, >>> >>> Since a month or so, I'm looking for ways to reduce SSL memory footprint, in vain? >>> >>> - I have read something about SSL_MODE_RELEASE_BUFFERS which can help you save some memory for idles connections (around 45Ko). >>> Since most of client connected are idle for a long time, I was thinking that would be a way to go. >>> >>> I'm wondering if Erlang R15B can work with openssl's version 1.0.X, and if so, how to active this option ? >>> >>> Thanks, >>> >>> Morgan. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > From msegalis@REDACTED Sun May 27 23:43:17 2012 From: msegalis@REDACTED (Morgan Segalis) Date: Sun, 27 May 2012 23:43:17 +0200 Subject: [erlang-questions] OpenSSL 1.0.X : SSL_MODE_RELEASE_BUFFERS In-Reply-To: References: <6B933F39-A381-4067-9483-714F971AB778@gmail.com> <1516726595418293131@unknownmsgid> <76BE3907-96F5-46B8-9630-1D7C21564811@gmail.com> Message-ID: <4B51483E-D2AB-4A3C-BF18-A822D107E639@gmail.com> Hi Ingela, Indeed I'm already using the hibernate option? - Just to be sure, how much an ssl connection should take with R15 ? just to be sure, I have not something wrong? Because when I check with "process_info(spawn(Fun), memory)." Here's what I get : before gc: {memory,13376} after gc: {memory,2108} Is ssl socket (present in the state tuple) included ? - any ETA for the next version ? Thank you ! Le 27 mai 2012 ? 23:35, Ingela Andin a ?crit : > Hi! > > You can use the hibernate option to reduce memory usage for idle > connections, however this might not be enough, we are currently > investegating more ways to reduce the memory footprint. It is correct > that we only use openssl for crypto functionality so openssl options > will not help your case. Ejabberd made its own solution a long time > ago when the old ssl implemantation was the only available one. We > also have removed some bottleneck problems in the upcoming version of > the erlang ssl application. > > Regards Ingela Erlang/OTP team - Ericsson AB > > On Sun, May 27, 2012 at 7:49 PM, Morgan Segalis wrote: >> Hi Antoine, >> >> I guess I have been mislead by ejabberd which can activate the option when available, but after investigating, it seems that they have done connection handling through a C-node using openssl. >> >> Then from where can come this "huge" memory footprint, is there a way in Erlang to check from where memory request are coming ? >> Because when I check with "process_info(spawn(Fun), memory)." >> >> Here's what I get : >> >> before gc: {memory,13376} >> after gc: {memory,2108} >> >> for each client? >> 2108 is that word size, or bits ? How can I get word size ? My system is 64bits, so word size is also ? >> >> Thanks, >> >> Morgan. >> >> Le 27 mai 2012 ? 16:39, Antoine Koener a ?crit : >> >>> Hi, Afaik >>> OpenSSL is only used in ssl for the crypto module. So buffers may not >>> be relates with your issue... >>> >>> -- >>> Was on the go >>> >>> On 27 mai 2012, at 14:20, Morgan Segalis wrote: >>> >>>> Hi everyone, >>>> >>>> Since a month or so, I'm looking for ways to reduce SSL memory footprint, in vain? >>>> >>>> - I have read something about SSL_MODE_RELEASE_BUFFERS which can help you save some memory for idles connections (around 45Ko). >>>> Since most of client connected are idle for a long time, I was thinking that would be a way to go. >>>> >>>> I'm wondering if Erlang R15B can work with openssl's version 1.0.X, and if so, how to active this option ? >>>> >>>> Thanks, >>>> >>>> Morgan. >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Mon May 28 00:40:35 2012 From: ok@REDACTED (Richard O'Keefe) Date: Mon, 28 May 2012 10:40:35 +1200 Subject: [erlang-questions] Getting rid of the preprocessor In-Reply-To: References: <390E1F41-608F-4FE3-B4B4-3B7685BD0D58@cs.otago.ac.nz> Message-ID: <763B6A1D-911F-496C-B15D-2B145D2A20C3@cs.otago.ac.nz> On 25/05/2012, at 7:12 PM, Vlad Dumitrescu wrote: > My real question got buried in the discussion about my particular use > case: previously when the preprocessor and its demise were discussed, > one main reason for not being able to do anything about it at the > moment was that first we need to do something about the records. But > the preprocessor doesn't do anything to records, they are processed by > the compiler. To quote you from a recent thread: "Abstract patterns > and frames are all part of a long-time project to make the > preprocessor unnecessary". Could you please enlighten me as to how the > preprocessor is involved here? If two modules need to use the same record, the definition is put in a .hrl file and each module -includes that file. This inclusion is done by the preprocessor. I just did a quick survey of the .hrl files in a directory that (sigh) has two releases of Erlang/OTP in it. defines=Y defines=N | records=Y 212 160 | 372 records=N 485 26 | 511 ========= ========= | ========= Totals 697 186 | 883 (The .hrl files with neither a -record nor a -define either -include other files, contain -compile directives, or contain generated code including function definitions.) > >> I presume it goes something like this: >> if it looks like a variable, display it as one; >> if it looks like a function call, display it as one; >> if the display is wrong, it's the programmer's fault >> for defining such a stupid macro. > > The displaying of fancy colors in the editor is not the important > issue. An IDE needs to know as much detail as possible about the code, > to support navigation and refactorings, to be able to suggest > meaningful completions and offer reasonable ways to fix any issues it > finds. This isn't possible if one can't understand (=parse) the code > properly. I find myself working on code that isn't _finished_ yet most of the time, for which there _are_ as yet no facts of the matter on which an IDE could depend. If memory serves me, the way Quintus handled the Emacs interface was for Emacs to ask Prolog to do the parsing. I for one do not expect "meaningful completions" for macros. It's worth reflecting that refactoring IDEs started in Smalltalk, where the syntax is simple, there is no compile time/run time distinction, and above all, no preprocessor. > The parser grammar is basically describing this nice language, except > that any grammar construct or combination thereof may be represented > by a macro call... The difficulty is acknowledged. As is the fact that you can't stop people using a processor even if there is none in the language. I have Smalltalk code generated using m4(1) and awk(1). The Smalltalk+m4 source code has to be edited *somehow*. I think "Delenda est preprocessor" was started while I was at RMIT, perhaps in 1997. My copy has the date 20 July 1998 on it, which is after it was sent to SERC. A copy can be found at http://www.cs.otago.ac.nz/staffpriv/ok/delenda.txt It was an internal report to SERC never meant for external publication, so it is _not_ beautifully formatted. From jozsef.berces@REDACTED Mon May 28 06:46:19 2012 From: jozsef.berces@REDACTED (=?iso-8859-1?Q?J=F3zsef_B=E9rces?=) Date: Mon, 28 May 2012 06:46:19 +0200 Subject: [erlang-questions] What causes nodes to become disconnected/reconnected? In-Reply-To: <026d01cd3a7b$c80b3ae0$5821b0a0$@com> References: <020101cd39c9$9b3f0260$d1bd0720$@com> <3717CEF51B134A4F8073627839A9DF412744B3B02D@ESESSCMS0353.eemea.ericsson.se> <026d01cd3a7b$c80b3ae0$5821b0a0$@com> Message-ID: <3717CEF51B134A4F8073627839A9DF412744B3B149@ESESSCMS0353.eemea.ericsson.se> Is there a way to detect from Erlang that windows just woke up from sleeping? ________________________________ From: David Mercer [mailto:dmercer@REDACTED] Sent: Friday, May 25, 2012 20:39 To: J?zsef B?rces; erlang-questions@REDACTED Subject: RE: [erlang-questions] What causes nodes to become disconnected/reconnected? These might be related, though. You are right, my case does not occur due to the host going to sleep, but both my hosts are Windows, and perhaps something is happening - whether it be with the network, firewall, virus protection, or whatever - that is causing the nodes to get disconnected, and they do not properly reconnect when the temporary issue is resolved. It might be the same thing in the Windows port of Erlang that prevents them from reconnecting after sleep that is preventing my nodes from reconnecting after something else. Cheers, DBM From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of J?zsef B?rces Sent: Friday, May 25, 2012 4:30 AM To: erlang-questions@REDACTED Subject: Re: [erlang-questions] What causes nodes to become disconnected/reconnected? It is somewhat offtopic as I do not believe David experenced this, but I have similar disconnection when Windows goes to sleep. After the wake-up, the nodes do not see each other. Does anyone have any idea how to fix this? Thanks, Jozsef ________________________________ From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Martynas Pumputis Sent: Friday, May 25, 2012 16:24 To: David Mercer Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] What causes nodes to become disconnected/reconnected? Erlang doesn't detect net splits by itself. You could start looking at net_kernel:set_net_ticktime/2 (try to increase this value if your node is suffering from high load/traffic) and kernel/src/dist_utils.erl to get a grasp how erlang handles node connections. Martynas On Thu, May 24, 2012 at 6:23 PM, David Mercer > wrote: (Yes, I am still working on my issue with a distributed application that keeps losing its connection to the other node.) It is not a problem only when I have 2 nodes on the same host. I was running the nodes on different hosts last night, and this morning the failover node had lost its connection to the main and so had started its own instance of the application. Calling nodes() on the failover returned []. Then I started a new node on the same host as the main (to see if it would restore the connections), and, yes, it did. After starting the third node, nodes() now on the failover node returns a list of two nodes, the two on the main host. However, the application on the failover node did not shut down, and so it is still running on both the main and failover nodes. To summarize: 1. Distributed application running on a node on host A ("main@REDACTED"), failover on node on host B ("failover@REDACTED"). 2. At some point, failover@REDACTED becomes disconnected from main@REDACTED, and the application starts on failover@REDACTED Now there are two instances of the application running. 3. From a network point of view, there is still (or again) a valid network connection between hosts A and B. I can't say for sure if some network/firewall/other issue caused a temporary disconnect, but I can say that by the time I got in this morning, when the application was running on both nodes, there was a firm network connection between the two hosts. 4. Calling nodes() on failover@REDACTED returns []. 5. A new (failover) node was started on host A ("failover@REDACTED"). It does not start the application (which is correct, since it is already running, albeit on both other nodes instead of just one). 6. Calling nodes() on failover@REDACTED now returns [failover@REDACTED,main@REDACTED]. 7. The application, however, is still running on failover@REDACTED, despite the fact that nodes/0 reports a connection to main@REDACTED I don't need someone to diagnose this for me. If someone could just educate me a little on how the connections work, how net splits are detected and nodes disconnected, etc., I might be able to take it from there. Anyone know enough and have the time to type out a little blurb? I can read source code, but having a little background knowledge would help put it into context for me. Thank-you! Cheers, David _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Mon May 28 08:04:34 2012 From: ulf@REDACTED (Ulf Wiger) Date: Mon, 28 May 2012 08:04:34 +0200 Subject: [erlang-questions] What causes nodes to become disconnected/reconnected? In-Reply-To: <3717CEF51B134A4F8073627839A9DF412744B3B149@ESESSCMS0353.eemea.ericsson.se> References: <020101cd39c9$9b3f0260$d1bd0720$@com> <3717CEF51B134A4F8073627839A9DF412744B3B02D@ESESSCMS0353.eemea.ericsson.se> <026d01cd3a7b$c80b3ae0$5821b0a0$@com> <3717CEF51B134A4F8073627839A9DF412744B3B149@ESESSCMS0353.eemea.ericsson.se> Message-ID: <55A37268-4AF9-4609-B77E-B2B63392DC4B@feuerlabs.com> Not sure if the erlang:now() auto-correction is active on Windows, but if it is, you should be able to detect a large diff between erlang:now() and os:timestamp(). It could of course also mean that someone has tampered with the system clock, but even so, it's a good idea to restart the node then. BR, Ulf W Ulf Wiger, Feuerlabs, Inc. http://www.feuerlabs.com 28 maj 2012 kl. 06:46 skrev J?zsef B?rces : > Is there a way to detect from Erlang that windows just woke up from sleeping? > > From: David Mercer [mailto:dmercer@REDACTED] > Sent: Friday, May 25, 2012 20:39 > To: J?zsef B?rces; erlang-questions@REDACTED > Subject: RE: [erlang-questions] What causes nodes to become disconnected/reconnected? > > These might be related, though. You are right, my case does not occur due to the host going to sleep, but both my hosts are Windows, and perhaps something is happening ? whether it be with the network, firewall, virus protection, or whatever ? that is causing the nodes to get disconnected, and they do not properly reconnect when the temporary issue is resolved. > > It might be the same thing in the Windows port of Erlang that prevents them from reconnecting after sleep that is preventing my nodes from reconnecting after something else. > > Cheers, > > DBM > > From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of J?zsef B?rces > Sent: Friday, May 25, 2012 4:30 AM > To: erlang-questions@REDACTED > Subject: Re: [erlang-questions] What causes nodes to become disconnected/reconnected? > > It is somewhat offtopic as I do not believe David experenced this, but I have similar disconnection when Windows goes to sleep. After the wake-up, the nodes do not see each other. Does anyone have any idea how to fix this? > > Thanks, > Jozsef > > From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Martynas Pumputis > Sent: Friday, May 25, 2012 16:24 > To: David Mercer > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] What causes nodes to become disconnected/reconnected? > > Erlang doesn't detect net splits by itself. You could start looking at net_kernel:set_net_ticktime/2 (try to increase this value if your node is suffering from high load/traffic) and kernel/src/dist_utils.erl to get a grasp how erlang handles node connections. > > Martynas > > On Thu, May 24, 2012 at 6:23 PM, David Mercer wrote: > (Yes, I am still working on my issue with a distributed application that keeps losing its connection to the other node.) > > It is not a problem only when I have 2 nodes on the same host. I was running the nodes on different hosts last night, and this morning the failover node had lost its connection to the main and so had started its own instance of the application. Calling nodes() on the failover returned []. > > Then I started a new node on the same host as the main (to see if it would restore the connections), and, yes, it did. After starting the third node, nodes() now on the failover node returns a list of two nodes, the two on the main host. However, the application on the failover node did not shut down, and so it is still running on both the main and failover nodes. > > To summarize: > > 1. Distributed application running on a node on host A (?main@REDACTED?), failover on node on host B (?failover@REDACTED?). > > > 2. At some point, failover@REDACTED becomes disconnected from main@REDACTED, and the application starts on failover@REDACTED Now there are two instances of the application running. > > > > 3. From a network point of view, there is still (or again) a valid network connection between hosts A and B. I can?t say for sure if some network/firewall/other issue caused a temporary disconnect, but I can say that by the time I got in this morning, when the application was running on both nodes, there was a firm network connection between the two hosts. > > > > 4. Calling nodes() on failover@REDACTED returns []. > > > > 5. A new (failover) node was started on host A (?failover@REDACTED?). It does not start the application (which is correct, since it is already running, albeit on both other nodes instead of just one). > > > > 6. Calling nodes() on failover@REDACTED now returns [failover@REDACTED,main@REDACTED]. > > > > 7. The application, however, is still running on failover@REDACTED, despite the fact that nodes/0 reports a connection to main@REDACTED > > > I don?t need someone to diagnose this for me. If someone could just educate me a little on how the connections work, how net splits are detected and nodes disconnected, etc., I might be able to take it from there. Anyone know enough and have the time to type out a little blurb? I can read source code, but having a little background knowledge would help put it into context for me. > > Thank-you! > > Cheers, > > David > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Mon May 28 10:07:38 2012 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 28 May 2012 10:07:38 +0200 Subject: [erlang-questions] Getting rid of the preprocessor In-Reply-To: <763B6A1D-911F-496C-B15D-2B145D2A20C3@cs.otago.ac.nz> References: <390E1F41-608F-4FE3-B4B4-3B7685BD0D58@cs.otago.ac.nz> <763B6A1D-911F-496C-B15D-2B145D2A20C3@cs.otago.ac.nz> Message-ID: Thank you for the answer, Richard. On Mon, May 28, 2012 at 12:40 AM, Richard O'Keefe wrote: > > If two modules need to use the same record, > the definition is put in a .hrl file > and each module -includes that file. > This inclusion is done by the preprocessor. Yes. So if the preprocessor only does inclusions and ifdefs (which don't affect the structure of the code because they only work outside form definitions), then "clean macros" could be part of the language proper (with a suitable list of restrictions on what they could resolve to). I can even imagine that a macro that isn't clean (value can't be parsed standalone) may be still handled as today, during a period of transition. > I find myself working on code that isn't _finished_ yet most of the > time, for which there _are_ as yet no facts of the matter on which > an IDE could depend. Why not? Even if the current module is not finished, you can still know things about the other modules and about the entities in this module that can be parsed. An IDE can also help find problems before runtime, because it knows about the whole code base while the compiler only handles one module at a time. If I know I am not using weird meta-programming techniques and I refer to a "foo:bar/3" where foo is in my project and bar/3 doesn't exist, I'd like to get a warning about that. The tests will probably catch that, but this is faster. > If memory serves me, the way Quintus handled > the Emacs interface was for Emacs to ask Prolog to do the parsing. That's what we do today, we use the Erlang scanner and parser. Problem is that they know nothing about macros, so we need to do manual magic. There are problems with having to talk to an Erlang node all the time like this (loss of synchronization, etc), so we are now moving to a Java-based parser. The disadvantage of that one is that we can't do any magic, so we have to be able to parse as much as possible. > I for one do not expect "meaningful completions" for macros. For example, "?DE" will complete to "?DEBUG" or give a list with "?DEBUG" and "?DELETE". >> The parser grammar is basically describing this nice language, except >> that any grammar construct or combination thereof may be represented >> by a macro call... > > The difficulty is acknowledged. ?As is the fact that you can't stop > people using a processor even if there is none in the language. ?I > have Smalltalk code generated using m4(1) and awk(1). ?The Smalltalk+m4 > source code has to be edited *somehow*. Yes, but if they do that, they habe basically a new language and are on their own, just like if they are using LFE, Elixir or Reia. I am talking about the base language. best regards, Vlad From freeakk@REDACTED Mon May 28 11:54:13 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Mon, 28 May 2012 13:54:13 +0400 Subject: [erlang-questions] managing OS processes In-Reply-To: References: <20120526220048.GA66695@alumni.caltech.edu> <0373D27A-3AF4-438E-ADD0-F2365A989617@gmail.com> <7CE35CA8-C56B-43E3-8F97-B259166A327D@gmail.com> Message-ID: > os:cmd("kill -" ++ Signal ++ " " ++ OsPid) It is platform-dependent. And there is no way to convert Port into OsPid. Only way is to "ask" the OS program using stdio. "with the current open_port implementation" it is easy to call PHP or python to control processes :) -- Best regards, Uvarov Michael From watson.timothy@REDACTED Mon May 28 11:56:31 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 28 May 2012 10:56:31 +0100 Subject: [erlang-questions] managing OS processes In-Reply-To: References: <20120526220048.GA66695@alumni.caltech.edu> <0373D27A-3AF4-438E-ADD0-F2365A989617@gmail.com> <7CE35CA8-C56B-43E3-8F97-B259166A327D@gmail.com> Message-ID: On 28 May 2012, at 10:54, Michael Uvarov wrote: >> os:cmd("kill -" ++ Signal ++ " " ++ OsPid) > It is platform-dependent. > And there is no way to convert Port into OsPid. Only way is to "ask" > the OS program using stdio. > "with the current open_port implementation" it is easy to call PHP or > python to control processes :) > Yes absolutely - nobody is saying this is correct. We *clearly* need a proper way to handle sending signals. BTW someone mentioned off-list that the name erlexec clashes with a built in executable. Perhaps as well as forking the project, it would be a good idea to rename it..... > -- > Best regards, > Uvarov Michael From jose.valim@REDACTED Mon May 28 13:01:05 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Mon, 28 May 2012 13:01:05 +0200 Subject: [erlang-questions] [ANN] Elixir v0.5.0 released In-Reply-To: References: Message-ID: Hello everyone, Some people on other forums found Elixir's moto, "Modern approach to programming for the Erlang VM", a bit offensive. The moto is meant to refer to the support for macros & meta-programming, primary syntax for associative data structures, protocols and other features. There are many discussions on this mailing list about how to bring some of these features to Erlang, like erl2 posted by Joe Armstrong, the frames proposal, improving parse transformations and so forth. It was never meant to denigrate Erlang at all, since Erlang assets are Elixir assets as well. That said, if someone was offended by the moto, I apologize. If those working directly on Erlang and OTP are not pleased by it, please let me know and I will do my best to come up with something different. Have a great week! * Jos? Valim www.plataformatec.com.br Founder and Lead Developer * On Fri, May 25, 2012 at 5:57 PM, Jos? Valim wrote: > Hello everyone, > > Today we have officially released Elixir. > > Elixir is a programming language built on top of the Erlang. It is > homoiconic, allowing meta-programming via macros, it also supports > polymorphism via protocols (similar to Clojure's), dynamic records and a > module reference mechanism. > > Elixir aims to stay compatible with Erlang, so it shares the same bytecode > and data types. This means you can invoke Erlang code from Elixir (and > vice-versa) without any conversion or performance hit. This allows a > developer to mix the expressiveness of Elixir with the robustness and > performance of Erlang. > > The getting started guide is here: > > http://elixir-lang.org/getting_started/1.html > > We also have a fast crash course for Erlang developers: > > http://elixir-lang.org/crash-course.html > > If anyone is interested in the release background, here is the > announcement: > > http://elixir-lang.org/blog/2012/05/25/elixir-v0-5-0-released/ > > If anyone would like to discuss Elixir or give feedback, feel free to join > #elixir-lang on irc.freenode.com > > Thanks! > > * > Jos? Valim > www.plataformatec.com.br > Founder and Lead Developer > * > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Mon May 28 16:13:47 2012 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Mon, 28 May 2012 09:13:47 -0500 Subject: [erlang-questions] managing OS processes In-Reply-To: <20120526220048.GA66695@alumni.caltech.edu> References: <20120526220048.GA66695@alumni.caltech.edu> Message-ID: There is at least one portable C++ process management library out there that could be leveraged... http://www.highscore.de/boost/process/process/introduction.html ... and nifpp might be a good way to talk to it (https://github.com/goertzenator/nifpp) This might be a quick and dirty way to get it done, but this probably not what you want for something that is shooting for inclusion into OTP. The library source should at least provide some good implementation guidance. Dan. On Sat, May 26, 2012 at 5:00 PM, Anthony Molinaro wrote: > I've found a need for this sort of thing as well. ?+1 for getting something > into OTP itself. ? I'm sure the real issue will be with Windows, although > I believe there was some code in couchdb for doing some control of windows > processes, and maybe erlexec already supports Windows? ?Anyway, I can't > say I'd do much other than testing, but just wanted to voice my support. > > -Anthony > > On Sat, May 26, 2012 at 09:52:32PM +0100, Tim Watson wrote: >> I'm also working on very similar requirements in >> https://github.com/nebularis/systest/blob/master/src/systest_cli.erl >> and like you two, I have to rely on a combination of open_port with >> its constituent events and os:cmd/1 in order to SIGHUP/SIGKILL which >> feels rather 'hackish' to me. I've also noticed that it makes consumer >> code rather timing dependent, which is frustrating. >> >> I would be willing to participate in either >> >> (a) improving the built in port functionality or >> (b) working on erlexec >> >> Although C I'm fairly comfortable with, but C++ (which erlexec appears >> to be written in) isn't really my bag. I do have to ask though, is >> erlexec in need of much additional work? I also have 'client' modules >> for the OTP slave and ssh modules and don't want to invest a mountain >> of time in replacing open_port, but a little TLC to bring erlexec up >> to scratch would be time well spent if it would give a more consistent >> interface. In particular I have some issues with open_port based code >> with now such as >> >> 1. cli clients my code launches can be 'detached' (usually erlang >> nodes run with -detached or scripts that launch erlang nodes using `sh >> -C` or setsid/nohup and the like), which makes the port handling code >> both complex and messy. >> 2. some client code I'm testing will execute another (different) >> script/command to shut down the node and the listening port doesn't >> always get the exit status properly >> 3. signal handling is a mess >> >> So erlexec does sound promising, but IMO it would be *better* if the >> built in open_port functionality could be extending to deal with these >> situations properly. >> >> Cheers, >> Tim >> >> On 26 May 2012 13:12, Gleb Peregud wrote: >> > On Sat, May 26, 2012 at 11:06 AM, Benoit Chesneau wrote: >> >> I'm trying to find a way to manage some os processes launched from >> >> Erlang. For example I need to relaunch them when the external process >> >> die for an unknown reason or send an HUP signal or just stop the >> >> external process. >> > >> > Hello Benoit >> > >> > Looks like there are two of us, who needs a better mechanism to manage >> > OS processes from Erlang. I need it for a CI server which I'm working >> > on, so my requirements are not as high as yours, but still os:cmd >> > doesn't work for me and ports are missing one feature which I need. >> > >> > Maybe we could reanimate erlexec together? >> > >> > Cheers, >> > Gleb >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -- > ------------------------------------------------------------------------ > Anthony Molinaro ? ? ? ? ? ? ? ? ? ? ? ? ? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From thomasl_erlang@REDACTED Mon May 28 17:25:54 2012 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Mon, 28 May 2012 08:25:54 -0700 (PDT) Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: <7E452DB1-8AB8-44C6-8CC9-5DA99EF8AB22@cs.otago.ac.nz> References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> <7E452DB1-8AB8-44C6-8CC9-5DA99EF8AB22@cs.otago.ac.nz> Message-ID: <1338218754.27626.YahooMailNeo@web111409.mail.gq1.yahoo.com> ----- Original Message ----- > From: Richard O'Keefe > To: Thomas Lindgren > Cc: Michael Turner ; "erlang-questions@REDACTED" > Sent: Thursday, May 10, 2012 8:10 AM > Subject: Re: [erlang-questions] Is there a good source for documentation on BEAM? > >( 1) We were told that BEAM documentation isn't needed because > ? ? there are other Erlang implementations. > (2) I ask whether any of those other implementations ever kept > ? ? up with The Real Thing.? (By the way, as far as I know, > ? ? none of them ever supported bit syntax, and my recent > ? ? attempt to install GERL failed miserably.) > (3) Suddenly we are told that the abandoning of those other > ? ? things just *proves* that we don't need BEAM documentation. > ? It looks like this discussion has terminated, and I think I've made whatever points I wanted to be made so I'll leave it at that. But since this seemed to be unclear, let me recapitulate: BEAM docs are not needed to produce a second source implementation, as shown by several examples. (1)? Also, there has so far been little practical interest shown by Erlang users in such a second source. So implementation efforts may be in vain. (3)? My personal view, at least, is that most of the difficulty in "keeping up with The Real Thing", Erlang/OTP, would be not in reproducing BEAM but in writing a fully compatible implementation tracking the rest of the runtime, ERTS. (2) So, is there a _practical_ case for doing these docs? In particular, will the effort result in useful external contributions that outweigh time spent? Not at all clear to me.? Hope it helps. Best regards, Thomas From watson.timothy@REDACTED Mon May 28 18:55:46 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 28 May 2012 17:55:46 +0100 Subject: [erlang-questions] managing OS processes In-Reply-To: References: <20120526220048.GA66695@alumni.caltech.edu> Message-ID: <70855800-F464-46C5-A57E-47B16FFD2B8D@gmail.com> On 28 May 2012, at 15:13, Daniel Goertzen wrote: > There is at least one portable C++ process management library out > there that could be leveraged... > > http://www.highscore.de/boost/process/process/introduction.html > > ... and nifpp might be a good way to talk to it > (https://github.com/goertzenator/nifpp) > > What I'd really like is native processes, one per externally managed pid, wrapped around a native sub process management api for each supported platform. Monitoring becomes simple that way, as once the native process dies you put the exit code into the 'EXIT' message unless it is zero. Signal handling should be do-able too and I/O will be more erlang-ish. > This might be a quick and dirty way to get it done, but this probably > not what you want for something that is shooting for inclusion into > OTP. The library source should at least provide some good > implementation guidance. > > Dan. > > > On Sat, May 26, 2012 at 5:00 PM, Anthony Molinaro > wrote: >> I've found a need for this sort of thing as well. +1 for getting something >> into OTP itself. I'm sure the real issue will be with Windows, although >> I believe there was some code in couchdb for doing some control of windows >> processes, and maybe erlexec already supports Windows? Anyway, I can't >> say I'd do much other than testing, but just wanted to voice my support. >> >> -Anthony >> >> On Sat, May 26, 2012 at 09:52:32PM +0100, Tim Watson wrote: >>> I'm also working on very similar requirements in >>> https://github.com/nebularis/systest/blob/master/src/systest_cli.erl >>> and like you two, I have to rely on a combination of open_port with >>> its constituent events and os:cmd/1 in order to SIGHUP/SIGKILL which >>> feels rather 'hackish' to me. I've also noticed that it makes consumer >>> code rather timing dependent, which is frustrating. >>> >>> I would be willing to participate in either >>> >>> (a) improving the built in port functionality or >>> (b) working on erlexec >>> >>> Although C I'm fairly comfortable with, but C++ (which erlexec appears >>> to be written in) isn't really my bag. I do have to ask though, is >>> erlexec in need of much additional work? I also have 'client' modules >>> for the OTP slave and ssh modules and don't want to invest a mountain >>> of time in replacing open_port, but a little TLC to bring erlexec up >>> to scratch would be time well spent if it would give a more consistent >>> interface. In particular I have some issues with open_port based code >>> with now such as >>> >>> 1. cli clients my code launches can be 'detached' (usually erlang >>> nodes run with -detached or scripts that launch erlang nodes using `sh >>> -C` or setsid/nohup and the like), which makes the port handling code >>> both complex and messy. >>> 2. some client code I'm testing will execute another (different) >>> script/command to shut down the node and the listening port doesn't >>> always get the exit status properly >>> 3. signal handling is a mess >>> >>> So erlexec does sound promising, but IMO it would be *better* if the >>> built in open_port functionality could be extending to deal with these >>> situations properly. >>> >>> Cheers, >>> Tim >>> >>> On 26 May 2012 13:12, Gleb Peregud wrote: >>>> On Sat, May 26, 2012 at 11:06 AM, Benoit Chesneau wrote: >>>>> I'm trying to find a way to manage some os processes launched from >>>>> Erlang. For example I need to relaunch them when the external process >>>>> die for an unknown reason or send an HUP signal or just stop the >>>>> external process. >>>> >>>> Hello Benoit >>>> >>>> Looks like there are two of us, who needs a better mechanism to manage >>>> OS processes from Erlang. I need it for a CI server which I'm working >>>> on, so my requirements are not as high as yours, but still os:cmd >>>> doesn't work for me and ports are missing one feature which I need. >>>> >>>> Maybe we could reanimate erlexec together? >>>> >>>> Cheers, >>>> Gleb >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> -- >> ------------------------------------------------------------------------ >> Anthony Molinaro >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From msegalis@REDACTED Mon May 28 18:58:18 2012 From: msegalis@REDACTED (Morgan Segalis) Date: Mon, 28 May 2012 18:58:18 +0200 Subject: [erlang-questions] does Erlang on 32Bit system can allocate more than 4Go ? Message-ID: <453CD325-66AB-47F4-9222-CA2C4C315F6F@gmail.com> I'm not really sure how Erlang does manage memory on a 32bit system. I have a lot of process, each does take something like 2KB. Will it hit a limit when it will get close to 4GB ? Thank, Morgan. From freeakk@REDACTED Mon May 28 19:26:15 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Mon, 28 May 2012 21:26:15 +0400 Subject: [erlang-questions] managing OS processes In-Reply-To: <70855800-F464-46C5-A57E-47B16FFD2B8D@gmail.com> References: <20120526220048.GA66695@alumni.caltech.edu> <70855800-F464-46C5-A57E-47B16FFD2B8D@gmail.com> Message-ID: Maybe I will write a wrapper around Boost.process as a driver. From freeakk@REDACTED Mon May 28 19:36:29 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Mon, 28 May 2012 21:36:29 +0400 Subject: [erlang-questions] [ANN] Erlang Bindings for Xapian Message-ID: I am writing bindings for Xapian as a part of the google summer of code project. Xapian is a search engine library under GPL. Code is here: https://github.com/freeakk/xapian -- Best regards, Uvarov Michael From zabrane3@REDACTED Mon May 28 20:04:06 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Mon, 28 May 2012 20:04:06 +0200 Subject: [erlang-questions] [ANN] Erlang Bindings for Xapian In-Reply-To: References: Message-ID: <375161C0-5CE4-4EEF-95D7-47F46155D2F5@gmail.com> Fantastic Michael. What's the status of the project? Are you sure you want to stick with that name? Why not "xapien", "exapian" ... Regards, Zabrane On May 28, 2012, at 7:36 PM, Michael Uvarov wrote: > I am writing bindings for Xapian as a part of the google summer of code project. > Xapian is a search engine library under GPL. > > Code is here: > https://github.com/freeakk/xapian > > -- > Best regards, > Uvarov Michael > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From freeakk@REDACTED Mon May 28 20:18:04 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Mon, 28 May 2012 22:18:04 +0400 Subject: [erlang-questions] [ANN] Erlang Bindings for Xapian In-Reply-To: <375161C0-5CE4-4EEF-95D7-47F46155D2F5@gmail.com> References: <375161C0-5CE4-4EEF-95D7-47F46155D2F5@gmail.com> Message-ID: On Mon, May 28, 2012 at 10:04 PM, Zabrane Mickael wrote: > Fantastic Michael. > > What's the status of the project? It is in the active development, but many features are already here. > Are you sure you want to stick with that name? Why not "xapien", "exapian" ... Xapian has bindings for many languages and there is a conversation just to use the same name, if it is free (some languages have few different bindings). From attila.r.nohl@REDACTED Mon May 28 20:22:28 2012 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Mon, 28 May 2012 20:22:28 +0200 Subject: [erlang-questions] does Erlang on 32Bit system can allocate more than 4Go ? In-Reply-To: <453CD325-66AB-47F4-9222-CA2C4C315F6F@gmail.com> References: <453CD325-66AB-47F4-9222-CA2C4C315F6F@gmail.com> Message-ID: 2012/5/28 Morgan Segalis : > I'm not really sure how Erlang does manage memory on a 32bit system. > I have a lot of process, each does take something like 2KB. > > Will it hit a limit when it will get close to 4GB ? Yes, the VM will crash. In my experience the VM usually crashes when only around 2.5 GB of memory is allocated and the garbage collector tries to allocate a further 1.5 GB in one step. From ok@REDACTED Tue May 29 00:08:45 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 29 May 2012 10:08:45 +1200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: <1338218754.27626.YahooMailNeo@web111409.mail.gq1.yahoo.com> References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> <7E452DB1-8AB8-44C6-8CC9-5DA99EF8AB22@cs.otago.ac.nz> <1338218754.27626.YahooMailNeo@web111409.mail.gq1.yahoo.com> Message-ID: On 29/05/2012, at 3:25 AM, Thomas Lindgren wrote: > BEAM docs are not needed to produce a second source implementation, as shown by several examples. (1) People were not asking for BEAM documentation in order to produce a second source implementation. That's a red herring. People ask for BEAM documentation in order to understand and perhaps revise the main implementation. One thing that Open Source is about is about community contributions and it is awfully hard to contribute to something that is under-documented. > > Also, there has so far been little practical interest shown by Erlang users in such a second source. So implementation efforts may be in vain. (3) Again, the red herring. This is *NOT* about second implementations. All of the additional implementations I've ever heard of used their own back end, *not* BEAM, and would *not* have received any great benefit from BEAM being documented. There is detailed documentation available for Lua and Icon and the WAM and several other VMs around, so there's not even any great advantage in learning about VMs from BEAM. > My personal view, at least, is that most of the difficulty in "keeping up with The Real Thing", Erlang/OTP, would be not in reproducing BEAM but in writing a fully compatible implementation tracking the rest of the runtime, ERTS. (2) Well, not _just_ that, but that's a big part of it. > So, is there a _practical_ case for doing these docs? In particular, will the effort result in useful external contributions that outweigh time spent? Not at all clear to me. Bearing in mind that "BEAM documentation" and "second system implementation" are about as irrelevant to each other as any two topics about the same language can be, what _might_ we realistically expect from BEAM documentation? (1) There are now several languages implemented on top of Erlang. Some are interpreted in Erlang, some are compiled to Erlang ASTs, and some are compiled to BEAM. Compiling to the BEAM is *much* harder than it needs to be because of the current undocumentation. This isn't *replacing* Erlang/OTP but *augmenting* it. (2) I've proposed a number of minor but handy extensions to Erlang syntax that lack a reference implementation because I have no idea what the compiler should generate for them; these could use the existing BEAM. (3) There are existing things that could be compiled more efficiently. I'm thinking here in particular of list comprehension. I'm not sure if the improved translation can be done with the existing BEAM or if minor extensions would be needed, because the undocumentation for the BEAM does not make clear any range limits or other restrictions on BEAM instructions. (4) There are even bigger changes, like frames, where even estimating the scope of the changes is hard because of the undocumentation. But above all you are making an assumption which I utterly reject, namely that documentation is a COST and ONLY a cost, that producing documentation provides no DIRECT benefits to the people writing the documentation. On the contrary, - you may find defects as you document - you may be able to structure the documentation so that test cases can automatically be extracted - you may in the very act of explaining a limitation see how it can be overcome - you may be able to extract parts of the implementation automatically from the documentation and I could go on. There's a slogan I learned from a business textbook: find the indispensable man and fire him! There was something I found unsettling: we were told in this thread that there wasn't any need for documentation because the Erlang/OTP maintainers had enough people who _knew_ this stuff. One of my colleagues here was managing a software project once; a key employee went on holiday and was murdered in a far country. A former student of mine was starting up a new company with me as a consultant, and being a worse driver than he thought, drove at speed into a tree. The tree survived. One of my former colleagues, a very intelligent and likable guy, was cycling down a Melbourne street and got knocked over by a hit-and-run driver. The result was head injury and someone who could dress himself but couldn't program if his life depended on it. When I was at Quintus, the founder who wrote the compiler and was the only person who really understood it quit and was never heard from again. Fergus O'Brien supervised an MSc on "Organisational Forgetting" and the fact of organisational forgetting has haunted the fringes of my mind ever since. People die. People get head injuries. People quit. Documents get lost. (If you ever find an architecture- of-the-ICL-2900 manual, I'd like to see it.) Even if things _are_ written down, organisations forget _where_. If they aren't written down, they WILL be forgotten sooner or later. People Well, my colleague Andrew Trotman once had a key employee We > > Hope it helps. > > > Best regards, > Thomas > From ok@REDACTED Tue May 29 00:09:39 2012 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 29 May 2012 10:09:39 +1200 Subject: [erlang-questions] managing OS processes In-Reply-To: <70855800-F464-46C5-A57E-47B16FFD2B8D@gmail.com> References: <20120526220048.GA66695@alumni.caltech.edu> <70855800-F464-46C5-A57E-47B16FFD2B8D@gmail.com> Message-ID: On 29/05/2012, at 4:55 AM, Tim Watson wrote: > > What I'd really like is native processes, one per externally managed pid, wrapped around a native sub process management api for each supported platform. Monitoring becomes simple that way, as once the native process dies you put the exit code into the 'EXIT' message unless it is zero. Signal handling should be do-able too and I/O will be more erlang-ish. +1 From watson.timothy@REDACTED Tue May 29 00:11:19 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 28 May 2012 23:11:19 +0100 Subject: [erlang-questions] more thoughts about package/dependency management Message-ID: <4FC3F807.3040701@gmail.com> http://hyperthunk.wordpress.com/2012/05/28/does-erlangotp-need-a-new-package-management-solution/ This is a summary of the Erlware-Questions discussions - hopefully I've been true to what was said on the list, but if I've misrepresented anyone's opinion then I apologise and hope that you'll put it down to my 'special' short/medium term memory. :) Comments/Feedback welcome. From ciprian.craciun@REDACTED Tue May 29 11:21:51 2012 From: ciprian.craciun@REDACTED (Ciprian Dorin Craciun) Date: Tue, 29 May 2012 12:21:51 +0300 Subject: [erlang-questions] managing OS processes In-Reply-To: References: Message-ID: On Sat, May 26, 2012 at 3:12 PM, Gleb Peregud wrote: > On Sat, May 26, 2012 at 11:06 AM, Benoit Chesneau wrote: >> I'm trying to find a way to manage some os processes launched from >> Erlang. For example I need to relaunch them when the external process >> die for an unknown reason or send an HUP signal or just stop the >> external process. > > Hello Benoit > > Looks like there are two of us, who needs a better mechanism to manage > OS processes from Erlang. I need it for a CI server which I'm working > on, so my requirements are not as high as yours, but still os:cmd > doesn't work for me and ports are missing one feature which I need. > > Maybe we could reanimate erlexec together? > > Cheers, > Gleb Hello all! I don't think its the right way (although it seems to be suggested by the Erlang manual or somewhere else I've read it), but the way I did it was to write a small C program that is the one "babysitting" the real process, and a small Erlang wrapper. The code is here (Apache 2.0 licensed): https://github.com/cipriancraciun/mosaic-node/tree/development/applications/mosaic-harness/sources https://github.com/cipriancraciun/mosaic-node/blob/development/applications/mosaic-harness/sources/mosaic_harness_backend.erl https://github.com/cipriancraciun/mosaic-node/blob/development/applications/mosaic-harness/sources/mosaic_harness.c The C program allows the following: * it uses a small protocol based on JSON to communicate with the Erlang wrapper, using solely `stdin` / `stdout` between itself and Erlang, and between itself and the "babysitted" program; * it allows a single process to be executed at a time (controlling environment, arguments, and working directory); (it could be extended to multiple process in parallel if wanted;) (but you can run it again if you want;) * it allows a signal to be sent; * it allows "exchange" messages to be sent to and from Erlang and the process; * it allows brutal termination; (i.e. SIGKILL;) * if `stdin` is closed by the Erlang side it kills the process; * if `stdin` is closed by the process side it informs the Erlang side; The C part is a self-contained `.c` file (of about 2k lines, of which 25% are structure definitions), written in an event driven manner (no threads), statically linkable, depending only on the `jansson` library (for JSON), using mostly POSIX syscalls. I've successfully used it for about a year now (in non-production systems) and it behaves nicely. If someone finds it useful I could try to extract it in a standalone project. Ciprian. From rtrlists@REDACTED Tue May 29 13:13:16 2012 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 29 May 2012 12:13:16 +0100 Subject: [erlang-questions] managing OS processes In-Reply-To: References: Message-ID: On Tue, May 29, 2012 at 10:21 AM, Ciprian Dorin Craciun < ciprian.craciun@REDACTED> wrote: > Hello all! > > I don't think its the right way (although it seems to be suggested > by the Erlang manual or somewhere else I've read it), but the way I > did it was to write a small C program that is the one "babysitting" > the real process, and a small Erlang wrapper. > > The code is here (Apache 2.0 licensed): > > https://github.com/cipriancraciun/mosaic-node/tree/development/applications/mosaic-harness/sources > > https://github.com/cipriancraciun/mosaic-node/blob/development/applications/mosaic-harness/sources/mosaic_harness_backend.erl > > https://github.com/cipriancraciun/mosaic-node/blob/development/applications/mosaic-harness/sources/mosaic_harness.c > > The C program allows the following: > * it uses a small protocol based on JSON to communicate with the > Erlang wrapper, using solely `stdin` / `stdout` between itself and > Erlang, and between itself and the "babysitted" program; > * it allows a single process to be executed at a time (controlling > environment, arguments, and working directory); (it could be extended > to multiple process in parallel if wanted;) (but you can run it again > if you want;) > * it allows a signal to be sent; > * it allows "exchange" messages to be sent to and from Erlang and > the process; > * it allows brutal termination; (i.e. SIGKILL;) > * if `stdin` is closed by the Erlang side it kills the process; > * if `stdin` is closed by the process side it informs the Erlang side; > > The C part is a self-contained `.c` file (of about 2k lines, of > which 25% are structure definitions), written in an event driven > manner (no threads), statically linkable, depending only on the > `jansson` library (for JSON), using mostly POSIX syscalls. > > I've successfully used it for about a year now (in non-production > systems) and it behaves nicely. > > If someone finds it useful I could try to extract it in a > standalone project. > > Ciprian. > > I have a similar "proxy" to run killable external programs on Windows. It's part of a larger project though, so I'd need to extract it. Would there be interest for me to do that? The approach is: Start the proxy program using open_port/2 and send it the command to run, the proxy then uses DuplicateHandle() and CreateProcessW() to kick off the command in a separate OS process but with stdin/stdout/stderr forwarded from the proxy to the new external command, prints the OS process id back to Erlang and waits for the external OS process to finish. Thus you now have your command running in the backgroud, able to communicate using stdio with the Erlang process that started the proxy. If you want to stop the command, you invoke the proxy with the process id you got earlier and it kills it using TerminateProcess(). Robby PS I vaguely remember getting the idea of passing back a process id for later kill purposes from some open source app that uses a similar approach to running killable commands from Erlang. Not that I can remember which one at the mooment, sorry. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Tue May 29 13:40:07 2012 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 29 May 2012 13:40:07 +0200 Subject: [erlang-questions] managing OS processes In-Reply-To: References: Message-ID: > I have a similar "proxy" to run killable external programs on Windows. It's part of a larger project though, so I'd need to extract it. Would there be interest for me to do that? Yep, could you share that code please? Regards, Zabrane From watson.timothy@REDACTED Tue May 29 14:38:56 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 29 May 2012 13:38:56 +0100 Subject: [erlang-questions] managing OS processes In-Reply-To: References: Message-ID: Guys, This stuff is really good. But fundamentally, we're still in the world where the limitations of open_port are what dominates. Dave Smith's sh modules/functions for both rebar and retest handle this kind of proxy approach just fine, simply by using `sh -C 'echo $$; exec ' ++ Cmd` and using port io. Admittedly it doesn't support signal handling as cleanly. I hear (on twitter) that there are improvements to port communication coming in R16, and I dearly hope that native processes will be with us by then. At that time, these issues *might* just go away, although they might not. The other problem I run into is, what about daemon/detached scripts, things launched using setsid/nohup and/or put straight into the background? And how is your proxy program going to exhibit characteristics any different in this situation, where you cannot 'spawn' a sub-process (or whatever the windows equivalent is). Anyway, despite my reticence, I think your programs both sounds pretty useful - although I'm bemused that you'd use JSON serialization instead of simply using the ei interface - and I would also suggest that packaging them as separate libraries would be very useful to the community, giving others the benefit of utilising your work! Cheers, Tim On 29 May 2012, at 12:13, Robert Raschke wrote: > > On Tue, May 29, 2012 at 10:21 AM, Ciprian Dorin Craciun wrote: > Hello all! > > I don't think its the right way (although it seems to be suggested > by the Erlang manual or somewhere else I've read it), but the way I > did it was to write a small C program that is the one "babysitting" > the real process, and a small Erlang wrapper. > > The code is here (Apache 2.0 licensed): > https://github.com/cipriancraciun/mosaic-node/tree/development/applications/mosaic-harness/sources > https://github.com/cipriancraciun/mosaic-node/blob/development/applications/mosaic-harness/sources/mosaic_harness_backend.erl > https://github.com/cipriancraciun/mosaic-node/blob/development/applications/mosaic-harness/sources/mosaic_harness.c > > The C program allows the following: > * it uses a small protocol based on JSON to communicate with the > Erlang wrapper, using solely `stdin` / `stdout` between itself and > Erlang, and between itself and the "babysitted" program; > * it allows a single process to be executed at a time (controlling > environment, arguments, and working directory); (it could be extended > to multiple process in parallel if wanted;) (but you can run it again > if you want;) > * it allows a signal to be sent; > * it allows "exchange" messages to be sent to and from Erlang and > the process; > * it allows brutal termination; (i.e. SIGKILL;) > * if `stdin` is closed by the Erlang side it kills the process; > * if `stdin` is closed by the process side it informs the Erlang side; > > The C part is a self-contained `.c` file (of about 2k lines, of > which 25% are structure definitions), written in an event driven > manner (no threads), statically linkable, depending only on the > `jansson` library (for JSON), using mostly POSIX syscalls. > > I've successfully used it for about a year now (in non-production > systems) and it behaves nicely. > > If someone finds it useful I could try to extract it in a > standalone project. > > Ciprian. > > > I have a similar "proxy" to run killable external programs on Windows. It's part of a larger project though, so I'd need to extract it. Would there be interest for me to do that? > > The approach is: > > Start the proxy program using open_port/2 and send it the command to run, the proxy then uses DuplicateHandle() and CreateProcessW() to kick off the command in a separate OS process but with stdin/stdout/stderr forwarded from the proxy to the new external command, prints the OS process id back to Erlang and waits for the external OS process to finish. > > Thus you now have your command running in the backgroud, able to communicate using stdio with the Erlang process that started the proxy. > > If you want to stop the command, you invoke the proxy with the process id you got earlier and it kills it using TerminateProcess(). > > Robby > > PS I vaguely remember getting the idea of passing back a process id for later kill purposes from some open source app that uses a similar approach to running killable commands from Erlang. Not that I can remember which one at the mooment, sorry. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From avinash@REDACTED Tue May 29 16:12:08 2012 From: avinash@REDACTED (Avinash Dhumane) Date: Tue, 29 May 2012 19:42:08 +0530 Subject: [erlang-questions] HttpC options Message-ID: I am not able to sense the difference between "max_keep_alive_length" and "max_pipeline_length" options ("httpc" module) in pipelining scenario (turned on by setting pipeline_timeout to timer:seconds(60)). Documentation says: max_keep_alive_length = maximum number of outstanding requests on the _same_ connection to a host max_pipeline_length = maximum number of outstanding requests on a _pipelined_ connection to a host Questions are: 1) Can both these options co-exist in pipelining scenario? If so, what aspects of requests submissions would they affect? 2) In "without pipelining" scenario, what is the meaning of keeping more than one outstanding request on the _same_ connection, i.e. max_keep_alive_length option? Can it really be done? If so, how different it is from pipelining? I am quite confused! :-) Please. Thanks. From ciprian.craciun@REDACTED Tue May 29 18:09:52 2012 From: ciprian.craciun@REDACTED (Ciprian Dorin Craciun) Date: Tue, 29 May 2012 19:09:52 +0300 Subject: [erlang-questions] managing OS processes In-Reply-To: References: Message-ID: On Tue, May 29, 2012 at 3:38 PM, Tim Watson wrote: > Guys, > > The other problem I run into is, what about daemon/detached scripts, things > launched using setsid/nohup and/or put straight into the background? From my point of view such application -- i.e. that daemonize themselves -- are broken to begin with and they live you with only limited options: * search another solution; (long live the open-source world;) :) * fix the respective applications and submit patches upstream; (again long live open-source, but unfortunately not everybody understands why always daemonizing is a bad approach -- I've encountered such cases...) * use OS-specific tricks to force it to stay in foreground (i.e. PID namespaces in Linux, etc.); As such I don't see this as being the purpose of such a library. > And how > is your proxy program going to exhibit characteristics any different in this > situation, where you cannot 'spawn' a sub-process (or whatever the windows > equivalent is). Again I don't see "not supporting Windows" as a big problem, because: * most Erlang developers (that would need such process control features) are educated enough to work in an UNIX compatible environment; * most serious Erlang deployments I guess live in an UNIX compatible environment; > Anyway, despite my reticence, I think your programs both sounds pretty > useful - although I'm bemused that you'd use JSON serialization instead of > simply using the ei interface Unfortunately the `ei` interface is available (natively) only for C/C++ and Erlang. But that leaves out things like Python and NodeJS which are mandatory in my case. And I still find more easily looking over JSON than over a binary message. (BTW I use the `packet` Erlang message framing `4 bytes length + payload`.) > - and I would also suggest that packaging them > as separate libraries would be very useful to the community, giving others > the benefit of utilising your work! :) Yes but most of the time I doubt someone will actually use it. That's why I've asked if someone wants to use it, I could extract it. Ciprian. From jack@REDACTED Tue May 29 18:13:08 2012 From: jack@REDACTED (Jack Moffitt) Date: Tue, 29 May 2012 10:13:08 -0600 Subject: [erlang-questions] managing OS processes In-Reply-To: References: Message-ID: > over JSON than over a binary message. (BTW I use the `packet` Erlang > message framing `4 bytes length + payload`.) I have been using {packet, 2}. What happens when (via bad data or malicious intent) someone sends you 0xffffffff as the message size? Is there some way I haven't noticed to set a maximum allowed length on {packet, 4} packets? jack. From ciprian.craciun@REDACTED Tue May 29 18:17:26 2012 From: ciprian.craciun@REDACTED (Ciprian Dorin Craciun) Date: Tue, 29 May 2012 19:17:26 +0300 Subject: [erlang-questions] managing OS processes In-Reply-To: References: Message-ID: On Tue, May 29, 2012 at 7:13 PM, Jack Moffitt wrote: >> over JSON than over a binary message. (BTW I use the `packet` Erlang >> message framing `4 bytes length + payload`.) > > I have been using {packet, 2}. What happens when (via bad data or > malicious intent) someone sends you 0xffffffff as the message size? Is > there some way I haven't noticed to set a maximum allowed length on > {packet, 4} packets? :) This problem hit me a few times before (someone wrote from the process side to `stdout`). Interestingly enough (being on x86-32) it tried to allocate 2-4 GB of data via `malloc`, returned `NULL`, the assertion was broken and the controlled process was killed in a graceful manner. And as seen from the code I didn't fix it yet. :) But indeed such a check should have been made. Ciprian. From freeakk@REDACTED Tue May 29 18:36:47 2012 From: freeakk@REDACTED (Michael Uvarov) Date: Tue, 29 May 2012 20:36:47 +0400 Subject: [erlang-questions] [ANN] Erlang Bindings for Xapian In-Reply-To: <1D24E487-3655-45BE-A5B0-0A6E3F9E248F@worrell.nl> References: <1D24E487-3655-45BE-A5B0-0A6E3F9E248F@worrell.nl> Message-ID: > Is Xapian linked to your bindings or is it a separately running process? It is a linked-in driver, but I also planning to support this binding as a port. From g@REDACTED Tue May 29 19:15:00 2012 From: g@REDACTED (Garrett Smith) Date: Tue, 29 May 2012 12:15:00 -0500 Subject: [erlang-questions] managing OS processes In-Reply-To: References: Message-ID: On Sat, May 26, 2012 at 4:06 AM, Benoit Chesneau wrote: > I'm trying to find a way to manage some os processes launched from > Erlang. For example I need to relaunch them when the external process > die for an unknown reason or send an HUP signal or just stop the > external process. > > For now I'm using erlang ports to open an external process and detect > when it die. I also send signals using the `os:cmd` and and always > send a SIGKILL signal using the `os:cmd`before closing the port. But > is there a better way to do that? Maybe a module somewhere ? I saw > erlexec [1] but it doesn't seem to be maintained since a while. I think it'd be great to get this into OTP core, but until that functionality is a) implemented and b) has demonstrated solid operations across range of platforms, I'm inclined to use something like runit. http://smarden.org/runit Runit will "daemonize" for you (wants child processes to run in the foreground) and provides a simple interface for sending signals to a child process. E.g. to send KILL to a process (shell example): printf k > $SERVICE/supervise/control The control scheme is described here: http://smarden.org/runit/runsv.8.html It's not as tightly integrated as a futuristic open_port might be. E.g. if you want to detect a pathological case (service flapping, app errors, etc.) you'd have to poll the environment (track pid changes, scan logs, etc.) But there are sometimes advantages to keeping OS process supervision and monitoring at arms length. Garrett From g@REDACTED Tue May 29 20:41:09 2012 From: g@REDACTED (Garrett Smith) Date: Tue, 29 May 2012 13:41:09 -0500 Subject: [erlang-questions] more thoughts about package/dependency management In-Reply-To: <4FC3F807.3040701@gmail.com> References: <4FC3F807.3040701@gmail.com> Message-ID: On Mon, May 28, 2012 at 5:11 PM, Tim Watson wrote: > http://hyperthunk.wordpress.com/2012/05/28/does-erlangotp-need-a-new-package-management-solution/ > > This is a summary of the Erlware-Questions discussions - hopefully I've been > true to what was said on the list, but if I've misrepresented anyone's > opinion then I apologise and hope that you'll put it down to my 'special' > short/medium term memory. :) Thanks for the summary! For fear-of-sheer-volume I didn't delve into the upstream artifacts :) I'd like to more directly respond to the post, but there's a lot there, so I'm jotting down "brain dump" points without thinking too hard about how they fit into the prior discussion. Your post covers a large chunk of "already solved" problems, if you leverage the last couple decades of system packaging experience. I'll point to Arch Linux's outstanding tool chain for package authoring and management as a starting point. Arch has achieved its remarkable growth I believe largely due to this. High points of Arch's package authoring/management facilities: - Drop-dead easy to build a package - Essentially line-of-sight automation for what people typically do when consuming software (e.g. "configure && make && make install" in the case of system packages) - Convention is to pull source code directly from upstream sources, whatever they may be (FTP, HTTP, git, hg, etc.) - Package building and installation tools are decoupled from publishing and hosting - Custom/private repositories are identical to official/core repositories and are easy to setup - Security has become very good over the years (using PGP, web-of-trust) Regarding packages, I don't think you need separate "publisher" metadata. E.g. in system package land, it's very common to have multiple packages of essentially the same software. Each has a separate name but may share common "provides" metadata. E.g. if I wanted to consume my fork of "rebar" I could the package "rebar-gar1t", which would "provide" "rebar" and satisfy that dependency if needed. This works well in practice as has a couple of advantages: - One less field to worry about - Easy to spot the "official" package and derivatives Here's a quick read and I think a comprehensive picture of the metadata one could need: https://wiki.archlinux.org/index.php/PKGBUILD Arch does not support multiple installed versions, which is obviously not workable for Erlang. That's a "must change/enhance" topic. +1 that the tool must support both system level installations (requiring privileged access) and local (suitable for unprivileged). I haven't seen a "global vs local" OTP app setup in Erlang -- but as I understand things, ERL_LIBS is a path, so it should work today based on the boot scripts, yes? I would not chose git for a repository format. This just requires simple file access: - A smallish index that can be downloaded separately - Each package is a file in a directory I think the packages themselves ought be securely signed (e.g. PGP scheme) and authorized by the installer. White listing sources is of course important, but I believe independent -- e.g. an installer might require HTTPS or for all mirrors. Does the preference for binary packages mean that source based packages are not supported? As a sanity check for your "list of tools" I've mapped each role to what Arch uses: * managing local and/or remote repositories Repositories are simply hierarchical stores that can be accessed using libcurl. There's no *one way* and you're free to use whatever tools make sense (push via WebDav, rsync, scp, git commit + pull, whatever) IMO there's no need or advantage to limit the repository format more than this. Package indexes are managed using repo-add: http://www.archlinux.org/pacman/repo-add.8.html * solving dependencies The Arch installer -- pacman -- does this. http://www.archlinux.org/pacman/pacman.8.html * fetching dependencies/indexes All using libcurl. Local indexes are synchronized independent of the packages. Dependency resolution is performed before any packages are downloaded. Packages are stored in a local cache for subsequent installs as needed (repairs mainly) but can be purged easily enough using pacman. * building Building is a part of the packaging process, which is handled by makepkg: http://www.archlinux.org/pacman/makepkg.8.html It's performed by the "build" function in PKGBUILD (defined by the package author). * packaging/assembling Packages are simply a gzipped tar of a directory that's created by the "package" function in PKGBUILD plus the package metadata and install scripts (used at install time). * publishing This is out-of-band in Arch -- it's up to the repository maintainer entirely. I think this is a good point of separation. You've nailed the list of tools/processes here -- nothing else to add! I hate sounding like an Arch fan boy (guilty) -- but this packaging problem is critical to keep simple yet effective, and that's hard to achieve. So I'd leverage as much as makes sense from the system packagers before (re)inventing anything new. And just change what doesn't work. IMO Arch is the best starting point for this. Naively, an Erlang equivalent of: - pacman - the program that performs all work on behalf of an installer - makepkg - the program that performs all work on behalf of a packager - repo-add - the program that manages a package index database - PKGBUILD - the package metadata and supporting scripts - /etc/pacman.conf - configuration for the installer (repositories, trust levels, etc.) - /etc/pacman.d/gnupg/ - local keyring, gpg related Nothing here about repository access protocols. Garrett P.S. Apologies for the excessively high level hand-waving -- again this is just a brain dump. From ingela.andin@REDACTED Wed May 30 10:41:54 2012 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 30 May 2012 10:41:54 +0200 Subject: [erlang-questions] Yaws and httpc In-Reply-To: References: Message-ID: Hi! 2012/5/26, Avinash Dhumane : [...] > > [*1] The test-application needs to simulate many "web browser" clients, so > > I suppose I keep a separate profile for each client process. Seems reasonable. [...] > one connection to web server since test-client is submitting the > transactions synchronously. But, I am seeing that the number of > connections to the web server grow more than 16000 even when there are > only 10000 "httpc" services running in the inets container. You need to tune your client configuration. Se also reply to your later mail with subject HttpC options. [...] Regards Ingela Erlang/OTP team Ericsson From ingela.andin@REDACTED Wed May 30 10:43:13 2012 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 30 May 2012 10:43:13 +0200 Subject: [erlang-questions] HttpC options In-Reply-To: References: Message-ID: Hi! I will try to clear it up for you. There is a consept called persistent conections and another one called pipelining that uses persistent connections in a different manner. So in both cases you reuse a connection to a server to send subsequent requests. You can not use both concepts at once you will have to choose. When using persistent connections without pipelining a request will be sent and the client waits for the reply and then the next request is sent and again the client waits for the reply and so on. When using pipelining you will send the next requests imediatly before receiving the answer to the previous request. The server will awnser the request in the same order that they where sent. If the connection is broken outstanding request will be automatically resent on a new connection. Also note from the documentation "If possible the client will keep its connections alive and use persistent connections with or without pipeline depending on configuration and current circumstances. The HTTP/1.1 specification does not provide a guideline for how many requests would be ideal to be sent on a persistent connection, this very much depends on the application. Note that a very long queue of requests may cause a user perceived delay as earlier requests may take a long time to complete. The HTTP/1.1 specification does suggest a limit of 2 persistent connections per server, which is the default value of the max_sessions option." Regards Ingela Erlang/OTP team - Ericsson AB 2012/5/30, Ingela Andin : > Hi! > > I will try to clear it up for you. > > There is a consept called persistent conections and another one called > pipelining that uses persistent connections in a different manner. > So in both cases you reuse a connection to a server to send subsequent > requests. You can not use both concepts at once you will have to > choose. > > When using persistent connections without pipelining a request will be > sent and the client waits for the reply and then the next request is > sent and again the client waits for the reply and so on. > > When using pipelining you will send the next requests imediatly before > receiving the answer to the previous request. The server will awnser > the request in the same order that they where sent. If the connection > is broken outstanding request will be automatically resent on a new > connection. > > Also note from the documentation > > "If possible the client will keep its connections alive and use > persistent connections with or without pipeline depending on > configuration and current circumstances. The HTTP/1.1 specification > does not provide a guideline for how many requests would be ideal to > be sent on a persistent connection, this very much depends on the > application. Note that a very long queue of requests may cause a user > perceived delay as earlier requests may take a long time to > complete. The HTTP/1.1 specification does suggest a limit of 2 > persistent connections per server, which is the default value of the > max_sessions option." > > Regards Ingela Erlang/OTP team - Ericsson AB > > > 2012/5/29, Avinash Dhumane : >> I am not able to sense the difference between "max_keep_alive_length" and >> "max_pipeline_length" options ("httpc" module) in pipelining scenario >> (turned on by setting pipeline_timeout to timer:seconds(60)). >> >> Documentation says: >> max_keep_alive_length = maximum number of outstanding requests on the >> _same_ connection to a host >> max_pipeline_length = maximum number of outstanding requests on a >> _pipelined_ connection to a host >> >> Questions are: >> 1) Can both these options co-exist in pipelining scenario? If so, what >> aspects of requests submissions would they affect? >> 2) In "without pipelining" scenario, what is the meaning of keeping more >> than one outstanding request on the _same_ connection, i.e. >> max_keep_alive_length option? Can it really be done? If so, how different >> it is from pipelining? >> >> I am quite confused! :-) >> >> Please. >> >> Thanks. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > From erlang@REDACTED Wed May 30 10:45:10 2012 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 30 May 2012 10:45:10 +0200 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> <7E452DB1-8AB8-44C6-8CC9-5DA99EF8AB22@cs.otago.ac.nz> <1338218754.27626.YahooMailNeo@web111409.mail.gq1.yahoo.com> Message-ID: On Tue, May 29, 2012 at 12:08 AM, Richard O'Keefe wrote: > > On 29/05/2012, at 3:25 AM, Thomas Lindgren wrote: >> BEAM docs are not needed to produce a second source implementation, as shown by several examples. (1) > > People were not asking for BEAM documentation in order to produce > a second source implementation. ?That's a red herring. ?People > ask for BEAM documentation in order to understand and perhaps > revise the main implementation. > > One thing that Open Source is about is about community contributions > and it is awfully hard to contribute to something that is > under-documented. >> >> Also, there has so far been little practical interest shown by Erlang users in such a second source. So implementation efforts may be in vain. (3) > > Again, the red herring. ?This is *NOT* about second implementations. > All of the additional implementations I've ever heard of used their > own back end, *not* BEAM, and would *not* have received any great > benefit from BEAM being documented. > > There is detailed documentation available for Lua and Icon and the > WAM and several other VMs around, so there's not even any great > advantage in learning about VMs from BEAM. > >> My personal view, at least, is that most of the difficulty in "keeping up with The Real Thing", Erlang/OTP, would be not in reproducing BEAM but in writing a fully compatible implementation tracking the rest of the runtime, ERTS. (2) > > Well, not _just_ that, but that's a big part of it. > >> So, is there a _practical_ case for doing these docs? In particular, will the effort result in useful external contributions that outweigh time spent? Not at all clear to me. > > Bearing in mind that "BEAM documentation" and "second system implementation" > are about as irrelevant to each other as any two topics about the same > language can be, what _might_ we realistically expect from BEAM documentation? > > (1) There are now several languages implemented on top of Erlang. > ? ?Some are interpreted in Erlang, some are compiled to Erlang ASTs, > ? ?and some are compiled to BEAM. ?Compiling to the BEAM is *much* > ? ?harder than it needs to be because of the current undocumentation. > > ? ?This isn't *replacing* Erlang/OTP but *augmenting* it. > > (2) I've proposed a number of minor but handy extensions to Erlang > ? ?syntax that lack a reference implementation because I have no idea > ? ?what the compiler should generate for them; these could use the > ? ?existing BEAM. > > (3) There are existing things that could be compiled more efficiently. > ? ?I'm thinking here in particular of list comprehension. ?I'm not > ? ?sure if the improved translation can be done with the existing > ? ?BEAM or if minor extensions would be needed, because the > ? ?undocumentation for the BEAM does not make clear any range > ? ?limits or other restrictions on BEAM instructions. > > (4) There are even bigger changes, like frames, where even estimating > ? ?the scope of the changes is hard because of the undocumentation. > > But above all you are making an assumption which I utterly reject, > namely that documentation is a COST and ONLY a cost, that producing > documentation provides no DIRECT benefits to the people writing the > documentation. > > On the contrary, > ?- you may find defects as you document > ?- you may be able to structure the documentation so that > ? test cases can automatically be extracted > ?- you may in the very act of explaining a limitation see > ? how it can be overcome > ?- you may be able to extract parts of the implementation > ? automatically from the documentation > and I could go on. > > There's a slogan I learned from a business textbook: > ?find the indispensable man and fire him! > There was something I found unsettling: we were told in this > thread that there wasn't any need for documentation because > the Erlang/OTP maintainers had enough people who _knew_ this > stuff. > > One of my colleagues here was managing a software project > once; a key employee went on holiday and was murdered in a > far country. ?A former student of mine was starting up a > new company with me as a consultant, and being a worse > driver than he thought, drove at speed into a tree. ?The > tree survived. ?One of my former colleagues, a very > intelligent and likable guy, was cycling down a Melbourne > street and got knocked over by a hit-and-run driver. ?The > result was head injury and someone who could dress himself > but couldn't program if his life depended on it. ?When I > was at Quintus, the founder who wrote the compiler and was > the only person who really understood it quit and was never > heard from again. > > Fergus O'Brien supervised an MSc on "Organisational > Forgetting" and the fact of organisational forgetting has > haunted the fringes of my mind ever since. > > People die. ?People get head injuries. ?People quit. > Documents get lost. ?(If you ever find an architecture- > of-the-ICL-2900 manual, I'd like to see it.) ?Even if > things _are_ written down, organisations forget _where_. > If they aren't written down, they WILL be forgotten > sooner or later. > Brilliant! and well argued. Thank you Richard. This last argument is excellent brain-food. So "stuff" should be well documented precisely because one day the creator of the stuff will get bored or die and the encompassing organisation will collectively forget how it works. I've notice a lot the "some other guy knows this stuff" phenomena - I've been chasing a particular problem for months. I always get the "I don't know but X knows" - I ask X and they refer to X1 etc. But X1 does not know ... I'm running out of leads. It appears that *nobody* knows. Fortunately the BEAM is not yet there. I know who knows - and when I ask them they *do* know so I just hope the entire OTP group aren't on the same plane to an Erlang conference in a far away land ... I conclude it should be documented. The next question is "priority" ... in the real world we juggle priorities. The case *is* made that we should document the BEAM but is this more or less important than implementing frames etc? As always, a tricky question... /Joe > > People > Well, my colleague Andrew Trotman once had a key employee and ... ?? > > > > > We >> >> Hope it helps. >> >> >> Best regards, >> Thomas >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vasdeveloper@REDACTED Wed May 30 11:26:58 2012 From: vasdeveloper@REDACTED (Kannan) Date: Wed, 30 May 2012 14:56:58 +0530 Subject: [erlang-questions] Erlang PP Question Message-ID: Hi There, If -define does just string replacement, why it does not let me do the following? -define(Z, z() -> io:fwrite("z")). -define(Y, y() -> io:fwrite("y")). -define(X, ?Y. ?Z.). ?X Thanks, Kannan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Wed May 30 11:33:42 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 30 May 2012 11:33:42 +0200 Subject: [erlang-questions] Erlang PP Question In-Reply-To: References: Message-ID: <4FC5E976.6010101@gmail.com> On 05/30/2012 11:26 AM, Kannan wrote: > Hi There, > > If -define does just string replacement, why it does not let me do the > following? > > -define(Z, z() -> io:fwrite("z")). > -define(Y, y() -> io:fwrite("y")). > -define(X, ?Y. ?Z.). In the last line, you have ?Y. followed by whitespace. This ends the define declaration. Erlang source code is divided into "forms", where each form ends with . followed by whitespace (usually newline, but any whitespace or even a comment can be used). So you have two forms at the end: '-define(X, ?Y.' and '?Z.).' and none of these are syntactically valid. It is not possible to make a single macro expand into two forms. /Richard From vasdeveloper@REDACTED Wed May 30 12:11:33 2012 From: vasdeveloper@REDACTED (Kannan) Date: Wed, 30 May 2012 15:41:33 +0530 Subject: [erlang-questions] Erlang PP Question In-Reply-To: <4FC5E976.6010101@gmail.com> References: <4FC5E976.6010101@gmail.com> Message-ID: Isn't PP takes place before parsing? Isn't PP is simply a string replacement? On Wed, May 30, 2012 at 3:03 PM, Richard Carlsson < carlsson.richard@REDACTED> wrote: > On 05/30/2012 11:26 AM, Kannan wrote: > >> Hi There, >> >> If -define does just string replacement, why it does not let me do the >> following? >> >> -define(Z, z() -> io:fwrite("z")). >> -define(Y, y() -> io:fwrite("y")). >> -define(X, ?Y. ?Z.). >> > > In the last line, you have ?Y. followed by whitespace. This ends the > define declaration. Erlang source code is divided into "forms", where each > form ends with . followed by whitespace (usually newline, but any > whitespace or even a comment can be used). So you have two forms at the > end: '-define(X, ?Y.' and '?Z.).' and none of these are syntactically > valid. It is not possible to make a single macro expand into two forms. > > /Richard > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Wed May 30 12:12:50 2012 From: vasdeveloper@REDACTED (Kannan) Date: Wed, 30 May 2012 15:42:50 +0530 Subject: [erlang-questions] Erlang PP Question In-Reply-To: References: <4FC5E976.6010101@gmail.com> Message-ID: My bad, please read the second PP as -define. On Wed, May 30, 2012 at 3:41 PM, Kannan wrote: > Isn't PP takes place before parsing? Isn't PP is simply a string > replacement? > > > > On Wed, May 30, 2012 at 3:03 PM, Richard Carlsson < > carlsson.richard@REDACTED> wrote: > >> On 05/30/2012 11:26 AM, Kannan wrote: >> >>> Hi There, >>> >>> If -define does just string replacement, why it does not let me do the >>> following? >>> >>> -define(Z, z() -> io:fwrite("z")). >>> -define(Y, y() -> io:fwrite("y")). >>> -define(X, ?Y. ?Z.). >>> >> >> In the last line, you have ?Y. followed by whitespace. This ends the >> define declaration. Erlang source code is divided into "forms", where each >> form ends with . followed by whitespace (usually newline, but any >> whitespace or even a comment can be used). So you have two forms at the >> end: '-define(X, ?Y.' and '?Z.).' and none of these are syntactically >> valid. It is not possible to make a single macro expand into two forms. >> >> /Richard >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Wed May 30 12:17:31 2012 From: vasdeveloper@REDACTED (Kannan) Date: Wed, 30 May 2012 15:47:31 +0530 Subject: [erlang-questions] Erlang PP Question In-Reply-To: References: <4FC5E976.6010101@gmail.com> Message-ID: Also, is there a way to expand a single macro into multiple macros, before a final replacement? On Wed, May 30, 2012 at 3:42 PM, Kannan wrote: > My bad, please read the second PP as -define. > > > On Wed, May 30, 2012 at 3:41 PM, Kannan wrote: > >> Isn't PP takes place before parsing? Isn't PP is simply a string >> replacement? >> >> >> >> On Wed, May 30, 2012 at 3:03 PM, Richard Carlsson < >> carlsson.richard@REDACTED> wrote: >> >>> On 05/30/2012 11:26 AM, Kannan wrote: >>> >>>> Hi There, >>>> >>>> If -define does just string replacement, why it does not let me do the >>>> following? >>>> >>>> -define(Z, z() -> io:fwrite("z")). >>>> -define(Y, y() -> io:fwrite("y")). >>>> -define(X, ?Y. ?Z.). >>>> >>> >>> In the last line, you have ?Y. followed by whitespace. This ends the >>> define declaration. Erlang source code is divided into "forms", where each >>> form ends with . followed by whitespace (usually newline, but any >>> whitespace or even a comment can be used). So you have two forms at the >>> end: '-define(X, ?Y.' and '?Z.).' and none of these are syntactically >>> valid. It is not possible to make a single macro expand into two forms. >>> >>> /Richard >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Wed May 30 12:23:27 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 30 May 2012 12:23:27 +0200 Subject: [erlang-questions] Erlang PP Question In-Reply-To: References: <4FC5E976.6010101@gmail.com> Message-ID: <4FC5F51F.5000701@gmail.com> On 05/30/2012 12:11 PM, Kannan wrote: > Isn't PP takes place before parsing? Isn't PP is simply a string > replacement? Almost - it's token level replacement (just as in the C preprocessor), and the splitting into forms is done before preprocessing. /Richard > On Wed, May 30, 2012 at 3:03 PM, Richard Carlsson > > wrote: > > On 05/30/2012 11:26 AM, Kannan wrote: > > Hi There, > > If -define does just string replacement, why it does not let me > do the > following? > > -define(Z, z() -> io:fwrite("z")). > -define(Y, y() -> io:fwrite("y")). > -define(X, ?Y. ?Z.). > > > In the last line, you have ?Y. followed by whitespace. This ends the > define declaration. Erlang source code is divided into "forms", > where each form ends with . followed by whitespace (usually newline, > but any whitespace or even a comment can be used). So you have two > forms at the end: '-define(X, ?Y.' and '?Z.).' and none of these are > syntactically valid. It is not possible to make a single macro > expand into two forms. > > /Richard > > From carlsson.richard@REDACTED Wed May 30 12:28:25 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 30 May 2012 12:28:25 +0200 Subject: [erlang-questions] Erlang PP Question In-Reply-To: References: <4FC5E976.6010101@gmail.com> Message-ID: <4FC5F649.9010701@gmail.com> On 05/30/2012 12:17 PM, Kannan wrote: > Also, is there a way to expand a single macro into multiple macros, > before a final replacement? If you mean into multiple macro definitions, it is not possible - each macro definition is a separate form, and you can't expand one form into multiple forms. But multiple uses of macros in the body of another macro is not a problem, if that's what you mean: -define(X, {foo, ?Y, ?Z}). /Richard > On Wed, May 30, 2012 at 3:42 PM, Kannan > wrote: > > My bad, please read the second PP as -define. > > > On Wed, May 30, 2012 at 3:41 PM, Kannan > wrote: > > Isn't PP takes place before parsing? Isn't PP is simply a string > replacement? > > > > On Wed, May 30, 2012 at 3:03 PM, Richard Carlsson > > > wrote: > > On 05/30/2012 11:26 AM, Kannan wrote: > > Hi There, > > If -define does just string replacement, why it does not > let me do the > following? > > -define(Z, z() -> io:fwrite("z")). > -define(Y, y() -> io:fwrite("y")). > -define(X, ?Y. ?Z.). > > > In the last line, you have ?Y. followed by whitespace. This > ends the define declaration. Erlang source code is divided > into "forms", where each form ends with . followed by > whitespace (usually newline, but any whitespace or even a > comment can be used). So you have two forms at the end: > '-define(X, ?Y.' and '?Z.).' and none of these are > syntactically valid. It is not possible to make a single > macro expand into two forms. > > /Richard > > > > From vasdeveloper@REDACTED Wed May 30 12:36:25 2012 From: vasdeveloper@REDACTED (Kannan) Date: Wed, 30 May 2012 16:06:25 +0530 Subject: [erlang-questions] Erlang PP Question In-Reply-To: <4FC5F649.9010701@gmail.com> References: <4FC5E976.6010101@gmail.com> <4FC5F649.9010701@gmail.com> Message-ID: Thanks Richard for your response. That makes sense. Actually what I wanted is to get a macro expand into two different functions in the source file. My intention was not producing a term. That is where the '.' came to be a problem. It seems it not possible. Anyways I will breakdown the macro into two different ones. Kind Regards, Kannan. On Wed, May 30, 2012 at 3:58 PM, Richard Carlsson < carlsson.richard@REDACTED> wrote: > On 05/30/2012 12:17 PM, Kannan wrote: > >> Also, is there a way to expand a single macro into multiple macros, >> before a final replacement? >> > > If you mean into multiple macro definitions, it is not possible - each > macro definition is a separate form, and you can't expand one form into > multiple forms. But multiple uses of macros in the body of another macro is > not a problem, if that's what you mean: > > -define(X, {foo, ?Y, ?Z}). > > /Richard > > > On Wed, May 30, 2012 at 3:42 PM, Kannan > > wrote: >> >> My bad, please read the second PP as -define. >> >> >> On Wed, May 30, 2012 at 3:41 PM, Kannan > > wrote: >> >> Isn't PP takes place before parsing? Isn't PP is simply a string >> replacement? >> >> >> >> On Wed, May 30, 2012 at 3:03 PM, Richard Carlsson >> >> >> >> >> wrote: >> >> On 05/30/2012 11:26 AM, Kannan wrote: >> >> Hi There, >> >> If -define does just string replacement, why it does not >> let me do the >> following? >> >> -define(Z, z() -> io:fwrite("z")). >> -define(Y, y() -> io:fwrite("y")). >> -define(X, ?Y. ?Z.). >> >> >> In the last line, you have ?Y. followed by whitespace. This >> ends the define declaration. Erlang source code is divided >> into "forms", where each form ends with . followed by >> whitespace (usually newline, but any whitespace or even a >> comment can be used). So you have two forms at the end: >> '-define(X, ?Y.' and '?Z.).' and none of these are >> syntactically valid. It is not possible to make a single >> macro expand into two forms. >> >> /Richard >> >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed May 30 12:40:52 2012 From: ok@REDACTED (ok@REDACTED) Date: Wed, 30 May 2012 22:40:52 +1200 Subject: [erlang-questions] Erlang PP Question In-Reply-To: References: Message-ID: "Kannan" wrote > If -define does just string replacement, It doesn't perform string replacement, never has done, and was never intended to. Like the (ANSI) C preprocessor, it performs TOKEN sequence replacement. If you happen to want string replacement, there's the UNIX m4 command, with a GNU version and a faster PDm4. From peter.mechlenborg@REDACTED Wed May 30 12:47:10 2012 From: peter.mechlenborg@REDACTED (Peter Mechlenborg) Date: Wed, 30 May 2012 12:47:10 +0200 Subject: [erlang-questions] Erlang and OCaml Message-ID: Hi all Do any of you have experience with combining Erlang and OCaml? I think this could be a great match, Erlang for coordination and fault tolerance and OCaml for number crunching, but I have only found http://code.google.com/p/erlocaml/ searching, and this project seems dead...? Best regards, -- Peter Mechlenborg -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed May 30 13:16:25 2012 From: ok@REDACTED (ok@REDACTED) Date: Wed, 30 May 2012 23:16:25 +1200 Subject: [erlang-questions] Erlang PP Question In-Reply-To: References: <4FC5E976.6010101@gmail.com> <4FC5F649.9010701@gmail.com> Message-ID: <00e88873920aa49375bfe620cd8edb8a.squirrel@chasm.otago.ac.nz> "Kannan" > Thanks Richard for your response. That makes sense. > Actually what I wanted is to get a macro expand into two different > functions in the source file. My intention was not producing a term. That > is where the '.' came to be a problem. As Richard Carlsson has explained, Erlang source files are tokenised, broken at ". " tokens into forms, and only then preprocessed. Preprocessing expands one sequence of tokens constituting the source version of a form into another sequence of tokens constituting the version that the parser then parses. At this point there is no token that a macro could possibly generated that would be interpreted as a form terminator or separator. (Indeed, we could have an amusing debate about whether ". " is _really_ a token... For the older among is, is a "tape mark" a tape block?) However, just because the built-in Erlang preprocessor cannot do it does not mean it cannot be done by preprocessing. For example, define(two, `$1`'(X) -> $2`'(X, X+1). $2`'(X, Y) -> X*Y.')dnl two(fred, nurke) fed to m4 gives you fred(X) -> nurke(X, X+1). nurke(X, Y) -> X*Y. M4 takes a bit of getting used to, but it _is_ a Turing-complete string processing language; you can do amazing things with it. (I use M4 with Java because Java generics don't support primitive types, and it is _amazing_ what a difference not creating a lot of Integer and Double boxes can make to your memory usage.) There is always Lisp-Flavoured Erlang, which puts a Lispy syntax complete with Lispy macros on top of Erlang; I don't recall if you can generate multiple definitions with LFE but it's definitely one thing Lisp can do. Lisp macros turn single forms into single forms, but a (progn Def ... Def) top level form can contain several definitions. From aleksandr.vin@REDACTED Wed May 30 14:29:25 2012 From: aleksandr.vin@REDACTED (Aleksandr Vinokurov) Date: Wed, 30 May 2012 14:29:25 +0200 Subject: [erlang-questions] What to use for calling COM on Windows? Message-ID: Hi all, What I can use for subj? NIF? :) -- Aleksandr, from #euc2012 From torben.lehoff@REDACTED Wed May 30 15:36:05 2012 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Wed, 30 May 2012 15:36:05 +0200 Subject: [erlang-questions] Erlang and OCaml In-Reply-To: References: Message-ID: <6D68F826-06DA-47B0-A487-4983F4A7D468@gmail.com> Talk to Eric Merritt - he has been using mlton for the number crunching. Cheers, Torben Sent from my iPad On 30/05/2012, at 12.47, Peter Mechlenborg wrote: > Hi all > > Do any of you have experience with combining Erlang and OCaml? > > I think this could be a great match, Erlang for coordination and fault tolerance and OCaml for number crunching, but I have only found http://code.google.com/p/erlocaml/ searching, and this project seems dead...? > > Best regards, > > -- Peter Mechlenborg > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From siraaj@REDACTED Wed May 30 15:53:46 2012 From: siraaj@REDACTED (Siraaj Khandkar) Date: Wed, 30 May 2012 09:53:46 -0400 Subject: [erlang-questions] Erlang and OCaml In-Reply-To: <6D68F826-06DA-47B0-A487-4983F4A7D468@gmail.com> References: <6D68F826-06DA-47B0-A487-4983F4A7D468@gmail.com> Message-ID: <4FC6266A.6050802@khandkar.net> I'd be VERY interested to hear about this as well! On 05/30/12 09:36, Torben Hoffmann wrote: > Talk to Eric Merritt - he has been using mlton for the number crunching. > > Cheers, > Torben > > Sent from my iPad > > On 30/05/2012, at 12.47, Peter Mechlenborg > wrote: > >> Hi all >> >> Do any of you have experience with combining Erlang and OCaml? >> >> I think this could be a great match, Erlang for coordination and fault >> tolerance and OCaml for number crunching, but I have only found >> http://code.google.com/p/erlocaml/ searching, and this project seems dead...? >> >> Best regards, >> >> -- Peter Mechlenborg -- Siraaj Khandkar .o. ..o ooo From demeshchuk@REDACTED Wed May 30 15:57:31 2012 From: demeshchuk@REDACTED (Dmitry Demeshchuk) Date: Wed, 30 May 2012 17:57:31 +0400 Subject: [erlang-questions] Erlang and OCaml In-Reply-To: <4FC6266A.6050802@khandkar.net> References: <6D68F826-06DA-47B0-A487-4983F4A7D468@gmail.com> <4FC6266A.6050802@khandkar.net> Message-ID: Echo ( http://aboutecho.com/ ) has been using Erlang with OCaml ports for about 3 years. https://github.com/EchoTeam/ocaml-erlang-port On Wed, May 30, 2012 at 5:53 PM, Siraaj Khandkar wrote: > I'd be VERY interested to hear about this as well! > > > > On 05/30/12 09:36, Torben Hoffmann wrote: >> >> Talk to Eric Merritt - he has been using mlton for the number crunching. >> >> Cheers, >> Torben >> >> Sent from my iPad >> >> On 30/05/2012, at 12.47, Peter Mechlenborg > > wrote: >> >>> Hi all >>> >>> Do any of you have experience with combining Erlang and OCaml? >>> >>> I think this could be a great match, Erlang for coordination and fault >>> tolerance and OCaml for number crunching, but I have only found >>> http://code.google.com/p/erlocaml/ searching, and this project seems >>> dead...? >>> >>> Best regards, >>> >>> -- Peter Mechlenborg > > > -- > Siraaj Khandkar > .o. > ..o > ooo > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Best regards, Dmitry Demeshchuk From james.hague@REDACTED Wed May 30 16:04:41 2012 From: james.hague@REDACTED (James Hague) Date: Wed, 30 May 2012 09:04:41 -0500 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> <7E452DB1-8AB8-44C6-8CC9-5DA99EF8AB22@cs.otago.ac.nz> <1338218754.27626.YahooMailNeo@web111409.mail.gq1.yahoo.com> Message-ID: When there was some documentation long ago, the bulk of it was about the dozens and dozens of BEAM opcodes, and that's not actually the important part. What *really* matters are all the undocumented assumptions in the system. What are the rules about floating point registers? Which BIFs can cause garbage collection and how are they treated differently than non-GC BIFs? When should registers be marked as unused? When are destructive tuple updates allowed? What inefficient-looking sequences are improved by the BEAM loader? I can learn BEAM by disassembling code (anyone who is interested can can use http://prog21.dadgum.com/127.html as a starting point), but I can't learn the underlying rules and philosophies of the VM. BEAM documentation can be as simple as: * One-line-per-instruction description of opcodes. * Couple of pages of VM architecture and rules. * Two HOWTO docs: adding a BIF and adding a VM instruction. * List of transformations done by the loader. From tuncer.ayaz@REDACTED Wed May 30 16:40:16 2012 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Wed, 30 May 2012 16:40:16 +0200 Subject: [erlang-questions] more thoughts about package/dependency management In-Reply-To: <4FC3F807.3040701@gmail.com> References: <4FC3F807.3040701@gmail.com> Message-ID: On Tue, May 29, 2012 at 12:11 AM, Tim Watson wrote: > http://hyperthunk.wordpress.com/2012/05/28/does-erlangotp-need-a-new-package-management-solution/ > > This is a summary of the Erlware-Questions discussions - hopefully > I've been true to what was said on the list, but if I've > misrepresented anyone's opinion then I apologise and hope that > you'll put it down to my 'special' short/medium term memory. :) > > Comments/Feedback welcome. Thanks Tim for the summary. I mostly agree with you and Garrett and think that distros and Hackage are a good source for inspiration. A few important points: - No need for storing (meta)data in a real vcs: + The index needs to be aware of all versions. + The versions should be kept as (archive) files in directories. + The index can be modified to add or remove entries. Replace doesn't seem like a good idea. It can be implemented, if needed. + Branches or merging shouldn't be required. - We should consider leveraging existing ftp mirror networks like distros and texlive do. For both the files and index. - We should look at dpkg (apt) and rpm (yum) for index and signature inspiration. - Sandboxing should be part of the initial design. - It has be trivial to get the tool(s) either as part of Erlang/OTP or with a simple download like rebar. - It should not require any fragile dependency as you need working tools to get packages. - The website can be made central, but should be avoided if possible. - Hayoo like search would be nice to have in a 2nd step. Do we need an infinite set of version or should it be limited like distros do with the various channels and a pool of versions referred to in the current indices? From watson.timothy@REDACTED Wed May 30 16:56:21 2012 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 30 May 2012 15:56:21 +0100 Subject: [erlang-questions] more thoughts about package/dependency management In-Reply-To: References: <4FC3F807.3040701@gmail.com> Message-ID: <5E61A0C2-D344-4E39-A4A5-C3BAC91F4E98@gmail.com> On 30 May 2012, at 15:40, Tuncer Ayaz wrote: > On Tue, May 29, 2012 at 12:11 AM, Tim Watson wrote: >> http://hyperthunk.wordpress.com/2012/05/28/does-erlangotp-need-a-new-package-management-solution/ >> >> This is a summary of the Erlware-Questions discussions - hopefully >> I've been true to what was said on the list, but if I've >> misrepresented anyone's opinion then I apologise and hope that >> you'll put it down to my 'special' short/medium term memory. :) >> >> Comments/Feedback welcome. > > Thanks Tim for the summary. I mostly agree with you and Garrett and > think that distros and Hackage are a good source for inspiration. > I agree in principle, and Garrett raised some good points. > A few important points: > > - No need for storing (meta)data in a real vcs: Ok so Eric and I were trying to avoid the need for hosting, making this completely free and community oriented. You want to go down some hosting route, that's fair enough, but who's going to provide the infrastructure? It worries me that this will put people off using the system and we'll end up getting nowhere. Apart from that I'm fine with the idea though. > + The index needs to be aware of all versions. +1 > + The versions should be kept as (archive) files in directories. Well that's fine if it's hosted, as the search can be very simple and the download just goes direct to the packaging using FTP or whatever. For 'non hosted' - read hosted on github or some such - then this was more of a sticking point, so I guess we need to iron out exactly *how* this stuff can/will be hosted before making that kind of decision about the repository layout. > + The index can be modified to add or remove entries. Replace > doesn't seem like a good idea. It can be implemented, if needed. Snapshot builds are a PITA, but some shops depend on them for CI etc. > + Branches or merging shouldn't be required. > - We should consider leveraging existing ftp mirror networks like > distros and texlive do. For both the files and index. How will this work in practice? > - We should look at dpkg (apt) and rpm (yum) for index and signature > inspiration. > - Sandboxing should be part of the initial design. +1 > - It has be trivial to get the tool(s) either as part of Erlang/OTP or > with a simple download like rebar. > - It should not require any fragile dependency as you need working > tools to get packages. > - The website can be made central, but should be avoided if possible. What does this mean? This is sounding more and more like a system that someone has to build and maintain. I'm *absolutely* in favour of doing that, but someone is going to stump up the money for it. > - Hayoo like search would be nice to have in a 2nd step. > Again, this is *great* idea, but someone, somewhere, covers the cost of hackage/hayoo - the same applies here. > Do we need an infinite set of version or should it be limited like > distros do with the various channels and a pool of versions referred > to in the current indices? From serge@REDACTED Wed May 30 17:48:53 2012 From: serge@REDACTED (Serge Aleynikov) Date: Wed, 30 May 2012 11:48:53 -0400 Subject: [erlang-questions] managing OS processes In-Reply-To: References: Message-ID: <4FC64165.2000109@aleynikov.org> Take a look at erlexec: http://code.google.com/p/erlexec/ I believe it does exactly what you need. On 5/26/2012 5:06 AM, Benoit Chesneau wrote: > I'm trying to find a way to manage some os processes launched from > Erlang. For example I need to relaunch them when the external process > die for an unknown reason or send an HUP signal or just stop the > external process. > > For now I'm using erlang ports to open an external process and detect > when it die. I also send signals using the `os:cmd` and and always > send a SIGKILL signal using the `os:cmd`before closing the port. But > is there a better way to do that? Maybe a module somewhere ? I saw > erlexec [1] but it doesn't seem to be maintained since a while. > > > - beno?t > > > [1] http://code.google.com/p/erlexec/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From desired.mta@REDACTED Wed May 30 22:39:44 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Wed, 30 May 2012 22:39:44 +0200 Subject: [erlang-questions] -type dict(_, _) ? Message-ID: Hi all, let's say I have a proplist of players: userid and player itself: -type players() :: list({mpserver:userid(), mpserver:player()}). Almost fine, but I know that keys are unique. In that case, it makes sense to use a dict: -type players() :: dict(mpserver:userid(), mpserver:player()). This throws "type not found" error. Why is it unsupported? Maybe something similar is supported, but I don't know how? What data structure do you use in this situation, and why? And do you benefit from dialyzer? Thanks, Motiejus Jak?tys From olivier.girondel@REDACTED Wed May 30 22:54:09 2012 From: olivier.girondel@REDACTED (Olivier Girondel) Date: Wed, 30 May 2012 22:54:09 +0200 Subject: [erlang-questions] Multi-core experience as of May, 2012 Message-ID: I was wondering what the state of the art is for Erlang, running on not-so-common (yet) processors/architectures (eg: Tilera, XLP/XLR, Octeon, ...) Since we may be few having access to such hardware, I'd be happy if you'd share your experiences, so a little survey would look like: * Hardware (the multi-core part: CPU, board, ...) * (optionally) Other hardware (RAM, disk, network devices, ...) * Erlang releases (tested or in production) * Number of nodes, if distributed * Use of proprietary SDKs * The C-side (Ports, linked-in drivers, NIFs) * Native / cross-compilation * Application use cases * Numbers -the ones that make you/executives say "GO" Just off the top of my head, feel free to extend.. Has the multicore era begun ? Best regards, -- Olivier PS/ Ok, there is http://www.paulgraham.com/avg.html, but the unasked questions never get answers :) From raould@REDACTED Wed May 30 22:59:15 2012 From: raould@REDACTED (Raoul Duke) Date: Wed, 30 May 2012 13:59:15 -0700 Subject: [erlang-questions] Multi-core experience as of May, 2012 In-Reply-To: References: Message-ID: On Wed, May 30, 2012 at 1:54 PM, Olivier Girondel wrote: > Has the multicore era begun ? what does that mean? most new machines, even just laptops, are multi core. :-) From olivier.girondel@REDACTED Wed May 30 23:03:56 2012 From: olivier.girondel@REDACTED (Olivier Girondel) Date: Wed, 30 May 2012 23:03:56 +0200 Subject: [erlang-questions] Multi-core experience as of May, 2012 In-Reply-To: References: Message-ID: On Wed, May 30, 2012 at 10:59 PM, Raoul Duke wrote: >> Has the multicore era begun ? > > what does that mean? most new machines, even just laptops, are multi core. :-) I meant "heavily/massively multicore", of course -- Olivier From raould@REDACTED Wed May 30 23:10:11 2012 From: raould@REDACTED (Raoul Duke) Date: Wed, 30 May 2012 14:10:11 -0700 Subject: [erlang-questions] Multi-core experience as of May, 2012 In-Reply-To: References: Message-ID: On Wed, May 30, 2012 at 2:03 PM, Olivier Girondel wrote: > I meant "heavily/massively multicore", of course it does seem like there are folks claiming erlang doesn't even scale well for non-massively-multicore situations. so it would be interesting to learn what the nuances / dials / variables / issues are; how to win / avoid losing when it comes to scaling erlang on platforms. From desired.mta@REDACTED Wed May 30 23:34:22 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Wed, 30 May 2012 23:34:22 +0200 Subject: [erlang-questions] PropEr statem: what does Res::term in next_state/3 mean? Message-ID: Hi, PropEr team, sorry for writing you (again) on holiday time. ;) I was reading about proper_statem, and could not understand one thing. >From PropEr tutorial[1]: next_state(S, V, {call,_,create_account,[_Name]}) -> S#state{users = [V|S#state.users]}; Instead, next_state/3 should return: S#state{users = [{call,erlang,hd,[V]}|S#state.users]} Two things are unclear: 1. What is V after all? 2. How can I return abstract code in *my own state*? I.e. does PropEr do some magic to look at my state and call the appropriate commands? I read the edoc[2] too, but still it didn't make it clearer. [1]: http://proper.softlab.ntua.gr/Tutorials/PropEr_testing_of_generic_servers.html [2]: file:///home/motiejus/code/erlmpc/deps/proper/doc/index.html I would be very glad if you could shed some light on this. Thanks a lot, Motiejus Jak?tys From james@REDACTED Wed May 30 23:39:22 2012 From: james@REDACTED (james) Date: Wed, 30 May 2012 22:39:22 +0100 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> <7E452DB1-8AB8-44C6-8CC9-5DA99EF8AB22@cs.otago.ac.nz> <1338218754.27626.YahooMailNeo@web111409.mail.gq1.yahoo.com> Message-ID: <4FC6938A.6090601@mansionfamily.plus.com> > If they aren't written down, they WILL be forgotten > sooner or later. I would also like to suggest that the act of writing down a design forces a degree of rigour that is hard to achieve otherwise. I have several times thought that I had considered a design from 'every angle', only to find that when I *write it down*, some of the angles were imperial and some were metric, and some were just smoke and mirrors. When something is written down, the handwaving stops and the engineering starts. From overminddl1@REDACTED Thu May 31 00:56:41 2012 From: overminddl1@REDACTED (OvermindDL1) Date: Wed, 30 May 2012 16:56:41 -0600 Subject: [erlang-questions] Multi-core experience as of May, 2012 In-Reply-To: References: Message-ID: On Wed, May 30, 2012 at 3:10 PM, Raoul Duke wrote: > On Wed, May 30, 2012 at 2:03 PM, Olivier Girondel > wrote: >> I meant "heavily/massively multicore", of course > > it does seem like there are folks claiming erlang doesn't even scale > well for non-massively-multicore situations. so it would be > interesting to learn what the nuances / dials / variables / issues > are; how to win / avoid losing when it comes to scaling erlang on > platforms. Speaking as one that looks in to this, but only uses Erlang on a 3 cluster set up with 1 core each on two and 6 on one, it works well on these. But from reading the mailing list, forums, articles, it seems Erlang starts to 'cap out' at a cluster of around 70, just due to mesh communications swamping everything else. *However* it is possible to partition an Erlang mesh by using hidden nodes and/or custom communication protocols, and doing that should allow you to expand pretty arbitrarily, based on the type of program(s) run on the cluster. Thus Erlang can do many-core, you just have to take it into account when you program for it. From overminddl1@REDACTED Thu May 31 01:01:03 2012 From: overminddl1@REDACTED (OvermindDL1) Date: Wed, 30 May 2012 17:01:03 -0600 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: <4FC6938A.6090601@mansionfamily.plus.com> References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> <1336418114.51449.YahooMailNeo@web111413.mail.gq1.yahoo.com> <1336586607.77117.YahooMailNeo@web111410.mail.gq1.yahoo.com> <7E452DB1-8AB8-44C6-8CC9-5DA99EF8AB22@cs.otago.ac.nz> <1338218754.27626.YahooMailNeo@web111409.mail.gq1.yahoo.com> <4FC6938A.6090601@mansionfamily.plus.com> Message-ID: On Wed, May 30, 2012 at 3:39 PM, james wrote: > I would also like to suggest that the act of writing down a design forces a > degree of rigour that is hard to achieve otherwise. ?I have several times > thought that I had considered a design from 'every angle', only to find that > when I *write it down*, some of the angles were imperial and some were > metric, and some were just smoke and mirrors. > > When something is written down, the handwaving stops and the engineering > starts. I *completely* agree with this. At my work we *have* to write down our systems, the API's, interfaces, backend, everything, as we code it. It really does help to solidify how the system should work, helps us to consider angles that we would not have considered before. Even if we never use the documentation again, the fact that we are forced to write it in *detail* and keep it up to date really does have a great boon on the quality of code. If you do not find problems in your code when writing documentation, then likely your documentation is not detailed enough. From aronisstav@REDACTED Thu May 31 01:23:50 2012 From: aronisstav@REDACTED (Stavros Aronis) Date: Wed, 30 May 2012 16:23:50 -0700 (PDT) Subject: [erlang-questions] -type dict(_, _) ? In-Reply-To: References: Message-ID: Hi, This is a planned extension for Dialyzer. Currently support for parametric types is rather weak (e.g. even a type name can appear only once, regardless of arity, so no dict/0 and dict/2 types can be declared together in the same module). Therefore only the dict/0 type has been defined, hence the error you get. If you are already implementing this with a dict, I would suggest that you define the type as: -type players :: dict(). %dict(mpserver:userid(), mpserver:player()). ... and hope that the commented out version will be supported soon! Regards, Stavros On Wednesday, May 30, 2012 10:39:44 PM UTC+2, Motiejus Jak?tys wrote: > > Hi all, > > let's say I have a proplist of players: userid and player itself: > > -type players() :: list({mpserver:userid(), mpserver:player()}). > > Almost fine, but I know that keys are unique. In that case, it makes > sense to use a dict: > -type players() :: dict(mpserver:userid(), mpserver:player()). > > This throws "type not found" error. Why is it unsupported? Maybe > something similar is supported, but I don't know how? What data > structure do you use in this situation, and why? And do you benefit > from dialyzer? > > Thanks, > Motiejus Jak?tys > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stu.bailey@REDACTED Thu May 31 01:42:22 2012 From: stu.bailey@REDACTED (Stu Bailey) Date: Wed, 30 May 2012 16:42:22 -0700 Subject: [erlang-questions] Is there a good source for documentation on BEAM? In-Reply-To: References: <1336380419.31641.YahooMailNeo@web111405.mail.gq1.yahoo.com> Message-ID: Excellent! This should be codified into Armstrong's Law of Technology Obfuscation! On Mon, May 7, 2012 at 4:47 AM, Joe Armstrong wrote: > I think it works like this: > > ? ?1) first you don't understand how the X works (X=Beam, JVM, X11, > ... you name it) > ? ?2) You struggle - and think - ?google and have a hot bath > ? ?3) Eureka - bath flows over > ? ?4) Now you can understand it - and you can also remember why you > could not understand it > ? ?5) Now it's easy you understand it > ? ?6) You see no reason to document it since it's obvious > > Round about 4) there is a small window of opportunity to explain to > other people how it works. > Once you get to 6) it's very difficult to remember what it felt like > at point 2) and consequently difficult > to write decent documentation. > > /Joe > > > On Mon, May 7, 2012 at 11:27 AM, Michael Turner > wrote: >> "Actually, I don't think such docs are all _that_ crucial -- who >> really needs to know, except a small number of VM implementors?" >> >> Aren't Erlang's chances of greater mindshare improved by making it >> easier to become a VM implementor? I doubt very much that Java would >> be where it is today had it not been for clear VM specification. >> That's not to say that Erlang should follow in all of Java's >> footsteps, even if it could. But I have to say I was a boggled to >> learn that you can't find out what the VM opcodes mean without reading >> the source (and maybe not even then, if the source contains bugs >> vis-a-vis some idealized machine model.) >> >> -michael turner >> >> >> >> On Mon, May 7, 2012 at 5:46 PM, Thomas Lindgren >> wrote: >>> >>> >>> >>>>________________________________ >>>> From: Jonathan Coveney >>>>To: erlang-questions@REDACTED >>>>Sent: Monday, May 7, 2012 8:39 AM >>>>Subject: [erlang-questions] Is there a good source for documentation on BEAM? >>>> >>>> >>>>This question seems to come up now and again, and it's surprising to me that a crucial part of the documentation isn't better documented. Is there a reason that it is the case? Is the reason that there is no VM spec to give the devs the flexibility to change the intermediate layer without having to worry about backwards compatibility to the degree that Java does? >>> >>> >>> Actually, I don't think such docs are all _that_ crucial -- who really needs to know, except a small number of VM implementors? (And they should read the source to get at all the goodies.) But perhaps someone on the list might be moved to do a tutorial presentation on an Erlang Factory or something? >>> >>> (By the way, I too assume not doing it is to avoid getting bogged down into minutiae.) >>> >>> If you want to learn more about some of the intellectual roots, try these: >>> http://wambook.sourceforge.net/ >>> >>> http://dl.acm.org/citation.cfm?id=188051 >>> >>> >>> Best regards, >>> Thomas >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From roberto@REDACTED Thu May 31 02:16:59 2012 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 30 May 2012 17:16:59 -0700 Subject: [erlang-questions] StringIO? Message-ID: Dear list, is there anything similar to Python's StringIO [1] in Erlang? I've a library that only accepts files as params of a function, but I have everything I need into a variable that holds a string(). So what I would need to do is write this string to a temp file, and pass that one to the library; I'd like to avoid doing that if possible. Thank you, r. [1] http://docs.python.org/library/stringio.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Thu May 31 02:25:16 2012 From: bob@REDACTED (Bob Ippolito) Date: Wed, 30 May 2012 17:25:16 -0700 Subject: [erlang-questions] StringIO? In-Reply-To: References: Message-ID: On Wed, May 30, 2012 at 5:16 PM, Roberto Ostinelli wrote: > Dear list, > > is there anything similar to Python's StringIO [1] in Erlang? > > I've a library that only accepts files as params of a function, but I have > everything I need into a variable that holds a string(). So what I would > need to do is write this string to a temp file, and pass that one to the > library; I'd like to avoid doing that if possible. > > Thank you, > > r. > > [1] http://docs.python.org/library/stringio.html > It's undocumented, but there's the ram_file module: https://github.com/erlang/otp/blob/maint/lib/kernel/src/ram_file.erl -bob -------------- next part -------------- An HTML attachment was scrubbed... URL: From manopapad@REDACTED Thu May 31 04:34:49 2012 From: manopapad@REDACTED (Manolis Papadakis) Date: Wed, 30 May 2012 19:34:49 -0700 Subject: [erlang-questions] PropEr statem: what does Res::term in next_state/3 mean? In-Reply-To: References: Message-ID: <4FC6D8C9.7060607@gmail.com> The first definition of the next_state/3 clause corresponds to the real code example, where the server returns a single password on each account creation. In this scenario, next_state/3 will be called in two possible contexts: during the generation of a command sequence, where V will be a symbolic variable (e.g. {var,42}), and during actual testing, where V will be an actual concrete password, as returned by the server (e.g. "Password1" -- a fairly secure password: it contains capital and lowercase letters, and even numbers). The second definition assumes a slightly different server interface, where create_account/1 returns a list of passwords, and the client is expected to only use the first one (this is obviously a contrived example). This alternative scenario is used to emphasize a side effect of the dual nature of next_state/3: Because it may be called with both symbolic and concrete values, it shouldn't treat its argument in a manner that is invalid for symbolic values. In the first scenario, we simply add the returned value to a list, which makes sense for both symbolic and concrete values, and is thus allowed. In the second scenario, however, we'd want to get the first password from the returned list of passwords and only store that in our model. If next_state/3 were to only be called with concrete values (e.g. ["Password1","123456","qwerty"]), then we would simply retrieve the head of the list directly: %% WRONG CODE next_state(S, V, {call,_,create_account,[_Name]}) -> S#state{users = [hd(V)|S#state.users]}; However, next_state/3 may also be called with a symbolic value, e.g. {var,42}, so the above code (which assumes that V is a list), will break. The way around this is to use symbolic calls to perform any computation that depends on the structure of the result: next_state(S, V, {call,_,create_account,[_Name]}) -> S#state{users = [{call,erlang,hd,[V]}|S#state.users]}; PropEr will automatically evaluate the symbolic calls every time next_state/3 is called with concrete values. On 05/30/2012 02:34 PM, Motiejus Jak?tys wrote: > Hi, PropEr team, > > sorry for writing you (again) on holiday time. ;) > > I was reading about proper_statem, and could not understand one thing. > From PropEr tutorial[1]: > next_state(S, V, {call,_,create_account,[_Name]}) -> > S#state{users = [V|S#state.users]}; > > Instead, next_state/3 should return: > > S#state{users = [{call,erlang,hd,[V]}|S#state.users]} > > Two things are unclear: > 1. What is V after all? > 2. How can I return abstract code in *my own state*? I.e. does PropEr > do some magic to look at my state and call the appropriate commands? > > I read the edoc[2] too, but still it didn't make it clearer. > > [1]: http://proper.softlab.ntua.gr/Tutorials/PropEr_testing_of_generic_servers.html > [2]: file:///home/motiejus/code/erlmpc/deps/proper/doc/index.html > > I would be very glad if you could shed some light on this. > > Thanks a lot, > Motiejus Jak?tys From bengt.kleberg@REDACTED Thu May 31 07:07:46 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 31 May 2012 07:07:46 +0200 Subject: [erlang-questions] StringIO? In-Reply-To: References: Message-ID: <1338440866.4779.6.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> Greetings, I have written a string_io module that gives an IO object (like in file:open/2). It is old, but still works with epp_dodger (that at the time did not work on strings. Has that changed?). bengt On Thu, 2012-05-31 at 02:16 +0200, Roberto Ostinelli wrote: > Dear list, > > > is there anything similar to Python's StringIO [1] in Erlang? > > > I've a library that only accepts files as params of a function, but I > have everything I need into a variable that holds a string(). So what > I would need to do is write this string to a temp file, and pass that > one to the library; I'd like to avoid doing that if possible. > > > Thank you, > > > r. > > > [1] http://docs.python.org/library/stringio.html From roberto@REDACTED Thu May 31 08:30:14 2012 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 30 May 2012 23:30:14 -0700 Subject: [erlang-questions] StringIO? In-Reply-To: References: Message-ID: Thank you Bob, that looks like it's exactly what I need. Thank you, r. On Wed, May 30, 2012 at 5:25 PM, Bob Ippolito wrote: > > It's undocumented, but there's the ram_file module: > > https://github.com/erlang/otp/blob/maint/lib/kernel/src/ram_file.erl > > -bob > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Thu May 31 08:41:43 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 31 May 2012 08:41:43 +0200 Subject: [erlang-questions] clarify: init_per_suite/0 in Common Test Message-ID: <1338446503.4779.22.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> Greetings, There is no Module:init_per_suite/0 documented for Common Test test suites. Should it be? bengt From carlsson.richard@REDACTED Thu May 31 09:00:39 2012 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 31 May 2012 09:00:39 +0200 Subject: [erlang-questions] gb_trees missing function!? In-Reply-To: <63E687EC-6E5F-4F36-9BE6-3355DE11B24C@gmail.com> References: <63E687EC-6E5F-4F36-9BE6-3355DE11B24C@gmail.com> Message-ID: <4FC71717.2080103@gmail.com> On 05/25/2012 09:30 PM, Dmitry Kolesnikov wrote: > Hello, > > gb_trees is pretty efficient in many use cases but I am starting to miss couple of features: > * range(From, To, Tree) -> [{Key, Val}] to retrieve multiple keys > * iterator(Key, Tree) -> Iter start iterator from Key > > of course lists:dropwhile(fun(X) -> X< Key end, gb_trees:to_list(Tree)) makes trick but I think it is not very efficient... > > What is your opinion? I think these are good ideas. If you mail me a suggested patch, I can try to get them into OTP. A while back I posted to the list that I've been working on a unified API for dict-like modules, and I think that at least the range function should probably be a standard function in that API (possibly under another name). /Richard From abdoo.mahran@REDACTED Thu May 31 10:04:10 2012 From: abdoo.mahran@REDACTED (Abdul Fattah Mahran) Date: Thu, 31 May 2012 10:04:10 +0200 Subject: [erlang-questions] clarify: init_per_suite/0 in Common Test In-Reply-To: <1338446503.4779.22.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> References: <1338446503.4779.22.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> Message-ID: Hi Bengt, It is documented http://www.erlang.org/doc/man/common_test.html#Module:init_per_suite-1 Module:init_per_suite(Config) -> NewConfig | {skip,Reason} | {skip_and_save,Reason,SaveConfig} Types: Config = NewConfig = SaveConfig = [{Key,Value}] Key = atom() Value = term() Reason = term() OPTIONAL This configuration function is called as the first function in the suite. It typically contains initializations which are common for all test cases in the suite, and which shall only be done once. The Config parameter is the configuration data which can be modified here. Whatever is returned from this function is given as Config to all configuration functions and test cases in the suite. If {skip,Reason} is returned, all test cases in the suite will be skipped and Reason printed in the overview log for the suite. For information on save_config and skip_and_save, please see Dependencies between Test Cases and Suitesin the User's Guide. Thanks, best regards, On Thu, May 31, 2012 at 8:41 AM, Bengt Kleberg wrote: > Greetings, > > There is no Module:init_per_suite/0 documented for Common Test test > suites. Should it be? > > > bengt > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Thanks Best Regards, Abd El-Fattah Mahran http://www.linkedin.com/in/abdoomahran -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Thu May 31 12:06:21 2012 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 31 May 2012 12:06:21 +0200 Subject: [erlang-questions] clarify: init_per_suite/0 in Common Test In-Reply-To: References: <1338446503.4779.22.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> Message-ID: <1338458781.4779.33.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> The documentation is for init_per_suite/1, I was wondering about init_per_suite/0. bengt On Thu, 2012-05-31 at 10:04 +0200, Abdul Fattah Mahran wrote: > Hi Bengt, > It is documented > http://www.erlang.org/doc/man/common_test.html#Module:init_per_suite-1 > > > > Module:init_per_suite(Config) -> NewConfig | {skip,Reason} | > {skip_and_save,Reason,SaveConfig} > > > Types: > > Config = NewConfig = SaveConfig = [{Key,Value}] > > Key = atom() > > Value = term() > > Reason = term() > > OPTIONAL > > This configuration function is called as the first function in the > suite. It typically contains initializations which are common for all > test cases in the suite, and which shall only be done once. The Config > parameter is the configuration data which can be modified here. > Whatever is returned from this function is given as Config to all > configuration functions and test cases in the suite. If {skip,Reason} > is returned, all test cases in the suite will be skipped and Reason > printed in the overview log for the suite. > > For information on save_config and skip_and_save, please see > Dependencies between Test Cases and Suites in the User's Guide. > > > Thanks, > > best regards, > > > > > > > On Thu, May 31, 2012 at 8:41 AM, Bengt Kleberg > wrote: > Greetings, > > There is no Module:init_per_suite/0 documented for Common Test > test > suites. Should it be? > > > bengt > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Thanks > Best Regards, > Abd El-Fattah Mahran > > http://www.linkedin.com/in/abdoomahran > From pekadan@REDACTED Thu May 31 12:12:47 2012 From: pekadan@REDACTED (Peter Andersson) Date: Thu, 31 May 2012 12:12:47 +0200 Subject: [erlang-questions] clarify: init_per_suite/0 in Common Test In-Reply-To: References: <1338446503.4779.22.camel@seasc1027.dyn.rnd.as.sw.ericsson.se> Message-ID: Hi, I suppose Bengt's asking if you can use an info function also with init_per_suite. Answer is yes, and it's documented in the user's guide, chapter 4.14: http://www.erlang.org/doc/apps/common_test/write_test_chapter.html#id71329 We should add info also in the reference manual that what goes for the testcase info function, "Testcase()", is valid for config functions init/end_per_suite/group too. Thanks! Best regards, Peter Ericsson AB, Erlang/OTP 2012/5/31, Abdul Fattah Mahran : > Hi Bengt, > It is documented > http://www.erlang.org/doc/man/common_test.html#Module:init_per_suite-1 > > Module:init_per_suite(Config) -> NewConfig | {skip,Reason} | > {skip_and_save,Reason,SaveConfig} > > Types: > Config = NewConfig = SaveConfig = [{Key,Value}] > Key = atom() > Value = term() > Reason = term() > > OPTIONAL > > This configuration function is called as the first function in the suite. > It typically contains initializations which are common for all test cases > in the suite, and which shall only be done once. The Config parameter is > the configuration data which can be modified here. Whatever is returned > from this function is given as Config to all configuration functions and > test cases in the suite. If {skip,Reason} is returned, all test cases in > the suite will be skipped and Reason printed in the overview log for the > suite. > > For information on save_config and skip_and_save, please see Dependencies > between Test Cases and > Suitesin > the User's Guide. > > > Thanks, > > best regards, > > > > > On Thu, May 31, 2012 at 8:41 AM, Bengt Kleberg > wrote: > >> Greetings, >> >> There is no Module:init_per_suite/0 documented for Common Test test >> suites. Should it be? >> >> >> bengt >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Thanks > Best Regards, > Abd El-Fattah Mahran > http://www.linkedin.com/in/abdoomahran > From desired.mta@REDACTED Thu May 31 13:06:27 2012 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Thu, 31 May 2012 13:06:27 +0200 Subject: [erlang-questions] {@docRoot } in edoc when using edown markdown format Message-ID: Hi, I want to get docRoot in overview.edoc, to link to a document in the same repository. Here is an extract from doc/overview.edoc: Detailed information about the whole X architecture is in
TD.markdown And the resulting README.md after ./rebar doc, is: Detailed information about the whole X architecture is in [TD.markdown](/TD.markdown) What I would like to see: [TD.markdown](http://github.com/Motiejus/X-architecture/doc/TD.markdown) rebar.config: {edoc_opts, [{doclet, edown_doclet}, {app_default, "http://www.erlang.org/doc/man"}, {top_level_readme, {"./README.md", "http://github.com/Motiejus/X-architecture"}}]}. Any way to get the absolute URL? Thanks, Motiejus Jak?tys From francesquini@REDACTED Thu May 31 13:29:04 2012 From: francesquini@REDACTED (Emilio De Camargo Francesquini) Date: Thu, 31 May 2012 13:29:04 +0200 Subject: [erlang-questions] About Erlang SMP scheduler In-Reply-To: References: <0F4D9489-7755-4FD2-8831-E4BEE2A8C8F1@gmail.com> <2FF9F129-C96C-48C2-BFE9-223F591D624B@gmail.com> <0773DEE9-E2B7-440A-814E-33C55A472A7C@gmail.com> Message-ID: Hello, > check_balance() sets up > migration paths and migration limits that are used in order to balance > the load between schedulers. Without balancing, processes with the > same assigned priority can get very different effective priorities. I have some questions relating to that. First, after the decision to rebalance has been made, what are the criteria to select which processes from each queue will be migrated? The last ones in the queue, the first, at random, ...? Second, how are these migration paths and limits defined? Does the definition of the migration paths take into consideration the architecture (SMP, multisocket SMP, NUMA, ...) of the machine? If we consider that the lifespan of the majority of the process is quite short, and if we are running on a NUMA machine for example, migrating processes at random to another numa-node might hurt the performance more than letting the execution queues unbalanced. Thanks! Best Regards Emilio Francesquini 2012/5/8 Rickard Green : > The work stealing is there to quickly distribute work between > schedulers, but it doesn't balance the load. check_balance() sets up > migration paths and migration limits that are used in order to balance > the load between schedulers. Without balancing, processes with the > same assigned priority can get very different effective priorities. > > Example: With 4 schedulers and 400 cpu bound processes executing the > same code, each scheduler will eventually end up with 100 processes to > manage. That is, all processes will have the same access to the cpu. > If you disable check_balance(), you may end up with three schedulers > only having one process each, and one scheduler with 397 processes. > These 397 processes will effectively have a much lower priority than > the other three processes. > > Regards, > Rickard Green, Erlang/OTP, Ericsson AB > > 2012/5/2 Siyao Zheng(???) : >> Hi, >> >> Yes, that's true. But the "try_steal_task()" afterwards is also controlled >> by ERTS_SMP macro. It doesn't make any sense to talk about workload balance >> outside multi-processor environment. I'm just wondering why "check_balance" >> is needed while there exists work stealing(try_steal_task). >> >> Cheers >> >> On May 2, 2012, at 12:04 PM, xu yu wrote: >> >> Hi, >> >> "check_balance" controlled by "ifdef ERTS_SMP", so... >> >> >> 2012/4/27 "Siyao Zheng(???)" >>> >>> Hi, >>> >>> schedule(), in which check_balance() is only called. >>> >>> In schedule(), when check_balance_reds reaches zero, scheduler decides to >>> "check_balance". But later in schedule(), when scheduler finds run queue >>> being empty, it will try to steal task from other schedulers (by calling >>> try_steal_task()), which is also a load balance mechanism. I'm just >>> wondering what the purpose of check_balance(). >>> >>> Cheers >>> Siyao >>> >>> On Apr 27, 2012, at 9:40 PM, Zabrane Mickael wrote: >>> >>> > Hi Siyao, >>> > >>> > Which "check_balance" you've commented out? >>> > >>> > Here is what I found so far: >>> > >>> > $ find /otp_src_R15B01 -type f | xargs grep check_balance | grep -v >>> > matches >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: int >>> > forced_check_balance; >>> > >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c:check_balance(ErtsRunQueue >>> > *c_rq) >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c:# error >>> > check_balance() assumes ERTS_MAX_PROCESS < (1 << 27) >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>> > c_rq->check_balance_reds = INT_MAX; >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>> > c_rq->check_balance_reds = INT_MAX; >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>> > rq->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: * >>> > check_balance() is never called in more threads >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: forced = >>> > balance_info.forced_check_balance; >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>> > balance_info.forced_check_balance = 0; >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>> > c_rq->check_balance_reds = INT_MAX; >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>> > rq->check_balance_reds = INT_MAX; >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>> > rq->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>> > rq->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>> > balance_info.forced_check_balance = 0; >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>> > (RQ)->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; \ >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>> > balance_info.forced_check_balance = 1; >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>> > ERTS_RUNQ_IX(0)->check_balance_reds = 0; >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: if >>> > (rq->check_balance_reds <= 0) >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.c: >>> > check_balance(rq); >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.h: int >>> > check_balance_reds; >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.h: >>> > (RQ)->check_balance_reds -= (REDS); \ >>> > /opt/otp_src_R15B01/erts/emulator/beam/erl_process.h: >>> > (RQ)->check_balance_reds -= (REDS); >>> > [...] >>> > >>> > For OTP Team: is this dangerous ? >>> > >>> > Regards, >>> > Zabrane >>> > >>> > >>> > On Apr 27, 2012, at 1:23 PM, Siyao Zheng(???) wrote: >>> > >>> >> Hi, >>> >> >>> >> In SMP version of Erlang, every scheduler periodically call >>> >> "check_balance()" to check load balance among all schedulers, then it might >>> >> migrate some processes to balance load, or shut down some schedulers with >>> >> low load. Does anyone know why scheduler should do this? check_balance() is >>> >> quite a big one, and it has to lock every run queue when it checks the run >>> >> queue. I think it's quite a big cost here. The work stealing at each >>> >> schedule step afterwards actually does work load balance very well. Actually >>> >> after I comment out the check_balance() step and run BigBang and Hackbench >>> >> benchmarks, they are really faster! >>> >> >>> >> So, I wonder what is the purpose of check_balance() step here, if it is >>> >> related to scalability on many core processors or something, and if there is >>> >> any other benchmark I can run to prove its usefulness. >>> >> >>> >> Cheers! >>> >> >>> >> Siyao >>> >> _______________________________________________ >>> >> erlang-questions mailing list >>> >> erlang-questions@REDACTED >>> >> http://erlang.org/mailman/listinfo/erlang-questions >>> > >>> > >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jose.valim@REDACTED Thu May 31 13:55:48 2012 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Thu, 31 May 2012 13:55:48 +0200 Subject: [erlang-questions] Optimization for the ++ operator Message-ID: I thought Erlang would optimize the ++ operator when the left side is known at compile time. For example, if the compiler sees the following outside of a function signature: "Foo" ++ Bar It could rewrite it as: [$F, $o, $o | assert_list(Bar)] However, I ran some benchmarks and it seems the optimization does not happen (on R15B). With a local dummy implementation of assert_list(Bar), I got that the first format is 50% slower than the second one. That said, given the possibility something is odd in my setup, does Erlang optimize it or not? If not, could it? * Jos? Valim www.plataformatec.com.br Founder and Lead Developer * -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Thu May 31 14:39:56 2012 From: tony@REDACTED (Tony Rogvall) Date: Thu, 31 May 2012 14:39:56 +0200 Subject: [erlang-questions] multiple behaviours Message-ID: <0CFCB8FA-DA25-428B-89DB-DBA34D62DFB3@rogvall.se> Hi! I am compiling the canopen stack (git://github.com/tonyrog/canopen.git) and get the following warning: ../src/co_os_app.erl:37: Warning: conflicting behaviours - callback index_specification/2 required by both 'co_stream_app' and 'co_app' (line 36) The question is: I can realize the reason for the warning, but we want to share some (one in this case) of the callbacks in the different flavours of the behaviours. One simple interface, one advance interface and the possibility to implement both .... In this case we could probably solve it by adding yet an other behaviour that does not require you to use both, but merge the parts needed. /Tony "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From arun11299@REDACTED Thu May 31 19:04:57 2012 From: arun11299@REDACTED (Arun Muralidharan) Date: Thu, 31 May 2012 22:34:57 +0530 Subject: [erlang-questions] Return status of gen_udp send Message-ID: Hi Folks! Is there any way to detect if the packet sent by issuing the statement "gen_udp:send" is really sent or got dropped because of buffer overflow in the kernel . I am using UDP socket with option set as active once and both send and receive buffer set to 2MB. Thanks, -A -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Thu May 31 19:22:13 2012 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Thu, 31 May 2012 18:22:13 +0100 Subject: [erlang-questions] SCTP multihoming In-Reply-To: References: Message-ID: Dear OTP team, The patch supplied by Tomas below works perfectly on Solaris 10 with R15B-1. Are there any plans to integrate it into an upcoming release? Chandru On 15 May 2012 14:20, Tomas Abrahamsson wrote: > Hi, > > On Linux, it worked on IPv4, but not IPv6 for me. > When I wrote the original patch I hadn't tried Solaris (or FreeBSD) > So I found additional issues on Solaris and FreeBSD: even IPv4 > SCTP multihoming didn't work there. Instead of trying out the > patch in my original post, I'd recommend trying out the > https://github.com/tomas-abrahamsson/otp/compare/sctp-multihoming > (referenced later in the thead: > http://erlang.org/pipermail/erlang-patches/2012-May/002788.html) > > BRs > Tomas > > On Tue, May 15, 2012 at 2:29 PM, Chandru > wrote: > > No, I didn't . Thanks for the pointer. Though the original poster says > it > > worked for IPv4 for him. It doesn't work for me even for IPv4 addresses. > > I'll try the patch out and see. > > > > Chandru > > > > On May 15, 2012 7:35 AM, "Lukas Larsson" wrote: > >> > >> I assume you have looked at this[1] thread on the erlang patches > >> mailinglist? > >> > >> [1] > http://erlang.org/pipermail/erlang-patches/2011-December/002559.html > >> > >> Sent from me > >> > >> On 15 maj 2012, at 06:06, Chandru > > >> wrote: > >> > >> > Hi all, > >> > > >> > I'm having trouble starting a server with multihoming support. > >> > > >> > The call: > >> > 16> (<0.804.0>) call gen_sctp:open([binary, > >> > {active,once}, > >> > {reuseaddr,true}, > >> > {port,3868}, > >> > {ip,{10,249,8,4}}, > >> > {ip,{10,249,8,12}}]) > >> > > >> > crashes with the report below. Anyone has a quick fix? I've traced it > >> > down to this: > >> > > >> > (<0.2364.0>) call > >> > > erlang:port_control(#Port<0.5560>,62,[[1],[[15,28],10,249,8,12],[[15,28],10,249,8,4]]) > >> > (<0.2364.0>) returned from erlang:port_control/3 -> > >> > [0,101,105,110,118,97,108] (which is einval basically) > >> > > >> > Any idea why this would be? I'm running this on Solaris 10, with R15B. > >> > > >> > cheers > >> > Chandru > >> > > >> > 22> rb:show(4). > >> > > >> > CRASH REPORT <0.866.0> 2012-05-15 > >> > 04:44:30 > >> > > >> > > =============================================================================== > >> > Crashing process > >> > initial_call > >> > {diameter_sctp,init,['Argument__1']} > >> > pid > >> > <0.866.0> > >> > registered_name > >> > [] > >> > error_info > >> > {error,badarg, > >> > [{gen_sctp,open, > >> > [[binary, > >> > {active,once}, > >> > {reuseaddr,true}, > >> > {port,3868}, > >> > {ip,{10,249,8,4}}, > >> > {ip,{10,249,8,12}}]], > >> > [{file,"gen_sctp.erl"},{line,126}]}, > >> > {diameter_sctp,open,3, > >> > [{file,"transport/diameter_sctp.erl"}, > >> > {line,227}]}, > >> > {diameter_sctp,i,1, > >> > [{file,"transport/diameter_sctp.erl"}, > >> > {line,147}]}, > >> > {diameter_sctp,init,1, > >> > [{file,"transport/diameter_sctp.erl"}, > >> > {line,141}]}, > >> > {proc_lib,init_p_do_apply,3, > >> > [{file,"proc_lib.erl"},{line,227}]}]} > >> > ancestors > >> > > >> > [diameter_sctp_listener_sup,diameter_transport_sup,diameter_sup, > >> > <0.779.0>] > >> > messages > >> > [] > >> > links > >> > [<0.803.0>] > >> > dictionary > >> > [] > >> > trap_exit > >> > false > >> > status > >> > running > >> > heap_size > >> > 75025 > >> > stack_size > >> > 24 > >> > reductions > >> > 736 > >> > _______________________________________________ > >> > erlang-questions mailing list > >> > erlang-questions@REDACTED > >> > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tomas.abrahamsson@REDACTED Thu May 31 19:45:57 2012 From: tomas.abrahamsson@REDACTED (Tomas Abrahamsson) Date: Thu, 31 May 2012 19:45:57 +0200 Subject: [erlang-questions] SCTP multihoming In-Reply-To: References: Message-ID: FYI: I've updated the branch on github https://github.com/tomas-abrahamsson/otp/compare/sctp-multihoming with one more commit to fix a compilation error on windows. I've also rebased it on top of the current 'maint'. BRs Tomas On Thu, May 31, 2012 at 7:22 PM, Chandru wrote: > Dear OTP team, > > The patch supplied by Tomas below works perfectly on Solaris 10 with R15B-1. > Are there any plans to integrate it into an upcoming release? > > Chandru > > > On 15 May 2012 14:20, Tomas Abrahamsson wrote: >> >> Hi, >> >> On Linux, it worked on IPv4, but not IPv6 for me. >> When I wrote the original patch I hadn't tried Solaris (or FreeBSD) >> So I found additional issues on Solaris and FreeBSD: even IPv4 >> SCTP multihoming didn't work there. Instead of trying out the >> patch in my original post, I'd recommend trying out the >> https://github.com/tomas-abrahamsson/otp/compare/sctp-multihoming >> (referenced later in the ?thead: >> http://erlang.org/pipermail/erlang-patches/2012-May/002788.html) >> >> BRs >> Tomas >> >> On Tue, May 15, 2012 at 2:29 PM, Chandru >> wrote: >> > No, I didn't . Thanks? for the pointer. Though the original poster says >> > it >> > worked for IPv4 for him. It doesn't work for me even for IPv4 addresses. >> > I'll try the patch out and see. >> > >> > Chandru >> > >> > On May 15, 2012 7:35 AM, "Lukas Larsson" wrote: >> >> >> >> I assume you have looked at this[1] thread on the erlang patches >> >> mailinglist? >> >> >> >> [1] >> >> http://erlang.org/pipermail/erlang-patches/2011-December/002559.html >> >> >> >> Sent from me >> >> >> >> On 15 maj 2012, at 06:06, Chandru >> >> >> >> wrote: >> >> >> >> > Hi all, >> >> > >> >> > I'm having trouble starting a server with multihoming support. >> >> > >> >> > The call: >> >> > 16> (<0.804.0>) call gen_sctp:open([binary, >> >> > ?{active,once}, >> >> > ?{reuseaddr,true}, >> >> > ?{port,3868}, >> >> > ?{ip,{10,249,8,4}}, >> >> > ?{ip,{10,249,8,12}}]) >> >> > >> >> > crashes with the report below. Anyone has a quick fix? I've traced it >> >> > down to this: >> >> > >> >> > (<0.2364.0>) call >> >> > >> >> > erlang:port_control(#Port<0.5560>,62,[[1],[[15,28],10,249,8,12],[[15,28],10,249,8,4]]) >> >> > (<0.2364.0>) returned from erlang:port_control/3 -> >> >> > [0,101,105,110,118,97,108] (which is einval basically) >> >> > >> >> > Any idea why this would be? I'm running this on Solaris 10, with >> >> > R15B. >> >> > >> >> > cheers >> >> > Chandru >> >> > >> >> > 22> rb:show(4). >> >> > >> >> > CRASH REPORT ?<0.866.0> >> >> > 2012-05-15 >> >> > 04:44:30 >> >> > >> >> > >> >> > =============================================================================== >> >> > Crashing process >> >> > ? ?initial_call >> >> > ?{diameter_sctp,init,['Argument__1']} >> >> > ? ?pid >> >> > ?<0.866.0> >> >> > ? ?registered_name >> >> > ? ? [] >> >> > ? ?error_info >> >> > ? ? ? ? ?{error,badarg, >> >> > ? ? ? ? ? ? ? ?[{gen_sctp,open, >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? [[binary, >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {active,once}, >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {reuseaddr,true}, >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {port,3868}, >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {ip,{10,249,8,4}}, >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? {ip,{10,249,8,12}}]], >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? [{file,"gen_sctp.erl"},{line,126}]}, >> >> > ? ? ? ? ? ? ? ? {diameter_sctp,open,3, >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[{file,"transport/diameter_sctp.erl"}, >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {line,227}]}, >> >> > ? ? ? ? ? ? ? ? {diameter_sctp,i,1, >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[{file,"transport/diameter_sctp.erl"}, >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {line,147}]}, >> >> > ? ? ? ? ? ? ? ? {diameter_sctp,init,1, >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[{file,"transport/diameter_sctp.erl"}, >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {line,141}]}, >> >> > ? ? ? ? ? ? ? ? {proc_lib,init_p_do_apply,3, >> >> > ? ? ? ? ? ? ? ? ? ? ? ? ? [{file,"proc_lib.erl"},{line,227}]}]} >> >> > ? ?ancestors >> >> > >> >> > ?[diameter_sctp_listener_sup,diameter_transport_sup,diameter_sup, >> >> > ? ? ? ? ?<0.779.0>] >> >> > ? ?messages >> >> > ? ? ?[] >> >> > ? ?links >> >> > ?[<0.803.0>] >> >> > ? ?dictionary >> >> > ? ? ?[] >> >> > ? ?trap_exit >> >> > ?false >> >> > ? ?status >> >> > running >> >> > ? ?heap_size >> >> > ?75025 >> >> > ? ?stack_size >> >> > ? ? ?24 >> >> > ? ?reductions >> >> > ? ? 736 >> >> > _______________________________________________ >> >> > erlang-questions mailing list >> >> > erlang-questions@REDACTED >> >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions >