From norton@REDACTED Thu Sep 1 01:30:30 2011 From: norton@REDACTED (Joseph Norton) Date: Thu, 1 Sep 2011 08:30:30 +0900 Subject: [erlang-questions] how can I make mnesia/sasl keep quiet In-Reply-To: References: Message-ID: <01C1F235-999A-4D28-BA97-A42ECF22B948@lovely.email.ne.jp> Try the following .... $ erl -sasl errlog_type error Joseph Norton norton@REDACTED On Sep 1, 2011, at 3:27 AM, Dmitry Demeshchuk wrote: > That's exactly what I want to do with rabbitmq erlang client as well :( > > On Wed, Aug 31, 2011 at 9:07 PM, Joe Armstrong wrote: >> When I do >> >>> mnesia:stop(). >> >> I see this >> >> =INFO REPORT==== 31-Aug-2011::19:03:57 === >> application: mnesia >> exited: stopped >> type: temporary >> >> How can I get rid of this printout? I never want to see excess output >> if the system has correctly done what I told it to do. >> >> /Joe >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Best regards, > Dmitry Demeshchuk > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jwatte@REDACTED Thu Sep 1 02:00:06 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 31 Aug 2011 17:00:06 -0700 Subject: [erlang-questions] record with dynamic item name In-Reply-To: <20110829073001.GB2394@jakstys.lt> References: <20110829073001.GB2394@jakstys.lt> Message-ID: You can also do that using record_info() without using a parse transform. record_info(name, fields) returns a list of field names, which map to their position in the tuple (remember that element 1 is the type). Use element(Tuple, N) to extract the element. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. 2011/8/29 Motiejus Jak?tys > On Sun, Aug 28, 2011 at 12:46:10PM +0200, Roberto Ostinelli wrote: > > Hi list, > > > > is there any way to retrieve a record item whose identifier is in a > > variable? For instance: > > > > -record(state, { > > type = 1 > > }). > > > > Ulf Wiger has done this job in parse_trans package: > https://github.com/uwiger/parse_trans > > Documentation here: > http://forum.trapexit.org/viewtopic.php?p=21790 > > Basically looks like this: > -compile({parse_transform, exprecs}). > -export_records(['r']). > -record(r, {a, b}). > > imho(MyField) -> > R = r{a = yadda, b = yabba}, > '#get-'(MyField, R). > > main() -> > imho(a). %% Returns yadda > > Motiejus > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Thu Sep 1 02:06:57 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 31 Aug 2011 17:06:57 -0700 Subject: [erlang-questions] REST interface to mnesia In-Reply-To: References: <698733B7-3FBC-418F-88A8-942660F7F11B@hypotheticalabs.com> Message-ID: Almost any data store in the world can do 5k reads/writes per second if you go to solid state disks. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Wed, Aug 31, 2011 at 9:58 AM, Joe Armstrong wrote: > On Wed, Aug 31, 2011 at 4:09 PM, Kevin Smith > wrote: > > How important is queryability? Would you want to use qlc, SQL, MapReduce, > or something else to write the queries? > > Not really. My order of priorities is: > > 1) Ease of install > "working out of the box" and "a few files" is important > I don't want lot's of files scattered all over the place > Entire database in one blob is nice (like SQLite) > should run forever on my machine without bothering me once installed > 2) Easy management > "it just works" - nice to have a web interface. > 3) Low storage overhead - don't be too profligate with storage > 4) Fast (ish) key-value storage. over 5K reads/writes/sec is fine > 5) Crash resiliant > 6) Can be controlled via a socket interface > 7) indexing or secondary keys > 8) full-text indexing > 9) Relational queries less important - I'm kind of used to Key-Value > queries > and designing data structures round them > > /Joe > > > > > > --Kevin > > On Aug 31, 2011, at 4:35 AM, Joe Armstrong wrote: > > > >> I want a resident Erlang database that always runs on my machine. > >> When I boot the system it's always running (like mySQL etc.) > >> > >> I want to be able to access it from any programming language. > >> > >> What I'd like > >> > >> 1) You install X > >> X might be erlang + mnesia + REST interface to mnesia > >> 2) X runs forever > >> 3) You manage X create tables etc via a web interface > >> or via REST commands or through a native-erlang interface > >> 4) You populate the tables via REST commands > >> or through a native-erlang interface > >> > >> /Joe > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From caox@REDACTED Thu Sep 1 12:01:48 2011 From: caox@REDACTED (caox) Date: Thu, 1 Sep 2011 18:01:48 +0800 Subject: [erlang-questions] erlang smp on aix Message-ID: <677D6E76-A4A8-460C-9607-4261D0F293BE@lightpole.net> hi we have a application used to run on centos but now we want to migrate it to AIX. During the test on Aix, we found the cpu scaling of the system is quite disappointing. With 8 cores and erlang smp auto, there were very high system cpu consumption and lots of thread_wait_lock call in the truss(strace) log, which would not appear on centos under the same test scenario. And the sys call will disappear if we turn smp off. So we guess there is something wrong with smp on Aix, perhaps caused by the schedulers switching. But we don't know where the thread_wait_call is from and what it is for. By the way, it is quite strange that the smp description of erl shell would be smp32:8 with smp auto on the 8 cores aix. Is there any hint to fix the problem? Appreciate your help. -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Thu Sep 1 14:17:23 2011 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 01 Sep 2011 13:17:23 +0100 (BST) Subject: [erlang-questions] erlang smp on aix In-Reply-To: <677D6E76-A4A8-460C-9607-4261D0F293BE@lightpole.net> Message-ID: <402ff4af-ab62-4c7c-a8dc-e5727274fe3e@knuth> Which version of Erlang ? ----- Original Message ----- > hi > we have a application used to run on centos but now we want to > migrate it to AIX. During the test on Aix, we found the cpu scaling > of the system is quite disappointing. With 8 cores and erlang smp > auto, there were very high system cpu consumption and lots of > thread_wait_lock call in the truss(strace) log, which would not > appear on centos under the same test scenario. And the sys call will > disappear if we turn smp off. > So we guess there is something wrong with smp on Aix, perhaps caused > by the schedulers switching. But we don't know where the > thread_wait_call is from and what it is for. > By the way, it is quite strange that the smp description of erl shell > would be smp32:8 with smp auto on the 8 cores aix. > Is there any hint to fix the problem? Appreciate your help. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Sep 1 14:57:06 2011 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 1 Sep 2011 14:57:06 +0200 Subject: [erlang-questions] how can I make mnesia/sasl keep quiet In-Reply-To: <01C1F235-999A-4D28-BA97-A42ECF22B948@lovely.email.ne.jp> References: <01C1F235-999A-4D28-BA97-A42ECF22B948@lovely.email.ne.jp> Message-ID: On Thu, Sep 1, 2011 at 1:30 AM, Joseph Norton wrote: > > Try the following .... > > $ erl -sasl errlog_type error > Thanks Any idea if this also be done if I start with $erl I prefer to do *everything* by calling functions and not with command line arguments /Joe > Joseph Norton > norton@REDACTED > > > > On Sep 1, 2011, at 3:27 AM, Dmitry Demeshchuk wrote: > >> That's exactly what I want to do with rabbitmq erlang client as well :( >> >> On Wed, Aug 31, 2011 at 9:07 PM, Joe Armstrong wrote: >>> When I do >>> >>>> mnesia:stop(). >>> >>> I see this >>> >>> =INFO REPORT==== 31-Aug-2011::19:03:57 === >>> ? ?application: mnesia >>> ? ?exited: stopped >>> ? ?type: temporary >>> >>> How can I get rid of this printout? I never want to see excess output >>> if the system has correctly done what I told it to do. >>> >>> /Joe >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> >> -- >> Best regards, >> Dmitry Demeshchuk >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From caox@REDACTED Thu Sep 1 14:58:47 2011 From: caox@REDACTED (caox) Date: Thu, 1 Sep 2011 20:58:47 +0800 Subject: [erlang-questions] erlang smp on aix In-Reply-To: <402ff4af-ab62-4c7c-a8dc-e5727274fe3e@knuth> References: <402ff4af-ab62-4c7c-a8dc-e5727274fe3e@knuth> Message-ID: Sorry.r14b01 ???? iPhone ? 2011-9-1???8:17?Robert Virding ??? > Which version of Erlang? > > hi > > we have a application used to run on centos but now we want to migrate it to AIX. During the test on Aix, we found the cpu scaling of the system is quite disappointing. With 8 cores and erlang smp auto, there were very high system cpu consumption and lots of thread_wait_lock call in the truss(strace) log, which would not appear on centos under the same test scenario. And the sys call will disappear if we turn smp off. > So we guess there is something wrong with smp on Aix, perhaps caused by the schedulers switching. But we don't know where the thread_wait_call is from and what it is for. > By the way, it is quite strange that the smp description of erl shell would be smp32:8 with smp auto on the 8 cores aix. > Is there any hint to fix the problem? Appreciate your help. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From demeshchuk@REDACTED Thu Sep 1 14:59:31 2011 From: demeshchuk@REDACTED (Dmitry Demeshchuk) Date: Thu, 1 Sep 2011 16:59:31 +0400 Subject: [erlang-questions] how can I make mnesia/sasl keep quiet In-Reply-To: References: <01C1F235-999A-4D28-BA97-A42ECF22B948@lovely.email.ne.jp> Message-ID: Perhaps application:set_env(errlog_type, error) ? On Thu, Sep 1, 2011 at 4:57 PM, Joe Armstrong wrote: > On Thu, Sep 1, 2011 at 1:30 AM, Joseph Norton wrote: >> >> Try the following .... >> >> $ erl -sasl errlog_type error >> > > Thanks > > Any idea if this also be done if I start with > > ?$erl > > I prefer to do *everything* by calling functions and not with command > line arguments > > /Joe > > > > >> Joseph Norton >> norton@REDACTED >> >> >> >> On Sep 1, 2011, at 3:27 AM, Dmitry Demeshchuk wrote: >> >>> That's exactly what I want to do with rabbitmq erlang client as well :( >>> >>> On Wed, Aug 31, 2011 at 9:07 PM, Joe Armstrong wrote: >>>> When I do >>>> >>>>> mnesia:stop(). >>>> >>>> I see this >>>> >>>> =INFO REPORT==== 31-Aug-2011::19:03:57 === >>>> ? ?application: mnesia >>>> ? ?exited: stopped >>>> ? ?type: temporary >>>> >>>> How can I get rid of this printout? I never want to see excess output >>>> if the system has correctly done what I told it to do. >>>> >>>> /Joe >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> >>> >>> -- >>> Best regards, >>> Dmitry Demeshchuk >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- Best regards, Dmitry Demeshchuk From fred.hebert@REDACTED Thu Sep 1 15:01:31 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Thu, 1 Sep 2011 09:01:31 -0400 Subject: [erlang-questions] how can I make mnesia/sasl keep quiet In-Reply-To: References: <01C1F235-999A-4D28-BA97-A42ECF22B948@lovely.email.ne.jp> Message-ID: You need to do application:set_env(sasl, errlog_type, Type). The function with only 2 arguments will do it to the application where the current process belongs -- it won't work if you're not running as a part of the SASL process group. Force it with 3 arguments, where the first one is the name of the application. -- Fred H?bert http://www.erlang-solutions.com On 2011-09-01, at 08:59 AM, Dmitry Demeshchuk wrote: > Perhaps application:set_env(errlog_type, error) ? > > On Thu, Sep 1, 2011 at 4:57 PM, Joe Armstrong wrote: >> On Thu, Sep 1, 2011 at 1:30 AM, Joseph Norton wrote: >>> >>> Try the following .... >>> >>> $ erl -sasl errlog_type error >>> >> >> Thanks >> >> Any idea if this also be done if I start with >> >> $erl >> >> I prefer to do *everything* by calling functions and not with command >> line arguments >> >> /Joe >> >> >> >> >>> Joseph Norton >>> norton@REDACTED >>> >>> >>> >>> On Sep 1, 2011, at 3:27 AM, Dmitry Demeshchuk wrote: >>> >>>> That's exactly what I want to do with rabbitmq erlang client as well :( >>>> >>>> On Wed, Aug 31, 2011 at 9:07 PM, Joe Armstrong wrote: >>>>> When I do >>>>> >>>>>> mnesia:stop(). >>>>> >>>>> I see this >>>>> >>>>> =INFO REPORT==== 31-Aug-2011::19:03:57 === >>>>> application: mnesia >>>>> exited: stopped >>>>> type: temporary >>>>> >>>>> How can I get rid of this printout? I never want to see excess output >>>>> if the system has correctly done what I told it to do. >>>>> >>>>> /Joe >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> >>>> >>>> >>>> -- >>>> Best regards, >>>> Dmitry Demeshchuk >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > > > > -- > Best regards, > Dmitry Demeshchuk > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From nick@REDACTED Thu Sep 1 17:04:56 2011 From: nick@REDACTED (Niclas Eklund) Date: Thu, 1 Sep 2011 17:04:56 +0200 Subject: [erlang-questions] how can I make mnesia/sasl keep quiet In-Reply-To: References: <01C1F235-999A-4D28-BA97-A42ECF22B948@lovely.email.ne.jp> Message-ID: Hi! Usually you should avoid application:set_env/3 if it's not your own application. From the reference manual: "Warning: Use this function only if you know what you are doing, that is, on your own applications. It is very application and configuration parameter dependent when and how often the value is read by the application, and careless use of this function may put the application in a weird, inconsistent, and malfunctioning state." Perhaps it works now, depending how SASL handles the parameter, but that might change at any given time without a warning. Best regards, Niclas @ Erlang/OTP On Thu, 1 Sep 2011, Fr?d?ric Trottier-H?bert wrote: > You need to do application:set_env(sasl, errlog_type, Type). The > function with only 2 arguments will do it to the application where the > current process belongs -- it won't work if you're not running as a part > of the SASL process group. Force it with 3 arguments, where the first > one is the name of the application. > > -- > Fred H?bert > http://www.erlang-solutions.com > > > > On 2011-09-01, at 08:59 AM, Dmitry Demeshchuk wrote: > >> Perhaps application:set_env(errlog_type, error) ? >> >> On Thu, Sep 1, 2011 at 4:57 PM, Joe Armstrong wrote: >>> On Thu, Sep 1, 2011 at 1:30 AM, Joseph Norton wrote: >>>> >>>> Try the following .... >>>> >>>> $ erl -sasl errlog_type error >>>> >>> >>> Thanks >>> >>> Any idea if this also be done if I start with >>> >>> $erl >>> >>> I prefer to do *everything* by calling functions and not with command >>> line arguments >>> >>> /Joe >>> >>> >>> >>> >>>> Joseph Norton >>>> norton@REDACTED >>>> >>>> >>>> >>>> On Sep 1, 2011, at 3:27 AM, Dmitry Demeshchuk wrote: >>>> >>>>> That's exactly what I want to do with rabbitmq erlang client as well :( >>>>> >>>>> On Wed, Aug 31, 2011 at 9:07 PM, Joe Armstrong wrote: >>>>>> When I do >>>>>> >>>>>>> mnesia:stop(). >>>>>> >>>>>> I see this >>>>>> >>>>>> =INFO REPORT==== 31-Aug-2011::19:03:57 === >>>>>> application: mnesia >>>>>> exited: stopped >>>>>> type: temporary >>>>>> >>>>>> How can I get rid of this printout? I never want to see excess output >>>>>> if the system has correctly done what I told it to do. >>>>>> >>>>>> /Joe >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Best regards, >>>>> Dmitry Demeshchuk >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >> >> >> >> -- >> Best regards, >> Dmitry Demeshchuk >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From fred.hebert@REDACTED Thu Sep 1 17:09:11 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Thu, 1 Sep 2011 11:09:11 -0400 Subject: [erlang-questions] how can I make mnesia/sasl keep quiet In-Reply-To: References: <01C1F235-999A-4D28-BA97-A42ECF22B948@lovely.email.ne.jp> Message-ID: <7760120C-F751-493A-B0EB-5226F17F7805@erlang-solutions.com> True, but otherwise, the call to application:set_env/2 will just not work. Still a good warning to give, yes. -- Fred H?bert http://www.erlang-solutions.com On 2011-09-01, at 11:04 AM, Niclas Eklund wrote: > > Hi! > > Usually you should avoid application:set_env/3 if it's not your own application. > > From the reference manual: > > "Warning: > Use this function only if you know what you are doing, that is, on your own applications. It is very application and configuration parameter dependent when and how often the value is read by the application, and careless use of this function may put the application in a weird, inconsistent, and malfunctioning state." > > Perhaps it works now, depending how SASL handles the parameter, but that might change at any given time without a warning. > > Best regards, > > Niclas @ Erlang/OTP > > On Thu, 1 Sep 2011, Fr?d?ric Trottier-H?bert wrote: > >> You need to do application:set_env(sasl, errlog_type, Type). The function with only 2 arguments will do it to the application where the current process belongs -- it won't work if you're not running as a part of the SASL process group. Force it with 3 arguments, where the first one is the name of the application. >> >> -- >> Fred H?bert >> http://www.erlang-solutions.com >> >> >> >> On 2011-09-01, at 08:59 AM, Dmitry Demeshchuk wrote: >> >>> Perhaps application:set_env(errlog_type, error) ? >>> >>> On Thu, Sep 1, 2011 at 4:57 PM, Joe Armstrong wrote: >>>> On Thu, Sep 1, 2011 at 1:30 AM, Joseph Norton wrote: >>>>> >>>>> Try the following .... >>>>> >>>>> $ erl -sasl errlog_type error >>>>> >>>> >>>> Thanks >>>> >>>> Any idea if this also be done if I start with >>>> >>>> $erl >>>> >>>> I prefer to do *everything* by calling functions and not with command >>>> line arguments >>>> >>>> /Joe >>>> >>>> >>>> >>>> >>>>> Joseph Norton >>>>> norton@REDACTED >>>>> >>>>> >>>>> >>>>> On Sep 1, 2011, at 3:27 AM, Dmitry Demeshchuk wrote: >>>>> >>>>>> That's exactly what I want to do with rabbitmq erlang client as well :( >>>>>> >>>>>> On Wed, Aug 31, 2011 at 9:07 PM, Joe Armstrong wrote: >>>>>>> When I do >>>>>>> >>>>>>>> mnesia:stop(). >>>>>>> >>>>>>> I see this >>>>>>> >>>>>>> =INFO REPORT==== 31-Aug-2011::19:03:57 === >>>>>>> application: mnesia >>>>>>> exited: stopped >>>>>>> type: temporary >>>>>>> >>>>>>> How can I get rid of this printout? I never want to see excess output >>>>>>> if the system has correctly done what I told it to do. >>>>>>> >>>>>>> /Joe >>>>>>> _______________________________________________ >>>>>>> erlang-questions mailing list >>>>>>> erlang-questions@REDACTED >>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Best regards, >>>>>> Dmitry Demeshchuk >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>> >>> >>> >>> >>> -- >>> Best regards, >>> Dmitry Demeshchuk >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From roberto@REDACTED Thu Sep 1 17:52:28 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 1 Sep 2011 17:52:28 +0200 Subject: [erlang-questions] record with dynamic item name In-Reply-To: References: <20110829073001.GB2394@jakstys.lt> Message-ID: thank you all for suggestions. i guess in this case performance jsut gets too much of a loss and i'll better simply code in 'verbose mode'. cheers, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Thu Sep 1 18:51:46 2011 From: jwatte@REDACTED (Jon Watte) Date: Thu, 1 Sep 2011 09:51:46 -0700 Subject: [erlang-questions] erlang smp on aix In-Reply-To: <677D6E76-A4A8-460C-9607-4261D0F293BE@lightpole.net> References: <677D6E76-A4A8-460C-9607-4261D0F293BE@lightpole.net> Message-ID: Did you enable kernel_poll? And does AIX actually support a suitable kernel polling mechanism (like epoll, etc)? Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Thu, Sep 1, 2011 at 3:01 AM, caox wrote: > hi > > we have a application used to run on centos but now we want to migrate it > to AIX. During the test on Aix, we found the cpu scaling of the system is > quite disappointing. With 8 cores and erlang smp auto, there were very high > system cpu consumption and lots of thread_wait_lock call in the > truss(strace) log, which would not appear on centos under the same test > scenario. And the sys call will disappear if we turn smp off. > So we guess there is something wrong with smp on Aix, perhaps caused by > the schedulers switching. But we don't know where the thread_wait_call is > from and what it is for. > By the way, it is quite strange that the smp description of erl shell would > be smp32:8 with smp auto on the 8 cores aix. > Is there any hint to fix the problem? Appreciate your help. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Thu Sep 1 18:54:35 2011 From: jwatte@REDACTED (Jon Watte) Date: Thu, 1 Sep 2011 09:54:35 -0700 Subject: [erlang-questions] how can I make mnesia/sasl keep quiet In-Reply-To: <7760120C-F751-493A-B0EB-5226F17F7805@erlang-solutions.com> References: <01C1F235-999A-4D28-BA97-A42ECF22B948@lovely.email.ne.jp> <7760120C-F751-493A-B0EB-5226F17F7805@erlang-solutions.com> Message-ID: I think that, as long as you call application:set_env(A,K,V) before you start the application A, you're probably safe. However, for some applications, this may be hard to accomplish. For example, if you use -boot start_sasl, then sasl will have started by the time you get to the interactive prompt. Maybe building your own boot script that configures this would be a suitable alternative? That would also work for non-interactive nodes (auto-start in /etc/init.d, etc) Sincerely, Jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. 2011/9/1 Fr?d?ric Trottier-H?bert > True, but otherwise, the call to application:set_env/2 will just not work. > Still a good warning to give, yes. > > -- > Fred H?bert > http://www.erlang-solutions.com > > > > On 2011-09-01, at 11:04 AM, Niclas Eklund wrote: > > > > > Hi! > > > > Usually you should avoid application:set_env/3 if it's not your own > application. > > > > From the reference manual: > > > > "Warning: > > Use this function only if you know what you are doing, that is, on > your own applications. It is very application and configuration parameter > dependent when and how often the value is read by the application, and > careless use of this function may put the application in a weird, > inconsistent, and malfunctioning state." > > > > Perhaps it works now, depending how SASL handles the parameter, but that > might change at any given time without a warning. > > > > Best regards, > > > > Niclas @ Erlang/OTP > > > > On Thu, 1 Sep 2011, Fr?d?ric Trottier-H?bert wrote: > > > >> You need to do application:set_env(sasl, errlog_type, Type). The > function with only 2 arguments will do it to the application where the > current process belongs -- it won't work if you're not running as a part of > the SASL process group. Force it with 3 arguments, where the first one is > the name of the application. > >> > >> -- > >> Fred H?bert > >> http://www.erlang-solutions.com > >> > >> > >> > >> On 2011-09-01, at 08:59 AM, Dmitry Demeshchuk wrote: > >> > >>> Perhaps application:set_env(errlog_type, error) ? > >>> > >>> On Thu, Sep 1, 2011 at 4:57 PM, Joe Armstrong > wrote: > >>>> On Thu, Sep 1, 2011 at 1:30 AM, Joseph Norton < > norton@REDACTED> wrote: > >>>>> > >>>>> Try the following .... > >>>>> > >>>>> $ erl -sasl errlog_type error > >>>>> > >>>> > >>>> Thanks > >>>> > >>>> Any idea if this also be done if I start with > >>>> > >>>> $erl > >>>> > >>>> I prefer to do *everything* by calling functions and not with command > >>>> line arguments > >>>> > >>>> /Joe > >>>> > >>>> > >>>> > >>>> > >>>>> Joseph Norton > >>>>> norton@REDACTED > >>>>> > >>>>> > >>>>> > >>>>> On Sep 1, 2011, at 3:27 AM, Dmitry Demeshchuk wrote: > >>>>> > >>>>>> That's exactly what I want to do with rabbitmq erlang client as well > :( > >>>>>> > >>>>>> On Wed, Aug 31, 2011 at 9:07 PM, Joe Armstrong > wrote: > >>>>>>> When I do > >>>>>>> > >>>>>>>> mnesia:stop(). > >>>>>>> > >>>>>>> I see this > >>>>>>> > >>>>>>> =INFO REPORT==== 31-Aug-2011::19:03:57 === > >>>>>>> application: mnesia > >>>>>>> exited: stopped > >>>>>>> type: temporary > >>>>>>> > >>>>>>> How can I get rid of this printout? I never want to see excess > output > >>>>>>> if the system has correctly done what I told it to do. > >>>>>>> > >>>>>>> /Joe > >>>>>>> _______________________________________________ > >>>>>>> erlang-questions mailing list > >>>>>>> erlang-questions@REDACTED > >>>>>>> http://erlang.org/mailman/listinfo/erlang-questions > >>>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Best regards, > >>>>>> Dmitry Demeshchuk > >>>>>> _______________________________________________ > >>>>>> erlang-questions mailing list > >>>>>> erlang-questions@REDACTED > >>>>>> http://erlang.org/mailman/listinfo/erlang-questions > >>>>> > >>>>> > >>>> > >>> > >>> > >>> > >>> -- > >>> Best regards, > >>> Dmitry Demeshchuk > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Thu Sep 1 19:05:47 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 1 Sep 2011 19:05:47 +0200 Subject: [erlang-questions] how can I make mnesia/sasl keep quiet In-Reply-To: References: <01C1F235-999A-4D28-BA97-A42ECF22B948@lovely.email.ne.jp> Message-ID: For those who want to change application environment variables in running systems, I recommend taking a look at gproc's support for environment variables. Basically, gproc helps with a few different problems with configuration environments: - there are so many ways to specify parameters: unix env vars, OTP env vars, mnesia, ? Gproc handles a number of them and caches the read value as a gproc property. - Updating the value is correspondingly supported. - If you want to notify all others who have read (cached) the same variable, just use gproc's pub/sub pattern. The other processes will have to take care of updating their own cached value*. - when debugging a process, its 'gproc_env' properties show which values it relies on. If a process failed to refresh its value, this will be apparent using a simple query against gproc. https://github.com/esl/gproc/blob/master/test/gproc_tests.erl#L395 for some illustration (unit test cases) BR, Ulf W * gproc:set_env(Scope, App, Key, Value, []) will accomplish this. On 1 Sep 2011, at 17:04, Niclas Eklund wrote: > > Hi! > > Usually you should avoid application:set_env/3 if it's not your own application. > > From the reference manual: > > "Warning: > Use this function only if you know what you are doing, that is, on your own applications. It is very application and configuration parameter dependent when and how often the value is read by the application, and careless use of this function may put the application in a weird, inconsistent, and malfunctioning state." > > Perhaps it works now, depending how SASL handles the parameter, but that might change at any given time without a warning. > > Best regards, > > Niclas @ Erlang/OTP > > On Thu, 1 Sep 2011, Fr?d?ric Trottier-H?bert wrote: > >> You need to do application:set_env(sasl, errlog_type, Type). The function with only 2 arguments will do it to the application where the current process belongs -- it won't work if you're not running as a part of the SASL process group. Force it with 3 arguments, where the first one is the name of the application. >> >> -- >> Fred H?bert >> http://www.erlang-solutions.com >> >> >> >> On 2011-09-01, at 08:59 AM, Dmitry Demeshchuk wrote: >> >>> Perhaps application:set_env(errlog_type, error) ? >>> >>> On Thu, Sep 1, 2011 at 4:57 PM, Joe Armstrong wrote: >>>> On Thu, Sep 1, 2011 at 1:30 AM, Joseph Norton wrote: >>>>> >>>>> Try the following .... >>>>> >>>>> $ erl -sasl errlog_type error >>>>> >>>> >>>> Thanks >>>> >>>> Any idea if this also be done if I start with >>>> >>>> $erl >>>> >>>> I prefer to do *everything* by calling functions and not with command >>>> line arguments >>>> >>>> /Joe >>>> >>>> >>>> >>>> >>>>> Joseph Norton >>>>> norton@REDACTED >>>>> >>>>> >>>>> >>>>> On Sep 1, 2011, at 3:27 AM, Dmitry Demeshchuk wrote: >>>>> >>>>>> That's exactly what I want to do with rabbitmq erlang client as well :( >>>>>> >>>>>> On Wed, Aug 31, 2011 at 9:07 PM, Joe Armstrong wrote: >>>>>>> When I do >>>>>>> >>>>>>>> mnesia:stop(). >>>>>>> >>>>>>> I see this >>>>>>> >>>>>>> =INFO REPORT==== 31-Aug-2011::19:03:57 === >>>>>>> application: mnesia >>>>>>> exited: stopped >>>>>>> type: temporary >>>>>>> >>>>>>> How can I get rid of this printout? I never want to see excess output >>>>>>> if the system has correctly done what I told it to do. >>>>>>> >>>>>>> /Joe >>>>>>> _______________________________________________ >>>>>>> erlang-questions mailing list >>>>>>> erlang-questions@REDACTED >>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Best regards, >>>>>> Dmitry Demeshchuk >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>> >>> >>> >>> >>> -- >>> Best regards, >>> Dmitry Demeshchuk >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From gnoblin@REDACTED Thu Sep 1 19:12:31 2011 From: gnoblin@REDACTED (Slav Pankratov) Date: Thu, 1 Sep 2011 20:12:31 +0300 Subject: [erlang-questions] misultin / unicode Message-ID: Hello! I'm using misultin to receive http calls, and they sometimes contain Russian symbols (unicode). ** > > % handle a GET on /users/{username} > handle('GET', ["users", UserName], Req) -> > io:format("~s~n", [UserName]), > Req:ok([{"Content-Type", "text/plain"}], "This is ~s's page.", [UserName]); In both terminal and browser I see incorrect output - what can be done about it? big thanks, Slav -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Thu Sep 1 22:21:52 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 1 Sep 2011 22:21:52 +0200 Subject: [erlang-questions] misultin / unicode In-Reply-To: References: Message-ID: 2011/9/1 Slav Pankratov > Hello! > > I'm using misultin to receive http calls, and they sometimes contain > Russian symbols (unicode). > > ** >> >> % handle a GET on /users/{username} >> handle('GET', ["users", UserName], Req) -> >> io:format("~s~n", [UserName]), >> Req:ok([{"Content-Type", "text/plain"}], "This is ~s's page.", [UserName]); > > > In both terminal and browser I see incorrect output - what can be done about it? > > big thanks, > Slav > > hi slav, this is not particularly related to misultin. when using uri you need to ensure that these are properly encoded for HTTP. You also need to ensure that the browser is rendering html encoded in UTF-8 so you should have in your code: % handle a GET on /users/{username} handle('GET', ["users", UserName], Req) -> Req:ok([{"Content-Type", "text/html; charset=UTF-8"}], "This is ~s's page.", [UserName]); then, you may serve the page: http://localhost:8080/users/%26%231055%3B%26%231088%3B%26%231080%3B%26%231074%3B%26%231077%3B%26%231090%3B that will print; This is ??????'s page. you probably also could read into IRI: http://tools.ietf.org/html/rfc3987 my 2c. r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sam@REDACTED Thu Sep 1 22:23:50 2011 From: sam@REDACTED (Samuel Elliott) Date: Thu, 1 Sep 2011 21:23:50 +0100 Subject: [erlang-questions] how can I make mnesia/sasl keep quiet In-Reply-To: References: <01C1F235-999A-4D28-BA97-A42ECF22B948@lovely.email.ne.jp> Message-ID: Does it count to put it in an app.config file that is referenced with $erl -config var/app ? yeah, essentially the same thing, but easier to version control and you can be sure its read at startup, which seems to be the issue mentioned to do with application:set_env/3 Sam On Thu, Sep 1, 2011 at 1:57 PM, Joe Armstrong wrote: > On Thu, Sep 1, 2011 at 1:30 AM, Joseph Norton > wrote: > > > > Try the following .... > > > > $ erl -sasl errlog_type error > > > > Thanks > > Any idea if this also be done if I start with > > $erl > > I prefer to do *everything* by calling functions and not with command > line arguments > > /Joe > > > > > > Joseph Norton > > norton@REDACTED > > > > > > > > On Sep 1, 2011, at 3:27 AM, Dmitry Demeshchuk wrote: > > > >> That's exactly what I want to do with rabbitmq erlang client as well :( > >> > >> On Wed, Aug 31, 2011 at 9:07 PM, Joe Armstrong > wrote: > >>> When I do > >>> > >>>> mnesia:stop(). > >>> > >>> I see this > >>> > >>> =INFO REPORT==== 31-Aug-2011::19:03:57 === > >>> application: mnesia > >>> exited: stopped > >>> type: temporary > >>> > >>> How can I get rid of this printout? I never want to see excess output > >>> if the system has correctly done what I told it to do. > >>> > >>> /Joe > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >>> > >> > >> > >> > >> -- > >> Best regards, > >> Dmitry Demeshchuk > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Samuel Elliott sam@REDACTED http://lenary.co.uk/ +44 (0)7891 993 664 -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangdlf@REDACTED Fri Sep 2 03:40:44 2011 From: erlangdlf@REDACTED (deng lifen) Date: Fri, 2 Sep 2011 09:40:44 +0800 Subject: [erlang-questions] erlang smp on aix In-Reply-To: References: <677D6E76-A4A8-460C-9607-4261D0F293BE@lightpole.net> Message-ID: I get this when starting yaws on AIX. kernel_poll not supported: "K" parameter ignored. ? 2011-9-2???12:51? Jon Watte ??? > Did you enable kernel_poll? And does AIX actually support a suitable kernel polling mechanism (like epoll, etc)? > > Sincerely, > > jw > > > -- > Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. > > > > On Thu, Sep 1, 2011 at 3:01 AM, caox wrote: > hi > > we have a application used to run on centos but now we want to migrate it to AIX. During the test on Aix, we found the cpu scaling of the system is quite disappointing. With 8 cores and erlang smp auto, there were very high system cpu consumption and lots of thread_wait_lock call in the truss(strace) log, which would not appear on centos under the same test scenario. And the sys call will disappear if we turn smp off. > So we guess there is something wrong with smp on Aix, perhaps caused by the schedulers switching. But we don't know where the thread_wait_call is from and what it is for. > By the way, it is quite strange that the smp description of erl shell would be smp32:8 with smp auto on the 8 cores aix. > Is there any hint to fix the problem? Appreciate your help. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions deng lifen erlangdlf@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From caox@REDACTED Fri Sep 2 03:42:51 2011 From: caox@REDACTED (caox) Date: Fri, 2 Sep 2011 09:42:51 +0800 Subject: [erlang-questions] erlang smp on aix In-Reply-To: References: <677D6E76-A4A8-460C-9607-4261D0F293BE@lightpole.net> Message-ID: <50882B59-F1AE-41BD-8514-9543863EF272@lightpole.net> No, the kernel poll is not supported on AIX. AIX has pollset interface like epoll but seems can not be used by erts. Is this the reason of thread_waitlock? Our test scenario is quite simple. Create 1000 erlang processes or more to execute our main function repeatedly. This function do nothing special but call a linkded-in driver with port_control/3 and do some template rendering then return. The whole function does not have any IO operations. During the test on a 2 cores AIX, we found no thread_waitlock() call in the strace log, which was full of poll() call. With same test scenario on a 4 cores Centos with kernel-poll off, there is no sys call in the strace log(even without poll() call). Cpu will be consumed perfectly and we can gain quite high system throughput. But when migrating to 4cores or 8cores AIX, thread_watilock() appears and there is lots of context switches according to vmstat. As a result, the throughput drop dramatically compared with Centos. So, what is role of poll() in the scenario, or in the erlang processes scheduling? BR ? 2011-9-2???12:51? Jon Watte ??? > Did you enable kernel_poll? And does AIX actually support a suitable kernel polling mechanism (like epoll, etc)? > > Sincerely, > > jw > > > -- > Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. > > > > On Thu, Sep 1, 2011 at 3:01 AM, caox wrote: > hi > > we have a application used to run on centos but now we want to migrate it to AIX. During the test on Aix, we found the cpu scaling of the system is quite disappointing. With 8 cores and erlang smp auto, there were very high system cpu consumption and lots of thread_wait_lock call in the truss(strace) log, which would not appear on centos under the same test scenario. And the sys call will disappear if we turn smp off. > So we guess there is something wrong with smp on Aix, perhaps caused by the schedulers switching. But we don't know where the thread_wait_call is from and what it is for. > By the way, it is quite strange that the smp description of erl shell would be smp32:8 with smp auto on the 8 cores aix. > Is there any hint to fix the problem? Appreciate your help. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vinoski@REDACTED Fri Sep 2 03:50:35 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Thu, 1 Sep 2011 21:50:35 -0400 Subject: [erlang-questions] erlang smp on aix In-Reply-To: References: <677D6E76-A4A8-460C-9607-4261D0F293BE@lightpole.net> Message-ID: 2011/9/1 deng lifen : > I get this when starting yaws on AIX. > kernel_poll not supported: "K" parameter ignored. Pass the --disable-kpoll option to yaws to avoid this. --steve From pfisher@REDACTED Fri Sep 2 04:00:19 2011 From: pfisher@REDACTED (Paul Fisher) Date: Thu, 1 Sep 2011 21:00:19 -0500 Subject: [erlang-questions] building erlang/otp dev on mac os x? Message-ID: I get roughly the following for both 32-bit and 64-bit builds on lion. Has anyone specifically diagnosed the issue building current releases up to, and including the current dev branch, on mac os x? Is this considered an gcc-llvm or otp issue? $ ./configure --enable-darwin-64bit --disable-hipe $ make === Entering application hipe erlc -W +debug_info +warn_exported_vars +warn_missing_spec +warn_untyped_record -o../ebin hipe_consttab.erl (no error logger present) error: "Error in process <0.1.0> with exit value: {{badfun,[<<5 bytes>>,<<36 bytes>>,<<9 bytes>>,<<3 bytes>>,<<2 bytes>>,<<5 bytes>>,<<14 bytes>>,<<2 bytes>>,<<8 bytes>>,<<8 bytes>>,<<5 bytes>>,<<7 bytes>>,<<5 bytes>>,<<11 bytes>>,<<2 bytes>>,<<11 bytes>>,<<15 bytes>>,<<4 bytes>>,<<40 bytes>>,<<5 bytes>>,<<1 byte>>,<<7 bytes>>,<<10 bytes>>,<<7 bytes>>,<<18 bytes>>,<<7 bytes>>,<<17 bytes>>,<<7 bytes>>,<<19 bytes>>,<<7 bytes>>,<<7 bytes>>,<<6 bytes>>,<<17 bytes>>]},[{erlang,apply,2}]}\n" ^C $ gcc --version i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00) Copyright (C) 2007 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. -- paul director, platform services alertlogic, inc. 713-484-8383 x2314 From dizzyd@REDACTED Fri Sep 2 04:21:50 2011 From: dizzyd@REDACTED (Dave Smith) Date: Thu, 1 Sep 2011 20:21:50 -0600 Subject: [erlang-questions] building erlang/otp dev on mac os x? In-Reply-To: References: Message-ID: LLVM on Lion appears to generate bad code when used with default optimizations. Use: CFLAGS=-O0 ./configure .... Be sure to start from a fresh unpack of OTP source too... D. On Thu, Sep 1, 2011 at 8:00 PM, Paul Fisher wrote: > I get roughly the following for both 32-bit and 64-bit builds on lion. ?Has anyone specifically diagnosed the issue building current releases up to, and including the current dev branch, on mac os x? ?Is this considered an gcc-llvm or otp issue? > > $ ./configure --enable-darwin-64bit --disable-hipe > $ make > > > > === Entering application hipe > erlc -W ?+debug_info +warn_exported_vars +warn_missing_spec +warn_untyped_record -o../ebin hipe_consttab.erl > (no error logger present) error: "Error in process <0.1.0> with exit value: {{badfun,[<<5 bytes>>,<<36 bytes>>,<<9 bytes>>,<<3 bytes>>,<<2 bytes>>,<<5 bytes>>,<<14 bytes>>,<<2 bytes>>,<<8 bytes>>,<<8 bytes>>,<<5 bytes>>,<<7 bytes>>,<<5 bytes>>,<<11 bytes>>,<<2 bytes>>,<<11 bytes>>,<<15 bytes>>,<<4 bytes>>,<<40 bytes>>,<<5 bytes>>,<<1 byte>>,<<7 bytes>>,<<10 bytes>>,<<7 bytes>>,<<18 bytes>>,<<7 bytes>>,<<17 bytes>>,<<7 bytes>>,<<19 bytes>>,<<7 bytes>>,<<7 bytes>>,<<6 bytes>>,<<17 bytes>>]},[{erlang,apply,2}]}\n" > ^C > > > $ gcc --version > i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00) > Copyright (C) 2007 Free Software Foundation, Inc. > This is free software; see the source for copying conditions. ?There is NO > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. > > > -- > paul > > director, platform services > alertlogic, inc. > 713-484-8383 x2314 > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From jwatte@REDACTED Fri Sep 2 09:35:55 2011 From: jwatte@REDACTED (Jon Watte) Date: Fri, 2 Sep 2011 00:35:55 -0700 Subject: [erlang-questions] erlang smp on aix In-Reply-To: <50882B59-F1AE-41BD-8514-9543863EF272@lightpole.net> References: <677D6E76-A4A8-460C-9607-4261D0F293BE@lightpole.net> <50882B59-F1AE-41BD-8514-9543863EF272@lightpole.net> Message-ID: I haven't kept up on the latest in AIX kernel development, but it seems that IBM on POWER worries more about virtualizing processors across multiple partitioned workloads, rather than the best possible API performance for a unified machine view. It's possible that the AIX kernel does not have the kind of primitives that Erlang needs to run well in highly parallel situations. When I worked with them, IBM technical support and ISV support was fairly well educated, and you could possible send an strace or a copy of your program to them to ask them what they think and get a real answer (as opposed to some other IHVs out there...) Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. 2011/9/1 caox > No, the kernel poll is not supported on AIX. > > AIX has pollset interface like epoll but seems can not be used by erts. Is > this the reason of thread_waitlock? > > Our test scenario is quite simple. Create 1000 erlang processes or more to > execute our main function repeatedly. This function do nothing special but > call a linkded-in driver with port_control/3 and do some template rendering > then return. The whole function does not have any IO operations. > > During the test on a 2 cores AIX, we found no thread_waitlock() call in > the strace log, which was full of poll() call. > > With same test scenario on a 4 cores Centos with kernel-poll off, there is > no sys call in the strace log(even without poll() call). Cpu will be > consumed perfectly and we can gain quite high system throughput. > > But when migrating to 4cores or 8cores AIX, thread_watilock() appears and > there is lots of context switches according to vmstat. As a result, the > throughput drop dramatically compared with Centos. > > So, what is role of poll() in the scenario, or in the erlang processes > scheduling? > > BR > > > ? 2011-9-2???12:51? Jon Watte ??? > > Did you enable kernel_poll? And does AIX actually support a suitable kernel > polling mechanism (like epoll, etc)? > > Sincerely, > > jw > > > -- > Americans might object: there is no way we would sacrifice our living > standards for the benefit of people in the rest of the world. Nevertheless, > whether we get there willingly or not, we shall soon have lower consumption > rates, because our present rates are unsustainable. > > > > On Thu, Sep 1, 2011 at 3:01 AM, caox wrote: > >> hi >> >> we have a application used to run on centos but now we want to migrate it >> to AIX. During the test on Aix, we found the cpu scaling of the system is >> quite disappointing. With 8 cores and erlang smp auto, there were very high >> system cpu consumption and lots of thread_wait_lock call in the >> truss(strace) log, which would not appear on centos under the same test >> scenario. And the sys call will disappear if we turn smp off. >> So we guess there is something wrong with smp on Aix, perhaps caused by >> the schedulers switching. But we don't know where the thread_wait_call is >> from and what it is for. >> By the way, it is quite strange that the smp description of erl shell >> would be smp32:8 with smp auto on the 8 cores aix. >> Is there any hint to fix the problem? Appreciate your help. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michal.niec@REDACTED Fri Sep 2 10:23:34 2011 From: michal.niec@REDACTED (Michal Niec) Date: Fri, 2 Sep 2011 10:23:34 +0200 Subject: [erlang-questions] building erlang/otp dev on mac os x? In-Reply-To: References: Message-ID: <311F0F49-D6DC-41A4-8052-6A99E5CE3245@erlang-solutions.com> You can also download the 64 bit Mac OS X Lion package from Erlang Solutions website: http://www.erlang-solutions.com/section/98/downloads/ . It was built using CFLAGS=-O0 ./configure --enable-darwin-64bit. You can also view the OTP test suite results for the package to decide if it will fulfill your requirements. Best regards, Michal Niec On Sep 2, 2011, at 4:21 AM, Dave Smith wrote: > LLVM on Lion appears to generate bad code when used with default > optimizations. Use: > > CFLAGS=-O0 ./configure .... > > Be sure to start from a fresh unpack of OTP source too... > > D. > > On Thu, Sep 1, 2011 at 8:00 PM, Paul Fisher wrote: >> I get roughly the following for both 32-bit and 64-bit builds on lion. Has anyone specifically diagnosed the issue building current releases up to, and including the current dev branch, on mac os x? Is this considered an gcc-llvm or otp issue? >> >> $ ./configure --enable-darwin-64bit --disable-hipe >> $ make >> >> >> >> === Entering application hipe >> erlc -W +debug_info +warn_exported_vars +warn_missing_spec +warn_untyped_record -o../ebin hipe_consttab.erl >> (no error logger present) error: "Error in process <0.1.0> with exit value: {{badfun,[<<5 bytes>>,<<36 bytes>>,<<9 bytes>>,<<3 bytes>>,<<2 bytes>>,<<5 bytes>>,<<14 bytes>>,<<2 bytes>>,<<8 bytes>>,<<8 bytes>>,<<5 bytes>>,<<7 bytes>>,<<5 bytes>>,<<11 bytes>>,<<2 bytes>>,<<11 bytes>>,<<15 bytes>>,<<4 bytes>>,<<40 bytes>>,<<5 bytes>>,<<1 byte>>,<<7 bytes>>,<<10 bytes>>,<<7 bytes>>,<<18 bytes>>,<<7 bytes>>,<<17 bytes>>,<<7 bytes>>,<<19 bytes>>,<<7 bytes>>,<<7 bytes>>,<<6 bytes>>,<<17 bytes>>]},[{erlang,apply,2}]}\n" >> ^C >> >> >> $ gcc --version >> i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00) >> Copyright (C) 2007 Free Software Foundation, Inc. >> This is free software; see the source for copying conditions. There is NO >> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. >> >> >> -- >> paul >> >> director, platform services >> alertlogic, inc. >> 713-484-8383 x2314 >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From antoine.koener@REDACTED Fri Sep 2 10:57:30 2011 From: antoine.koener@REDACTED (Antoine Koener) Date: Fri, 2 Sep 2011 10:57:30 +0200 Subject: [erlang-questions] misultin / unicode In-Reply-To: References: Message-ID: On Thu, Sep 1, 2011 at 10:21 PM, Roberto Ostinelli wrote: > > > 2011/9/1 Slav Pankratov >> >> Hello! >> >> >> >> >> I'm using misultin to receive http calls, and they sometimes contain >> Russian symbols (unicode). >> >> >>> >>> >>> % handle a GET on /users/{username} >>> >>> >>> handle('GET', ["users", UserName], Req) -> >>> >>> >>> io:format("~s~n", [UserName]), I may be completely wrong but: try with ~ts instead of ~s in your io:format >>> >>> >>> Req:ok([{"Content-Type", "text/plain"}], "This is ~s's page.", >>> [UserName]); >> >> >> >> >> In both terminal and browser I see incorrect output - what can be done >> about it? >> >> big thanks, >> Slav > > hi slav, > > this is not particularly related to misultin. when using uri you need to > ensure that these are properly encoded for HTTP. You also need to ensure > that the browser is rendering html encoded in UTF-8 so you should have in > your code: > > % handle a GET on /users/{username} > handle('GET', ["users", UserName], Req) -> > ??? Req:ok([{"Content-Type", "text/html; charset=UTF-8"}], "This is ~s's > page.", [UserName]); > > then, you may serve the page: > > http://localhost:8080/users/%26%231055%3B%26%231088%3B%26%231080%3B%26%231074%3B%26%231077%3B%26%231090%3B > > that will print; > > This is ??????'s page. > > you probably also could read into IRI: http://tools.ietf.org/html/rfc3987 > > my 2c. > > r. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From bangonkali@REDACTED Fri Sep 2 12:21:49 2011 From: bangonkali@REDACTED (Bangon Kali) Date: Fri, 2 Sep 2011 03:21:49 -0700 (PDT) Subject: [erlang-questions] Newbie Question, brute force method. Message-ID: <1314958909984-3785742.post@n4.nabble.com> Hi guys! I'm a newbie here. I'm very new to Erlang. Although I've been studying well im having problems converting sequential programs in to Recursive ones. I'm using this book Pragmatic Programming Erlang (2007). I've been half way in my readings. Although I kinda get some of the pieces along the way, I have been having problems with some parts. Generally on how to convert very sequential programs in to recursive ones. In order to test my knowledge in Erlang I have setup a challenge to myself, by converting http://j.mp/r8hS3C this program that I've made in several other languages (C, Python, Javscript) into Erlang. This program basically just Finds the set of Realistic Resistors and Capacitor for an Astable 555 IC implementation. The program uses the brute force technique by testing all possible values and calculating the frequency. It will then compare the percentage difference between the previous frequency to the current frequency, if the new frequency is lower, then it will save the R1, R2, and C combinations. It will do in a loop until all possible realistic values of R1, R2 and C are tested. a simple pseudo code might be easier to understand where: get_error returns the percentage difference between 2 values get_frequency returns the frequency of an R1, R2 and C combination f_desired is the frequency target the complete source code for this program (in some languages) are available on this bitbucket site: http://j.mp/r8hS3C i'm not asking for a direct translation in to erlang, but if you can easily make one, then that would really help, since i can study it directly and somehow learn to implement erlang on some other problems with the same situation such as this. here are some of my attempts, but im at loss at how to continue this. i hope you guys can help. just some clue, if i may ask. thanks! -- View this message in context: http://erlang.2086793.n4.nabble.com/Newbie-Question-brute-force-method-tp3785742p3785742.html Sent from the Erlang Questions mailing list archive at Nabble.com. From gnoblin@REDACTED Fri Sep 2 13:02:30 2011 From: gnoblin@REDACTED (Slav Pankratov) Date: Fri, 2 Sep 2011 14:02:30 +0300 Subject: [erlang-questions] misultin / unicode In-Reply-To: References: Message-ID: Roberto, thanks! Antoine, yes in my case ~ts helps together with unicode:characters_to_list(list_to_binary()) (and it seems I have latin1 text coming in, not unicode - my mistake!) thanks, Slav 2011/9/2 Antoine Koener > On Thu, Sep 1, 2011 at 10:21 PM, Roberto Ostinelli > wrote: > > > > > > 2011/9/1 Slav Pankratov > >> > >> Hello! > >> > >> > >> > >> > >> I'm using misultin to receive http calls, and they sometimes contain > >> Russian symbols (unicode). > >> > >> > >>> > >>> > >>> % handle a GET on /users/{username} > >>> > >>> > >>> handle('GET', ["users", UserName], Req) -> > >>> > >>> > >>> io:format("~s~n", [UserName]), > > I may be completely wrong but: try with ~ts instead of ~s in your io:format > > > >>> > >>> > >>> Req:ok([{"Content-Type", "text/plain"}], "This is ~s's page.", > >>> [UserName]); > >> > >> > >> > >> > >> In both terminal and browser I see incorrect output - what can be done > >> about it? > >> > >> big thanks, > >> Slav > > > > hi slav, > > > > this is not particularly related to misultin. when using uri you need to > > ensure that these are properly encoded for HTTP. You also need to ensure > > that the browser is rendering html encoded in UTF-8 so you should have in > > your code: > > > > % handle a GET on /users/{username} > > handle('GET', ["users", UserName], Req) -> > > Req:ok([{"Content-Type", "text/html; charset=UTF-8"}], "This is ~s's > > page.", [UserName]); > > > > then, you may serve the page: > > > > > http://localhost:8080/users/%26%231055%3B%26%231088%3B%26%231080%3B%26%231074%3B%26%231077%3B%26%231090%3B > > > > that will print; > > > > This is ??????'s page. > > > > you probably also could read into IRI: > http://tools.ietf.org/html/rfc3987 > > > > my 2c. > > > > r. > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From konrad.kaplita@REDACTED Fri Sep 2 16:06:02 2011 From: konrad.kaplita@REDACTED (Konrad Kaplita) Date: Fri, 2 Sep 2011 15:06:02 +0100 Subject: [erlang-questions] SSL verify_fun option Message-ID: Hi all, I need an advice from some SSL guru here. I want to implement SSL mutual authentication. I'm using R14B02. During certificate validation I want also to obtain Common Name value. To do this I try to use verify_fun option but with no luck. Documentation states: > The verify fun will be called during the X509-path validation when an error > or an extension unknown to the ssl application is encountered. Additionally > it will be called when a certificate is considered valid by the path > validation to allow access to each certificate in the path to the user > application. My SSL Options tuple looks like this: > {ssl_opts, [{certfile, > filename:join([code:priv_dir(ws), "ssl", "server.crt"])}, > {keyfile, > filename:join([code:priv_dir(ws), "ssl", "server.key"])}, > {verify, verify_peer}, > {cacertfile, > filename:join([code:priv_dir(ws), "ssl", "ca.crt"])}, > {fail_if_no_peer_cert, true}, > {verify_fun, {fun validate/3, []}} I expect that my validate fun will be invoked after successfull SSL handshake, but it's not. After some digging I discovered that ssl_handshake:certify/6 function has been changed in this commit: https://github.com/erlang/otp/commit/4dbf3c9e4ae7cfd19b247353369166d31b8f15e5#diff-0 It basically wraps my verify_fun in it's own, and passes this further to pubkey_cert:verify_fun/4, and it never invokes my verify_fun, due to the change introduced by the commit mentioned above (when you have valid vertificate, your verify_fun is never called, which is in opposite to the documentation). Is current behaviour intentional or a bug? Or maybe I'm doing something wrong along the way? Kind regards, Konrad -------------- next part -------------- An HTML attachment was scrubbed... URL: From baliulia@REDACTED Fri Sep 2 18:02:10 2011 From: baliulia@REDACTED (=?UTF-8?B?SWduYXMgVnnFoW5pYXVza2Fz?=) Date: Fri, 02 Sep 2011 19:02:10 +0300 Subject: [erlang-questions] how can I make mnesia/sasl keep quiet In-Reply-To: <01C1F235-999A-4D28-BA97-A42ECF22B948@lovely.email.ne.jp> References: <01C1F235-999A-4D28-BA97-A42ECF22B948@lovely.email.ne.jp> Message-ID: <4E60FE02.7060408@gmail.com> On 09/01/2011 02:30 AM, Joseph Norton wrote: > > Try the following .... > > $ erl -sasl errlog_type error > This does not seem to work for me with R14B02 and I have others confirm under R14B03. All of mnesia:stop(), application:stop(mnesia) and error_logger:info_msg("test") print info reports. Any ideas why? Thanks /Ignas From ingela@REDACTED Fri Sep 2 21:09:42 2011 From: ingela@REDACTED (Ingela Andin) Date: Fri, 2 Sep 2011 21:09:42 +0200 Subject: [erlang-questions] ssl_esock leaking file descriptors In-Reply-To: References: Message-ID: Hi! In R15 you will be able to run the Erlang distrubution over the new ssl implementation. The plan is also to drop the old ssl implementation in R15. Regards Ingela Erlang/OTP-team, Ericsson AB 2011/8/31 Gordon Guthrie : > We get an intermittent ssl_esock problem which I have never successful > reproduced. It goes to 100% and the process needs to be manually killed. > Richard Andrews also reported a problem with it going to 100% CPU in 2009: > http://erlang.2086793.n4.nabble.com/ssl-esock-spinning-out-of-control-in-poll-td2117067.html > He has a patch for that. > It is on my 'long list' of things to fix but more frequent/reproducable ones > allways get in the way. > Gordon > > On 31 August 2011 15:13, Justin Milam wrote: >> >> I've started to notice a slow leak of file descriptors in the ssl_esock >> port. I'm running Erlang R14B and using SSL to encrypt traffic over the >> Erlang distribution protocol. The cluster has 10 nodes minimum with >> transient nodes joining and leaving the cluster regularly. From checking the >> ssl_esock process with lsof it appears to be slowly leaking file >> descriptors. The number of open file descriptors seems to increase after a >> node joins the cluster and then leaves. Eventually ssl_esock holds open >> enough file descriptors to hit the ulimit (currently 8192) in which case >> ssl_esock goes into an infinite loop using near 100% of one of the CPUs. >> I've been able to reproduce the issue by lowering the ulimit and >> continually connecting/disconnecting a remote shell to a local running node >> until the ulimit is reached. When ssl_esock is running in debug mode I see >> the following being logged continually: >> ==========LOOP============= >> MASKS SET FOR FD: 27 (read) 26 (read) 25 (read) 24 (read) 19 (read) 18 >> (read) 17 (read) 16 (read) 12 (read) 11 (read) 10 (read) 9 (read) 8 (read) 7 >> (read) 6 (read) >> CONNECTIONS: >> ?- DEFUNCT [0x8772978] (fd = 29) >> ?- DEFUNCT [0x86f9950] (fd = 28) >> ?- JOINED [0x875ae30] (origin = accept) >> ? ? ? ?(fd = 26, eof = 0, wq = 0, bp = 0) >> ? ? ? ?(proxyfd = 27, eof = 0, wq = 0, bp = 0) >> ?- JOINED [0x86fa970] (origin = accept) >> ? ? ? ?(fd = 24, eof = 0, wq = 0, bp = 0) >> ? ? ? ?(proxyfd = 25, eof = 0, wq = 0, bp = 0) >> ?- DEFUNCT [0x8733600] (fd = 21) >> ?- DEFUNCT [0x8732c38] (fd = 20) >> ?- JOINED [0x8733958] (origin = accept) >> ? ? ? ?(fd = 18, eof = 0, wq = 0, bp = 0) >> ? ? ? ?(proxyfd = 19, eof = 0, wq = 0, bp = 0) >> ?- JOINED [0x8734f78] (origin = accept) >> ? ? ? ?(fd = 16, eof = 0, wq = 0, bp = 0) >> ? ? ? ?(proxyfd = 17, eof = 0, wq = 0, bp = 0) >> ?- CONNECTED [0x87134a8] (fd = 15) >> ?- DEFUNCT [0x871f220] (fd = 13) >> ?- JOINED [0x87147d0] (origin = accept) >> ? ? ? ?(fd = 11, eof = 0, wq = 0, bp = 0) >> ? ? ? ?(proxyfd = 12, eof = 0, wq = 0, bp = 0) >> ?- JOINED [0x87083d0] (origin = connect) >> ? ? ? ?(fd = 9, eof = 0, wq = 0, bp = 0) >> ? ? ? ?(proxyfd = 10, eof = 0, wq = 0, bp = 0) >> ?- JOINED [0x86f29e8] (origin = connect) >> ? ? ? ?(fd = 7, eof = 0, wq = 0, bp = 0) >> ? ? ? ?(proxyfd = 8, eof = 0, wq = 0, bp = 0) >> ?- ACTIVE_LISTENING [0x86f2258] (fd = 6, acceptors = 1) >> Before poll/select: 15 descriptors (total 29) >> Error calling accept() >> accept error (proxy_listensock): emfile >> Has anyone else experienced such?behavior? >> Thanks >> -justin >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Gordon Guthrie > CEO hypernumbers > > http://hypernumbers.com > t: hypernumbers > +44 7776 251669 > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From gnoblin@REDACTED Fri Sep 2 21:30:28 2011 From: gnoblin@REDACTED (Slav Pankratov) Date: Fri, 2 Sep 2011 22:30:28 +0300 Subject: [erlang-questions] TonyGen mongodb-erlang / incrementing a field Message-ID: Hello! I'm struggling to increment a field ($inc) in mongodb from erlang via the official mongo driver (https://github.com/TonyGen/mongodb-erlang). Could anyone point me in the right direction of one of the correct ways to do it? thanks, Slav -------------- next part -------------- An HTML attachment was scrubbed... URL: From norton@REDACTED Sat Sep 3 07:12:28 2011 From: norton@REDACTED (Joseph Norton) Date: Sat, 3 Sep 2011 14:12:28 +0900 Subject: [erlang-questions] erl_nif: nif_to_binary and binary_to_nif API calls Message-ID: For the erl_nif module, are there any plans to add API calls for nif_to_binary and binary_to_nif? regards, Joseph Norton From rapsey@REDACTED Sat Sep 3 09:04:22 2011 From: rapsey@REDACTED (Rapsey) Date: Sat, 3 Sep 2011 09:04:22 +0200 Subject: [erlang-questions] erl_nif: nif_to_binary and binary_to_nif API calls In-Reply-To: References: Message-ID: I'm not sure what you mean. There is: enif_inspect_binary enif_inspect_iolist_as_binary enif_make_binary What's missing? Sergej On Sat, Sep 3, 2011 at 7:12 AM, Joseph Norton wrote: > > For the erl_nif module, are there any plans to add API calls for > nif_to_binary and binary_to_nif? > > regards, > > Joseph Norton > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From norton@REDACTED Sat Sep 3 09:53:51 2011 From: norton@REDACTED (Joseph Norton) Date: Sat, 3 Sep 2011 16:53:51 +0900 Subject: [erlang-questions] erl_nif: nif_to_binary and binary_to_nif API calls In-Reply-To: References: Message-ID: It would be handy to have the same functionality that is provided by erlang:term_to_binary and erlang:binary_to_term available to a NIF author. erlang:term_to_binary(term()) -> ext_binary() erlang:binary_to_term(ext_binary()) -> term() except in C where term() is ERL_NIF_TERM and ext_binary() is ErlNifBinary. - Joe On Sep 3, 2011, at 4:04 PM, Rapsey wrote: > I'm not sure what you mean. There is: > enif_inspect_binary > enif_inspect_iolist_as_binary > enif_make_binary > > What's missing? > > > Sergej > > > > On Sat, Sep 3, 2011 at 7:12 AM, Joseph Norton wrote: > > For the erl_nif module, are there any plans to add API calls for nif_to_binary and binary_to_nif? > > regards, > > Joseph Norton > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From caox@REDACTED Sat Sep 3 12:23:07 2011 From: caox@REDACTED (caox) Date: Sat, 3 Sep 2011 18:23:07 +0800 Subject: [erlang-questions] erlang smp on aix In-Reply-To: References: <677D6E76-A4A8-460C-9607-4261D0F293BE@lightpole.net> <50882B59-F1AE-41BD-8514-9543863EF272@lightpole.net> Message-ID: <206BB439-0B12-4D27-926C-BE999F1D25E2@lightpole.net> After we implemented template rendering with native erlang code instead of linked-in c driver, the problem seemed resolved.But the driver works fine on Centos. Thanks for your help. ? 2011-9-2???3:35? Jon Watte ??? > I haven't kept up on the latest in AIX kernel development, but it seems that IBM on POWER worries more about virtualizing processors across multiple partitioned workloads, rather than the best possible API performance for a unified machine view. It's possible that the AIX kernel does not have the kind of primitives that Erlang needs to run well in highly parallel situations. > When I worked with them, IBM technical support and ISV support was fairly well educated, and you could possible send an strace or a copy of your program to them to ask them what they think and get a real answer (as opposed to some other IHVs out there...) > > Sincerely, > > jw > > -- > Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. > > > > 2011/9/1 caox > No, the kernel poll is not supported on AIX. > > AIX has pollset interface like epoll but seems can not be used by erts. Is this the reason of thread_waitlock? > > Our test scenario is quite simple. Create 1000 erlang processes or more to execute our main function repeatedly. This function do nothing special but call a linkded-in driver with port_control/3 and do some template rendering then return. The whole function does not have any IO operations. > > During the test on a 2 cores AIX, we found no thread_waitlock() call in the strace log, which was full of poll() call. > > With same test scenario on a 4 cores Centos with kernel-poll off, there is no sys call in the strace log(even without poll() call). Cpu will be consumed perfectly and we can gain quite high system throughput. > > But when migrating to 4cores or 8cores AIX, thread_watilock() appears and there is lots of context switches according to vmstat. As a result, the throughput drop dramatically compared with Centos. > > So, what is role of poll() in the scenario, or in the erlang processes scheduling? > > BR > > > ? 2011-9-2???12:51? Jon Watte ??? > >> Did you enable kernel_poll? And does AIX actually support a suitable kernel polling mechanism (like epoll, etc)? >> >> Sincerely, >> >> jw >> >> >> -- >> Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. >> >> >> >> On Thu, Sep 1, 2011 at 3:01 AM, caox wrote: >> hi >> >> we have a application used to run on centos but now we want to migrate it to AIX. During the test on Aix, we found the cpu scaling of the system is quite disappointing. With 8 cores and erlang smp auto, there were very high system cpu consumption and lots of thread_wait_lock call in the truss(strace) log, which would not appear on centos under the same test scenario. And the sys call will disappear if we turn smp off. >> So we guess there is something wrong with smp on Aix, perhaps caused by the schedulers switching. But we don't know where the thread_wait_call is from and what it is for. >> By the way, it is quite strange that the smp description of erl shell would be smp32:8 with smp auto on the 8 cores aix. >> Is there any hint to fix the problem? Appreciate your help. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Sat Sep 3 13:03:42 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 3 Sep 2011 13:03:42 +0200 Subject: [erlang-questions] ssl_esock leaking file descriptors In-Reply-To: References: Message-ID: <96299989-4593-4126-86B1-E0FBBF69452D@erlang-solutions.com> On 2 Sep 2011, at 21:09, Ingela Andin wrote: > Hi! > > In R15 you will be able to run the Erlang distrubution over the new > ssl implementation. The plan is also to drop the old ssl > implementation in R15. Hmm, interesting? Does this also mean that it will be easier to define other carriers (erlang-implemented) for the Erlang distribution? BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Sat Sep 3 19:26:37 2011 From: jwatte@REDACTED (Jon Watte) Date: Sat, 3 Sep 2011 10:26:37 -0700 Subject: [erlang-questions] Newbie Question, brute force method. In-Reply-To: <1314958909984-3785742.post@n4.nabble.com> References: <1314958909984-3785742.post@n4.nabble.com> Message-ID: With tail recursion, each for loop turns into a function of its own. stuff_a() for (a=0; a != b; ++a) { c += stuff_b(a) } stuff_c() turns into: stuff_a() stuff_b_loop(a, b, 0); stuff_c() stuff_b_loop(A, A, R) -> R; stuff_b_loop(A, B, R) -> stuff_b_loop(A+1, B, R + stuff_b(A)). This can almost always be re-written "mechanically" if you just think of the loop as a tail-recursive function with start/stop/result arguments. You could even declare a "for()" function that takes a fun(): for(Limit, Limit, _, Result) -> Result; for(Start, Limit, Fun, Result) -> for(Start+1, Limit, Fun, Fun(Result)). This would be used in the above as: stuff_a(), for(a, b, fun(C) -> C + stuff_b(C) end, 0), stuff_c() This, in turn, is the core of the "fold" operation on sequences and containers. (Look at lists:foldl(), dict:fold(), and friends). Thus, for the parts of your program that just iterate over the items in an element, you can use lists:foldl() as the loop construct. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Fri, Sep 2, 2011 at 3:21 AM, Bangon Kali wrote: > Hi guys! I'm a newbie here. I'm very new to Erlang. Although I've been > studying well im having problems converting sequential programs in to > Recursive ones. I'm using this book Pragmatic Programming Erlang (2007). > I've been half way in my readings. Although I kinda get some of the pieces > along the way, I have been having problems with some parts. Generally on > how > to convert very sequential programs in to recursive ones. > > In order to test my knowledge in Erlang I have setup a challenge to myself, > by converting http://j.mp/r8hS3C this program that I've made in several > other languages (C, Python, Javscript) into Erlang. This program basically > just Finds the set of Realistic Resistors and Capacitor for an Astable 555 > IC implementation. > > The program uses the brute force technique by testing all possible values > and calculating the frequency. It will then compare the percentage > difference between the previous frequency to the current frequency, if the > new frequency is lower, then it will save the R1, R2, and C combinations. > It > will do in a loop until all possible realistic values of R1, R2 and C are > tested. > > a simple pseudo code might be easier to understand > > > > where: > get_error returns the percentage difference between 2 values > get_frequency returns the frequency of an R1, R2 and C combination > f_desired is the frequency target > > the complete source code for this program (in some languages) are available > on this bitbucket site: http://j.mp/r8hS3C > > i'm not asking for a direct translation in to erlang, but if you can easily > make one, then that would really help, since i can study it directly and > somehow learn to implement erlang on some other problems with the same > situation such as this. > > here are some of my attempts, but im at loss at how to continue this. > > i hope you guys can help. just some clue, if i may ask. thanks! > > > -- > View this message in context: > http://erlang.2086793.n4.nabble.com/Newbie-Question-brute-force-method-tp3785742p3785742.html > Sent from the Erlang Questions mailing list archive at Nabble.com. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From groups.erlang@REDACTED Sat Sep 3 21:54:38 2011 From: groups.erlang@REDACTED (Rommel Garcia) Date: Sat, 3 Sep 2011 15:54:38 -0400 Subject: [erlang-questions] Problem with debugger:start() Message-ID: Hello, I'm currently going through the tutorials from Joe Armstrong's book "Programming Erlang" and was interested to use the erlang debugging tool. I tried running debugger:start but it's complaining about a driver my Mac OSX Snow Leopard doesn't have. Please see statements below. Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.2 (abort with ^G) 1> debugger:start(). =ERROR REPORT==== 3-Sep-2011::15:18:58 === ERROR: Could not find 'wxe_driver.so' in: /opt/local/lib/erlang/lib/wx-0.98.8/priv {ok,<0.35.0>} How do I install this wxe_driver.so driver? Haven't found anything in google with this issue. I would appreciate your comments. Thanks. Rom From garazdawi@REDACTED Sat Sep 3 23:34:14 2011 From: garazdawi@REDACTED (Lukas Larsson) Date: Sat, 3 Sep 2011 23:34:14 +0200 Subject: [erlang-questions] Problem with debugger:start() In-Reply-To: References: Message-ID: wx widgets are not supported for os x 64 bit, so if you want any wx window applications on os x you have to run a 32 bit erlang VM Lukas On Sat, Sep 3, 2011 at 9:54 PM, Rommel Garcia wrote: > Hello, > > I'm currently going through the tutorials from Joe Armstrong's book > "Programming Erlang" and was interested to use the erlang debugging tool. > I tried running debugger:start but it's complaining about a driver my Mac > OSX Snow Leopard doesn't have. Please see statements below. > > > Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:4:4] [rq:4] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.2 (abort with ^G) > 1> debugger:start(). > > =ERROR REPORT==== 3-Sep-2011::15:18:58 === > ERROR: Could not find 'wxe_driver.so' in: > /opt/local/lib/erlang/lib/wx-0.98.8/priv > {ok,<0.35.0>} > > > How do I install this wxe_driver.so driver? Haven't found anything in > google with this issue. I would appreciate your comments. > > Thanks. > > Rom > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dan@REDACTED Sun Sep 4 03:31:33 2011 From: dan@REDACTED (Daniel Dormont) Date: Sat, 3 Sep 2011 21:31:33 -0400 Subject: [erlang-questions] When to catch (was: Module config data and functional programming) Message-ID: This was an earlier thread of mine that inspired a new question. As the months go by I'm getting more comfortable with "let it crash" as an idea and seeing its benefits. But it occurs to me: surely there are exceptions. Or rather: in order for "let it crash" to work at one level of abstraction or functionality, there must be actual crash-handling code at a level beneath it. In Erlang, this seems to often involve letting individual processes crash and their supervisors react accordingly. But perhaps that's not always the right answer. So my question is: When is "catch" the right tool for the job? In the example below, Ejabberd's "hooks" handlers use catch when calling functions in foreign modules that are (probably) not critical path to whatever it's doing. That seems like a pretty decent case to me. Would you agree? What are some cases common in Erlang where exceptions need to be caught and handled (or ignored) within a single process? dan On Fri, Apr 22, 2011 at 1:30 PM, Per Melin wrote: > On Fri, Apr 22, 2011 at 4:32 PM, Daniel Dormont > wrote: >> It also occurs to me that switching to gen_server would offer another advantage in that it would be easy to recover from crashes in my own code. If I were not to do that though, is the recommended practice just to put a 'catch' call around code that might break? > > I don't have enough context, but it sounds absolutely unnecessary to > create a gen_server for this. > > I would consider it ejabberd's responsibility to handle any crashes in > your code. They have asked you for a callback function and they must > consider that it may break. And they are in a better position than you > to know how to proceed from a crash here. I did have a look in > ejabberd_hooks and they do catch exits from your function. Let it > crash. > > I'm hesitant to touch the question of what is recommended practice > since this is the kind of thread where Dr Richard O'Keefe usually > comments after a while and turns everyone that posted before him into > fools. > From norton@REDACTED Sun Sep 4 05:33:08 2011 From: norton@REDACTED (Joseph Norton) Date: Sun, 4 Sep 2011 12:33:08 +0900 Subject: [erlang-questions] how can I make mnesia/sasl keep quiet In-Reply-To: <4E60FE02.7060408@gmail.com> References: <01C1F235-999A-4D28-BA97-A42ECF22B948@lovely.email.ne.jp> <4E60FE02.7060408@gmail.com> Message-ID: <766D19E4-1318-4D49-A05F-219BA2EF6665@lovely.email.ne.jp> I was mistaken. This option seems to only disable SASL progress reports. "error reports" seems to include info level log events as well. Excerpt from sasl modules documentation: errlog_type = error | progress | all Restricts the error logging performed by the specified sasl_error_logger to error reports, progress reports, or both. Default is all. Excerpt from lib/sasl/src/sasl.erl module: type(error) -> error; type(progress) -> progress; type(_) -> all. Joseph Norton On Sep 3, 2011, at 1:02 AM, Ignas Vy?niauskas wrote: > On 09/01/2011 02:30 AM, Joseph Norton wrote: >> >> Try the following .... >> >> $ erl -sasl errlog_type error >> > This does not seem to work for me with R14B02 and I have others confirm > under R14B03. > All of mnesia:stop(), application:stop(mnesia) and > error_logger:info_msg("test") print info reports. > Any ideas why? > > Thanks > /Ignas > > > > From jwatte@REDACTED Sun Sep 4 08:33:59 2011 From: jwatte@REDACTED (Jon Watte) Date: Sat, 3 Sep 2011 23:33:59 -0700 Subject: [erlang-questions] When to catch (was: Module config data and functional programming) In-Reply-To: References: Message-ID: How expensive is the state of the process to re-create? If it's super expensive, "let it crash" may not be the right choice. Then again, if the reason you crash is that the state has gotten somehow corrupted, then crashing might be the only way to "clean it out." And, on the third hand, when you've sufficiently hardened your application so that remote connection action cannot disturb it in unintended ways, no matter what crap gets thrown at it, then "let it crash" doesn't matter, because you won't crash anyway :-) Really, I find crash-and-restart to be fine for things that really only happens once in a blue moon. For anything that's a little more common -- unexpected remote connection disconnect, bad input data or unexpected message sequencing, for example, those really are design goals of the system. If all that crashes is the connection of the badly behaving user, then that's a fine way to disconnect that user. If it's further into the core of the application, perhaps not so. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Sat, Sep 3, 2011 at 6:31 PM, Daniel Dormont wrote: > This was an earlier thread of mine that inspired a new question. > > As the months go by I'm getting more comfortable with "let it crash" > as an idea and seeing its benefits. But it occurs to me: surely there > are exceptions. Or rather: in order for "let it crash" to work at one > level of abstraction or functionality, there must be actual > crash-handling code at a level beneath it. In Erlang, this seems to > often involve letting individual processes crash and their supervisors > react accordingly. But perhaps that's not always the right answer. So > my question is: > > When is "catch" the right tool for the job? In the example below, > Ejabberd's "hooks" handlers use catch when calling functions in > foreign modules that are (probably) not critical path to whatever it's > doing. That seems like a pretty decent case to me. Would you agree? > What are some cases common in Erlang where exceptions need to be > caught and handled (or ignored) within a single process? > > dan > > On Fri, Apr 22, 2011 at 1:30 PM, Per Melin wrote: > > On Fri, Apr 22, 2011 at 4:32 PM, Daniel Dormont > > wrote: > >> It also occurs to me that switching to gen_server would offer another > advantage in that it would be easy to recover from crashes in my own code. > If I were not to do that though, is the recommended practice just to put a > 'catch' call around code that might break? > > > > I don't have enough context, but it sounds absolutely unnecessary to > > create a gen_server for this. > > > > I would consider it ejabberd's responsibility to handle any crashes in > > your code. They have asked you for a callback function and they must > > consider that it may break. And they are in a better position than you > > to know how to proceed from a crash here. I did have a look in > > ejabberd_hooks and they do catch exits from your function. Let it > > crash. > > > > I'm hesitant to touch the question of what is recommended practice > > since this is the kind of thread where Dr Richard O'Keefe usually > > comments after a while and turns everyone that posted before him into > > fools. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zerthurd@REDACTED Sun Sep 4 13:09:19 2011 From: zerthurd@REDACTED (Maxim Treskin) Date: Sun, 4 Sep 2011 18:09:19 +0700 Subject: [erlang-questions] capturing all traces in a running system In-Reply-To: <483A36B7-2A6F-4785-BEAA-1A8A60CDE229@gmail.com> References: <483A36B7-2A6F-4785-BEAA-1A8A60CDE229@gmail.com> Message-ID: Hello, Joel Look at https://github.com/metachord/mctrace It has similar functionality On 31 August 2011 21:06, Joel Reymont wrote: > Has anyone tried to capture all traces in a running system for later > analysis? > > What kind of performance impact does trace collection have on a running > system, particularly if traces are shipped over TCP to a non-Erlang server? > > It seems that it should be possible to reconstruct the behavior of a > running system by logging what module started what process and by capturing > all traces for offline analysis. > > Has this been tried before? > > Thanks, Joel > > -------------------------------------------------------------------------- > - for hire: mac osx device driver ninja, kernel extensions and usb drivers > ---------------------+------------+--------------------------------------- > http://wagerlabs.com | @wagerlabs | http://www.linkedin.com/in/joelreymont > ---------------------+------------+--------------------------------------- > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Maxim Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.elsgaard@REDACTED Sun Sep 4 18:06:04 2011 From: thomas.elsgaard@REDACTED (Thomas Elsgaard) Date: Sun, 4 Sep 2011 18:06:04 +0200 Subject: [erlang-questions] Newbie question, finite state Machine failover Message-ID: Hi I am looking into using erlang for a FSM (finite state machine) and i will need to implement a statefull failover mechanism between two physical servers, which approch would you recommend for this? Any Best practice advices? Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From maggio.vincenzo@REDACTED Sun Sep 4 18:24:07 2011 From: maggio.vincenzo@REDACTED (Vincenzo Maggio) Date: Sun, 4 Sep 2011 18:24:07 +0200 Subject: [erlang-questions] Newbie question, finite state Machine failover In-Reply-To: References: Message-ID: Hello, I should think the OTP compliant way would be to use application failover mechanism, in which the application on second server would start up if first server fails. Check out http://www.erlang.org/doc/design_principles/distributed_applications.html Vinz Date: Sun, 4 Sep 2011 18:06:04 +0200 From: thomas.elsgaard@REDACTED To: erlang-questions@REDACTED Subject: [erlang-questions] Newbie question, finite state Machine failover Hi I am looking into using erlang for a FSM (finite state machine) and i will need to implement a statefull failover mechanism between two physical servers, which approch would you recommend for this? Any Best practice advices? Thomas _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel@REDACTED Sun Sep 4 19:10:43 2011 From: daniel@REDACTED (Daniel Luna) Date: Sun, 4 Sep 2011 13:10:43 -0400 Subject: [erlang-questions] TonyGen mongodb-erlang / incrementing a field In-Reply-To: References: Message-ID: On 2 September 2011 15:30, Slav Pankratov wrote: > I'm struggling to increment a field ($inc) in mongodb from erlang via the > official mongo driver?(https://github.com/TonyGen/mongodb-erlang). > Could anyone point me in the right direction of one of the correct ways to > do it? You can do something like the following: F = fun() -> mongo:modify(some_collection, {some_indexed_field, Key}, {'$inc', {'field.to.increase', Increase}}) end, mongo:do(safe, master, Connection, db_name, F). If the field to increase doesn't exist it will be created with value Increase. Take a look at mongo:repsert/3 if the element might be non-existent. Cheers, Daniel From groups.erlang@REDACTED Sun Sep 4 20:05:33 2011 From: groups.erlang@REDACTED (Rommel Garcia) Date: Sun, 4 Sep 2011 14:05:33 -0400 Subject: [erlang-questions] Problem with debugger:start() In-Reply-To: References: Message-ID: Thanks Lukas. Do you know if macports will be able to install a 32bit erlang vm considering my machine already has one which is 64bit? Rommel On Sep 3, 2011, at 5:34 PM, Lukas Larsson wrote: > wx widgets are not supported for os x 64 bit, so if you want any wx window applications on os x you have to run a 32 bit erlang VM > > Lukas > > On Sat, Sep 3, 2011 at 9:54 PM, Rommel Garcia wrote: > Hello, > > I'm currently going through the tutorials from Joe Armstrong's book "Programming Erlang" and was interested to use the erlang debugging tool. > I tried running debugger:start but it's complaining about a driver my Mac OSX Snow Leopard doesn't have. Please see statements below. > > > Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.2 (abort with ^G) > 1> debugger:start(). > > =ERROR REPORT==== 3-Sep-2011::15:18:58 === > ERROR: Could not find 'wxe_driver.so' in: /opt/local/lib/erlang/lib/wx-0.98.8/priv > {ok,<0.35.0>} > > > How do I install this wxe_driver.so driver? Haven't found anything in google with this issue. I would appreciate your comments. > > Thanks. > > Rom > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > Rom -------------- next part -------------- An HTML attachment was scrubbed... URL: From bfulgham@REDACTED Sun Sep 4 20:26:25 2011 From: bfulgham@REDACTED (Brent Fulgham) Date: Sun, 4 Sep 2011 11:26:25 -0700 Subject: [erlang-questions] Problem with debugger:start() In-Reply-To: References: Message-ID: <172F8CE8-2F3D-4D8E-AC99-5B2B44FEA924@gmail.com> MacPorts defaults to 64-bit when the machine can support it. Unfortunately, it does not seem possible to install a 32-bit VM in a 64-bit MacPorts install. Instead, you have to clear out your entire MacPorts install, and set the 32-bit architecture as the default before you install packages. I am trying to figure out a way around this issue. That does not require port removal. It might be enough to just adjust the build settings in the Portfile. If that works, I'll add a variant case and commit it. -Brent Sent from my iPhone On Sep 4, 2011, at 11:05 AM, Rommel Garcia wrote: > Thanks Lukas. Do you know if macports will be able to install a 32bit erlang vm considering my machine already has one which is 64bit? > > Rommel > > On Sep 3, 2011, at 5:34 PM, Lukas Larsson wrote: > >> wx widgets are not supported for os x 64 bit, so if you want any wx window applications on os x you have to run a 32 bit erlang VM >> >> Lukas >> >> On Sat, Sep 3, 2011 at 9:54 PM, Rommel Garcia wrote: >> Hello, >> >> I'm currently going through the tutorials from Joe Armstrong's book "Programming Erlang" and was interested to use the erlang debugging tool. >> I tried running debugger:start but it's complaining about a driver my Mac OSX Snow Leopard doesn't have. Please see statements below. >> >> >> Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] >> >> Eshell V5.8.2 (abort with ^G) >> 1> debugger:start(). >> >> =ERROR REPORT==== 3-Sep-2011::15:18:58 === >> ERROR: Could not find 'wxe_driver.so' in: /opt/local/lib/erlang/lib/wx-0.98.8/priv >> {ok,<0.35.0>} >> >> >> How do I install this wxe_driver.so driver? Haven't found anything in google with this issue. I would appreciate your comments. >> >> Thanks. >> >> Rom >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > Rom > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From crusso@REDACTED Mon Sep 5 00:45:05 2011 From: crusso@REDACTED (Claudio Russo) Date: Sun, 4 Sep 2011 22:45:05 +0000 Subject: [erlang-questions] Final CFP: PADL'12 - Practical Aspects of Declarative Languages 2012 Message-ID: <88D1F4047EA9A2468BCC6B743186880F6409410C@DB3EX14MBXC313.europe.corp.microsoft.com> [Apologies if you receive multiple copies.] Final Call for Papers ===================== 14th International Symposium on Practical Aspects of Declarative Languages (PADL 2012) http://research.microsoft.com/~crusso/padl12 Philadelphia, Pennsylvania, USA, January 23-24, 2012 Co-located with ACM POPL'12 Abstract/Paper submission deadline: September 10th/17th, 2011 Conference Description ====================== Declarative languages build on sound theoretical bases to provide attractive frameworks for application development. These languages have been successfully applied to many different real-world situations, ranging from data base management to active networks to software engineering to decision support systems. New developments in theory and implementation have opened up new application areas. At the same time, applications of declarative languages to novel problems raise numerous interesting research issues. Well-known questions include designing for scalability, language extensions for application deployment, and programming environments. Thus, applications drive the progress in the theory and implementation of declarative systems, and benefit from this progress as well. PADL is a forum for researchers and practitioners to present original work emphasizing novel applications and implementation techniques for all forms of declarative concepts, including, functional, logic, constraints, etc. Topics of interest include, but are not limited to: * Innovative applications of declarative languages * Declarative domain-specific languages and applications * Practical applications of theoretical results * New language developments and their impact on applications * Declarative languages and Software Engineering * Evaluation of implementation techniques on practical applications * Practical experiences and industrial applications * Novel uses of declarative languages in the classroom * Practical extensions such as constraint-based, probabilistic, and reactive languages. PADL'12 welcomes new ideas and approaches pertaining to applications and implementation of declarative languages. In this occasion PADL is co-located, as traditionally, with ACM POPL, which will be held immediately following PADL, January 25-27. The symposium will be held in Philadelphia, Pennsylvania, USA. Important Dates and Submission Guidelines ========================================= Abstract Submission: September 10, 2011 Paper Submission: September 17, 2011 Notification: October 22, 2011 Camera-ready: November 5, 2011 Symposium: January 23-24, 2012 Authors should submit an electronic copy of the full paper in PDF using the Springer LNCS format. The submission will be done through EasyChair conference system. If electronic submission is impossible, please contact the program chairs for information on how to submit hard copies. All submissions must be original work written in English. Submissions must be unpublished and not submitted for publication elsewhere. Work that already appeared in unpublished or informally published workshops proceedings may be submitted. PADL'12 will accept both technical and application papers: * Technical papers must describe original, previously unpublished research results. Technical papers must not exceed 15 pages in Springer LNCS format. * Application papers are a mechanism to present important practical applications of declarative languages that occur in industry or in areas of research other than Computer Science. Application papers will be published in the Springer-Verlag conference proceedings, and will be presented in a separate session. Application papers are expected to describe complex and/or real-world applications that rely on an innovative use of declarative languages. Application descriptions, engineering solutions and real-world experiences (both positive and negative) are solicited. The limit for application papers is 6 pages in Springer LNCS format. Program Committee ================= Marcello Balduccini, Intelligent Systems Department, Kodak Research Labs Edwin Brady, University of St Andrews, Scotland Henning Christiansen, Roskilde University, Denmark Agostino Dovier, University of Udine, Italy Matthew Flatt, University of Utah, USA Gopal Gupta, University of Texas at Dallas, USA John Hughes, Chalmers University of Technology, Sweden; Quviq AB Gabriele Keller, University of New South Wales, Australia Lunjin Lu, Oakland University, USA Marc Pouzet, ?cole normale sup?rieure, France Ricardo Rocha, University of Porto, Portugal Andreas Rossberg, Google Germany GmbH, Germany Claudio Russo, Microsoft Research Cambridge, UK (co-chair) Kostis Sagonas, Uppsala University, Sweden; NTUA, Greece Satnam Singh, Microsoft Research Cambridge, UK Zoltan Somogyi, University of Melbourne, Australia Eijiro Sumii, Tohoku University, Japan Terrance Swift, Universidade Nova de Lisboa, Portugal; Johns Hopkins University, USA Andrew Tolmach, Portland State University, USA Jan Wielemaker, University of Amsterdam, The Netherlands Roland Yap, National University of Singapore, Republic of Singapore Kwangkeun Yi, Seoul National University, Korea Neng-Fa Zhou, Brooklyn College, City University of New York, USA (co-chair) Contacts ======== For additional information about papers and submissions, please contact the Program Chairs: Claudio Russo Microsoft Research Cambridge,UK Email: crusso microsoft com Neng-Fa Zhou Brooklyn College, The City University of New York, USA Email: zhou sci brooklyn cuny edu With the Cooperation of ======================= The Association for Logic Programming (ALP) ACM SIGPLAN Microsoft Research, Cambridge =================================== From steven.charles.davis@REDACTED Mon Sep 5 01:54:30 2011 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sun, 4 Sep 2011 16:54:30 -0700 (PDT) Subject: [erlang-questions] When to catch (was: Module config data and functional programming) In-Reply-To: References: Message-ID: On Sep 3, 8:31?pm, Daniel Dormont wrote: > When is "catch" the right tool for the job? I'm going to simply repeat some advice I got from (IIRC) Robert Virding, which was given to me a while back, and which has thoroughly resonated for me... "Protect the boundary of your system, inside let it crash". To go further than the question you asked, you could possibly also explore the ideas behind Joe Armstrong's "man in the middle" as exemplified by UBF contracts. HTH, /s From wmacgyver@REDACTED Mon Sep 5 03:08:24 2011 From: wmacgyver@REDACTED (Wilson MacGyver) Date: Sun, 4 Sep 2011 21:08:24 -0400 Subject: [erlang-questions] misultin on erlang R13B04? Message-ID: Does misultin no longer support R13 series? I tried to build it via make and get make erlc -W -I include -o ebin src/*.erl src/misultin_acceptor.erl:85: referring to built-in type tid as a remote type; please take out the module name src/misultin_acceptor.erl:149: referring to built-in type tid as a remote type; please take out the module name src/misultin_acceptor.erl:168: referring to built-in type tid as a remote type; please take out the module name make: *** [all] Error 1 on R14 it builds fine. Thanks -- Omnem crede diem tibi diluxisse supremum. From groups.erlang@REDACTED Mon Sep 5 03:42:10 2011 From: groups.erlang@REDACTED (Rommel Garcia) Date: Sun, 4 Sep 2011 21:42:10 -0400 Subject: [erlang-questions] Problem with debugger:start() In-Reply-To: <172F8CE8-2F3D-4D8E-AC99-5B2B44FEA924@gmail.com> References: <172F8CE8-2F3D-4D8E-AC99-5B2B44FEA924@gmail.com> Message-ID: Thanks Brent! Please let me know once it's ready for download... Appreciate all the help guys! Rommel On Sep 4, 2011, at 2:26 PM, Brent Fulgham wrote: > MacPorts defaults to 64-bit when the machine can support it. Unfortunately, it does not seem possible to install a 32-bit VM in a 64-bit MacPorts install. Instead, you have to clear out your entire MacPorts install, and set the 32-bit architecture as the default before you install packages. > > I am trying to figure out a way around this issue. That does not require port removal. It might be enough to just adjust the build settings in the Portfile. > > If that works, I'll add a variant case and commit it. > > -Brent > > Sent from my iPhone > > On Sep 4, 2011, at 11:05 AM, Rommel Garcia wrote: > >> Thanks Lukas. Do you know if macports will be able to install a 32bit erlang vm considering my machine already has one which is 64bit? >> >> Rommel >> >> On Sep 3, 2011, at 5:34 PM, Lukas Larsson wrote: >> >>> wx widgets are not supported for os x 64 bit, so if you want any wx window applications on os x you have to run a 32 bit erlang VM >>> >>> Lukas >>> >>> On Sat, Sep 3, 2011 at 9:54 PM, Rommel Garcia wrote: >>> Hello, >>> >>> I'm currently going through the tutorials from Joe Armstrong's book "Programming Erlang" and was interested to use the erlang debugging tool. >>> I tried running debugger:start but it's complaining about a driver my Mac OSX Snow Leopard doesn't have. Please see statements below. >>> >>> >>> Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] >>> >>> Eshell V5.8.2 (abort with ^G) >>> 1> debugger:start(). >>> >>> =ERROR REPORT==== 3-Sep-2011::15:18:58 === >>> ERROR: Could not find 'wxe_driver.so' in: /opt/local/lib/erlang/lib/wx-0.98.8/priv >>> {ok,<0.35.0>} >>> >>> >>> How do I install this wxe_driver.so driver? Haven't found anything in google with this issue. I would appreciate your comments. >>> >>> Thanks. >>> >>> Rom >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> Rom >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions Rom -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela@REDACTED Mon Sep 5 09:38:51 2011 From: ingela@REDACTED (Ingela Andin) Date: Mon, 5 Sep 2011 09:38:51 +0200 Subject: [erlang-questions] ssl_esock leaking file descriptors In-Reply-To: <96299989-4593-4126-86B1-E0FBBF69452D@erlang-solutions.com> References: <96299989-4593-4126-86B1-E0FBBF69452D@erlang-solutions.com> Message-ID: Hi! Well the main problem with Erlang-implemented carriers are that when the distribution is started the system is in such an early stage that you can not start Erlang applications. The way we solved this for ssl is that we have a special supervisor that we hook into the kernel application that will create a cloone of the ssl application. The drawback with this is that you can not soft upgrade the ssl-application if you use ssl as distribution carrier (of course if you do not use ssl for the distribution soft upgrade will be possible in R15). Regards Ingela Erlang/OTP team - Ericssson AB 2011/9/3 Ulf Wiger : > > On 2 Sep 2011, at 21:09, Ingela Andin wrote: > > Hi! > > In R15 you will be able to run the Erlang distrubution over the new > ssl implementation. ?The plan is also to drop the old ssl > implementation in R15. > > Hmm, interesting? > Does this also mean that it will be easier to define other carriers > (erlang-implemented) for the Erlang distribution? > BR, > Ulf W > Ulf Wiger,?CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From roberto@REDACTED Mon Sep 5 09:57:11 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 5 Sep 2011 09:57:11 +0200 Subject: [erlang-questions] misultin on erlang R13B04? In-Reply-To: References: Message-ID: Hi Wilson, all the issues you are referring to concern type specs definitions. On R13, tid() was a built-in type, and it has been changed on R14 to be a remote type [which in my opinion is more correct]. Thus the error you are getting while compiling on R13. I did report this question here: http://erlang.org/pipermail/erlang-questions/2011-August/060838.html Kostis answered "But I would really recommend to the users of R13 to switch to R14. There are many good reasons to do that". It's not the first time I hear this, even the OTP team has recommended on many occasions to upgrade [I remember some SSL issues too]. Therefore, I haven't added support to R13 on misultin 0.8. At least until now. r. On Mon, Sep 5, 2011 at 3:08 AM, Wilson MacGyver wrote: > Does misultin no longer support R13 series? > > I tried to build it via make > and get > > make > erlc -W -I include -o ebin src/*.erl > src/misultin_acceptor.erl:85: referring to built-in type tid as a > remote type; please take out the module name > src/misultin_acceptor.erl:149: referring to built-in type tid as a > remote type; please take out the module name > src/misultin_acceptor.erl:168: referring to built-in type tid as a > remote type; please take out the module name > make: *** [all] Error 1 > > > on R14 it builds fine. > > Thanks > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela@REDACTED Mon Sep 5 10:00:35 2011 From: ingela@REDACTED (Ingela Andin) Date: Mon, 5 Sep 2011 10:00:35 +0200 Subject: [erlang-questions] SSL verify_fun option In-Reply-To: References: Message-ID: Hi! This commit will have the effect that your verify_fun will not be called with the "extended_key_usage-extension" as a unknown extension that has not been verified, as it has been verified by the ssl application. However your verify_fun should be called with the whole certificate giving you the chance to verify this extension further if you so desire. I am referring to one of the clause: (Cert, valid, UserState) -> ... (Cert, valid_peer, UserState) -> ... Regards Ingela Erlang/OTP team - Ericsson AB 2011/9/2 Konrad Kaplita : > Hi all, > I need an advice from some SSL guru here. I want to implement SSL mutual > authentication. I'm using R14B02. During certificate validation I want also > to obtain Common Name value. To do this I try to use verify_fun option but > with no luck. > Documentation states: >> >> The verify fun will be called during the X509-path validation when an >> error or an extension unknown to the ssl application is encountered. >> Additionally it will be called when a certificate is considered valid by the >> path validation to allow access to each certificate in the path to the user >> application. > > My SSL Options tuple looks like this: >> >> ? ? ? ? ? ? ? ? ? ? {ssl_opts, [{certfile, >> filename:join([code:priv_dir(ws), "ssl", "server.crt"])}, >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {keyfile, >> filename:join([code:priv_dir(ws), "ssl", "server.key"])}, >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {verify, verify_peer}, >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {cacertfile, >> filename:join([code:priv_dir(ws), "ssl", "ca.crt"])}, >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {fail_if_no_peer_cert, true}, >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {verify_fun, {fun validate/3, []}} > > I expect that my validate fun will be invoked after successfull SSL > handshake, but it's not. > After some digging I discovered that ssl_handshake:certify/6 function has > been changed in this > commit:?https://github.com/erlang/otp/commit/4dbf3c9e4ae7cfd19b247353369166d31b8f15e5#diff-0 > It basically wraps my verify_fun in it's own, and passes this further to > pubkey_cert:verify_fun/4, and it never invokes my verify_fun, due to the > change introduced by the commit mentioned above (when you have valid > vertificate, your verify_fun is never called, which is in opposite to the > documentation). > Is current behaviour intentional or a bug? Or maybe I'm doing something > wrong along the way? > Kind regards, > Konrad > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From roberto@REDACTED Mon Sep 5 10:05:10 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 5 Sep 2011 10:05:10 +0200 Subject: [erlang-questions] misultin on erlang R13B04? In-Reply-To: References: Message-ID: However, since I do plan to add support for previous versions [misultin does support R13xx except for type and specs definitions], can someone point me in a right direction for how to do so? Kostis is referring to Scalaris code for some configuration option but a more precise help would be appreciated. Thank you, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wmacgyver@REDACTED Mon Sep 5 11:43:53 2011 From: wmacgyver@REDACTED (Wilson MacGyver) Date: Mon, 5 Sep 2011 05:43:53 -0400 Subject: [erlang-questions] misultin on erlang R13B04? In-Reply-To: References: Message-ID: No problem, as long as it's an expected behavior, I can live with that. You may want to put in the docs that it requires R14+. Thanks, On Mon, Sep 5, 2011 at 3:57 AM, Roberto Ostinelli wrote: > Hi Wilson, > > all the issues you are referring to concern type specs definitions. > > On R13, tid() was a built-in type, and it has been changed on R14 to be a > remote type [which in my opinion is more correct]. Thus the error you are > getting while compiling on R13. > > I did report this question here: > http://erlang.org/pipermail/erlang-questions/2011-August/060838.html > > Kostis answered "But I would really recommend to the users of R13 to switch > to R14. There are many good reasons to do that". It's not the first time I > hear this, even the OTP team has recommended on many occasions to upgrade [I > remember some SSL issues too]. > > Therefore, I haven't added support to R13 on misultin 0.8. At least until > now. > > r. > > > On Mon, Sep 5, 2011 at 3:08 AM, Wilson MacGyver wrote: >> >> Does misultin no longer support R13 series? >> >> I tried to build it via make >> and get >> >> make >> erlc -W -I include -o ebin src/*.erl >> src/misultin_acceptor.erl:85: referring to built-in type tid as a >> remote type; please take out the module name >> src/misultin_acceptor.erl:149: referring to built-in type tid as a >> remote type; please take out the module name >> src/misultin_acceptor.erl:168: referring to built-in type tid as a >> remote type; please take out the module name >> make: *** [all] Error 1 >> >> >> on R14 it builds fine. >> >> Thanks > -- Omnem crede diem tibi diluxisse supremum. From erikseres@REDACTED Mon Sep 5 11:48:23 2011 From: erikseres@REDACTED (Erik Seres) Date: Mon, 5 Sep 2011 11:48:23 +0200 Subject: [erlang-questions] Mnesia - Partitioned network problem Message-ID: Hello, I am running distributed mnesia on a cluster of two servers, x@REDACTED and x@REDACTED interaction with mnesia occurs on x@REDACTED x@REDACTED replicates from x@REDACTED Due to an intermittent network failure, replication had stopped. In the logs for x@REDACTED was the following message: =ERROR REPORT==== 23-Aug-2011::23:56:37 === ** Node 'x@REDACTED' not responding ** ** Removing (timedout) connection ** This message went unnoticed until x@REDACTED was restarted for an unrelated reason and it x@REDACTED logged the following error at startup: =ERROR REPORT==== 24-Aug-2011::14:22:09 === Mnesia('x@REDACTED'): ** ERROR ** mnesia_event got {inconsistent_database, running_partitioned_network, 'x@REDACTED'} Question 1: How is it possible that only x@REDACTED detected the disconnect from x@REDACTED and x@REDACTED did not detect a disconnect from x@REDACTED? Question 2: How is it possible that x@REDACTED did not log the running_paritioned_network error? I realize that the answer to Q1 may lead to an answer to Q2, as well. Note, that neither server was set as master. Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Mon Sep 5 12:57:48 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 5 Sep 2011 12:57:48 +0200 Subject: [erlang-questions] Mnesia - Partitioned network problem In-Reply-To: References: Message-ID: On 5 Sep 2011, at 11:48, Erik Seres wrote: > Due to an intermittent network failure, replication had stopped. In the logs for x@REDACTED was the following message: > > =ERROR REPORT==== 23-Aug-2011::23:56:37 === > ** Node 'x@REDACTED' not responding ** > ** Removing (timedout) connection ** Sometimes, this message appears on both sides, and sometimes only on one. It depends a bit on the cause. The other side (if it doesn't timeout and issue the above message) will detect the socket being closed, but this doesn't cause a message to be logged. > This message went unnoticed until x@REDACTED was restarted for an unrelated reason and it x@REDACTED logged the following error at startup: > > =ERROR REPORT==== 24-Aug-2011::14:22:09 === > Mnesia('x@REDACTED'): ** ERROR ** mnesia_event got {inconsistent_database, running_partitioned_network, 'x@REDACTED'} This comes when the nodes try to reconnect. The node that restarted will hail the one that is already running (x@REDACTED), which finds out that both nodes considered the other 'down'. Unfortunately, this event is not issued on both nodes. That the systems didn't reconnect until x@REDACTED restarted has to do with the fact that you happen to have no processes that actively try to send messages to the other node. By default, Erlang nodes will connect automatically on-demand, but this can be turned off (using -kernel dist_auto_connect X, where X :: once | never). Sometimes, people will have a process that simply sends a UDP message to corresponding processes on the other nodes. If such a message is received from a node that you thought was down (not in the nodes() list), then you have a partitioned network, and need to take steps to resolve it. Mnesia doesn't do this for you; but it's really not that hard to supply the logic - once you realise that you have to do it yourself. ;-) BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From v@REDACTED Mon Sep 5 13:47:31 2011 From: v@REDACTED (Valentin Micic) Date: Mon, 5 Sep 2011 13:47:31 +0200 Subject: [erlang-questions] Mnesia - Partitioned network problem In-Reply-To: References: Message-ID: <80DE992E-22A4-4C0F-B703-6476F24AF968@pharos-avantgard.com> Q1: This may sound wired, but this is not necessarily a networking problem! A couple of years ago, I've been conducting some tests using two machines connected "back-to-back" (*), thus limiting the network to two directly connected nodes. My mnesia environment consisted of two highly fragmented disk-only tables (256 and 768 dets files) hosted by two nodes, having only one interacting with a "traffic source" (let's call it ACTIVE), whilst using the second for replication only. Then, I've exposed the ACTIVE mnesia node to a continuous traffic, using a great deal of inserts, pushing the rate as high as 600 inserts per second (Before I forget -- I've used a huge data set, say, more than 300 million records) After running smoothly for a while (that is, until the general population per dets file reached approx 200,000 records), I've noticed the ACTIVE mnesia node warned about the network partitioning, although there was no reason to do so (e.g. cable was still connected ;-)). After a considerable fact finding effort, I've concluded that it was the disk activity (that caused CPU to go into the uninterruptible sleep state waiting for disk I/O to complete) causing the erts to slow-down, hence build-up process message queue, which in turn caused mnesia to report network partitioning -- not because response hasn't been received from the peer node, but because the response has not been processed on time. Since the remote node did not have the same issue with CPU load, it was able to "see" the peer mnesia node. Indeed, I've been able to fix the issue (at least short term), by increasing the file I/O thread pool (+A flag). Q2: see above :-) Kind regards V/ (*) that is, by connecting two machines using a single cable, thus avoiding any additional points of failure. On 05 Sep 2011, at 11:48 AM, Erik Seres wrote: > Hello, > > I am running distributed mnesia on a cluster of two servers, x@REDACTED and x@REDACTED All interaction with mnesia occurs on x@REDACTED while x@REDACTED just replicates from x@REDACTED > > Due to an intermittent network failure, replication had stopped. In the logs for x@REDACTED was the following message: > > =ERROR REPORT==== 23-Aug-2011::23:56:37 === > ** Node 'x@REDACTED' not responding ** > ** Removing (timedout) connection ** > > This message went unnoticed until x@REDACTED was restarted for an unrelated reason and it x@REDACTED logged the following error at startup: > > =ERROR REPORT==== 24-Aug-2011::14:22:09 === > Mnesia('x@REDACTED'): ** ERROR ** mnesia_event got {inconsistent_database, running_partitioned_network, 'x@REDACTED'} > > > Question 1: How is it possible that only x@REDACTED detected the disconnect from x@REDACTED and x@REDACTED did not detect a disconnect from x@REDACTED? > Question 2: How is it possible that x@REDACTED did not log the running_paritioned_network error? > > I realize that the answer to Q1 may lead to an answer to Q2, as well. > > Note, that neither server was set as master. > > Thank you. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From robert.virding@REDACTED Mon Sep 5 15:33:25 2011 From: robert.virding@REDACTED (Robert Virding) Date: Mon, 05 Sep 2011 14:33:25 +0100 (BST) Subject: [erlang-questions] misultin on erlang R13B04? In-Reply-To: Message-ID: <366b7a82-4bac-4999-9412-2e41f40acb77@knuth> ----- Original Message ----- > However, since I do plan to add support for previous versions > [misultin does support R13xx except for type and specs definitions], > can someone point me in a right direction for how to do so? Remove all type and spec definitions. :-) > Kostis is referring to Scalaris code for some configuration option > but a more precise help would be appreciated. > Thank you, > r. Robert -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Sep 5 17:11:47 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Mon, 05 Sep 2011 17:11:47 +0200 Subject: [erlang-questions] [ANN] Cowboy 0.2.0 - HTTP server and acceptor pool Message-ID: <4E64E6B3.80209@dev-extend.eu> Hello guys. I would like to formally announce Cowboy here today with the first tagged version: 0.2.0. This is a stable build of a development HTTP server and pool acceptor. It is released under the ISC license, which is a BSD license with deprecated clauses removed. https://github.com/extend/cowboy Cowboy cuts network communication into three parts: transport (tcp, ssl), protocol (http, ftp...) and application handlers for handling and responding to client requests. It comes with an acceptor pool that takes transport and protocol modules and use them to listen, accept and process requests. Each listener come with its own supervision tree that can be started and stopped independently. The current features include: * TCP and SSL transport handlers * HTTP protocol handler * Optional dispatcher for HTTP requests * WebSocket support (all recent browsers supported) * Cookie support (adapted from mochiweb by Tom Burdick) * OTP-like handler API for HTTP: init/3, handle/2, terminate/2 * And for WebSocket: websocket_init/3, websocket_handle/3, websocket_terminate/3 * EUnit, common_test and PropEr tests * No dialyzer warnings! http://dialyzer.softlab.ntua.gr/nowarn/cowboy/ Cowboy also uses binaries instead of lists, largely reducing memory usage. It also only uses one process per request a la gen_server, catching and outputting a stacktrace when errors occur. Note that WebSocket is still a draft and that changes still occur in its specifications. Browsers do not fully implement the specs either, so we are just aiming for supporting browsers for now. While this release is fairly stable, keep in mind that Cowboy is still a work in progress and that its features and API may change at some point. Me and others are already using it in production however. Future developments include: * FastCGI support (Anthony Ramine) * Sendfile support (Magnus Klaar) * Webmachine-like REST handler * Multipart support for file upload and compression (Anthony Ramine) * Allow reloading a listener's options without restarting it * More PropEr tests! Finally, I'll be setting up a benchmarking environment of 4 servers to properly test performance improvements and make sure unrelated changes don't have a negative impact on performance. This should happen in the next few weeks and it will have a public URL. We might also setup the tests for other servers including C ones later on. If you are interested in helping out, using it or have questions I'm available by email, on the issues tracker on github or on IRC on #erlounge or #erlang. Thanks for reading. -- Lo?c Hoguin Dev:Extend From roberto@REDACTED Mon Sep 5 17:16:45 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 5 Sep 2011 17:16:45 +0200 Subject: [erlang-questions] misultin on erlang R13B04? In-Reply-To: <366b7a82-4bac-4999-9412-2e41f40acb77@knuth> References: <366b7a82-4bac-4999-9412-2e41f40acb77@knuth> Message-ID: On Mon, Sep 5, 2011 at 3:33 PM, Robert Virding < robert.virding@REDACTED> wrote: > ------------------------------ > > However, since I do plan to add support for previous versions [misultin > does support R13xx except for type and specs definitions], can someone point > me in a right direction for how to do so? > > > Remove all type and spec definitions. :-) > ..seriously? i'm quite fond in using these, as they add to methodology and tracking of own code. also, with tools such as PropEr there's an additional good reason to add them. id' rather add a >=R14 spec to misultin than to remove specs. however, ideally, since Kostis has pointed out there are ways to have them working on different Erlang versions, I'd like to hear how to do this. r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pfisher@REDACTED Mon Sep 5 17:22:19 2011 From: pfisher@REDACTED (Paul Fisher) Date: Mon, 5 Sep 2011 10:22:19 -0500 Subject: [erlang-questions] building erlang/otp dev on mac os x? In-Reply-To: References: Message-ID: <55D701EA-6AB1-4D05-8F5C-E79732E3CD67@alertlogic.com> Just a follow-up on this. Installing gcc 4.5.3 from macports, allows the current dev branch to build and pass most of the tests. On Sep 1, 2011, at 9:00 PM, Paul Fisher wrote: > I get roughly the following for both 32-bit and 64-bit builds on lion. Has anyone specifically diagnosed the issue building current releases up to, and including the current dev branch, on mac os x? Is this considered an gcc-llvm or otp issue? > > $ ./configure --enable-darwin-64bit --disable-hipe > $ make > > > > === Entering application hipe > erlc -W +debug_info +warn_exported_vars +warn_missing_spec +warn_untyped_record -o../ebin hipe_consttab.erl > (no error logger present) error: "Error in process <0.1.0> with exit value: {{badfun,[<<5 bytes>>,<<36 bytes>>,<<9 bytes>>,<<3 bytes>>,<<2 bytes>>,<<5 bytes>>,<<14 bytes>>,<<2 bytes>>,<<8 bytes>>,<<8 bytes>>,<<5 bytes>>,<<7 bytes>>,<<5 bytes>>,<<11 bytes>>,<<2 bytes>>,<<11 bytes>>,<<15 bytes>>,<<4 bytes>>,<<40 bytes>>,<<5 bytes>>,<<1 byte>>,<<7 bytes>>,<<10 bytes>>,<<7 bytes>>,<<18 bytes>>,<<7 bytes>>,<<17 bytes>>,<<7 bytes>>,<<19 bytes>>,<<7 bytes>>,<<7 bytes>>,<<6 bytes>>,<<17 bytes>>]},[{erlang,apply,2}]}\n" > ^C > > > $ gcc --version > i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00) > Copyright (C) 2007 Free Software Foundation, Inc. > This is free software; see the source for copying conditions. There is NO > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. > > > -- > paul > > director, platform services > alertlogic, inc. > 713-484-8383 x2314 > -- paul director, platform services alertlogic, inc. 713-484-8383 x2314 From kostis@REDACTED Mon Sep 5 17:40:47 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 05 Sep 2011 18:40:47 +0300 Subject: [erlang-questions] building erlang/otp dev on mac os x? In-Reply-To: <55D701EA-6AB1-4D05-8F5C-E79732E3CD67@alertlogic.com> References: <55D701EA-6AB1-4D05-8F5C-E79732E3CD67@alertlogic.com> Message-ID: <4E64ED7F.1050900@cs.ntua.gr> On 09/05/11 18:22, Paul Fisher wrote: > Just a follow-up on this. Installing gcc 4.5.3 from macports, allows the current dev branch to build and pass most of the tests. I do not have a Mac and for this reason perhaps I should not really care, but if I had one I would have found this information a bit uninformative. The interesting question is whether the dev branch passes at least all tests as those when compiled with -O0, i.e. whether the optimization passes in the updated macports compiler are still causing trouble for (parts of) Erlang/OTP or not. If not, either the compiler or Erlang/OTP needs fixing. Till then I would recommend to all MacOS users to keep using -O0 anyway. Kostis From techabc@REDACTED Mon Sep 5 17:46:02 2011 From: techabc@REDACTED (techabc) Date: Mon, 5 Sep 2011 23:46:02 +0800 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: References: Message-ID: Import QT to erlang world, QT c++ or QML, Nice thing it is! Dose helpful of Native Process with R15? 2011/8/6 Daniel Goertzen > How about interfacing Erlang to the Qt UI library? > > I don't mean a convential binding like wxErlang or PyQt, but components to > facilitate message passing between Erlang and a Qt port program or NIF > thread running a Qt message loop. > > Qt has relatively recently introduced QML, which is basically a fancy GUI > designer that lets you incorporate javascript. The application development > model is that the GUI is written in QML/javascript, and the underlying > application logic is in C++ (very Web 2.0-ish). Now instead of writing C++, > you have your QML/javascript talk to the "ErQt" object which passes messages > to/from Erlang, and Erlang implements the application logic. > > Conceivably one could create a single Qt port executable, driver, or NIF > library that could run any number of QML/javascript programs. Creating a > GUI for Erlang would involve using the native Qt tools for authoring QML and > javascript, and implementing your comm protocol on both the javascript and > Erlang sides. Your QML/javascript GUI would run on the standard Qt port > exectable...no additional C++ compiling required. > > Now QML is currently only available for mobile devices, but the Qt folks > have stated that they will port it to the desktop and this will become the > preferred way to develop desktop Qt apps. > > This project would likely involve more C++ than Erlang, so I'm not sure how > you feel about that. ;) > > Cheers, > Dan. > > > On Mon, Aug 1, 2011 at 5:53 PM, Nick S wrote: > >> I have just finished implementing a "professional" Erlang server, and with >> time and enthusiasm in hand, I would like to start working on something >> personal. >> I must say, it has been really fun writing apps in OTP and see it >> processing millions of messages without ever crashing...! >> >> Would appreciate some new ideas, that would be useful to community :) >> >> - XMPP, AMQP etc are already in market! >> >> - Any audio/video processing layer? >> >> - Any telecom related...? >> >> - Some standard protocol implementation? >> >> - Something new for next generation...? (Buzz words... cloud computing >> etc..) >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.james.barry@REDACTED Mon Sep 5 17:53:48 2011 From: paul.james.barry@REDACTED (Paul Barry) Date: Mon, 5 Sep 2011 16:53:48 +0100 Subject: [erlang-questions] building erlang/otp dev on mac os x? In-Reply-To: <4E64ED7F.1050900@cs.ntua.gr> References: <55D701EA-6AB1-4D05-8F5C-E79732E3CD67@alertlogic.com> <4E64ED7F.1050900@cs.ntua.gr> Message-ID: FWIW: I am running on Mac OS X and installed Erlang with Homebrew and have had no issues: http://mxcl.github.com/homebrew/ On 5 September 2011 16:40, Kostis Sagonas wrote: > On 09/05/11 18:22, Paul Fisher wrote: >> >> Just a follow-up on this. ?Installing gcc 4.5.3 from macports, allows the >> current dev branch to build and pass most of the tests. > > I do not have a Mac and for this reason perhaps I should not really care, > but if I had one I would have found this information a bit uninformative. > > The interesting question is whether the dev branch passes at least all tests > as those when compiled with -O0, i.e. whether the optimization passes in the > updated macports compiler are still causing trouble for (parts of) > Erlang/OTP or not. > > If not, either the compiler or Erlang/OTP needs fixing. Till then I would > recommend to all MacOS users to keep using -O0 anyway. > > Kostis > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Paul Barry, w: http://paulbarry.itcarlow.ie, e: paul.barry@REDACTED Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From kostis@REDACTED Mon Sep 5 17:57:13 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 05 Sep 2011 18:57:13 +0300 Subject: [erlang-questions] misultin on erlang R13B04? In-Reply-To: References: <366b7a82-4bac-4999-9412-2e41f40acb77@knuth> Message-ID: <4E64F159.50000@cs.ntua.gr> On 09/05/11 18:16, Roberto Ostinelli wrote: > On Mon, Sep 5, 2011 at 3:33 PM, Robert Virding > > wrote: > > ------------------------------------------------------------------------ > > However, since I do plan to add support for previous versions > [misultin does support R13xx except for type and specs > definitions], can someone point me in a right direction for how > to do so? > > > Remove all type and spec definitions. :-) > > > ..seriously? i'm quite fond in using these, as they add to methodology > and tracking of own code. also, with tools such as PropEr there's an > additional good reason to add them. id' rather add a >=R14 spec to > misultin than to remove specs. Good choice! :) > however, ideally, since Kostis has pointed out there are ways to have > them working on different Erlang versions, I'd like to hear how to do this. I suggested looking how it's done at the 'configure' of Scalaris. Its code is here: http://scalaris.googlecode.com/svn/trunk/configure Search for e.g. gb_tree() or DIALYZER to see how to write (portions of) a configure script that allows to generate appropriate defines for various types that have become built-ins or changed status between different OTP versions. Kostis From norton@REDACTED Mon Sep 5 18:04:12 2011 From: norton@REDACTED (Joseph Wayne Norton) Date: Tue, 6 Sep 2011 01:04:12 +0900 Subject: [erlang-questions] misultin on erlang R13B04? In-Reply-To: <4E64F159.50000@cs.ntua.gr> References: <366b7a82-4bac-4999-9412-2e41f40acb77@knuth> <4E64F159.50000@cs.ntua.gr> Message-ID: <8ADDDB50-0ED4-43C0-AF07-93D826D80DE8@lovely.email.ne.jp> One other suggestion. If you are using rebar as your build tool, the platform_define directive works quite well. Here is an example for "old_inets": https://github.com/norton/ubf-jsonrpc/blob/master/rebar.config https://github.com/norton/ubf-jsonrpc/blob/master/src/ubf_jsonrpc_inets_httpc_simple.erl thanks, - Joe N On 2011/09/06, at 0:57, Kostis Sagonas wrote: > On 09/05/11 18:16, Roberto Ostinelli wrote: >> On Mon, Sep 5, 2011 at 3:33 PM, Robert Virding >> > > wrote: >> >> ------------------------------------------------------------------------ >> >> However, since I do plan to add support for previous versions >> [misultin does support R13xx except for type and specs >> definitions], can someone point me in a right direction for how >> to do so? >> >> >> Remove all type and spec definitions. :-) >> >> >> ..seriously? i'm quite fond in using these, as they add to methodology >> and tracking of own code. also, with tools such as PropEr there's an >> additional good reason to add them. id' rather add a >=R14 spec to >> misultin than to remove specs. > > Good choice! :) > >> however, ideally, since Kostis has pointed out there are ways to have >> them working on different Erlang versions, I'd like to hear how to do this. > > I suggested looking how it's done at the 'configure' of Scalaris. > > Its code is here: http://scalaris.googlecode.com/svn/trunk/configure > > Search for e.g. gb_tree() or DIALYZER to see how to write (portions of) a configure script that allows to generate appropriate defines for various types that have become built-ins or changed status between different OTP versions. > > ____Kostis > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Joseph Wayne Norton From zabrane3@REDACTED Mon Sep 5 18:11:07 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Mon, 5 Sep 2011 18:11:07 +0200 Subject: [erlang-questions] [ANN] Cowboy 0.2.0 - HTTP server and acceptor pool In-Reply-To: <4E64E6B3.80209@dev-extend.eu> References: <4E64E6B3.80209@dev-extend.eu> Message-ID: Congratulations ! On Sep 5, 2011, at 5:11 PM, Lo?c Hoguin wrote: > Hello guys. > > I would like to formally announce Cowboy here today with the first > tagged version: 0.2.0. This is a stable build of a development HTTP > server and pool acceptor. It is released under the ISC license, which is > a BSD license with deprecated clauses removed. > > https://github.com/extend/cowboy > > Cowboy cuts network communication into three parts: transport (tcp, > ssl), protocol (http, ftp...) and application handlers for handling and > responding to client requests. It comes with an acceptor pool that takes > transport and protocol modules and use them to listen, accept and > process requests. Each listener come with its own supervision tree that > can be started and stopped independently. > > The current features include: > > * TCP and SSL transport handlers > * HTTP protocol handler > * Optional dispatcher for HTTP requests > * WebSocket support (all recent browsers supported) > * Cookie support (adapted from mochiweb by Tom Burdick) > * OTP-like handler API for HTTP: init/3, handle/2, terminate/2 > * And for WebSocket: websocket_init/3, websocket_handle/3, > websocket_terminate/3 > * EUnit, common_test and PropEr tests > * No dialyzer warnings! http://dialyzer.softlab.ntua.gr/nowarn/cowboy/ > > Cowboy also uses binaries instead of lists, largely reducing memory > usage. It also only uses one process per request a la gen_server, > catching and outputting a stacktrace when errors occur. > > Note that WebSocket is still a draft and that changes still occur in its > specifications. Browsers do not fully implement the specs either, so we > are just aiming for supporting browsers for now. > > While this release is fairly stable, keep in mind that Cowboy is still a > work in progress and that its features and API may change at some point. > Me and others are already using it in production however. > > Future developments include: > > * FastCGI support (Anthony Ramine) > * Sendfile support (Magnus Klaar) > * Webmachine-like REST handler > * Multipart support for file upload and compression (Anthony Ramine) > * Allow reloading a listener's options without restarting it > * More PropEr tests! > > Finally, I'll be setting up a benchmarking environment of 4 servers to > properly test performance improvements and make sure unrelated changes > don't have a negative impact on performance. This should happen in the > next few weeks and it will have a public URL. We might also setup the > tests for other servers including C ones later on. > > If you are interested in helping out, using it or have questions I'm > available by email, on the issues tracker on github or on IRC on > #erlounge or #erlang. > > Thanks for reading. > From reynaldomic@REDACTED Mon Sep 5 18:59:20 2011 From: reynaldomic@REDACTED (Reynaldo Baquerizo) Date: Mon, 5 Sep 2011 11:59:20 -0500 Subject: [erlang-questions] tcp connections dropped in gen_server Message-ID: I have a running application that consist in a supervisor and two generic servers, one of them wraps around odbc and the other handles tcp connections, a fragment of the relevant code is: init([]) -> process_flag(trap_exit, true), {ok, ListenSocket} = gen_tcp:listen(Port, [binary, {packet, 0}, {reuseaddr, true}, {active, true}]), proc_lib:spawn_link(?MODULE, acceptor, [ListenSocket]) acceptor(ListenSocket) -> {ok, Socket} = gen_tcp:accept(ListenSocket), error_logger:info_msg("New connection from ~p~n", [Socket]), _Pid = proc_lib:spawn(?MODULE, acceptor, [ListenSocket]), inet:setopts(Socket, [binary, {nodelay, true}, {active, true}]), loop(Socket). loop(Socket) -> receive {tcp, Socket, Data} -> error_logger:info_msg("Messaged received from ~p: ~p~n", [Socket, Data]), comm_lib:handle_message(Socket, Data), loop(Socket); {tcp_closed, Socket} -> error_logger:info_msg("Device at ~p disconnected~n", [Socket]); _Any -> %% skip this loop(Socket) end. So, I basically start a new unlinked process for every new tcp connection. It works just fine for a couple hours but then every tcp connection is dropped gradually with message "Device at ~p disconnected". The client will try to reconnect if connection is closed. The tcp connection should only terminate if remote end closes it or spawned proccess in the server crashes. After all connections were dropped, I can see with inet:i() that there are established connections but no logging! Can anyone give some insight or point to the right direction to debug this? From essen@REDACTED Mon Sep 5 19:41:52 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Mon, 05 Sep 2011 19:41:52 +0200 Subject: [erlang-questions] When to catch (was: Module config data and functional programming) In-Reply-To: References: Message-ID: <4E6509E0.70307@dev-extend.eu> On 09/04/2011 03:31 AM, Daniel Dormont wrote: > This was an earlier thread of mine that inspired a new question. > > As the months go by I'm getting more comfortable with "let it crash" > as an idea and seeing its benefits. But it occurs to me: surely there > are exceptions. Or rather: in order for "let it crash" to work at one > level of abstraction or functionality, there must be actual > crash-handling code at a level beneath it. In Erlang, this seems to > often involve letting individual processes crash and their supervisors > react accordingly. But perhaps that's not always the right answer. So > my question is: > > When is "catch" the right tool for the job? In the example below, > Ejabberd's "hooks" handlers use catch when calling functions in > foreign modules that are (probably) not critical path to whatever it's > doing. That seems like a pretty decent case to me. Would you agree? > What are some cases common in Erlang where exceptions need to be > caught and handled (or ignored) within a single process? Another use: You should catch when you run foreign code from the same process. Examples of this include gen_server, gen_fsm and friends. The gen_server code will call functions defined in your module, which can crash. gen_server will catch exceptions to print a meaningful error and cleanup. -- Lo?c Hoguin Dev:Extend From tristan.sloughter@REDACTED Mon Sep 5 19:52:42 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Mon, 5 Sep 2011 12:52:42 -0500 Subject: [erlang-questions] Alternative Default Webmachine Functions Message-ID: This should be a generic Erlang question even though its being used for writing Webmachine resources. Basically my problem stems from extends not adding the base modules exports to the exports of the module extending it, as far as I can tell at least form module_info(functions). So something like: -model(resource_user). -extends(resource_base). Where resource_base exports some Webmachine functions like: -exports([init/1]). init([]) -> {ok, #ctx{}}. Does not work becase Webmachine does not see a resource_user:init/1. Does anyone have a suggestion on an alternative way to achieve this? I want to be able to override the function in resource_user so it does not use the resource_base or default Webmachine function. Thanks, Tristan -------------- next part -------------- An HTML attachment was scrubbed... URL: From pfisher@REDACTED Mon Sep 5 19:57:40 2011 From: pfisher@REDACTED (Paul Fisher) Date: Mon, 5 Sep 2011 12:57:40 -0500 Subject: [erlang-questions] building erlang/otp dev on mac os x? In-Reply-To: <4E64ED7F.1050900@cs.ntua.gr> References: <55D701EA-6AB1-4D05-8F5C-E79732E3CD67@alertlogic.com> <4E64ED7F.1050900@cs.ntua.gr> Message-ID: <9B059C13-3DE0-4738-86F6-430124A1E172@alertlogic.com> Yeah, sorry about that. The tests take a long time to run and I wanted to let other mac users know that there was a possible viable alternative. I'm running more tests off and on today and will report those results, if there is interest. Was kinda hoping to move mac users past the "use -O0" impasse... On Sep 5, 2011, at 10:40 AM, Kostis Sagonas wrote: > On 09/05/11 18:22, Paul Fisher wrote: >> Just a follow-up on this. Installing gcc 4.5.3 from macports, allows the current dev branch to build and pass most of the tests. > > I do not have a Mac and for this reason perhaps I should not really > care, but if I had one I would have found this information a bit > uninformative. > > The interesting question is whether the dev branch passes at least all > tests as those when compiled with -O0, i.e. whether the optimization > passes in the updated macports compiler are still causing trouble for > (parts of) Erlang/OTP or not. > > If not, either the compiler or Erlang/OTP needs fixing. Till then I > would recommend to all MacOS users to keep using -O0 anyway. > > Kostis > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- paul director, platform services alertlogic, inc. 713-484-8383 x2314 "When in doubt, use brute force." -- Ken Thompson From essen@REDACTED Mon Sep 5 20:27:17 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Mon, 05 Sep 2011 20:27:17 +0200 Subject: [erlang-questions] Alternative Default Webmachine Functions In-Reply-To: References: Message-ID: <4E651485.8060308@dev-extend.eu> On 09/05/2011 07:52 PM, Tristan Sloughter wrote: > This should be a generic Erlang question even though its being used for > writing Webmachine resources. > > Basically my problem stems from extends not adding the base modules > exports to the exports of the module extending it, as far as I can tell > at least form module_info(functions). So something like: > > -model(resource_user). > -extends(resource_base). > > Where resource_base exports some Webmachine functions like: > > -exports([init/1]). > > init([]) -> > {ok, #ctx{}}. > > Does not work becase Webmachine does not see a resource_user:init/1. > > Does anyone have a suggestion on an alternative way to achieve this? I > want to be able to override the function in resource_user so it does not > use the resource_base or default Webmachine function. Not answering your question directly, but can't you solve that using a parse_transform instead? Add the missing functions at compile time with it. -- Lo?c Hoguin Dev:Extend From tristan.sloughter@REDACTED Mon Sep 5 20:33:43 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Mon, 5 Sep 2011 13:33:43 -0500 Subject: [erlang-questions] Alternative Default Webmachine Functions In-Reply-To: <4E651485.8060308@dev-extend.eu> References: <4E651485.8060308@dev-extend.eu> Message-ID: Hehe, yeah, that is my backup solution. When I go to reach for a parse transform or writing my own behaviour I always try to just assume I'm missing something simple. And not until I'm sure of no other option I go with those. Tristan On Mon, Sep 5, 2011 at 1:27 PM, Lo?c Hoguin wrote: > On 09/05/2011 07:52 PM, Tristan Sloughter wrote: > > This should be a generic Erlang question even though its being used for > > writing Webmachine resources. > > > > Basically my problem stems from extends not adding the base modules > > exports to the exports of the module extending it, as far as I can tell > > at least form module_info(functions). So something like: > > > > -model(resource_user). > > -extends(resource_base). > > > > Where resource_base exports some Webmachine functions like: > > > > -exports([init/1]). > > > > init([]) -> > > {ok, #ctx{}}. > > > > Does not work becase Webmachine does not see a resource_user:init/1. > > > > Does anyone have a suggestion on an alternative way to achieve this? I > > want to be able to override the function in resource_user so it does not > > use the resource_base or default Webmachine function. > > Not answering your question directly, but can't you solve that using a > parse_transform instead? Add the missing functions at compile time with it. > > -- > Lo?c Hoguin > Dev:Extend > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lenartlad@REDACTED Mon Sep 5 20:51:57 2011 From: lenartlad@REDACTED (Ladislav Lenart) Date: Mon, 05 Sep 2011 20:51:57 +0200 Subject: [erlang-questions] tcp connections dropped in gen_server In-Reply-To: References: Message-ID: <4E651A4D.4070705@volny.cz> Hello. I think the problem of missing log messages (and hanging connection processes) might be in the fact that you don't handle messages of the form {tcp_error, Socket, Reason} in your loop/1 function. Also, if I understand the code correctly, the newly created connection processes (acceptors) are not supervised. To prevent future problems with this I strongly recommend you to modify your code slightly as suggested in the book "Erlang and OTP in action". NOTE: I haven't even attempted to compile the following code (taken from the book and adapted to your use case). Modified process structure: simple_one_for_one - one for each ListenSocket loop - one for each existing TCP connection on the ListenSocket acceptor - one on the ListenSocket %%%%%%%%%%%%%%%%%%%%%% %%% TCP supervisor %%% %%%%%%%%%%%%%%%%%%%%%% -module(tcp_sup). -behaviour(supervisor). -export([start_link/1, start_child/1]). -export([init/1]). start_link({port, Port}) -> {ok, ListenSocket} = gen_tcp:listen(Port, [binary, {packet, 0}, {reuseaddr, true}, {active, true}]), start_link({listen_socket, ListenSocket}); start_link({listen_socket, ListenSocket}) -> supervisor:start_link(?MODULE, [ListenSocket]). start_child(SupPid) -> supervisor:start_child(SupPid, []). init([ListenSocket]) -> Server = {tcp_srv, {tcp_srv, start_link, [self(), ListenSocket]}, temporary, brutal_kill, worker, [tcp_srv]}, RestartStrategy = {simple_one_for_one, 0, 1}, % <-- tune for production demands {ok, {RestartStrategy, [Server]}}. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%% TCP server (aceptor + loop for one TCP connection %%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -module(tcp_srv). -export([start_link/2]). -export([acceptor/2]). start_link(SupPid, ListenSocket) -> proc_lib:spawn_link(?MODULE, acceptor, [SupPid, ListenSocket]). acceptor(SupPid, ListenSocket) -> {ok, Socket} = gen_tcp:accept(ListenSocket), tcp_sup:start_child(SupPid, ListenSocket), % <-- Instruct the tcp_sup SupPid to start new acceptor process. error_logger:info_msg("New connection from ~p~n", [Socket]), inet:setopts(Socket, [binary, {nodelay, true}, {active, true}]), loop(Socket). loop(Socket) -> %% As before. You should also consider to introduce a flow control to limit unbounded memory usage under heavy load using {active, false} for ListenSocket and {active, once} for Socket: 1. [in tcp_sup:start_link/1] {ok, ListenSocket} = gen_tcp:listen(Port, [binary, {packet, 0}, {reuseaddr, true}, {active, false}]), 2. [in tcp_srv:acceptor/2] inet:setopts(Socket, [binary, {nodelay, true}, {active, once}]), 3. [modify tcp_srv:loop/1] loop(Socket) -> receive {tcp, Socket, Data} -> inet:setopts(Socket, [{active, once}]), % <-- added line error_logger:info_msg("Messaged received from ~p: ~p~n", [Socket, Data]), comm_lib:handle_message(Socket, Data), loop(Socket); {tcp_closed, Socket} -> error_logger:info_msg("Device at ~p disconnected~n", [Socket]); _Any -> %% skip this loop(Socket) end. HTH, Ladislav Lenart On 5.9.2011 18:59, Reynaldo Baquerizo wrote: > I have a running application that consist in a supervisor and two > generic servers, one of them wraps around odbc and the other handles > tcp connections, a fragment of the relevant code is: > > > init([]) -> > process_flag(trap_exit, true), > {ok, ListenSocket} = gen_tcp:listen(Port, [binary, {packet, 0}, > {reuseaddr, true}, > {active, true}]), > proc_lib:spawn_link(?MODULE, acceptor, [ListenSocket]) > > acceptor(ListenSocket) -> > {ok, Socket} = gen_tcp:accept(ListenSocket), > error_logger:info_msg("New connection from ~p~n", [Socket]), > _Pid = proc_lib:spawn(?MODULE, acceptor, [ListenSocket]), > inet:setopts(Socket, [binary, {nodelay, true}, {active, true}]), > loop(Socket). > > loop(Socket) -> > receive > {tcp, Socket, Data} -> > error_logger:info_msg("Messaged received from ~p: ~p~n", [Socket, Data]), > comm_lib:handle_message(Socket, Data), > loop(Socket); > {tcp_closed, Socket} -> > error_logger:info_msg("Device at ~p disconnected~n", [Socket]); > _Any -> > %% skip this > loop(Socket) > end. > > So, I basically start a new unlinked process for every new tcp > connection. It works just fine for a couple hours but then every tcp > connection is dropped gradually with message "Device at ~p > disconnected". The client will try to reconnect if connection is > closed. The tcp connection should only terminate if remote end closes > it or spawned proccess in the server crashes. > > After all connections were dropped, I can see with inet:i() that there > are established connections but no logging! > > Can anyone give some insight or point to the right direction to debug this? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From cgsmcmlxxv@REDACTED Mon Sep 5 23:40:51 2011 From: cgsmcmlxxv@REDACTED (George Catalin Serbanut) Date: Mon, 5 Sep 2011 23:40:51 +0200 Subject: [erlang-questions] tcp connections dropped in gen_server In-Reply-To: References: Message-ID: Hi, I've been testing one code found on internet and it works smoothly. The code can be found at: http://20bits.com/articles/erlang-a-generalized-tcp-server/ Take a look at the code and compare it with your code. It will help in solving your problem. About your code, it may be your code gets a bit confused by which connection belongs to which spawned thread and then releases an error which generates tcp_closed message. Nevertheless, I might be wrong. Better try to dump your tcp traffic and see where is the problem in the connections. Cheers, CGS On Mon, Sep 5, 2011 at 6:59 PM, Reynaldo Baquerizo wrote: > I have a running application that consist in a supervisor and two > generic servers, one of them wraps around odbc and the other handles > tcp connections, a fragment of the relevant code is: > > > init([]) -> > process_flag(trap_exit, true), > {ok, ListenSocket} = gen_tcp:listen(Port, [binary, {packet, 0}, > > {reuseaddr, true}, > > {active, true}]), > proc_lib:spawn_link(?MODULE, acceptor, [ListenSocket]) > > acceptor(ListenSocket) -> > {ok, Socket} = gen_tcp:accept(ListenSocket), > error_logger:info_msg("New connection from ~p~n", [Socket]), > _Pid = proc_lib:spawn(?MODULE, acceptor, [ListenSocket]), > inet:setopts(Socket, [binary, {nodelay, true}, {active, true}]), > loop(Socket). > > loop(Socket) -> > receive > {tcp, Socket, Data} -> > error_logger:info_msg("Messaged received from ~p: ~p~n", > [Socket, Data]), > comm_lib:handle_message(Socket, Data), > loop(Socket); > {tcp_closed, Socket} -> > error_logger:info_msg("Device at ~p disconnected~n", [Socket]); > _Any -> > %% skip this > loop(Socket) > end. > > So, I basically start a new unlinked process for every new tcp > connection. It works just fine for a couple hours but then every tcp > connection is dropped gradually with message "Device at ~p > disconnected". The client will try to reconnect if connection is > closed. The tcp connection should only terminate if remote end closes > it or spawned proccess in the server crashes. > > After all connections were dropped, I can see with inet:i() that there > are established connections but no logging! > > Can anyone give some insight or point to the right direction to debug this? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From antoine.koener@REDACTED Tue Sep 6 00:05:32 2011 From: antoine.koener@REDACTED (Antoine Koener) Date: Tue, 6 Sep 2011 00:05:32 +0200 Subject: [erlang-questions] tcp connections dropped in gen_server In-Reply-To: References: Message-ID: On Mon, Sep 5, 2011 at 11:40 PM, George Catalin Serbanut < cgsmcmlxxv@REDACTED> wrote: > Hi, > > I've been testing one code found on internet and it works smoothly. The > code can be found at: > > http://20bits.com/articles/erlang-a-generalized-tcp-server/ > > Take a look at the code and compare it with your code. It will help in > solving your problem. > > I find that http://www.trapexit.org/Building_a_Non-blocking_TCP_server_using_OTP_principles is the best since it's - OTP based - never blocks - is a little bit more l33t because of "prim_inet:async_accept(Listen_socket, -1)". (-; my 2c. > > > > On Mon, Sep 5, 2011 at 6:59 PM, Reynaldo Baquerizo wrote: > >> I have a running application that consist in a supervisor and two >> generic servers, one of them wraps around odbc and the other handles >> tcp connections, a fragment of the relevant code is: >> >> >> init([]) -> >> process_flag(trap_exit, true), >> {ok, ListenSocket} = gen_tcp:listen(Port, [binary, {packet, 0}, >> >> {reuseaddr, true}, >> >> {active, true}]), >> proc_lib:spawn_link(?MODULE, acceptor, [ListenSocket]) >> >> acceptor(ListenSocket) -> >> {ok, Socket} = gen_tcp:accept(ListenSocket), >> error_logger:info_msg("New connection from ~p~n", [Socket]), >> _Pid = proc_lib:spawn(?MODULE, acceptor, [ListenSocket]), >> inet:setopts(Socket, [binary, {nodelay, true}, {active, true}]), >> loop(Socket). >> >> loop(Socket) -> >> receive >> {tcp, Socket, Data} -> >> error_logger:info_msg("Messaged received from ~p: ~p~n", >> [Socket, Data]), >> comm_lib:handle_message(Socket, Data), >> loop(Socket); >> {tcp_closed, Socket} -> >> error_logger:info_msg("Device at ~p disconnected~n", [Socket]); >> _Any -> >> %% skip this >> loop(Socket) >> end. >> >> So, I basically start a new unlinked process for every new tcp >> connection. It works just fine for a couple hours but then every tcp >> connection is dropped gradually with message "Device at ~p >> disconnected". The client will try to reconnect if connection is >> closed. The tcp connection should only terminate if remote end closes >> it or spawned proccess in the server crashes. >> >> After all connections were dropped, I can see with inet:i() that there >> are established connections but no logging! >> >> Can anyone give some insight or point to the right direction to debug >> this? >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Sep 6 00:14:35 2011 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 6 Sep 2011 10:14:35 +1200 Subject: [erlang-questions] Newbie Question, brute force method. In-Reply-To: <1314958909984-3785742.post@n4.nabble.com> References: <1314958909984-3785742.post@n4.nabble.com> Message-ID: <9467F092-27D4-4F37-8301-117102161DF3@cs.otago.ac.nz> Consider a generic loop State := Initial for Var in Collection do State := Update(State, Context) od where State is the set of variables defined outside the loop and updated inside the loop and Context is the set of variables defined outside the loop but invariant inside the loop. To functional programmers, this is a "fold". fold([X|Xs], A, F) -> fold(Xs, F(A, X), F); fold([], A, _) -> A. ... fold(Collection, Initial, fun (State) -> Update(State, Context) end) (fold/3 is available in the lists module.) Let's consider the case of "arg min": arg_min(Xs, F) -> {Arg_Min, _} = fold(Xs, nothing, fun (Old_State, X) -> FX = F(X), case Old_State of {_, FY} when FY <= FX -> Old_State ; _ -> {X, FX} end), Arg_Min. This will crash if Xs is empty, as it should. Now your brute force search for the best resistor and capacitor values is basically an arg min, it's just iterating over three lists instead of one. You should be able to take it from there. (By the way, very very very few people implement Erlang. A lot more people just use it.) From fritchie@REDACTED Tue Sep 6 07:37:39 2011 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Tue, 06 Sep 2011 00:37:39 -0500 Subject: [erlang-questions] Introducing new work on adding DTrace probes to the Erlang VM Message-ID: <62454.1315287459@snookles.snookles.com> Good evening. If you hadn't read the tweet that I'd made last week, I've started work on introducing DTrace probes to the Erlang VM. Some DTrace work was done on the R12 Erlang runtime by Garry Bulmer and Tim Watson. If anyone out there knows how to contact either of them, please contact me. Much of their work has been lost (at least to me) in the mists of time. The work is on a branch called "dtrace-experiment" over on GitHub, see https://github.com/slfritchie/otp/tree/dtrace-experiment and then scroll down far enough so that you can see the README text. The branch was created off of the "dev" branch from the Ericsson OTP repo on 23 August. Code contributions are welcome! This is a side project for me (SLF), so things would go faster if other people are willing to pitch in. Please use the GitHub pull request mechanism or send me an email message. At the moment, the only thing that has had probes added thoroughly is the efile_drv driver. (There are some toy probes for erlang:spawn(), but it was just a proof-of-concept toy.) But that's a big chunk of code. According to "diff", I've removed about 400 lines of text and added about 1,400 lines. % /path/to/debug-build/bin/erl +A 6 Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:8:8] [rq:8] \ [async-threads:6] [hipe] [kernel-poll:false] [type-assertions] \ [debug-compiled] [lock-checking] [dtrace] Eshell V5.8.5 (abort with ^G) 1> Happy hacking! -Scott From bgustavsson@REDACTED Tue Sep 6 07:51:41 2011 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Tue, 6 Sep 2011 07:51:41 +0200 Subject: [erlang-questions] building erlang/otp dev on mac os x? In-Reply-To: <9B059C13-3DE0-4738-86F6-430124A1E172@alertlogic.com> References: <55D701EA-6AB1-4D05-8F5C-E79732E3CD67@alertlogic.com> <4E64ED7F.1050900@cs.ntua.gr> <9B059C13-3DE0-4738-86F6-430124A1E172@alertlogic.com> Message-ID: On Mon, Sep 5, 2011 at 7:57 PM, Paul Fisher wrote: > Yeah, sorry about that. ?The tests take a long time to run and I wanted to let other mac users know that there was a possible viable alternative. > > I'm running more tests off and on today and will report those results, if there is interest. > > Was kinda hoping to move mac users past the "use -O0" impasse... > We hope to have R14B04 to build out-of-the-box on Lion. Our plan is to have 'configure' set up the Makefiles so that beam_emu.c is built with gcc-4.2 and everything else with the default compiler (usually llvm-gcc-4.2.). Until then, the easiest way to build Erlang/OTP seems to be: CC=gcc-4.2 ./configure make We have not run all test suites on Lion yet, but I have successfully built Erlang/OTP and run the emulator and compiler test suites. Our long-term plan is to find out exactly what causes the problem compiling beam_emu.c and submit a bug report to the llvm team. /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From erlang@REDACTED Tue Sep 6 10:24:53 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 6 Sep 2011 10:24:53 +0200 Subject: [erlang-questions] When to catch (was: Module config data and functional programming) In-Reply-To: References: Message-ID: I just thought of another rule: "Never return a false answer" - so the answer to "when should I let it crash" is "you should should crash if the alternative to crashing is to return a false answer" Often I use the rule "you should crash if you don't know what to do". All of this is motivated by some code I saw long, long ago. I was reading some code. It read (something like): f(...) -> ...; f(X) -> io:format("I don't know what to do in function f X=~p~n", [X]). Now f was a function of type int() -> int(). The last clause, however returns the atom 'ok' (this is what io:format returns). f now produces a type incorrect return value (ie it lies about it's type) - this will cause something else to crash somewhere else - but we don't want the program to crash "somewhere else" we want it to crash *immediately* so we can debug the original error and not the knock-on error caused by it crashing "somewhere else". If you *omit* the last clause of f this is exactly what will happen. if you want *force* the programmers attention to this error then write the last clause as exit({an_informative_tuple}) If when you wrote the program you didn't know what to do at this point in the code - then no amount of catching in the world will help - all you can do is report the error and return to some stable state. This situation occurs extremely often in programming - as you write code you get to points where you don't know what to do this is due to bad specifications and incomplete understanding of the problem. Programming *is* the art of understanding a problem - once you have complete understood the problem the program is in a sense "written" (at least in your head) - it remains to type it into the machine. Thus when the problem is not solved you will often not know what to do. You should always crash immediate to draw attention to the problem and not make matters worse (that's while you are developing) - when you ship the product you change strategy. Add a top level catch and hide the error from the user - put as much debugging info as possible in some error log and tell the developer - try to return the system to a stable state. Basically, functions should "never lie" ie if they say they are going to do something they should do what then say and nothing else - they should crash rather than lie. This is the programmatic version of "do no evil". Hope that helps /Joe On Sun, Sep 4, 2011 at 3:31 AM, Daniel Dormont wrote: > This was an earlier thread of mine that inspired a new question. > > As the months go by I'm getting more comfortable with "let it crash" > as an idea and seeing its benefits. But it occurs to me: surely there > are exceptions. Or rather: in order for "let it crash" to work at one > level of abstraction or functionality, there must be actual > crash-handling code at a level beneath it. In Erlang, this seems to > often involve letting individual processes crash and their supervisors > react accordingly. But perhaps that's not always the right answer. So > my question is: > > When is "catch" the right tool for the job? In the example below, > Ejabberd's "hooks" handlers use catch when calling functions in > foreign modules that are (probably) not critical path to whatever it's > doing. That seems like a pretty decent case to me. Would you agree? > What are some cases common in Erlang where exceptions need to be > caught and handled (or ignored) within a single process? > > dan > > On Fri, Apr 22, 2011 at 1:30 PM, Per Melin wrote: >> On Fri, Apr 22, 2011 at 4:32 PM, Daniel Dormont >> wrote: >>> It also occurs to me that switching to gen_server would offer another advantage in that it would be easy to recover from crashes in my own code. If I were not to do that though, is the recommended practice just to put a 'catch' call around code that might break? >> >> I don't have enough context, but it sounds absolutely unnecessary to >> create a gen_server for this. >> >> I would consider it ejabberd's responsibility to handle any crashes in >> your code. They have asked you for a callback function and they must >> consider that it may break. And they are in a better position than you >> to know how to proceed from a crash here. I did have a look in >> ejabberd_hooks and they do catch exits from your function. Let it >> crash. >> >> I'm hesitant to touch the question of what is recommended practice >> since this is the kind of thread where Dr Richard O'Keefe usually >> comments after a while and turns everyone that posted before him into >> fools. >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From michal.niec@REDACTED Tue Sep 6 10:42:25 2011 From: michal.niec@REDACTED (Michal Niec) Date: Tue, 6 Sep 2011 10:42:25 +0200 Subject: [erlang-questions] building erlang/otp dev on mac os x? In-Reply-To: References: <55D701EA-6AB1-4D05-8F5C-E79732E3CD67@alertlogic.com> <4E64ED7F.1050900@cs.ntua.gr> <9B059C13-3DE0-4738-86F6-430124A1E172@alertlogic.com> Message-ID: <79ACCFD3-01D2-462C-90E8-285ED4EE5FEC@erlang-solutions.com> We built and ran the full OTP test suite on R14B03 for Mac OS X Lion with llvm-gcc (CFLAGS=-O0 --enable-darwin-64bit --enable-hipe). Here is the link to the tests results http://binaries.erlang-solutions.com/R14B03/Mac%20OSX%2010.7/test_server/ . We will also have the tests results for dev branch soon. Regards, Michal Niec On Sep 6, 2011, at 7:51 AM, Bj?rn Gustavsson wrote: > On Mon, Sep 5, 2011 at 7:57 PM, Paul Fisher wrote: >> Yeah, sorry about that. The tests take a long time to run and I wanted to let other mac users know that there was a possible viable alternative. >> >> I'm running more tests off and on today and will report those results, if there is interest. >> >> Was kinda hoping to move mac users past the "use -O0" impasse... >> > > We hope to have R14B04 to build out-of-the-box > on Lion. Our plan is to have 'configure' set up > the Makefiles so that beam_emu.c is built with > gcc-4.2 and everything else with the default > compiler (usually llvm-gcc-4.2.). > > Until then, the easiest way to build Erlang/OTP > seems to be: > > CC=gcc-4.2 ./configure > make > > We have not run all test suites on Lion yet, > but I have successfully built Erlang/OTP and > run the emulator and compiler test suites. > > Our long-term plan is to find out exactly what > causes the problem compiling beam_emu.c > and submit a bug report to the llvm team. > > /Bjorn > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Tue Sep 6 10:47:34 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Tue, 6 Sep 2011 10:47:34 +0200 Subject: [erlang-questions] When to catch (was: Module config data and functional programming) In-Reply-To: References: Message-ID: On 6 Sep 2011, at 10:24, Joe Armstrong wrote: > Programming *is* the art of understanding a problem - once you have > complete understood the problem > the program is in a sense "written" (at least in your head) - it > remains to type it into the machine. > Thus when the problem is not solved you will often not know what to do. I completely agree. This also leads into the domain of "accidental complexity", where the answer to why you don't know what to do in a particular spot in your code, may well be that you shouldn't ever get there in the first place. The solution then could be to choose a simpler (or more adequate) overall approach. This is very easy to fall into in concurrency programming, and I have given some examples, e.g. in my EUC 2005 talk [1], and later in the Death by Accidental Complexity talk [2]. This comment is an example similar to Joe's, but where the authors eventually figured out what to do - even if it was a "false answer": %% Getting a ReleasePath when pending a Notify Released event is a bit %% complicated. We need to check for which path the ReleasePath is for and %% for which path the notify is for. If they are for different paths we are %% in a dilemma since we only can be in pending for one of them. As a simple %% way out we just treat this as an abnormal release for now. The "can only be in pending for one of them" was, AFAIK, an consequence of the implementation, and not of the problem itself. The code in question was later rewritten and _much_ simplified - to the point where it could even be understood by more than one person... [1] http://www.erlang.org/euc/05/1500Wiger.ppt [2] http://www.infoq.com/presentations/Death-by-Accidental-Complexity BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From lenartlad@REDACTED Tue Sep 6 13:15:32 2011 From: lenartlad@REDACTED (Ladislav Lenart) Date: Tue, 06 Sep 2011 13:15:32 +0200 Subject: [erlang-questions] tcp connections dropped in gen_server In-Reply-To: References: <4E651A4D.4070705@volny.cz> Message-ID: <4E6600D4.8030502@volny.cz> Hello. On 5.9.2011 23:40, Reynaldo Baquerizo wrote: >> [snip] >> >> Also, if I understand the code correctly, the newly created connection >> processes (acceptors) are not supervised. To prevent future problems >> with this I strongly recommend you to modify your code slightly as >> suggested in the book "Erlang and OTP in action". > > I didn't feel the need to supervised those connections. I fail to see > the difference between leaving them unattended and simple_one_for_one > with no restart. There was a comment tcp_sup:init/1 saying "tune for production demands" to give you a hint to change the restart strategy according to your needs. The advantage of having the connection processes supervised (with suitable restart strategy) is that the supervisor should log all unexpected crashes of the connection processes. It is also nice to look at appmon and see where the processes belong. > Thanks for the feedback ! You're welcome :-) NOTE (to anyone who should later read this thread): I've made a few mistakes in my previous code sample: * The supervisor should start its first child (acceptor) as part of its own init (hidden behind tcp:start_link/1 API call. Otherwise noone will be listening on the ListenSocket. * tcp_srv:start_link/2 should return {ok, pid()} instead of pid() to adhere to common expectations for start_link/X functions. In module tcp_sup: start_link({port, Port}) -> {ok, ListenSocket} = gen_tcp:listen(Port, [binary, {packet, 0}, {reuseaddr, true}, {active, true}]), start_link({listen_socket, ListenSocket}); start_link({listen_socket, ListenSocket}) -> {ok, SupPid} = supervisor:start_link(?MODULE, [ListenSocket]), {ok, _Pid} = tcp_sup:startChild(SupPid), % <-- Start the first acceptor. {ok, SupPid}. In module tcp_srv: start_link(SupPid, ListenSocket) -> {ok, proc_lib:spawn_link(?MODULE, acceptor, [SupPid, ListenSocket])}. Ladislav Lenart >> NOTE: I haven't even attempted to compile the following code (taken >> from the book and adapted to your use case). >> >> Modified process structure: >> simple_one_for_one - one for each ListenSocket >> loop - one for each existing TCP connection on the ListenSocket >> acceptor - one on the ListenSocket >> >> >> %%%%%%%%%%%%%%%%%%%%%% >> %%% TCP supervisor %%% >> %%%%%%%%%%%%%%%%%%%%%% >> -module(tcp_sup). >> >> -behaviour(supervisor). >> >> -export([start_link/1, start_child/1]). >> -export([init/1]). >> >> start_link({port, Port}) -> >> {ok, ListenSocket} = gen_tcp:listen(Port, [binary, {packet, 0}, >> {reuseaddr, true}, {active, true}]), >> start_link({listen_socket, ListenSocket}); >> start_link({listen_socket, ListenSocket}) -> >> supervisor:start_link(?MODULE, [ListenSocket]). >> >> start_child(SupPid) -> >> supervisor:start_child(SupPid, []). >> >> init([ListenSocket]) -> >> Server = {tcp_srv, {tcp_srv, start_link, [self(), ListenSocket]}, >> temporary, brutal_kill, worker, [tcp_srv]}, >> RestartStrategy = {simple_one_for_one, 0, 1}, %<-- tune for >> production demands >> {ok, {RestartStrategy, [Server]}}. >> >> >> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% >> %%% TCP server (aceptor + loop for one TCP connection %%% >> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% >> -module(tcp_srv). >> >> -export([start_link/2]). >> -export([acceptor/2]). >> >> start_link(SupPid, ListenSocket) -> >> proc_lib:spawn_link(?MODULE, acceptor, [SupPid, ListenSocket]). >> >> acceptor(SupPid, ListenSocket) -> >> {ok, Socket} = gen_tcp:accept(ListenSocket), >> tcp_sup:start_child(SupPid, ListenSocket), %<-- Instruct the tcp_sup >> SupPid to start new acceptor process. >> error_logger:info_msg("New connection from ~p~n", [Socket]), >> inet:setopts(Socket, [binary, {nodelay, true}, {active, true}]), >> loop(Socket). >> >> loop(Socket) -> >> %% As before. >> >> >> You should also consider to introduce a flow control to limit >> unbounded memory usage under heavy load using {active, false} >> for ListenSocket and {active, once} for Socket: >> 1. [in tcp_sup:start_link/1] {ok, ListenSocket} = gen_tcp:listen(Port, >> [binary, {packet, 0}, {reuseaddr, true}, {active, false}]), >> 2. [in tcp_srv:acceptor/2] inet:setopts(Socket, [binary, {nodelay, true}, >> {active, once}]), >> 3. [modify tcp_srv:loop/1] >> loop(Socket) -> >> receive >> {tcp, Socket, Data} -> >> inet:setopts(Socket, [{active, once}]), %<-- added line >> error_logger:info_msg("Messaged received from ~p: ~p~n", >> [Socket, Data]), >> comm_lib:handle_message(Socket, Data), >> loop(Socket); >> {tcp_closed, Socket} -> >> error_logger:info_msg("Device at ~p disconnected~n", >> [Socket]); >> _Any -> >> %% skip this >> loop(Socket) >> end. >> >> >> HTH, >> >> Ladislav Lenart >> >> >> On 5.9.2011 18:59, Reynaldo Baquerizo wrote: >>> >>> I have a running application that consist in a supervisor and two >>> generic servers, one of them wraps around odbc and the other handles >>> tcp connections, a fragment of the relevant code is: >>> >>> >>> init([]) -> >>> process_flag(trap_exit, true), >>> {ok, ListenSocket} = gen_tcp:listen(Port, [binary, {packet, 0}, >>> >>> {reuseaddr, true}, >>> >>> {active, true}]), >>> proc_lib:spawn_link(?MODULE, acceptor, [ListenSocket]) >>> >>> acceptor(ListenSocket) -> >>> {ok, Socket} = gen_tcp:accept(ListenSocket), >>> error_logger:info_msg("New connection from ~p~n", [Socket]), >>> _Pid = proc_lib:spawn(?MODULE, acceptor, [ListenSocket]), >>> inet:setopts(Socket, [binary, {nodelay, true}, {active, true}]), >>> loop(Socket). >>> >>> loop(Socket) -> >>> receive >>> {tcp, Socket, Data} -> >>> error_logger:info_msg("Messaged received from ~p: ~p~n", [Socket, >>> Data]), >>> comm_lib:handle_message(Socket, Data), >>> loop(Socket); >>> {tcp_closed, Socket} -> >>> error_logger:info_msg("Device at ~p disconnected~n", [Socket]); >>> _Any -> >>> %% skip this >>> loop(Socket) >>> end. >>> >>> So, I basically start a new unlinked process for every new tcp >>> connection. It works just fine for a couple hours but then every tcp >>> connection is dropped gradually with message "Device at ~p >>> disconnected". The client will try to reconnect if connection is >>> closed. The tcp connection should only terminate if remote end closes >>> it or spawned proccess in the server crashes. >>> >>> After all connections were dropped, I can see with inet:i() that there >>> are established connections but no logging! >>> >>> Can anyone give some insight or point to the right direction to debug >>> this? From dizzyd@REDACTED Tue Sep 6 15:12:02 2011 From: dizzyd@REDACTED (Dave Smith) Date: Tue, 6 Sep 2011 07:12:02 -0600 Subject: [erlang-questions] building erlang/otp dev on mac os x? In-Reply-To: References: <55D701EA-6AB1-4D05-8F5C-E79732E3CD67@alertlogic.com> <4E64ED7F.1050900@cs.ntua.gr> <9B059C13-3DE0-4738-86F6-430124A1E172@alertlogic.com> Message-ID: 2011/9/5 Bj?rn Gustavsson : > > Our long-term plan is to find out exactly what > causes the problem compiling beam_emu.c > and submit a bug report to the llvm team. Bjorn, the only context that I'm aware of is: http://llvm.org/bugs/show_bug.cgi?id=7883 https://github.com/lht/otp/commit/d57d2f47fd4a0fd28bfeeb3b5ebbe8201463dcb7 (Sorry if you've already seen those). It's also worth noting that a number of other projects have had to disable optimization (some variation of Ruby, gmplib, etc) on Lion due to all these issues. D. From caox@REDACTED Tue Sep 6 17:50:50 2011 From: caox@REDACTED (caox) Date: Tue, 6 Sep 2011 23:50:50 +0800 Subject: [erlang-questions] marco definition in beam_emu.c Message-ID: <3EFC8E23-CB4F-4CBA-8F15-47A1EFC06053@lightpole.net> Hi When reading the source code of beam_emu.c, I found the following marcos: #if defined(NO_JUMP_TABLE) # define OpCase(OpCode) case op_##OpCode: lb_##OpCode # define CountCase(OpCode) case op_count_##OpCode # define OpCode(OpCode) ((Uint*)op_##OpCode) # define Goto(Rel) {Go = (int)(Rel); goto emulator_loop;} # define LabelAddr(Addr) &&##Addr #else # define OpCase(OpCode) lb_##OpCode # define CountCase(OpCode) lb_count_##OpCode # define Goto(Rel) goto *(Rel) # define LabelAddr(Label) &&Label # define OpCode(OpCode) (&&lb_##OpCode) #endif which I couldn't understand. What is the meaning of 'op_##' and 'lb_##'? BR -------------- next part -------------- An HTML attachment was scrubbed... URL: From ngreco@REDACTED Tue Sep 6 18:01:18 2011 From: ngreco@REDACTED (Nahuel Greco) Date: Tue, 6 Sep 2011 13:01:18 -0300 Subject: [erlang-questions] marco definition in beam_emu.c In-Reply-To: <3EFC8E23-CB4F-4CBA-8F15-47A1EFC06053@lightpole.net> References: <3EFC8E23-CB4F-4CBA-8F15-47A1EFC06053@lightpole.net> Message-ID: Thats the gcc (or C99?) way to mark you want to evaluate a macro variable when is in contact with another string, so if you define: #define OpCase(OpCode)? ? case op_##OpCode: lb_##OpCode #define OpCase2(OpCode)? case op_OpCode: lb_OpCode OpCase(444) will be translated to "case op_444: lb_444" OpCase2(444) will be translated to "case_OpCode: lb_OpCode", and surely you don't want that. Saludos, Nahuel Greco. On Tue, Sep 6, 2011 at 12:50 PM, caox wrote: > Hi > When reading the source code of beam_emu.c, I found the following marcos: > #if defined(NO_JUMP_TABLE) > #? define OpCase(OpCode)? ? case op_##OpCode: lb_##OpCode > #? define CountCase(OpCode) case op_count_##OpCode > #? define OpCode(OpCode)? ? ((Uint*)op_##OpCode) > #? define Goto(Rel) {Go = (int)(Rel); goto emulator_loop;} > #? define LabelAddr(Addr) &&##Addr > #else > #? define OpCase(OpCode)? ? lb_##OpCode > #? define CountCase(OpCode) lb_count_##OpCode > #? define Goto(Rel) goto *(Rel) > #? define LabelAddr(Label) &&Label > #? define OpCode(OpCode)? (&&lb_##OpCode) > #endif > which I couldn't understand. What is the meaning of 'op_##' and 'lb_##'? > BR > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From lenartlad@REDACTED Tue Sep 6 19:55:24 2011 From: lenartlad@REDACTED (Ladislav Lenart) Date: Tue, 06 Sep 2011 19:55:24 +0200 Subject: [erlang-questions] tcp connections dropped in gen_server In-Reply-To: References: <4E651A4D.4070705@volny.cz> <4E6600D4.8030502@volny.cz> Message-ID: <4E665E8C.5020709@volny.cz> Hello. [I hope it's ok that I reply to the list - someone else might find this information useful as well. I am mentioning this only because you keep replying to me privately.] On 6.9.2011 17:52, Reynaldo Baquerizo wrote: >>>> Also, if I understand the code correctly, the newly created connection >>>> processes (acceptors) are not supervised. To prevent future problems >>>> with this I strongly recommend you to modify your code slightly as >>>> suggested in the book "Erlang and OTP in action". >>> >>> I didn't feel the need to supervised those connections. I fail to see >>> the difference between leaving them unattended and simple_one_for_one >>> with no restart. >> >> There was a comment tcp_sup:init/1 saying "tune for production demands" to > > Achh.. I read it as "tuned" :-) Good one! I have to be careful next time I invent "helpful comments" :-) >> give you a hint to change the restart strategy according to your needs. The >> advantage of having the connection processes supervised (with suitable >> restart > > Will those processes keep their state when restarted? > For instance, the socket will be closed or the restart will happen earlier? They won't be restarted. The socket will be closed because its controlling process (the one that performed gen_tcp:accept/1) just died (and the socket was linked to it). My sole aim was to put connection processes under supervision. This way you will be informed about their abnormal crashes. I proposed the following supervisor (in tcp_sup): init([ListenSocket]) -> Server = {tcp_srv, {tcp_srv, start_link, [self(), ListenSocket]}, temporary, brutal_kill, worker, [tcp_srv]}, RestartStrategy = {simple_one_for_one, 0, 1}, % <-- tune for production demands {ok, {RestartStrategy, [Server]}}. As you can see from the child specification (Server): * temporary - Instructs the supervisor to never restart its terminated children. * brutal_kill - Instructs the supervisor to kill its children (via exit(ChildPid, kill)) when it itself is about to terminate without giving them any chance to react whatsoever (e.g. clean up) but also without any possibility of failure of this operation. The brutal_kill is the only way to terminate the one process blocked indefinitely in call to gen_tcp:accept(ListenSocket). It makes very little sense to restart these processes (to me), because the TCP connection will die as well. The external client can reconnect and start anew. Or am I missing something? > I use a gen_server to encapsulate some state, does it matter? your > example of tcp_srv isn't a gen_server. Well, neither was yours, I just kept it that way to minimize changes in your original code :-) But gen_server it is... NOTES: * Again, I haven't even made an attempt to compile the following code. * The tricky bit is to postpone the initialization (i.e. gen_tcp:accept/1). * The server has no application specific state. However it should be straightforward to add it, preferably with something like -record(state, {supervisor, listen_socket, socket, ...}). %%% TCP connection process -module(tcp_srv). -behaviour(gen_server). -export([start_link/2, stop/1]). -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). %%% API start_link(SupPid, ListenSocket) -> gen_server:start_link(?MODULE, [SupPid, ListenSocket], []). stop(Pid) -> gen_server:cast(Pid, stop). %%% Callbacks init([SupPid, ListenSocket]) -> %% Return from the init now because the supervisor is waiting for me. %% Postpone my initialization for later. %% accept will be the first message I will receive and I will block %% there indefinitely. self() ! accept, {ok, {SupPid, ListenSocket}}. handle_call(Msg, _From, State) -> {reply, {error, {unknown_request, Msg}}, State}. handle_cast(stop, State) -> {stop, normal, State}; handle_cast(_Msg, State) -> {noreply, State}. handle_info(accept, {SupPid, ListenSocket}) -> {ok, Socket} = gen_tcp:accept(ListenSocket), tcp_sup:start_child(SupPid, ListenSocket), % <-- Instruct the tcp_sup SupPid to start new acceptor process. error_logger:info_msg("New connection from ~p~n", [Socket]), inet:setopts(Socket, [binary, {nodelay, true}, {active, true}]), {noreply, Socket}; handle_info({tcp, Socket, Data}, State) -> error_logger:info_msg("Messaged received from ~p: ~p~n", [Socket, Data]), %% comm_lib:handle_message/3 is expected to return NewState. {noreply, comm_lib:handle_message(Socket, Data, State)}; handle_info({tcp_closed, Socket}, State) -> error_logger:info_msg("Device at ~p disconnected~n", [Socket]), {stop, normal, State}; handle_info(_Any, State) -> {noreply, State}. terminate(_Reason, _State) -> ok. code_change(_OldVsn, State, _Extra) -> {ok, State}. > I think I will reestructure my supervision tree as follow. One root > supervisor with one_for_one strategy, a child supervisor > (simple_one_for_one) for tcp client connections and child gen_server > worker, pretty much like the example hanging out there. Sounds good to me. Ladislav Lenart From bfulgham@REDACTED Tue Sep 6 20:08:27 2011 From: bfulgham@REDACTED (Brent Fulgham) Date: Tue, 6 Sep 2011 11:08:27 -0700 Subject: [erlang-questions] Problem with debugger:start() In-Reply-To: References: <172F8CE8-2F3D-4D8E-AC99-5B2B44FEA924@gmail.com> Message-ID: Hi Rommel, On Sun, Sep 4, 2011 at 6:42 PM, Rommel Garcia wrote: > Thanks Brent! Please let me know once it's ready for download... Appreciate > all the help guys! I've checked in a few updates to the erlang port to help with this problem. 1. Please be sure your libxml2 and libxslt libraries are built with the "+universal" variant. The Portfile syntax does not provide a mechanism to ensure that dependencies are built with specific variant state, so you must do this manually. 2. If you build erlang with +wxwidgets, you will get a 32-bit build of the VM. This allows you to run the wx examples. 3. I also updated ESDL, so you can play with that as well. It also requires a 32-bit build because it relies on wx for its OpenGL support. I tried to get Wings to build as well, but am running into a problem with the erl_steal_main_thread symbol, which is provided by the Erlang VM, but cannot be found by Wings for some reason. :-( Hopefully this will get you back to studying with the Armstrong book :-) -Brent From gnoblin@REDACTED Tue Sep 6 22:19:26 2011 From: gnoblin@REDACTED (Slav Pankratov) Date: Tue, 6 Sep 2011 23:19:26 +0300 Subject: [erlang-questions] launching rebar release on windows server 2008 Message-ID: Hello! Is it possible to launch a release generated with rebar (I create the release on MacOS) on windows (windows server 2008 r2)? I get "erlexec: can't execute binary file" error. thanks, Slav -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Tue Sep 6 23:09:41 2011 From: jwatte@REDACTED (Jon Watte) Date: Tue, 6 Sep 2011 14:09:41 -0700 Subject: [erlang-questions] Newbie question, finite state Machine failover In-Reply-To: References: Message-ID: Stateful, as in the fail-over needs to be "hot" and "online" and replicating the state of the first application faithfully? The danger with such approaches is that, if the state becomes corrupt through some chain of events, then the replicated copy may also be corrupt, and the "slave" crashes when the "master" crashes. It still works great in case of hardware failure on the master instance, of course. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Sun, Sep 4, 2011 at 9:06 AM, Thomas Elsgaard wrote: > Hi > > I am looking into using erlang for a FSM (finite state machine) and i will > need to implement a statefull failover mechanism between two physical > servers, which approch would you recommend for this? Any Best practice > advices? > > Thomas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pfisher@REDACTED Wed Sep 7 01:13:56 2011 From: pfisher@REDACTED (Paul Fisher) Date: Tue, 6 Sep 2011 18:13:56 -0500 Subject: [erlang-questions] building erlang/otp dev on mac os x? In-Reply-To: References: <55D701EA-6AB1-4D05-8F5C-E79732E3CD67@alertlogic.com> <4E64ED7F.1050900@cs.ntua.gr> <9B059C13-3DE0-4738-86F6-430124A1E172@alertlogic.com> Message-ID: <8FAE78A1-D6CD-4172-8E0F-AE853CE784D0@alertlogic.com> Bj?rn, Thank you for the information and the directional statement, it is very much appreciated. I did manage to run a test results comparison of "gcc-llvm-4.2 -O0" versus "gcc-4.2 -O3" for the emulator, test_server, kernel, stdlib, and erl_interface applications. Using "gcc-4.2 -O3" yields fewer failures for kernel and stdlib applications, and the same results for the rest. kernel: 298 gen_tcp_misc_SUITE iter_max_socks FAILED {timetrap_timeout,{gen_tcp_misc_SUITE,653}} stdlib: 765 shell_SUITE start_restricted_on_command_line FAILED {shell_SUITE,201} {badmatch,"** exception error: undefined function foo:local_...} While I did not have a chance yet to do an exhaustive search over the results, it appears that the gcc-4.2 version completes the test cases faster for the emulator package. To me, these results make: $ CC=gcc-4.2 ./configure --enable-darwin-64bit a viable alternative, at least for now. On Sep 6, 2011, at 12:51 AM, Bj?rn Gustavsson wrote: > On Mon, Sep 5, 2011 at 7:57 PM, Paul Fisher wrote: >> Yeah, sorry about that. The tests take a long time to run and I wanted to let other mac users know that there was a possible viable alternative. >> >> I'm running more tests off and on today and will report those results, if there is interest. >> >> Was kinda hoping to move mac users past the "use -O0" impasse... >> > > We hope to have R14B04 to build out-of-the-box > on Lion. Our plan is to have 'configure' set up > the Makefiles so that beam_emu.c is built with > gcc-4.2 and everything else with the default > compiler (usually llvm-gcc-4.2.). > > Until then, the easiest way to build Erlang/OTP > seems to be: > > CC=gcc-4.2 ./configure > make > > We have not run all test suites on Lion yet, > but I have successfully built Erlang/OTP and > run the emulator and compiler test suites. > > Our long-term plan is to find out exactly what > causes the problem compiling beam_emu.c > and submit a bug report to the llvm team. > > /Bjorn > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB -- paul director, platform services alertlogic, inc. 713-484-8383 x2314 "When in doubt, use brute force." -- Ken Thompson From mjtruog@REDACTED Wed Sep 7 06:24:59 2011 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 06 Sep 2011 21:24:59 -0700 Subject: [erlang-questions] [ANN] CloudI 0.1.8 Released Message-ID: <4E66F21B.2020807@gmail.com> Changes in this release include: * HTTP GET query string parsing for C/C++, Java, Python, and Ruby * All request/responses have a terminating null for sloppy C code * rebar build fix for R14B03 * autconf/installation fixes CloudI is currently alpha and in-test. 0.2.0 will be the beta release. Download from http://sourceforge.net/projects/cloudi/files/0.1.8a/cloudi-0.1.8.tar.gz/download Anyone with questions, comments, ideas, suggestions, criticisms, or concerns is welcome to send email. Thanks, Michael From ok@REDACTED Wed Sep 7 07:01:39 2011 From: ok@REDACTED (Richard O'Keefe) Date: Wed, 7 Sep 2011 17:01:39 +1200 Subject: [erlang-questions] marco definition in beam_emu.c In-Reply-To: References: <3EFC8E23-CB4F-4CBA-8F15-47A1EFC06053@lightpole.net> Message-ID: <68FDB280-A50D-4A64-B522-3B14EB116C8F@cs.otago.ac.nz> On 7/09/2011, at 4:01 AM, Nahuel Greco wrote: > Thats the gcc (or C99?) way to mark you want to evaluate a macro > variable when is in contact with another string, so if you define: It's C89, and it doesn't mark evaluation but concatenation. From fdmanana@REDACTED Wed Sep 7 08:28:52 2011 From: fdmanana@REDACTED (Filipe David Manana) Date: Tue, 6 Sep 2011 23:28:52 -0700 Subject: [erlang-questions] Mac OS X Lion (64bits at least), XCode 4.1 and Erlang OTP R14 Message-ID: I was having an issue with R14 (tried B01, B02, B03 and otp/dev git branch) on Mac OS X Lion with XCode 4.1. Basically very frequent crashes on init:restart/0 calls when the crypto application was loaded. I posted a solution and gdb debug session in case someone finds the same issue: https://gist.github.com/1199903 Hope it helps others. best regards, -- Filipe David Manana, "Reasonable men adapt themselves to the world. ?Unreasonable men adapt the world to themselves. ?That's why all progress depends on unreasonable men." From ulf.wiger@REDACTED Wed Sep 7 09:51:00 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 7 Sep 2011 09:51:00 +0200 Subject: [erlang-questions] Newbie question, finite state Machine failover In-Reply-To: References: Message-ID: <3AC559E3-721B-4859-9857-B77DEFEBCBAF@erlang-solutions.com> On 6 Sep 2011, at 23:09, Jon Watte wrote: > Stateful, as in the fail-over needs to be "hot" and "online" and replicating the state of the first application faithfully? > > The danger with such approaches is that, if the state becomes corrupt through some chain of events, then the replicated copy may also be corrupt, and the "slave" crashes when the "master" crashes. It still works great in case of hardware failure on the master instance, of course. You are right. One way to mitigate this is to put some effort into designing a replication format, which is not just mirroring the internal state. Not only will this reduce the likelihood of propagating corrupted state; it will also simplify potential future upgrades and extensions, and make it easier to analyse the traffic flowing between nodes. One should also think through at which points it is at all meaningful to replicate. I like to refer to "stable-state replication", which doesn't really say anything about the frequency of updates, but rather highlights that there are usually discrete points where recovery from error is meaningful. The transition states between these points tend to be volatile, and replicating them may serve little purpose. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From baliulia@REDACTED Wed Sep 7 10:14:13 2011 From: baliulia@REDACTED (=?UTF-8?B?SWduYXMgVnnFoW5pYXVza2Fz?=) Date: Wed, 07 Sep 2011 11:14:13 +0300 Subject: [erlang-questions] slave:start/3 time out when EUnit started with ERL_FLAGS Message-ID: <4E6727D5.4050400@gmail.com> Hi fellow Erlangers, I believe there might be a bug in EUnit. Consider this module: foo.erl ----------------------------------------------------------- -module(foo). -compile(export_all). works_ok() -> {ok, Host} = inet:gethostname(), {ok, _Node} = slave:start(Host, slavefoo, []). bug_test() -> error_logger:info_msg("Node ~p is alive? ~p~n", [node(), is_alive()]), {ok, _Node} = works_ok(). ----------------------------------------------------------- and a Makefile: ----------------------------------------------------------- compile: erlc foo.erl nobug: compile EUNIT="verbose" erl -sname foobar -s eunit test foo bug: compile EUNIT="verbose" ERL_FLAGS="-sname foobar" erl -s eunit test foo works: compile ERL_FLAGS="-sname foobar" erl -s foo works_ok works_too: compile erl -sname foobar -s foo works_ok ----------------------------------------------------------- Now if you run 'make nobug', everything is ok: =INFO REPORT==== 7-Sep-2011::11:00:33 === Node foobar@REDACTED is alive? true [0.197 s] ok ====================================================== However if you 'make bug' you get the following: =INFO REPORT==== 7-Sep-2011::11:00:19 === Node foobar@REDACTED is alive? true *timed out* undefined ======================================================= As you can see from the info reports, the node is alive in both cases. The slave:start/3 also works fine when EUnit is not involved. Can someone confirm that this is indeed a bug or explain this behaviour please? I am running Erlang R14B-2 on Linux and this has also been tested on another machine. Kind regards, Ignas From ulf.wiger@REDACTED Wed Sep 7 10:26:29 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 7 Sep 2011 10:26:29 +0200 Subject: [erlang-questions] slave:start/3 time out when EUnit started with ERL_FLAGS In-Reply-To: <4E6727D5.4050400@gmail.com> References: <4E6727D5.4050400@gmail.com> Message-ID: On 7 Sep 2011, at 10:14, Ignas Vy?niauskas wrote: > bug: compile > EUNIT="verbose" ERL_FLAGS="-sname foobar" erl -s eunit test foo From the init(3) man page on the '-s' flag: "The functions are executed sequentially in an initialization process, which then terminates normally and passes control to the user. This means that a -s call which does not return will block further processing; to avoid this, use some variant of spawn in such cases." eunit:test/1 runs to completion, blocking until the tests are done. Therefore, it can't be used directly with the -s flag. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From czinkos@REDACTED Wed Sep 7 10:27:35 2011 From: czinkos@REDACTED (Zsolt Czinkos) Date: Wed, 7 Sep 2011 10:27:35 +0200 Subject: [erlang-questions] register existing process as event handler Message-ID: Hello I have one beginner-question regarding gen_event: Is it possible to register an existing process as an event handler to an event manager (or to multiple event managers)? Thanks Best, Zsolt From cgsmcmlxxv@REDACTED Wed Sep 7 10:34:02 2011 From: cgsmcmlxxv@REDACTED (George Catalin Serbanut) Date: Wed, 7 Sep 2011 10:34:02 +0200 Subject: [erlang-questions] slave:start/3 time out when EUnit started with ERL_FLAGS In-Reply-To: References: <4E6727D5.4050400@gmail.com> Message-ID: Try also "-s init stop" at the end of your commends. That will ensure control release after the process is consumed. Cheers, CGS 2011/9/7 Ulf Wiger > > On 7 Sep 2011, at 10:14, Ignas Vy?niauskas wrote: > > bug: compile > EUNIT="verbose" ERL_FLAGS="-sname foobar" erl -s eunit test foo > > > From the init(3) man page on the '-s' flag: > > "The functions are executed sequentially in an initialization process, > which then terminates normally and passes control to the user. This means > that a -s call which does not return will block further processing; to avoid > this, use some variant of spawn in such cases." > > eunit:test/1 runs to completion, blocking until the tests are done. > Therefore, it can't be used directly with the -s flag. > > BR, > Ulf W > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Wed Sep 7 10:52:50 2011 From: erlangsiri@REDACTED (Siri Hansen) Date: Wed, 7 Sep 2011 10:52:50 +0200 Subject: [erlang-questions] systools:make_tar/2 fails because of {filename, join, [[]]} In-Reply-To: References: Message-ID: A correction for this is now included in the upcoming R14B04. Regards /siri 2011/8/31 Siri Hansen > Hi Nikola! > > A workaround for the problem is to give the path as "./ebin" instead of > just "ebin". > > The root cause is here: > > 35> file:path_open(["ebin"],"a.app",[read]). > {ok,<0.189.0>,"ebin/a.app"} > > 36> file:path_open(["./ebin"],"a.app",[read]). > {ok,<0.191.0>,"./ebin/a.app"} > > The last element of the returned tuple is used for setting #application.dir > ("ebin" vs "./ebin") - later on systools_make:appDir/1 tries to find the > parent directory of this... and that's where it fails. > > I'll try to get a solution into R14B04. > Regards > /siri > > > > 2011/8/31 Nikola Skoric > >> Hello everybody, >> >> Here is my problem: >> >> nick@REDACTED:~/code/egon_server$ erl -pa ebin >> Erlang R13B03 (erts-5.7.4) [source] [smp:4:4] [rq:4] [async-threads:0] >> [hipe] [kernel-poll:false] >> >> Eshell V5.7.4 (abort with ^G) >> 1> systools:make_script("es_rel-0.0.1", [{path, ["ebin"]}]). >> ok >> 2> systools:make_tar("es_rel-0.0.1", [{path, ["ebin"]},{erts, >> code:root_dir()}]). >> {{case_clause, >> {'EXIT', >> {function_clause, >> [{filename,join,[[]]}, >> {systools_make,add_appl,7}, >> {systools_make,'-add_applications/5-fun-0-',6}, >> {lists,foldl,3}, >> {systools_make,add_applications,5}, >> {systools_make,mk_tar,6}, >> {systools_make,mk_tar,5}, >> {systools_make,make_tar,2}]}}}, >> [{systools_make,'-add_applications/5-fun-0-',6}, >> {lists,foldl,3}, >> {systools_make,add_applications,5}, >> {systools_make,mk_tar,6}, >> {systools_make,mk_tar,5}, >> {systools_make,make_tar,2}, >> {erl_eval,do_apply,5}, >> {shell,exprs,6}]} >> error >> 3> >> >> Now, that is exactly the same problem as these two: >> >> http://erlang.2086793.n4.nabble.com/troubleshooting-systools-make-tar-td2117171.html >> http://erlang.org/pipermail/erlang-questions/2008-August/037463.html >> >> I tried things advised in those threads, but nothing helped. I'm >> trying to build a release of this project: >> https://github.com/egon-sim/egon_server. Does anybody have any hints >> where to start looking for error? >> >> Thanks in advance, >> Nikola >> >> -- >> "Strange women lying in ponds distributing swords is no basis for a >> system of government." >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Wed Sep 7 11:01:46 2011 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Wed, 7 Sep 2011 10:01:46 +0100 Subject: [erlang-questions] Developing killer / open source apps In-Reply-To: References: Message-ID: On 1 August 2011 23:53, Nick S wrote: > I have just finished implementing a "professional" Erlang server, and with > time and enthusiasm in hand, I would like to start working on something > personal. > I must say, it has been really fun writing apps in OTP and see it > processing millions of messages without ever crashing...! > > Would appreciate some new ideas, that would be useful to community :) > > Support for web services would be nice. There is some support for SOAP using a combination of code from erlsoap and erlsom, but it is a bit clunky. What would be nice is comprehensive support for this as it exists in other languages such as Java and Python. This would also encourage adoption of Erlang in the enterprise community. cheers Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Wed Sep 7 11:12:35 2011 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 07 Sep 2011 10:12:35 +0100 (BST) Subject: [erlang-questions] register existing process as event handler In-Reply-To: Message-ID: Event managers/handlers are different to gen servers/fsms. With a server/fsm each one is a separate process with one callback module which is either a gen_server or a gen_fsm behaviour. With event managers/handlers only the event manager is a separate process. Event handlers are just callback modules which you register with event managers. You can register many event handlers with each manager and register a handler with many managers. When an event manager receives an event it calls a callback function in each event handler module which has been registered with it. The callback functions are evaluated directly in the event manager process, so event handlers as such are not separate processes. Of course an event handler module can send messages for the events it processes, this is the common case, but such processes are not really the actual event handlers. Robert ----- Original Message ----- > Hello > > I have one beginner-question regarding gen_event: > > Is it possible to register an existing process as an event handler to > an event manager (or to multiple event managers)? > > Thanks > > Best, > > Zsolt > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From baliulia@REDACTED Wed Sep 7 11:14:32 2011 From: baliulia@REDACTED (yfyf) Date: Wed, 07 Sep 2011 12:14:32 +0300 Subject: [erlang-questions] slave:start/3 time out when EUnit started with ERL_FLAGS In-Reply-To: References: <4E6727D5.4050400@gmail.com> Message-ID: <4E6735F8.9010502@gmail.com> On 09/07/2011 11:26 AM, Ulf Wiger wrote: > > On 7 Sep 2011, at 10:14, Ignas Vy?niauskas wrote: > >> bug: compile >> EUNIT="verbose" ERL_FLAGS="-sname foobar" erl -s eunit test foo > > From the init(3) man page on the '-s' flag: > > "The functions are executed sequentially in an initialization process, > which then terminates normally and passes control to the user. This > means that a -s call which does not return will block further > processing; to avoid this, use some variant of spawn in such cases." > > eunit:test/1 runs to completion, blocking until the tests are done. > Therefore, it can't be used directly with the -s flag. > Is the '-s' flag really relevant here? Consider this: ------------------------------------------------------------ [~/dev/fooApp]$ ERL_FLAGS="-sname foobar" erl Erlang R14B (erts-5.8.1) [source] [rq:1] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) (foobar@REDACTED)1> eunit:test(foo). =INFO REPORT==== 7-Sep-2011::11:58:33 === Node foobar@REDACTED is alive? true foo: bug_test (module 'foo')...*timed out* undefined ======================================================= Failed: 0. Skipped: 0. Passed: 0. One or more tests were cancelled. ------------------------------------------------------------ However: ------------------------------------------------------------ [~/dev/fooApp]$ erl -sname foobar Erlang R14B (erts-5.8.1) [source] [rq:1] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) (foobar@REDACTED)1> eunit:test(foo). =INFO REPORT==== 7-Sep-2011::12:00:11 === Node foobar@REDACTED is alive? true Test passed. ok ------------------------------------------------------------ Did I misunderstand your answer? If so, could you maybe explain it in a bit more detail. The reason why I ended up in this situation is that before I was using net_kernel:start([eunit, shortnames]) in my test setup functions, but then Mnesia stopped responding to EUnit's calls for some reason and I started digging for the problem and found this. An off topic, but related question: is using net_kernel:start/1 in EUnit a bad idea in general? Thank you for your help. From rtrlists@REDACTED Wed Sep 7 11:25:27 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 7 Sep 2011 10:25:27 +0100 Subject: [erlang-questions] marco definition in beam_emu.c In-Reply-To: <3EFC8E23-CB4F-4CBA-8F15-47A1EFC06053@lightpole.net> References: <3EFC8E23-CB4F-4CBA-8F15-47A1EFC06053@lightpole.net> Message-ID: On Tue, Sep 6, 2011 at 4:50 PM, caox wrote: > Hi > > When reading the source code of beam_emu.c, I found the following marcos: > > #if defined(NO_JUMP_TABLE) > # define OpCase(OpCode) case op_##OpCode: lb_##OpCode > # define CountCase(OpCode) case op_count_##OpCode > # define OpCode(OpCode) ((Uint*)op_##OpCode) > # define Goto(Rel) {Go = (int)(Rel); goto emulator_loop;} > # define LabelAddr(Addr) &&##Addr > #else > # define OpCase(OpCode) lb_##OpCode > # define CountCase(OpCode) lb_count_##OpCode > # define Goto(Rel) goto *(Rel) > # define LabelAddr(Label) &&Label > # define OpCode(OpCode) (&&lb_##OpCode) > #endif > > which I couldn't understand. What is the meaning of 'op_##' and 'lb_##'? > > BR > > It allows you to glue together a literal piece of text and a passed in argument of the macro. For example, OpCase(FooBar) becomes case op_FooBar: lb_FooBar . Robby -------------- next part -------------- An HTML attachment was scrubbed... URL: From cgsmcmlxxv@REDACTED Wed Sep 7 12:21:32 2011 From: cgsmcmlxxv@REDACTED (George Catalin Serbanut) Date: Wed, 7 Sep 2011 12:21:32 +0200 Subject: [erlang-questions] slave:start/3 time out when EUnit started with ERL_FLAGS In-Reply-To: <4E6735F8.9010502@gmail.com> References: <4E6727D5.4050400@gmail.com> <4E6735F8.9010502@gmail.com> Message-ID: I thought you used make bug without stopping the first shell. Sorry, misunderstanding. ERL_FLAGS is equivalent with ERL_ZFLAGS which appends at the end of the command line. Now, it depends on how "command line" is interpreted (Eshell can be considered a continuation of the "erl" command line). Try ERL_AFLAGS. If you still get the problem, then it may be a misbehavior of the interpretation of those FLAGS. Cheers, CGS On Wed, Sep 7, 2011 at 11:14 AM, yfyf wrote: > On 09/07/2011 11:26 AM, Ulf Wiger wrote: > > > > On 7 Sep 2011, at 10:14, Ignas Vy?niauskas wrote: > > > >> bug: compile > >> EUNIT="verbose" ERL_FLAGS="-sname foobar" erl -s eunit test foo > > > > From the init(3) man page on the '-s' flag: > > > > "The functions are executed sequentially in an initialization process, > > which then terminates normally and passes control to the user. This > > means that a -s call which does not return will block further > > processing; to avoid this, use some variant of spawn in such cases." > > > > eunit:test/1 runs to completion, blocking until the tests are done. > > Therefore, it can't be used directly with the -s flag. > > > Is the '-s' flag really relevant here? Consider this: > > ------------------------------------------------------------ > [~/dev/fooApp]$ ERL_FLAGS="-sname foobar" erl > Erlang R14B (erts-5.8.1) [source] [rq:1] [async-threads:0] [hipe] > [kernel-poll:false] > > Eshell V5.8.1 (abort with ^G) > (foobar@REDACTED)1> eunit:test(foo). > > =INFO REPORT==== 7-Sep-2011::11:58:33 === > Node foobar@REDACTED is alive? true > foo: bug_test (module 'foo')...*timed out* > undefined > ======================================================= > Failed: 0. Skipped: 0. Passed: 0. > One or more tests were cancelled. > ------------------------------------------------------------ > > However: > > ------------------------------------------------------------ > [~/dev/fooApp]$ erl -sname foobar > Erlang R14B (erts-5.8.1) [source] [rq:1] [async-threads:0] [hipe] > [kernel-poll:false] > > Eshell V5.8.1 (abort with ^G) > (foobar@REDACTED)1> eunit:test(foo). > > =INFO REPORT==== 7-Sep-2011::12:00:11 === > Node foobar@REDACTED is alive? true > Test passed. > ok > ------------------------------------------------------------ > > > Did I misunderstand your answer? If so, could you maybe explain it in a > bit more detail. > The reason why I ended up in this situation is that before I was using > net_kernel:start([eunit, shortnames]) in my test setup functions, but > then Mnesia stopped responding to EUnit's calls for some reason and I > started digging for the problem and found this. > > An off topic, but related question: is using net_kernel:start/1 in EUnit > a bad idea in general? > > Thank you for your help. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From reynaldomic@REDACTED Wed Sep 7 12:55:35 2011 From: reynaldomic@REDACTED (Reynaldo Baquerizo) Date: Wed, 7 Sep 2011 05:55:35 -0500 Subject: [erlang-questions] tcp connections dropped in gen_server In-Reply-To: <4E665E8C.5020709@volny.cz> References: <4E651A4D.4070705@volny.cz> <4E6600D4.8030502@volny.cz> <4E665E8C.5020709@volny.cz> Message-ID: <20110907055535.7fef7c02@bertello.localdomain> > [I hope it's ok that I reply to the list - someone else might find > this information useful as well. I am mentioning this only because > you keep replying to me privately.] Ooops, apologies ... I certainly didn't mean to write to you alone. > It makes very little sense to restart these processes (to me), because > the TCP connection will die as well. The external client can reconnect > and start anew. Or am I missing something? Indeed, the client will reconnect. But I think I found the problem. The process that is listening for new connections in gen_tcp:accept/1 dies at some point, all other processes with established connections are fine but eventually crash (cause of bad input), no further reconnections will be possible. How can I isolate the listening process? or reestructure to restart it if it crashes? At the end, I simplified it and have this: -module(comm_tcp). -export([start_link/2, acceptor/2]). start_link(SupPid, ListenSocket) -> {ok, proc_lib:spawn_link(?MODULE, acceptor, [SupPid, ListenSocket])}. acceptor(SupPid, ListenSocket) -> {ok, Socket} = gen_tcp:accept(ListenSocket), {ok, Pid} = comm_client_sup:start_child(SupPid), %% gen_tcp:controlling_process(Socket, Pid), error_logger:info_msg("New connection from ~p~n", [Socket]), inet:setopts(Socket, [binary, {nodelay, true}, {active, true}]), loop(Socket). loop(Socket) -> receive {tcp, Socket, Data} -> error_logger:info_msg("Messaged received from ~p: ~p~n",[Socket, Data]), comm_lib:handle_message(Socket, Data), loop(Socket); {tcp_closed, Socket} -> error_logger:info_msg("Device at ~p disconnected~n",[Socket]); _Any -> loop(Socket) end. -- Reynaldo From baliulia@REDACTED Wed Sep 7 13:42:45 2011 From: baliulia@REDACTED (=?UTF-8?B?SWduYXMgVnnFoW5pYXVza2Fz?=) Date: Wed, 07 Sep 2011 14:42:45 +0300 Subject: [erlang-questions] slave:start/3 time out when EUnit started with ERL_FLAGS In-Reply-To: References: <4E6727D5.4050400@gmail.com> <4E6735F8.9010502@gmail.com> Message-ID: <4E6758B5.5050504@gmail.com> On 09/07/2011 01:21 PM, George Catalin Serbanut wrote: > ERL_FLAGS is equivalent with ERL_ZFLAGS which appends at the end of > the command line. Now, it depends on how "command line" is > interpreted (Eshell can be considered a continuation of the "erl" > command line). Try ERL_AFLAGS. If you still get the problem, then it > may be a misbehavior of the interpretation of those FLAGS. Unfortunately, I have already tried ERL_AFLAGS and the result is identical. From desired.mta@REDACTED Wed Sep 7 12:34:27 2011 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Wed, 7 Sep 2011 13:34:27 +0300 Subject: [erlang-questions] slave:start/3 time out when EUnit started with ERL_FLAGS In-Reply-To: References: <4E6727D5.4050400@gmail.com> <4E6735F8.9010502@gmail.com> Message-ID: <20110907103427.GA18850@jakstys.lt> On Wed, Sep 07, 2011 at 12:21:32PM +0200, George Catalin Serbanut wrote: > ERL_FLAGS is equivalent with ERL_ZFLAGS which appends at the end of the > command line. Now, it depends on how "command line" is interpreted (Eshell > can be considered a continuation of the "erl" command line). Thanks for response. Can this be a catch? > Try ERL_AFLAGS. If you still get the problem, then it may be a > misbehavior of the interpretation of those FLAGS. Does not work as well. Let me rephrase: running $ ERL_FLAGS="-sname eunit" erl -pa . -s eunit test foo according the documentation of ERL_FLAGS, should be equivalent to $ erl -pa . -s eunit test foo -sname eunit However, these two cases behave differently. Regarding Ulf's response, if "-s eunit test" blocked until completion, the second case would not work, and at least, -sname would not be parsed? However, we see the node name. Do I misunderstand what ERL_FLAGS does? As far as I see, it should alter the command line before any module execution. As we see from printed node name, the command line is parsed. Motiejus From cgsmcmlxxv@REDACTED Wed Sep 7 14:49:18 2011 From: cgsmcmlxxv@REDACTED (George Catalin Serbanut) Date: Wed, 7 Sep 2011 14:49:18 +0200 Subject: [erlang-questions] slave:start/3 time out when EUnit started with ERL_FLAGS In-Reply-To: <20110907103427.GA18850@jakstys.lt> References: <4E6727D5.4050400@gmail.com> <4E6735F8.9010502@gmail.com> <20110907103427.GA18850@jakstys.lt> Message-ID: It may be that ERL_FLAGS can be the problem. Try "erl -sname foobar" directly instead of using ERL_FLAGS. 2011/9/7 Motiejus Jak?tys > On Wed, Sep 07, 2011 at 12:21:32PM +0200, George Catalin Serbanut wrote: > > ERL_FLAGS is equivalent with ERL_ZFLAGS which appends at the end of the > > command line. Now, it depends on how "command line" is interpreted > (Eshell > > can be considered a continuation of the "erl" command line). > > Thanks for response. Can this be a catch? > > > Try ERL_AFLAGS. If you still get the problem, then it may be a > > misbehavior of the interpretation of those FLAGS. > > Does not work as well. > > Let me rephrase: > running > $ ERL_FLAGS="-sname eunit" erl -pa . -s eunit test foo > according the documentation of ERL_FLAGS, should be equivalent to > $ erl -pa . -s eunit test foo -sname eunit > > However, these two cases behave differently. > > Regarding Ulf's response, if "-s eunit test" blocked until completion, > the second case would not work, and at least, -sname would not be > parsed? However, we see the node name. > > Do I misunderstand what ERL_FLAGS does? As far as I see, it should alter > the command line before any module execution. As we see from printed > node name, the command line is parsed. > > Motiejus > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lambdadmitry@REDACTED Wed Sep 7 15:42:19 2011 From: lambdadmitry@REDACTED (Dmitry Groshev) Date: Wed, 7 Sep 2011 17:42:19 +0400 Subject: [erlang-questions] Strange behaviour of =:= applied to int and int64 Message-ID: I'm using R14B03 and writing a little NIF library. Today I've encountered a strange behaviour of =:= when it is used against number that is set in sources and term that was constructed as enif_make_unit64. Here is an example: (circa_dev@REDACTED)1> {ok, T} = ecirca:new(5, last, small). {ok,{ecirca,#Ref<0.0.0.57>,<<>>,<0.48.0>,small}} (circa_dev@REDACTED)2> {ok, A} = ecirca:size(T). {ok,5} (circa_dev@REDACTED)3> B = 5. 5 (circa_dev@REDACTED)4> A == B. true (circa_dev@REDACTED)5> A =:= B. false (circa_dev@REDACTED)6> A. 5 (circa_dev@REDACTED)7> B. 5 This behaviour looks quite weird to me. Is it expected? From jws@REDACTED Wed Sep 7 16:03:04 2011 From: jws@REDACTED (Jeff Schultz) Date: Thu, 8 Sep 2011 00:03:04 +1000 Subject: [erlang-questions] Strange behaviour of =:= applied to int and int64 In-Reply-To: References: Message-ID: <20110907140304.GA9479@mulga.csse.unimelb.edu.au> On Wed, Sep 07, 2011 at 05:42:19PM +0400, Dmitry Groshev wrote: > I'm using R14B03 and writing a little NIF library. Today I've > encountered a strange behaviour of =:= when it is used against number > that is set in sources and term that was constructed as > enif_make_unit64. Here is an example: Is it possible that you've made a "big" integer with a value of 5 in your NIF? I've not looked at how BEAM handles this, but one common arrangement is to use tagged immediate values for "small" integers, and tagged pointers to some big-integer implementation for *all* other integers. The lack of overlap between the representations is then used as an opportunity for optimisation because no small integer can equal a big integer. As we have 1> 1==1.0. true 2> 1=:=1.0. false it would be unsurprising if =:=/2 (a.k.a. "exactly equal to") were false when applied to 5 as a "small" integer and 5 as a "big" integer. > (circa_dev@REDACTED)1> {ok, T} = ecirca:new(5, last, small). > {ok,{ecirca,#Ref<0.0.0.57>,<<>>,<0.48.0>,small}} > (circa_dev@REDACTED)2> {ok, A} = ecirca:size(T). > {ok,5} > (circa_dev@REDACTED)3> B = 5. > 5 > (circa_dev@REDACTED)4> A == B. > true > (circa_dev@REDACTED)5> A =:= B. > false > (circa_dev@REDACTED)6> A. > 5 > (circa_dev@REDACTED)7> B. > 5 Jeff Schultz From lenartlad@REDACTED Wed Sep 7 16:15:52 2011 From: lenartlad@REDACTED (Ladislav Lenart) Date: Wed, 07 Sep 2011 16:15:52 +0200 Subject: [erlang-questions] tcp connections dropped in gen_server In-Reply-To: <20110907055535.7fef7c02@bertello.localdomain> References: <4E651A4D.4070705@volny.cz> <4E6600D4.8030502@volny.cz> <4E665E8C.5020709@volny.cz> <20110907055535.7fef7c02@bertello.localdomain> Message-ID: <4E677C98.4040004@volny.cz> Hello. On 7.9.2011 12:55, Reynaldo Baquerizo wrote: >> [I hope it's ok that I reply to the list - someone else might find >> this information useful as well. I am mentioning this only because >> you keep replying to me privately.] > > Ooops, apologies ... I certainly didn't mean to write to you alone. No problem :-) >> It makes very little sense to restart these processes (to me), because >> the TCP connection will die as well. The external client can reconnect >> and start anew. Or am I missing something? > > Indeed, the client will reconnect. But I think I found the problem. The > process that is listening for new connections in gen_tcp:accept/1 dies > at some point, all other processes with established connections are fine > but eventually crash (cause of bad input), no further reconnections > will be possible. I see. Can you provide us with the exact cause of it (i.e. what gen_tcp:accept/1 returned)? It seems to me that under normal operation (if the accept was successful at least once before), there should be no problem of this kind... > How can I isolate the listening process? or reestructure to restart it > if it crashes? You need to restructure the processes then: port (service) supervisor - one_to_one acceptor - worker session supervisor - simple_one_to_one connection - worker Notes: * The above is a complete hierarchy of one TCP service listening on a given port. * There's exactly one acceptor under a port supervisor. The acceptor creates a listening socket and calls gen_tcp:accept/1 on it. It's terminated via brutal_kill (because of the blocking nature of the gen_tcp:accept/1 call). If it crashes it will be restarted by the port supervisor. * session_supervisor is basically a tcp_sup. * connection is essentially the loop(Socket) part of the previous connection process. These don't have to be killed via brutal_kill because they don't block. All the "magic" happens in the acceptor process... %%%%%%%%%%%%%%%%%%%% %%% TCP acceptor %%% %%%%%%%%%%%%%%%%%%%% -module(tcp_acceptor). start_link(SupPid, Port) -> {ok, proc_lib:spawn_link(?MODULE, init, [SupPid, Port])}. init(SupPid, Port) -> %% IMPORTANT: active must be set to false. {ok, ListenSocket} = gen_tcp:listen(Port, [binary, {packet, 0}, {reuseaddr, true}, {active, false}]), accept(SupPid, ListenSocket). accept(SupPid, ListenSocket) -> {ok, Socket} = gen_tcp:accept(ListenSocket), start_connection(SupPid, Socket), accept(SupPid, ListenSocket). start_connection(SupPid, Socket) -> %% tcp_session_sup is the id of the simple_one_to_one session supervisor %% in the child specification of the port supervisor (SupPid here). Kids = supervisor:which_children(SupPid), {value, {tcp_session_sup, SessionSup, _, _}} = lists:keysearch(tcp_session_sup, 1, Kids), {ok, Pid}} = supervisor:start_child(SessionSup, []), %% Force Socket to send future messages to Pid and not to me. ok = gen_tcp:controlling_process(Socket, Pid), %% Inform Pid about Socket (let it initialize the TCP session). %% As a bare minimum Pid should remember Socket in its internal state %% and set active flag to one of {active, true} or {active, once}. Pid ! {init_tcp_session, Socket}. I hope the rest of the picture is clear now. Note also that this can be turned into a generic TCP service / application fairly easily. You just need to parametrize the above with: * CallbackModule - name of the module that implements application specific connection process on top of TCP. Session supervisor needs it to create a desired child specification. * CallbackOptions - list of initial arguments passed to connection process via message {init_tcp_session, Socket, CallbackOptions}. * PacketOptions - list of PacketOptions passed to tcp_acceptor. If you have further questions, please do not hesitate and ask! :-) Ladislav Lenart From thomas.elsgaard@REDACTED Wed Sep 7 16:27:44 2011 From: thomas.elsgaard@REDACTED (Thomas Elsgaard) Date: Wed, 7 Sep 2011 16:27:44 +0200 Subject: [erlang-questions] Newbie question, finite state Machine failover In-Reply-To: <3AC559E3-721B-4859-9857-B77DEFEBCBAF@erlang-solutions.com> References: <3AC559E3-721B-4859-9857-B77DEFEBCBAF@erlang-solutions.com> Message-ID: On Wed, Sep 7, 2011 at 9:51 AM, Ulf Wiger wrote: > > On 6 Sep 2011, at 23:09, Jon Watte wrote: > >> Stateful, as in the fail-over needs to be "hot" and "online" and replicating the state of the first application faithfully? >> >> The danger with such approaches is that, if the state becomes corrupt through some chain of events, then the replicated copy may also be corrupt, and the "slave" crashes when the "master" crashes. It still works great in case of hardware failure on the master instance, of course. > > You are right. One way to mitigate this is to put some effort into designing a replication format, which is not just mirroring the internal state. Not only will this reduce the likelihood of propagating corrupted state; it will also simplify potential future upgrades and extensions, and make it easier to analyse the traffic flowing between nodes. > > One should also think through at which points it is at all meaningful to replicate. I like to refer to "stable-state replication", which doesn't really say anything about the frequency of updates, but rather highlights that there are usually discrete points where recovery from error is meaningful. The transition states between these points tend to be volatile, and replicating them may serve little purpose. > > BR, > Ulf W > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > Hi All Thanks for the input, there is no easy way ;-) But i will take it into my considerations ///Thomas From norton@REDACTED Wed Sep 7 16:42:45 2011 From: norton@REDACTED (Joseph Norton) Date: Wed, 7 Sep 2011 23:42:45 +0900 Subject: [erlang-questions] tcp connections dropped in gen_server In-Reply-To: <4E677C98.4040004@volny.cz> References: <4E651A4D.4070705@volny.cz> <4E6600D4.8030502@volny.cz> <4E665E8C.5020709@volny.cz> <20110907055535.7fef7c02@bertello.localdomain> <4E677C98.4040004@volny.cz> Message-ID: Hi. I can not help but offer a suggestion. As one working example, the UBF framework contains a TCP/IP listener - called proc_socket_server. https://github.com/norton/ubf/blob/master/src/proc_socket_server.erl The structure is simple and effective and doesn't use supervisors or even gen_servers. As previously suggested, supervisors adds unnecessary complexity (and possibly negative performance impact) than seems required for the task. I suspect your worker process will be different than UBF's worker. Nevertheless, the UBF worker's loop for reading and writing from the socket is simple and easy to follow. The worker is called - contract_driver. https://github.com/norton/ubf/blob/master/src/contract_driver.erl regards, Joseph Norton On Sep 7, 2011, at 11:15 PM, Ladislav Lenart wrote: > Hello. > > > On 7.9.2011 12:55, Reynaldo Baquerizo wrote: >>> [I hope it's ok that I reply to the list - someone else might find >>> this information useful as well. I am mentioning this only because >>> you keep replying to me privately.] >> >> Ooops, apologies ... I certainly didn't mean to write to you alone. > > No problem :-) > > >>> It makes very little sense to restart these processes (to me), because >>> the TCP connection will die as well. The external client can reconnect >>> and start anew. Or am I missing something? >> >> Indeed, the client will reconnect. But I think I found the problem. The >> process that is listening for new connections in gen_tcp:accept/1 dies >> at some point, all other processes with established connections are fine >> but eventually crash (cause of bad input), no further reconnections >> will be possible. > > I see. Can you provide us with the exact cause of it (i.e. what > gen_tcp:accept/1 returned)? It seems to me that under normal > operation (if the accept was successful at least once before), > there should be no problem of this kind... > > >> How can I isolate the listening process? or reestructure to restart it >> if it crashes? > > You need to restructure the processes then: > > port (service) supervisor - one_to_one > acceptor - worker > session supervisor - simple_one_to_one > connection - worker > > Notes: > * The above is a complete hierarchy of one TCP service listening on > a given port. > * There's exactly one acceptor under a port supervisor. The acceptor > creates a listening socket and calls gen_tcp:accept/1 on it. It's > terminated via brutal_kill (because of the blocking nature of the > gen_tcp:accept/1 call). If it crashes it will be restarted by the > port supervisor. > * session_supervisor is basically a tcp_sup. > * connection is essentially the loop(Socket) part of the previous > connection process. These don't have to be killed via brutal_kill > because they don't block. > > All the "magic" happens in the acceptor process... > > > %%%%%%%%%%%%%%%%%%%% > %%% TCP acceptor %%% > %%%%%%%%%%%%%%%%%%%% > -module(tcp_acceptor). > > start_link(SupPid, Port) -> > {ok, proc_lib:spawn_link(?MODULE, init, [SupPid, Port])}. > > init(SupPid, Port) -> > %% IMPORTANT: active must be set to false. > {ok, ListenSocket} = gen_tcp:listen(Port, [binary, {packet, 0}, {reuseaddr, true}, {active, false}]), > accept(SupPid, ListenSocket). > > accept(SupPid, ListenSocket) -> > {ok, Socket} = gen_tcp:accept(ListenSocket), > start_connection(SupPid, Socket), > accept(SupPid, ListenSocket). > > start_connection(SupPid, Socket) -> > %% tcp_session_sup is the id of the simple_one_to_one session supervisor > %% in the child specification of the port supervisor (SupPid here). > Kids = supervisor:which_children(SupPid), > {value, {tcp_session_sup, SessionSup, _, _}} = lists:keysearch(tcp_session_sup, 1, Kids), > {ok, Pid}} = supervisor:start_child(SessionSup, []), > %% Force Socket to send future messages to Pid and not to me. > ok = gen_tcp:controlling_process(Socket, Pid), > %% Inform Pid about Socket (let it initialize the TCP session). > %% As a bare minimum Pid should remember Socket in its internal state > %% and set active flag to one of {active, true} or {active, once}. > Pid ! {init_tcp_session, Socket}. > > > I hope the rest of the picture is clear now. Note also that this can be > turned into a generic TCP service / application fairly easily. You just > need to parametrize the above with: > * CallbackModule - name of the module that implements application specific > connection process on top of TCP. Session supervisor needs it to create > a desired child specification. > * CallbackOptions - list of initial arguments passed to connection process > via message {init_tcp_session, Socket, CallbackOptions}. > * PacketOptions - list of PacketOptions passed to tcp_acceptor. > > > If you have further questions, please do not hesitate and ask! :-) > > > Ladislav Lenart > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ulf.wiger@REDACTED Wed Sep 7 16:42:49 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 7 Sep 2011 16:42:49 +0200 Subject: [erlang-questions] Newbie question, finite state Machine failover In-Reply-To: References: <3AC559E3-721B-4859-9857-B77DEFEBCBAF@erlang-solutions.com> Message-ID: I should also say that some protocols allow for a "recovery window", which further simplifies stable-state replication. I know that some people will overlook this and try for hottest possible redundancy, but in several of the products I've been involved in, this property has been crucial. If you have resources allocated in hardware (e.g. forwarding engines, DSP:s, etc.), you will need to audit the resource management situation after a failure. If session state information is missing at some level, this may force you to release the entire session, even if most of the data is still there. Traditional Telecoms protocols expect sessions to have a data plane component and a control plane component, and will tolerate temporary loss of the control signaling. Normally, you will have 10-15 seconds to get your house in order and respond sanely to status inquiries. This is actually part of the secret behind the high availability figures achieved in Telecoms. Even the interaction with end users is designed to be fault-tolerant, giving the serving infrastructure some margin for recovery. The separation of the rather sensitive data path and the complex but fault-tolerant control plane is also important. Data processing units are kept as simple as possible, and are fault-isolated from the control plane. This model is becoming blurred by the trend towards tighter integration, but as a mental model, it is good to recall what the benefits of separation were. BR, Ulf W On 7 Sep 2011, at 16:27, Thomas Elsgaard wrote: > On Wed, Sep 7, 2011 at 9:51 AM, Ulf Wiger > wrote: >> >> On 6 Sep 2011, at 23:09, Jon Watte wrote: >> >>> Stateful, as in the fail-over needs to be "hot" and "online" and replicating the state of the first application faithfully? >>> >>> The danger with such approaches is that, if the state becomes corrupt through some chain of events, then the replicated copy may also be corrupt, and the "slave" crashes when the "master" crashes. It still works great in case of hardware failure on the master instance, of course. >> >> You are right. One way to mitigate this is to put some effort into designing a replication format, which is not just mirroring the internal state. Not only will this reduce the likelihood of propagating corrupted state; it will also simplify potential future upgrades and extensions, and make it easier to analyse the traffic flowing between nodes. >> >> One should also think through at which points it is at all meaningful to replicate. I like to refer to "stable-state replication", which doesn't really say anything about the frequency of updates, but rather highlights that there are usually discrete points where recovery from error is meaningful. The transition states between these points tend to be volatile, and replicating them may serve little purpose. >> >> BR, >> Ulf W >> >> Ulf Wiger, CTO, Erlang Solutions, Ltd. >> http://erlang-solutions.com >> >> >> >> > > Hi All > > Thanks for the input, there is no easy way ;-) But i will take it into > my considerations > > ///Thomas Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From lambdadmitry@REDACTED Wed Sep 7 16:49:31 2011 From: lambdadmitry@REDACTED (Dmitry Groshev) Date: Wed, 7 Sep 2011 07:49:31 -0700 (PDT) Subject: [erlang-questions] Strange behaviour of =:= applied to int and int64 In-Reply-To: <20110907140304.GA9479@mulga.csse.unimelb.edu.au> References: <20110907140304.GA9479@mulga.csse.unimelb.edu.au> Message-ID: <18611838.4589.1315406971081.JavaMail.geo-discussion-forums@yqaa12> >Is it possible that you've made a "big" integer with a value of 5 in your NIF I'm not sure how BEAM handle enif_make_uint64, but I think that it would be strange if BEAM represent such numbers as "big integers". Anyway, it's a clear abstraction leak. -------------- next part -------------- An HTML attachment was scrubbed... URL: From reynaldomic@REDACTED Wed Sep 7 19:33:39 2011 From: reynaldomic@REDACTED (Reynaldo Baquerizo) Date: Wed, 7 Sep 2011 12:33:39 -0500 Subject: [erlang-questions] tcp connections dropped in gen_server In-Reply-To: <4E677C98.4040004@volny.cz> References: <4E651A4D.4070705@volny.cz> <4E6600D4.8030502@volny.cz> <4E665E8C.5020709@volny.cz> <20110907055535.7fef7c02@bertello.localdomain> <4E677C98.4040004@volny.cz> Message-ID: >>> It makes very little sense to restart these processes (to me), because >>> the TCP connection will die as well. The external client can reconnect >>> and start anew. Or am I missing something? >> >> Indeed, the client will reconnect. But I think I found the problem. The >> process that is listening for new connections in gen_tcp:accept/1 dies >> at some point, all other processes with established connections are fine >> but eventually crash (cause of bad input), no further reconnections >> will be possible. > > I see. Can you provide us with the exact cause of it (i.e. what > gen_tcp:accept/1 returned)? It seems to me that under normal > operation (if the accept was successful at least once before), > there should be no problem of this kind... Achh,, found it (think so) I've hit the ERL_MAX_PORTS limit Log Report: =CRASH REPORT==== 7-Sep-2011::10:42:53 === crasher: initial call: comm_tcp:acceptor/2 pid: <0.1833.0> registered_name: [] exception error: no match of right hand side value {error,enfile} in function comm_tcp:acceptor/2 ancestors: [comm_client_sup,comm_sup,<0.51.0>] messages: [] links: [<0.61.0>] dictionary: [] trap_exit: false status: running heap_size: 233 stack_size: 24 reductions: 96 neighbours: =SUPERVISOR REPORT==== 7-Sep-2011::10:42:53 === Supervisor: {local,comm_client_sup} Context: child_terminated Reason: {badmatch,{error,enfile}} Offender: [{pid,<0.1833.0>}, {name,comm_tcp}, {mfargs,{comm_tcp,start_link,undefined}}, {restart_type,temporary}, {shutdown,brutal_kill}, {child_type,worker}] For further reference I am running Erlang on a Windows Server 2008. Besides setting ERL_MAX_PORTS to a higher number, any other suggestion? I have already found this thread http://erlang.2086793.n4.nabble.com/error-emfile-on-windows-td3064840.html in case anyone else find it useful. >> How can I isolate the listening process? or reestructure to restart it >> if it crashes? > > You need to restructure the processes then: > > ? ?port (service) supervisor - one_to_one > ? ? ? ?acceptor - worker > ? ? ? ?session supervisor - simple_one_to_one > ? ? ? ? ? ?connection - worker I don't want to augment the complexity, I will tune ERL_MAX_PORTS and handle the exhaustion with a case clause to avoid terminating the process. Thanks for the lengthy explanation > > If you have further questions, please do not hesitate and ask! :-) I will. -- Reynaldo From reynaldomic@REDACTED Wed Sep 7 19:38:08 2011 From: reynaldomic@REDACTED (Reynaldo Baquerizo) Date: Wed, 7 Sep 2011 12:38:08 -0500 Subject: [erlang-questions] tcp connections dropped in gen_server In-Reply-To: References: <4E651A4D.4070705@volny.cz> <4E6600D4.8030502@volny.cz> <4E665E8C.5020709@volny.cz> <20110907055535.7fef7c02@bertello.localdomain> <4E677C98.4040004@volny.cz> Message-ID: > Hi. ?I can not help but offer a suggestion. ?As one working example, the UBF framework contains a TCP/IP listener - called proc_socket_server. Thanks, I will study them. -- Reynaldo From paul.joseph.davis@REDACTED Wed Sep 7 19:48:53 2011 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Wed, 7 Sep 2011 12:48:53 -0500 Subject: [erlang-questions] Strange behaviour of =:= applied to int and int64 In-Reply-To: <18611838.4589.1315406971081.JavaMail.geo-discussion-forums@yqaa12> References: <20110907140304.GA9479@mulga.csse.unimelb.edu.au> <18611838.4589.1315406971081.JavaMail.geo-discussion-forums@yqaa12> Message-ID: There was a recent thread related to the halfword emulator that would cause this behavior. Not sure if its the same bug or not, but the thread has a patch that fixed it for me. Thread starts at [1] and the patch is at [2]. [1] http://erlang.org/pipermail/erlang-questions/2011-June/059650.html [2] http://erlang.org/pipermail/erlang-questions/2011-June/059728.html On Wed, Sep 7, 2011 at 9:49 AM, Dmitry Groshev wrote: >>Is it possible that you've made a "big" integer with a value of 5 in > your NIF > I'm not sure how BEAM handle enif_make_uint64, but I think that it would be > strange if BEAM represent such numbers as "big integers". > Anyway, it's a clear abstraction leak. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From oscar@REDACTED Wed Sep 7 23:20:15 2011 From: oscar@REDACTED (=?UTF-8?Q?Oscar_Hellstr=C3=B6m?=) Date: Wed, 07 Sep 2011 22:20:15 +0100 Subject: [erlang-questions] ssl:setopts/2 and packet type httph Message-ID: <4b50db2306ff96e1325b2ec1fa92ed78@hellstrom.st> Hi, I'm trying to use lhttpc with OTP HEAD from github and ran in to troubles when trying to call ssl:setopts(S, [{packet, httph}]). I started poking around and found this in the inet manual: "Note that the packet type httph is not needed when reading from a socket." Apart from that I can't find any reference to httph any more. I'm not sure what's meant by that but I guess I'm missing something here. I'm using packet type httph to read trailers after chunked transfer coding. Is packet type httph intentionally removed, and will it be so from pure tcp sockets also? Thanks /Oscar -- Oscar Hellstr?m, oscar@REDACTED web: http://oscar.hellstrom.st From jesper.louis.andersen@REDACTED Wed Sep 7 23:21:23 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 7 Sep 2011 23:21:23 +0200 Subject: [erlang-questions] Newbie question, finite state Machine failover In-Reply-To: References: <3AC559E3-721B-4859-9857-B77DEFEBCBAF@erlang-solutions.com> Message-ID: On Wed, Sep 7, 2011 at 16:42, Ulf Wiger wrote: > > I should also say that some protocols allow for a "recovery window", which further simplifies stable-state replication. I know that some people will overlook this and try for hottest possible redundancy, but in several of the products I've been involved in, this property has been crucial. > On another plane, having a smaller stable-state often makes it easier to verify for correctness. If you have some way to make sure that the state is stable, then by all means check it! You can often export this state to another process for periodic verification as well. While it does not _mend_ the error, it _detects_. Think about a system managing money. Like a certain Friar back in the day, Luca Pacioli, your system can do double-entry bookkeeping by having two processes, each playing the role of a ledger account. This means that you can check the system for a stable state by cross query on the ledgers. The stable state is small, namely the end balance of the account and it is enough for verification. The basic idea is that your process has an internal state s(), and your system defines several projections of the form -spec projectionX( s() ) -> t()., where t() is the type of the projection image. You use these projections for verification, for state marshalling, for system inspection and so on. It may be you need several different projections for different purposes, but often they can be coalesced into a few. As Ulf mentioned, simple t()'s means easier upgrade paths as well. Notice that state does not come equal. A lot of the internal state of a process is not valid if something goes wrong anyway. So there is little reason to keep it around. Sometimes, you keep state which acts as a scratch pad for your calculations, most often on the stack. This is not important either when things crash. Most crashes are due to state inconsistencies anyway, so keeping a leash on the scratchpad will definitely make your life worse. Sometimes, you are lucky and data are self-verifying. Sometimes not. But thinking about what properties your data will have is good for several reasons - It also makes it easier to write QuickCheck/ProPer tests. This is also why there is no easy way to do this. If you want to make a system fault tolerant, you need two machines as a start. But then you need to make sure that the right kind of information flows between the two machines. Erlang will make it easy to transfer information. But no language can verify the properties of the information that flows. At least not easily. -- J. From wilsoncj1@REDACTED Thu Sep 8 04:09:21 2011 From: wilsoncj1@REDACTED (Christopher Wilson) Date: Wed, 7 Sep 2011 22:09:21 -0400 Subject: [erlang-questions] Pass Name to Dynamic Supervisors Message-ID: Is there a way to pass a name/ID into the call to supervisor:start_child, more specifically to init()? I'd like to specify the name of the worker or supervisor such that when I ran supervisor:which_children I could identify what's what. Also, I've tried [unsuccessfully] to dynamically create a child spec and add pass that in. Are there any decent examples out there on how to do this outside of the man pages? Thank you. -Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: From wmacgyver@REDACTED Thu Sep 8 07:03:50 2011 From: wmacgyver@REDACTED (Wilson MacGyver) Date: Thu, 8 Sep 2011 01:03:50 -0400 Subject: [erlang-questions] building erlang/otp dev on mac os x? In-Reply-To: References: <55D701EA-6AB1-4D05-8F5C-E79732E3CD67@alertlogic.com> <4E64ED7F.1050900@cs.ntua.gr> <9B059C13-3DE0-4738-86F6-430124A1E172@alertlogic.com> Message-ID: Hi, Is there a target date for R14B04? I look around on the erlang.org site and can't seem to find it. Thanks 2011/9/6 Bj?rn Gustavsson : > We hope to have R14B04 to build out-of-the-box > on Lion. Our plan is to have 'configure' set up > the Makefiles so that beam_emu.c is built with > gcc-4.2 and everything else with the default > compiler (usually llvm-gcc-4.2.). -- Omnem crede diem tibi diluxisse supremum. From lenartlad@REDACTED Thu Sep 8 10:04:29 2011 From: lenartlad@REDACTED (Ladislav Lenart) Date: Thu, 08 Sep 2011 10:04:29 +0200 Subject: [erlang-questions] tcp connections dropped in gen_server In-Reply-To: References: <4E651A4D.4070705@volny.cz> <4E6600D4.8030502@volny.cz> <4E665E8C.5020709@volny.cz> <20110907055535.7fef7c02@bertello.localdomain> <4E677C98.4040004@volny.cz> Message-ID: <4E68770D.6020703@volny.cz> Hello. On 7.9.2011 19:33, Reynaldo Baquerizo wrote: >>>> It makes very little sense to restart these processes (to me), because >>>> the TCP connection will die as well. The external client can reconnect >>>> and start anew. Or am I missing something? >>> >>> Indeed, the client will reconnect. But I think I found the problem. The >>> process that is listening for new connections in gen_tcp:accept/1 dies >>> at some point, all other processes with established connections are fine >>> but eventually crash (cause of bad input), no further reconnections >>> will be possible. >> >> I see. Can you provide us with the exact cause of it (i.e. what >> gen_tcp:accept/1 returned)? It seems to me that under normal >> operation (if the accept was successful at least once before), >> there should be no problem of this kind... > > Achh,, found it (think so) > I've hit the ERL_MAX_PORTS limit Ok. > For further reference I am running Erlang on a Windows Server 2008. > > Besides setting ERL_MAX_PORTS to a higher number, any other suggestion? > I have already found this thread > http://erlang.2086793.n4.nabble.com/error-emfile-on-windows-td3064840.html > in case anyone else find it useful. No, I am afraid. Maybe someone else can help... >>> How can I isolate the listening process? or reestructure to restart it >>> if it crashes? >> >> You need to restructure the processes then: >> >> port (service) supervisor - one_to_one >> acceptor - worker >> session supervisor - simple_one_to_one >> connection - worker > > I don't want to augment the complexity, I will tune ERL_MAX_PORTS and > handle the exhaustion with a case clause to avoid terminating the > process. One question, how do you plan to handle the exhaustion of the ports, i.e. what will you do in the newly added case clause? > Thanks for the lengthy explanation You're welcome. Ladislav Lenart From sam@REDACTED Thu Sep 8 10:44:34 2011 From: sam@REDACTED (Samuel Elliott) Date: Thu, 8 Sep 2011 09:44:34 +0100 Subject: [erlang-questions] Pass Name to Dynamic Supervisors In-Reply-To: References: Message-ID: I managed this when writing an OS erlang app recently. in the supervisor: https://github.com/lenary/ironmaster/blob/master/src/im_provider_sup.erl#L38-40 and in the worker: https://github.com/lenary/ironmaster/blob/master/src/im_provider.erl#L56-62 Essentially, the supervisor's `start_child/1` takes the callback module, and adds a childspec to the supervisor. This childspec then calls down into the worker's `start_link/1` which takes the same callback module name as an argument, and starts a gen_server using itself. Yes, it sounds complicated, but it's not really. Sam On Thu, Sep 8, 2011 at 3:09 AM, Christopher Wilson wrote: > Is there a way to pass a name/ID into the call to supervisor:start_child, > more specifically to init()? I'd like to specify the name of the worker or > supervisor such that when I ran supervisor:which_children I could identify > what's what. > > Also, I've tried [unsuccessfully] to dynamically create a child spec and > add pass that in. Are there any decent examples out there on how to do this > outside of the man pages? > > Thank you. > > -Chris > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Samuel Elliott sam@REDACTED http://lenary.co.uk/ +44 (0)7891 993 664 -------------- next part -------------- An HTML attachment was scrubbed... URL: From ttom.kelly@REDACTED Thu Sep 8 10:56:14 2011 From: ttom.kelly@REDACTED (tom kelly) Date: Thu, 8 Sep 2011 09:56:14 +0100 Subject: [erlang-questions] Pass Name to Dynamic Supervisors In-Reply-To: References: Message-ID: Hi Chris, We do something like this in our application. First, in our "static" supervision tree we start a dynamic supervisor: init(Args) -> {ok, { {one_for_one, 5, 5}, [ {dynamic_supervisor_id, {dynamic_supervisor, start_link, [Args]}, permanent, infinity, supervisor, [dynamic_supervisor]}, Where the dynamic_supervisor module has the functions: start_link(Args) -> supervisor:start_link({local,?MODULE}, ?MODULE, [Args]). init(_Args) -> {ok, { {one_for_one, 5, 5}, [] }}. Then in our code we can dynamically start supervised processes with registered names, ProcName: supervisor:start_child(dynamic_supervisor, {ProcName, {gen_server, start_link, [{local, ProcName}, ?MODULE, [StartArgs], []]}, permanent, 10000, worker, [?MODULE] }) And shut them down with: supervisor:terminate_child(dynamic_supervisor,PName), supervisor:delete_child(dynamic_supervisor,PName), This is just a few clippings from our code base here with the names made more generic, so there might be a syntax error or two but I think it helps do what you want. //Tom. On Thu, Sep 8, 2011 at 3:09 AM, Christopher Wilson wrote: > Is there a way to pass a name/ID into the call to supervisor:start_child, > more specifically to init()? I'd like to specify the name of the worker or > supervisor such that when I ran supervisor:which_children I could identify > what's what. > > Also, I've tried [unsuccessfully] to dynamically create a child spec and > add pass that in. Are there any decent examples out there on how to do this > outside of the man pages? > > Thank you. > > -Chris > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From per@REDACTED Thu Sep 8 13:23:27 2011 From: per@REDACTED (Per Hedeland) Date: Thu, 8 Sep 2011 13:23:27 +0200 (CEST) Subject: [erlang-questions] tcp connections dropped in gen_server In-Reply-To: Message-ID: <201109081123.p88BNR4H022360@pluto.hedeland.org> Reynaldo Baquerizo wrote: > >Achh,, found it (think so) >I've hit the ERL_MAX_PORTS limit > exception error: no match of right hand side value {error,enfile} >For further reference I am running Erlang on a Windows Server 2008. I don't know if things are radically different on Windows (I would hope not), but on *nix 'enfile' means 1> file:format_error(enfile). "file table overflow" which is a system-wide (i.e. in your OS) limit on the number of open file descriptors, as opposed to 2> file:format_error(emfile). "too many open files" which is a per-OS-process limit for the same thing, i.e. in this case for the Erlang VM, whereas hitting ERL_MAX_PORTS results in 3> file:format_error(system_limit). "a system limit was hit, probably not enough ports" Btw, on *nix, ERL_MAX_PORTS defaults to the closest power of 2 at or above the limit that gives 'emfile' - but you can of course hit ERL_MAX_PORTS without getting 'emfile' when you use ports for things that aren't file descriptors - notably drivers. --Per Hedeland From lambdadmitry@REDACTED Thu Sep 8 15:58:38 2011 From: lambdadmitry@REDACTED (Dmitry Groshev) Date: Thu, 8 Sep 2011 06:58:38 -0700 (PDT) Subject: [erlang-questions] Strange behaviour of =:= applied to int and int64 In-Reply-To: References: <20110907140304.GA9479@mulga.csse.unimelb.edu.au> <18611838.4589.1315406971081.JavaMail.geo-discussion-forums@yqaa12> Message-ID: <2491271.1146.1315490318664.JavaMail.geo-discussion-forums@yqic37> I'm more interested in fixing this behaviour in future releases than patching BEAM myself, because I can easily avoid this error. Is there any hope for this? -------------- next part -------------- An HTML attachment was scrubbed... URL: From magnus.henoch@REDACTED Thu Sep 8 16:18:48 2011 From: magnus.henoch@REDACTED (Magnus Henoch) Date: Thu, 08 Sep 2011 15:18:48 +0100 (BST) Subject: [erlang-questions] Strange behaviour of =:= applied to int and int64 In-Reply-To: <2491271.1146.1315490318664.JavaMail.geo-discussion-forums@yqic37> Message-ID: <5001d350-cb59-41dc-a225-5292de9a3173@knuth> > I'm more interested in fixing this behaviour in future releases than > patching BEAM myself, because I can easily avoid this error. Is > there any hope for this? Yes, the patch in question has been applied to the dev branch: https://github.com/erlang/otp/blob/dev/erts/emulator/beam/erl_nif.c and will thus appear in R14B04, due to be released on 5th October IIRC. Regards, -- Magnus Henoch Erlang Solutions Ltd http://www.erlang-solutions.com/ From max.lapshin@REDACTED Thu Sep 8 17:17:49 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 8 Sep 2011 19:17:49 +0400 Subject: [erlang-questions] Pack whole project into one escript file Message-ID: Hi. I like the way that rebar is packaged. All files are zipped onto one big package and almost everything is cool. I have two questions: 1) how easy it would be to learn escript to die on SIGHUP? I want very, very much to learn erlyvideo to run under runit. It requires strict death on SIGHUP on closing stdin. 2) Is it possible to detect in runtime, that we are running in packaged mode and access files directly in this zip file? Currently I package wwwroot with erlyvideo and it would be good to serve files out of this big zip file. From dizzyd@REDACTED Thu Sep 8 17:45:56 2011 From: dizzyd@REDACTED (Dave Smith) Date: Thu, 8 Sep 2011 09:45:56 -0600 Subject: [erlang-questions] Pack whole project into one escript file In-Reply-To: References: Message-ID: On Thu, Sep 8, 2011 at 9:17 AM, Max Lapshin wrote: > 2) Is it possible to detect in runtime, that we are running in > packaged mode and access files directly in this zip file? Yes. Rebar does this for templates; see the code in src/rebar_templater.erl, esp. cache_escript_files/0. Also, rebar does some of the packaging work for you -- try "rebar escriptize". You can even include deps in that ZIP file by using the {escript_incl_apps, [apps...]}. See also, basho_bench's rebar.config. D. From smith.winston.101@REDACTED Thu Sep 8 19:37:33 2011 From: smith.winston.101@REDACTED (Winston Smith) Date: Thu, 8 Sep 2011 13:37:33 -0400 Subject: [erlang-questions] How to configure otp for a debug build (with otp_build)? Message-ID: Trying to get to the bottom of a segmentation fault on my cross compiled version of R14B03 on an AVR32 (this is running a reltool/rebar generated "node"): beam[1440]: segfault at 36842a0e pc 2b7cc05e sp 7ff27668 ecr 24 I had no luck with a core file (even with +d) and I also tried adding +v, but I get the message: warning: -v (only in debug compiled code) I need to rebuild in debug mode, but I couldn't immediately find any documentation on using otp_build to produce a debug build. Since this is cross compiled, I'm using otp_build with the xcomp configuration file erl-xcomp-avr32-atmel-linux-gnu.conf. Could someone give me a pointer on how to configure a cross compiled erlang for a debug build? Many thanks. From gustav.simonsson@REDACTED Fri Sep 9 10:47:00 2011 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Fri, 09 Sep 2011 09:47:00 +0100 (BST) Subject: [erlang-questions] How to configure otp for a debug build (with otp_build)? In-Reply-To: Message-ID: <076c88cc-f5df-413c-adef-9e9e3a3df577@knuth> There is a line with "CFLAGS=" in the xcomp file where you can add flags to gcc, such as debug options found in http://gcc.gnu.org/onlinedocs/gcc-4.6.1/gcc/Debugging-Options.html#Debugging-Options Cheers, Gustav ----- Ursprungligt meddelande ----- Fr?n: "Winston Smith" Till: erlang-questions@REDACTED Kopia: "Rickard Green" Skickat: torsdag, 8 sep 2011 19:37:33 ?mne: [erlang-questions] How to configure otp for a debug build (with otp_build)? Trying to get to the bottom of a segmentation fault on my cross compiled version of R14B03 on an AVR32 (this is running a reltool/rebar generated "node"): beam[1440]: segfault at 36842a0e pc 2b7cc05e sp 7ff27668 ecr 24 I had no luck with a core file (even with +d) and I also tried adding +v, but I get the message: warning: -v (only in debug compiled code) I need to rebuild in debug mode, but I couldn't immediately find any documentation on using otp_build to produce a debug build. Since this is cross compiled, I'm using otp_build with the xcomp configuration file erl-xcomp-avr32-atmel-linux-gnu.conf. Could someone give me a pointer on how to configure a cross compiled erlang for a debug build? Many thanks. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From johannes.auer@REDACTED Fri Sep 9 13:35:54 2011 From: johannes.auer@REDACTED (Johannes Auer) Date: Fri, 09 Sep 2011 13:35:54 +0200 Subject: [erlang-questions] Garbage Collector Details Message-ID: Greetings! I am interested in some details about the GC of Erlang. I found some papers explaining the private heap architecture of the GC, but the paper was lacking some important details: It says that a message is first copied to the private heap of the receiving process, and afterwards a pointer to that message is placed in the message queue. But copying to the heap would require locking, since many processes can send messages concurrently to the same receiving process. Could someone explain what is happening *exactly* when a message is sent from one process to another? Thanks for you help! Johannes From olivier.boudeville@REDACTED Fri Sep 9 14:00:55 2011 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Fri, 9 Sep 2011 14:00:55 +0200 Subject: [erlang-questions] Type specs expressivity Message-ID: Hi, I would have three small questions regarding what can be expressed with type specs: - we can define immediate constants and even ranges for integer values (ex: '-spec f() -> 42.' or '-spec f() -> 0..42.'), but apparently we cannot do the same for floats? Is there any particular reason for that (besides the lack of meaning of exact comparisons for floating-point numbers), or maybe it is a planned enhancement? - it is a still more minor point, but we are using a hashtable type in pure Erlang, which is implemented as a tuple containing buckets. The number of elements of the tuple is determined at run-time (depending on a targeted ideal load factor, based on the expected number of elements to be stored in the hashtable). We can define a type for bucket(), but for the hashtable is there any way of expressing a more precise declaration than '-opaque hashtable() :: tuple()', i.e. to tell that even if we do not know the size of the tuple, we know it will contain bucket() instances at each position? - what is the recommended way of handling functions that may or may not throw exceptions? If otherwise returning a type T, should we use T | none() or T | no_return() as return type, or should we consider that type specs focus on actual return types under normal circumstances, and that exceptions are in "another dimension"? I suppose it is the latter option, but then one could imagine that type specs for functions could be enriched with exception specifications, for example to check whether a catch clause has any chance of being triggered at all? Thanks in advance for any hint, Best regards, Olivier. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Fri Sep 9 17:07:33 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 9 Sep 2011 17:07:33 +0200 Subject: [erlang-questions] Garbage Collector Details In-Reply-To: References: Message-ID: On Fri, Sep 9, 2011 at 13:35, Johannes Auer wrote: > It says that a message is first copied to the private heap of the receiving process, and afterwards a pointer to that message is placed in the message queue. But copying to the heap would require locking, since many processes can send messages concurrently to the same receiving process. Could someone explain what is happening *exactly* when a message is sent from one process to another? There are three tricks being employed, and I am sure one from the Erlang/OTP team will correct me if I am wrong here. First, an Erlang process struct is governed by a series of locks (4 at the time being). For our purposes the most important locks are the MSGQ and STATUS locks which together can be called the SENDRECV lock. The two locks protect some specific fields in the process struct - we are concerned with the message queue called the InQueue. The InQueue is a linked list of incoming messages, protected by the MSGQ lock. But we also need the STATUS lock, so we will take both of these and call them the SENDRECV lock. There is also another important lock, the MAIN lock which protects every field not protected by a more specific lock. Second, notice that the heap of a process can be fragmented. That is, rather than consist of a contiguous piece of memory, it may consist of several smaller fragments. When we run a garbage collection in the process, these fragments will be combined and the fragments will be released. Third, the message queue is split into two. There is the above mentioned InQueue, protected by MSGQ. And there is also the PrivQueue, the private queue protected by MAIN. The idea is that if you hold the MSGQ lock, you can send messages but the process can still do stuff as long as it only needs MAIN. ++ Whenever a process is running, it holds MAIN. This allows it exclusive access to most fields and the PrivQueue. When process S wants to send to R, it does the following (in a very simplified explanation): 1. Allocate/Grab a new heap fragment. I don't know how much fragment reuse there is, but there could be some. 2. Copy the message to the heap fragment. We have exclusive right to the fragment. 3. Grab the MSGQ and STATUS locks of R. 4. Add the message (and fragment) to the InQueue. 5. Release the MSGQ and STATUS locks of R. 4 is mostly just a pointer assignment, so the time we hold the lock is extremely small. Also, we don't stop another scheduler with MAIN from executing the process. Even the copy is "free" because it happens outside the lock. When the process executes, it will grab MAIN and do its work off the PrivQueue. If it exhausts, it'll grab the MSGQ lock and evacuate the messages to the PrivQueue. There is also an important invariant, which is that InQueue *never* contains pointers into the process "main heap". This means that the Garbage Collector does not need to include InQueue in its root set. Garbage collection then amounts to combining all the fragments and collecting garbage. The neat trick here is that the fragments works as a handoff algorithm. The pointer to the fragment is a "token" and he who holds that token is responsible for its collection. When we allocate a new fragment/token and then hand it off to the R process, we give the R process the responsibility of collecting and reclaiming the data. But the allocation and copying work is done in S. Optimistically, S rarely blocks R and R rarely blocks S. Even if we have multiple senders, they hold the lock for a very short time, so they are unlikely to trip over each other much. Of course it goes wrong if there are scores of threads trying to get that lock, but chances are you have a badly designed program then. I should also mention that this is a message from S to R locally on the same machine with S and R different processes. The code path if S = R or if S and R are on different distributed nodes is different. -- J. From sverker@REDACTED Fri Sep 9 18:43:22 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Fri, 9 Sep 2011 18:43:22 +0200 Subject: [erlang-questions] Strange behaviour of =:= applied to int and int64 In-Reply-To: <5001d350-cb59-41dc-a225-5292de9a3173@knuth> References: <5001d350-cb59-41dc-a225-5292de9a3173@knuth> Message-ID: <4E6A422A.1070607@erix.ericsson.se> Magnus Henoch wrote: >> I'm more interested in fixing this behaviour in future releases than >> patching BEAM myself, because I can easily avoid this error. Is >> there any hope for this? >> > > Yes, the patch in question has been applied to the dev branch: > > https://github.com/erlang/otp/blob/dev/erts/emulator/beam/erl_nif.c > > and will thus appear in R14B04, due to be released on 5th October IIRC. > > Regards, > The mentioned patch in dev for R14B04 does only apply for halfword emulator: https://github.com/erlang/otp/commit/1c0966044b66d20f3f6607e91076e0a61918f1a4 So... are you running halfword emulator, Dmitry. /Sverker, Erlang/OTP From lambdadmitry@REDACTED Fri Sep 9 18:48:32 2011 From: lambdadmitry@REDACTED (Dmitry Groshev) Date: Fri, 9 Sep 2011 09:48:32 -0700 (PDT) Subject: [erlang-questions] Strange behaviour of =:= applied to int and int64 In-Reply-To: <4E6A422A.1070607@erix.ericsson.se> References: <5001d350-cb59-41dc-a225-5292de9a3173@knuth> <4E6A422A.1070607@erix.ericsson.se> Message-ID: <32800883.157.1315586912970.JavaMail.geo-discussion-forums@yqic37> yes. -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Fri Sep 9 19:03:30 2011 From: g@REDACTED (Garrett Smith) Date: Fri, 9 Sep 2011 12:03:30 -0500 Subject: [erlang-questions] Pass Name to Dynamic Supervisors In-Reply-To: References: Message-ID: On Wed, Sep 7, 2011 at 9:09 PM, Christopher Wilson wrote: > Is there a way to pass a name/ID into the call to supervisor:start_child, > more specifically to init()? ?I'd like to specify the name of the worker or > supervisor such that when I ran supervisor:which_children I could identify > what's what. > Also, I've tried [unsuccessfully] to dynamically create a child spec and add > pass that in. ?Are there any decent examples out there on how to do this > outside of the man pages? Take a look at: https://github.com/esl/gproc I'm not sure if that's the canonical repository -- maybe Ulf can weigh in. It's tempting, but I find it's best not to mess with the vanilla supervisory facility and instead use gproc for getting a list of named processes. You can also use gproc to publish interesting process stats to consumers, thus avoiding costly and intrusive queries to the processes themselves. This paper: https://github.com/esl/gproc/blob/master/doc/erlang07-wiger.pdf?raw=true provides the background for gproc and is otherwise an excellent read! You'll have a slight learning curve with gproc, but it's an *indispensable* tool so well worth it. It needs to be in the core :) Garrett From aronisstav@REDACTED Fri Sep 9 19:40:05 2011 From: aronisstav@REDACTED (Stavros Aronis) Date: Fri, 9 Sep 2011 10:40:05 -0700 (PDT) Subject: [erlang-questions] Type specs expressivity In-Reply-To: References: Message-ID: <33472150.2515.1315590005781.JavaMail.geo-discussion-forums@yqah42> Hi Olivier, Some answers to your questions: - we can define immediate constants and even ranges for integer > values (ex: '-spec f() -> 42.' or '-spec f() -> 0..42.'), but apparently we > cannot do the same for floats? Is there any particular reason for that > (besides the lack of meaning of exact comparisons for floating-point > numbers), or maybe it is a planned enhancement? > I am not aware of any specific reason behind that decision, apart from perhaps it being one step further from the functionality that was initially desired. To simply be able to write such a spec would be trivial, but to actually have tools like Dialyzer infer float values or ranges might be not that easy (or even significantly useful). > - it is a still more minor point, but we are using a hashtable type > in pure Erlang, which is implemented as a tuple containing buckets. The > number of elements of the tuple is determined at run-time (depending on a > targeted ideal load factor, based on the expected number of elements to be > stored in the hashtable). We can define a type for bucket(), but for the > hashtable is there any way of expressing a more precise declaration than > '-opaque hashtable() :: tuple()', i.e. to tell that even if we do not know > the size of the tuple, we know it will contain bucket() instances at each > position? Same as before, a type language can be extended to accommodate whatever your heart wishes. This one is rather simpler I guess and I have noted it down as a possible extension (it is also a matter of pride... PropEr has such loose_tuples(), Dialyzer should have them too!). Parametrical types in general are not currently fully integrated in the tool (apart from lists) but might become so in the near future! > - what is the recommended way of handling functions that may or may > not throw exceptions? If otherwise returning a type T, should we use T | > none() or T | no_return() as return type, or should we consider that type > specs focus on actual return types under normal circumstances, and that > exceptions are in "another dimension"? I suppose it is the latter option, > but then one could imagine that type specs for functions could be enriched > with exception specifications, for example to check whether a catch clause > has any chance of being triggered at all? > Adding none() or no_return() in a regular union of types might be useful from a documentation point of view, but is currently silently ignored by Dialyzer (if you read some of the publications about it's type system, namely success typings, you will understand why). Still, I wouldn't advice against writing such specs, as such kind of analysis might become available in the future (from Dialyzer or a completely different tool). Kostis might have more to add or have a different opinion on these. I hope these points were helpful nevertheless! Regards, Stavros Aronis -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Fri Sep 9 19:55:50 2011 From: jwatte@REDACTED (Jon Watte) Date: Fri, 9 Sep 2011 10:55:50 -0700 Subject: [erlang-questions] Garbage Collector Details In-Reply-To: References: Message-ID: That was a fantastically interesting description of the VM internals! Thank you very much! 1. Allocate/Grab a new heap fragment. Is there a global lock for allocating these fragments, or is the heap fragment allocator smarter than this? How is contention for this allocator resolved? Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Fri, Sep 9, 2011 at 8:07 AM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Fri, Sep 9, 2011 at 13:35, Johannes Auer wrote: > > > It says that a message is first copied to the private heap of the > receiving process, and afterwards a pointer to that message is placed in the > message queue. But copying to the heap would require locking, since many > processes can send messages concurrently to the same receiving process. > Could someone explain what is happening *exactly* when a message is sent > from one process to another? > > There are three tricks being employed, and I am sure one from the > Erlang/OTP team will correct me if I am wrong here. > > First, an Erlang process struct is governed by a series of locks (4 at > the time being). For our purposes the most important locks are the > MSGQ and STATUS locks which together can be called the SENDRECV lock. > The two locks protect some specific fields in the process struct - we > are concerned with the message queue called the InQueue. The InQueue > is a linked list of incoming messages, protected by the MSGQ lock. But > we also need the STATUS lock, so we will take both of these and call > them the SENDRECV lock. There is also another important lock, the MAIN > lock which protects every field not protected by a more specific lock. > > Second, notice that the heap of a process can be fragmented. That is, > rather than consist of a contiguous piece of memory, it may consist of > several smaller fragments. When we run a garbage collection in the > process, these fragments will be combined and the fragments will be > released. > > Third, the message queue is split into two. There is the above > mentioned InQueue, protected by MSGQ. And there is also the PrivQueue, > the private queue protected by MAIN. The idea is that if you hold the > MSGQ lock, you can send messages but the process can still do stuff as > long as it only needs MAIN. > > ++ > > Whenever a process is running, it holds MAIN. This allows it exclusive > access to most fields and the PrivQueue. When process S wants to send > to R, it does the following (in a very simplified explanation): > > 1. Allocate/Grab a new heap fragment. I don't know how much fragment > reuse there is, but there could be some. > 2. Copy the message to the heap fragment. We have exclusive right to > the fragment. > 3. Grab the MSGQ and STATUS locks of R. > 4. Add the message (and fragment) to the InQueue. > 5. Release the MSGQ and STATUS locks of R. > > 4 is mostly just a pointer assignment, so the time we hold the lock is > extremely small. Also, we don't stop another scheduler with MAIN from > executing the process. Even the copy is "free" because it happens > outside the lock. > > When the process executes, it will grab MAIN and do its work off the > PrivQueue. If it exhausts, it'll grab the MSGQ lock and evacuate the > messages to the PrivQueue. There is also an important invariant, which > is that InQueue *never* contains pointers into the process "main > heap". This means that the Garbage Collector does not need to include > InQueue in its root set. > > Garbage collection then amounts to combining all the fragments and > collecting garbage. The neat trick here is that the fragments works as > a handoff algorithm. The pointer to the fragment is a "token" and he > who holds that token is responsible for its collection. When we > allocate a new fragment/token and then hand it off to the R process, > we give the R process the responsibility of collecting and reclaiming > the data. But the allocation and copying work is done in S. > > Optimistically, S rarely blocks R and R rarely blocks S. Even if we > have multiple senders, they hold the lock for a very short time, so > they are unlikely to trip over each other much. Of course it goes > wrong if there are scores of threads trying to get that lock, but > chances are you have a badly designed program then. > > I should also mention that this is a message from S to R locally on > the same machine with S and R different processes. The code path if S > = R or if S and R are on different distributed nodes is different. > > > -- > J. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kukhyun@REDACTED Fri Sep 9 20:28:19 2011 From: kukhyun@REDACTED (KukHyun Lee) Date: Sat, 10 Sep 2011 03:28:19 +0900 Subject: [erlang-questions] mnesia clear_table() bug ? Message-ID: Hi. This is disc_only_copies table. Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.4 (abort with ^G) ... (db@REDACTED)103> mnesia:table_info(table, size). 151944 (db@REDACTED)104> mnesia:table_info(table, memory). 16295224 (db@REDACTED)105> mnesia:clear_table(table). {atomic,ok} (db@REDACTED)106> mnesia:table_info(table, size). 0 (db@REDACTED)107> mnesia:table_info(table, memory). 16295224 Why did not changed table's memory? Is this a bug? How can i set a table to zero memory. Thanks, KukHyun -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Fri Sep 9 21:24:59 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 9 Sep 2011 21:24:59 +0200 Subject: [erlang-questions] Garbage Collector Details In-Reply-To: References: Message-ID: On Fri, Sep 9, 2011 at 19:55, Jon Watte wrote: > Is there a global lock for allocating these fragments, or is the heap > fragment allocator smarter than this? How is contention for this allocator > resolved? Rickard Green is the right guy here, because he wrote the allocator framework. But you can in general use the same tricks as is done by parallel malloc() implementations. The idea is to split the (operating system) heap up into arenas and make each thread go to a different arena. This way, by default, lock contention is kept low since you will usually not have to compete for the lock at all and you can get your allocation done, free the lock and go on. Notice that you can get the uncontended lock/unlock case fast by using e.g., the linux futex(7) calls in which case uncontended locking doesn't even need the kernel involved. More advanced allocators keep a local thread cache of recently free'd objects which are allocated in an arena. So rather than give back such an object to the Arena as it is freed, the thread speculates that the object will be allocated again soon and keeps it in the thread as if it were allocated. Requests of an object whose size is in the cache already can skip the synchronization completely because the thread already has it, out of the main arena. This solution is also extremely (CPU) cache-friendly since a recently released object is more commonly in the (CPU) cache. I can imagine this is fairly easy to do when you are allocating data for a Garbage Collector process heap because objects will not have extremely fine granularity but be a bit more coarse, and often be of certain fixed sizes. One very cool thing about Erlang is that the erts_alloc framework is plugin-based. You can configure it and tune it if you need it. I can also imagine it is a very important part of coming SMP improvements. -- J. From comptekki@REDACTED Fri Sep 9 22:16:14 2011 From: comptekki@REDACTED (Wes James) Date: Fri, 9 Sep 2011 14:16:14 -0600 Subject: [erlang-questions] erl -run - what's needed Message-ID: I've tried erl -run module with a start function in the code and erl -run module fun, but it keeps crashing. What is needed for this to work right? I just want to run erl and have mod:fun be called as I start erl. Thanks, -wes From comptekki@REDACTED Fri Sep 9 22:21:13 2011 From: comptekki@REDACTED (Wes James) Date: Fri, 9 Sep 2011 14:21:13 -0600 Subject: [erlang-questions] erl -run - what's needed In-Reply-To: References: Message-ID: Never mind. I thought I had the .erl file compiled, but it wasn't. When I erlc file.erl and then ran erl -run mod it worked. -wes On Fri, Sep 9, 2011 at 2:16 PM, Wes James wrote: > I've tried erl -run module with a start function in the code and erl > -run module fun, but it keeps crashing. ?What is needed for this to > work right? ?I just want to run erl and have mod:fun be called as I > start erl. > > Thanks, > > -wes > From james@REDACTED Sat Sep 10 02:10:30 2011 From: james@REDACTED (James Aimonetti) Date: Fri, 9 Sep 2011 17:10:30 -0700 Subject: [erlang-questions] PropEr way to deal with recursive types Message-ID: <4E6AAAF6.30205@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm wondering how to handle testing with recursive types, especially with regards to handling the depth at which those structures are created in the generators. Take, for instance, a decoded json object ({struct, [{K,V},...]} in this case). V can obviously be a json object itself or even a list of json objects. I'd like to write a function is_json_object/1 that will deeply inspect a passed json object and verify it is, in fact, a json object. Assuming simplified types similar to: - -type json_prop_key() :: binary(). - -type json_prop_value() :: binary() | json_object(). - -type json_object() :: {struct, [{json_prop_key(), json_prop_value()}]}. and a function definition - -spec is_json_object/1 :: (term()) -> boolean(). is_json_object(MaybeJObj) -> % Implementation here ... % return a boolean Now, I'd like to create the PropEr tests for valid json objects. Here's my first crack: prop_is_json_object() -> ?FORALL(JObj, json_object_builder(), ?WHENFAIL(io:format("Failed prop_is_json_object ~p~n", [JObj]), ?MODULE:is_json_object(JObj) )). %% GENERATORS %% create json objects json_object_builder() -> ?LET(Prop, a_proplist(), {struct, Prop}). a_proplist() -> ?SUCHTHAT(L, list( ?LET({K,V}, {binary(), proplist_value()}, {K, V}) ), length(L) < 5). proplist_value() -> ?LAZY(weighted_union([{15, binary()}, {1, json_object_builder()}])). I tried limiting the proplist to 5 tuples, heavily weighting the proplist_value union towards binary(), etc. When I run "proper:module(json_tester).", I see a string of periods (varies how far it gets), then the VM hangs and I have to get into the JCL to interrupt the process. My assumption is that the generator is getting stuck creating a big data structure, as CPU and memory increase rapidly. I've looked at three examples[1][2][3] that Fred Herbert shared but haven't found the mental bridge between his examples and my attempt. I don't doubt that I'm misusing or ignorant of the right functions or macros in PropEr to make this easier. Any hints? :) James [1] https://bitbucket.org/ferd/zippers/src/9548780f0261/test/prop_zipper_lists.erl#cl-92 [2] https://bitbucket.org/ferd/zippers/src/9548780f0261/test/prop_zipper_bintrees.erl#cl-88 [3] https://bitbucket.org/ferd/zippers/src/9548780f0261/test/prop_zipper_forests.erl#cl-198 - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOaqr2AAoJENc77s1OYoGgAYsH/jXoReq7uXkPxVrBvzseQULe lOgFcJoWf1/Ese2pyIOC9VmNt5uj1XxMfhu5RinjSmXarep77nRUYTc6DMWDHYk+ YWSM9seNYMfR7tgcUmztpZkhpNdYvwKhPGHkvuuw3bEMzizT9qh+NmUOFvhUiwpU kngonWOY00tFr+kpJ/JXQtcyCpl/WGTodXGOwflAwosFwTtJPLjOjS3fgqeF/X3s NJv4rM1p55H6LmraX/t7mOvDPJFuMaDL/AfOqn1ndQM8BwXxjx6m8EcpN3B+noe8 LsAKEm3r+qXwGTImT6/H1PC+Y0NdGgSUbWDKQrTVNJlMyn8MstwBOPC7emJVhxI= =4Srb -----END PGP SIGNATURE----- From norton@REDACTED Sat Sep 10 04:17:19 2011 From: norton@REDACTED (Joseph Norton) Date: Sat, 10 Sep 2011 11:17:19 +0900 Subject: [erlang-questions] PropEr way to deal with recursive types In-Reply-To: <4E6AAAF6.30205@2600hz.com> References: <4E6AAAF6.30205@2600hz.com> Message-ID: <52BF961C-FC74-40EC-B995-2E3903F8EC4C@lovely.email.ne.jp> James - See the SIZED macro and resize command. Here is an example that decrements the size for each recursive call: type(Gen,Type) -> ?SIZED(Size,resize(if Size > 0 -> Size-1; true -> 0 end,type1(Gen,Type))). https://github.com/norton/ubf/blob/master/test/qc/qc_ubf_types.erl Depending on your needs, you may want to divide by 2, take the square root, etc. thanks, Joseph Norton p.s. Take note - there is an open issue related to PropEr's resize implementation: https://github.com/manopapad/proper/issues/10 On Sep 10, 2011, at 9:10 AM, James Aimonetti wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I'm wondering how to handle testing with recursive types, especially > with regards to handling the depth at which those structures are created > in the generators. > > Take, for instance, a decoded json object ({struct, [{K,V},...]} in this > case). V can obviously be a json object itself or even a list of json > objects. > > I'd like to write a function is_json_object/1 that will deeply inspect a > passed json object and verify it is, in fact, a json object. > > Assuming simplified types similar to: > > - -type json_prop_key() :: binary(). > - -type json_prop_value() :: binary() | json_object(). > - -type json_object() :: {struct, [{json_prop_key(), json_prop_value()}]}. > > and a function definition > - -spec is_json_object/1 :: (term()) -> boolean(). > is_json_object(MaybeJObj) -> > % Implementation here > ... > % return a boolean > > Now, I'd like to create the PropEr tests for valid json objects. Here's > my first crack: > > prop_is_json_object() -> > ?FORALL(JObj, json_object_builder(), > ?WHENFAIL(io:format("Failed prop_is_json_object ~p~n", [JObj]), > ?MODULE:is_json_object(JObj) > )). > > %% GENERATORS > %% create json objects > json_object_builder() -> > ?LET(Prop, a_proplist(), {struct, Prop}). > > a_proplist() -> > ?SUCHTHAT(L, list( ?LET({K,V}, {binary(), proplist_value()}, {K, V}) > ), length(L) < 5). > > proplist_value() -> > ?LAZY(weighted_union([{15, binary()}, {1, json_object_builder()}])). > > I tried limiting the proplist to 5 tuples, heavily weighting the > proplist_value union towards binary(), etc. When I run > "proper:module(json_tester).", I see a string of periods (varies how far > it gets), then the VM hangs and I have to get into the JCL to interrupt > the process. My assumption is that the generator is getting stuck > creating a big data structure, as CPU and memory increase rapidly. > > I've looked at three examples[1][2][3] that Fred Herbert shared but > haven't found the mental bridge between his examples and my attempt. I > don't doubt that I'm misusing or ignorant of the right functions or > macros in PropEr to make this easier. > > Any hints? :) > > James > > [1] > https://bitbucket.org/ferd/zippers/src/9548780f0261/test/prop_zipper_lists.erl#cl-92 > [2] > https://bitbucket.org/ferd/zippers/src/9548780f0261/test/prop_zipper_bintrees.erl#cl-88 > [3] > https://bitbucket.org/ferd/zippers/src/9548780f0261/test/prop_zipper_forests.erl#cl-198 > > - -- > James Aimonetti > Distributed Systems Engineer / DJ MC_ > > 2600hz | http://2600hz.com > sip:james@REDACTED > tel: 415.886.7905 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iQEcBAEBAgAGBQJOaqr2AAoJENc77s1OYoGgAYsH/jXoReq7uXkPxVrBvzseQULe > lOgFcJoWf1/Ese2pyIOC9VmNt5uj1XxMfhu5RinjSmXarep77nRUYTc6DMWDHYk+ > YWSM9seNYMfR7tgcUmztpZkhpNdYvwKhPGHkvuuw3bEMzizT9qh+NmUOFvhUiwpU > kngonWOY00tFr+kpJ/JXQtcyCpl/WGTodXGOwflAwosFwTtJPLjOjS3fgqeF/X3s > NJv4rM1p55H6LmraX/t7mOvDPJFuMaDL/AfOqn1ndQM8BwXxjx6m8EcpN3B+noe8 > LsAKEm3r+qXwGTImT6/H1PC+Y0NdGgSUbWDKQrTVNJlMyn8MstwBOPC7emJVhxI= > =4Srb > -----END PGP SIGNATURE----- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From smith.winston.101@REDACTED Sat Sep 10 06:02:25 2011 From: smith.winston.101@REDACTED (Winston Smith) Date: Sat, 10 Sep 2011 00:02:25 -0400 Subject: [erlang-questions] How to configure otp for a debug build (with otp_build)? In-Reply-To: <076c88cc-f5df-413c-adef-9e9e3a3df577@knuth> References: <076c88cc-f5df-413c-adef-9e9e3a3df577@knuth> Message-ID: On Fri, Sep 9, 2011 at 4:47 AM, Gustav Simonsson wrote: > There is a line with "CFLAGS=" in the xcomp file where you can add flags to gcc, > such as debug options found in > http://gcc.gnu.org/onlinedocs/gcc-4.6.1/gcc/Debugging-Options.html#Debugging-Options I think there's more to it than that. At the very least, you need to swap the -O2 (or whatever Erlang builds with by default) for -g and also define -DDEBUG, but there's also the fact that you don't want symbols stripped for a debug build. So a debug version is something that really has to be "embraced" by otp_build. After some digging, I found the following notes: http://www.erlang.org/doc/installation_guide/INSTALL.html#How-to-Build-a-Debug-Enabled-Erlang-RunTime-System These basically that say that after configuring to go into $ERL_TOP/erts/emulator and run: make debug FLAVOR=plain But for an xcomp (cross compile) configuration, this simply built a debug version of the host system's Erlang (not the target systems which is what I need). Note that in this case, the host system is my Ubuntu 11.04 x64 system, the target is my AVR32 Linux board. I found that if you invoke make as follows, then it'll build a debug version for the target (not the host!): make debug FLAVOR=plain TARGET=avr32-atmel-linux-gnu Since my segmentation fault seems to be related to inet_gethost, I found that this wasn't building a debug version of inet_gethost, so I also had to go into $ERL_TOP/erts/etc/common and run the same make command, again as follows: make debug FLAVOR=plain TARGET=avr32-atmel-linux-gnu In order to get a debug version of inet_gethost. Note that this make-debug invocation does in fact produce unstripped debug executables which is a necessity for debugging. I don't know how successful this was/will be [in terms of tracking down my segmentation fault], but I'm about to start testing it now ...! Thanks! Winston. From kostis@REDACTED Sat Sep 10 09:20:18 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Sat, 10 Sep 2011 10:20:18 +0300 Subject: [erlang-questions] Type specs expressivity In-Reply-To: References: Message-ID: <4E6B0FB2.4060608@cs.ntua.gr> On 09/09/11 15:00, Olivier BOUDEVILLE wrote: > > I would have three small questions regarding what can be expressed with > type specs: These are all very good questions! Perhaps we should make sure they find their way verbatim in an FAQ in dialyzer's website. > - we can define immediate constants and even ranges for integer values > (ex: '-spec f() -> 42.' or '-spec f() -> 0..42.'), but apparently we > cannot do the same for floats? Is there any particular reason for that > (besides the lack of meaning of exact comparisons for floating-point > numbers), or maybe it is a planned enhancement? Admittedly, the decision has been quite arbitrary. We first started with atoms as singleton types and then extended this capability to integers because they were many natural cases where this turned out handy (e.g. bits_in_word() :: 32 | 64). Then we added integer ranges and integer range analysis to the system. We've stopped there. Call me unimaginative, but the only cases where we could see singleton floats as handy were constants like pi() :: 3.141592653589793, epsilon() and all their other Greek relatives ;) so we never bothered implementing singleton types for floats in the type system. But we can change that, easily I think. So, if you have a convincing example where you need singleton floats, I would love to hear about it, even off list and possibly put singleton floats on the list of planned extensions. Aside: there is no support for singletons in binaries, bitstrings or strings either... > - it is a still more minor point, but we are using a hashtable type in > pure Erlang, which is implemented as a tuple containing buckets. The > number of elements of the tuple is determined at run-time (depending on > a targeted ideal load factor, based on the expected number of elements > to be stored in the hashtable). We can define a type for bucket(), but > for the hashtable is there any way of expressing a more precise > declaration than '-opaque hashtable() :: tuple()', i.e. to tell that > even if we do not know the size of the tuple, we know it will contain > bucket() instances at each position? Currently, there is no tuple(T) type where T is a type. Perhaps there should be one but I have to think more of the consequences here. Incidentally, the opaque declaration all it does is to detect whether code *outside* the defining module (the module containing the declaration) somehow depends/checks the structure of the data structure that has been declared as opaque. > - what is the recommended way of handling functions that may or may not > throw exceptions? If otherwise returning a type T, should we use T | > none() or T | no_return() as return type, or should we consider that > type specs focus on actual return types under normal circumstances, and > that exceptions are in "another dimension"? I suppose it is the latter > option, but then one could imagine that type specs for functions could > be enriched with exception specifications, for example to check whether > a catch clause has any chance of being triggered at all? Exceptions are in "another dimension" as you write. For a long time it has been on my TODO list to add a -throws attribute (perhaps as part of the spec declaration) so that functions can declare their explicitly thrown exceptions. In any case, you should definitely NOT add none() or no_return() to the return type if the function has some other return type under normal circumstances. There is no harm but this none() or no_return() will simply be ignored by the analysis. Note that in Erlang all functions that do not have any() as their input type T have an implicit none() in their return type (for calls with inputs that do not belong to T) so, in some sense, the none() is always there. I think that the proper way to specify exceptions is by explicit declarations in a -throws like attribute. Alas, currently such a declaration exists only in the Edoc language... Kostis From desired.mta@REDACTED Sat Sep 10 13:46:33 2011 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Sat, 10 Sep 2011 14:46:33 +0300 Subject: [erlang-questions] PropEr way to deal with recursive types In-Reply-To: <4E6AAAF6.30205@2600hz.com> References: <4E6AAAF6.30205@2600hz.com> Message-ID: <20110910114633.GA17866@jakstys.lt> On Fri, Sep 09, 2011 at 05:10:30PM -0700, James Aimonetti wrote: > I tried limiting the proplist to 5 tuples, heavily weighting the > proplist_value union towards binary(), etc. When I run > "proper:module(json_tester).", I see a string of periods (varies how far > it gets), then the VM hangs and I have to get into the JCL to interrupt > the process. My assumption is that the generator is getting stuck > creating a big data structure, as CPU and memory increase rapidly. > > I've looked at three examples[1][2][3] that Fred Herbert shared but > haven't found the mental bridge between his examples and my attempt. I > don't doubt that I'm misusing or ignorant of the right functions or > macros in PropEr to make this easier. > > Any hints? :) > You can limit recursion depth as well. Just pass max_size to the proper:quickcheck() function. That helped me as I didn't have to write my own recursive generator. :) As the topic is quite similar, I would like to remind about creating custom types in PropEr. http://erlang.org/pipermail/erlang-questions/2011-August/060619.html Thanks Motiejus From kostis@REDACTED Sat Sep 10 15:06:06 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Sat, 10 Sep 2011 16:06:06 +0300 Subject: [erlang-questions] PropEr way to deal with recursive types In-Reply-To: <20110910114633.GA17866@jakstys.lt> References: <4E6AAAF6.30205@2600hz.com> <20110910114633.GA17866@jakstys.lt> Message-ID: <4E6B60BE.6080109@cs.ntua.gr> On 09/10/11 14:46, Motiejus Jak?tys wrote: > On Fri, Sep 09, 2011 at 05:10:30PM -0700, James Aimonetti wrote: >> I tried limiting the proplist to 5 tuples, heavily weighting the >> proplist_value union towards binary(), etc. When I run >> "proper:module(json_tester).", I see a string of periods (varies how far >> it gets), then the VM hangs and I have to get into the JCL to interrupt >> the process. My assumption is that the generator is getting stuck >> creating a big data structure, as CPU and memory increase rapidly. >> >> I've looked at three examples[1][2][3] that Fred Herbert shared but >> haven't found the mental bridge between his examples and my attempt. I >> don't doubt that I'm misusing or ignorant of the right functions or >> macros in PropEr to make this easier. >> >> Any hints? :) >> > > You can limit recursion depth as well. Just pass max_size to the > proper:quickcheck() function. That helped me as I didn't have to write > my own recursive generator. :) The PropEr way to deal with recursive types is to delay writing your own generators as long as possible. You can start with writing just the recursive type definitions and use them as a "for free" generator. If that proves not to be adequate for your application, you can try to globally limit the recursion depth in the way suggested by Motiejus. Only if even that is not enough (for some reason) should you try to resort to ?SIZED, resize, ?LAZY, and friends. If you encounter any problem at any point in the above we would like to hear about it (preferably with a small test that exhibits the problem). Please use either the proper@REDACTED address or github for that as not all PropEr developers read erlang-questions. > As the topic is quite similar, I would like to remind about creating > custom types in PropEr. > http://erlang.org/pipermail/erlang-questions/2011-August/060619.html We are (slowly) working on this one... Kostis From ahmet@REDACTED Sat Sep 10 17:20:53 2011 From: ahmet@REDACTED (Ahmet Nalcacioglu) Date: Sat, 10 Sep 2011 11:20:53 -0400 Subject: [erlang-questions] Compiling R14B03 with hipe on Ubuntu Message-ID: I have a 64 bit ubuntu 10.10 I tried everything to build erlang with hipe support, to no avail. I tried using kerl KERL_CONFIGURE_OPTIONS="--enable-hipe" kerl build R14B03 r14b03_hipe Then I tried on a source I dowloaded using ./configure --enable-hipe I even removed the following lines in otp_build script which disables hipe for linux " if target_contains linux; then XX=`echo $* | grep -v able-hipe` if [ "$*" = "$XX" ]; then CONFIG_FLAGS="$CONFIG_FLAGS --disable-hipe" fi fi " then do a ./configure --enable-hipe I can't get a build with hipe enabled. I don't understand why for linux hipe is disabled in the first place. The only way I can get a hipe enabled erl is through ubuntu software center, which has a erlang-base-hipe module. However that is an old build R13. Can anyone help me? I tried but I don't know why simply ./configure --enable-hipe does not work. From grimlog@REDACTED Sat Sep 10 17:49:57 2011 From: grimlog@REDACTED (Michael Schreckenbauer) Date: Sat, 10 Sep 2011 17:49:57 +0200 Subject: [erlang-questions] Compiling R14B03 with hipe on Ubuntu In-Reply-To: References: Message-ID: <5357676.PKUot2Agu2@pc> Hi Ahmet, On Saturday, 10. September 2011 11:20:53 Ahmet Nalcacioglu wrote: > I have a 64 bit ubuntu 10.10 > > I tried everything to build erlang with hipe support, to no avail. > > I tried using kerl > KERL_CONFIGURE_OPTIONS="--enable-hipe" kerl build R14B03 r14b03_hipe > > Then I tried on a source I dowloaded using > ./configure --enable-hipe > > I even removed the following lines in otp_build script which disables > hipe for linux > " > > if target_contains linux; then > XX=`echo $* | grep -v able-hipe` > if [ "$*" = "$XX" ]; then > CONFIG_FLAGS="$CONFIG_FLAGS --disable-hipe" > fi > fi > " you are misreading that script. It adds --disable-hipe, if there's no explicit --enable-hipe or --disable-hipe given to ./configure XX is *empty*, if there's something containing "able-hipe" on the commandline, so it *differs* from "$*", so nothing is added to CONFIG_FLAGS. If there is nothing on the commandline matching "able-hipe", then XX is the given commandline, so it matches "$*" and "--disable-hipe" get's added. > then do a ./configure --enable-hipe > > I can't get a build with hipe enabled. I don't understand why for > linux hipe is disabled in the first place. > > The only way I can get a hipe enabled erl is through ubuntu software > center, which has a erlang-base-hipe module. However that is an old > build R13. > > Can anyone help me? I tried but I don't know why simply ./configure > --enable-hipe does not work. What's the exact error you are getting? It works for me (I use gentoo, FWIW) Regards, Michael From james@REDACTED Sat Sep 10 22:25:23 2011 From: james@REDACTED (James Aimonetti) Date: Sat, 10 Sep 2011 13:25:23 -0700 Subject: [erlang-questions] PropEr way to deal with recursive types In-Reply-To: <4E6B60BE.6080109@cs.ntua.gr> References: <4E6AAAF6.30205@2600hz.com> <20110910114633.GA17866@jakstys.lt> <4E6B60BE.6080109@cs.ntua.gr> Message-ID: <4E6BC7B3.8040001@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 09/10/2011 06:06 AM, Kostis Sagonas wrote: > On 09/10/11 14:46, Motiejus Jak?tys wrote: >> On Fri, Sep 09, 2011 at 05:10:30PM -0700, James Aimonetti wrote: >>> I tried limiting the proplist to 5 tuples, heavily weighting the >>> proplist_value union towards binary(), etc. When I run >>> "proper:module(json_tester).", I see a string of periods (varies how far >>> it gets), then the VM hangs and I have to get into the JCL to interrupt >>> the process. My assumption is that the generator is getting stuck >>> creating a big data structure, as CPU and memory increase rapidly. >>> >>> I've looked at three examples[1][2][3] that Fred Herbert shared but >>> haven't found the mental bridge between his examples and my attempt. I >>> don't doubt that I'm misusing or ignorant of the right functions or >>> macros in PropEr to make this easier. >>> >>> Any hints? :) >>> >> >> You can limit recursion depth as well. Just pass max_size to the >> proper:quickcheck() function. That helped me as I didn't have to write >> my own recursive generator. :) > > The PropEr way to deal with recursive types is to delay writing your own > generators as long as possible. You can start with writing just the > recursive type definitions and use them as a "for free" generator. If > that proves not to be adequate for your application, you can try to > globally limit the recursion depth in the way suggested by Motiejus. > Only if even that is not enough (for some reason) should you try to > resort to ?SIZED, resize, ?LAZY, and friends. > > If you encounter any problem at any point in the above we would like to > hear about it (preferably with a small test that exhibits the problem). > Please use either the proper@REDACTED address or github for that as not > all PropEr developers read erlang-questions. > >> As the topic is quite similar, I would like to remind about creating >> custom types in PropEr. >> http://erlang.org/pipermail/erlang-questions/2011-August/060619.html > > We are (slowly) working on this one... > > Kostis > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > Thanks for the replies everyone. Kostis, I will start over, using your steps above, to generate the test cases and send the PropEr team code samples if I can't seem to get the recursive types working. James - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOa8ezAAoJENc77s1OYoGgjtMH/iQmsOe/MXsJdoH5QcXGmBRu WKhxUp/ue4LBdqfJd//zHjwoiluZQR9CuZUXZ+hRE3JGi+kHTl7pR3SUS9vkD2mJ 21WAPOSlVPHY3LZXcFmsKODJM54/faqshoM4uk9Q5EN8UcwrXWp74+sHDIzt3Mai vP3N5mIePR1qXXW3MW12gi/32NwPLpTcr3beFBYGW9emnv2JXHb+8FeqaSqy1+KQ 9FOy8kpBxUFKS4SzaL6IeECPwDwsVd+fphDcP+AOYLUot/bHe48X+XDVxvxFgvxf qQ1GiNu00Oesex5cpWkL+KukeoSqAISo+aF2ya/Sd30uFj76gqKQ/PqwxcDu4/U= =pi/E -----END PGP SIGNATURE----- From wilsoncj1@REDACTED Sat Sep 10 22:57:59 2011 From: wilsoncj1@REDACTED (Christopher Wilson) Date: Sat, 10 Sep 2011 16:57:59 -0400 Subject: [erlang-questions] Pass Name to Dynamic Supervisors In-Reply-To: References: Message-ID: Thanks, that's pretty close. How, or more importantly, when do I register the name. Do I put this in the child worker/sup as part of the MFA used to spawn it or can this be called by the parent once start_child has been called? Best practice is all I'm looking for. Thanks again for pointing this one out. -Chris On Fri, Sep 9, 2011 at 1:03 PM, Garrett Smith wrote: > On Wed, Sep 7, 2011 at 9:09 PM, Christopher Wilson > wrote: > > Is there a way to pass a name/ID into the call to supervisor:start_child, > > more specifically to init()? I'd like to specify the name of the worker > or > > supervisor such that when I ran supervisor:which_children I could > identify > > what's what. > > Also, I've tried [unsuccessfully] to dynamically create a child spec and > add > > pass that in. Are there any decent examples out there on how to do this > > outside of the man pages? > > Take a look at: > > https://github.com/esl/gproc > > I'm not sure if that's the canonical repository -- maybe Ulf can weigh in. > > It's tempting, but I find it's best not to mess with the vanilla > supervisory facility and instead use gproc for getting a list of named > processes. You can also use gproc to publish interesting process stats > to consumers, thus avoiding costly and intrusive queries to the > processes themselves. > > This paper: > > https://github.com/esl/gproc/blob/master/doc/erlang07-wiger.pdf?raw=true > > provides the background for gproc and is otherwise an excellent read! > > You'll have a slight learning curve with gproc, but it's an > *indispensable* tool so well worth it. > > It needs to be in the core :) > > Garrett > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Sun Sep 11 11:28:22 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Sun, 11 Sep 2011 12:28:22 +0300 Subject: [erlang-questions] Compiling R14B03 with hipe on Ubuntu In-Reply-To: References: Message-ID: <4E6C7F36.2050202@cs.ntua.gr> On 09/10/11 18:20, Ahmet Nalcacioglu wrote: > I have a 64 bit ubuntu 10.10 > > I tried everything to build erlang with hipe support, to no avail. > > I tried using kerl > KERL_CONFIGURE_OPTIONS="--enable-hipe" kerl build R14B03 r14b03_hipe > > Then I tried on a source I dowloaded using > ./configure --enable-hipe > > I even removed the following lines in otp_build script which disables > hipe for linux > " > > if target_contains linux; then > XX=`echo $* | grep -v able-hipe` > if [ "$*" = "$XX" ]; then > CONFIG_FLAGS="$CONFIG_FLAGS --disable-hipe" > fi > fi > " > then do a ./configure --enable-hipe > > I can't get a build with hipe enabled. I don't understand why for > linux hipe is disabled in the first place. > > The only way I can get a hipe enabled erl is through ubuntu software > center, which has a erlang-base-hipe module. However that is an old > build R13. > > Can anyone help me? I tried but I don't know why simply ./configure > --enable-hipe does not work. I really do not understand what (you think) the problem is and more importantly I do not understand what it is you are doing. Are you compiling from source and if so where did you get this source from. (My advice: get it from Erlang's github repo.) On most (all?) Linuxes, hipe is enabled by default, so you should really not need --enable-hipe. What is the slogan when you start the system? If it's something like: Erlang R14B0? [source] [64-bit] ...[hipe] ... then HiPE is enabled. Kostis From magnus.klaar@REDACTED Sun Sep 11 16:49:50 2011 From: magnus.klaar@REDACTED (Magnus Klaar) Date: Sun, 11 Sep 2011 16:49:50 +0200 Subject: [erlang-questions] Pass Name to Dynamic Supervisors In-Reply-To: References: Message-ID: Hi! If you are using gproc you are expected to register the name of a process from the same process. I've never found a reason to do this the other way around. A registered name is a property of the child process/module and therefore it should be responsible for this. We are using gproc in etorrent, if you grep the source code for register it should give you some usage examples. MVH Magnus Klaar On Sat, Sep 10, 2011 at 10:57 PM, Christopher Wilson wrote: > Thanks, that's pretty close. How, or more importantly, when do I register > the name. Do I put this in the child worker/sup as part of the MFA used to > spawn it or can this be called by the parent once start_child has been > called? Best practice is all I'm looking for. Thanks again for pointing > this one out. > > -Chris > > > On Fri, Sep 9, 2011 at 1:03 PM, Garrett Smith wrote: > >> On Wed, Sep 7, 2011 at 9:09 PM, Christopher Wilson >> wrote: >> > Is there a way to pass a name/ID into the call to >> supervisor:start_child, >> > more specifically to init()? I'd like to specify the name of the worker >> or >> > supervisor such that when I ran supervisor:which_children I could >> identify >> > what's what. >> > Also, I've tried [unsuccessfully] to dynamically create a child spec and >> add >> > pass that in. Are there any decent examples out there on how to do this >> > outside of the man pages? >> >> Take a look at: >> >> https://github.com/esl/gproc >> >> I'm not sure if that's the canonical repository -- maybe Ulf can weigh in. >> >> It's tempting, but I find it's best not to mess with the vanilla >> supervisory facility and instead use gproc for getting a list of named >> processes. You can also use gproc to publish interesting process stats >> to consumers, thus avoiding costly and intrusive queries to the >> processes themselves. >> >> This paper: >> >> https://github.com/esl/gproc/blob/master/doc/erlang07-wiger.pdf?raw=true >> >> provides the background for gproc and is otherwise an excellent read! >> >> You'll have a slight learning curve with gproc, but it's an >> *indispensable* tool so well worth it. >> >> It needs to be in the core :) >> >> Garrett >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Mon Sep 12 02:21:17 2011 From: g@REDACTED (Garrett Smith) Date: Sun, 11 Sep 2011 19:21:17 -0500 Subject: [erlang-questions] Pass Name to Dynamic Supervisors In-Reply-To: References: Message-ID: As Magnus pointed out, registration is the responsibility of the process, so a good place is in the init/1 function. Lookup/queries however are from outside the process. The way I look at this pattern is this: - Keep processes as black box as possible - To get information from a process, the process should publish it where it can be read without involving the process This pattern is common in OS apps (e.g. MySQL) where counters/stats are maintained in a way that they can be read/queried efficiently by external consumers, leaving the app to focus solely on its area of concern. On Sat, Sep 10, 2011 at 3:57 PM, Christopher Wilson wrote: > Thanks, that's pretty close. ?How, or more importantly, when do I register > the name. ?Do I put this in the child worker/sup as part of the MFA used to > spawn it or can this be called by the parent once start_child has been > called? ?Best practice is all I'm looking for. ?Thanks again for pointing > this one out. > -Chris > > On Fri, Sep 9, 2011 at 1:03 PM, Garrett Smith wrote: >> >> On Wed, Sep 7, 2011 at 9:09 PM, Christopher Wilson >> wrote: >> > Is there a way to pass a name/ID into the call to >> > supervisor:start_child, >> > more specifically to init()? ?I'd like to specify the name of the worker >> > or >> > supervisor such that when I ran supervisor:which_children I could >> > identify >> > what's what. >> > Also, I've tried [unsuccessfully] to dynamically create a child spec and >> > add >> > pass that in. ?Are there any decent examples out there on how to do this >> > outside of the man pages? >> >> Take a look at: >> >> https://github.com/esl/gproc >> >> I'm not sure if that's the canonical repository -- maybe Ulf can weigh in. >> >> It's tempting, but I find it's best not to mess with the vanilla >> supervisory facility and instead use gproc for getting a list of named >> processes. You can also use gproc to publish interesting process stats >> to consumers, thus avoiding costly and intrusive queries to the >> processes themselves. >> >> This paper: >> >> https://github.com/esl/gproc/blob/master/doc/erlang07-wiger.pdf?raw=true >> >> provides the background for gproc and is otherwise an excellent read! >> >> You'll have a slight learning curve with gproc, but it's an >> *indispensable* tool so well worth it. >> >> It needs to be in the core :) >> >> Garrett > > From ulf.wiger@REDACTED Mon Sep 12 09:25:55 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 12 Sep 2011 09:25:55 +0200 Subject: [erlang-questions] Pass Name to Dynamic Supervisors In-Reply-To: References: Message-ID: <4E6DB403.1010705@erlang-solutions.com> On 11/09/2011 16:49, Magnus Klaar wrote: > > We are using gproc in etorrent, if you grep the source code for register > it should give you some usage examples. I have also started to document Gproc on the github wiki: https://github.com/esl/gproc/wiki It's obviously only in the beginning stages. Any contributions and instructive examples are most welcome. BR, Ulf W -- Ulf Wiger CTO, Erlang Solutions Ltd, formerly Erlang Training & Consulting Ltd http://www.erlang-solutions.com From essen@REDACTED Mon Sep 12 13:20:18 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Mon, 12 Sep 2011 13:20:18 +0200 Subject: [erlang-questions] Pass Name to Dynamic Supervisors In-Reply-To: <4E6DB403.1010705@erlang-solutions.com> References: <4E6DB403.1010705@erlang-solutions.com> Message-ID: <4E6DEAF2.8030606@dev-extend.eu> On 09/12/2011 09:25 AM, Ulf Wiger wrote: > On 11/09/2011 16:49, Magnus Klaar wrote: >> >> We are using gproc in etorrent, if you grep the source code for register >> it should give you some usage examples. > > I have also started to document Gproc on the github wiki: > > https://github.com/esl/gproc/wiki > > It's obviously only in the beginning stages. Any contributions and > instructive examples are most welcome. While we are on the subject, if anyone has any advice on how best to optimize gproc options for performance (beyond what's explained in the gproc.erl file) I'd be grateful. -- Lo?c Hoguin Dev:Extend From vladdu55@REDACTED Mon Sep 12 15:07:07 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 12 Sep 2011 15:07:07 +0200 Subject: [erlang-questions] testing the erlang parser Message-ID: Hi! I see that there is no test suite for the parser, only for the scanner and the preprocessor, and I would need to have at least a file (or a few) that makes use of all the parser rules, especially the type language. I started to write something like that myself, but it would save a lot of work if that was already available. Thanks in advance! best regards, Vlad From kostis@REDACTED Mon Sep 12 16:01:49 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 12 Sep 2011 17:01:49 +0300 Subject: [erlang-questions] testing the erlang parser In-Reply-To: References: Message-ID: <4E6E10CD.7050902@cs.ntua.gr> On 09/12/11 16:07, Vlad Dumitrescu wrote: > Hi! > > I see that there is no test suite for the parser, only for the scanner > and the preprocessor, and I would need to have at least a file (or a > few) that makes use of all the parser rules, especially the type > language. > > I started to write something like that myself, but it would save a lot > of work if that was already available. Thanks in advance! Probably not exactly what you are looking for, but I've seen a test suite for (pretty) printing types and specs from their internal representation in the OTP tests. As a start at least you could probably turn that relatively easily to a parser test suite. Kostis From norton@REDACTED Mon Sep 12 17:46:24 2011 From: norton@REDACTED (Joseph Wayne Norton) Date: Tue, 13 Sep 2011 00:46:24 +0900 Subject: [erlang-questions] ets:next/2 behavior - set vs. ordered_set Message-ID: <00E5DFDF-EF45-4A8A-8871-227911A55C78@lovely.email.ne.jp> I'm currently developing a disk based clone of the ets application. During testing with QuickCheck and Proper, I stumbled upon a difference between set and ordered_set for the real ets:next/2 operation. The last command exits with badarg rather than '$end_of_table' as I would expect. Can someone comment on this behavior? Is this a bug? Is this a documentation bug? thanks, 18> ets:new(ordered_set,[ordered_set,public,named_table,{keypos,2}]). ordered_set 19> ets:first(ordered_set). '$end_of_table' 20> ets:next(ordered_set,0). '$end_of_table' 21> 21> ets:new(set,[set,public,named_table,{keypos,2}]). set 22> ets:first(set). '$end_of_table' 23> ets:next(set,0). ** exception error: bad argument in function ets:next/2 called as ets:next(set,0) 24> $ erl -v Erlang R14B03 (erts-5.8.4) [source] [64-bit] [rq:1] [async-threads:0] [kernel-poll:false] Eshell V5.8.4 (abort with ^G) next(Tab, Key1) -> Key2 | '$end_of_table' Types Tab = tid() | atom() Key1 = Key2 = term() Returns the next key Key2 , following the key Key1 in the table Tab . If the table is of the ordered_set type, the next key in Erlang term order is returned. If the table is of any other type, the next key according to the table's internal order is returned. If there is no next key, '$end_of_table' is returned. Joseph Wayne Norton norton@REDACTED From freza@REDACTED Mon Sep 12 23:17:40 2011 From: freza@REDACTED (Jachym Holecek) Date: Mon, 12 Sep 2011 17:17:40 -0400 Subject: [erlang-questions] ets:next/2 behavior - set vs. ordered_set In-Reply-To: <00E5DFDF-EF45-4A8A-8871-227911A55C78@lovely.email.ne.jp> References: <00E5DFDF-EF45-4A8A-8871-227911A55C78@lovely.email.ne.jp> Message-ID: <20110912211740.GA23065@circlewave.net> # Joseph Wayne Norton 2011-09-12: > I'm currently developing a disk based clone of the ets application. Interesting, I was considering doing something like that for an upcoming project -- are you planning to make the result open source by any chance? ;-) > During testing with QuickCheck and Proper, I stumbled upon a difference between > set and ordered_set for the real ets:next/2 operation. > > The last command exits with badarg rather than '$end_of_table' as I would expect. > > Can someone comment on this behavior? Is this a bug? Is this a documentation bug? FWIW I've always read this: next(Tab, Key1) -> Key2 | '$end_of_table' Returns the next key Key2, following the key Key1 in the table Tab. as "behaviour is undefined if Key1 isn't a key in Tab". That ordered_set tables support even non-key items has been mentioned on the mailing list in passing some while ago, I've found it very surprising... BR, -- Jachym From norton@REDACTED Tue Sep 13 01:30:24 2011 From: norton@REDACTED (Joseph Norton) Date: Tue, 13 Sep 2011 08:30:24 +0900 Subject: [erlang-questions] ets:next/2 behavior - set vs. ordered_set In-Reply-To: <20110912211740.GA23065@circlewave.net> References: <00E5DFDF-EF45-4A8A-8871-227911A55C78@lovely.email.ne.jp> <20110912211740.GA23065@circlewave.net> Message-ID: Yes, that is my plan after the basic features and tests are in place. On 2011/09/13, at 6:17, Jachym Holecek wrote: > # Joseph Wayne Norton 2011-09-12: >> I'm currently developing a disk based clone of the ets application. > > Interesting, I was considering doing something like that for an upcoming > project -- are you planning to make the result open source by any chance? > > ;-) > >> During testing with QuickCheck and Proper, I stumbled upon a difference between >> set and ordered_set for the real ets:next/2 operation. >> >> The last command exits with badarg rather than '$end_of_table' as I would expect. >> >> Can someone comment on this behavior? Is this a bug? Is this a documentation bug? > > FWIW I've always read this: > > next(Tab, Key1) -> Key2 | '$end_of_table' > > Returns the next key Key2, following the key Key1 in the table Tab. > > as "behaviour is undefined if Key1 isn't a key in Tab". That ordered_set > tables support even non-key items has been mentioned on the mailing list > in passing some while ago, I've found it very surprising... > > BR, > -- Jachym From overminddl1@REDACTED Tue Sep 13 02:55:07 2011 From: overminddl1@REDACTED (OvermindDL1) Date: Mon, 12 Sep 2011 18:55:07 -0600 Subject: [erlang-questions] ets:next/2 behavior - set vs. ordered_set In-Reply-To: <00E5DFDF-EF45-4A8A-8871-227911A55C78@lovely.email.ne.jp> References: <00E5DFDF-EF45-4A8A-8871-227911A55C78@lovely.email.ne.jp> Message-ID: On Mon, Sep 12, 2011 at 9:46 AM, Joseph Wayne Norton wrote: > > I'm currently developing a disk based clone of the ets application. Can you elaborate how this is different from dets, or are you doing it for learning? From oribrost@REDACTED Tue Sep 13 03:51:48 2011 From: oribrost@REDACTED (ori brost) Date: Tue, 13 Sep 2011 03:51:48 +0200 Subject: [erlang-questions] Difference of gigs between beam.smp memory and erlang:memory(total) Message-ID: I've been doing a simple experiment in which I attempt to connect about 500000 websocket clients to RJ's websocket patched mochiweb fork. I've noticed that there is a difference of several gigabytes between memory consumed (I'm talking about working set here, not virtual) by beam.smp and the value returned from erlang:memory(total). Could this be related to the problem in http://erlang.org/pipermail/erlang-questions/2011-June/059611.html? Regards, Ori Bar. From norton@REDACTED Tue Sep 13 04:07:54 2011 From: norton@REDACTED (Joseph Norton) Date: Tue, 13 Sep 2011 11:07:54 +0900 Subject: [erlang-questions] ets:next/2 behavior - set vs. ordered_set In-Reply-To: References: <00E5DFDF-EF45-4A8A-8871-227911A55C78@lovely.email.ne.jp> Message-ID: <4D010521-D005-44C4-82C9-75438256A773@lovely.email.ne.jp> The basic goal is to build and test a production worthy replacement for ets with disk based storage. After I have the basics in place, I can elaborate further. thanks, Joseph Norton On Sep 13, 2011, at 9:55 AM, OvermindDL1 wrote: > On Mon, Sep 12, 2011 at 9:46 AM, Joseph Wayne Norton > wrote: >> >> I'm currently developing a disk based clone of the ets application. > > Can you elaborate how this is different from dets, or are you doing it > for learning? From sam@REDACTED Tue Sep 13 07:17:13 2011 From: sam@REDACTED (Sam Bobroff) Date: Tue, 13 Sep 2011 15:17:13 +1000 Subject: [erlang-questions] Erlang release names and versions Message-ID: Hi Erlangers, I'm trying to package up some Erlang projects for distribution, possibly by creating Releases, and I'm getting quite confused. I'd like to do it the "correct" way because we may want to use release_handler to perform upgrades. Maybe someone can help me out :-) The documentation for Releases (http://www.erlang.org/doc/design_principles/release_structure.html) shows a sample directory structure produced by make_tar, which includes this part: releases/A/start.boot releases/ch_rel-1.rel Where ch_rel-1 is the name of the release and A is the version of the release. Question 1: If the release file is placed in releases/ch_rel-1.rel, then isn't installing a second version of the release (say, "B") going to overwrite the file? Isn't this a problem? Is the addition of "-1" to the release name supposed to address this? Shouldn't the release name appear somewhere in the path? Question 2: If another release, say "foo", is installed with the same version (say "A"), isn't it going to overwrite the start.boot file from release "ch_rel-1"? Question 3: Should I even be trying to install more than one release at once? Would I need a completely new Erlang install directory (or at least, releases directory, by using RELDIR)? (I have, incidentally, seen somewhere that the placement of the .rel file directly under releases/ by make_tar is a bug. Is this so? It certainly doesn't seem to match the structure created by create_target.) Thanks in advance for any help, Sam. From mjtruog@REDACTED Tue Sep 13 09:09:15 2011 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 13 Sep 2011 00:09:15 -0700 Subject: [erlang-questions] Erlang release names and versions In-Reply-To: References: Message-ID: <4E6F019B.40405@gmail.com> If you are doing this from scratch it should be much easier to use rebar for building the release: https://github.com/basho/rebar Otherwise, most older projects use to use shell scripts combined with erlang source code to build releases, and much of that was custom ad-hoc source code. So, that is good to avoid. On 09/12/2011 10:17 PM, Sam Bobroff wrote: > Hi Erlangers, > I'm trying to package up some Erlang projects for distribution, > possibly by creating Releases, and I'm getting quite confused. I'd > like to do it the "correct" way because we may want to use > release_handler to perform upgrades. > > Maybe someone can help me out :-) > > The documentation for Releases > (http://www.erlang.org/doc/design_principles/release_structure.html) > shows a sample directory structure produced by make_tar, which > includes this part: > > releases/A/start.boot > releases/ch_rel-1.rel > > Where ch_rel-1 is the name of the release and A is the version of the release. > > Question 1: If the release file is placed in releases/ch_rel-1.rel, > then isn't installing a second version of the release (say, "B") going > to overwrite the file? Isn't this a problem? Is the addition of "-1" > to the release name supposed to address this? Shouldn't the release > name appear somewhere in the path? > > Question 2: If another release, say "foo", is installed with the same > version (say "A"), isn't it going to overwrite the start.boot file > from release "ch_rel-1"? > > Question 3: Should I even be trying to install more than one release > at once? Would I need a completely new Erlang install directory (or at > least, releases directory, by using RELDIR)? > > (I have, incidentally, seen somewhere that the placement of the .rel > file directly under releases/ by make_tar is a bug. Is this so? It > certainly doesn't seem to match the structure created by > create_target.) > > Thanks in advance for any help, > Sam. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From erlang@REDACTED Tue Sep 13 09:19:13 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 13 Sep 2011 09:19:13 +0200 Subject: [erlang-questions] where did my code come from? Message-ID: I had an idea on my way to work ... When you write code, you have *implicit* knowledge of where the external code comes from. When I write the following: -module(foo). ... start() -> X = lists:reverse(...), Y = elib1_misc:zap(...) Z = misultin:request(...) ... I "know" that lists is part of my local OTP install, elib1_misc is my own library installed in ~/code/elib2_1/ebin and misultin is an imported project stored in ~/imports/misultin I also know that my paths etc are setup so this code will work when I run the program. The problem is the *nobody else* knows this. I could tell the system like this: -module(foo). -location(lists, "https://github.com/erlang/otp/blob/dev/lib/stdlib/src/lists.erl"). -location(elib1_misc, "https://github.com/joearms/elib1/blob/master/lib/src/elib1_misc.erl"). -location(misultin, "https://github.com/ostinelli/misultin/blob/master/src/misultin.erl"). ... The location annotation give a *definitive source" for the module I am using in the module What could you do with this information? Answer - a lot - for starters - automatically check for "latest versions" of libraries download them when they change - provide "who uses my code" feedback to the authors of the code - publish (globally) lists of "definitive" versions of code - recursive track and code dependencies (What do I mean) when my system discovers that I use misultin - it downloads misultin.erl misultin.erl will have location dependencies which I can follow, thus the libraries that misultin calls can be fetched. - automate code loading Most often this kind of "additional" information is kept "outside the program" by strictly annotating the program with the location dependencies we bring this information *into* the program in a form where it cannot be detached from the source code. Comments? /Joe From vladdu55@REDACTED Tue Sep 13 09:29:01 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 13 Sep 2011 09:29:01 +0200 Subject: [erlang-questions] where did my code come from? In-Reply-To: References: Message-ID: Hi! On Tue, Sep 13, 2011 at 09:19, Joe Armstrong wrote: > When you write code, you have *implicit* knowledge of where the > external code comes from. > > When I write the following: > > ? ? -module(foo). > ? ? ?... > ? ? start() -> > ? ? ? ? ?X = lists:reverse(...), > ? ? ? ? ?Y = elib1_misc:zap(...) > ? ? ? ? ?Z = misultin:request(...) > ? ? ... > > I "know" that lists is part of my local OTP install, elib1_misc is my > own library installed > in ~/code/elib2_1/ebin and misultin is an imported project stored in > ~/imports/misultin > I also know that my paths etc are setup so this code will work when I > run the program. > > The problem is the *nobody else* knows this. > > I could tell the system like this: > > ? ?-module(foo). > > ? ?-location(lists, > "https://github.com/erlang/otp/blob/dev/lib/stdlib/src/lists.erl"). > ? ?-location(elib1_misc, > "https://github.com/joearms/elib1/blob/master/lib/src/elib1_misc.erl"). > ? ?-location(misultin, > "https://github.com/ostinelli/misultin/blob/master/src/misultin.erl"). While the idea of having explicit references to external libraries is good, I'm not sure about the way you suggest to do it... First, should these annotations be found in all modules? Wouldn't that mean a lot of duplication? I would rather have this information in the .app file instead. Second, what about code that isn't available from a public repository? Then the reference would be meaningless outside the original environment. best regards, Vlad From demeshchuk@REDACTED Tue Sep 13 09:53:10 2011 From: demeshchuk@REDACTED (Dmitry Demeshchuk) Date: Tue, 13 Sep 2011 11:53:10 +0400 Subject: [erlang-questions] where did my code come from? In-Reply-To: References: Message-ID: This seems to produce a handful of duplicate and almost duplicate code. Say, you use 11 modules from a certain external app, and you need to annotate them all. For example, such tools as rebar just refer to the application itself, allowing to download it, update it from external sources, build and easily include to your code path. What it doesn't provide is, as you described it, implicit knowledge of where the code comes from for the programmer. Partially, this can be solved by some rebar-dependent extensions for vim/emacs/whatever. But, of course, it won't be a universal solution in any way (we'll have 2 dependency points here: rebar and editor). Adding locations as you are suggesting (whether it's done manually, or semi-automatically) is surely more environment-independent, but looks quite clumsy. So, as for me, I'd rather write a scope plugin for vim that takes rebar.config into account (probably, the only major Erlang open-source project that still *doesn't* use rebar is rabbitmq) and allows to display the given module's location. The rest of the stuff is easily done by rebar itself. On Tue, Sep 13, 2011 at 11:19 AM, Joe Armstrong wrote: > I had an idea on my way to work ... > > When you write code, you have *implicit* knowledge of where the > external code comes from. > > When I write the following: > > ? ? -module(foo). > ? ? ?... > ? ? start() -> > ? ? ? ? ?X = lists:reverse(...), > ? ? ? ? ?Y = elib1_misc:zap(...) > ? ? ? ? ?Z = misultin:request(...) > ? ? ... > > I "know" that lists is part of my local OTP install, elib1_misc is my > own library installed > in ~/code/elib2_1/ebin and misultin is an imported project stored in > ~/imports/misultin > I also know that my paths etc are setup so this code will work when I > run the program. > > The problem is the *nobody else* knows this. > > I could tell the system like this: > > ? ?-module(foo). > > ? ?-location(lists, > "https://github.com/erlang/otp/blob/dev/lib/stdlib/src/lists.erl"). > ? ?-location(elib1_misc, > "https://github.com/joearms/elib1/blob/master/lib/src/elib1_misc.erl"). > ? ?-location(misultin, > "https://github.com/ostinelli/misultin/blob/master/src/misultin.erl"). > > ? ?... > > > ? The location annotation give a *definitive source" for the module I > am using in the module > > ? ?What could you do with this information? > > ? ?Answer - a lot - for starters > > ? ? ? ? - automatically check for "latest versions" of libraries > download them when they change > ? ? ? ? - provide "who uses my code" feedback to the authors of the code > ? ? ? ? - publish (globally) lists of "definitive" versions of code > ? ? ? ? - recursive track and code dependencies (What do I mean) > ? ? ? ? ? ?when my system discovers that I use misultin - it > downloads misultin.erl > ? ? ? ? ? ?misultin.erl will have location dependencies which I can > follow, thus the libraries > ? ? ? ? ? ?that misultin calls can be fetched. > ? ? ? ? - automate code loading > > ? ?Most often this kind of "additional" information is kept "outside > the program" by strictly > annotating the program with the location dependencies we bring this > information *into* the program > in a form where it cannot be detached from the source code. > > ? ?Comments? > > ? /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Best regards, Dmitry Demeshchuk From mjtruog@REDACTED Tue Sep 13 09:57:06 2011 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 13 Sep 2011 00:57:06 -0700 Subject: [erlang-questions] where did my code come from? In-Reply-To: References: Message-ID: <4E6F0CD2.8050005@gmail.com> This sounds like a very interesting approach to aid collaboration across different software projects and avoiding fragmentation of effort (i.e., avoiding siloed development efforts, without any sharing). I think this is important because much software seems to be perpetually rewritten, just due to difficulties locating the proper dependencies. With explicit dependencies at a per-module level, it would be easier to manage and track source code dependencies. My hope is that, you can explicitly reference a particular version of each dependency so your source code remains stable while the other dependencies change independently (so I would want a version referenced somehow within the location specification). Within Erlang modules, this could be handled with the vsn, but this type of version might need to be separate, just since that may complicate code upgrades. I am tempted to think this would benefit from integration with version control, but that would not provide a flexible system unless there was a large integration effort to handle all major version control systems. Either way, I think this is a good way of managing external source code dependencies which should encourage the reuse of Erlang source code and accelerate the iterations to make source code stable. This type of development would benefit both proprietary development and open source development and may become a more natural way to develop source code over a Wide Area Network (WAN, so, both any intranet and the Internet). On 09/13/2011 12:19 AM, Joe Armstrong wrote: > I had an idea on my way to work ... > > When you write code, you have *implicit* knowledge of where the > external code comes from. > > When I write the following: > > -module(foo). > ... > start() -> > X = lists:reverse(...), > Y = elib1_misc:zap(...) > Z = misultin:request(...) > ... > > I "know" that lists is part of my local OTP install, elib1_misc is my > own library installed > in ~/code/elib2_1/ebin and misultin is an imported project stored in > ~/imports/misultin > I also know that my paths etc are setup so this code will work when I > run the program. > > The problem is the *nobody else* knows this. > > I could tell the system like this: > > -module(foo). > > -location(lists, > "https://github.com/erlang/otp/blob/dev/lib/stdlib/src/lists.erl"). > -location(elib1_misc, > "https://github.com/joearms/elib1/blob/master/lib/src/elib1_misc.erl"). > -location(misultin, > "https://github.com/ostinelli/misultin/blob/master/src/misultin.erl"). > > ... > > > The location annotation give a *definitive source" for the module I > am using in the module > > What could you do with this information? > > Answer - a lot - for starters > > - automatically check for "latest versions" of libraries > download them when they change > - provide "who uses my code" feedback to the authors of the code > - publish (globally) lists of "definitive" versions of code > - recursive track and code dependencies (What do I mean) > when my system discovers that I use misultin - it > downloads misultin.erl > misultin.erl will have location dependencies which I can > follow, thus the libraries > that misultin calls can be fetched. > - automate code loading > > Most often this kind of "additional" information is kept "outside > the program" by strictly > annotating the program with the location dependencies we bring this > information *into* the program > in a form where it cannot be detached from the source code. > > Comments? > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From erlang@REDACTED Tue Sep 13 10:01:05 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 13 Sep 2011 10:01:05 +0200 Subject: [erlang-questions] where did my code come from? In-Reply-To: References: Message-ID: On Tue, Sep 13, 2011 at 9:53 AM, Dmitry Demeshchuk wrote: > This seems to produce a handful of duplicate and almost duplicate > code. Say, you use 11 modules from a certain external app, and you > need to annotate them all. -include("dependencies"). :-) > > For example, such tools as rebar just refer to the application itself, > allowing to download it, update it from external sources, build and > easily include to your code path. What it doesn't provide is, as you > described it, implicit knowledge of where the code comes from for the > programmer. Partially, this can be solved by some rebar-dependent > extensions for vim/emacs/whatever. But, of course, it won't be a > universal solution in any way (we'll have 2 dependency points here: > rebar and editor). > > Adding locations as you are suggesting (whether it's done manually, or > semi-automatically) is surely more environment-independent, but looks > quite clumsy. But it's *precise* - do you want precision or convenience? It's actually the shortest possible way to *exactly* specify what the symbol (for example lists) means If I say -location(lists, "https://github.com/erlang/otp/blob/dev/lib/stdlib/src/lists.erl"). I mean the module lists is to be found at exactly this address If several modules had the same base address I could use a macro - but this does alter the semantics > > So, as for me, I'd rather write a scope plugin for vim that takes > rebar.config into account (probably, the only major Erlang open-source > project that still *doesn't* use rebar is rabbitmq) and allows to > display the given module's location. The rest of the stuff is easily > done by rebar itself. > > On Tue, Sep 13, 2011 at 11:19 AM, Joe Armstrong wrote: >> I had an idea on my way to work ... >> >> When you write code, you have *implicit* knowledge of where the >> external code comes from. >> >> When I write the following: >> >> ? ? -module(foo). >> ? ? ?... >> ? ? start() -> >> ? ? ? ? ?X = lists:reverse(...), >> ? ? ? ? ?Y = elib1_misc:zap(...) >> ? ? ? ? ?Z = misultin:request(...) >> ? ? ... >> >> I "know" that lists is part of my local OTP install, elib1_misc is my >> own library installed >> in ~/code/elib2_1/ebin and misultin is an imported project stored in >> ~/imports/misultin >> I also know that my paths etc are setup so this code will work when I >> run the program. >> >> The problem is the *nobody else* knows this. >> >> I could tell the system like this: >> >> ? ?-module(foo). >> >> ? ?-location(lists, >> "https://github.com/erlang/otp/blob/dev/lib/stdlib/src/lists.erl"). >> ? ?-location(elib1_misc, >> "https://github.com/joearms/elib1/blob/master/lib/src/elib1_misc.erl"). >> ? ?-location(misultin, >> "https://github.com/ostinelli/misultin/blob/master/src/misultin.erl"). >> >> ? ?... >> >> >> ? The location annotation give a *definitive source" for the module I >> am using in the module >> >> ? ?What could you do with this information? >> >> ? ?Answer - a lot - for starters >> >> ? ? ? ? - automatically check for "latest versions" of libraries >> download them when they change >> ? ? ? ? - provide "who uses my code" feedback to the authors of the code >> ? ? ? ? - publish (globally) lists of "definitive" versions of code >> ? ? ? ? - recursive track and code dependencies (What do I mean) >> ? ? ? ? ? ?when my system discovers that I use misultin - it >> downloads misultin.erl >> ? ? ? ? ? ?misultin.erl will have location dependencies which I can >> follow, thus the libraries >> ? ? ? ? ? ?that misultin calls can be fetched. >> ? ? ? ? - automate code loading >> >> ? ?Most often this kind of "additional" information is kept "outside >> the program" by strictly >> annotating the program with the location dependencies we bring this >> information *into* the program >> in a form where it cannot be detached from the source code. >> >> ? ?Comments? >> >> ? /Joe >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Best regards, > Dmitry Demeshchuk > From kostis@REDACTED Tue Sep 13 10:03:04 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 13 Sep 2011 11:03:04 +0300 Subject: [erlang-questions] Difference of gigs between beam.smp memory and erlang:memory(total) In-Reply-To: References: Message-ID: <4E6F0E38.8060008@cs.ntua.gr> On 09/13/11 04:51, ori brost wrote: > I've been doing a simple experiment in which I attempt to connect > about 500000 websocket clients to RJ's websocket patched mochiweb > fork. I've noticed that there is a difference of several gigabytes > between memory consumed (I'm talking about working set here, not > virtual) by beam.smp and the value returned from erlang:memory(total). > Could this be related to the problem in > http://erlang.org/pipermail/erlang-questions/2011-June/059611.html? Yet another guy who thinks that Erlang developers reading this list are superheros with amazing powers that allow them to magically guess the version of Erlang/OTP he is using, the platform/OS, the details of his application and of WebSocket's code, the patches to e.g. mochiweb he has done, etc. and provide a verdict whether the problem he is experiencing could be related to an equally underspecified question somebody posted three months ago. Sorry to disappoint you but the fact is that they are mere mortals. ;) If you helped them a bit more by giving some more details and more importantly by posting some code that exhibited the problem you are experiencing, you chances of getting an answer would increase. Cheers, Kostis From erlang@REDACTED Tue Sep 13 10:06:54 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 13 Sep 2011 10:06:54 +0200 Subject: [erlang-questions] where did my code come from? In-Reply-To: <4E6F0CD2.8050005@gmail.com> References: <4E6F0CD2.8050005@gmail.com> Message-ID: On Tue, Sep 13, 2011 at 9:57 AM, Michael Truog wrote: > This sounds like a very interesting approach to aid collaboration across different software projects and avoiding fragmentation of effort (i.e., avoiding siloed development efforts, without any sharing). ?I think this is important because much software seems to be perpetually rewritten, just due to difficulties locating the proper dependencies. Yes^100 I also imagine some quite complex software in the background trying to nail down the precise dependencies. When software gets cloned copied and modified the changes are often not back-propagated to the original source - so the idea of a definitive version gets lost. We need to think of how to automate the back-propagation of changes to code. Forward propagation is easy - back propagation is far more difficult - some code vanishes from github and reappears two years later in a soureforce project with significant improvements. You google for some code and find ten different versions and haven't a clue which is best. At least writing down the exact dependencies would allow this to be tracked and analyzed. /joe With explicit dependencies at a per-module level, it would be easier to manage and track source code dependencies. ?My hope is that, you can explicitly reference a particular version of each dependency so your source code remains stable while the other dependencies change independently (so I would want a version referenced somehow within the location specification). ?Within Erlang modules, this could be handled with the vsn, but this type of version might need to be separate, just since that may complicate code upgrades. ?I am tempted to think this would benefit from integration with version control, but that would not provide a > flexible system unless there was a large integration effort to handle all major version control systems. > > Either way, I think this is a good way of managing external source code dependencies which should encourage the reuse of Erlang source code and accelerate the iterations to make source code stable. ?This type of development would benefit both proprietary development and open source development and may become a more natural way to develop source code over a Wide Area Network (WAN, so, both any intranet and the Internet). > > On 09/13/2011 12:19 AM, Joe Armstrong wrote: >> I had an idea on my way to work ... >> >> When you write code, you have *implicit* knowledge of where the >> external code comes from. >> >> When I write the following: >> >> ? ? ?-module(foo). >> ? ? ? ... >> ? ? ?start() -> >> ? ? ? ? ? X = lists:reverse(...), >> ? ? ? ? ? Y = elib1_misc:zap(...) >> ? ? ? ? ? Z = misultin:request(...) >> ? ? ?... >> >> I "know" that lists is part of my local OTP install, elib1_misc is my >> own library installed >> in ~/code/elib2_1/ebin and misultin is an imported project stored in >> ~/imports/misultin >> I also know that my paths etc are setup so this code will work when I >> run the program. >> >> The problem is the *nobody else* knows this. >> >> I could tell the system like this: >> >> ? ? -module(foo). >> >> ? ? -location(lists, >> "https://github.com/erlang/otp/blob/dev/lib/stdlib/src/lists.erl"). >> ? ? -location(elib1_misc, >> "https://github.com/joearms/elib1/blob/master/lib/src/elib1_misc.erl"). >> ? ? -location(misultin, >> "https://github.com/ostinelli/misultin/blob/master/src/misultin.erl"). >> >> ? ? ... >> >> >> ? ?The location annotation give a *definitive source" for the module I >> am using in the module >> >> ? ? What could you do with this information? >> >> ? ? Answer - a lot - for starters >> >> ? ? ? ? ?- automatically check for "latest versions" of libraries >> download them when they change >> ? ? ? ? ?- provide "who uses my code" feedback to the authors of the code >> ? ? ? ? ?- publish (globally) lists of "definitive" versions of code >> ? ? ? ? ?- recursive track and code dependencies (What do I mean) >> ? ? ? ? ? ? when my system discovers that I use misultin - it >> downloads misultin.erl >> ? ? ? ? ? ? misultin.erl will have location dependencies which I can >> follow, thus the libraries >> ? ? ? ? ? ? that misultin calls can be fetched. >> ? ? ? ? ?- automate code loading >> >> ? ? Most often this kind of "additional" information is kept "outside >> the program" by strictly >> annotating the program with the location dependencies we bring this >> information *into* the program >> in a form where it cannot be detached from the source code. >> >> ? ? Comments? >> >> ? ?/Joe >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > From mjtruog@REDACTED Tue Sep 13 10:37:16 2011 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 13 Sep 2011 01:37:16 -0700 Subject: [erlang-questions] where did my code come from? In-Reply-To: References: <4E6F0CD2.8050005@gmail.com> Message-ID: <4E6F163C.3080302@gmail.com> On 09/13/2011 01:06 AM, Joe Armstrong wrote: > On Tue, Sep 13, 2011 at 9:57 AM, Michael Truog wrote: >> This sounds like a very interesting approach to aid collaboration across different software projects and avoiding fragmentation of effort (i.e., avoiding siloed development efforts, without any sharing). I think this is important because much software seems to be perpetually rewritten, just due to difficulties locating the proper dependencies. > Yes^100 > > I also imagine some quite complex software in the background > trying to nail down the precise dependencies. > > When software gets cloned copied and modified the changes are often not > back-propagated to the original source - so the idea of a definitive > version gets > lost. We need to think of how to automate the back-propagation of > changes to code. Forward propagation is easy - back propagation is far more > difficult - some code vanishes from github and reappears two years later > in a soureforce project with significant improvements. You google for > some code and find ten different versions and haven't a clue which is best. > At least writing down the exact dependencies would allow this to be tracked > and analyzed. Yes, I agree. There would need to be a tool that does dependency analysis that would aid in modifying the dependency versions across all the modules, since that would need to happen easily to facilitate testing (which should help back-propagation take place). This seems to require keeping a local copy of all versions of the remote source code, at least to avoid problems when dependencies disappear or change their location. It might be easier to manage the dependencies if they were held in a file separate from the source code. However, I think separating the dependencies from the source code would compromise the software reuse benefits (i.e., you copy the module and you have all the dependencies). To really know what the dependency is though, you would need to specify the version in the same place. > /joe > > With explicit dependencies at a per-module level, it would be easier > to manage and track source code dependencies. My hope is that, you > can explicitly reference a particular version of each dependency so > your source code remains stable while the other dependencies change > independently (so I would want a version referenced somehow within the > location specification). Within Erlang modules, this could be handled > with the vsn, but this type of version might need to be separate, just > since that may complicate code upgrades. I am tempted to think this > would benefit from integration with version control, but that would > not provide a >> flexible system unless there was a large integration effort to handle all major version control systems. >> >> Either way, I think this is a good way of managing external source code dependencies which should encourage the reuse of Erlang source code and accelerate the iterations to make source code stable. This type of development would benefit both proprietary development and open source development and may become a more natural way to develop source code over a Wide Area Network (WAN, so, both any intranet and the Internet). >> >> On 09/13/2011 12:19 AM, Joe Armstrong wrote: >>> I had an idea on my way to work ... >>> >>> When you write code, you have *implicit* knowledge of where the >>> external code comes from. >>> >>> When I write the following: >>> >>> -module(foo). >>> ... >>> start() -> >>> X = lists:reverse(...), >>> Y = elib1_misc:zap(...) >>> Z = misultin:request(...) >>> ... >>> >>> I "know" that lists is part of my local OTP install, elib1_misc is my >>> own library installed >>> in ~/code/elib2_1/ebin and misultin is an imported project stored in >>> ~/imports/misultin >>> I also know that my paths etc are setup so this code will work when I >>> run the program. >>> >>> The problem is the *nobody else* knows this. >>> >>> I could tell the system like this: >>> >>> -module(foo). >>> >>> -location(lists, >>> "https://github.com/erlang/otp/blob/dev/lib/stdlib/src/lists.erl"). >>> -location(elib1_misc, >>> "https://github.com/joearms/elib1/blob/master/lib/src/elib1_misc.erl"). >>> -location(misultin, >>> "https://github.com/ostinelli/misultin/blob/master/src/misultin.erl"). >>> >>> ... >>> >>> >>> The location annotation give a *definitive source" for the module I >>> am using in the module >>> >>> What could you do with this information? >>> >>> Answer - a lot - for starters >>> >>> - automatically check for "latest versions" of libraries >>> download them when they change >>> - provide "who uses my code" feedback to the authors of the code >>> - publish (globally) lists of "definitive" versions of code >>> - recursive track and code dependencies (What do I mean) >>> when my system discovers that I use misultin - it >>> downloads misultin.erl >>> misultin.erl will have location dependencies which I can >>> follow, thus the libraries >>> that misultin calls can be fetched. >>> - automate code loading >>> >>> Most often this kind of "additional" information is kept "outside >>> the program" by strictly >>> annotating the program with the location dependencies we bring this >>> information *into* the program >>> in a form where it cannot be detached from the source code. >>> >>> Comments? >>> >>> /Joe >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> From erlang@REDACTED Tue Sep 13 11:08:45 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 13 Sep 2011 11:08:45 +0200 Subject: [erlang-questions] where did my code come from? In-Reply-To: <4E6F163C.3080302@gmail.com> References: <4E6F0CD2.8050005@gmail.com> <4E6F163C.3080302@gmail.com> Message-ID: On Tue, Sep 13, 2011 at 10:37 AM, Michael Truog wrote: > On 09/13/2011 01:06 AM, Joe Armstrong wrote: >> On Tue, Sep 13, 2011 at 9:57 AM, Michael Truog wrote: >>> This sounds like a very interesting approach to aid collaboration across different software projects and avoiding fragmentation of effort (i.e., avoiding siloed development efforts, without any sharing). ?I think this is important because much software seems to be perpetually rewritten, just due to difficulties locating the proper dependencies. >> Yes^100 >> >> I also imagine some quite complex software in the background >> trying to nail down the precise dependencies. >> >> When software gets cloned copied and modified the changes are often not >> back-propagated to the original source - so the idea of a definitive >> version gets >> lost. We need to think of how to automate the back-propagation of >> changes to code. Forward propagation is easy - back propagation is far more >> difficult - some code vanishes from github and reappears two years later >> in a soureforce project with significant improvements. You google for >> some code and find ten different versions and haven't a clue which is best. >> At least writing down the exact dependencies would allow this to be tracked >> and analyzed. > > Yes, I agree. ?There would need to be a tool that does dependency analysis that would aid in modifying the dependency versions across all the modules, since that would need to happen easily to facilitate testing (which should help back-propagation take place). ?This seems to require keeping a local copy of all versions of the remote source code, at least to avoid problems when dependencies disappear or change their location. ?It might be easier to manage the dependencies if they were held in a file separate from the source code. Possible - I have taken to adding a file called meta.info to most of my directories - it contains an erlang term saying what's in the directory. Then I can rename and move the directory - and the meta file moves as well - then I scan my disk and put all the meta data into a database This is part of a long-term clean-up project - I have about 50K directories to go :-) /Joe ?However, I think separating the dependencies from the source code would compromise the software reuse benefits (i.e., you copy the module and you have all the dependencies). ?To really know what the dependency is though, you would need to specify the version in the same place. > >> /joe >> >> With explicit dependencies at a per-module level, it would be easier >> to manage and track source code dependencies. ?My hope is that, you >> can explicitly reference a particular version of each dependency so >> your source code remains stable while the other dependencies change >> independently (so I would want a version referenced somehow within the >> location specification). ?Within Erlang modules, this could be handled >> with the vsn, but this type of version might need to be separate, just >> since that may complicate code upgrades. ?I am tempted to think this >> would benefit from integration with version control, but that would >> not provide a >>> flexible system unless there was a large integration effort to handle all major version control systems. >>> >>> Either way, I think this is a good way of managing external source code dependencies which should encourage the reuse of Erlang source code and accelerate the iterations to make source code stable. ?This type of development would benefit both proprietary development and open source development and may become a more natural way to develop source code over a Wide Area Network (WAN, so, both any intranet and the Internet). >>> >>> On 09/13/2011 12:19 AM, Joe Armstrong wrote: >>>> I had an idea on my way to work ... >>>> >>>> When you write code, you have *implicit* knowledge of where the >>>> external code comes from. >>>> >>>> When I write the following: >>>> >>>> ? ? ?-module(foo). >>>> ? ? ? ... >>>> ? ? ?start() -> >>>> ? ? ? ? ? X = lists:reverse(...), >>>> ? ? ? ? ? Y = elib1_misc:zap(...) >>>> ? ? ? ? ? Z = misultin:request(...) >>>> ? ? ?... >>>> >>>> I "know" that lists is part of my local OTP install, elib1_misc is my >>>> own library installed >>>> in ~/code/elib2_1/ebin and misultin is an imported project stored in >>>> ~/imports/misultin >>>> I also know that my paths etc are setup so this code will work when I >>>> run the program. >>>> >>>> The problem is the *nobody else* knows this. >>>> >>>> I could tell the system like this: >>>> >>>> ? ? -module(foo). >>>> >>>> ? ? -location(lists, >>>> "https://github.com/erlang/otp/blob/dev/lib/stdlib/src/lists.erl"). >>>> ? ? -location(elib1_misc, >>>> "https://github.com/joearms/elib1/blob/master/lib/src/elib1_misc.erl"). >>>> ? ? -location(misultin, >>>> "https://github.com/ostinelli/misultin/blob/master/src/misultin.erl"). >>>> >>>> ? ? ... >>>> >>>> >>>> ? ?The location annotation give a *definitive source" for the module I >>>> am using in the module >>>> >>>> ? ? What could you do with this information? >>>> >>>> ? ? Answer - a lot - for starters >>>> >>>> ? ? ? ? ?- automatically check for "latest versions" of libraries >>>> download them when they change >>>> ? ? ? ? ?- provide "who uses my code" feedback to the authors of the code >>>> ? ? ? ? ?- publish (globally) lists of "definitive" versions of code >>>> ? ? ? ? ?- recursive track and code dependencies (What do I mean) >>>> ? ? ? ? ? ? when my system discovers that I use misultin - it >>>> downloads misultin.erl >>>> ? ? ? ? ? ? misultin.erl will have location dependencies which I can >>>> follow, thus the libraries >>>> ? ? ? ? ? ? that misultin calls can be fetched. >>>> ? ? ? ? ?- automate code loading >>>> >>>> ? ? Most often this kind of "additional" information is kept "outside >>>> the program" by strictly >>>> annotating the program with the location dependencies we bring this >>>> information *into* the program >>>> in a form where it cannot be detached from the source code. >>>> >>>> ? ? Comments? >>>> >>>> ? ?/Joe >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> > > From sverker@REDACTED Tue Sep 13 11:42:19 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Tue, 13 Sep 2011 11:42:19 +0200 Subject: [erlang-questions] ets:next/2 behavior - set vs. ordered_set In-Reply-To: <00E5DFDF-EF45-4A8A-8871-227911A55C78@lovely.email.ne.jp> References: <00E5DFDF-EF45-4A8A-8871-227911A55C78@lovely.email.ne.jp> Message-ID: <4E6F257B.4050406@erix.ericsson.se> Joseph Wayne Norton wrote: > I'm currently developing a disk based clone of the ets application. During testing with QuickCheck and Proper, I stumbled upon a difference between set and ordered_set for the real ets:next/2 operation. > > The last command exits with badarg rather than '$end_of_table' as I would expect. > > Can someone comment on this behavior? Is this a bug? Is this a documentation bug It is the intended behavior. I don't think the documentation is contradicting but it could be much more clearer and contain something like: For tables of the ordered_set type, next/2 will succeed regardless of Key1. For other tables Key1 must exist in the table or have existed the last time the tables was "unfixed" (see safe_fixtable/2), otherwise badarg is thrown. /Sverker, Erlang/OTP From fred.hebert@REDACTED Tue Sep 13 13:01:45 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Tue, 13 Sep 2011 07:01:45 -0400 Subject: [erlang-questions] where did my code come from? In-Reply-To: References: Message-ID: <45244C3D-2B90-422C-BE50-9CF422150FD5@erlang-solutions.com> Problems I would foresee: 1. What if I try to include the code that is in charge of fetching code remotely, and that the new one I include is broken. Do I just break the downloads of other files? 2. What happens if many modules try to include conflicting versions of a dependency? This is a problem already, but managed at a higher level with applications 3. If my module depends on other local modules (before I've uploaded anything to github) do I add local paths or just none of these clauses? 4. How would we represent differences between depending on applications and depending on modules? Applications might require drivers, NIFs, etc. which will need specific compiling options. 5. How does this work for code updates? 6. What about protocols that are not git, hg, http or whatever and are not known to the VM? In my opinion, the basic code-sharing block should be the OTP application (or OTP library application), a bit similar to what Rebar and Agner are using at the moment. All these dependencies can be somewhat described as part of the '.app' file of each application you write, and likely are already described in there for whatever build tool to be used. The problem is that OTP has no way to react to these dependencies as of now, although it can check for conflicts, registered names, etc. Related to the list above, it would be simpler to handle point 1 (stuff can still be built, but not assembled into a release, or dependencies will be missing at startup), point 2 (versions are handled as part of releases, but still not optimal), point 3 (the {modules, ...} tuple does this). Point 3 would have no clear way to do it, no obvious fallback. Point 4 is only handled by tools like rebar or agner, that will try to compile them for you; Agner takes a command given by the developer to tell you how to build the code. Point 5 works with appups and relups and dependencies -- very hard to do, still. No fix possible for point 6 that I can think of. It would be somewhat simpler and safer, I think, to build the mechanism on top of OTP apps rather than raw modules. The problem is that releases and applications are somewhat an advanced concept that not all developers know and understand. -- Fred H?bert http://www.erlang-solutions.com On 2011-09-13, at 03:19 AM, Joe Armstrong wrote: > I had an idea on my way to work ... > > When you write code, you have *implicit* knowledge of where the > external code comes from. > > When I write the following: > > -module(foo). > ... > start() -> > X = lists:reverse(...), > Y = elib1_misc:zap(...) > Z = misultin:request(...) > ... > > I "know" that lists is part of my local OTP install, elib1_misc is my > own library installed > in ~/code/elib2_1/ebin and misultin is an imported project stored in > ~/imports/misultin > I also know that my paths etc are setup so this code will work when I > run the program. > > The problem is the *nobody else* knows this. > > I could tell the system like this: > > -module(foo). > > -location(lists, > "https://github.com/erlang/otp/blob/dev/lib/stdlib/src/lists.erl"). > -location(elib1_misc, > "https://github.com/joearms/elib1/blob/master/lib/src/elib1_misc.erl"). > -location(misultin, > "https://github.com/ostinelli/misultin/blob/master/src/misultin.erl"). > > ... > > > The location annotation give a *definitive source" for the module I > am using in the module > > What could you do with this information? > > Answer - a lot - for starters > > - automatically check for "latest versions" of libraries > download them when they change > - provide "who uses my code" feedback to the authors of the code > - publish (globally) lists of "definitive" versions of code > - recursive track and code dependencies (What do I mean) > when my system discovers that I use misultin - it > downloads misultin.erl > misultin.erl will have location dependencies which I can > follow, thus the libraries > that misultin calls can be fetched. > - automate code loading > > Most often this kind of "additional" information is kept "outside > the program" by strictly > annotating the program with the location dependencies we bring this > information *into* the program > in a form where it cannot be detached from the source code. > > Comments? > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Tue Sep 13 13:19:18 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 13 Sep 2011 15:19:18 +0400 Subject: [erlang-questions] Erlang and beam-based scripting languages. Message-ID: Hi. I've written a letter several times to Santa Claus to have some easy Rails <-> erlang integration for my needs, but he seems to be too busy. So, my problem is: it is not convenient for me to write all customer logic in erlang, because it involves process of compiling and loading modules. Also, I would be very great to share the same file between web site and erlang daemon. It seems, that most developed beam-based language is elixir (https://github.com/josevalim/elixir ), but there are some erlang problems, that looks like blocking for have convenient scripting in erlang. I have erlyvideo server and it has config. There is option in config: {detectors, [rewrite, akamai, livestream]}. When user requests stream, media_provider takes this list of detectors, goes one by one and calls: case Module:check(Host, Name, Options) of .. for each atom. Now we meet the problem. When I write Module:check, code_server is loading module and it can load only .beam and .app files. If I have any way to hook inside with some code that will say: Hi, I know how to load this module, i think it will be easier to add simple non-compilable scripting to my app and have something like Java world have: Ruby/Python classed mixed with "native" Java classes. From erlang@REDACTED Tue Sep 13 13:25:57 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 13 Sep 2011 13:25:57 +0200 Subject: [erlang-questions] where did my code come from? In-Reply-To: <45244C3D-2B90-422C-BE50-9CF422150FD5@erlang-solutions.com> References: <45244C3D-2B90-422C-BE50-9CF422150FD5@erlang-solutions.com> Message-ID: 2011/9/13 Fr?d?ric Trottier-H?bert : > Problems I would foresee: > > 1. What if I try to include the code that is in charge of fetching code remotely, and that the new one I include is broken. Do I just break the downloads of other files? > > 2. What happens if many modules try to include conflicting versions of a dependency? This is a problem already, but managed at a higher level with applications > > 3. If my module depends on other local modules (before I've uploaded anything to github) do I add local paths or just none of these clauses? > > 4. How would we represent differences between depending on applications and depending on modules? Applications might require drivers, NIFs, etc. which will need specific compiling options. > > 5. How does this work for code updates? > > 6. What about protocols that are not git, hg, http or whatever and are not known to the VM? > > In my opinion, the basic code-sharing block should be the OTP application (or OTP library application), a bit similar to what Rebar and Agner are using at the moment. All these dependencies can be somewhat described as part of the '.app' file of each application you write, and likely are already described in there for whatever build tool to be used. The problem is that OTP has no way to react to these dependencies as of now, although it can check for conflicts, registered names, etc. > > Related to the list above, it would be simpler to handle point 1 (stuff can still be built, but not assembled into a release, or dependencies will be missing at startup), point 2 (versions are handled as part of releases, but still not optimal), point 3 (the {modules, ...} tuple does this). Point 3 would have no clear way to do it, no obvious fallback. Point 4 is only handled by tools like rebar or agner, that will try to compile them for you; Agner takes a command given by the developer to tell you how to build the code. Point 5 works with appups and relups and dependencies -- very hard to do, still. No fix possible for point 6 that I can think of. > > It would be somewhat simpler and safer, I think, to build the mechanism on top of OTP apps rather than raw modules. I'm not sure about this - often (very often) modules escape from their context - so for example, individual modules in one application escape from that application and find themselves being included in other applications. This is just done by googling and cut-and-paste. I do this frequently. How do I solve X? - Google a bit. I find some code in yaws or mochiweb that does what I want. I just take the individual module remove it from its context and either use it or modify it. The changes I make are not back-propagated into the original module. If you keep the meta data in the module itself it won't get lost if the module is seen "in the wild" - if you put the information outside the module then one day you will loose it. >The problem is that releases and applications are somewhat an advanced >concept that not all developers know and understand. > > -- > Fred H?bert > http://www.erlang-solutions.com > > > > On 2011-09-13, at 03:19 AM, Joe Armstrong wrote: > >> I had an idea on my way to work ... >> >> When you write code, you have *implicit* knowledge of where the >> external code comes from. >> >> When I write the following: >> >> ? ? -module(foo). >> ? ? ?... >> ? ? start() -> >> ? ? ? ? ?X = lists:reverse(...), >> ? ? ? ? ?Y = elib1_misc:zap(...) >> ? ? ? ? ?Z = misultin:request(...) >> ? ? ... >> >> I "know" that lists is part of my local OTP install, elib1_misc is my >> own library installed >> in ~/code/elib2_1/ebin and misultin is an imported project stored in >> ~/imports/misultin >> I also know that my paths etc are setup so this code will work when I >> run the program. >> >> The problem is the *nobody else* knows this. >> >> I could tell the system like this: >> >> ? ?-module(foo). >> >> ? ?-location(lists, >> "https://github.com/erlang/otp/blob/dev/lib/stdlib/src/lists.erl"). >> ? ?-location(elib1_misc, >> "https://github.com/joearms/elib1/blob/master/lib/src/elib1_misc.erl"). >> ? ?-location(misultin, >> "https://github.com/ostinelli/misultin/blob/master/src/misultin.erl"). >> >> ? ?... >> >> >> ? The location annotation give a *definitive source" for the module I >> am using in the module >> >> ? ?What could you do with this information? >> >> ? ?Answer - a lot - for starters >> >> ? ? ? ? - automatically check for "latest versions" of libraries >> download them when they change >> ? ? ? ? - provide "who uses my code" feedback to the authors of the code >> ? ? ? ? - publish (globally) lists of "definitive" versions of code >> ? ? ? ? - recursive track and code dependencies (What do I mean) >> ? ? ? ? ? ?when my system discovers that I use misultin - it >> downloads misultin.erl >> ? ? ? ? ? ?misultin.erl will have location dependencies which I can >> follow, thus the libraries >> ? ? ? ? ? ?that misultin calls can be fetched. >> ? ? ? ? - automate code loading >> >> ? ?Most often this kind of "additional" information is kept "outside >> the program" by strictly >> annotating the program with the location dependencies we bring this >> information *into* the program >> in a form where it cannot be detached from the source code. >> >> ? ?Comments? >> >> ? /Joe >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From fred.hebert@REDACTED Tue Sep 13 13:38:39 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Tue, 13 Sep 2011 07:38:39 -0400 Subject: [erlang-questions] where did my code come from? In-Reply-To: References: <45244C3D-2B90-422C-BE50-9CF422150FD5@erlang-solutions.com> Message-ID: <48740D12-0407-4F09-8688-48E2F80271F4@erlang-solutions.com> On 2011-09-13, at 07:25 AM, Joe Armstrong wrote: > 2011/9/13 Fr?d?ric Trottier-H?bert : >> Problems I would foresee: >> >> ... >> >> In my opinion, the basic code-sharing block should be the OTP application (or OTP library application), a bit similar to what Rebar and Agner are using at the moment. All these dependencies can be somewhat described as part of the '.app' file of each application you write, and likely are already described in there for whatever build tool to be used. The problem is that OTP has no way to react to these dependencies as of now, although it can check for conflicts, registered names, etc. >> >> Related to the list above, it would be simpler to handle point 1 (stuff can still be built, but not assembled into a release, or dependencies will be missing at startup), point 2 (versions are handled as part of releases, but still not optimal), point 3 (the {modules, ...} tuple does this). Point 3 would have no clear way to do it, no obvious fallback. Point 4 is only handled by tools like rebar or agner, that will try to compile them for you; Agner takes a command given by the developer to tell you how to build the code. Point 5 works with appups and relups and dependencies -- very hard to do, still. No fix possible for point 6 that I can think of. >> >> It would be somewhat simpler and safer, I think, to build the mechanism on top of OTP apps rather than raw modules. > > I'm not sure about this - often (very often) modules escape from their > context - so for example, individual modules in one application > escape from that application and find themselves being included in other > applications. This is just done by googling and cut-and-paste. I do > this frequently. How do I solve X? - Google a bit. I find some code in > yaws > or mochiweb that does what I want. I just take the individual module > remove it from its context and either use it or modify it. The changes > I make are not back-propagated into the original module. If you > keep the meta data in the module itself it won't get lost if the module > is seen "in the wild" - if you put the information outside the module > then one day you will loose it. > For individual modules escaping their original context, this is something that can be handled with Reltool using filters, although it's not fully optimal or easy to do. I've had this in some code where I depended on mochiweb modules, but not the whole thing. In my case, my release config file (https://bitbucket.org/ferd/blogerl/src/e8317b92af3a/rel/rel.conf) has this policy for mochiweb: {app, mochiweb, [{incl_app_filters, [ "^ebin/mochiutf8.beam$", "^ebin/mochiweb_charref.beam$", "^ebin/mochiweb_html.beam$" ]}, {incl_cond, include}]} While in the release in general applications are not included by default ({incl_cond, exclude}). This lets me build a specific mochiweb library for my release, only containing the 3 files mentioned there. By doing so and depending on mochiweb in rebar (or agner, or any other build system), I'm able to create a final application with only the files I need, but the source version can still be kept up to date with all the related libraries. The problem here is how accessible this way of doing things is to Erlang programmers. Most of us do not know how to build releases with a lot of ease. In any case, this doesn't solve the other issue you mentioned -- if I modify the mochiweb modules, then I need to fork it, make a branch or copy the repository myself and then link to that in rebar/agner/whatever, hoping that the online repo will contain all the information I need. There will be no quick way to relate *my* mochiweb to the *original* mochiweb unless someone checks the link and sees that github, bitbucket or google code has some link relating it back to the original version. From vladdu55@REDACTED Tue Sep 13 13:42:57 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 13 Sep 2011 13:42:57 +0200 Subject: [erlang-questions] Erlang and beam-based scripting languages. In-Reply-To: References: Message-ID: Hi, On Tue, Sep 13, 2011 at 13:19, Max Lapshin wrote: > I have erlyvideo server and it has config. There is option in config: > {detectors, [rewrite, akamai, livestream]}. > > When user requests stream, media_provider takes this list of > detectors, goes one by one and calls: > > case Module:check(Host, Name, Options) of > ? .. > > for each atom. Now we meet the problem. When I write ?Module:check, > code_server is loading module and it can load only .beam and .app > files. > If I have any way to hook inside with some code that will say: Hi, I > know how to load this module, i think it will be easier to add simple > non-compilable scripting to my app and have something like Java world > have: ?Ruby/Python classed mixed with "native" Java classes. Do I understand correctly that you would like to have a module loader that accepts registrations for file type handlers and whenever it doesn't find a .beam file for a module, asks these handlers if they can find a source file and compile+load it? That would be cool. The only delicate issue I see right now is how to find these source files, if the source directories aren't added to the code path. regards, Vlad From sam@REDACTED Tue Sep 13 13:43:29 2011 From: sam@REDACTED (Samuel Elliott) Date: Tue, 13 Sep 2011 12:43:29 +0100 Subject: [erlang-questions] Erlang release names and versions In-Reply-To: References: Message-ID: Did you see this? It may have answers to some of your questions http://learnyousomeerlang.com/relups (including if you need to move generated files) Sam On Tue, Sep 13, 2011 at 6:17 AM, Sam Bobroff wrote: > Hi Erlangers, > I'm trying to package up some Erlang projects for distribution, > possibly by creating Releases, and I'm getting quite confused. I'd > like to do it the "correct" way because we may want to use > release_handler to perform upgrades. > > Maybe someone can help me out :-) > > The documentation for Releases > (http://www.erlang.org/doc/design_principles/release_structure.html) > shows a sample directory structure produced by make_tar, which > includes this part: > > releases/A/start.boot > releases/ch_rel-1.rel > > Where ch_rel-1 is the name of the release and A is the version of the > release. > > Question 1: If the release file is placed in releases/ch_rel-1.rel, > then isn't installing a second version of the release (say, "B") going > to overwrite the file? Isn't this a problem? Is the addition of "-1" > to the release name supposed to address this? Shouldn't the release > name appear somewhere in the path? > > Question 2: If another release, say "foo", is installed with the same > version (say "A"), isn't it going to overwrite the start.boot file > from release "ch_rel-1"? > > Question 3: Should I even be trying to install more than one release > at once? Would I need a completely new Erlang install directory (or at > least, releases directory, by using RELDIR)? > > (I have, incidentally, seen somewhere that the placement of the .rel > file directly under releases/ by make_tar is a bug. Is this so? It > certainly doesn't seem to match the structure created by > create_target.) > > Thanks in advance for any help, > Sam. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Samuel Elliott sam@REDACTED http://lenary.co.uk/ +44 (0)7891 993 664 -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue Sep 13 14:05:56 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 13 Sep 2011 16:05:56 +0400 Subject: [erlang-questions] Erlang and beam-based scripting languages. In-Reply-To: References: Message-ID: On Tue, Sep 13, 2011 at 3:42 PM, Vlad Dumitrescu wrote: > Do I understand correctly that you would like to have a module loader > that accepts registrations for file type handlers and whenever it > doesn't find a .beam file for a module, asks these handlers if they > can find a source file and compile+load it? That would be cool. Exactly. But there is comment in code_server looking like "leave hope anyone, who is going to change this module, many brave Swedens are lying here" and I'm a bit afraid to edit it =) > The > only delicate issue I see right now is how to find these source files, > if the source directories aren't added to the code path. > I think, best behaviour is to look inside code path. No code path, no modules. I've tried to take elixir and load it. Currently it is looking so (doesn't do any real job, just for example): digital_access.ex: module DigitalAccess def check(host, name, options) IO.puts "Called with #{host} #{name} #{options.inspect()}" {ok, headers, reply} = Erlang.http_stream.get_with_body("http://ya.ru/", []) url = "http://video.ru/movies/"+headers['"Content-Length"] [{'type, 'file},{'url, url},{'file_access,'http_file}] end end (ev@REDACTED)1> code:add_pathz(code:lib_dir(elixir,exbin)). true (ev@REDACTED)2> application:start(elixir). ok (ev@REDACTED)3> elixir:file("digital_access.ex"). {{elixir_slate__,'exUnboundMethod::Behavior', [{arity,3},{name,check},{owner,exDigitalAccess}]}, []} (ev@REDACTED)4> SomeUnknownArgument = []. [] (ev@REDACTED)5> 'exDigitalAccess':check([], default, <<"http://ya.ru/">>, [{url, <<"zzz">>}]). Called with default http://ya.ru/ [{'url,"zzz"}] [{type,file}, {url,<<"http://video.ru/movies/6747">>}, {file_access,http_file}] (ev@REDACTED)6> So it really looks very close to what I want, except module name and automatic loading. From vladimir.kirichenko@REDACTED Tue Sep 13 14:18:17 2011 From: vladimir.kirichenko@REDACTED (Volodymyr Kyrychenko) Date: Tue, 13 Sep 2011 15:18:17 +0300 Subject: [erlang-questions] calling FUN from NIF Message-ID: Hi All, Where can I find info/example about calling erlang functions (callback funs in particular) from NIFs? -- Best Regards, Volodymyr Kyrychenko From cgsmcmlxxv@REDACTED Tue Sep 13 14:41:05 2011 From: cgsmcmlxxv@REDACTED (George Catalin Serbanut) Date: Tue, 13 Sep 2011 14:41:05 +0200 Subject: [erlang-questions] Erlang and beam-based scripting languages. In-Reply-To: References: Message-ID: Hi, Max, I don't know if I understood correctly, but what you are looking for is kind of web server. Is that statement correct? If yes, see YAWS (Yet Another Web Server - http://yaws.hyber.org/). Other options: 1. If you look for a database (with web pages capability), see CouchDB ( http://couchdb.apache.org/). 2. If you look for a messenger-like server, see Ejabberd ( http://www.ejabberd.im/). 3. If you are looking for messenger-like client, see Exmpp ( https://support.process-one.net/doc/display/EXMPP/exmpp+home). 4. If you are looking to pass arguments to a process in a dynamic way via a configuration file, see os:cmd/1 function from Erlang (e.g., Customer = "Max", CustomerPhoneNumber = os:cmd("cat phone_agend.txt | grep \""++Customer++"\" | awk'{print $2}' ", or Type = "video", Video = "crazy_cow", os:cmd("exec $(cat mime.cfg | grep \""++Type++"\" | awk '{printf("%s", $2)}') "++Video)), where in mime.cfg you put a line `video vlc'). Just use your imagination here. ;) I hope this will help you. Good luck! Cheers, CGS On Tue, Sep 13, 2011 at 2:05 PM, Max Lapshin wrote: > On Tue, Sep 13, 2011 at 3:42 PM, Vlad Dumitrescu > wrote: > > > Do I understand correctly that you would like to have a module loader > > that accepts registrations for file type handlers and whenever it > > doesn't find a .beam file for a module, asks these handlers if they > > can find a source file and compile+load it? That would be cool. > > Exactly. But there is comment in code_server looking like "leave hope > anyone, who is going to change this module, many brave Swedens are > lying here" > and I'm a bit afraid to edit it =) > > > > The > > only delicate issue I see right now is how to find these source files, > > if the source directories aren't added to the code path. > > > > I think, best behaviour is to look inside code path. No code path, no > modules. > > I've tried to take elixir and load it. Currently it is looking so > (doesn't do any real job, just for example): > > digital_access.ex: > > module DigitalAccess > def check(host, name, options) > IO.puts "Called with #{host} #{name} #{options.inspect()}" > {ok, headers, reply} = Erlang.http_stream.get_with_body("http://ya.ru/", > []) > url = "http://video.ru/movies/"+headers['"Content-Length"] > [{'type, 'file},{'url, url},{'file_access,'http_file}] > end > end > > > > (ev@REDACTED)1> code:add_pathz(code:lib_dir(elixir,exbin)). > true > (ev@REDACTED)2> application:start(elixir). > ok > (ev@REDACTED)3> elixir:file("digital_access.ex"). > {{elixir_slate__,'exUnboundMethod::Behavior', > [{arity,3},{name,check},{owner,exDigitalAccess}]}, > []} > (ev@REDACTED)4> SomeUnknownArgument = []. > [] > (ev@REDACTED)5> 'exDigitalAccess':check([], default, <<"http://ya.ru/">>, > [{url, <<"zzz">>}]). > Called with default http://ya.ru/ [{'url,"zzz"}] > [{type,file}, > {url,<<"http://video.ru/movies/6747">>}, > {file_access,http_file}] > (ev@REDACTED)6> > > > > So it really looks very close to what I want, except module name and > automatic loading. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Tue Sep 13 14:49:10 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 13 Sep 2011 14:49:10 +0200 Subject: [erlang-questions] Erlang and beam-based scripting languages. In-Reply-To: References: Message-ID: On Tue, Sep 13, 2011 at 14:05, Max Lapshin wrote: > I've tried to take ?elixir and load it. > ...... > So it really looks very close to what I want, except module name and > automatic loading. Elixir compiles to beam, so it might be enough to have a daemon watch for changes in the source directory and automatically compile the elixir files? Then the only problem is with the module names, but that I would say is something that elixir should address - possibly with a compilation flag to not mangle module names. regards, Vlad From kblachowski@REDACTED Tue Sep 13 15:25:42 2011 From: kblachowski@REDACTED (Krzysztof Blachowski) Date: Tue, 13 Sep 2011 15:25:42 +0200 Subject: [erlang-questions] Two network interfaces. Message-ID: Hi all. Let's imagine that we have two machines A and B with erlang VM. On each erlang machine there is application which uses mnesia database. The database is replicated between nodes. Each node has two interfaces "I1" which is extremely fast and "I2" which is slow. There is also a bunch of erlang nodes which are clients for A and B nodes and they are using native erlang interface. I was trying to configure erlang nodes A and B to utilize this two interfaces but without any effect. How to connect nodes A i B with fast link (for mnesia replication) and at this same time connect them with slow link with external clients? I've tried: machine A: erl -name master -setcookie secret inet:getif(): {ok,[{{192,168,1,10},{192,168,255,255},{255,255,0,0}}, {{10,132,24,10},{10,132,25,255},{255,255,254,0}}, {{127,0,0,1},{0,0,0,0},{255,0,0,0}}]} machine B: erl -name replica -setcookie secret net_adm:ping('master@REDACTED'): pong net_adm:ping('master@REDACTED'): pang Why? What I'm doing wrong? best regards ---------------------------------------------------------------- Skoda Octavia z 2009 roku juz za 11 tys. zl http://linkint.pl/f2a36 From paul.joseph.davis@REDACTED Tue Sep 13 15:55:37 2011 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Tue, 13 Sep 2011 13:55:37 +0000 Subject: [erlang-questions] calling FUN from NIF In-Reply-To: References: Message-ID: There is currently no such behavior in the API. For the moment your best bet would be to write an Erlang shim that responded to messages from the NIF that would be written in to be resumable or uses threads to communicate with the Erlang VM. On the other hand, there are planned enhancements to the NIF api to allow such awesomeness. See Rickard's talk at [1]. Paul Davis [1] http://www.erlang-factory.com/conference/SFBay2011/speakers/RickardGreen On Tue, Sep 13, 2011 at 12:18 PM, Volodymyr Kyrychenko wrote: > Hi All, > > Where can I find info/example about calling erlang functions (callback > funs in particular) from NIFs? > > -- > Best Regards, > Volodymyr Kyrychenko > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From max.lapshin@REDACTED Tue Sep 13 15:57:25 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 13 Sep 2011 17:57:25 +0400 Subject: [erlang-questions] Erlang and beam-based scripting languages. In-Reply-To: References: Message-ID: This is proof-of-concept error_handler replacement, that can pass calls to elixir modules: https://gist.github.com/1213853 Currently it already works as: elixir_loader:undefined_function(digital_access, check, [default, <<"ya.ru">>, [{key,value}]]). It looks if there is file digital_access.ex, loads it, try to find if module with proper name was loaded and calls function from there, according to naming and call conventions. But doesn't work as an error_handler =(( From watson.timothy@REDACTED Tue Sep 13 17:17:36 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 13 Sep 2011 16:17:36 +0100 Subject: [erlang-questions] where did my code come from? In-Reply-To: References: <45244C3D-2B90-422C-BE50-9CF422150FD5@erlang-solutions.com> Message-ID: > > > I'm not sure about this - often (very often) modules escape from their > context - so for example, individual modules in one application > escape from that application and find themselves being included in other > applications. This is just done by googling and cut-and-paste. I do > this frequently. How do I solve X? - Google a bit. I find some code in > yaws > or mochiweb that does what I want. I just take the individual module > remove it from its context and either use it or modify it. The changes > I make are not back-propagated into the original module. If you > keep the meta data in the module itself it won't get lost if the module > is seen "in the wild" - if you put the information outside the module > then one day you will loose it. > > If you pulled these module(s) you want to take code from out into a separate library, published it on github and sent a request to the original author(s) to adopt this as a separate dependency, there would be a bit more granular reuse going on. Naturally the original authors might reject your dependency for their code, but with the right metadata available in your new library, others would be able to discover and potentially also use it. >The problem is that releases and applications are somewhat an advanced > >concept that not all developers know and understand. > But they ought to learn about them. Making releases and libraries/applications the standard unit of distribution and dependency use/reuse (as rebar, agner and other tools do) should reinforce this and encourage people to learn. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gordon@REDACTED Tue Sep 13 17:28:23 2011 From: gordon@REDACTED (Gordon Guthrie) Date: Tue, 13 Sep 2011 16:28:23 +0100 Subject: [erlang-questions] Erlang and beam-based scripting languages. In-Reply-To: References: Message-ID: We have some proof of concept Erlang/Ruby integration in our source base. Essentially is uses the erlectricity gem to provide a message passing interface between Erlang and Ruby. The erlang shim starts a driver which it sends messages to. These messages are unpacked and the 'applyied' to ruby functions which return values. The returns are packed into a message and returned. There is some details on it here: http://www.rubyinside.com/erlang-ruby-bridge-its-erlectricity-487.html We used it for a bit and then stopped. Dunno if it is still supported. YMMV, price of shares can go down as well as up, terms and conditions apply, etc, etc G On 13 September 2011 12:19, Max Lapshin wrote: > Hi. > > I've written a letter several times to Santa Claus to have some easy > Rails <-> erlang integration for my needs, but he seems to be too > busy. > > So, my problem is: it is not convenient for me to write all customer > logic in erlang, because it involves process of compiling and loading > modules. > Also, I would be very great to share the same file between web site > and erlang daemon. > > It seems, that most developed beam-based language is elixir > (https://github.com/josevalim/elixir ), but there are some erlang > problems, that looks like blocking for > have convenient scripting in erlang. > > > I have erlyvideo server and it has config. There is option in config: > > {detectors, [rewrite, akamai, livestream]}. > > When user requests stream, media_provider takes this list of > detectors, goes one by one and calls: > > case Module:check(Host, Name, Options) of > .. > > for each atom. Now we meet the problem. When I write Module:check, > code_server is loading module and it can load only .beam and .app > files. > If I have any way to hook inside with some code that will say: Hi, I > know how to load this module, i think it will be easier to add simple > non-compilable scripting to my app and have something like Java world > have: Ruby/Python classed mixed with "native" Java classes. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue Sep 13 17:33:35 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 13 Sep 2011 19:33:35 +0400 Subject: [erlang-questions] Erlang and beam-based scripting languages. In-Reply-To: References: Message-ID: This is familiar to me. I've written my own erlang external term dumper in ruby with full support for EventMachine, including simple creation of erlang-like processes. It was big and 100% useless job =)) External program is not very usefull, because I really want to make funcalls into erlang VM from a script. Without these funcall, I'd better make HTTP request with JSON request to _any_ scripted web-server. From michael.santos@REDACTED Tue Sep 13 18:14:10 2011 From: michael.santos@REDACTED (Michael Santos) Date: Tue, 13 Sep 2011 12:14:10 -0400 Subject: [erlang-questions] Two network interfaces. In-Reply-To: References: Message-ID: <20110913161410.GA29881@ecn.lan> On Tue, Sep 13, 2011 at 03:25:42PM +0200, Krzysztof Blachowski wrote: > Hi all. > > Let's imagine that we have two machines A and B with erlang VM. > > On each erlang machine there is application which uses mnesia database. The database is replicated between nodes. > Each node has two interfaces "I1" which is extremely fast and "I2" which is slow. There is also a bunch of erlang nodes which are clients > for A and B nodes and they are using native erlang interface. > > I was trying to configure erlang nodes A and B to utilize this two interfaces but without any effect. > How to connect nodes A i B with fast link (for mnesia replication) and at this same time connect them with slow link with external clients? > > I've tried: > > > machine A: > > erl -name master -setcookie secret > > inet:getif(): > {ok,[{{192,168,1,10},{192,168,255,255},{255,255,0,0}}, > {{10,132,24,10},{10,132,25,255},{255,255,254,0}}, > {{127,0,0,1},{0,0,0,0},{255,0,0,0}}]} > > machine B: > > erl -name replica -setcookie secret > > net_adm:ping('master@REDACTED'): > pong > > net_adm:ping('master@REDACTED'): > pang > > Why? What I'm doing wrong? Since the name of the node is 'master@REDACTED', it won't respond to 'master@REDACTED'. What might work is changing the DNS resolution on either the clients or the hosts. If the interfaces are mapped as follows: I1: 192.168.1.10 I2: 10.132.24.10 The clients will do a DNS lookup on 'my.domain' and get 10.132.24.10. On the mnesia nodes, force the resolution to the fast interface by adding an entry to the /etc/hosts file: 192.168.1.10 my.domain Or if you don't want to change the name lookup for everything on the host, you can set it just for Erlang, something like: cat<erl_inetrc {host, {192,168,1,10}, ["my.domain"]}. {lookup, [file, dns]}. EOF The run: erl -name master -kernel inetrc '"erl_inetrc"' It's all explained here: http://www.erlang.org/doc/apps/erts/inet_cfg.html From jesper.louis.andersen@REDACTED Tue Sep 13 19:21:36 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 13 Sep 2011 19:21:36 +0200 Subject: [erlang-questions] where did my code come from? In-Reply-To: References: Message-ID: On Tue, Sep 13, 2011 at 09:19, Joe Armstrong wrote: > ? ?-location(lists, > "https://github.com/erlang/otp/blob/dev/lib/stdlib/src/lists.erl"). > ? ?Comments? I like the idea quite a lot. In a modern world, the whole notion of "a module lives on a single file system" is wrong. It is rather the case that a module lives its life behind an URI, is subject to change and that locally we are more concerned about caching and having the correct version. While you are at it, your -location() notion also allows for import renaming, which is something that is needed dearly in Erlang. Who says a module is in a file? It could be in an HTTP stream, or sent over a websocket or XMPP even. There is no reason to tie it to a specific carrier. As for security, security is to be had by forming releases. When you create a release, you can use signatures to verify the authenticity of modules from module authors. The signature is even present easily in git with a merkle-tree-like construction. The alternative, which is probably some 20-30 years out in the future is Proof-carrying-code where the code itself provides a proof of what it does - so lets stick with signatures for now. Releases also provide a stable notion of the system. It is not subject to change once formed. Then there is the part of self-documenting the dependencies, which I also really like. In the Standard ML community, there is a nice MLB concept where you can form bundles of software in the large. You can say "This application consists of these files, and it exports these modules and functors under these (renamed) names". Upon import, you can also rename the API names to avoid clashes locally. The standard library is just a MLB application reference. The older standard library can be had by another reference. Essentially we have a notion of lexically scoped module availability and exportation in the MLB files. But they never considered a world where everything is just an URI. Which is why I like the location idea. It incorporates that notion, and it is a tighter coupling than an external MLB file, which I am not sure is the way to go. In any case, if one were to revamp the module system of Erlang, -location() should be a consideration. -- J. From max.lapshin@REDACTED Tue Sep 13 20:13:55 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 13 Sep 2011 22:13:55 +0400 Subject: [erlang-questions] Erlang and beam-based scripting languages. In-Reply-To: References: Message-ID: This is maximum, that I can do for now. https://github.com/maxlapshin/elixir/blob/master/src/elixir_loader.erl This code can load elixir modules and call functions from them. But it can't work as error_handler. It seems, that guys from OTP have made some sacrifice to Odin to make this piece of magic work =)) But if to make it work, it will be able to reused to make any kind of magic, like automatic calling Corba methods, or anything else. From ds.erl@REDACTED Tue Sep 13 20:19:07 2011 From: ds.erl@REDACTED (ds.erl@REDACTED) Date: Tue, 13 Sep 2011 20:19:07 +0200 Subject: [erlang-questions] Erlang and beam-based scripting languages. In-Reply-To: References: Message-ID: On 13 Sep 2011, at 13:19, Max Lapshin wrote: > So, my problem is: it is not convenient for me to write all customer > logic in erlang, because it involves process of compiling and loading > modules. You can always read erlang source code with file:path_script/3 and evaluate it in an environment set up to your liking. IMO this offers all the advantages of scripting *and* you're using the "native" language of your app. Regards. -Daniel From erlang@REDACTED Tue Sep 13 21:42:30 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 13 Sep 2011 21:42:30 +0200 Subject: [erlang-questions] where did my code come from? In-Reply-To: References: Message-ID: On Tue, Sep 13, 2011 at 7:21 PM, Jesper Louis Andersen wrote: > On Tue, Sep 13, 2011 at 09:19, Joe Armstrong wrote: >> ? ?-location(lists, >> "https://github.com/erlang/otp/blob/dev/lib/stdlib/src/lists.erl"). > >> ? ?Comments? > > I like the idea quite a lot. In a modern world, the whole notion of "a > module lives on a single file system" is wrong. It is rather the case > that a module lives its life behind an URI, is subject to change and > that locally we are more concerned about caching and having the > correct version. While you are at it, your -location() notion also > allows for import renaming, which is something that is needed dearly > in Erlang. Who says a module is in a file? It could be in an HTTP > stream, or sent over a websocket or XMPP even. There is no reason to > tie it to a specific carrier. YES^10 - given that it can be somewhat tricky to install an application locally one might want to say -location(mod23, "erlmod://a.y.z/foo/bar"). after which mod23:foo(...) would work via a RPC ton the remote host could be useful for quick-and-dirty testing or keeping code private > > As for security, security is to be had by forming releases. When you > create a release, you can use signatures to verify the authenticity of > modules from module authors. The signature is even present easily in > git with a merkle-tree-like construction. The alternative, which is > probably some 20-30 years out in the future is Proof-carrying-code > where the code itself provides a proof of what it does - so lets stick > with signatures for now. > > Releases also provide a stable notion of the system. It is not subject > to change once formed. > > Then there is the part of self-documenting the dependencies, which I > also really like. In the Standard ML community, there is a nice MLB > concept where you can form bundles of software in the large. You can > say "This application consists of these files, and it exports these > modules and functors under these (renamed) names". Upon import, you > can also rename the API names to avoid clashes locally. The standard > library is just a MLB application reference. The older standard > library can be had by another reference. yes - I made a experiment where I renamed modules with md5(content) if you see what I mean and signed/encrypted them with RSA keys :-) > > Essentially we have a notion of lexically scoped module availability > and exportation in the MLB files. But they never considered a world > where everything is just an URI. Which is why I like the location > idea. It incorporates that notion, and it is a tighter coupling than > an external MLB file, which I am not sure is the way to go. In any > case, if one were to revamp the module system of Erlang, -location() > should be a consideration. > > > -- > J. > From dave@REDACTED Tue Sep 13 21:44:30 2011 From: dave@REDACTED (David Goehrig) Date: Tue, 13 Sep 2011 15:44:30 -0400 Subject: [erlang-questions] where did my code come from? In-Reply-To: References: Message-ID: <36F7D21B-B7C7-4CCF-A64D-F0E3543BEF3C@nexttolast.com> This is basically how most real world JavaScript applications already work, so it simply falls to building a nice distributed registry that can be universally queried. Erlang txt fields in DNS anyone? Also since rebar already allows you to specify your dependencies as git tags, we already have a system that will automatically pull the specific source (or latest if unspecified) and build your application. Dave -=-=- dave@REDACTED -=-=- On Sep 13, 2011, at 3:19 AM, Joe Armstrong wrote: > I had an idea on my way to work ... > > When you write code, you have *implicit* knowledge of where the > external code comes from. > > When I write the following: > > -module(foo). > ... > start() -> > X = lists:reverse(...), > Y = elib1_misc:zap(...) > Z = misultin:request(...) > ... > > I "know" that lists is part of my local OTP install, elib1_misc is my > own library installed > in ~/code/elib2_1/ebin and misultin is an imported project stored in > ~/imports/misultin > I also know that my paths etc are setup so this code will work when I > run the program. > > The problem is the *nobody else* knows this. > > I could tell the system like this: > > -module(foo). > > -location(lists, > "https://github.com/erlang/otp/blob/dev/lib/stdlib/src/lists.erl"). > -location(elib1_misc, > "https://github.com/joearms/elib1/blob/master/lib/src/elib1_misc.erl"). > -location(misultin, > "https://github.com/ostinelli/misultin/blob/master/src/misultin.erl"). > > ... > > > The location annotation give a *definitive source" for the module I > am using in the module > > What could you do with this information? > > Answer - a lot - for starters > > - automatically check for "latest versions" of libraries > download them when they change > - provide "who uses my code" feedback to the authors of the code > - publish (globally) lists of "definitive" versions of code > - recursive track and code dependencies (What do I mean) > when my system discovers that I use misultin - it > downloads misultin.erl > misultin.erl will have location dependencies which I can > follow, thus the libraries > that misultin calls can be fetched. > - automate code loading > > Most often this kind of "additional" information is kept "outside > the program" by strictly > annotating the program with the location dependencies we bring this > information *into* the program > in a form where it cannot be detached from the source code. > > Comments? > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Tue Sep 13 21:53:31 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 13 Sep 2011 21:53:31 +0200 Subject: [erlang-questions] where did my code come from? In-Reply-To: <36F7D21B-B7C7-4CCF-A64D-F0E3543BEF3C@nexttolast.com> References: <36F7D21B-B7C7-4CCF-A64D-F0E3543BEF3C@nexttolast.com> Message-ID: On Tue, Sep 13, 2011 at 9:44 PM, David Goehrig wrote: > This is basically how most real world JavaScript applications already work, so it simply falls to building a nice distributed registry that can be universally queried. I was thinking of statically using the information to fetch and update the code. But you could do it dynamically by hacking the code loader and caching the results. I guess you could just say -location(mymod,"http://ww.a.b/foo/mymod.erl") and use a regular web server to serve up the code and cache the result. To implement this you'd just need a parse transform and a small hack to error_handler.erl and code.erl I would do this myself but I'm off on holiday tomorrow with no computer /Joe > > Erlang txt fields in DNS anyone? > > Also since rebar already allows you to specify your dependencies as git tags, we already have a system that will automatically pull the specific source (or latest if unspecified) and build your application. > > Dave > > -=-=- dave@REDACTED -=-=- > > On Sep 13, 2011, at 3:19 AM, Joe Armstrong wrote: > >> I had an idea on my way to work ... >> >> When you write code, you have *implicit* knowledge of where the >> external code comes from. >> >> When I write the following: >> >> ? ? -module(foo). >> ? ? ?... >> ? ? start() -> >> ? ? ? ? ?X = lists:reverse(...), >> ? ? ? ? ?Y = elib1_misc:zap(...) >> ? ? ? ? ?Z = misultin:request(...) >> ? ? ... >> >> I "know" that lists is part of my local OTP install, elib1_misc is my >> own library installed >> in ~/code/elib2_1/ebin and misultin is an imported project stored in >> ~/imports/misultin >> I also know that my paths etc are setup so this code will work when I >> run the program. >> >> The problem is the *nobody else* knows this. >> >> I could tell the system like this: >> >> ? ?-module(foo). >> >> ? ?-location(lists, >> "https://github.com/erlang/otp/blob/dev/lib/stdlib/src/lists.erl"). >> ? ?-location(elib1_misc, >> "https://github.com/joearms/elib1/blob/master/lib/src/elib1_misc.erl"). >> ? ?-location(misultin, >> "https://github.com/ostinelli/misultin/blob/master/src/misultin.erl"). >> >> ? ?... >> >> >> ? The location annotation give a *definitive source" for the module I >> am using in the module >> >> ? ?What could you do with this information? >> >> ? ?Answer - a lot - for starters >> >> ? ? ? ? - automatically check for "latest versions" of libraries >> download them when they change >> ? ? ? ? - provide "who uses my code" feedback to the authors of the code >> ? ? ? ? - publish (globally) lists of "definitive" versions of code >> ? ? ? ? - recursive track and code dependencies (What do I mean) >> ? ? ? ? ? ?when my system discovers that I use misultin - it >> downloads misultin.erl >> ? ? ? ? ? ?misultin.erl will have location dependencies which I can >> follow, thus the libraries >> ? ? ? ? ? ?that misultin calls can be fetched. >> ? ? ? ? - automate code loading >> >> ? ?Most often this kind of "additional" information is kept "outside >> the program" by strictly >> annotating the program with the location dependencies we bring this >> information *into* the program >> in a form where it cannot be detached from the source code. >> >> ? ?Comments? >> >> ? /Joe >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From masklinn@REDACTED Tue Sep 13 21:55:36 2011 From: masklinn@REDACTED (Masklinn) Date: Tue, 13 Sep 2011 21:55:36 +0200 Subject: [erlang-questions] where did my code come from? In-Reply-To: <36F7D21B-B7C7-4CCF-A64D-F0E3543BEF3C@nexttolast.com> References: <36F7D21B-B7C7-4CCF-A64D-F0E3543BEF3C@nexttolast.com> Message-ID: <70747CCA-9D8E-48C0-9816-097B40D27C53@masklinn.net> On 2011-09-13, at 21:44 , David Goehrig wrote: > This is basically how most real world JavaScript applications already work in-browser, I don't know any server-side javascript implementation which fetches modules over HTTP when importing by default. In fact, the CommonJS "require" spec does not allow allow the colon character at all. > so it simply falls to building a nice distributed registry that can be universally queried. Regardless of remote-loading (which would ideally require signed everything), finally having an easy to use and community-accepted CPAN.pm or gem-type tool for Erlang would be quite nice. Even linked to a pypi-type centralized repository. From jakob@REDACTED Tue Sep 13 22:28:56 2011 From: jakob@REDACTED (Jakob Praher) Date: Tue, 13 Sep 2011 22:28:56 +0200 Subject: [erlang-questions] where did my code come from? In-Reply-To: References: <36F7D21B-B7C7-4CCF-A64D-F0E3543BEF3C@nexttolast.com> Message-ID: <4E6FBD08.6040809@praher.info> Am 13.09.11 21:53, schrieb Joe Armstrong: > On Tue, Sep 13, 2011 at 9:44 PM, David Goehrig wrote: >> This is basically how most real world JavaScript applications already work, so it simply falls to building a nice distributed registry that can be universally queried. > I was thinking of statically using the information to fetch and > update the code. But you could do it dynamically by hacking the > code loader and caching the results. I guess you could just say > > -location(mymod,"http://ww.a.b/foo/mymod.erl") Sorry for my ignorance, I have not written much erlang (unfortunately). But just to get this concept -location is a clause directing the laoder, right? So you have three entities: a) the exporter site (exporting mymod) b) the importer site (depending on mymod) c) the loader (knowing where to find mymod) Where a) and b) should be transparent of c). So you can plug in different loders and load code from different sources. Where is code depended by mymod loaded then? In Javascript (CommonJS module system) the trick is to interpret a code site as a closure and pass in a function called require as a parameter (for hygenic reasons) when evaluating the code loaded from a location, that has a certain lookup rule for again loading code (relative to that site). What about security issues - can you trust www.a.b even if it depends on some internal modules (that are not sandboxed)? Where would such a -location annotation typically be placed? Can you give me an example? Since you have more deployments than libraries and applications there needs to be some wiring. Cheers, Jakob From fritchie@REDACTED Tue Sep 13 23:29:20 2011 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Tue, 13 Sep 2011 16:29:20 -0500 Subject: [erlang-questions] calling FUN from NIF In-Reply-To: Message of "Tue, 13 Sep 2011 13:55:37 -0000." Message-ID: <54522.1315949360@snookles.snookles.com> Paul Davis wrote: pd> On the other hand, there are planned enhancements to the NIF api to pd> allow such awesomeness. See Rickard's talk at [1]. If you can't wait for that kind of awesomeness, I *know* that it's possible to do what you're looking for via a driver instead of a NIF. (Because I saw some of Tony Rogvall's driver code from a decade ago to it. :-) With a bit of enginuity, perhaps the technique could also be applied to NIFs. Note: this is just a sketch from memory poisoned by a lot of intervening time. Also note that the Erlang process that makes the initial call to the driver is the samem process that executes the driver's Erlang callback function, i.e., PID . This is a mash-up attempt to show that time flows as you read from top to bottom. Note that the way this is sketched, the driver can request more than one callback be executed on the Erlang side of the world. Erlang code C code in driver Erlang process Deep in the virtual machine ---------------------- --------------------------- drv_command(Port, SerializedCommandData) driver's struct erl_drv_entry.outputv function(): calculate stuff, decides that it needs to call an Erlang callback function, sends reply to request a callback function call. Store intermediate state in driver's private data. %% Code path on first call: %% driver sends us ?WantCallback get_port_answer(Port) -> receive {Port, {data, [Response|Rest]=_Data}} -> case Response of ?WantCallback -> {M, F, As} = decode_callback_request(Rest), Answer = erlang:apply(M, F, As), % Encode answer and send it back to % driver via drv_command() or whatever get_port_answer(Port); ?FinalAnswer -> % Decode and return to caller decode_final_answer(Rest) end end. Erlang send us an encoded callback reply. Decode the reply, fetch our intermediate state out of my private data, and continue calculating. [...] Ah, I'm done, so I reply with my final answer. %% This time, get_port_answer() %% will receive a message with %% ?FinalAnswer tag. -Scott From joe@REDACTED Tue Sep 13 23:32:37 2011 From: joe@REDACTED (Joe Williams) Date: Tue, 13 Sep 2011 14:32:37 -0700 Subject: [erlang-questions] Funargs: Ruby-like blocks for Erlang In-Reply-To: References: Message-ID: <771F610E4F1B42EB9627A71C1B83D0A4@joetify.com> > Xml:posts do > lists:each(Posts) do |Post| > Xml:post do > Xml:title(Post#post.title), > Xml:body(Post#post.body), > Xml:published_at(Post#post.published_at), > Xml:comments do > lists:each(Post#post.comments) do |Comment| > Xml.comment do > Xml:body(Comment#comment.body) > end > end > end > end > end > end Just happened across this thread, late to the party. Perhaps you could write your patch to do "ennnnnnd" rather than six "ends". -Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave@REDACTED Wed Sep 14 01:29:29 2011 From: dave@REDACTED (David Goehrig) Date: Tue, 13 Sep 2011 19:29:29 -0400 Subject: [erlang-questions] where did my code come from? In-Reply-To: <4E6FBD08.6040809@praher.info> References: <36F7D21B-B7C7-4CCF-A64D-F0E3543BEF3C@nexttolast.com> <4E6FBD08.6040809@praher.info> Message-ID: On Sep 13, 2011, at 4:28 PM, Jakob Praher wrote: > What about security issues - can you trust www.a.b even if it depends on > some internal modules (that are not sandboxed)? Do you trust the code you download via cpan, gems, easy_install, npm, yum, apt, or your favorite module system here? Even if you have PGP signatures and programmatically check them, you still implicitly trust the developer whose code you are using. The best you can do is lock down to an assumed good state and save the hash. My suspicion is in the long term systems will continue to be so complex no one will be able to verify trust, and we'll adapt by living with a certain level of insecurity and abuse. Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Sep 14 02:12:44 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 14 Sep 2011 02:12:44 +0200 Subject: [erlang-questions] where did my code come from? In-Reply-To: References: <36F7D21B-B7C7-4CCF-A64D-F0E3543BEF3C@nexttolast.com> <4E6FBD08.6040809@praher.info> Message-ID: On Wed, Sep 14, 2011 at 01:29, David Goehrig wrote: > My suspicion is in the long term systems will continue to be so complex no > one will be able to verify trust, and we'll adapt by living with a certain > level of insecurity and abuse. Well, there is one way out, but it is not particularly implemented, nor easy to pull off. George Necula has written a paper on proof-carrying-code. Here is the trick. We start by setting up a security policy. The SP has to be formally specified since we want a machine to check it. Necula does this for out-of-bounds memory checks, but you can in principle do it for any security predicate you want, as long as you are able to make a formulation of it formally. When we all agree on an SP, the next step can begin. The author of the code is obliged to pass two things. The code C, and a proof P which states that the code obeys the SP. From C a set of verification conditions are extracted which must be true for SP to hold. The proof P is a list of how to show these conditions to be true. How the author comes up with P is interesting. Basically you can have the compiler infer everything about P you need. In a case the compiler can't prove something, for a memory access it won't be able to prove it is not out-of-bounds, the compiler inserts enough checking so it is able to carry out the proof. This is mostly mechanical. The proof is output as an Oracle stream which allows us to compress the size of the proof down. The idea is that the oracle stream provides a set of hints about what to do if an automated proof checker gets stuck. The user of the code gets C and P. He then proceed by using a verification condition generator on C which tells him what conditions need to be true - the same generator which the author uses. And he uses P on the conditions to check that the proof actually is the right one. If the proof goes through, he may use the code. The beauty of the approach is that the user of the code does not have to trust the author. They only have to agree on SP. If the Code is altered, the Proof may or may not work. If it works, it is still safe. If not, we can reject the program. If the proof is altered, it may or may not work. if it works, it is still safe. If not, we can reject the proof (and thus the program). Now, it is a 10 year old idea. So why hasn't it seen more use? For one, it requires some compiler tech support. Then there is the whole part about SP, which can be pretty nasty to build up. My bet is that it has to incubate for more years, until our knowledge about mechanical theorem proving is better. But a dreamy vision it is! It would allow us to distribute code over the internet, and execute code from untrusted sources, just by checking that the proof they provide is correct. -- J. From jeffm@REDACTED Wed Sep 14 03:18:20 2011 From: jeffm@REDACTED (jm) Date: Wed, 14 Sep 2011 11:18:20 +1000 Subject: [erlang-questions] where did my code come from? In-Reply-To: <36F7D21B-B7C7-4CCF-A64D-F0E3543BEF3C@nexttolast.com> References: <36F7D21B-B7C7-4CCF-A64D-F0E3543BEF3C@nexttolast.com> Message-ID: <4E7000DC.4010609@ghostgun.com> You appear to be the first person in this thread that has addressed the question that I've been thinking. "How to find the repository?" Most, so far, have used fixed URLs, but as we all know these tend to be out of date on a regular basis. The larger the project, the more dependency, the more you desperately need this thing to work, the more pushed for time you are, the more likely you are to find that the site which has the module you need is down or no longer exists. Here's roughly what I've been think as I read this thread, Have a file which lists mirror sites from which you can pull a list of repositories. Much like the /etc/resolve.conf file for dns. Have a global conf file to tune repo/lookup site selection and other parameters similar to debian package management. Have a project conf file to fine tune the global settings. Have a per project file which lists app/module/fun dependances. This could be auto-generated using tools. As part of this a tool to make it easy to system packages (eg rpm, deb, open package, etc). So far I don't see a need for a -location compiler directive at the source file level. May be I'm missing something. However, the app/module/fun dependance list would need a better signature than app:module:fun/count or version. It's would need some form of crypto hash. How would you calculate such a hash? Having a resolution down to the function level would allow you to know if you can safely upgrade to a future (or past) module version or not automatically even before compiling. It seems that there are projects out there that do A+C or A+B, but none that do the full raft of things. Finally, the key I think is to eliminate single points of failure such as being reliant on one key website. A P2P module distribution system would be great. Jeff. On 14/09/11 5:44 AM, David Goehrig wrote: > This is basically how most real world JavaScript applications already work, so it simply falls to building a nice distributed registry that can be universally queried. > > Erlang txt fields in DNS anyone? > > Also since rebar already allows you to specify your dependencies as git tags, we already have a system that will automatically pull the specific source (or latest if unspecified) and build your application. > > Dave > From jeffm@REDACTED Wed Sep 14 04:01:35 2011 From: jeffm@REDACTED (jm) Date: Wed, 14 Sep 2011 12:01:35 +1000 Subject: [erlang-questions] Rough thought on a P2P package distribution model for Erlang Message-ID: <4E700AFF.7090309@ghostgun.com> In my previous email I said that I thought P2P package distribution system would be a good idea. This was due to it elimination the single points of failure with relying of the future of websites. There are a number of problems with using a P2P module. Chief among these are how to get packages into the system and how to know that these packages are trust worthy. With that in mind here's some rough thoughts on a P2P module repository for Erlang: Publisher: the person who maintains the package. Typically, the author of the module being published. Node: a server which is a member of the P2P module repository system Indexer: a person who creates an index of packages that they say meets some criteria ie, they vouch for the packages. Administrator: the person who looks after a node The process would work something like this, Some one writes a wonderful module the one everyone has been waiting for. Either the original author or someone on their behalf packages it up. The Publisher then makes this publicly available on a website or through git/mercurial/etc The Publisher notifies one or more indexers. Each Indexers check that the package meets their criteria. The Indexer then injects the package into the p2p distribution system along with an updated signed versioned index file. This index file lists which packages the Indexer has verified and the cryptographic hash for each package. The Administrators of other nodes select which Indexers they wish to follow and keep copies each Indexers public key (obtained out of band). The Nodes then replicates the index file of each Indexer of interest and the packages listed by those index files. These nodes then make this information available of ftp/http/p2p or other means to other nodes and end developers. Using an Indexer has a couple of advantages: 1) it eliminates the need for everyone to have certificates. Making the system cleaner to use and lowering the barrier to entry of package maintainers allow them to easily submit their work without distraction. 2) It maintains a concept similar to existing repositories with which people are familiar. This makes it easy for people to bring up and maintain additional nodes. It also means that the number of people that have to wade though all the packages out there is reduced down to the Indexer. You simple select the Indexer who has a package criteria which reflects your own. This is separate what packages are and who Erlang handles dependances. This is merely a distribution model. Excuse the broad description I merely intend this to give people ideas. Jeff. From bengt.kleberg@REDACTED Wed Sep 14 08:46:16 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Wed, 14 Sep 2011 08:46:16 +0200 Subject: [erlang-questions] where did my code come from? In-Reply-To: References: Message-ID: <1315982776.5065.2.camel@seasc1137> Greetings, This email mentions import renaming. Does that include solving the problem that two modules I want to load have the same name? bengt On Tue, 2011-09-13 at 19:21 +0200, Jesper Louis Andersen wrote: > On Tue, Sep 13, 2011 at 09:19, Joe Armstrong wrote: > > -location(lists, > > "https://github.com/erlang/otp/blob/dev/lib/stdlib/src/lists.erl"). > > > Comments? > > I like the idea quite a lot. In a modern world, the whole notion of "a > module lives on a single file system" is wrong. It is rather the case > that a module lives its life behind an URI, is subject to change and > that locally we are more concerned about caching and having the > correct version. While you are at it, your -location() notion also > allows for import renaming, which is something that is needed dearly > in Erlang. Who says a module is in a file? It could be in an HTTP > stream, or sent over a websocket or XMPP even. There is no reason to > tie it to a specific carrier. > > As for security, security is to be had by forming releases. When you > create a release, you can use signatures to verify the authenticity of > modules from module authors. The signature is even present easily in > git with a merkle-tree-like construction. The alternative, which is > probably some 20-30 years out in the future is Proof-carrying-code > where the code itself provides a proof of what it does - so lets stick > with signatures for now. > > Releases also provide a stable notion of the system. It is not subject > to change once formed. > > Then there is the part of self-documenting the dependencies, which I > also really like. In the Standard ML community, there is a nice MLB > concept where you can form bundles of software in the large. You can > say "This application consists of these files, and it exports these > modules and functors under these (renamed) names". Upon import, you > can also rename the API names to avoid clashes locally. The standard > library is just a MLB application reference. The older standard > library can be had by another reference. > > Essentially we have a notion of lexically scoped module availability > and exportation in the MLB files. But they never considered a world > where everything is just an URI. Which is why I like the location > idea. It incorporates that notion, and it is a tighter coupling than > an external MLB file, which I am not sure is the way to go. In any > case, if one were to revamp the module system of Erlang, -location() > should be a consideration. > > From norton@REDACTED Wed Sep 14 11:21:43 2011 From: norton@REDACTED (Joseph Norton) Date: Wed, 14 Sep 2011 18:21:43 +0900 Subject: [erlang-questions] Obsolete exported functions file:raw_{read, write}_file_info/2 - why? Message-ID: <953BD0D9-D0E2-4033-88A9-E4254B3A5379@lovely.email.ne.jp> I'm curious if someone happens to know why the file:raw_read_file_info/1 and file:raw_read_file_info/2 methods were made obsolete? Joseph Norton norton@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwertymaniac@REDACTED Wed Sep 14 13:01:01 2011 From: qwertymaniac@REDACTED (Harsh J) Date: Wed, 14 Sep 2011 16:31:01 +0530 Subject: [erlang-questions] Obsolete exported functions file:raw_{read, write}_file_info/2 - why? In-Reply-To: <953BD0D9-D0E2-4033-88A9-E4254B3A5379@lovely.email.ne.jp> References: <953BD0D9-D0E2-4033-88A9-E4254B3A5379@lovely.email.ne.jp> Message-ID: Am not very sure of this, but I think Bjorn's answer about file:rawopen applies to this as well: http://erlang.2086793.n4.nabble.com/Problem-in-file-rawopen-2-td2103932.html On Wed, Sep 14, 2011 at 2:51 PM, Joseph Norton wrote: > > I'm curious if someone happens to know why the file:raw_read_file_info/1 and > file:raw_read_file_info/2 methods were made obsolete? > Joseph Norton > norton@REDACTED > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Harsh J http://harshj.com From max.lapshin@REDACTED Wed Sep 14 14:23:51 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 14 Sep 2011 16:23:51 +0400 Subject: [erlang-questions] Erlang and beam-based scripting languages. In-Reply-To: References: Message-ID: At least I've ended with such solution: https://github.com/maxlapshin/elixir/blob/master/src/elixir_tracker.erl it is just like mochiweb reloader, that looks for changed scripts and create proxy modules for them. From s.j.thompson@REDACTED Wed Sep 14 16:08:58 2011 From: s.j.thompson@REDACTED (Simon Thompson) Date: Wed, 14 Sep 2011 15:08:58 +0100 Subject: [erlang-questions] PEPM Final call for papers Message-ID: Final Call For Papers Paper submission deadline: Mon, October 10, 2011, 23:59, GMT ACM SIGPLAN 2012 Workshop on Partial Evaluation and Program Manipulation January 23-24, 2012. Philadelphia, Pennsylvania, USA (co-located with POPL'12) http://www.program-transformation.org/PEPM12 The PEPM Symposium/Workshop series aims to bring together researchers and practitioners working in the broad area of program transformation, which spans from refactoring, partial evaluation, supercompilation, fusion and other metaprogramming to model-driven development, program analyses including termination, inductive programming, program generation and applications of machine learning and probabilistic search. PEPM focuses on techniques, supporting theory, tools, and applications of the analysis and manipulation of programs. Each technique or tool of program manipulation should have a clear, although perhaps informal, statement of desired properties, along with an argument how these properties could be achieved. Topics of interest for PEPM'12 include, but are not limited to: - Program and model manipulation techniques such as: supercompilation, partial evaluation, fusion, on-the-fly program adaptation, active libraries, program inversion, slicing, symbolic execution, refactoring, decompilation, and obfuscation. - Program analysis techniques that are used to drive program/model manipulation such as: abstract interpretation, termination checking, binding-time analysis, constraint solving, type systems, automated testing and test case generation. - Techniques that treat programs/models as data objects including metaprogramming, generative programming, embedded domain-specific languages, program synthesis by sketching and inductive programming, staged computation, and model-driven program generation and transformation. - Application of the above techniques including case studies of program manipulation in real-world (industrial, open-source) projects and software development processes, descriptions of robust tools capable of effectively handling realistic applications, benchmarking. Examples of application domains include legacy program understanding and transformation, DSL implementations, visual languages and end-user programming, scientific computing, middleware frameworks and infrastructure needed for distributed and web-based applications, resource-limited computation, and security. To maintain the dynamic and interactive nature of PEPM, we will continue the category of `short papers' for tool demonstrations and for presentations of exciting if not fully polished research, and of interesting academic, industrial and open-source applications that are new or unfamiliar. Student attendants with accepted papers can apply for a SIGPLAN PAC grant to help cover travel expenses and other support. All accepted papers, short papers included, will appear in formal proceedings published by ACM Press and will be included in the ACM Digital Library. Selected papers may later on be invited for a journal special issue dedicated to PEPM'12. Submission Categories and Guidelines Authors are strongly encouraged to consult the advice for authoring research papers and tool papers before submitting. The PC Chairs welcome any inquiries about the authoring advice. Regular research papers must not exceed 10 pages in ACM Proceedings style. Short papers are up to 4 pages in ACM Proceedings style. Authors of tool demonstration proposals are expected to present a live demonstration of the described tool at the workshop (tool papers should include an additional appendix of up to 6 extra pages giving the outline, screenshots, examples, etc. to indicate the content of the proposed live demo at the workshop). Important Dates - Paper submission: Mon, October 10, 2011, 23:59, GMT - Author notification: Tue, November 8, 2011 - Workshop: Mon-Tue, January 23-24, 2012 Invited Speakers - Markus Pueschel (ETH Zurich, Switzerland) - Martin Berger (University of Sussex, UK) Program Chairs - Oleg Kiselyov (Monterey, CA, USA) - Simon Thompson (University of Kent, UK) Program Committee Members - Emilie Balland (INRIA, France) - Ewen Denney (NASA Ames Research Center, USA) - Martin Erwig (Oregon State University, USA) - Sebastian Fischer (National Institute of Informatics, Japan) - Lidia Fuentes (Universidad de Malaga, Spain) - John Gallagher (Roskilde University, Denmark and IMDEA Software, Spain) - Dave Herman (Mozilla Research, USA) - Stefan Holdermans (Vector Fabrics, the Netherlands) - Christian Kaestner (University of Marburg, Germany) - Emanuel Kitzelmann (International Computer Science Institute, USA) - Andrei Klimov (Keldysh Institute of Applied Mathematics, Russian Academy of Sciences) - Shin-Cheng Mu (Academia Sinica, Taiwan) - Alberto Pardo (Universidad de la Repu'blica, Uruguay) - Kostis Sagonas (Uppsala University, Sweden and National Technical University of Athens, Greece) - Anthony M. Sloane (Macquarie University, Australia) - Armando Solar-Lezama (MIT, USA) - Aaron Stump (The University of Iowa, USA) - Kohei Suenaga (University of Kyoto, Japan) - Eric Van Wyk (University of Minnesota, USA) - Kwangkeun Yi (Seoul National University, Korea) Simon Thompson | Professor of Logic and Computation School of Computing | University of Kent | Canterbury, CT2 7NF, UK s.j.thompson@REDACTED | M +44 7986 085754 | W www.cs.kent.ac.uk/~sjt From ericbmerritt@REDACTED Wed Sep 14 17:24:23 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Wed, 14 Sep 2011 10:24:23 -0500 Subject: [erlang-questions] where did my code come from? In-Reply-To: References: Message-ID: > > When I write the following: > > ? ? -module(foo). > ? ? ?... > ? ? start() -> > ? ? ? ? ?X = lists:reverse(...), > ? ? ? ? ?Y = elib1_misc:zap(...) > ? ? ? ? ?Z = misultin:request(...) > ? ? ... > > I "know" that lists is part of my local OTP install, elib1_misc is my > own library installed > in ~/code/elib2_1/ebin and misultin is an imported project stored in > ~/imports/misultin > I also know that my paths etc are setup so this code will work when I > run the program. > > The problem is the *nobody else* knows this. > > I could tell the system like this: > > ? ?-module(foo). > > ? ?-location(lists, > "https://github.com/erlang/otp/blob/dev/lib/stdlib/src/lists.erl"). > ? ?-location(elib1_misc, > "https://github.com/joearms/elib1/blob/master/lib/src/elib1_misc.erl"). > ? ?-location(misultin, > "https://github.com/ostinelli/misultin/blob/master/src/misultin.erl"). I would say this is a build system issue more then anything else. In fact, I would say this is one of the major points of a build system. Knowing how to assemble your project and where things in your project live. It could be that the current build systems do a poor job of advertising this information though. > ? ?... > > > ? The location annotation give a *definitive source" for the module I > am using in the module > > ? ?What could you do with this information? > > ? ?Answer - a lot - for starters > > ? ? ? ? - automatically check for "latest versions" of libraries > download them when they change On compile or at run time? I suspect that changing versions of a dependency without running associated testing code is going to result in some interesting and perhaps hard to resolve errors. It could be possible if the libraries strictly follow semantic versioning and the 'updater', understands patch and minor versions well. However, relying on a library maintainer to follow a spec without exception is probably not a good idea. As always the tests tell you the truth of the viability of a new arrangement of your system (regardless of the new arrangement). It could be that auto upgrades happen and all the tests are run and pass before 'cementing' the new dependencies, I suppose. > ? ? ? ? - provide "who uses my code" feedback to the authors of the code > ? ? ? ? - publish (globally) lists of "definitive" versions of code > ? ? ? ? - recursive track and code dependencies (What do I mean) > ? ? ? ? ? ?when my system discovers that I use misultin - it > downloads misultin.erl > ? ? ? ? ? ?misultin.erl will have location dependencies which I can > follow, thus the libraries > ? ? ? ? ? ?that misultin calls can be fetched. > ? ? ? ? - automate code loading This really does sound much more like a build system then something at the language level. Though, I suppose you could build the build system into the platform. It could also just be poor semantics. It could be that this is what build systems currently do but there is no reason for that to be in a separate system. > ? ?Most often this kind of "additional" information is kept "outside > the program" by strictly > annotating the program with the location dependencies we bring this > information *into* the program > in a form where it cannot be detached from the source code. You could bring this into the system without actually having to provide additional annotations to the Erlang source code. Erlang or at least OTP has quite a lot of metadata that is required to run. These are first class members of the project/app and are part of the source code though not part of the Erlang code. I suspect adding a bit of required metadata could solve this problem without having redundant declarations in the source code itself. > > ? ?Comments? > > ? /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From jesper.louis.andersen@REDACTED Wed Sep 14 18:26:28 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 14 Sep 2011 18:26:28 +0200 Subject: [erlang-questions] where did my code come from? In-Reply-To: <1315982776.5065.2.camel@seasc1137> References: <1315982776.5065.2.camel@seasc1137> Message-ID: On Wed, Sep 14, 2011 at 08:46, Bengt Kleberg wrote: > Greetings, > > This email mentions import renaming. Does that include solving the > problem that two modules I want to load have the same name? Well it depends, You can rename a module when you import it, or you can rename a module when you export it. The latter has no meaning in Erlang. All modules are by default exported under the name they were written with and there is full visibility. So in Erlang, only import renaming makes (easy) sense. The idea is then, that you can write an import declaration with renaming to make a more digestible name locally in an application. But this doesn't allow for two modules with the same name in the Erlang Runtime. It is silly I have to write etorrent_event:notify/2 when in my application, event:notify/2 would be as good. On the other hand, it doesn't save me much in the typing department, and it is another level of proxying my brain has to process. Another way is to allow modules of the same name, but add some qualifier when referring to them. So you can say "The module in the stdlib application" when you import them. I initally wanted a more expressive module system because I saw it as necessary for being able to evolve the standard library, while keeping the old systems on track. The idea was to bulid an alternative stdlib next to the current one and then prefer it for applications. I've however come to the conclusion it would be better to add a tool like Golangs "gofix". Gofix is an indentation-aware syntax fixer and library upgrader. The idea is that when the basis library is changed, gofix can be invoked to automatically bring programs up to date as well. In many cases the application is completely mechanic and can be used to ease the burden on programmers when using old versions of the code. In Erlang the basis is already there. One just has to consider Wrangler. With an upgrade tool, we could begin evolving the stdlib to make it more streamlined and consistent. Since Erlang has no types, we can't easily fix the order of function arguments, but many other cases can be fixed - and that while providing backwards compatibility through mechanical upgrade of old source code. It also avoids the issue with maintaining more than one standard library. -- J. From fritchie@REDACTED Wed Sep 14 18:42:37 2011 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 14 Sep 2011 11:42:37 -0500 Subject: [erlang-questions] Obsolete exported functions file:raw_{read, write}_file_info/2 - why? In-Reply-To: Message of "Wed, 14 Sep 2011 18:21:43 +0900." <953BD0D9-D0E2-4033-88A9-E4254B3A5379@lovely.email.ne.jp> Message-ID: <19976.1316018557@snookles.snookles.com> Joseph Norton wrote: jn> I'm curious if someone happens to know why the jn> file:raw_read_file_info/1 and file:raw_read_file_info/2 methods were jn> made obsolete? Joe, I stumbled across those functions while putting DTrace probes into the efile_drv driver. I thought, hey, those would be quite useful. I recommend reading the file.erl source. It's quite instructive to see how many file I/O functions are redirected to the 'file_server_2' process. For file I/O-intensive applications (e.g. Hibari and Riak I know, CouchDB and RabbitMQ I'd guess), having all calls to(*) file:read_file_info/1 serialized by the file server process is a source of latency that we (DB authors) may desire to live without. Having planted bugs in prim_file.erl accidentally, it is also quite instructive to see how many different ways the VM's bootstrap process can be broken. So I now understand reluctance to deprecate functions that may be necessary at some weird situation when booting. However, there may come a time when some of us submit a patch to expose *more* raw file I/O functions in file.erl, not fewer. -Scott (*) Or file:delete/1 or file:rename/2 or ... From erlangy@REDACTED Wed Sep 14 18:57:16 2011 From: erlangy@REDACTED (erlang) Date: Wed, 14 Sep 2011 18:57:16 +0200 Subject: [erlang-questions] Obsolete exported functions file:raw_{read, write}_file_info/2 - why? In-Reply-To: <19976.1316018557@snookles.snookles.com> References: <19976.1316018557@snookles.snookles.com> Message-ID: <09AC9E70-E4A4-4EEF-9ADD-B89C188B427F@gmail.com> > I recommend reading the file.erl source. It's quite instructive to see > how many file I/O functions are redirected to the 'file_server_2' > process. For file I/O-intensive applications (e.g. Hibari and Riak I > know, CouchDB and RabbitMQ I'd guess), having all calls to(*) > file:read_file_info/1 serialized by the file server process is a source > of latency that we (DB authors) may desire to live without. How did you proceed to avoid these calls and reduce latency Scott? Any hints? From pfisher@REDACTED Wed Sep 14 19:22:51 2011 From: pfisher@REDACTED (Paul Fisher) Date: Wed, 14 Sep 2011 12:22:51 -0500 Subject: [erlang-questions] Obsolete exported functions file:raw_{read, write}_file_info/2 - why? In-Reply-To: <19976.1316018557@snookles.snookles.com> References: <19976.1316018557@snookles.snookles.com> Message-ID: Agreed. For us, file_server_2 represented a significant bottleneck on our 8 core nodes (e.g. delete/2, rename/2, etc.) We created a module that used the low-level prim_file interface directly to get around the bottlenecks. We have a bounded number of processes that perform file operations, so we ended up with the following basic approach in this module: -spec delete(Name :: file:name()) -> 'ok' | {'error', file:posix()}. delete(Name) -> Port = get_port(), prim_file:delete( Port, Name ). ? get_port() -> case get( ?MODULE ) of undefined -> Port = case prim_file:start() of {ok, Port0} -> Port0; {error, Reason} -> exit( {?MODULE, failed_to_create_prim_file, Reason} ) end, put( ?MODULE, Port ), Port; Port -> Port end. There was some discussion a few months ago about someone experimenting/testing an alternative file implementation (NIF-based, I believe), but I forget the source of the comment. On Sep 14, 2011, at 11:42 AM, Scott Lystig Fritchie wrote: > Joseph Norton wrote: > > jn> I'm curious if someone happens to know why the > jn> file:raw_read_file_info/1 and file:raw_read_file_info/2 methods were > jn> made obsolete? > > Joe, I stumbled across those functions while putting DTrace probes into > the efile_drv driver. I thought, hey, those would be quite useful. > > I recommend reading the file.erl source. It's quite instructive to see > how many file I/O functions are redirected to the 'file_server_2' > process. For file I/O-intensive applications (e.g. Hibari and Riak I > know, CouchDB and RabbitMQ I'd guess), having all calls to(*) > file:read_file_info/1 serialized by the file server process is a source > of latency that we (DB authors) may desire to live without. > > Having planted bugs in prim_file.erl accidentally, it is also quite > instructive to see how many different ways the VM's bootstrap process > can be broken. So I now understand reluctance to deprecate functions > that may be necessary at some weird situation when booting. However, > there may come a time when some of us submit a patch to expose *more* > raw file I/O functions in file.erl, not fewer. > > -Scott > > (*) Or file:delete/1 or file:rename/2 or ... > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- paul director, platform services alertlogic, inc. 713-484-8383 x2314 "When in doubt, use brute force." -- Ken Thompson From dave@REDACTED Wed Sep 14 19:35:55 2011 From: dave@REDACTED (David Goehrig) Date: Wed, 14 Sep 2011 13:35:55 -0400 Subject: [erlang-questions] where did my code come from? In-Reply-To: References: <36F7D21B-B7C7-4CCF-A64D-F0E3543BEF3C@nexttolast.com> <4E6FBD08.6040809@praher.info> Message-ID: On Sep 13, 2011, at 8:12 PM, Jesper Louis Andersen wrote: > George Necula has written a paper on > proof-carrying-code. Here is the trick. > I think PCC works only in theory, but would not work in reality. Here's the reason: cost To prove a system is correct must be cheaper than the cost of any damage done by a potential exploit. The cost of proving a system correct is also an up front cost that requires your entire tool chain and infrastructure also be tested. Since the cost of creating proofs for and validating those proof systems (replacing all existing infrastructure) exceeds the cost of not doing so, we are unlikely to ever see mass adoption. And since the risk of economic damage due failure/compromise is relatively small, and the opportunity cost associated with not going to market quite high, there is always a disincentive to prove/test your new systems before entering production. And so it is unlikely to ever be used outside of critical infrastructure and military application. Security is fundamentally an economic issue, and it rarely makes sense to devote resources to securing systems with short lifetimes. Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Wed Sep 14 19:37:36 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 14 Sep 2011 19:37:36 +0200 Subject: [erlang-questions] Obsolete exported functions file:raw_{read, write}_file_info/2 - why? In-Reply-To: <19976.1316018557@snookles.snookles.com> References: <953BD0D9-D0E2-4033-88A9-E4254B3A5379@lovely.email.ne.jp> <19976.1316018557@snookles.snookles.com> Message-ID: 2011/9/14, Scott Lystig Fritchie : > Joseph Norton wrote: > > jn> I'm curious if someone happens to know why the > jn> file:raw_read_file_info/1 and file:raw_read_file_info/2 methods were > jn> made obsolete? > > Joe, I stumbled across those functions while putting DTrace probes into > the efile_drv driver. I thought, hey, those would be quite useful. > > I recommend reading the file.erl source. It's quite instructive to see > how many file I/O functions are redirected to the 'file_server_2' > process. For file I/O-intensive applications (e.g. Hibari and Riak I > know, CouchDB and RabbitMQ I'd guess), having all calls to(*) > file:read_file_info/1 serialized by the file server process is a source > of latency that we (DB authors) may desire to live without. That's interesting. I was also chasing a performance problem a couple of weeks ago (the server crawled to a halt for a minute or two, the load of the Linux OS went over 50, then everything went back to normal) and noticed that the file_server process used a lot of CPU. My solution(?) was to randomize the jobs that would write to the disk, so the 30 processes tried not to write to the disk at the same time. From fritchie@REDACTED Wed Sep 14 23:17:03 2011 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 14 Sep 2011 16:17:03 -0500 Subject: [erlang-questions] Obsolete exported functions file:raw_{read, write}_file_info/2 - why? In-Reply-To: Message of "Wed, 14 Sep 2011 19:37:36 +0200." Message-ID: <35408.1316035023@snookles.snookles.com> Attila Rajmund Nohl wrote: arn> That's interesting. I was also chasing a performance problem a arn> couple of weeks ago (the server crawled to a halt for a minute or arn> two, the load of the Linux OS went over 50, then everything went arn> back to normal) and noticed that the file_server process used a lot arn> of CPU. My solution(?) was to randomize the jobs that would write arn> to the disk, so the 30 processes tried not to write to the disk at arn> the same time. Hrm, you didn't mention how you're writing to disk. If you're using file:writefile/2, that's another func that ends up going through the file_server_2. If you're opening file descriptors for each file (and opening them in 'raw' mode), then you probably should also be using the +A flag when starting the VM so that all computation by the VM won't be blocked by slow file I/O. And if you're already using separate file descriptors and the +A flag, simply doing too much I/O at once can be a Bad Idea. If your disk is overloaded (measure using "iostat -x 1" or equivalent), then all you can really do is wait(*). If you believe that your disk(s) are not yet saturated and that you believe that your bottleneck is the Erlang VM, it may be possible that you've got too much parallel I/O relative to the size of the +A I/O worker pool. For example, say that you use "+A 4" and (as mentioned above) have 50 file writes happening simultaneously. The code in efile_drv.c assigns a worker Pthread based on the Erlang port number(**). So you could have many ports' worth of I/O assigned to each worker Pthread, 12-13 on average if everything is sync'ed perfectly. In a pathological worst case, where you opened 4 ports and discarded every 3, you could have all 50 ports assigned to the same worker Pthread.(***) -Scott (*) Or fundamentally change the data you're writing and how you do it and when. (**) The assignment of port -> worker thread is *not* done by "first idle Pthread in the pool". (***) The lack of visibility into this part of the efile_drv.c driver is a motivating reason that got me active in patching the VM to add DTrace probes. See https://github.com/slfritchie/otp/tree/dtrace-experiment for source. Also, Dustin Stallings is hacking on DTrace, starting from a different direction, see https://github.com/dustin/otp/tree/dtrace. From fritchie@REDACTED Wed Sep 14 23:26:06 2011 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 14 Sep 2011 16:26:06 -0500 Subject: [erlang-questions] Obsolete exported functions file:raw_{read, write}_file_info/2 - why? In-Reply-To: Message of "Wed, 14 Sep 2011 16:17:03 CDT." <35408.1316035023@snookles.snookles.com> Message-ID: <35999.1316035566@snookles.snookles.com> Scott Lystig Fritchie wrote: slf> Also, Dustin Stallings is hacking on DTrace, starting from slf> a different direction, see slf> https://github.com/dustin/otp/tree/dtrace. Oops, I should have spelled "Sallings". Many apologies, Dustin! -Scott From lambdadmitry@REDACTED Wed Sep 14 20:17:12 2011 From: lambdadmitry@REDACTED (Dmitry Groshev) Date: Wed, 14 Sep 2011 11:17:12 -0700 (PDT) Subject: [erlang-questions] Strange behaviour of atoms in NIF Message-ID: <22176595.1348.1316024232112.JavaMail.geo-discussion-forums@yqih9> I'm writing a NIF-based circular array library and experiencing a strange behaviour of atoms in there. I have a NIF that should return an atom. It's fairly simple, but I get different atoms when calling this NIF from different places. Here is an issue with quite comprehensive explanation, please take a look at it at github: https://github.com/band115/ecirca/issues/31 I've tried to compare atoms that are returned by printf'ing them, but they seems identical in despite of obvious difference in shell. -------------- next part -------------- An HTML attachment was scrubbed... URL: From maxtqm@REDACTED Thu Sep 15 09:21:59 2011 From: maxtqm@REDACTED (max tan) Date: Thu, 15 Sep 2011 15:21:59 +0800 Subject: [erlang-questions] Is it possible to align binary's byte array to cache line boundary? Message-ID: It seems not supported in current VM, right? -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Thu Sep 15 09:39:38 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 15 Sep 2011 09:39:38 +0200 Subject: [erlang-questions] where did my code come from? In-Reply-To: References: <1315982776.5065.2.camel@seasc1137> Message-ID: <1316072378.7747.5.camel@seasc1137> In my mind the problem that module renaming should solve is "modules with the same name in the Erlang Runtime". The gofix tool, does it handle leagacy/binary code? bengt On Wed, 2011-09-14 at 18:26 +0200, Jesper Louis Andersen wrote: > On Wed, Sep 14, 2011 at 08:46, Bengt Kleberg wrote: > > Greetings, > > > > This email mentions import renaming. Does that include solving the > > problem that two modules I want to load have the same name? > > Well it depends, > > You can rename a module when you import it, or > you can rename a module when you export it. > > The latter has no meaning in Erlang. All modules are by default > exported under the name they were written with and there is full > visibility. So in Erlang, only import renaming makes (easy) sense. The > idea is then, that you can write an import declaration with renaming > to make a more digestible name locally in an application. But this > doesn't allow for two modules with the same name in the Erlang > Runtime. > > It is silly I have to write etorrent_event:notify/2 when in my > application, event:notify/2 would be as good. On the other hand, it > doesn't save me much in the typing department, and it is another level > of proxying my brain has to process. > > Another way is to allow modules of the same name, but add some > qualifier when referring to them. So you can say "The module in the > stdlib application" when you import them. > > I initally wanted a more expressive module system because I saw it as > necessary for being able to evolve the standard library, while keeping > the old systems on track. The idea was to bulid an alternative stdlib > next to the current one and then prefer it for applications. I've > however come to the conclusion it would be better to add a tool like > Golangs "gofix". Gofix is an indentation-aware syntax fixer and > library upgrader. The idea is that when the basis library is changed, > gofix can be invoked to automatically bring programs up to date as > well. In many cases the application is completely mechanic and can be > used to ease the burden on programmers when using old versions of the > code. > > In Erlang the basis is already there. One just has to consider > Wrangler. With an upgrade tool, we could begin evolving the stdlib to > make it more streamlined and consistent. Since Erlang has no types, we > can't easily fix the order of function arguments, but many other cases > can be fixed - and that while providing backwards compatibility > through mechanical upgrade of old source code. It also avoids the > issue with maintaining more than one standard library. > > > From bengt.kleberg@REDACTED Thu Sep 15 10:50:13 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 15 Sep 2011 10:50:13 +0200 Subject: [erlang-questions] beginner: dialyzer for the first time Message-ID: <1316076613.7747.32.camel@seasc1137> Greetings, When I want to use dialyzer on some Erlang source files for the first time I do: dialyzer --src *erl This gives me the error: Checking whether the PLT /home//.dialyzer_plt is up-to-date...dialyzer: exit 1 dialyzer: Could not find the PLT: /home//.dialyzer_plt Use the options: --build_plt to build a new PLT; or --add_to_plt to add to an existing PLT So I try: dialyzer --build_plt --src *erl This gives me the error: dialyzer: {dialyzer_error,"Byte code compiled with debug_info is needed to build the PLT"} [{dialyzer_options,check_output_plt,1}, {dialyzer_options,postprocess_opts,1}, {dialyzer_options,build,1}, {dialyzer_cl_parse,cl,1}, {dialyzer_cl_parse,start,0}, {dialyzer,plain_cl,0}, {init,start_it,1}, {init,start_em,1}] dialyzer: exit 1 What should I do next? The goal is to use dialyzer on some Erlang source files. bengt From ingela@REDACTED Thu Sep 15 10:55:43 2011 From: ingela@REDACTED (Ingela Andin) Date: Thu, 15 Sep 2011 10:55:43 +0200 Subject: [erlang-questions] ssl:setopts/2 and packet type httph In-Reply-To: <4b50db2306ff96e1325b2ec1fa92ed78@hellstrom.st> References: <4b50db2306ff96e1325b2ec1fa92ed78@hellstrom.st> Message-ID: Hi! This was discussed a bit of list as I happend to reply from the wrong email address (sorry about that). Anyway here is the conclution. Packet httph was invented for the statless erlang:decode_packet/3 function (used for example by the ssl application), howerver gen_tcp and ssl sockets will switch atomatically from http to httph after receiving the request/status line, hence we did not see the need to be able to set httph an ssl option. When using gen_tcp you happen to be able to use packet httph in an other context e.i. parsing trailers from chunked encoding, which is quite logical as they are defined to be entity-headers in the HTTP RFC. So what happens to work for gen_tcp we think is the desired behaviour and we will update ssl to handle it too. We wil also update the inet documentation. Regards Ingela Erlang/OTP team - Ericsson AB Den 7 september 2011 23:20 skrev Oscar Hellstr?m : > Hi, > > I'm trying to use lhttpc with OTP HEAD from github and ran in to troubles > when trying to call ssl:setopts(S, [{packet, httph}]). I started poking > around and found this in the inet manual: > "Note that the packet type httph is not needed when reading from a socket." > Apart from that I can't find any reference to httph any more. I'm not sure > what's meant by that but I guess I'm missing something here. I'm using > packet type httph to read trailers after chunked transfer coding. > > Is packet type httph intentionally removed, and will it be so from pure tcp > sockets also? > > Thanks > /Oscar > > -- > Oscar Hellstr?m, oscar@REDACTED > web: http://oscar.hellstrom.st > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From kostis@REDACTED Thu Sep 15 11:14:14 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 15 Sep 2011 12:14:14 +0300 Subject: [erlang-questions] beginner: dialyzer for the first time In-Reply-To: <1316076613.7747.32.camel@seasc1137> References: <1316076613.7747.32.camel@seasc1137> Message-ID: <4E71C1E6.1080706@cs.ntua.gr> On 09/15/11 11:50, Bengt Kleberg wrote: > > When I want to use dialyzer on some Erlang source files for the first > time I do: > dialyzer --src *erl > > This gives me the error: > Checking whether the PLT /home//.dialyzer_plt is > up-to-date...dialyzer: exit 1 > > dialyzer: Could not find the PLT: /home//.dialyzer_plt > Use the options: > --build_plt to build a new PLT; or > --add_to_plt to add to an existing PLT > > > So I try: > dialyzer --build_plt --src *erl > > This gives me the error: > dialyzer: {dialyzer_error,"Byte code compiled with debug_info is needed > to build the PLT"} > ... > > What should I do next? The goal is to use dialyzer on some Erlang source > files. You have misunderstood the (main) purpose of the PLT. It is intended to contain type information for modules of the Erlang standard library (so that this information is known to the analysis), not the files you want to analyze. The message you get is more helpful in later versions (e.g. R13B*) and reads as follows: ======================================================================= dialyzer: Could not find the PLT: /home/USER/.dialyzer_plt Use the options: --build_plt to build a new PLT; or --add_to_plt to add to an existing PLT For example, use a command like the following: dialyzer --build_plt --apps erts kernel stdlib mnesia Note that building a PLT such as the above may take 20 mins or so If you later need information about other applications, say crypto, you can extend the PLT by the command: dialyzer --add_to_plt --apps crypto For applications that are not in Erlang/OTP use an absolute file name. ======================================================================= Execute the line: dialyzer --build_plt --apps erts kernel stdlib first and then you can analyze your files in the way you started: dialyzer --src *.erl Kostis From bengt.kleberg@REDACTED Thu Sep 15 11:21:25 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 15 Sep 2011 11:21:25 +0200 Subject: [erlang-questions] beginner: dialyzer for the first time In-Reply-To: <4E71C1E6.1080706@cs.ntua.gr> References: <1316076613.7747.32.camel@seasc1137> <4E71C1E6.1080706@cs.ntua.gr> Message-ID: <1316078485.7747.34.camel@seasc1137> Thank you for the explanation. Is there a reason that this PLT (for the Erlang standard library) is not included with the Erlang standard library? ebngt On Thu, 2011-09-15 at 11:14 +0200, Kostis Sagonas wrote: > On 09/15/11 11:50, Bengt Kleberg wrote: > > > > When I want to use dialyzer on some Erlang source files for the first > > time I do: > > dialyzer --src *erl > > > > This gives me the error: > > Checking whether the PLT /home//.dialyzer_plt is > > up-to-date...dialyzer: exit 1 > > > > dialyzer: Could not find the PLT: /home//.dialyzer_plt > > Use the options: > > --build_plt to build a new PLT; or > > --add_to_plt to add to an existing PLT > > > > > > So I try: > > dialyzer --build_plt --src *erl > > > > This gives me the error: > > dialyzer: {dialyzer_error,"Byte code compiled with debug_info is needed > > to build the PLT"} > > ... > > > > What should I do next? The goal is to use dialyzer on some Erlang source > > files. > > You have misunderstood the (main) purpose of the PLT. It is intended to > contain type information for modules of the Erlang standard library (so > that this information is known to the analysis), not the files you want > to analyze. The message you get is more helpful in later versions (e.g. > R13B*) and reads as follows: > > ======================================================================= > dialyzer: Could not find the PLT: /home/USER/.dialyzer_plt > Use the options: > --build_plt to build a new PLT; or > --add_to_plt to add to an existing PLT > > For example, use a command like the following: > dialyzer --build_plt --apps erts kernel stdlib mnesia > Note that building a PLT such as the above may take 20 mins or so > > If you later need information about other applications, say crypto, > you can extend the PLT by the command: > dialyzer --add_to_plt --apps crypto > For applications that are not in Erlang/OTP use an absolute file name. > ======================================================================= > > Execute the line: > > dialyzer --build_plt --apps erts kernel stdlib > > first and then you can analyze your files in the way you started: > > dialyzer --src *.erl > > Kostis > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From norton@REDACTED Thu Sep 15 11:25:00 2011 From: norton@REDACTED (Joseph Norton) Date: Thu, 15 Sep 2011 18:25:00 +0900 Subject: [erlang-questions] Obsolete exported functions file:raw_{read, write}_file_info/2 - why? In-Reply-To: <09AC9E70-E4A4-4EEF-9ADD-B89C188B427F@gmail.com> References: <19976.1316018557@snookles.snookles.com> <09AC9E70-E4A4-4EEF-9ADD-B89C188B427F@gmail.com> Message-ID: Hi. I'm working on a patch for the file.erl module itself. In the meantime, see the attached module as a working example. We have been using this approach for benchmarking purposes since early summer. The performance difference is dramatically better than the default file implementation. thanks, Joseph Norton norton@REDACTED -------------- next part -------------- A non-text attachment was scrubbed... Name: basho_bench_erlang_file_alternative.erl Type: application/octet-stream Size: 3415 bytes Desc: not available URL: -------------- next part -------------- On Sep 15, 2011, at 1:57 AM, erlang wrote: >> I recommend reading the file.erl source. It's quite instructive to see >> how many file I/O functions are redirected to the 'file_server_2' >> process. For file I/O-intensive applications (e.g. Hibari and Riak I >> know, CouchDB and RabbitMQ I'd guess), having all calls to(*) >> file:read_file_info/1 serialized by the file server process is a source >> of latency that we (DB authors) may desire to live without. > > How did you proceed to avoid these calls and reduce latency Scott? > Any hints? From zabrane3@REDACTED Thu Sep 15 11:36:18 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 15 Sep 2011 11:36:18 +0200 Subject: [erlang-questions] Obsolete exported functions file:raw_{read, write}_file_info/2 - why? In-Reply-To: References: <19976.1316018557@snookles.snookles.com> <09AC9E70-E4A4-4EEF-9ADD-B89C188B427F@gmail.com> Message-ID: Thanks for sharing Joseph. One more question: Is the call prim_file:read_file/1 a better alternative to file:read_file/1? Regards, Zabrane On Sep 15, 2011, at 11:25 AM, Joseph Norton wrote: > > Hi. > > I'm working on a patch for the file.erl module itself. In the meantime, see the attached module as a working example. We have been using this approach for benchmarking purposes since early summer. The performance difference is dramatically better than the default file implementation. > > thanks, > > Joseph Norton > norton@REDACTED > > > On Sep 15, 2011, at 1:57 AM, erlang wrote: > >>> I recommend reading the file.erl source. It's quite instructive to see >>> how many file I/O functions are redirected to the 'file_server_2' >>> process. For file I/O-intensive applications (e.g. Hibari and Riak I >>> know, CouchDB and RabbitMQ I'd guess), having all calls to(*) >>> file:read_file_info/1 serialized by the file server process is a source >>> of latency that we (DB authors) may desire to live without. >> >> How did you proceed to avoid these calls and reduce latency Scott? >> Any hints? > From norton@REDACTED Thu Sep 15 12:25:57 2011 From: norton@REDACTED (Joseph Norton) Date: Thu, 15 Sep 2011 19:25:57 +0900 Subject: [erlang-questions] Obsolete exported functions file:raw_{read, write}_file_info/2 - why? In-Reply-To: References: <19976.1316018557@snookles.snookles.com> <09AC9E70-E4A4-4EEF-9ADD-B89C188B427F@gmail.com> Message-ID: Zabrane - Based on benchmarking results, I believe prim_file:read_file is a better alternative for performance. The other reason the prim_file approach is better is it should prevent callers with file i/o to different disks and/or different disk controllers from impacting each other. We have seen a case in production where failure of one disk crashed erlang processes doing file i/o to an unrelated disk. The root cause (not yet repeated and not yet proven though) is the singleton file i/o server process. Nevertheless, I was hoping to learn from my original post why the file module hasn't been written (or re-written) to provide raw support for other operations beyond just open. My only guess is that file i/o hasn't been a bottleneck for most Erlang applications. thanks, Joseph Norton norton@REDACTED On Sep 15, 2011, at 6:36 PM, Zabrane Mickael wrote: > Thanks for sharing Joseph. > > One more question: > Is the call prim_file:read_file/1 a better alternative to file:read_file/1? > > Regards, > Zabrane > > On Sep 15, 2011, at 11:25 AM, Joseph Norton wrote: > >> >> Hi. >> >> I'm working on a patch for the file.erl module itself. In the meantime, see the attached module as a working example. We have been using this approach for benchmarking purposes since early summer. The performance difference is dramatically better than the default file implementation. >> >> thanks, >> >> Joseph Norton >> norton@REDACTED >> >> >> On Sep 15, 2011, at 1:57 AM, erlang wrote: >> >>>> I recommend reading the file.erl source. It's quite instructive to see >>>> how many file I/O functions are redirected to the 'file_server_2' >>>> process. For file I/O-intensive applications (e.g. Hibari and Riak I >>>> know, CouchDB and RabbitMQ I'd guess), having all calls to(*) >>>> file:read_file_info/1 serialized by the file server process is a source >>>> of latency that we (DB authors) may desire to live without. >>> >>> How did you proceed to avoid these calls and reduce latency Scott? >>> Any hints? >> > > From sverker@REDACTED Thu Sep 15 12:32:13 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Thu, 15 Sep 2011 12:32:13 +0200 Subject: [erlang-questions] Strange behaviour of atoms in NIF Message-ID: <4E71D42D.9070707@erix.ericsson.se> Dmitry Groshev wrote: > I'm writing a NIF-based circular array library and experiencing a strange > behaviour of atoms in there. I have a NIF that should return an atom. It's > fairly simple, but I get different atoms when calling this NIF from > different places. Here is an issue with quite comprehensive explanation, > please take a look at it at github: > https://github.com/band115/ecirca/issues/31 I've tried to compare atoms that > are returned by printf'ing them, but they seems identical in despite of > obvious difference in shell. > > > NIF debugging tip #1: int erts_printf(const char *, ...); Works the same as printf with the additional feature of %T to print ERL_NIF_TERM's. Is not part of the official API (yet) and does therefore not work on Windows. NIF debugging tip #2: Use debug compiled emulator. > cd $ERL_TOP/erts/emulator > make FLAVOR=smp debug (or FLAVOR=plain) > $ERL_TOP/bin/cerl -debug /Sverker, Erlang/OTP Ericsson From zabrane3@REDACTED Thu Sep 15 12:33:34 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Thu, 15 Sep 2011 12:33:34 +0200 Subject: [erlang-questions] Obsolete exported functions file:raw_{read, write}_file_info/2 - why? In-Reply-To: References: <19976.1316018557@snookles.snookles.com> <09AC9E70-E4A4-4EEF-9ADD-B89C188B427F@gmail.com> Message-ID: <9AB62D44-EEB1-4731-87F7-B69453D84724@gmail.com> Thanks a lot. On Sep 15, 2011, at 12:25 PM, Joseph Norton wrote: > > Zabrane - > > Based on benchmarking results, I believe prim_file:read_file is a better alternative for performance. The other reason the prim_file approach is better is it should prevent callers with file i/o to different disks and/or different disk controllers from impacting each other. We have seen a case in production where failure of one disk crashed erlang processes doing file i/o to an unrelated disk. The root cause (not yet repeated and not yet proven though) is the singleton file i/o server process. > > Nevertheless, I was hoping to learn from my original post why the file module hasn't been written (or re-written) to provide raw support for other operations beyond just open. My only guess is that file i/o hasn't been a bottleneck for most Erlang applications. > > thanks, > > Joseph Norton > norton@REDACTED From gordon@REDACTED Thu Sep 15 14:05:58 2011 From: gordon@REDACTED (Gordon Guthrie) Date: Thu, 15 Sep 2011 13:05:58 +0100 Subject: [erlang-questions] Mnesia repair is failing Message-ID: Folks The system crashed on me due to memory and left Mnesia in a state. On restart it goes into repair mode then then crashes out with: ** FATAL ** {error, {"Cannot open dets table", 'temp-dla3.hypernumbers.com&80&logging', [{file, "/hn/hypernumbers/var/db/temp-dla3.hypernumbers.com &80&logging.DAT"}, {keypos,2}, {repair,true}, {type,bag}], {no_more_space_on_file, "/hn/hypernumbers/var/db/temp-dla3.hypernumbers.com &80&logging.DAT.TMP"}}} I am presuming that while the mnesia table is OK the underlying dets table created as part of the repair is just too damn big and breaks the 2GB limit on dets. What do I do now? It is only a logging table so I could trash it, but I need to bring the system up. Can I start Mnesia and do stuff to the schema to take it out without kicking off the repair function? Gordon -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Sep 15 17:11:14 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 15 Sep 2011 17:11:14 +0200 Subject: [erlang-questions] where did my code come from? In-Reply-To: <1316072378.7747.5.camel@seasc1137> References: <1315982776.5065.2.camel@seasc1137> <1316072378.7747.5.camel@seasc1137> Message-ID: On Thu, Sep 15, 2011 at 09:39, Bengt Kleberg wrote: > In my mind the problem that module renaming should solve is "modules > with the same name in the Erlang Runtime". I agree it would be a very good thing to solve. It would be nice to have a solution where you could have multiple modules of the same names in the runtime. But perhaps it is not a question of renaming modules, but rather a question of module visibility. Making a module visible under another name is a way to avoid name clashes. > The gofix tool, does it handle leagacy/binary code? No it doesn't. I think the assumption is that the source code is available. Handling legacy/binary code is a more involved problem. -- J. From dmercer@REDACTED Thu Sep 15 17:51:29 2011 From: dmercer@REDACTED (David Mercer) Date: Thu, 15 Sep 2011 10:51:29 -0500 Subject: [erlang-questions] where did my code come from? In-Reply-To: <1316072378.7747.5.camel@seasc1137> References: <1315982776.5065.2.camel@seasc1137> <1316072378.7747.5.camel@seasc1137> Message-ID: <003501cc73bf$56797550$036c5ff0$@com> On Thursday, September 15, 2011, Bengt Kleberg wrote: > In my mind the problem that module renaming should solve is "modules > with the same name in the Erlang Runtime". Kind of like an -import_lib directive, saying, for instance, to pull references to the lists module from a different library than stdlib? Cheers, DBM From anthonym@REDACTED Thu Sep 15 18:06:50 2011 From: anthonym@REDACTED (Anthony Molinaro) Date: Thu, 15 Sep 2011 09:06:50 -0700 Subject: [erlang-questions] Any Erlang GIS software out there? Message-ID: <20110915160650.GB1632@alumni.caltech.edu> Hi, I'm interested in doing some reverse geocoding (taking lat/long and getting nearby cities). While looking around I've not seen any GIS related software written in erlang. So just wondering if I'm missing anything? Anyone written any GIS related software in Erlang? Thanks, -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro From jack@REDACTED Thu Sep 15 18:27:29 2011 From: jack@REDACTED (Jack Moffitt) Date: Thu, 15 Sep 2011 10:27:29 -0600 Subject: [erlang-questions] Any Erlang GIS software out there? In-Reply-To: <20110915160650.GB1632@alumni.caltech.edu> References: <20110915160650.GB1632@alumni.caltech.edu> Message-ID: > ?I'm interested in doing some reverse geocoding (taking lat/long and > getting nearby cities). ?While looking around I've not seen any GIS > related software written in erlang. ?So just wondering if I'm missing > anything? ?Anyone written any GIS related software in Erlang? SimpleGeo has a context API that will give you what you are looking for I believe. Another alternative is to use the openstreetmap data and roll your own. As for general GIS stuff in Erlang, I don't know of anything open source, but plenty of people have been building products with erlang and geo data. I know of several that use Ejabberd (they wrote custom geo modules that plug in to it) and we use Erlang almost exclusively at Shadow Cities, which is a location-based mobile MMO. Also, there is GeoCouch I believe, which does have some open source code in Erlang. jack. From reynaldomic@REDACTED Thu Sep 15 18:47:21 2011 From: reynaldomic@REDACTED (Reynaldo Baquerizo) Date: Thu, 15 Sep 2011 11:47:21 -0500 Subject: [erlang-questions] Any Erlang GIS software out there? In-Reply-To: <20110915160650.GB1632@alumni.caltech.edu> References: <20110915160650.GB1632@alumni.caltech.edu> Message-ID: Hi, > ?I'm interested in doing some reverse geocoding (taking lat/long and > getting nearby cities). ?While looking around I've not seen any GIS > related software written in erlang. ?So just wondering if I'm missing > anything? ?Anyone written any GIS related software in Erlang? I have played a little with it. A short example: -export([reverse_geocode/0]). -define(BASE_URL, "http://sampleserver1.arcgisonline.com/ArcGIS/rest/services/" ++ "Locators/ESRI_Geocode_USA/GeocodeServer/reverseGeocode?" ++ "location=-117.195681386%2C34.057517097&distance=5&outSR=&f=pjson"). reverse_geocode() -> reverse_geocode(?BASE_URL). reverse_geocode(Url) -> {ok, Result} = httpc:request(Url), {{_,200,_},_,Body} = Result, {_Json} = mochijson2:decode(Body), {Data} = proplists:get_value(<<"address">>, _Json), Data. -- Reynaldo From norton@REDACTED Thu Sep 15 19:23:32 2011 From: norton@REDACTED (Joseph Wayne Norton) Date: Fri, 16 Sep 2011 02:23:32 +0900 Subject: [erlang-questions] Obsolete exported functions file:raw_{read, write}_file_info/2 - why? In-Reply-To: References: <19976.1316018557@snookles.snookles.com> <09AC9E70-E4A4-4EEF-9ADD-B89C188B427F@gmail.com> Message-ID: As a followup to my previous e-mail, here is one possible patch for the file.erl module itself. I have not tested this patch so please consider it as pseudo-code only. https://github.com/norton/otp/commit/6541f14800ab687dc6aa262e464f5ff9d72c2deb thanks, On 2011/09/15, at 18:25, Joseph Norton wrote: > > Hi. > > I'm working on a patch for the file.erl module itself. In the meantime, see the attached module as a working example. We have been using this approach for benchmarking purposes since early summer. The performance difference is dramatically better than the default file implementation. > > thanks, > > Joseph Norton > norton@REDACTED > > > On Sep 15, 2011, at 1:57 AM, erlang wrote: > >>> I recommend reading the file.erl source. It's quite instructive to see >>> how many file I/O functions are redirected to the 'file_server_2' >>> process. For file I/O-intensive applications (e.g. Hibari and Riak I >>> know, CouchDB and RabbitMQ I'd guess), having all calls to(*) >>> file:read_file_info/1 serialized by the file server process is a source >>> of latency that we (DB authors) may desire to live without. >> >> How did you proceed to avoid these calls and reduce latency Scott? >> Any hints? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Joseph Wayne Norton norton@REDACTED From roberto@REDACTED Thu Sep 15 22:39:50 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 15 Sep 2011 22:39:50 +0200 Subject: [erlang-questions] Any Erlang GIS software out there? In-Reply-To: References: <20110915160650.GB1632@alumni.caltech.edu> Message-ID: might this be what you need? https://github.com/mochi/egeoip r. On Thu, Sep 15, 2011 at 6:47 PM, Reynaldo Baquerizo wrote: > Hi, > > > I'm interested in doing some reverse geocoding (taking lat/long and > > getting nearby cities). While looking around I've not seen any GIS > > related software written in erlang. So just wondering if I'm missing > > anything? Anyone written any GIS related software in Erlang? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonym@REDACTED Fri Sep 16 04:54:35 2011 From: anthonym@REDACTED (Anthony Molinaro) Date: Thu, 15 Sep 2011 19:54:35 -0700 Subject: [erlang-questions] Any Erlang GIS software out there? In-Reply-To: References: <20110915160650.GB1632@alumni.caltech.edu> Message-ID: <8C4FC719-ADD5-419D-BACF-13584FBD3EE1@alumni.caltech.edu> No egeoip is an ip address to geo location library which I have used and know about. I want to take a latitude and longitude and get a geo location which is a bit different. Using a web service is not an option as I only have a millisecond or so to get this information. Also I'm likely to have to call it a few billion times a day eventually so am spooking for software to run locally. I created a hackish reverse geo location by using ets and an ets:select/2 which works with the locations I have. But it basically just uses point data so would be interested in any systems which use vector based data for better accuracy. Thanks for the pointers, -Anthony On Sep 15, 2011, at 1:39 PM, Roberto Ostinelli wrote: > might this be what you need? > https://github.com/mochi/egeoip > > r. > > > On Thu, Sep 15, 2011 at 6:47 PM, Reynaldo Baquerizo wrote: > Hi, > > > I'm interested in doing some reverse geocoding (taking lat/long and > > getting nearby cities). While looking around I've not seen any GIS > > related software written in erlang. So just wondering if I'm missing > > anything? Anyone written any GIS related software in Erlang? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Fri Sep 16 07:45:12 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 16 Sep 2011 07:45:12 +0200 Subject: [erlang-questions] where did my code come from? In-Reply-To: <003501cc73bf$56797550$036c5ff0$@com> References: <1315982776.5065.2.camel@seasc1137> <1316072378.7747.5.camel@seasc1137> <003501cc73bf$56797550$036c5ff0$@com> Message-ID: <1316151912.4778.5.camel@seasc1137> I can think of ways to make my code flexible enough to handle choosing between two different modules. What I can not do is to have two modules called 'lists' loaded/used at the same time. bengt On Thu, 2011-09-15 at 17:51 +0200, David Mercer wrote: > On Thursday, September 15, 2011, Bengt Kleberg wrote: > > > In my mind the problem that module renaming should solve is "modules > > with the same name in the Erlang Runtime". > > Kind of like an -import_lib directive, saying, for instance, to pull > references to the lists module from a different library than stdlib? > > Cheers, > > DBM > From bookjovi@REDACTED Fri Sep 16 10:38:52 2011 From: bookjovi@REDACTED (Jovi Zhang) Date: Fri, 16 Sep 2011 16:38:52 +0800 Subject: [erlang-questions] process will stack overflow after received many message? Message-ID: Hi, I am a newbie of Erlang, here I have a question for receive message in Erlang loop. Like below Erlang code, when process receive one message, it will invoke loop again, then it can service like a server, receive message forever. BUT is possible that process will stack overflow after receive too many message? the process invoke loop again and again. I saw there have many code write like this in <> book. loop() -> receive hello -> io:format("hello\n"), loop(); Other -> io:format("I don't know what is this message, ~p is ~n" ,[Other]), loop() end. From rtrlists@REDACTED Fri Sep 16 10:44:39 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Fri, 16 Sep 2011 09:44:39 +0100 Subject: [erlang-questions] process will stack overflow after received many message? In-Reply-To: References: Message-ID: On Fri, Sep 16, 2011 at 9:38 AM, Jovi Zhang wrote: > Hi, > I am a newbie of Erlang, here I have a question for receive > message in Erlang loop. > Like below Erlang code, when process receive one message, it will > invoke loop again, > then it can service like a server, receive message forever. > BUT is possible that process will stack overflow after receive too > many message? the process invoke loop again and again. > > I saw there have many code write like this in <> > book. > > loop() -> > receive > hello -> > io:format("hello\n"), > loop(); > Other -> > io:format("I don't know what is this message, ~p is ~n" > ,[Other]), > loop() > end. > > No stack overflow, due to tail call optimisation. See also http://www.erlang.org/doc/reference_manual/functions.html#id74170 Robby -------------- next part -------------- An HTML attachment was scrubbed... URL: From bookjovi@REDACTED Fri Sep 16 10:57:36 2011 From: bookjovi@REDACTED (Jovi Zhang) Date: Fri, 16 Sep 2011 16:57:36 +0800 Subject: [erlang-questions] process will stack overflow after received many message? In-Reply-To: References: Message-ID: On Fri, Sep 16, 2011 at 4:44 PM, Robert Raschke wrote: > > On Fri, Sep 16, 2011 at 9:38 AM, Jovi Zhang wrote: >> >> Hi, >> ? ?I am a newbie of Erlang, here I have a question for receive >> message in Erlang loop. >> ? ?Like below Erlang code, when process receive one message, it will >> invoke loop again, >> ? ?then it can service like a server, receive message forever. >> ? ?BUT is possible that process will stack overflow after receive too >> many message? the process invoke loop again and again. >> >> ? ?I saw there have many code write like this in <> >> book. >> >> loop() -> >> ? ?receive >> ? ? ? ?hello -> >> ? ? ? ? ? ?io:format("hello\n"), >> ? ? ? ? ? ?loop(); >> ? ? ? ?Other -> >> ? ? ? ? ? ?io:format("I don't know what is this message, ~p is ~n" >> ,[Other]), >> ? ? ? ? ? ?loop() >> ? ?end. >> > > No stack overflow, due to tail call optimisation. See also > http://www.erlang.org/doc/reference_manual/functions.html#id74170 > > Robby > Thanks, but when I test it using escript, the result is like below, "loop end" printed two times, it means it stacked! [root@REDACTED Erlang]# cat jovi.erl #!/usr/bin/env escript loop() -> receive one -> io:format("one\n"), loop(); two -> io:format("two\n") end, io:format("loop end\n"). main(_) -> Pid = spawn(fun() -> loop() end), Pid ! one, Pid ! two, receive _Any -> void end. [root@REDACTED Erlang]# ./jovi.erl one two loop end loop end ^C From hm@REDACTED Fri Sep 16 11:05:16 2011 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Fri, 16 Sep 2011 11:05:16 +0200 Subject: [erlang-questions] process will stack overflow after received many message? In-Reply-To: References: Message-ID: Your last call to io:format/2 is making the function non tail recursive. /H?kan On Fri, Sep 16, 2011 at 10:57 AM, Jovi Zhang wrote: > On Fri, Sep 16, 2011 at 4:44 PM, Robert Raschke wrote: >> >> On Fri, Sep 16, 2011 at 9:38 AM, Jovi Zhang wrote: >>> >>> Hi, >>> ? ?I am a newbie of Erlang, here I have a question for receive >>> message in Erlang loop. >>> ? ?Like below Erlang code, when process receive one message, it will >>> invoke loop again, >>> ? ?then it can service like a server, receive message forever. >>> ? ?BUT is possible that process will stack overflow after receive too >>> many message? the process invoke loop again and again. >>> >>> ? ?I saw there have many code write like this in <> >>> book. >>> >>> loop() -> >>> ? ?receive >>> ? ? ? ?hello -> >>> ? ? ? ? ? ?io:format("hello\n"), >>> ? ? ? ? ? ?loop(); >>> ? ? ? ?Other -> >>> ? ? ? ? ? ?io:format("I don't know what is this message, ~p is ~n" >>> ,[Other]), >>> ? ? ? ? ? ?loop() >>> ? ?end. >>> >> >> No stack overflow, due to tail call optimisation. See also >> http://www.erlang.org/doc/reference_manual/functions.html#id74170 >> >> Robby >> > > Thanks, but when I test it using escript, the result is like below, > "loop end" printed two times, it means it stacked! > > [root@REDACTED Erlang]# cat jovi.erl > #!/usr/bin/env escript > > loop() -> > ? ?receive > ? ? ? ?one -> > ? ? ? ? ? ?io:format("one\n"), > ? ? ? ? ? ?loop(); > ? ? ? ?two -> > ? ? ? ? ? ?io:format("two\n") > ? ?end, > ? ?io:format("loop end\n"). > > > main(_) -> > ? ?Pid = spawn(fun() -> loop() end), > ? ?Pid ! one, > ? ?Pid ! two, > ? ?receive > ? ? ? ?_Any -> void > ? ?end. > > > [root@REDACTED Erlang]# ./jovi.erl > one > two > loop end > loop end From steve@REDACTED Fri Sep 16 11:06:14 2011 From: steve@REDACTED (Steve Strong) Date: Fri, 16 Sep 2011 11:06:14 +0200 Subject: [erlang-questions] process will stack overflow after received many message? In-Reply-To: References: Message-ID: You might find this link useful as well http://en.wikipedia.org/wiki/Tail_call - the reason that your new example is stacking is that having code after the recursion means that it's no longer tail-recursive, and will indeed consume stack space. -- Steve Strong @srstrong Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Friday, 16 September 2011 at 10:57, Jovi Zhang wrote: > On Fri, Sep 16, 2011 at 4:44 PM, Robert Raschke wrote: > > > > On Fri, Sep 16, 2011 at 9:38 AM, Jovi Zhang wrote: > > > > > > Hi, > > > I am a newbie of Erlang, here I have a question for receive > > > message in Erlang loop. > > > Like below Erlang code, when process receive one message, it will > > > invoke loop again, > > > then it can service like a server, receive message forever. > > > BUT is possible that process will stack overflow after receive too > > > many message? the process invoke loop again and again. > > > > > > I saw there have many code write like this in <> > > > book. > > > > > > loop() -> > > > receive > > > hello -> > > > io:format("hello\n"), > > > loop(); > > > Other -> > > > io:format("I don't know what is this message, ~p is ~n" > > > ,[Other]), > > > loop() > > > end. > > > > No stack overflow, due to tail call optimisation. See also > > http://www.erlang.org/doc/reference_manual/functions.html#id74170 > > > > Robby > > Thanks, but when I test it using escript, the result is like below, > "loop end" printed two times, it means it stacked! > > [root@REDACTED Erlang]# cat jovi.erl > #!/usr/bin/env escript > > loop() -> > receive > one -> > io:format("one\n"), > loop(); > two -> > io:format("two\n") > end, > io:format("loop end\n"). > > > main(_) -> > Pid = spawn(fun() -> loop() end), > Pid ! one, > Pid ! two, > receive > _Any -> void > end. > > > [root@REDACTED Erlang]# ./jovi.erl > one > two > loop end > loop end > ^C > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Fri Sep 16 07:48:26 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 16 Sep 2011 07:48:26 +0200 Subject: [erlang-questions] where did my code come from? In-Reply-To: References: <1315982776.5065.2.camel@seasc1137> <1316072378.7747.5.camel@seasc1137> Message-ID: <1316152106.4778.8.camel@seasc1137> Since I am working at a place that has all the source, but still does not accept changing legacy, I think goflex is a no-go. (Sorry about that feeble attempt of humor). bengt On Thu, 2011-09-15 at 17:11 +0200, Jesper Louis Andersen wrote: > On Thu, Sep 15, 2011 at 09:39, Bengt Kleberg wrote: > > In my mind the problem that module renaming should solve is "modules > > with the same name in the Erlang Runtime". > > I agree it would be a very good thing to solve. It would be nice to > have a solution where you could have multiple modules of the same > names in the runtime. But perhaps it is not a question of renaming > modules, but rather a question of module visibility. Making a module > visible under another name is a way to avoid name clashes. > > > The gofix tool, does it handle leagacy/binary code? > > No it doesn't. I think the assumption is that the source code is > available. Handling legacy/binary code is a more involved problem. > From bookjovi@REDACTED Fri Sep 16 11:45:08 2011 From: bookjovi@REDACTED (Jovi Zhang) Date: Fri, 16 Sep 2011 17:45:08 +0800 Subject: [erlang-questions] process will stack overflow after received many message? In-Reply-To: References: Message-ID: On Fri, Sep 16, 2011 at 5:06 PM, Steve Strong wrote: > You might find this link useful as > well?http://en.wikipedia.org/wiki/Tail_call?- the reason that your new > example is stacking is that having code after the recursion means that it's > no longer tail-recursive, and will indeed consume stack space. > -- > Steve Strong > @srstrong > Sent with Sparrow > > On Friday, 16 September 2011 at 10:57, Jovi Zhang wrote: > > On Fri, Sep 16, 2011 at 4:44 PM, Robert Raschke > wrote: > > On Fri, Sep 16, 2011 at 9:38 AM, Jovi Zhang wrote: > > Hi, > ? ?I am a newbie of Erlang, here I have a question for receive > message in Erlang loop. > ? ?Like below Erlang code, when process receive one message, it will > invoke loop again, > ? ?then it can service like a server, receive message forever. > ? ?BUT is possible that process will stack overflow after receive too > many message? the process invoke loop again and again. > > ? ?I saw there have many code write like this in <> > book. > > loop() -> > ? ?receive > ? ? ? ?hello -> > ? ? ? ? ? ?io:format("hello\n"), > ? ? ? ? ? ?loop(); > ? ? ? ?Other -> > ? ? ? ? ? ?io:format("I don't know what is this message, ~p is ~n" > ,[Other]), > ? ? ? ? ? ?loop() > ? ?end. > > No stack overflow, due to tail call optimisation. See also > http://www.erlang.org/doc/reference_manual/functions.html#id74170 > > Robby > > Thanks, but when I test it using escript, the result is like below, > "loop end" printed two times, it means it stacked! > > [root@REDACTED Erlang]# cat jovi.erl > #!/usr/bin/env escript > > loop() -> > receive > one -> > io:format("one\n"), > loop(); > two -> > io:format("two\n") > end, > io:format("loop end\n"). > > > main(_) -> > Pid = spawn(fun() -> loop() end), > Pid ! one, > Pid ! two, > receive > _Any -> void > end. > > > [root@REDACTED Erlang]# ./jovi.erl > one > two > loop end > loop end > ^C > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > Hmm, Thanks very much, I understand it now! .jovi From raimo+erlang-questions@REDACTED Fri Sep 16 11:52:14 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 16 Sep 2011 11:52:14 +0200 Subject: [erlang-questions] FreeBSD patches Message-ID: <20110916095213.GA11461@erix.ericsson.se> Hi. I am just curious about the patches that are in FreeBSD's port tree for building Erlang, in particular these erts ones: # cat files/patch-erts-emulator-Makefile.in $FreeBSD: ports/lang/erlang/files/patch-erts-emulator-Makefile.in,v 1.3 2008/11/05 23:32:22 olgeni Exp $ --- erts/emulator/Makefile.in.orig +++ erts/emulator/Makefile.in @@ -544,7 +546,7 @@ endif $(OBJDIR)/%.o: beam/%.c - $(CC) $(subst -O2, $(GEN_OPT_FLGS), $(CFLAGS)) $(INCLUDES) -c $< -o $@ + $(CC) $(INCLUDES) $(subst -O2, $(GEN_OPT_FLGS), $(CFLAGS)) -c $< -o $@ else # cat files/patch-erts_etc_common_erlc.c $FreeBSD: ports/lang/erlang/files/patch-erts_etc_common_erlc.c,v 1.1 2010/04/07 22:04:49 olgeni Exp $ --- erts/etc/common/erlc.c.orig +++ erts/etc/common/erlc.c @@ -186,6 +186,7 @@ */ PUSH("-noinput"); + PUSH2("-smp", "disable"); PUSH2("-mode", "minimal"); PUSH2("-boot", "start_clean"); PUSH3("-s", "erl_compile", "compile_cmdline"); Given a good explanation we could include them in R15. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From jesper.louis.andersen@REDACTED Fri Sep 16 17:31:01 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 16 Sep 2011 17:31:01 +0200 Subject: [erlang-questions] process will stack overflow after received many message? In-Reply-To: References: Message-ID: On Fri, Sep 16, 2011 at 11:45, Jovi Zhang wrote: > > Hmm, Thanks very much, I understand it now! > Let me add that the perhaps most subtle way to make a function stacking is to put an exception handler on the stack. Beware of that case if you wonder why something is stacking when it should not be. It only happens if the exception handler is pushed by each recursive call though. -- J. From jwatte@REDACTED Fri Sep 16 20:46:42 2011 From: jwatte@REDACTED (Jon Watte) Date: Fri, 16 Sep 2011 11:46:42 -0700 Subject: [erlang-questions] ets:next/2 behavior - set vs. ordered_set In-Reply-To: <4D010521-D005-44C4-82C9-75438256A773@lovely.email.ne.jp> References: <00E5DFDF-EF45-4A8A-8871-227911A55C78@lovely.email.ne.jp> <4D010521-D005-44C4-82C9-75438256A773@lovely.email.ne.jp> Message-ID: In what way is dets not production worthy? Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Mon, Sep 12, 2011 at 7:07 PM, Joseph Norton wrote: > > The basic goal is to build and test a production worthy replacement for ets > with disk based storage. After I have the basics in place, I can elaborate > further. > > thanks, > > Joseph Norton > > On Sep 13, 2011, at 9:55 AM, OvermindDL1 wrote: > > > On Mon, Sep 12, 2011 at 9:46 AM, Joseph Wayne Norton > > wrote: > >> > >> I'm currently developing a disk based clone of the ets application. > > > > Can you elaborate how this is different from dets, or are you doing it > > for learning? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Fri Sep 16 20:48:26 2011 From: jwatte@REDACTED (Jon Watte) Date: Fri, 16 Sep 2011 11:48:26 -0700 Subject: [erlang-questions] Two network interfaces. In-Reply-To: References: Message-ID: One alternative is to do interface selection using a static route. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Tue, Sep 13, 2011 at 6:25 AM, Krzysztof Blachowski < kblachowski@REDACTED> wrote: > Hi all. > > Let's imagine that we have two machines A and B with erlang VM. > > On each erlang machine there is application which uses mnesia database. The > database is replicated between nodes. > Each node has two interfaces "I1" which is extremely fast and "I2" which is > slow. There is also a bunch of erlang nodes which are clients > for A and B nodes and they are using native erlang interface. > > I was trying to configure erlang nodes A and B to utilize this two > interfaces but without any effect. > How to connect nodes A i B with fast link (for mnesia replication) and at > this same time connect them with slow link with external clients? > > I've tried: > > > machine A: > > erl -name master -setcookie secret > > inet:getif(): > {ok,[{{192,168,1,10},{192,168,255,255},{255,255,0,0}}, > {{10,132,24,10},{10,132,25,255},{255,255,254,0}}, > {{127,0,0,1},{0,0,0,0},{255,0,0,0}}]} > > machine B: > > erl -name replica -setcookie secret > > net_adm:ping('master@REDACTED'): > pong > > net_adm:ping('master@REDACTED'): > pang > > Why? What I'm doing wrong? > > best regards > > ---------------------------------------------------------------- > Skoda Octavia z 2009 roku juz za 11 tys. zl > http://linkint.pl/f2a36 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Fri Sep 16 20:51:40 2011 From: jwatte@REDACTED (Jon Watte) Date: Fri, 16 Sep 2011 11:51:40 -0700 Subject: [erlang-questions] Rough thought on a P2P package distribution model for Erlang In-Reply-To: <4E700AFF.7090309@ghostgun.com> References: <4E700AFF.7090309@ghostgun.com> Message-ID: How is this different from the already-solved problem of peer-to-peer authenticated file distribution? Tracker-based systems like bittorrent, and fully peer-to-peer systems like freenet have been around for a very long time, and solve all of those problems, with different trade-offs for performance, security, susceptibility, etc. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Tue, Sep 13, 2011 at 7:01 PM, jm wrote: > In my previous email I said that I thought P2P package distribution system > would be a good idea. This was due to it elimination the single points of > failure with relying of the future of websites. There are a number of > problems with using a P2P module. Chief among these are how to get packages > into the system and how to know that these packages are trust worthy. > > With that in mind here's some rough thoughts on a P2P module repository for > Erlang: > > Publisher: the person who maintains the package. Typically, the author of > the module being published. > Node: a server which is a member of the P2P module repository system > Indexer: a person who creates an index of packages that they say meets > some criteria ie, they vouch for the packages. > Administrator: the person who looks after a node > > The process would work something like this, > > Some one writes a wonderful module the one everyone has been waiting for. > Either the original author or someone on their behalf packages it up. > The Publisher then makes this publicly available on a website or through > git/mercurial/etc > The Publisher notifies one or more indexers. > Each Indexers check that the package meets their criteria. > The Indexer then injects the package into the p2p distribution system along > with an updated signed versioned index file. > This index file lists which packages the Indexer has verified and the > cryptographic hash for each package. > The Administrators of other nodes select which Indexers they wish to follow > and keep copies each Indexers public key (obtained out of band). > The Nodes then replicates the index file of each Indexer of interest and > the packages listed by those index files. > These nodes then make this information available of ftp/http/p2p or other > means to other nodes and end developers. > > Using an Indexer has a couple of advantages: > 1) it eliminates the need for everyone to have certificates. Making the > system cleaner to use and lowering the barrier to entry of package > maintainers allow them to easily submit their work without distraction. > 2) It maintains a concept similar to existing repositories with which > people are familiar. This makes it easy for people to bring up and maintain > additional nodes. It also means that the number of people that have to wade > though all the packages out there is reduced down to the Indexer. You simple > select the Indexer who has a package criteria which reflects your own. > > This is separate what packages are and who Erlang handles dependances. This > is merely a distribution model. > > Excuse the broad description I merely intend this to give people ideas. > > Jeff. > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Fri Sep 16 20:56:17 2011 From: jwatte@REDACTED (Jon Watte) Date: Fri, 16 Sep 2011 11:56:17 -0700 Subject: [erlang-questions] Erlang and beam-based scripting languages. In-Reply-To: References: Message-ID: If your main goal is to use erlang as an application server for web requests, and you want a fast development iteration cycle, have you checked out the "reloader" module? (I think it comes with mochiweb) This module will re-load BEAM files as they are re-compiled, into a running Erlang server. I find it's quite convenient for quick turn-around development. Yes, you still need to type "make" after you save the changes, before you actually hit the service on the web, but you can also just bind a macro to "save and make" :-) The benefit of this is that, if you have a syntax error, you don't have to poke at the application through a web browser to find it; you will be told by the compiler immediately. I wouldn't recommend using reloader for production use, though. Also, there are some issues with code references hanging around past the "old code" stage -- long-lived funs are the biggest culprit. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Tue, Sep 13, 2011 at 4:19 AM, Max Lapshin wrote: > Hi. > > I've written a letter several times to Santa Claus to have some easy > Rails <-> erlang integration for my needs, but he seems to be too > busy. > > So, my problem is: it is not convenient for me to write all customer > logic in erlang, because it involves process of compiling and loading > modules. > Also, I would be very great to share the same file between web site > and erlang daemon. > > It seems, that most developed beam-based language is elixir > (https://github.com/josevalim/elixir ), but there are some erlang > problems, that looks like blocking for > have convenient scripting in erlang. > > > I have erlyvideo server and it has config. There is option in config: > > {detectors, [rewrite, akamai, livestream]}. > > When user requests stream, media_provider takes this list of > detectors, goes one by one and calls: > > case Module:check(Host, Name, Options) of > .. > > for each atom. Now we meet the problem. When I write Module:check, > code_server is loading module and it can load only .beam and .app > files. > If I have any way to hook inside with some code that will say: Hi, I > know how to load this module, i think it will be easier to add simple > non-compilable scripting to my app and have something like Java world > have: Ruby/Python classed mixed with "native" Java classes. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Fri Sep 16 21:47:23 2011 From: bob@REDACTED (Bob Ippolito) Date: Fri, 16 Sep 2011 12:47:23 -0700 Subject: [erlang-questions] ets:next/2 behavior - set vs. ordered_set In-Reply-To: References: <00E5DFDF-EF45-4A8A-8871-227911A55C78@lovely.email.ne.jp> <4D010521-D005-44C4-82C9-75438256A773@lovely.email.ne.jp> Message-ID: The main issues that I'm aware of are that a dets file can't be larger than 2GB, ordered_set isn't supported, repair is often not very fast, and the only way to defragment a table is to close it and re-open it with repair set to force (mostly a problem because of the 2GB limit). http://www.erlang.org/doc/man/dets.html On Fri, Sep 16, 2011 at 11:46 AM, Jon Watte wrote: > In what way is dets not production worthy? > Sincerely, > jw > > -- > Americans might object: there is no way we would sacrifice our living > standards for the benefit of people in the rest of the world. Nevertheless, > whether we get there willingly or not, we shall soon have lower consumption > rates, because our present rates are unsustainable. > > > > On Mon, Sep 12, 2011 at 7:07 PM, Joseph Norton > wrote: >> >> The basic goal is to build and test a production worthy replacement for >> ets with disk based storage. ? After I have the basics in place, I can >> elaborate further. >> >> thanks, >> >> Joseph Norton >> >> On Sep 13, 2011, at 9:55 AM, OvermindDL1 wrote: >> >> > On Mon, Sep 12, 2011 at 9:46 AM, Joseph Wayne Norton >> > wrote: >> >> >> >> I'm currently developing a disk based clone of the ets application. >> > >> > Can you elaborate how this is different from dets, or are you doing it >> > for learning? >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From norton@REDACTED Sat Sep 17 00:23:08 2011 From: norton@REDACTED (Joseph Norton) Date: Sat, 17 Sep 2011 07:23:08 +0900 Subject: [erlang-questions] ets:next/2 behavior - set vs. ordered_set In-Reply-To: References: <00E5DFDF-EF45-4A8A-8871-227911A55C78@lovely.email.ne.jp> <4D010521-D005-44C4-82C9-75438256A773@lovely.email.ne.jp> Message-ID: That is my understanding too. jwnorton@REDACTED On 2011/09/17, at 4:47, Bob Ippolito wrote: > The main issues that I'm aware of are that a dets file can't be larger > than 2GB, ordered_set isn't supported, repair is often not very fast, > and the only way to defragment a table is to close it and re-open it > with repair set to force (mostly a problem because of the 2GB limit). > > http://www.erlang.org/doc/man/dets.html > > On Fri, Sep 16, 2011 at 11:46 AM, Jon Watte wrote: >> In what way is dets not production worthy? >> Sincerely, >> jw >> >> -- >> Americans might object: there is no way we would sacrifice our living >> standards for the benefit of people in the rest of the world. Nevertheless, >> whether we get there willingly or not, we shall soon have lower consumption >> rates, because our present rates are unsustainable. >> >> >> >> On Mon, Sep 12, 2011 at 7:07 PM, Joseph Norton >> wrote: >>> >>> The basic goal is to build and test a production worthy replacement for >>> ets with disk based storage. After I have the basics in place, I can >>> elaborate further. >>> >>> thanks, >>> >>> Joseph Norton >>> >>> On Sep 13, 2011, at 9:55 AM, OvermindDL1 wrote: >>> >>>> On Mon, Sep 12, 2011 at 9:46 AM, Joseph Wayne Norton >>>> wrote: >>>>> >>>>> I'm currently developing a disk based clone of the ets application. >>>> >>>> Can you elaborate how this is different from dets, or are you doing it >>>> for learning? >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> From jwatte@REDACTED Sat Sep 17 06:47:59 2011 From: jwatte@REDACTED (Jon Watte) Date: Fri, 16 Sep 2011 21:47:59 -0700 Subject: [erlang-questions] ets:next/2 behavior - set vs. ordered_set In-Reply-To: References: <00E5DFDF-EF45-4A8A-8871-227911A55C78@lovely.email.ne.jp> <4D010521-D005-44C4-82C9-75438256A773@lovely.email.ne.jp> Message-ID: So, by transitivity, the same reasons mnesia is annoying for large data sets? Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Fri, Sep 16, 2011 at 12:47 PM, Bob Ippolito wrote: > The main issues that I'm aware of are that a dets file can't be larger > than 2GB, ordered_set isn't supported, repair is often not very fast, > and the only way to defragment a table is to close it and re-open it > with repair set to force (mostly a problem because of the 2GB limit). > > http://www.erlang.org/doc/man/dets.html > > On Fri, Sep 16, 2011 at 11:46 AM, Jon Watte wrote: > > In what way is dets not production worthy? > > Sincerely, > > jw > > > > -- > > Americans might object: there is no way we would sacrifice our living > > standards for the benefit of people in the rest of the world. > Nevertheless, > > whether we get there willingly or not, we shall soon have lower > consumption > > rates, because our present rates are unsustainable. > > > > > > > > On Mon, Sep 12, 2011 at 7:07 PM, Joseph Norton < > norton@REDACTED> > > wrote: > >> > >> The basic goal is to build and test a production worthy replacement for > >> ets with disk based storage. After I have the basics in place, I can > >> elaborate further. > >> > >> thanks, > >> > >> Joseph Norton > >> > >> On Sep 13, 2011, at 9:55 AM, OvermindDL1 wrote: > >> > >> > On Mon, Sep 12, 2011 at 9:46 AM, Joseph Wayne Norton > >> > wrote: > >> >> > >> >> I'm currently developing a disk based clone of the ets application. > >> > > >> > Can you elaborate how this is different from dets, or are you doing it > >> > for learning? > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Sat Sep 17 12:23:23 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 17 Sep 2011 11:23:23 +0100 Subject: [erlang-questions] Erlang and beam-based scripting languages. In-Reply-To: References: Message-ID: > > Yes, you still need to type "make" after you save the changes, before you > actually hit the service on the web, but you can also just bind a macro to > "save and make" :-) The benefit of this is that, if you have a syntax error, > you don't have to poke at the application through a web browser to find it; > you will be told by the compiler immediately. > I was using https://github.com/n0gg1n/devenv/blob/master/erlcd for a while to avoid the `make' step. I might rewrite it as an escript some time though. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sat Sep 17 12:40:46 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 17 Sep 2011 14:40:46 +0400 Subject: [erlang-questions] Erlang and beam-based scripting languages. In-Reply-To: References: Message-ID: Look. I can do "make", "rebar compile", "cap deploy" or whatever else. Question is: what can do a low-priced deploy engineer, that will be hired by my customer? How does it raise TCO of my product? I think (and I will check during following year) that using of ruby-like scripting will make it easier and cheaper. Scripting, that works without any deploy commands. From jesper.louis.andersen@REDACTED Sat Sep 17 15:07:54 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 17 Sep 2011 15:07:54 +0200 Subject: [erlang-questions] Is it possible to align binary's byte array to cache line boundary? In-Reply-To: References: Message-ID: On Thu, Sep 15, 2011 at 09:21, max tan wrote: > It seems not supported in current VM, right? If it is supported, it will be automatic and done for all binaries. Take a look at the allocator used for allocation of binaries, since it is a different one than the one used for allocation of process heaps. Why do you need this guarantee? -- J. From maxtqm@REDACTED Sat Sep 17 18:30:48 2011 From: maxtqm@REDACTED (max tan) Date: Sun, 18 Sep 2011 00:30:48 +0800 Subject: [erlang-questions] Is it possible to align binary's byte array to cache line boundary? In-Reply-To: References: Message-ID: > > On Thu, Sep 15, 2011 at 09:21, max tan wrote: > > It seems not supported in current VM, right? > > If it is supported, it will be automatic and done for all binaries. > Take a look at the allocator used for allocation of binaries, since it > is a different one than the one used for allocation of process heaps. I saw in BEAM's sources, that when the binary's size is 64 bytes or less, BEAM will allocate a ErlHeapBin on local heap, in "erl_binary.h": typedef struct erl_heap_bin { Eterm thing_word; /* Subtag HEAP_BINARY_SUBTAG. */ Uint size; /* Binary size in bytes. */ Eterm data[1]; /* The data in the binary. */ } ErlHeapBin; in "binary.c": /* * Create a brand new binary from scratch. */ Eterm new_binary(Process *p, byte *buf, Uint len) { ProcBin* pb; Binary* bptr; if (len <= ERL_ONHEAP_BIN_LIMIT) { ErlHeapBin* hb = (ErlHeapBin *) HAlloc(p, heap_bin_size(len)); hb->thing_word = header_heap_bin(len); hb->size = len; if (buf != NULL) { sys_memcpy(hb->data, buf, len); } return make_binary(hb); } ................ So, I can be sure that BEAM will align ProcBin instead of binary "data". When len <= ( cache_line_size - sizeof(Eterm) - sizeof(Uint) ), it's OK, the whole ProcBin can be fit in 1 cache line. Otherwise, for example when len = cache_line_size, the whole ProcBin will surely span 2 or more cache lines. > > Why do you need this guarantee? > For example, my application reads/creates/stores/sends binaries of 64 bytes repeatedly, if I can be sure that the binaries is allocated along cache line boundary ( which is normally 64/128 byte multiplied ), each processing will need just 1 memory fetch, so cache-friendly, so I can maximize the processing throughput. > > -- > J. From jesper.louis.andersen@REDACTED Sat Sep 17 22:50:45 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 17 Sep 2011 22:50:45 +0200 Subject: [erlang-questions] Is it possible to align binary's byte array to cache line boundary? In-Reply-To: References: Message-ID: On Sat, Sep 17, 2011 at 18:30, max tan wrote: > I saw in BEAM's sources, that when the binary's size is 64 bytes or less, > BEAM will allocate a ErlHeapBin on local heap, If we limit ourselves to small binaries only, everything will be heap allocated. The heap is garbage collected, and the size of the heap will govern what kind of garbage collector there is in play here. If the heap is small enough, all of it will be in cache and there will be little penalty for pulling it in from the first level cache, one or two cache lines needed. Garbage collectors can some times outperform a manual memory management routine, especially if you think about how much garbage you create in your application. Unfortunately, I only have fairly old papers which study the impact of garbage collectors quo malloc()/free() style implementations. I dare not think they still hold these days with multiple cores, cache coherency algorithms, "shared memory" and new cache sizes. But for small heaps with few objects living and most objects dead, all GCs will be in cache and run fast as a result. Also note that messaging your binary is a 64 byte copy (+ its header and other things - chances are that it can't fit into a single cache line in the first place!). So if your binaries are small, I'd rather worry about other things than cahce alignment, since it is then in the hands of the GC. I think that the mantra of Erlang is "mod out" such details in the program and let the underlying VM handle it. If it proves to be too slow in practice, you can always outsource the heavy work to a NIF, in which you can get more explicit control over memory layout and use. In my experience though, the binaries of Erlang have a performance that is not too shabby to put it mildly. Depending on the problem you can often frame it in ways that will lead to less binary allocation and work - that is optimize algorithmically what is going on. What I'd suggest is that you quantify how fast you need your operations to be and then go measure and optimize if you can reach that goal - and how many cores and nodes you need to reach it at that speed. This will guide you if you need to worry about cache alignment performance in the GC. Erlang is focused on being robust first, correct second and speedy third. My experience is that it is often _fast enough_ but it can't compete in raw CPU-bound kernel performance besides Ocaml, Java, C or C++. But then you throw robustness overboard. -- J. From bookjovi@REDACTED Sun Sep 18 09:10:43 2011 From: bookjovi@REDACTED (Jovi Zhang) Date: Sun, 18 Sep 2011 15:10:43 +0800 Subject: [erlang-questions] How erlang handle complicated data structure like C? Message-ID: Hi, Is there any good way to handle complicated data structure like C in Erlang? struct packet_t { int a, double b, struct xx_t, union c {int d, char e}, char f:1, char g:3, char h:5 } Some protocol have complicated data structure like above, how to represent those structure in Erlang? and operate on each field freely like C? .jovi From cgsmcmlxxv@REDACTED Sun Sep 18 09:17:07 2011 From: cgsmcmlxxv@REDACTED (George Catalin Serbanut) Date: Sun, 18 Sep 2011 09:17:07 +0200 Subject: [erlang-questions] How erlang handle complicated data structure like C? In-Reply-To: References: Message-ID: Hi Jovi Nested records (http://www.erlang.org/doc/reference_manual/records.html). CGS On Sun, Sep 18, 2011 at 9:10 AM, Jovi Zhang wrote: > Hi, > Is there any good way to handle complicated data structure like C in > Erlang? > > struct packet_t { > int a, > double b, > struct xx_t, > union c {int d, char e}, > char f:1, > char g:3, > char h:5 > } > > Some protocol have complicated data structure like above, how to > represent those structure in Erlang? and operate on each field freely > like C? > > .jovi > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sun Sep 18 14:17:01 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 18 Sep 2011 14:17:01 +0200 Subject: [erlang-questions] How erlang handle complicated data structure like C? In-Reply-To: References: Message-ID: On Sun, Sep 18, 2011 at 09:10, Jovi Zhang wrote: > Hi, > ? ?Is there any good way to handle complicated data structure like C in Erlang? > > ? ?struct packet_t { > ? ? ? ?int a, > ? ? ? ?double b, > ? ? ? ?struct xx_t, > ? ? ? ?union c {int d, char e}, > ? ? ? ?char f:1, > ? ? ? ?char g:3, > ? ? ? ?char h:5 > ? ?} > > ? ?Some protocol have complicated data structure like above, how to > represent those structure in Erlang? and operate on each field freely > like C? Your tools are: Binary Pattern Matches. The above can be matched with something like: case Packet of <> -> S_XX = decode(XX), % Assume you have a separate decoding function for this beast {A, B, S_XX, {d, D}, F, G, H}; <> -> S_XX = decode(XX), {A, B, S_XX, {e, E}, F, G, H} end It probably won't work as expected as the bit size doesn't match up to a multiple of 8. Also note that some protocols use a length field early to specify the size of later data, which you can also do. Using the binary as an expression and not as a match will enable you to construct the struct. Records, possibly nested. The idea here is that you are taking an external packet representation apart and converting it to an internal form. The internal form are usually not a 8-element tuple as above, but via using a (nested) record as George C. Serbanut recommends. In C, you keep, more or less, an 1-to-1 representation to what is on the wire (beware of the packed attribute!). In Erlang you decode the packets external representation to an internal one and then proceed to use the internal representation in your code. When you want to output the internal representation, you render it into binary() form and shove that out over the wire. . . . Structures can be more complicated than this. If the structure size is statically known and you can break your input up in the right chunk size, the above works. If the size is dynamic, you will have to write code which scrutinizes a partial decode to learn how to decode the rest. Finally, if there are more packets coming in on, say, the same TCP stream, you may want to process it recursively by adding a <<...., Rest/binary>> field to the match and then handle the Rest of the input in another call. For TCP streams you may also need some buffering to make sure you have a full packet to decode, but this isn't something new compared to C. -- J. From rapsey@REDACTED Sun Sep 18 14:40:59 2011 From: rapsey@REDACTED (Rapsey) Date: Sun, 18 Sep 2011 14:40:59 +0200 Subject: [erlang-questions] How erlang handle complicated data structure like C? In-Reply-To: References: Message-ID: Your options are: propery lists and records. Records are faster, propery lists are more flexible. Sergej On Sun, Sep 18, 2011 at 9:10 AM, Jovi Zhang wrote: > Hi, > Is there any good way to handle complicated data structure like C in > Erlang? > > struct packet_t { > int a, > double b, > struct xx_t, > union c {int d, char e}, > char f:1, > char g:3, > char h:5 > } > > Some protocol have complicated data structure like above, how to > represent those structure in Erlang? and operate on each field freely > like C? > > .jovi > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From silent_vendetta@REDACTED Sun Sep 18 18:49:02 2011 From: silent_vendetta@REDACTED (Chris Hicks) Date: Sun, 18 Sep 2011 09:49:02 -0700 Subject: [erlang-questions] escript question Message-ID: Quick question, what would be considered the easiest way to "return" data to a running Erlang application from an escript? For example I might want to run an escript with a data structure I want it to modify and, once it is all done, get the modified structure back. -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Sun Sep 18 19:44:24 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Sun, 18 Sep 2011 18:44:24 +0100 Subject: [erlang-questions] escript question In-Reply-To: References: Message-ID: 1. write it to a file and use file:consult/1 to grab the result. 2. write it back to stdout and use erl_parse/erl_eval 3. use emu_args to get the escript hooked into the distribution protocol and use rcp:call to send the result back Not actually sure about (3) as I've never done that, but the other two work fine and are simple to implement. Cheers, Tim On 18 September 2011 17:49, Chris Hicks wrote: > Quick question, what would be considered the easiest way to "return" data > to a running Erlang application from an escript? For example I might want to > run an escript with a data structure I want it to modify and, once it is all > done, get the modified structure back. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From silent_vendetta@REDACTED Sun Sep 18 19:57:24 2011 From: silent_vendetta@REDACTED (Chris Hicks) Date: Sun, 18 Sep 2011 10:57:24 -0700 Subject: [erlang-questions] escript question In-Reply-To: References: , Message-ID: Thanks for the response, Tim. The first two were along the lines of what I was thinking I'd need to do, though I didn't know about file:consult/1 which could come in very handy for a couple other things I'm working on. I'm leaning a bit more towards #1 though simply because in the system I'm building there could be anywhere from dozens to hundreds of these scripts being fired off each second and, without having spent much time thinking about it, I imagine trying to filter the appropriate structure back to the appropriate process which fired off the escript via method #2 would just add some unnecessary complexity. Date: Sun, 18 Sep 2011 18:44:24 +0100 Subject: Re: [erlang-questions] escript question From: watson.timothy@REDACTED To: silent_vendetta@REDACTED CC: erlang-questions@REDACTED 1. write it to a file and use file:consult/1 to grab the result.2. write it back to stdout and use erl_parse/erl_eval3. use emu_args to get the escript hooked into the distribution protocol and use rcp:call to send the result back Not actually sure about (3) as I've never done that, but the other two work fine and are simple to implement. Cheers, Tim On 18 September 2011 17:49, Chris Hicks wrote: Quick question, what would be considered the easiest way to "return" data to a running Erlang application from an escript? For example I might want to run an escript with a data structure I want it to modify and, once it is all done, get the modified structure back. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From silent_vendetta@REDACTED Sun Sep 18 23:15:01 2011 From: silent_vendetta@REDACTED (Chris Hicks) Date: Sun, 18 Sep 2011 14:15:01 -0700 Subject: [erlang-questions] escript question In-Reply-To: References: , , , Message-ID: I was thinking about this some more, and doing some more reading, and if I have a process which uses os:cmd on a unix box to run the escript and if doing so starts up a full Erlang node, is there any reason that the Pid of the calling process couldn't be passed as one of the arguments? Couldn't the script simply send a message to that Pid and once the script exits and returns a value to the process that called os:cmd, that process would then do a receive to retrieve the message? On a related note, can an escript be run as a text string instead of a file residing on the system? Basically, I'm trying to figure out if I can just hold all my scripts in a DB, pull them out as needed and execute them rather than actually creating a file that is run. From: silent_vendetta@REDACTED To: watson.timothy@REDACTED Date: Sun, 18 Sep 2011 10:57:24 -0700 CC: erlang-questions@REDACTED Subject: Re: [erlang-questions] escript question Thanks for the response, Tim. The first two were along the lines of what I was thinking I'd need to do, though I didn't know about file:consult/1 which could come in very handy for a couple other things I'm working on. I'm leaning a bit more towards #1 though simply because in the system I'm building there could be anywhere from dozens to hundreds of these scripts being fired off each second and, without having spent much time thinking about it, I imagine trying to filter the appropriate structure back to the appropriate process which fired off the escript via method #2 would just add some unnecessary complexity. Date: Sun, 18 Sep 2011 18:44:24 +0100 Subject: Re: [erlang-questions] escript question From: watson.timothy@REDACTED To: silent_vendetta@REDACTED CC: erlang-questions@REDACTED 1. write it to a file and use file:consult/1 to grab the result.2. write it back to stdout and use erl_parse/erl_eval3. use emu_args to get the escript hooked into the distribution protocol and use rcp:call to send the result back Not actually sure about (3) as I've never done that, but the other two work fine and are simple to implement. Cheers, Tim On 18 September 2011 17:49, Chris Hicks wrote: Quick question, what would be considered the easiest way to "return" data to a running Erlang application from an escript? For example I might want to run an escript with a data structure I want it to modify and, once it is all done, get the modified structure back. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Mon Sep 19 06:22:54 2011 From: jwatte@REDACTED (Jon Watte) Date: Sun, 18 Sep 2011 21:22:54 -0700 Subject: [erlang-questions] Erlang and beam-based scripting languages. In-Reply-To: References: Message-ID: I don't get it. You will never edit code that's running live, right? And you need to arrange to make the update at a minimum be atomic, so that modules that depend on other modules are all upgraded together, else you'll get random runtime errors while upgrading. Thus, you need some way to get code out of your source control, and into production, in an atomic way. That needs to be some kind of script. Having that script also do a "make" first doesn't seem like it will cost anything. So, where is the cost of "make"? I don't see it. Are you saying that typing "make" to run tests and check your source code for problems is somehow a less good use of time for a maintenance engineer than for a regular engineer? Are you saying that all sites out there built in Java (another build-based compilation model), or ASP.NET (another build-based compilation model) are somehow doomed? All large-scale Ruby sites where I have insight (as well as PHP; another interpreter-based web services language) would dearly love to be able to catch errors through a "make" rather than have to poke at the code with HTTP requests. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Sat, Sep 17, 2011 at 3:40 AM, Max Lapshin wrote: > Look. I can do "make", "rebar compile", "cap deploy" or whatever else. > > Question is: what can do a low-priced deploy engineer, that will be > hired by my customer? > How does it raise TCO of my product? > > > I think (and I will check during following year) that using of > ruby-like scripting will make it easier and cheaper. > Scripting, that works without any deploy commands. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Mon Sep 19 06:33:13 2011 From: jwatte@REDACTED (Jon Watte) Date: Sun, 18 Sep 2011 21:33:13 -0700 Subject: [erlang-questions] escript question In-Reply-To: References: Message-ID: How about binary_to_term and term_to_binary? Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Sun, Sep 18, 2011 at 9:49 AM, Chris Hicks wrote: > Quick question, what would be considered the easiest way to "return" data > to a running Erlang application from an escript? For example I might want to > run an escript with a data structure I want it to modify and, once it is all > done, get the modified structure back. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bookjovi@REDACTED Mon Sep 19 07:09:38 2011 From: bookjovi@REDACTED (Jovi Zhang) Date: Mon, 19 Sep 2011 13:09:38 +0800 Subject: [erlang-questions] How erlang handle complicated data structure like C? In-Reply-To: References: Message-ID: But how to change record format to binary correctly? because the binary need to transport in network. When I try below, the output seems wrong. -record(semper_packet_t, {tdm_leg = <<1:8, 3:8>>, ip_leg = 2 }). term_to_binary(#semper_packet_t.tdm_leg) result: <<131,97,2>> From masklinn@REDACTED Mon Sep 19 07:39:36 2011 From: masklinn@REDACTED (Masklinn) Date: Mon, 19 Sep 2011 07:39:36 +0200 Subject: [erlang-questions] How erlang handle complicated data structure like C? In-Reply-To: References: Message-ID: On 19 sept. 2011, at 07:09, Jovi Zhang wrote: > But how to change record format to binary correctly? because the > binary need to transport in network. > When I try below, the output seems wrong. > > -record(semper_packet_t, > {tdm_leg = <<1:8, 3:8>>, > ip_leg = 2 > }). > > term_to_binary(#semper_packet_t.tdm_leg) > > result: <<131,97,2>> you need an encoder and a decoder, to and from your binary wire format. You don't generally put raw binary data in a record, and there is just about no way erlang's binary packing of its terms (which exists for e<->e exchanges) is going to match a struct defined by an arbitrary third party. From cgsmcmlxxv@REDACTED Mon Sep 19 08:26:41 2011 From: cgsmcmlxxv@REDACTED (George Catalin Serbanut) Date: Mon, 19 Sep 2011 08:26:41 +0200 Subject: [erlang-questions] How erlang handle complicated data structure like C? In-Reply-To: References: Message-ID: Considering all variables integers, here is an example how to play with records: 1. Record definition: -record(tdm_leg,{a,b}). -record(semper_packet_t,{tdm_leg = #tdm_leg{},ip_leg}). -define(SEPARATOR,"--/--"). 2. Inserting values: insert_values(A,B,C) -> Semper_packet_t = #semper_packet_t{#tdm_leg{a=A,b=B},ip_leg=C}. 3. Read values: read_values(Semper_packet_t = #semper{Tdm_leg = tdm_leg#tdm_leg{a=A,b=B},ip_leg=C}) -> do_something_with(Semper_packet_t,Tdm_leg,A,B,C). 4. Encoder: encode(#semper{#tdm_leg{a=A,b=B},ip_leg=C}) -> list_to_binary(io_lib:format("~p~s~p~s~p",[A,?SEPARATOR,B,?SEPARATOR,C])). 5. Decoder: decode(Binary) -> [A,B,C] = re:split(binary_to_list(Binary),?SEPARATOR,[{return,list}]), Semper_packet_t = #semper_packet_t{#tdm_leg{a=list_to_integer(A),b=list_to_interger(B)},ip_leg=list_to_integer(C)}. 6. Binary tester: is_my_record(Binary) -> L = re:split(binary_to_list(Binary),?SEPARATOR,[{return,list}]), if length(L) == 3 -> Result = lists:all(fun(E) -> TL = string:join(string:tokens(E,"0123456789"),""), if length(TL) == 0 -> Result = true; length(TL) /= 0 -> Result = false end, Result end,L), length(L) /= 3 -> Result = false, end, Result. 7. How to use the test: receive_function(Packet) when is_my_record(Packet) == true -> do_something_with_my_packet; receive_function(Packet) when is_my_record(Packet) /= true -> I_do_not_care. Take this an example only. There are other options more elegant. For the moment I created something which is easily understandable for someone coming from other programming languages. Have fun with Erlang! ;) CGS On Mon, Sep 19, 2011 at 7:39 AM, Masklinn wrote: > On 19 sept. 2011, at 07:09, Jovi Zhang wrote: > > But how to change record format to binary correctly? because the > > binary need to transport in network. > > When I try below, the output seems wrong. > > > > -record(semper_packet_t, > > {tdm_leg = <<1:8, 3:8>>, > > ip_leg = 2 > > }). > > > > term_to_binary(#semper_packet_t.tdm_leg) > > > > result: <<131,97,2>> > you need an encoder and a decoder, to and from your binary wire format. You > don't generally put raw binary data in a record, and there is just about no > way erlang's binary packing of its terms (which exists for e<->e exchanges) > is going to match a struct defined by an arbitrary third party. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wiener.guy@REDACTED Mon Sep 19 10:14:21 2011 From: wiener.guy@REDACTED (Guy Wiener) Date: Mon, 19 Sep 2011 11:14:21 +0300 Subject: [erlang-questions] Erlang on ARM9? Message-ID: Hello Erlangers, I am checking an option to use Erlang to control a Linux-based device (a programmable helicopter). Is there an efficient port of Erlang that works on top of an ARM9 processor? If so, the usage scenario will be many processes running on a single node, with frequent message-passing, some access to binary files, and no shared data (no ETS, DETS or Mnesia). Is this scenario likely to perform well on an embedded Erlang? Thanks, Guy Wiener, Dept. of Computer Science and Applied Math, Weizmann Institute for Science. -------------- next part -------------- An HTML attachment was scrubbed... URL: From banibrata.dutta@REDACTED Mon Sep 19 10:17:33 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Mon, 19 Sep 2011 13:47:33 +0530 Subject: [erlang-questions] Erlang on ARM9? In-Reply-To: References: Message-ID: On Mon, Sep 19, 2011 at 1:44 PM, Guy Wiener wrote: > I am checking an option to use Erlang to control a Linux-based device (a > programmable helicopter). Is there an efficient port of Erlang that works > on top of an ARM9 processor? > Here's a part answer. http://www.erlang-embedded.com/ > If so, the usage scenario will be many processes running on a single node, > with frequent message-passing, some access to binary files, and no shared > data (no ETS, DETS or Mnesia). Is this scenario likely to perform well on > an embedded Erlang? > > -- regards, Banibrata http://www.linkedin.com/in/bdutta http://twitter.com/edgeliving -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Mon Sep 19 11:17:11 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 19 Sep 2011 10:17:11 +0100 Subject: [erlang-questions] escript question In-Reply-To: References: Message-ID: On 18 September 2011 22:15, Chris Hicks wrote: > I was thinking about this some more, and doing some more reading, and if I > have a process which uses os:cmd on a unix box to run the escript and if > doing so starts up a full Erlang node, is there any reason that the Pid of > the calling process couldn't be passed as one of the arguments? Couldn't the > script simply send a message to that Pid and once the script exits and > returns a value to the process that called os:cmd, that process would then > do a receive to retrieve the message? > As I said previously, I'm not 100% sure about escripts being part of a distributed system, but check it out and see if it works. In terms of passing on the pid, as Jon mentioned you should probably pass on a globally registered name, which you can handle with either the global module or https://github.com/esl/gproc (I favour the latter personally). I'd also suggest rather than using os:cmd, that you consider erlang:open_port so that you can check the return code to see whether the script executed successfully or not. I'll post an example at the bottom of this main. > > On a related note, can an escript be run as a text string instead of a file > residing on the system? Basically, I'm trying to figure out if I can just > hold all my scripts in a DB, pull them out as needed and execute them rather > than actually creating a file that is run. > Yes sort of. The escript module has functions to handle this kind of thing, for example from http://www.erlang.org/doc/man/escript.html: {ok, Bin} = escript:create(binary, [shebang, comment, {emu_args, "-smp disable"}, {source, list_to_binary(Source)}]), %% {ok,<<"#!/usr/bin/env escript\n%% This is an -*- erlang -*- file\n%%!-smp disabl"...>>} file:write_file("demo.escript", Bin). You can also zip together a bunch of module beam files to form into an escript, which allows you to guarantee modules will be loaded by the code server when the script is executed. Rebar provides support for doing this as part of a build process: take a look at https://github.com/basho/rebar/blob/master/src/rebar_escripter.erl to see how it works. Using a mechanism like this, you could store a blob containing the code and pull this out at runtime, optionally modify it and then write it to a temporary file location prior to executing the script(s). To get feedback from the executable, you can do something like this quick, not production worthy, hack: exec(Command) -> exec(Command, []). exec(Command, Env) -> PortSettings = [exit_status, {line, 16384}, stderr_to_stdout, hide], sh_loop(open_port({spawn, Command}, PortSettings ++ Env), []). sh_loop(Port, Acc) -> receive {Port, {data, {eol, Line}}} -> sh_loop(Port, [Line ++ "\n" | Acc]); {Port, {data, {noeol, Line}}} -> sh_loop(Port, [Line | Acc]); {Port, {exit_status, 0}} -> {ok, lists:flatten(lists:reverse(Acc))}; {Port, {exit_status, Rc}} -> {error, {Rc, lists:flatten(lists:reverse(Acc))}} end. Cheers, Tim ------------------------------ > From: silent_vendetta@REDACTED > To: watson.timothy@REDACTED > Date: Sun, 18 Sep 2011 10:57:24 -0700 > > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] escript question > > Thanks for the response, Tim. The first two were along the lines of what I > was thinking I'd need to do, though I didn't know about file:consult/1 which > could come in very handy for a couple other things I'm working on. I'm > leaning a bit more towards #1 though simply because in the system I'm > building there could be anywhere from dozens to hundreds of these scripts > being fired off each second and, without having spent much time thinking > about it, I imagine trying to filter the appropriate structure back to the > appropriate process which fired off the escript via method #2 would just add > some unnecessary complexity. > > ------------------------------ > Date: Sun, 18 Sep 2011 18:44:24 +0100 > Subject: Re: [erlang-questions] escript question > From: watson.timothy@REDACTED > To: silent_vendetta@REDACTED > CC: erlang-questions@REDACTED > > 1. write it to a file and use file:consult/1 to grab the result. > 2. write it back to stdout and use erl_parse/erl_eval > 3. use emu_args to get the escript hooked into the distribution protocol > and use rcp:call to send the result back > > Not actually sure about (3) as I've never done that, but the other two work > fine and are simple to implement. > > Cheers, > > Tim > > On 18 September 2011 17:49, Chris Hicks wrote: > > Quick question, what would be considered the easiest way to "return" data > to a running Erlang application from an escript? For example I might want to > run an escript with a data structure I want it to modify and, once it is all > done, get the modified structure back. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ erlang-questions mailing > list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Mon Sep 19 15:12:02 2011 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Mon, 19 Sep 2011 08:12:02 -0500 Subject: [erlang-questions] Erlang on ARM9? In-Reply-To: References: Message-ID: I think that is cortex only, not ARM9 (correct me if I'm wrong). I've built a working Erlang for a cortex-a8 before using vanilla sources; no special port was required. I only tested it very briefly, but it seemed to work well. I can't think of any reason why it wouldn't also work on an ARM9. I built it from a Gentoo stage3 chroot on the target platform. Browsing the gentoo mirrors, I see that stage3 tarballs are available for armv4, v5 (ARM926), v6, and v7(cortex). I have also run Erlang on a 100MHz x86. It was quite slow to start, but once up and running it was fine. I suspect the slowness was due more to filesystem and io problems than Erlang itself. Just wanted to give this datapoint because I know ARM9 devices operate in the low 100's of MHz. Cheers, Dan. On Mon, Sep 19, 2011 at 3:17 AM, Banibrata Dutta wrote: > On Mon, Sep 19, 2011 at 1:44 PM, Guy Wiener wrote: > >> I am checking an option to use Erlang to control a Linux-based device (a >> programmable helicopter). Is there an efficient port of Erlang that works >> on top of an ARM9 processor? >> > > Here's a part answer. > http://www.erlang-embedded.com/ > > >> If so, the usage scenario will be many processes running on a single >> node, with frequent message-passing, some access to binary files, and no >> shared data (no ETS, DETS or Mnesia). Is this scenario likely to perform >> well on an embedded Erlang? >> >> > -- > regards, > Banibrata > http://www.linkedin.com/in/bdutta > http://twitter.com/edgeliving > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Mon Sep 19 16:22:01 2011 From: g@REDACTED (Garrett Smith) Date: Mon, 19 Sep 2011 09:22:01 -0500 Subject: [erlang-questions] escript question In-Reply-To: References: Message-ID: On Sun, Sep 18, 2011 at 4:15 PM, Chris Hicks wrote: > I was thinking about this some more, and doing some more reading, and if I > have a process which uses os:cmd on a unix box to run the escript and if > doing so starts up a full Erlang node, is there any reason that the Pid of > the calling process couldn't be passed as one of the arguments? Couldn't the > script simply send a message to that Pid and once the script exits and > returns a value to the process that called os:cmd, that process would then > do a receive to retrieve the message? > > On a related note, can an escript be run as a text string instead of a file > residing on the system? Basically, I'm trying to figure out if I can just > hold all my scripts in a DB, pull them out as needed and execute them rather > than actually creating a file that is run. By running as an escript, you're forcing yourself to go through stdio. IMO the only reason you'd want to do this would be run the escript from a shell or other programs. If all you want to do is ship around your code to be used dynamically, check out erl_parse and erl_eval (as suggested by Tim). E.g. store a function in your database that increments a number by 1: FStr = "fun(N) -> N + 1 end." When you're ready to use it: {ok, Tokens, _} = erl_scan:string(FStr), {ok, Exprs} = erl_parse:parse_exprs(Tokens), {value, F, []} = erl_eval:exprs(Exprs, []). Now: 2 = F(1). From ulf.wiger@REDACTED Mon Sep 19 16:37:53 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 19 Sep 2011 16:37:53 +0200 Subject: [erlang-questions] escript question In-Reply-To: References: Message-ID: <9469322B-7150-4167-9AD1-704E9B114E8E@erlang-solutions.com> On 19 Sep 2011, at 16:22, Garrett Smith wrote: > > E.g. store a function in your database that increments a number by 1: > > FStr = "fun(N) -> N + 1 end." > > When you're ready to use it: > > {ok, Tokens, _} = erl_scan:string(FStr), > {ok, Exprs} = erl_parse:parse_exprs(Tokens), > {value, F, []} = erl_eval:exprs(Exprs, []). > > Now: > > 2 = F(1). I'd go one step further and store Exprs in the database directly, eliminating the parsing step. To avoid having to learn the abstract form representation, you can use parse_trans_codegen.erl. http://github.com/esl/parse_trans Example: -module(storefun). -export([f/1]). -include_lib("parse_trans/include/codegen.hrl"). f(1) -> codegen:exprs(fun() -> fun(N) -> N + 1 end end). Compiling this with parse_trans/ebin in the path: Eshell V5.8.4 (abort with ^G) 1> c(storefun). {ok,storefun} 2> storefun:f(1). [{'fun',8, {clauses,[{clause,8, [{var,8,'N'}], [], [{op,9,'+',{var,9,'N'},{integer,9,1}}]}]}}] 3> {value, F, []} = erl_eval:exprs(v(2),[]). {value,#Fun,[]} 4> F(2). 3 That is, codegen:exprs(fun() -> end) returns the abstract form list for . If you want the expressions to inherit variables, you can do that by defining those variables as arguments to the enclosing fun: f(2) -> codegen:exprs(fun(X) -> fun(N) -> N + X end end). 5> c(storefun). {ok,storefun} 6> storefun:f(2). [{'fun',14, {clauses,[{clause,14, [{var,14,'N'}], [], [{op,15,'+',{var,15,'N'},{var,15,'X'}}]}]}}] 7> {value, F2, []} = erl_eval:exprs(v(6),[]). ** exception error: {unbound_var,'X'} 8> {value, F2, []} = erl_eval:exprs(v(6),[{'X',17}]). ** exception error: no match of right hand side value {value,#Fun,[{'X',17}]} 9> {value, F2, _} = erl_eval:exprs(v(6),[{'X',17}]). {value,#Fun,[{'X',17}]} 10> F2(3). 20 The abstract code is generated at compile-time, so there is very little overhead. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From g@REDACTED Mon Sep 19 16:40:04 2011 From: g@REDACTED (Garrett Smith) Date: Mon, 19 Sep 2011 09:40:04 -0500 Subject: [erlang-questions] Erlang and beam-based scripting languages. In-Reply-To: References: Message-ID: On Sat, Sep 17, 2011 at 5:40 AM, Max Lapshin wrote: > Look. I can do "make", "rebar compile", "cap deploy" or whatever else. > > Question is: what can do a low-priced deploy engineer, that will be > hired by my customer? > How does it raise TCO of my product? > > > I think (and I will check during following year) that using of > ruby-like scripting will make it easier and cheaper. > Scripting, that works without any deploy commands. I can see where this would be desirable. PHP enjoys this natively. Python, with some trivial dynamic reloading, etc. I don't think you need to go as far as using a different language though. If had enough incentive to go write (or wait for) a different language for this application, here's what I'd try first: - Reconceive of "escript" files as your unit of dynamic code, replacing the "main" function with whatever makes sense for your application (e.g. a web request handler) - Use Erlang's excellent scan/parse/eval tool chain to dynamically load the file's list of functions and call your entry point You could do this with dynamically loaded functions as well, ala erlydtl. This would be an application specific framework that you provided your customers, rather than a generic solution. But I think it has some advantages over reimplimenting the entire language: - You avoid reimplementing the entire language :) It has the disadvantage of shoe-horning your users into a limited framework, but it sounds like this might be what you want anyway. You'd have to provide a tool to verify your customer's code. But not hard. Garrett From max.lapshin@REDACTED Mon Sep 19 16:47:20 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 19 Sep 2011 18:47:20 +0400 Subject: [erlang-questions] Erlang and beam-based scripting languages. In-Reply-To: References: Message-ID: Ok, maybe I'm wrong with idea of using some syntax-sugar on top of erlang, but the idea of not-using make is simplifying administration. When I deploy server with chef, I want to put some files into "/etc/erlyvideo" and run "apt-get install erlyvideo" If erlyvideo takes these files and compile them at runtime, they are just loaded. I've already tested and it showed to be rather useful. Yes: there are no atomic updates. If I break module, while editing, it will be not accessible. But the same problem will be with editing erlang files: if you load syntax-right, but semantic-wrong file, users will be served in wrong way. My first results, show that autoreloader in production seems to work nice. If it will make my life a horror, I will remove it =)) From ericbmerritt@REDACTED Mon Sep 19 16:56:58 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Mon, 19 Sep 2011 09:56:58 -0500 Subject: [erlang-questions] Escript and multiple OTP application archives (*.ez) or handling dependencies in an escript Message-ID: Hey guys, I am looking at escript and wondering how to go about distributing more then one OTP app in an escript. That is, a project and it's dependencies. From the documentation and tests on my part that validate that it looks like escript:create can have one of an escript or a code archive (one and only one) as its body. From reading the docs on code archives an archive may contain one and only one OTP Application and be named for that OTP Application. Unless I am missing something, which could be very true, it looks like an escript must conform to one of the following. - All dependencies must be on the users box (ie part of the erlang distribution) - All dependencies must be copied into a single OTP Application I am hoping that I am wrong on this as these constraints are a problem. Eric From cgsmcmlxxv@REDACTED Mon Sep 19 17:08:56 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Mon, 19 Sep 2011 17:08:56 +0200 Subject: [erlang-questions] Escript and multiple OTP application archives (*.ez) or handling dependencies in an escript In-Reply-To: References: Message-ID: Hi, I am not expert, but a simple solution would be to create an application which can call other applications. You can call at any time application:start(...) inside your application. Cheers, CGS On Mon, Sep 19, 2011 at 4:56 PM, Eric Merritt wrote: > Hey guys, > > I am looking at escript and wondering how to go about distributing > more then one OTP app in an escript. That is, a project and it's > dependencies. > > From the documentation and tests on my part that validate that it > looks like escript:create can have one of an escript or a code archive > (one and only one) as its body. From reading the docs on code archives > an archive may contain one and only one OTP Application and be named > for that OTP Application. > > Unless I am missing something, which could be very true, it looks like > an escript must conform to one of the following. > > - All dependencies must be on the users box (ie part of the erlang > distribution) > - All dependencies must be copied into a single OTP Application > > I am hoping that I am wrong on this as these constraints are a problem. > > Eric > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hm@REDACTED Mon Sep 19 17:13:29 2011 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Mon, 19 Sep 2011 17:13:29 +0200 Subject: [erlang-questions] Escript and multiple OTP application archives (*.ez) or handling dependencies in an escript In-Reply-To: References: Message-ID: On Mon, Sep 19, 2011 at 4:56 PM, Eric Merritt wrote: > Hey guys, > > ?I am looking at escript and wondering how to go about distributing > more then one OTP app in an escript. That is, a project and it's > dependencies. > > ?From the documentation and tests on my part that validate that it > looks like escript:create can have one of an escript or a code archive > (one and only one) as its body. From reading the docs on code archives > an archive may contain one and only one OTP Application and be named > for that OTP Application. No, you can have several applications in an escript archive. It is just the .ez files that are limited to contain one application. /H?kan From dizzyd@REDACTED Mon Sep 19 17:20:16 2011 From: dizzyd@REDACTED (Dave Smith) Date: Mon, 19 Sep 2011 09:20:16 -0600 Subject: [erlang-questions] Escript and multiple OTP application archives (*.ez) or handling dependencies in an escript In-Reply-To: References: Message-ID: FWIW, basho_bench does distribute multiple apps in the same .ez. You can see the config for rebar: https://github.com/basho/basho_bench/blob/master/rebar.config has a {escript_incl_apps, [...]} config item. It's a bit of a hack, but it works. :) D. 2011/9/19 H?kan Mattsson : > On Mon, Sep 19, 2011 at 4:56 PM, Eric Merritt wrote: >> Hey guys, >> >> ?I am looking at escript and wondering how to go about distributing >> more then one OTP app in an escript. That is, a project and it's >> dependencies. >> >> ?From the documentation and tests on my part that validate that it >> looks like escript:create can have one of an escript or a code archive >> (one and only one) as its body. From reading the docs on code archives >> an archive may contain one and only one OTP Application and be named >> for that OTP Application. > > No, you can have several applications in an escript archive. > It is just the .ez files that are limited to contain one application. > > /H?kan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From hm@REDACTED Mon Sep 19 17:29:22 2011 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Mon, 19 Sep 2011 17:29:22 +0200 Subject: [erlang-questions] Escript and multiple OTP application archives (*.ez) or handling dependencies in an escript In-Reply-To: References: Message-ID: You can also take a look at the test case escript_SUITE:archive_script/1: https://github.com/erlang/otp/blob/dev/lib/stdlib/test/escript_SUITE.erl It creates an archive with to applications and puts it in an escript. /H?kan 2011/9/19 Dave Smith : > FWIW, basho_bench does distribute multiple apps in the same .ez. You > can see the config for rebar: > > https://github.com/basho/basho_bench/blob/master/rebar.config > > has a {escript_incl_apps, [...]} config item. It's a bit of a hack, > but it works. :) > > D. > > 2011/9/19 H?kan Mattsson : >> On Mon, Sep 19, 2011 at 4:56 PM, Eric Merritt wrote: >>> Hey guys, >>> >>> ?I am looking at escript and wondering how to go about distributing >>> more then one OTP app in an escript. That is, a project and it's >>> dependencies. >>> >>> ?From the documentation and tests on my part that validate that it >>> looks like escript:create can have one of an escript or a code archive >>> (one and only one) as its body. From reading the docs on code archives >>> an archive may contain one and only one OTP Application and be named >>> for that OTP Application. >> >> No, you can have several applications in an escript archive. >> It is just the .ez files that are limited to contain one application. >> >> /H?kan >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From amir_fire_2005@REDACTED Mon Sep 19 17:32:05 2011 From: amir_fire_2005@REDACTED (Amir) Date: Mon, 19 Sep 2011 08:32:05 -0700 (PDT) Subject: [erlang-questions] Lists!! In-Reply-To: <1316446046.43502.YahooMailNeo@web30402.mail.mud.yahoo.com> References: <1316446046.43502.YahooMailNeo@web30402.mail.mud.yahoo.com> Message-ID: <1316446325.5297.YahooMailNeo@web30401.mail.mud.yahoo.com> Hey Everyone, Here is the question:? If you try to enter this on Erlang shell?[97,98,99].? you will get the following answer: "abc" But I need the original list to be returned, is there any solution ? By the way, init:script_id().? {"OTP ?APN 181 01","R14B03"} -------------- next part -------------- An HTML attachment was scrubbed... URL: From silent_vendetta@REDACTED Mon Sep 19 17:33:25 2011 From: silent_vendetta@REDACTED (Chris Hicks) Date: Mon, 19 Sep 2011 08:33:25 -0700 Subject: [erlang-questions] escript question In-Reply-To: <9469322B-7150-4167-9AD1-704E9B114E8E@erlang-solutions.com> References: , <9469322B-7150-4167-9AD1-704E9B114E8E@erlang-solutions.com> Message-ID: Thank you everyone for responding, the answers have been very informative. For my purposes, the method Ulf proposed looks to be the the optimal solution. It's going to make things so much easier than I had anticipated and now I just want this grind to end so I can get back to the fun stuff of writing my own code! > Subject: Re: [erlang-questions] escript question > From: ulf.wiger@REDACTED > Date: Mon, 19 Sep 2011 16:37:53 +0200 > CC: silent_vendetta@REDACTED; erlang-questions@REDACTED > To: g@REDACTED > > > On 19 Sep 2011, at 16:22, Garrett Smith wrote: > > > > > E.g. store a function in your database that increments a number by 1: > > > > FStr = "fun(N) -> N + 1 end." > > > > When you're ready to use it: > > > > {ok, Tokens, _} = erl_scan:string(FStr), > > {ok, Exprs} = erl_parse:parse_exprs(Tokens), > > {value, F, []} = erl_eval:exprs(Exprs, []). > > > > Now: > > > > 2 = F(1). > > I'd go one step further and store Exprs in the database directly, eliminating the parsing step. > > To avoid having to learn the abstract form representation, you can use parse_trans_codegen.erl. > > http://github.com/esl/parse_trans > > Example: > > -module(storefun). > -export([f/1]). > > -include_lib("parse_trans/include/codegen.hrl"). > > f(1) -> > codegen:exprs(fun() -> > fun(N) -> > N + 1 > end > end). > > Compiling this with parse_trans/ebin in the path: > > Eshell V5.8.4 (abort with ^G) > 1> c(storefun). > {ok,storefun} > 2> storefun:f(1). > [{'fun',8, > {clauses,[{clause,8, > [{var,8,'N'}], > [], > [{op,9,'+',{var,9,'N'},{integer,9,1}}]}]}}] > 3> {value, F, []} = erl_eval:exprs(v(2),[]). > {value,#Fun,[]} > 4> F(2). > 3 > > That is, codegen:exprs(fun() -> end) returns the abstract form list for . If you want the expressions to inherit variables, you can do that by defining those variables as arguments to the enclosing fun: > > f(2) -> > codegen:exprs(fun(X) -> > fun(N) -> > N + X > end > end). > > > 5> c(storefun). > {ok,storefun} > 6> storefun:f(2). > [{'fun',14, > {clauses,[{clause,14, > [{var,14,'N'}], > [], > [{op,15,'+',{var,15,'N'},{var,15,'X'}}]}]}}] > 7> {value, F2, []} = erl_eval:exprs(v(6),[]). > ** exception error: {unbound_var,'X'} > 8> {value, F2, []} = erl_eval:exprs(v(6),[{'X',17}]). > ** exception error: no match of right hand side value > {value,#Fun,[{'X',17}]} > 9> {value, F2, _} = erl_eval:exprs(v(6),[{'X',17}]). > {value,#Fun,[{'X',17}]} > 10> F2(3). > 20 > > > The abstract code is generated at compile-time, so there is very little overhead. > > BR, > Ulf W > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Mon Sep 19 17:33:51 2011 From: bob@REDACTED (Bob Ippolito) Date: Mon, 19 Sep 2011 08:33:51 -0700 Subject: [erlang-questions] Lists!! In-Reply-To: <1316446325.5297.YahooMailNeo@web30401.mail.mud.yahoo.com> References: <1316446046.43502.YahooMailNeo@web30402.mail.mud.yahoo.com> <1316446325.5297.YahooMailNeo@web30401.mail.mud.yahoo.com> Message-ID: These lists are indistinguishable, this is just how the shell displays lists that could be interpreted as strings. "abc" =:= [97,98.99]. On Mon, Sep 19, 2011 at 8:32 AM, Amir wrote: > > > Hey Everyone, > Here is the question: > If you try to enter this on Erlang shell?[97,98,99]. > you will get the following answer: "abc" > But I need the original list to be returned, is there any solution ? > By the way, > init:script_id(). > {"OTP ?APN 181 01","R14B03"} > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From silent_vendetta@REDACTED Mon Sep 19 17:39:00 2011 From: silent_vendetta@REDACTED (Chris Hicks) Date: Mon, 19 Sep 2011 08:39:00 -0700 Subject: [erlang-questions] Lists!! In-Reply-To: References: <1316446046.43502.YahooMailNeo@web30402.mail.mud.yahoo.com>, <1316446325.5297.YahooMailNeo@web30401.mail.mud.yahoo.com>, Message-ID: If you are trying to print the list, take a look at the options for the io:format/1 module. If you are going to be performing mathematical calculations on the different items in the list, the Erlang system will automatically treat them as numbers, because that's what they are. The issue you are seeing is due to the fact that there is no string type in Erlang, simply lists of ASCII codes corresponding to the characters and the shell makes the assumption that you have input a string, in this case. It is just a formatting issue, not a change of the underlying structure. > Date: Mon, 19 Sep 2011 08:33:51 -0700 > From: bob@REDACTED > To: amir_fire_2005@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Lists!! > > These lists are indistinguishable, this is just how the shell displays > lists that could be interpreted as strings. "abc" =:= [97,98.99]. > > On Mon, Sep 19, 2011 at 8:32 AM, Amir wrote: > > > > > > Hey Everyone, > > Here is the question: > > If you try to enter this on Erlang shell [97,98,99]. > > you will get the following answer: "abc" > > But I need the original list to be returned, is there any solution ? > > By the way, > > init:script_id(). > > {"OTP APN 181 01","R14B03"} > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From amir_fire_2005@REDACTED Mon Sep 19 17:44:33 2011 From: amir_fire_2005@REDACTED (Amir) Date: Mon, 19 Sep 2011 08:44:33 -0700 (PDT) Subject: [erlang-questions] Fw: Lists!! In-Reply-To: <1316446852.88918.YahooMailNeo@web30405.mail.mud.yahoo.com> References: <1316446046.43502.YahooMailNeo@web30402.mail.mud.yahoo.com> <1316446325.5297.YahooMailNeo@web30401.mail.mud.yahoo.com> <1316446852.88918.YahooMailNeo@web30405.mail.mud.yahoo.com> Message-ID: <1316447073.19004.YahooMailNeo@web30404.mail.mud.yahoo.com> No I am not trying to print them.? I want the function to return the list containing a number? The requirement says: the function should return ? ? ? ?[Number] For example ? ? [8] ? ? ? ?but the shell shows me "\b" As a result of?previous replied, when the function returns "\b" instead of [8], I should imagine the requirement is covered, but it just shows that way? --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ----- Forwarded Message ----- From:?Chris Hicks To:?bob@REDACTED; amir_fire_2005@REDACTED Cc:?erlang-questions@REDACTED Sent:?Monday, September 19, 2011 5:39 PM Subject:?RE: [erlang-questions] Lists!! If you are trying to print the list, take a look at the options for the?io:format/1?module. If you are going to be performing mathematical calculations on the different items in the list, the Erlang system will automatically treat them as numbers, because that's what they are. The issue you are seeing is due to the fact that there is no string type in Erlang, simply lists of ASCII codes corresponding to the characters and the shell makes the assumption that you have input a string, in this case. It is just a formatting issue, not a change of the underlying structure. > Date: Mon, 19 Sep 2011 08:33:51 -0700 > From: bob@REDACTED > To: amir_fire_2005@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Lists!! >? > These lists are indistinguishable, this is just how the shell displays > lists that could be interpreted as strings. "abc" =:= [97,98.99]. >? > On Mon, Sep 19, 2011 at 8:32 AM, Amir wrote: > > > > > > Hey Everyone, > > Here is the question: > > If you try to enter this on Erlang shell?[97,98,99]. > > you will get the following answer: "abc" > > But I need the original list to be returned, is there any solution ? > > By the way, > > init:script_id(). > > {"OTP ?APN 181 01","R14B03"} > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions ________________________________ From: Bob Ippolito To: Amir Cc: "erlang-questions@REDACTED" Sent: Monday, September 19, 2011 5:33 PM Subject: Re: [erlang-questions] Lists!! These lists are indistinguishable, this is just how the shell displays lists that could be interpreted as strings. "abc" =:= [97,98.99]. On Mon, Sep 19, 2011 at 8:32 AM, Amir wrote: > > > Hey Everyone, > Here is the question: > If you try to enter this on Erlang shell?[97,98,99]. > you will get the following answer: "abc" > But I need the original list to be returned, is there any solution ? > By the way, > init:script_id(). > {"OTP ?APN 181 01","R14B03"} > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dale@REDACTED Mon Sep 19 17:46:29 2011 From: dale@REDACTED (Dale Harvey) Date: Mon, 19 Sep 2011 16:46:29 +0100 Subject: [erlang-questions] Fw: Lists!! In-Reply-To: <1316447073.19004.YahooMailNeo@web30404.mail.mud.yahoo.com> References: <1316446046.43502.YahooMailNeo@web30402.mail.mud.yahoo.com> <1316446325.5297.YahooMailNeo@web30401.mail.mud.yahoo.com> <1316446852.88918.YahooMailNeo@web30405.mail.mud.yahoo.com> <1316447073.19004.YahooMailNeo@web30404.mail.mud.yahoo.com> Message-ID: > "\b" =:= [8]. true they are exactly the same thing, just different ways to display On 19 September 2011 16:44, Amir wrote: > > > No I am not trying to print them. > I want the function to return the list containing a number > > The requirement says: > > the function should return [Number] > For example [8] but the shell shows me "\b" > > As a result of previous replied, when the function returns "\b" instead of > [8], I should imagine the requirement is covered, but it just shows that > way? > > > > > --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > ----- Forwarded Message ----- > *From:* Chris Hicks > *To:* bob@REDACTED; amir_fire_2005@REDACTED > *Cc:* erlang-questions@REDACTED > *Sent:* Monday, September 19, 2011 5:39 PM > *Subject:* RE: [erlang-questions] Lists!! > > If you are trying to print the list, take a look at the options for the > io:format/1 module. > If you are going to be performing mathematical calculations on the different > items in the list, the Erlang system will automatically treat them as > numbers, because that's what they are. The issue you are seeing is due to > the fact that there is no string type in Erlang, simply lists of ASCII codes > corresponding to the characters and the shell makes the assumption that you > have input a string, in this case. It is just a formatting issue, not a > change of the underlying structure. > > > Date: Mon, 19 Sep 2011 08:33:51 -0700 > > From: bob@REDACTED > > To: amir_fire_2005@REDACTED > > CC: erlang-questions@REDACTED > > Subject: Re: [erlang-questions] Lists!! > > > > These lists are indistinguishable, this is just how the shell displays > > lists that could be interpreted as strings. "abc" =:= [97,98.99]. > > > > On Mon, Sep 19, 2011 at 8:32 AM, Amir wrote: > > > > > > > > > Hey Everyone, > > > Here is the question: > > > If you try to enter this on Erlang shell [97,98,99]. > > > you will get the following answer: "abc" > > > But I need the original list to be returned, is there any solution ? > > > By the way, > > > init:script_id(). > > > {"OTP APN 181 01","R14B03"} > > > > > > > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > ------------------------------ > *From:* Bob Ippolito > *To:* Amir > *Cc:* "erlang-questions@REDACTED" > *Sent:* Monday, September 19, 2011 5:33 PM > > *Subject:* Re: [erlang-questions] Lists!! > > These lists are indistinguishable, this is just how the shell displays > lists that could be interpreted as strings. "abc" =:= [97,98.99]. > > On Mon, Sep 19, 2011 at 8:32 AM, Amir wrote: > > > > > > Hey Everyone, > > Here is the question: > > If you try to enter this on Erlang shell [97,98,99]. > > you will get the following answer: "abc" > > But I need the original list to be returned, is there any solution ? > > By the way, > > init:script_id(). > > {"OTP APN 181 01","R14B03"} > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amir_fire_2005@REDACTED Mon Sep 19 17:47:33 2011 From: amir_fire_2005@REDACTED (Amir) Date: Mon, 19 Sep 2011 08:47:33 -0700 (PDT) Subject: [erlang-questions] Fw: Fw: Lists!! In-Reply-To: References: <1316446046.43502.YahooMailNeo@web30402.mail.mud.yahoo.com> <1316446325.5297.YahooMailNeo@web30401.mail.mud.yahoo.com> <1316446852.88918.YahooMailNeo@web30405.mail.mud.yahoo.com> <1316447073.19004.YahooMailNeo@web30404.mail.mud.yahoo.com> Message-ID: <1316447253.32668.YahooMailNeo@web30406.mail.mud.yahoo.com> Thanks all... ----- Forwarded Message ----- From: Dale Harvey To: Amir Cc: "erlang-questions@REDACTED" Sent: Monday, September 19, 2011 5:46 PM Subject: Re: [erlang-questions] Fw: Lists!! > "\b" =:= [8]. true they are exactly the same thing, just different ways to display On 19 September 2011 16:44, Amir wrote: > > > >No I am not trying to print them.? >I want the function to return the list containing a number? > > >The requirement says: > > >the function should return ? ? ? ?[Number] >For example ? ? [8] ? ? ? ?but the shell shows me "\b" > > >As a result of?previous replied, when the function returns "\b" instead of [8], I should imagine the requirement is covered, but it just shows that way? > > > > > > >--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > > > >----- Forwarded Message ----- >From:?Chris Hicks >To:?bob@REDACTED; amir_fire_2005@REDACTED >Cc:?erlang-questions@REDACTED >Sent:?Monday, September 19, 2011 5:39 PM >Subject:?RE: [erlang-questions] Lists!! > > >If you are trying to print the list, take a look at the options for the?io:format/1?module. If you are going to be performing mathematical calculations on the different items in the list, the Erlang system will automatically treat them as numbers, because that's what they are. The issue you are seeing is due to the fact that there is no string type in Erlang, simply lists of ASCII codes corresponding to the characters and the shell makes the assumption that you have input a string, in this case. It is just a formatting issue, not a change of the underlying structure. > > >> Date: Mon, 19 Sep 2011 08:33:51 -0700 >> From: bob@REDACTED >> To: amir_fire_2005@REDACTED >> CC: erlang-questions@REDACTED >> Subject: Re: [erlang-questions] Lists!! >>? > >> These lists are indistinguishable, this is just how the shell displays >> lists that could be interpreted as strings. "abc" =:= [97,98.99]. >>? >> On Mon, Sep 19, 2011 at 8:32 AM, Amir wrote: >> > >> > >> > Hey Everyone, >> > Here is the question: >> > If you try to enter this on Erlang shell?[97,98,99]. >> > you will get the following answer: "abc" >> > But I need the original list to be returned, is there any solution ? >> > By the way, >> > init:script_id(). >> > {"OTP ?APN 181 01","R14B03"} >> > >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > >________________________________ > From: Bob Ippolito >To: Amir >Cc: "erlang-questions@REDACTED" >Sent: Monday, September 19, 2011 5:33 PM > >Subject: Re: [erlang-questions] Lists!! > > >These lists are indistinguishable, this is just how the shell displays >lists that could be interpreted as strings. "abc" =:= [97,98.99]. > >On Mon, Sep 19, 2011 at 8:32 AM, Amir wrote: >> >> >> Hey Everyone, >> Here is the question: >> If you try to enter this on Erlang shell?[97,98,99]. >> you will get the following answer: "abc" >> But I need the original list to be returned, is there any solution ? >> By the way, >> init:script_id(). >> {"OTP ?APN 181 01","R14B03"} >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > > > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From silent_vendetta@REDACTED Mon Sep 19 17:50:15 2011 From: silent_vendetta@REDACTED (Chris Hicks) Date: Mon, 19 Sep 2011 08:50:15 -0700 Subject: [erlang-questions] Fw: Lists!! In-Reply-To: <1316447073.19004.YahooMailNeo@web30404.mail.mud.yahoo.com> References: <1316446046.43502.YahooMailNeo@web30402.mail.mud.yahoo.com>, <1316446325.5297.YahooMailNeo@web30401.mail.mud.yahoo.com>, , <1316446852.88918.YahooMailNeo@web30405.mail.mud.yahoo.com>, <1316447073.19004.YahooMailNeo@web30404.mail.mud.yahoo.com> Message-ID: That is correct. Internally the result of the function is stored as [8]. You are simply seeing the shell's assumption of what [8] means. I'm not in a place to open an erlang shell so I'll show you how to provie it to yourself with a series of commands: [8] <-- Shows you "\b' as you said so do: [N] = [8]. Then: N * 4. <-- You will get a result of 32 Date: Mon, 19 Sep 2011 08:44:33 -0700 From: amir_fire_2005@REDACTED To: erlang-questions@REDACTED Subject: [erlang-questions] Fw: Lists!! No I am not trying to print them. I want the function to return the list containing a number The requirement says: the function should return [Number]For example [8] but the shell shows me "\b" As a result of previous replied, when the function returns "\b" instead of [8], I should imagine the requirement is covered, but it just shows that way? --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ----- Forwarded Message ----- From: Chris Hicks To: bob@REDACTED; amir_fire_2005@REDACTED Cc: erlang-questions@REDACTED Sent: Monday, September 19, 2011 5:39 PM Subject: RE: [erlang-questions] Lists!! If you are trying to print the list, take a look at the options for the io:format/1 module. If you are going to be performing mathematical calculations on the different items in the list, the Erlang system will automatically treat them as numbers, because that's what they are. The issue you are seeing is due to the fact that there is no string type in Erlang, simply lists of ASCII codes corresponding to the characters and the shell makes the assumption that you have input a string, in this case. It is just a formatting issue, not a change of the underlying structure. > Date: Mon, 19 Sep 2011 08:33:51 -0700 > From: bob@REDACTED > To: amir_fire_2005@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Lists!! > > These lists are indistinguishable, this is just how the shell displays > lists that could be interpreted as strings. "abc" =:= [97,98.99]. > > On Mon, Sep 19, 2011 at 8:32 AM, Amir wrote: > > > > > > Hey Everyone, > > Here is the question: > > If you try to enter this on Erlang shell [97,98,99]. > > you will get the following answer: "abc" > > But I need the original list to be returned, is there any solution ? > > By the way, > > init:script_id(). > > {"OTP APN 181 01","R14B03"} > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From: Bob Ippolito To: Amir Cc: "erlang-questions@REDACTED" Sent: Monday, September 19, 2011 5:33 PM Subject: Re: [erlang-questions] Lists!! These lists are indistinguishable, this is just how the shell displays lists that could be interpreted as strings. "abc" =:= [97,98.99]. On Mon, Sep 19, 2011 at 8:32 AM, Amir wrote: > > > Hey Everyone, > Here is the question: > If you try to enter this on Erlang shell [97,98,99]. > you will get the following answer: "abc" > But I need the original list to be returned, is there any solution ? > By the way, > init:script_id(). > {"OTP APN 181 01","R14B03"} > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf.wiger@REDACTED Mon Sep 19 18:03:44 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 19 Sep 2011 18:03:44 +0200 Subject: [erlang-questions] escript question In-Reply-To: References: , <9469322B-7150-4167-9AD1-704E9B114E8E@erlang-solutions.com> Message-ID: <8B384DC3-E464-48D5-9901-437A273D6B56@erlang-solutions.com> On 19 Sep 2011, at 17:33, Chris Hicks wrote: > Thank you everyone for responding, the answers have been very informative. For my purposes, the method Ulf proposed looks to be the the optimal solution. It's going to make things so much easier than I had anticipated and now I just want this grind to end so I can get back to the fun stuff of writing my own code! Let me know if you encounter any issues with it. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From dgud@REDACTED Mon Sep 19 19:18:49 2011 From: dgud@REDACTED (Dan Gudmundsson) Date: Mon, 19 Sep 2011 19:18:49 +0200 Subject: [erlang-questions] mnesia clear_table() bug ? In-Reply-To: References: Message-ID: That is because I use [d]ets:match_delete(Tab, '_') instead of [d]ets:delete_all_objects(Tab) Will fix in next release. Thanks /Dan On Fri, Sep 9, 2011 at 8:28 PM, KukHyun Lee wrote: > Hi. > This is disc_only_copies table. > > Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:4:4] [rq:4] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.4 (abort with ^G) > ... > > (db@REDACTED)103> mnesia:table_info(table, size). > 151944 > (db@REDACTED)104> mnesia:table_info(table, memory). > 16295224 > (db@REDACTED)105> mnesia:clear_table(table). > {atomic,ok} > (db@REDACTED)106> mnesia:table_info(table, size). > 0 > (db@REDACTED)107> mnesia:table_info(table, memory). > 16295224 > > Why did not changed table's memory? Is this a bug? > How can i set a table to zero memory. > > Thanks, > KukHyun > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandu.gokul138@REDACTED Tue Sep 20 00:20:20 2011 From: chandu.gokul138@REDACTED (Gokul Evuri) Date: Tue, 20 Sep 2011 00:20:20 +0200 Subject: [erlang-questions] What happened to my debugger and why a crashed-shell?? Message-ID: When tried to set break points through the code and debug, got debugger and shell crash. exit:{'Exit',<0.76.0>,{port_handler,#Port,0.2433>,badsig}} Reason:{'Exit',{gs_kernel_died,{port_handler,#Port<0.2433>,badsig}}} %%attached a screen shot -- *Gokul Reddy Evuri,* * * -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Screenshot-44.png Type: image/png Size: 41749 bytes Desc: not available URL: From mailparmalat@REDACTED Tue Sep 20 00:24:12 2011 From: mailparmalat@REDACTED (Steven Yu) Date: Tue, 20 Sep 2011 00:24:12 +0200 Subject: [erlang-questions] What happened to my debugger and why a crashed-shell?? In-Reply-To: References: Message-ID: Hi, I had a similar problem but with toolbar / tv / pman where it just doesn't startup. Toolbar gave a timeout on startup. TV gave a PID and nothing happened. Tried 4 different versions R9, R11, R12, R14 and all gave same results. I'm thinking it could be Windows 7 updates related. Regards, On Tue, Sep 20, 2011 at 12:20 AM, Gokul Evuri wrote: > When tried to set break points through the code and debug, > got debugger and shell crash. > > exit:{'Exit',<0.76.0>,{port_handler,#Port,0.2433>,badsig}} > Reason:{'Exit',{gs_kernel_died,{port_handler,#Port<0.2433>,badsig}}} > > %%attached a screen shot > > > -- > *Gokul Reddy Evuri,* > * > * > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Sep 20 04:26:11 2011 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 20 Sep 2011 14:26:11 +1200 Subject: [erlang-questions] Lists!! In-Reply-To: <1316446325.5297.YahooMailNeo@web30401.mail.mud.yahoo.com> References: <1316446046.43502.YahooMailNeo@web30402.mail.mud.yahoo.com> <1316446325.5297.YahooMailNeo@web30401.mail.mud.yahoo.com> Message-ID: <869F324C-C613-4BE8-9FC0-F550E248A774@cs.otago.ac.nz> On 20/09/2011, at 3:32 AM, Amir wrote: > > > Hey Everyone, > Here is the question: > If you try to enter this on Erlang shell [97,98,99]. > you will get the following answer: "abc" > > But I need the original list to be returned, is there any solution ? That *IS* the original list. From ok@REDACTED Tue Sep 20 04:28:11 2011 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 20 Sep 2011 14:28:11 +1200 Subject: [erlang-questions] Fw: Lists!! In-Reply-To: <1316447073.19004.YahooMailNeo@web30404.mail.mud.yahoo.com> References: <1316446046.43502.YahooMailNeo@web30402.mail.mud.yahoo.com> <1316446325.5297.YahooMailNeo@web30401.mail.mud.yahoo.com> <1316446852.88918.YahooMailNeo@web30405.mail.mud.yahoo.com> <1316447073.19004.YahooMailNeo@web30404.mail.mud.yahoo.com> Message-ID: <2EB3C2A7-43C4-4D4D-8A2D-115F4AC20F74@cs.otago.ac.nz> On 20/09/2011, at 3:44 AM, Amir wrote: > > > No I am not trying to print them. > I want the function to return the list containing a number > > The requirement says: > > the function should return [Number] > For example [8] but the shell shows me "\b" [8] and "\b" are the SAME THING, as a quick > [8] = "\b". would have shown you. This is all in the documentation and the textbooks; they do repay reading. From wgwi@REDACTED Tue Sep 20 09:41:37 2011 From: wgwi@REDACTED (Wang Wei) Date: Tue, 20 Sep 2011 15:41:37 +0800 Subject: [erlang-questions] Any way to operate windows Active directory with erlang? Message-ID: <4E7843B1.2070005@sxu.edu.cn> Hi, I wish to operate windows Active directory with erlang, Is there any projects do the same things. thanks. From antoine.koener@REDACTED Tue Sep 20 11:00:23 2011 From: antoine.koener@REDACTED (Antoine Koener) Date: Tue, 20 Sep 2011 11:00:23 +0200 Subject: [erlang-questions] Any way to operate windows Active directory with erlang? In-Reply-To: <4E7843B1.2070005@sxu.edu.cn> References: <4E7843B1.2070005@sxu.edu.cn> Message-ID: Do you mean querying ad using ldap ? Search for "eldap" On 20 sept. 2011, at 09:41, Wang Wei wrote: > Hi, I wish to operate windows Active directory with erlang, Is there any > projects do the same things. thanks. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From torben.lehoff@REDACTED Tue Sep 20 15:04:02 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Tue, 20 Sep 2011 15:04:02 +0200 Subject: [erlang-questions] How to find out which types a module is exporting? Message-ID: Hi, I would like to get information about which types a module exports - are there any way of doing this that is smarter than looking in the source file? Cheers, Torben -- http://www.linkedin.com/in/torbenhoffmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Sep 20 15:45:18 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 20 Sep 2011 15:45:18 +0200 Subject: [erlang-questions] How erlang handle complicated data structure like C? In-Reply-To: References: Message-ID: On Mon, Sep 19, 2011 at 08:26, George Catalin Serbanut wrote: > Considering all variables integers, here is an example how to play with > records: I'll move the quotes around a bit... > 7. How to use the test: > receive_function(Packet) when is_my_record(Packet) == true -> > do_something_with_my_packet; > receive_function(Packet) when is_my_record(Packet) /= true -> I_do_not_care. You can't do this in Erlang. Guard expressions are limited so you can be sure they terminate. > 6. Binary tester: > is_my_record(Binary) -> > ??? L = re:split(binary_to_list(Binary),?SEPARATOR,[{return,list}]), > ??? if length(L) == 3 -> Result = lists:all(fun(E) -> TL = > string:join(string:tokens(E,"0123456789"),""), if length(TL) == 0 -> Result > = true; length(TL) /= 0 -> Result = false end, Result end,L), > ?????? length(L) /= 3 -> Result = false, > ??? end, > ??? Result. This part can be done much more beautifully. length is generally a bad idea when a match will do: is_my_record(B) -> case re:split(binary_to_list(B), ?SEP, [{return, list}]) of [A, B, C] = L -> lists:all(fun(E) -> string:join(string:tokens(E,"0123456789"), "") == [] end, L); _ -> false end. I'd probably write (not tested, subject to infinite failure): is_digit(C) when C >= $0 andalso C =< $9 -> true is_digit(C) -> false. all_digits(B) when is_binary(B) -> all_digits(binary_to_list(B)); % Building a traverser over binaries will be faster all_digits(L) -> lists:all(fun is_digit/1, L). is_my_record(B) -> case binary:split(B, <>, [global]) of [A, B, C] = L -> lists:all(fun all_digits/1, L); _ -> false end. all_digits/1 is inefficient. But we can write: all_digits(F, <<>>) -> true; all_digits(F, <>) -> F(C) andalso all_digits(F, R). % I can't remember if andalso is tail recursive. It should be. If not, replace this with a case. This avoids the creation of lots and lots of garbage. It also avoids the pesky re-library which is usually a bad idea :) And I believe it is a tad more readable. -- J. From cgsmcmlxxv@REDACTED Tue Sep 20 16:13:14 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Tue, 20 Sep 2011 16:13:14 +0200 Subject: [erlang-questions] How erlang handle complicated data structure like C? In-Reply-To: References: Message-ID: Hi Jesper, Can you explain why length is a bad idea? I do not deny that matching left and right expressions is also a solution, but I see no difference. So, please, if you know a bug about length, I am interested in knowing about. ------------------- > 7. How to use the test: > receive_function(Packet) when is_my_record(Packet) == true -> > do_something_with_my_packet; > receive_function(Packet) when is_my_record(Packet) /= true -> I_do_not_care. You can't do this in Erlang. Guard expressions are limited so you can be sure they terminate. ------------------- Please, take a better look and you will notice I haven't used guarded function for recursion, but for condition. What you said is okay for recursion. In addition, I didn't say that was the best way to do it, but just an example. Jovi needed an example and I gave. I am sorry if my code offended you so much to call it "garbage". Feel free to propose the best working idea for this case. Cheers, CGS On Tue, Sep 20, 2011 at 3:45 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Mon, Sep 19, 2011 at 08:26, George Catalin Serbanut > wrote: > > Considering all variables integers, here is an example how to play with > > records: > > I'll move the quotes around a bit... > > > 7. How to use the test: > > receive_function(Packet) when is_my_record(Packet) == true -> > > do_something_with_my_packet; > > receive_function(Packet) when is_my_record(Packet) /= true -> > I_do_not_care. > > You can't do this in Erlang. Guard expressions are limited so you can > be sure they terminate. > > > 6. Binary tester: > > is_my_record(Binary) -> > > L = re:split(binary_to_list(Binary),?SEPARATOR,[{return,list}]), > > if length(L) == 3 -> Result = lists:all(fun(E) -> TL = > > string:join(string:tokens(E,"0123456789"),""), if length(TL) == 0 -> > Result > > = true; length(TL) /= 0 -> Result = false end, Result end,L), > > length(L) /= 3 -> Result = false, > > end, > > Result. > > This part can be done much more beautifully. length is generally a bad > idea when a match will do: > > is_my_record(B) -> > case re:split(binary_to_list(B), ?SEP, [{return, list}]) of > [A, B, C] = L -> > lists:all(fun(E) -> string:join(string:tokens(E,"0123456789"), > "") == [] end, L); > _ -> false > end. > > I'd probably write (not tested, subject to infinite failure): > > is_digit(C) when C >= $0 andalso C =< $9 -> true > is_digit(C) -> false. > > all_digits(B) when is_binary(B) -> all_digits(binary_to_list(B)); % > Building a traverser over binaries will be faster > all_digits(L) -> lists:all(fun is_digit/1, L). > > is_my_record(B) -> > case binary:split(B, <>, [global]) of > [A, B, C] = L -> lists:all(fun all_digits/1, L); > _ -> false > end. > > all_digits/1 is inefficient. But we can write: > > all_digits(F, <<>>) -> true; > all_digits(F, <>) -> > F(C) andalso all_digits(F, R). % I can't remember if andalso is tail > recursive. It should be. If not, replace this with a case. > > This avoids the creation of lots and lots of garbage. It also avoids > the pesky re-library which is usually a bad idea :) And I believe it > is a tad more readable. > > -- > J. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pdf@REDACTED Tue Sep 20 16:16:12 2011 From: pdf@REDACTED (Paulo Ferreira) Date: Tue, 20 Sep 2011 15:16:12 +0100 Subject: [erlang-questions] Erlang on ARM9? In-Reply-To: References: Message-ID: <0977DC71-22ED-43E0-8BD3-F2889C6FC370@isep.ipp.pt> On 2011/09/19, at 09:14, Guy Wiener wrote: > Hello Erlangers, > I am checking an option to use Erlang to control a Linux-based device (a programmable helicopter). Is there an efficient port of Erlang that works on top of an ARM9 processor? If so, the usage scenario will be many processes running on a single node, with frequent message-passing, some access to binary files, and no shared data (no ETS, DETS or Mnesia). Is this scenario likely to perform well on an embedded Erlang? > > Thanks, > Guy Wiener, > Dept. of Computer Science and Applied Math, > Weizmann Institute for Science. Well, just it just compiles from source well. I have experience using a S3C2440 board at 400 Mhz and on some informal benchmarks ( ring ) it is just 7 times slower than a 2,2 Ghz core 2 Duo Mac book. From other erlang embedded experiences: How do you control the hardware in C? Linux device drivers are very slow, when compared to direct memory access. Doing NIFs and dynamic libs may be hard. ARM processors are usually supported by the latest gcc version, and that is VERY good for Erlang porting. If you try to compile Erlang with an older gcc version, you may get all the kind of "strange and wonderful" errors. Check the gcc version of your toolchain! My best regards Paulo Ferreira P.S.: Please replace "strange and wonderful" by the suitable words in your native language.... From jesper.louis.andersen@REDACTED Tue Sep 20 17:00:26 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 20 Sep 2011 17:00:26 +0200 Subject: [erlang-questions] How erlang handle complicated data structure like C? In-Reply-To: References: Message-ID: On Tue, Sep 20, 2011 at 16:13, CGS wrote: > Hi Jesper, > > Can you explain why length is a bad idea? I do not deny that matching left > and right expressions is also a solution, but I see no difference. So, > please, if you know a bug about length, I am interested in knowing about. length/1 is not wrong, but note that it has to traverse all of the list to come up with its answer. In our case, the list is fairly small (length/1 returns 3), so this may not matter. But for a large list, the call to length is going to be rather expensive. Contrast this with a match on the structure which is either [] or [_|_]. This match can be done simply by examining the head of the list. We don't have to traverse the list fully to get the answer. In general, a good idea is to prefer matching over calling length/1 if you are in doubt. It usually translates to simpler code. Another worthy point is that matching on lists avoids "Boolean blindness", which often follows due to a matching on == 3 or similar. Boolean Blindness is a rather deep concept, but it is worth understanding. See, http://existentialtype.wordpress.com/2011/03/15/boolean-blindness/ The exposition could be written in a more approachable way I think, but I have not gotten around to doing it yet :P In a certain sense, length/1 tells us _too much_. It yields a number, namely the length of the list. But we are only interested in two limited cases, namely sizes 0 and 3. That is, length/1 can be seen as being too powerful for our cause. That is, there is nothing wrong with using length/1 here. You should just know the implications of doing so for other code pieces. >> 7. How to use the test: >> receive_function(Packet) when is_my_record(Packet) == true -> >> do_something_with_my_packet; >> receive_function(Packet) when is_my_record(Packet) /= true -> >> I_do_not_care. > > You can't do this in Erlang. Guard expressions are limited so you can > be sure they terminate. > ------------------- > > Please, take a better look and you will notice I haven't used guarded > function for recursion, but for condition. What you said is okay for > recursion. The problem is shown in the following silly module: ---- -module(f). -compile(export_all). my_own_guard(B) when is_binary(B) -> binary; my_own_guard(S) when is_list(S) -> list. f(X) when my_own_guard(X) == binary -> binary_to_list(X); f(X) when my_own_guard(X) == list -> X. --- When we try to compile this in the Erlang shell, it fails: Eshell V5.8.5 (abort with ^G) 1> c(f). ./f.erl:8: call to local/imported function my_own_guard/1 is illegal in guard ./f.erl:10: call to local/imported function my_own_guard/1 is illegal in guard error 2> The relevant part in the reference manual is the part about "Guard Sequences", http://www.erlang.org/doc/reference_manual/expressions.html#id78951 which states exactly what subset of all expressions that are valid as a Guard. In particular we are only allowed to call functions in Tables 7.4 and 7.5 and f:my_own_guard/1 is not in that table which is why the above compilation fail. It is worth it to contrast this behavior with Haskell, where arbitrary guard sequences *are* allowed and Haskell is lazy. Or Coq, where are functions are total, including recursive functions. In these cases, termination is less of a problem. > In addition, I didn't say that was the best way to do it, but just an > example. Jovi needed an example and I gave. I am sorry if my code offended > you so much to call it "garbage". Feel free to propose the best working idea > for this case. Written code is never garbage if it is correct and solves the problem in an adequate way. Even if it contains minor errors which are not deep design errors, it isn't a problem - because those are quickly fixed. To me, that the code is there has a value in its own. This is mere "golfing" in the sense of producing simpler and smaller code. My goal was not to show the code to be bad, but rather to show an alternative way of implementing a solution which I do think is a bit more idiomatic and functional. Why? Because I think it is better you see more styles and then form an opinion of your own about what style you prefer when writing programs. In any event, consistency tends to win over smug code :) Your style may be easier to comprehend for a person who is more accustomed to an imperative code writing style. My style is, perhaps, easier for one with an ML or Haskell background. -- J. From jussi.ao.malinen@REDACTED Tue Sep 20 18:02:54 2011 From: jussi.ao.malinen@REDACTED (Jussi Malinen) Date: Tue, 20 Sep 2011 19:02:54 +0300 Subject: [erlang-questions] ASN.1 Value notation not working because of 'Externalvaluereference'? Message-ID: Hello! I am working with an ASN.1 protocol (S1AP to be specific [1]) and I have managed to compile the specifications and decoding messages works fine. However when trying to use value notation with S1AP I get an error. I have created a minimal value notation example below. See the entire file in [2]. (I got this from OSS Nokalva's ASN.1-Step, so it should be valid I assume). Compiling this value notation causes the following warning: "Warning: could not check default value 14". There is a long list of other warnings about other default values. resetSuccess S1AP-PDU ::= successfulOutcome : { procedureCode 14, criticality reject, value ResetAcknowledge : { protocolIEs {} } } The compilation goes through, but the tuple that the resetSuccess() function returns contains a lot of extra elements like Externalvaluereference, which I dont get when decoding this same message from a protocol capture: {successfulOutcome, {'SuccessfulOutcome',14, {'Externalvaluereference',13,'S1APValue',reject}, {opentypefieldvalue, {type,[], {'Externaltypereference',14,'S1APValue','ResetAcknowledge'}, [],[],no}, [{protocolIEs,[]}]}}} The error I get when trying to encode this is: 1> 'S1AP':encode('S1AP-PDU', 'S1APValue':resetSuccess()). {error,{asn1,{enumerated,{'Externalvaluereference',13, 'S1APValue',reject}}}} If someone wants to give this a try, the project is open source (still only a prototype for sure) and available online at google-code [3]. To replicate just clone the project, run "./compile.py asn1", "export ERL_LIBS=." and then on erl "'S1AP':encode('S1AP-PDU', 'S1APValue':resetSuccess())." Are there some gotchas with value notation that I should be aware of? Has someone managed to get S1AP or similar protocol to work with value notation? Any pointers how to continue with this are appreciated! Thanks, Jussi Malinen [1] http://www.3gpp.org/ftp/Specs/html-info/36413.htm [2] https://code.google.com/p/rammbock-asn1/source/browse/src/asn1/S1AP/S1APValue.asn1 [3] https://code.google.com/p/rammbock-asn1 From cgsmcmlxxv@REDACTED Tue Sep 20 20:26:48 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Tue, 20 Sep 2011 20:26:48 +0200 Subject: [erlang-questions] How erlang handle complicated data structure like C? In-Reply-To: References: Message-ID: I agree with the illegal guard. The reason I put it that way was to cover two cases: receive internally or via TCP connection. Your observation is correct that such a guard doesn't work. Its sole purpose was to see the condition. I missed your point first time and I apologize. About the length, it depends what you prefer: slower code, but smaller, or bigger code, but faster (I usually prefer to work with vectors even if they are more memory consumers, but I can insert very fast search algorithms and that is important for me at large lists). Maybe it's my style, but I start all the time with implementing the skeleton, after that I expand it and in the end I do optimizations. I think it's easy to guess why, so, not entering details. Nevertheless, this is about styles of implementing things. Cheers, CGS On Tue, Sep 20, 2011 at 5:00 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Tue, Sep 20, 2011 at 16:13, CGS wrote: > > Hi Jesper, > > > > Can you explain why length is a bad idea? I do not deny that matching > left > > and right expressions is also a solution, but I see no difference. So, > > please, if you know a bug about length, I am interested in knowing about. > > length/1 is not wrong, but note that it has to traverse all of the > list to come up with its answer. In our case, the list is fairly small > (length/1 returns 3), so this may not matter. But for a large list, > the call to length is going to be rather expensive. Contrast this with > a match on the structure which is either [] or [_|_]. This match can > be done simply by examining the head of the list. We don't have to > traverse the list fully to get the answer. > > In general, a good idea is to prefer matching over calling length/1 if > you are in doubt. It usually translates to simpler code. Another > worthy point is that matching on lists avoids "Boolean blindness", > which often follows due to a matching on == 3 or similar. Boolean > Blindness is a rather deep concept, but it is worth understanding. > See, > > http://existentialtype.wordpress.com/2011/03/15/boolean-blindness/ > > The exposition could be written in a more approachable way I think, > but I have not gotten around to doing it yet :P > > In a certain sense, length/1 tells us _too much_. It yields a number, > namely the length of the list. But we are only interested in two > limited cases, namely sizes 0 and 3. That is, length/1 can be seen as > being too powerful for our cause. > > That is, there is nothing wrong with using length/1 here. You should > just know the implications of doing so for other code pieces. > > >> 7. How to use the test: > >> receive_function(Packet) when is_my_record(Packet) == true -> > >> do_something_with_my_packet; > >> receive_function(Packet) when is_my_record(Packet) /= true -> > >> I_do_not_care. > > > > You can't do this in Erlang. Guard expressions are limited so you can > > be sure they terminate. > > ------------------- > > > > Please, take a better look and you will notice I haven't used guarded > > function for recursion, but for condition. What you said is okay for > > recursion. > > The problem is shown in the following silly module: > > ---- > -module(f). > > -compile(export_all). > > my_own_guard(B) when is_binary(B) -> binary; > my_own_guard(S) when is_list(S) -> list. > > f(X) when my_own_guard(X) == binary -> > binary_to_list(X); > f(X) when my_own_guard(X) == list -> > X. > > --- > > When we try to compile this in the Erlang shell, it fails: > > Eshell V5.8.5 (abort with ^G) > 1> c(f). > ./f.erl:8: call to local/imported function my_own_guard/1 is illegal in > guard > ./f.erl:10: call to local/imported function my_own_guard/1 is illegal in > guard > error > 2> > > The relevant part in the reference manual is the part about "Guard > Sequences", > > http://www.erlang.org/doc/reference_manual/expressions.html#id78951 > > which states exactly what subset of all expressions that are valid as > a Guard. In particular we are only allowed to call functions in Tables > 7.4 and 7.5 and f:my_own_guard/1 is not in that table which is why the > above compilation fail. > > It is worth it to contrast this behavior with Haskell, where arbitrary > guard sequences *are* allowed and Haskell is lazy. Or Coq, where are > functions are total, including recursive functions. In these cases, > termination is less of a problem. > > > In addition, I didn't say that was the best way to do it, but just an > > example. Jovi needed an example and I gave. I am sorry if my code > offended > > you so much to call it "garbage". Feel free to propose the best working > idea > > for this case. > > Written code is never garbage if it is correct and solves the problem > in an adequate way. Even if it contains minor errors which are not > deep design errors, it isn't a problem - because those are quickly > fixed. To me, that the code is there has a value in its own. This is > mere "golfing" in the sense of producing simpler and smaller code. My > goal was not to show the code to be bad, but rather to show an > alternative way of implementing a solution which I do think is a bit > more idiomatic and functional. > > Why? Because I think it is better you see more styles and then form an > opinion of your own about what style you prefer when writing programs. > In any event, consistency tends to win over smug code :) Your style > may be easier to comprehend for a person who is more accustomed to an > imperative code writing style. My style is, perhaps, easier for one > with an ML or Haskell background. > > -- > J. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.james.barry@REDACTED Tue Sep 20 21:09:05 2011 From: paul.james.barry@REDACTED (Paul Barry) Date: Tue, 20 Sep 2011 20:09:05 +0100 Subject: [erlang-questions] O'Reilly Webcast: Reminder Message-ID: Hi folks. Just a reminder of my upcoming O'Reilly webcast on Erlang scheduled for this Thursday: http://oreillynet.com/pub/e/2022 at 10am PST. The O'Reilly folk tell me that over 200 people have registered for this event (although, typically, less than this will actually "turn up" on the day). I'd be thrilled if some of you could try to catch my talk, as I'm also presenting it again at PyCon Ireland 2011 on October 9th: http://python.ie/pycon/2011/conference/#talks - if you can think of any enhancement to my talk, I'd be delighted to include your suggestions at PyCon. Regards. Paul. -- Paul Barry, w: http://paulbarry.itcarlow.ie, e: paul.barry@REDACTED Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. From jeffm@REDACTED Wed Sep 21 01:43:40 2011 From: jeffm@REDACTED (jm) Date: Wed, 21 Sep 2011 09:43:40 +1000 Subject: [erlang-questions] Rough thought on a P2P package distribution model for Erlang In-Reply-To: References: <4E700AFF.7090309@ghostgun.com> Message-ID: <4E79252C.9070005@ghostgun.com> I'm not saying it is any different. I'm just outlining a distribution system for packages which is an alternative to the more tradition tiered ftp or http site mirrors. Even in the sense I'm outlining it's not that novel. One linux distribution, rubyx, used a similar model. Their package distribution program was called "white water". The project appears to be dead as the website seems to be unresponsive. As you point out these systems "have been around for a very long time" so why aren't we making better use of these technologies for things like this? Jeff. On 17/09/11 4:51 AM, Jon Watte wrote: > How is this different from the already-solved problem of peer-to-peer > authenticated file distribution? > > Tracker-based systems like bittorrent, and fully peer-to-peer systems > like freenet have been around for a very long time, and solve all of > those problems, with different trade-offs for performance, security, > susceptibility, etc. > > Sincerely, > > jw > From ericbmerritt@REDACTED Wed Sep 21 05:12:09 2011 From: ericbmerritt@REDACTED (Eric Merritt) Date: Tue, 20 Sep 2011 22:12:09 -0500 Subject: [erlang-questions] Escript and multiple OTP application archives (*.ez) or handling dependencies in an escript In-Reply-To: References: Message-ID: Thanks guys, You where quite correct. You can load as much as you would like into an archive. I realize why the manpage for the code module was misleading. Thanks again. 2011/9/19 H?kan Mattsson : > You can also take a look at the test case escript_SUITE:archive_script/1: > > ?https://github.com/erlang/otp/blob/dev/lib/stdlib/test/escript_SUITE.erl > > It creates an archive with to applications and puts it in an escript. > > /H?kan > > 2011/9/19 Dave Smith : >> FWIW, basho_bench does distribute multiple apps in the same .ez. You >> can see the config for rebar: >> >> https://github.com/basho/basho_bench/blob/master/rebar.config >> >> has a {escript_incl_apps, [...]} config item. It's a bit of a hack, >> but it works. :) >> >> D. >> >> 2011/9/19 H?kan Mattsson : >>> On Mon, Sep 19, 2011 at 4:56 PM, Eric Merritt wrote: >>>> Hey guys, >>>> >>>> ?I am looking at escript and wondering how to go about distributing >>>> more then one OTP app in an escript. That is, a project and it's >>>> dependencies. >>>> >>>> ?From the documentation and tests on my part that validate that it >>>> looks like escript:create can have one of an escript or a code archive >>>> (one and only one) as its body. From reading the docs on code archives >>>> an archive may contain one and only one OTP Application and be named >>>> for that OTP Application. >>> >>> No, you can have several applications in an escript archive. >>> It is just the .ez files that are limited to contain one application. >>> >>> /H?kan >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > From sam@REDACTED Wed Sep 21 06:32:15 2011 From: sam@REDACTED (Sam Bobroff) Date: Wed, 21 Sep 2011 14:32:15 +1000 Subject: [erlang-questions] Erlang release names and versions In-Reply-To: References: Message-ID: Thanks a lot for the tips but I already knew most of it -- I must admit that I was attempting to provoke a more "official" answer from one of the OTP people ;-) Does anyone from the OTP team wish to comment on my original question? All I can get so far is: the provided code and documentation is either wrong or doesn't work so you have to build your own, although some other people have started doing that. That's a pretty poor official answer for how to release Erlang applications, especially when it is strongly tied to code reloading (one of Erlang's "killer features"... if only anyone could understand how to use it). Thanks, Sam. On 13 September 2011 21:43, Samuel Elliott wrote: > Did you see this? It may have answers to some of your questions > http://learnyousomeerlang.com/relups > (including if you need to move generated files) > > Sam > > On Tue, Sep 13, 2011 at 6:17 AM, Sam Bobroff wrote: >> >> Hi Erlangers, >> I'm trying to package up some Erlang projects for distribution, >> possibly by creating Releases, and I'm getting quite confused. I'd >> like to do it the "correct" way because we may want to use >> release_handler to perform upgrades. >> >> Maybe someone can help me out :-) >> >> The documentation for Releases >> (http://www.erlang.org/doc/design_principles/release_structure.html) >> shows a sample directory structure produced by make_tar, which >> includes this part: >> >> releases/A/start.boot >> releases/ch_rel-1.rel >> >> Where ch_rel-1 is the name of the release and A is the version of the >> release. >> >> Question 1: If the release file is placed in releases/ch_rel-1.rel, >> then isn't installing a second version of the release (say, "B") going >> to overwrite the file? Isn't this a problem? Is the addition of "-1" >> to the release name supposed to address this? Shouldn't the release >> name appear somewhere in the path? >> >> Question 2: If another release, say "foo", is installed with the same >> version (say "A"), isn't it going to overwrite the start.boot file >> from release "ch_rel-1"? >> >> Question 3: Should I even be trying to install more than one release >> at once? Would I need a completely new Erlang install directory (or at >> least, releases directory, by using RELDIR)? >> >> (I have, incidentally, seen somewhere that the placement of the .rel >> file directly under releases/ by make_tar is a bug. Is this so? It >> certainly doesn't seem to match the structure created by >> create_target.) >> >> Thanks in advance for any help, >> Sam. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Samuel Elliott > sam@REDACTED > http://lenary.co.uk/ > +44 (0)7891 993 664 > > From daniel.widgren@REDACTED Wed Sep 21 06:50:47 2011 From: daniel.widgren@REDACTED (Daniel Widgren) Date: Wed, 21 Sep 2011 06:50:47 +0200 Subject: [erlang-questions] common test Message-ID: Hi, I tried to do some tests with common test but stumble into some issues. I think it is either spec/config fault. I'm using R14B03. Got some code in usual directories: src ebin priv test doc In test I have the my_SUITE.erl file and in src some .erl files. I have a module that do some stuff before calling ct_master:run with a list of specs. But when ct_master runs the my_SUITE it will not have the path for ebin. When the SUITE is starting up it will get undef on functions that are in modules that are in ebin. So I'm wondering how and where I should add the path to ebin. Function works fine in the module that calls ct_master but not in ct_master. regards, Daniel From neumark.peter@REDACTED Wed Sep 21 09:41:09 2011 From: neumark.peter@REDACTED (Peter Neumark) Date: Wed, 21 Sep 2011 09:41:09 +0200 Subject: [erlang-questions] Connecting Erlang and Javascript using apache thrift over socket.io Message-ID: Hi, Just wanted to let anybody interested know that I've created a proof-of-concept demo for makeing thrift RPC calls between erlang and javascript. What makes it interesting is that both erlang and javascript can act as the server and client. It's still very rough around the edges, but I encourage anyone interested to take a look: https://github.com/postlock/thrift-over-socketio/ Cheers, Peter From watson.timothy@REDACTED Wed Sep 21 10:26:09 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 21 Sep 2011 09:26:09 +0100 Subject: [erlang-questions] common test In-Reply-To: References: Message-ID: > > > So I'm wondering how and where I should add the path to ebin. Function > works fine in the module that calls ct_master but not in ct_master. > > It all depends on how you're starting common_test. If you're using the ct_run script that ships with OTP, then you can pass arguments to it in order to add additional elements to the code path. If you're using rebar then the 'ct' command deals with all of this for you, and if not, looking at the code which rebar uses to build up a command to execute common test should be illuminating: https://github.com/basho/rebar/blob/master/src/rebar_ct.erl#L156 should demonstrate most of the command line options you'll need. Cheers -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Wed Sep 21 10:29:37 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 21 Sep 2011 09:29:37 +0100 Subject: [erlang-questions] logging from linked-in drivers Message-ID: Hi all, Can anyone recommend the best way to do logging inside a linked-in driver? I have debugging printf statements in the code, but I want to do some proper logging (probably to the file system) in production too. The driver is likely to be multi-threaded (using either the async driver API or a custom thread pool) and I wonder how other people have dealt with this in the past. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemenkov@REDACTED Wed Sep 21 10:40:20 2011 From: lemenkov@REDACTED (Peter Lemenkov) Date: Wed, 21 Sep 2011 12:40:20 +0400 Subject: [erlang-questions] logging from linked-in drivers In-Reply-To: References: Message-ID: 2011/9/21 Tim Watson : > Hi all, > Can anyone recommend the best way to do logging inside a linked-in driver? I man 3 syslog -- With best regards, Peter Lemenkov. From erlangsiri@REDACTED Wed Sep 21 10:48:15 2011 From: erlangsiri@REDACTED (Siri Hansen) Date: Wed, 21 Sep 2011 10:48:15 +0200 Subject: [erlang-questions] Erlang release names and versions In-Reply-To: References: Message-ID: Hi Sam - sorry for the delay. This is not yet the "official" answer - it's just to let you know that this OTP person is indeed looking into the issue with the intention of producing a good answer really soon ... :) Regards /siri@REDACTED 2011/9/21 Sam Bobroff > Thanks a lot for the tips but I already knew most of it -- I must > admit that I was attempting to provoke a more "official" answer from > one of the OTP people ;-) > > Does anyone from the OTP team wish to comment on my original question? > > All I can get so far is: the provided code and documentation is either > wrong or doesn't work so you have to build your own, although some > other people have started doing that. > > That's a pretty poor official answer for how to release Erlang > applications, especially when it is strongly tied to code reloading > (one of Erlang's "killer features"... if only anyone could understand > how to use it). > > Thanks, > Sam. > > On 13 September 2011 21:43, Samuel Elliott wrote: > > Did you see this? It may have answers to some of your questions > > http://learnyousomeerlang.com/relups > > (including if you need to move generated files) > > > > Sam > > > > On Tue, Sep 13, 2011 at 6:17 AM, Sam Bobroff wrote: > >> > >> Hi Erlangers, > >> I'm trying to package up some Erlang projects for distribution, > >> possibly by creating Releases, and I'm getting quite confused. I'd > >> like to do it the "correct" way because we may want to use > >> release_handler to perform upgrades. > >> > >> Maybe someone can help me out :-) > >> > >> The documentation for Releases > >> (http://www.erlang.org/doc/design_principles/release_structure.html) > >> shows a sample directory structure produced by make_tar, which > >> includes this part: > >> > >> releases/A/start.boot > >> releases/ch_rel-1.rel > >> > >> Where ch_rel-1 is the name of the release and A is the version of the > >> release. > >> > >> Question 1: If the release file is placed in releases/ch_rel-1.rel, > >> then isn't installing a second version of the release (say, "B") going > >> to overwrite the file? Isn't this a problem? Is the addition of "-1" > >> to the release name supposed to address this? Shouldn't the release > >> name appear somewhere in the path? > >> > >> Question 2: If another release, say "foo", is installed with the same > >> version (say "A"), isn't it going to overwrite the start.boot file > >> from release "ch_rel-1"? > >> > >> Question 3: Should I even be trying to install more than one release > >> at once? Would I need a completely new Erlang install directory (or at > >> least, releases directory, by using RELDIR)? > >> > >> (I have, incidentally, seen somewhere that the placement of the .rel > >> file directly under releases/ by make_tar is a bug. Is this so? It > >> certainly doesn't seem to match the structure created by > >> create_target.) > >> > >> Thanks in advance for any help, > >> Sam. > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > > Samuel Elliott > > sam@REDACTED > > http://lenary.co.uk/ > > +44 (0)7891 993 664 > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Sep 21 10:49:16 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 21 Sep 2011 12:49:16 +0400 Subject: [erlang-questions] logging from linked-in drivers In-Reply-To: References: Message-ID: Tim. I may be 10 times wrong, but it seems for me that the most convenient and simple way of logging is stderr/stdout, captured by runit If you wish to do complicated logging, I recommend sending messages to owner process, that will be rerouted to some erlang level logger. From jesper.louis.andersen@REDACTED Wed Sep 21 14:25:48 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 21 Sep 2011 14:25:48 +0200 Subject: [erlang-questions] Rough thought on a P2P package distribution model for Erlang In-Reply-To: <4E79252C.9070005@ghostgun.com> References: <4E700AFF.7090309@ghostgun.com> <4E79252C.9070005@ghostgun.com> Message-ID: On Wed, Sep 21, 2011 at 01:43, jm wrote: > As you point out these systems "have been around for a very long time" so > why aren't we making better use of these technologies for things like this? Mainly because there is no need to do it. Data distribution systems like BitTorrent has the specific distinct advantage that it can transfer data at high bandwidths, even if the initial source is highly bandwidth constrained in its upstream. Upstream bandwidth scales with demand in a BT network. Hence, the applicability of BitTorrent (and like) protocols hinges on a need to thwart an upstream bandwidth constraint. Example: You are Facebook and need a 1 gigabyte image distributed quickly to 50.000 nodes from a single deploy machine. But as soon as there is demand for a package distribution, and the demand is high enough, mirrors form and mirrors have ample amounts of upstream bandwidth available. Far more than what is needed. Thus, the additional complexity of adding BitTorrent into the mix isn't needed. Add that HTTP transport is well-known and simple. You can say it is a locally extreme value which is currently good enough. The proof is Content Delivery Networks (CDNs) which does not use BitTorrent to distribute content. -- J. From watson.timothy@REDACTED Wed Sep 21 15:29:46 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 21 Sep 2011 14:29:46 +0100 Subject: [erlang-questions] logging from linked-in drivers In-Reply-To: References: Message-ID: On 21 September 2011 09:49, Max Lapshin wrote: > Tim. > I may be 10 times wrong, but it seems for me that the most convenient > and simple way of logging is stderr/stdout, captured by runit > > Yeah, this is what I do with debugging statements. I'd like to put individual log lines/entries out to the file system rather than the console. Given that posix requires stdio operations dealing with FILE* structures to work atomically, do you think fprintf calls are a reasonable thing to do in a multi-threaded driver?. > If you wish to do complicated logging, I recommend sending messages to > owner process, that will be rerouted to some erlang level logger. > I'll have a look into this. I had considered doing so using driver_send_term to send data to an arbitrary (e.g., logging) process that is specified during driver initialisation, but wondered whether the overhead was going to be too high. Also that call is only thread safe when the SMP emulator is in use, which means using a driver mutex the rest of the time. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bighostkim@REDACTED Wed Sep 21 16:43:48 2011 From: bighostkim@REDACTED (Allen Kim) Date: Wed, 21 Sep 2011 10:43:48 -0400 Subject: [erlang-questions] erlang gen_tcp:connect/3 not working with rpc:call/4, anyone knows why? Message-ID: Hi, Could not solve this by myself 1. launched a new node A with ssh command 2. started a new node B 3. gen_tcp:connect/3 works on B, but rpc:call(B,gen_tcp,connect,Params) not works. Both nodes are running on local laptop and one node returns ok and the other node returns error. I don't understand. Anyone knows why? ~ $ssh allen@REDACTED 'erl -name loadtest@REDACTED -detached -setcookie loadtest' ~ $erl -name allen@REDACTED -setcookie loadtest Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.4 (abort with ^G) (allen@REDACTED)1> gen_tcp:connect("www.google.com",80,[]). {ok,#Port<0.630>} (allen@REDACTED)2> rpc:call('loadtest@REDACTED',gen_tcp,connect,["www.google.com",80,[]]). {error,nxdomain} -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Wed Sep 21 16:46:07 2011 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Wed, 21 Sep 2011 15:46:07 +0100 Subject: [erlang-questions] How to run Erlang without installing it Message-ID: Hello, I encountered a regression in erlang dev version and want to bisect it. What I do: $ git checkout otp; cd otp $ ./build_otp all $ bin/erl Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Eshell V5.8.5 (abort with ^G) 1> application:start(mnesia). {error,{"no such file or directory","mnesia.app"}} What I expect to get: 1> application:start(mnesia). ok 2> io:format("~p~n", [code:get_path()]). [".","/dev/shm/otp/lib/kernel/ebin","/dev/shm/otp/lib/stdlib/ebin", "/dev/shm/otp/lib/xmerl/ebin","/dev/shm/otp/lib/wx/ebin", "/dev/shm/otp/lib/webtool/ebin","/dev/shm/otp/lib/typer/ebin", "/dev/shm/otp/lib/tv/ebin","/dev/shm/otp/lib/tools/ebin", "/dev/shm/otp/lib/toolbar/ebin","/dev/shm/otp/lib/test_server/ebin", "/dev/shm/otp/lib/syntax_tools/ebin","/dev/shm/otp/lib/ssl/ebin", "/dev/shm/otp/lib/ssh/ebin","/dev/shm/otp/lib/snmp/ebin", "/dev/shm/otp/lib/sasl/ebin","/dev/shm/otp/lib/runtime_tools/ebin", "/dev/shm/otp/lib/reltool/ebin","/dev/shm/otp/lib/public_key/ebin", "/dev/shm/otp/lib/pman/ebin","/dev/shm/otp/lib/percept/ebin", "/dev/shm/otp/lib/parsetools/ebin","/dev/shm/otp/lib/otp_mibs/ebin", "/dev/shm/otp/lib/os_mon/ebin","/dev/shm/otp/lib/orber/ebin", "/dev/shm/otp/lib/odbc/ebin","/dev/shm/otp/lib/observer/ebin", "/dev/shm/otp/lib/mnesia/ebin","/dev/shm/otp/lib/megaco/ebin", "/dev/shm/otp/lib/jinterface","/dev/shm/otp/lib/inviso/ebin", "/dev/shm/otp/lib/inets/ebin","/dev/shm/otp/lib/ic/ebin", "/dev/shm/otp/lib/hipe/ebin","/dev/shm/otp/lib/gs/ebin", "/dev/shm/otp/lib/eunit/ebin","/dev/shm/otp/lib/et/ebin", "/dev/shm/otp/lib/erl_interface","/dev/shm/otp/lib/erl_docgen/ebin", "/dev/shm/otp/lib/edoc/ebin","/dev/shm/otp/lib/docbuilder/ebin", "/dev/shm/otp/lib/diameter/ebin","/dev/shm/otp/lib/dialyzer/ebin", "/dev/shm/otp/lib/debugger/ebin","/dev/shm/otp/lib/crypto/ebin", "/dev/shm/otp/lib/cosTransactions/ebin","/dev/shm/otp/lib/cosTime/ebin", "/dev/shm/otp/lib/cosProperty/ebin","/dev/shm/otp/lib/cosNotification/ebin", "/dev/shm/otp/lib/cosFileTransfer/ebin", "/dev/shm/otp/lib/cosEventDomain/ebin","/dev/shm/otp/lib/cosEvent/ebin", "/dev/shm/otp/lib/compiler/ebin","/dev/shm/otp/lib/common_test/ebin", "/dev/shm/otp/lib/autom4te.cache","/dev/shm/otp/lib/asn1/ebin", "/dev/shm/otp/lib/appmon/ebin"] I tried to set OTPROOT to `pwd`, ERL_LIBS to `pwd/lib`, but it did not change the outcome. -- Motiejus Jak?tys From kaiduanx@REDACTED Wed Sep 21 17:20:34 2011 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Wed, 21 Sep 2011 11:20:34 -0400 Subject: [erlang-questions] erlang gen_tcp:connect/3 not working with rpc:call/4, anyone knows why? In-Reply-To: References: Message-ID: (kaiduanx@REDACTED)6> net_adm:ping('loadtest@REDACTED'). pong (kaiduanx@REDACTED)7> nodes(). ['loadtest@REDACTED'] (kaiduanx@REDACTED)8> rpc:call('loadtest@REDACTED', gen_tcp, connect, ["www.google.com", 80, []]). {ok,#Port<5752.374>} You need to connect the node first. /Kaiduan On Wed, Sep 21, 2011 at 10:43 AM, Allen Kim wrote: > Hi, > > Could not solve this by myself > > launched a new node A with ssh command > started a new node B > gen_tcp:connect/3 works on B, but rpc:call(B,gen_tcp,connect,Params) not > works. > > Both nodes are running on local laptop > > and one node returns ok and the other node returns error. > > I don't understand. > > Anyone knows why? > > ~ $ssh allen@REDACTED 'erl -name loadtest@REDACTED -detached -setcookie > loadtest' > ~ $erl -name allen@REDACTED -setcookie loadtest > Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > Eshell V5.8.4 (abort with ^G) > (allen@REDACTED)1> gen_tcp:connect("www.google.com",80,[]). > {ok,#Port<0.630>} > (allen@REDACTED)2> > rpc:call('loadtest@REDACTED',gen_tcp,connect,["www.google.com",80,[]]). > {error,nxdomain} > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From lukas.larsson@REDACTED Wed Sep 21 17:49:47 2011 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Wed, 21 Sep 2011 16:49:47 +0100 (BST) Subject: [erlang-questions] How to run Erlang without installing it In-Reply-To: Message-ID: Hello! You probably want to pass the -a switch to otp_build so that all applications get built, i.e. `./otp_build all -a`. The 'all' phrase that you have supplied refers to all stages of compilation (autoconf, configure, emulator and minimal apps). To get all applications as well you need -a. If you type `./otp_build --help` you will get a description of what otp_build does and this is kind of explained in the first lines of the help message. Lukas ----- Original Message ----- From: "Motiejus Jak?tys" To: erlang-questions@REDACTED Sent: Wednesday, September 21, 2011 4:46:07 PM Subject: [erlang-questions] How to run Erlang without installing it Hello, I encountered a regression in erlang dev version and want to bisect it. What I do: $ git checkout otp; cd otp $ ./build_otp all $ bin/erl Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] Eshell V5.8.5 (abort with ^G) 1> application:start(mnesia). {error,{"no such file or directory","mnesia.app"}} What I expect to get: 1> application:start(mnesia). ok 2> io:format("~p~n", [code:get_path()]). [".","/dev/shm/otp/lib/kernel/ebin","/dev/shm/otp/lib/stdlib/ebin", "/dev/shm/otp/lib/xmerl/ebin","/dev/shm/otp/lib/wx/ebin", "/dev/shm/otp/lib/webtool/ebin","/dev/shm/otp/lib/typer/ebin", "/dev/shm/otp/lib/tv/ebin","/dev/shm/otp/lib/tools/ebin", "/dev/shm/otp/lib/toolbar/ebin","/dev/shm/otp/lib/test_server/ebin", "/dev/shm/otp/lib/syntax_tools/ebin","/dev/shm/otp/lib/ssl/ebin", "/dev/shm/otp/lib/ssh/ebin","/dev/shm/otp/lib/snmp/ebin", "/dev/shm/otp/lib/sasl/ebin","/dev/shm/otp/lib/runtime_tools/ebin", "/dev/shm/otp/lib/reltool/ebin","/dev/shm/otp/lib/public_key/ebin", "/dev/shm/otp/lib/pman/ebin","/dev/shm/otp/lib/percept/ebin", "/dev/shm/otp/lib/parsetools/ebin","/dev/shm/otp/lib/otp_mibs/ebin", "/dev/shm/otp/lib/os_mon/ebin","/dev/shm/otp/lib/orber/ebin", "/dev/shm/otp/lib/odbc/ebin","/dev/shm/otp/lib/observer/ebin", "/dev/shm/otp/lib/mnesia/ebin","/dev/shm/otp/lib/megaco/ebin", "/dev/shm/otp/lib/jinterface","/dev/shm/otp/lib/inviso/ebin", "/dev/shm/otp/lib/inets/ebin","/dev/shm/otp/lib/ic/ebin", "/dev/shm/otp/lib/hipe/ebin","/dev/shm/otp/lib/gs/ebin", "/dev/shm/otp/lib/eunit/ebin","/dev/shm/otp/lib/et/ebin", "/dev/shm/otp/lib/erl_interface","/dev/shm/otp/lib/erl_docgen/ebin", "/dev/shm/otp/lib/edoc/ebin","/dev/shm/otp/lib/docbuilder/ebin", "/dev/shm/otp/lib/diameter/ebin","/dev/shm/otp/lib/dialyzer/ebin", "/dev/shm/otp/lib/debugger/ebin","/dev/shm/otp/lib/crypto/ebin", "/dev/shm/otp/lib/cosTransactions/ebin","/dev/shm/otp/lib/cosTime/ebin", "/dev/shm/otp/lib/cosProperty/ebin","/dev/shm/otp/lib/cosNotification/ebin", "/dev/shm/otp/lib/cosFileTransfer/ebin", "/dev/shm/otp/lib/cosEventDomain/ebin","/dev/shm/otp/lib/cosEvent/ebin", "/dev/shm/otp/lib/compiler/ebin","/dev/shm/otp/lib/common_test/ebin", "/dev/shm/otp/lib/autom4te.cache","/dev/shm/otp/lib/asn1/ebin", "/dev/shm/otp/lib/appmon/ebin"] I tried to set OTPROOT to `pwd`, ERL_LIBS to `pwd/lib`, but it did not change the outcome. -- Motiejus Jak?tys _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From allen.kim@REDACTED Wed Sep 21 17:55:32 2011 From: allen.kim@REDACTED (Allen Kim) Date: Wed, 21 Sep 2011 10:55:32 -0500 Subject: [erlang-questions] erlang gen_tcp:connect/3 not working with rpc:call/4, anyone knows why? In-Reply-To: Message-ID: Is it environmental issue? I see the same error on my Mac OS X laptop. ~/open_source/EMG/loadtesterl$ssh allen@REDACTED 'erl -name loadtest@REDACTED -detached -setcookie loadtest' ~/open_source/EMG/loadtesterl$erl -name allen@REDACTED -setcookie loadtest Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.4 (abort with ^G) (allen@REDACTED)1> gen_tcp:connect("www.google.com",80,[]). {ok,#Port<0.617>} (allen@REDACTED)2> net_adm:ping('loadtest@REDACTED'). pong (allen@REDACTED)3> rpc:call('loadtest@REDACTED',gen_tcp,connect,["www.google.com",80,[]]). {error,nxdomain} (allen@REDACTED)4> nodes(). ['loadtest@REDACTED'] On 11-09-21 11:20 AM, "Kaiduan Xie" wrote: >(kaiduanx@REDACTED)6> net_adm:ping('loadtest@REDACTED'). >pong >(kaiduanx@REDACTED)7> nodes(). >['loadtest@REDACTED'] >(kaiduanx@REDACTED)8> rpc:call('loadtest@REDACTED', gen_tcp, >connect, ["www.google.com", 80, []]). >{ok,#Port<5752.374>} > >You need to connect the node first. > >/Kaiduan >On Wed, Sep 21, 2011 at 10:43 AM, Allen Kim wrote: >> Hi, >> >> Could not solve this by myself >> >> launched a new node A with ssh command >> started a new node B >> gen_tcp:connect/3 works on B, but rpc:call(B,gen_tcp,connect,Params) not >> works. >> >> Both nodes are running on local laptop >> >> and one node returns ok and the other node returns error. >> >> I don't understand. >> >> Anyone knows why? >> >> ~ $ssh allen@REDACTED 'erl -name loadtest@REDACTED -detached >>-setcookie >> loadtest' >> ~ $erl -name allen@REDACTED -setcookie loadtest >> Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] >> [async-threads:0] [hipe] [kernel-poll:false] >> Eshell V5.8.4 (abort with ^G) >> (allen@REDACTED)1> gen_tcp:connect("www.google.com",80,[]). >> {ok,#Port<0.630>} >> (allen@REDACTED)2> >> rpc:call('loadtest@REDACTED',gen_tcp,connect,["www.google.com",80,[]]). >> {error,nxdomain} >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> From kaiduanx@REDACTED Wed Sep 21 18:32:36 2011 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Wed, 21 Sep 2011 12:32:36 -0400 Subject: [erlang-questions] erlang gen_tcp:connect/3 not working with rpc:call/4, anyone knows why? In-Reply-To: References: Message-ID: I tried on Ubuntu Linux, and net_adm:ping() is not necessary. How is your DNS configured? What it looks like in /etc/resolve.conf file? On Wed, Sep 21, 2011 at 11:55 AM, Allen Kim wrote: > Is it environmental issue? > I see the same error on my Mac OS X laptop. > > ~/open_source/EMG/loadtesterl$ssh allen@REDACTED ?'erl -name > loadtest@REDACTED -detached -setcookie loadtest' > ~/open_source/EMG/loadtesterl$erl -name allen@REDACTED -setcookie loadtest > Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.4 ?(abort with ^G) > (allen@REDACTED)1> gen_tcp:connect("www.google.com",80,[]). > {ok,#Port<0.617>} > (allen@REDACTED)2> net_adm:ping('loadtest@REDACTED'). > pong > (allen@REDACTED)3> > rpc:call('loadtest@REDACTED',gen_tcp,connect,["www.google.com",80,[]]). > {error,nxdomain} > (allen@REDACTED)4> nodes(). > ['loadtest@REDACTED'] > > > > > > > > > > On 11-09-21 11:20 AM, "Kaiduan Xie" wrote: > >>(kaiduanx@REDACTED)6> net_adm:ping('loadtest@REDACTED'). >>pong >>(kaiduanx@REDACTED)7> nodes(). >>['loadtest@REDACTED'] >>(kaiduanx@REDACTED)8> rpc:call('loadtest@REDACTED', gen_tcp, >>connect, ["www.google.com", 80, []]). >>{ok,#Port<5752.374>} >> >>You need to connect the node first. >> >>/Kaiduan >>On Wed, Sep 21, 2011 at 10:43 AM, Allen Kim wrote: >>> Hi, >>> >>> Could not solve this by myself >>> >>> launched a new node A with ssh command >>> started a new node B >>> gen_tcp:connect/3 works on B, but rpc:call(B,gen_tcp,connect,Params) not >>> works. >>> >>> Both nodes are running on local laptop >>> >>> and one node returns ok and the other node returns error. >>> >>> I don't understand. >>> >>> Anyone knows why? >>> >>> ~ $ssh allen@REDACTED ?'erl -name loadtest@REDACTED -detached >>>-setcookie >>> loadtest' >>> ~ $erl -name allen@REDACTED -setcookie loadtest >>> Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] >>> [async-threads:0] [hipe] [kernel-poll:false] >>> Eshell V5.8.4 ?(abort with ^G) >>> (allen@REDACTED)1> gen_tcp:connect("www.google.com",80,[]). >>> {ok,#Port<0.630>} >>> (allen@REDACTED)2> >>> rpc:call('loadtest@REDACTED',gen_tcp,connect,["www.google.com",80,[]]). >>> {error,nxdomain} >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> > > From allen.kim@REDACTED Wed Sep 21 18:35:49 2011 From: allen.kim@REDACTED (Allen Kim) Date: Wed, 21 Sep 2011 11:35:49 -0500 Subject: [erlang-questions] erlang gen_tcp:connect/3 not working with rpc:call/4, anyone knows why? In-Reply-To: Message-ID: $cat /etc/resolv.conf # # Mac OS X Notice # # This file is not used by the host name and address resolution # or the DNS query routing mechanisms used by most processes on # this Mac OS X system. # # This file is automatically generated. # nameserver 192.168.14.10nameserver 192.168.24.10 On 11-09-21 12:32 PM, "Kaiduan Xie" wrote: >I tried on Ubuntu Linux, and net_adm:ping() is not necessary. How is >your DNS configured? What it looks like in /etc/resolve.conf file? > >On Wed, Sep 21, 2011 at 11:55 AM, Allen Kim > wrote: >> Is it environmental issue? >> I see the same error on my Mac OS X laptop. >> >> ~/open_source/EMG/loadtesterl$ssh allen@REDACTED 'erl -name >> loadtest@REDACTED -detached -setcookie loadtest' >> ~/open_source/EMG/loadtesterl$erl -name allen@REDACTED -setcookie >>loadtest >> Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] >> [async-threads:0] [hipe] [kernel-poll:false] >> >> Eshell V5.8.4 (abort with ^G) >> (allen@REDACTED)1> gen_tcp:connect("www.google.com",80,[]). >> {ok,#Port<0.617>} >> (allen@REDACTED)2> net_adm:ping('loadtest@REDACTED'). >> pong >> (allen@REDACTED)3> >> rpc:call('loadtest@REDACTED',gen_tcp,connect,["www.google.com",80,[]]). >> {error,nxdomain} >> (allen@REDACTED)4> nodes(). >> ['loadtest@REDACTED'] >> >> >> >> >> >> >> >> >> >> On 11-09-21 11:20 AM, "Kaiduan Xie" wrote: >> >>>(kaiduanx@REDACTED)6> net_adm:ping('loadtest@REDACTED'). >>>pong >>>(kaiduanx@REDACTED)7> nodes(). >>>['loadtest@REDACTED'] >>>(kaiduanx@REDACTED)8> rpc:call('loadtest@REDACTED', gen_tcp, >>>connect, ["www.google.com", 80, []]). >>>{ok,#Port<5752.374>} >>> >>>You need to connect the node first. >>> >>>/Kaiduan >>>On Wed, Sep 21, 2011 at 10:43 AM, Allen Kim >>>wrote: >>>> Hi, >>>> >>>> Could not solve this by myself >>>> >>>> launched a new node A with ssh command >>>> started a new node B >>>> gen_tcp:connect/3 works on B, but rpc:call(B,gen_tcp,connect,Params) >>>>not >>>> works. >>>> >>>> Both nodes are running on local laptop >>>> >>>> and one node returns ok and the other node returns error. >>>> >>>> I don't understand. >>>> >>>> Anyone knows why? >>>> >>>> ~ $ssh allen@REDACTED 'erl -name loadtest@REDACTED -detached >>>>-setcookie >>>> loadtest' >>>> ~ $erl -name allen@REDACTED -setcookie loadtest >>>> Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] >>>> [async-threads:0] [hipe] [kernel-poll:false] >>>> Eshell V5.8.4 (abort with ^G) >>>> (allen@REDACTED)1> gen_tcp:connect("www.google.com",80,[]). >>>> {ok,#Port<0.630>} >>>> (allen@REDACTED)2> >>>> >>>>rpc:call('loadtest@REDACTED',gen_tcp,connect,["www.google.com",80,[]]) >>>>. >>>> {error,nxdomain} >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >> >> From allen.kim@REDACTED Wed Sep 21 19:03:49 2011 From: allen.kim@REDACTED (Allen Kim) Date: Wed, 21 Sep 2011 12:03:49 -0500 Subject: [erlang-questions] erlang gen_tcp:connect/3 not working with rpc:call/4, anyone knows why? In-Reply-To: Message-ID: It IS Mac OS X issue, definitely. I ran it from CentOS box, and it works fine. Don't know why it does not work on Mac OS X though. On 11-09-21 12:35 PM, "Allen Kim" wrote: >$cat /etc/resolv.conf ># ># Mac OS X Notice ># ># This file is not used by the host name and address resolution ># or the DNS query routing mechanisms used by most processes on ># this Mac OS X system. ># ># This file is automatically generated. ># >nameserver 192.168.14.10nameserver 192.168.24.10 > > > > >On 11-09-21 12:32 PM, "Kaiduan Xie" wrote: > >>I tried on Ubuntu Linux, and net_adm:ping() is not necessary. How is >>your DNS configured? What it looks like in /etc/resolve.conf file? >> >>On Wed, Sep 21, 2011 at 11:55 AM, Allen Kim >> wrote: >>> Is it environmental issue? >>> I see the same error on my Mac OS X laptop. >>> >>> ~/open_source/EMG/loadtesterl$ssh allen@REDACTED 'erl -name >>> loadtest@REDACTED -detached -setcookie loadtest' >>> ~/open_source/EMG/loadtesterl$erl -name allen@REDACTED -setcookie >>>loadtest >>> Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] >>> [async-threads:0] [hipe] [kernel-poll:false] >>> >>> Eshell V5.8.4 (abort with ^G) >>> (allen@REDACTED)1> gen_tcp:connect("www.google.com",80,[]). >>> {ok,#Port<0.617>} >>> (allen@REDACTED)2> net_adm:ping('loadtest@REDACTED'). >>> pong >>> (allen@REDACTED)3> >>> >>>rpc:call('loadtest@REDACTED',gen_tcp,connect,["www.google.com",80,[]]). >>> {error,nxdomain} >>> (allen@REDACTED)4> nodes(). >>> ['loadtest@REDACTED'] >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> On 11-09-21 11:20 AM, "Kaiduan Xie" wrote: >>> >>>>(kaiduanx@REDACTED)6> net_adm:ping('loadtest@REDACTED'). >>>>pong >>>>(kaiduanx@REDACTED)7> nodes(). >>>>['loadtest@REDACTED'] >>>>(kaiduanx@REDACTED)8> rpc:call('loadtest@REDACTED', gen_tcp, >>>>connect, ["www.google.com", 80, []]). >>>>{ok,#Port<5752.374>} >>>> >>>>You need to connect the node first. >>>> >>>>/Kaiduan >>>>On Wed, Sep 21, 2011 at 10:43 AM, Allen Kim >>>>wrote: >>>>> Hi, >>>>> >>>>> Could not solve this by myself >>>>> >>>>> launched a new node A with ssh command >>>>> started a new node B >>>>> gen_tcp:connect/3 works on B, but rpc:call(B,gen_tcp,connect,Params) >>>>>not >>>>> works. >>>>> >>>>> Both nodes are running on local laptop >>>>> >>>>> and one node returns ok and the other node returns error. >>>>> >>>>> I don't understand. >>>>> >>>>> Anyone knows why? >>>>> >>>>> ~ $ssh allen@REDACTED 'erl -name loadtest@REDACTED -detached >>>>>-setcookie >>>>> loadtest' >>>>> ~ $erl -name allen@REDACTED -setcookie loadtest >>>>> Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] >>>>> [async-threads:0] [hipe] [kernel-poll:false] >>>>> Eshell V5.8.4 (abort with ^G) >>>>> (allen@REDACTED)1> gen_tcp:connect("www.google.com",80,[]). >>>>> {ok,#Port<0.630>} >>>>> (allen@REDACTED)2> >>>>> >>>>>rpc:call('loadtest@REDACTED',gen_tcp,connect,["www.google.com",80,[]] >>>>>) >>>>>. >>>>> {error,nxdomain} >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>> >>> > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions From dave@REDACTED Wed Sep 21 19:40:21 2011 From: dave@REDACTED (David Goehrig) Date: Wed, 21 Sep 2011 13:40:21 -0400 Subject: [erlang-questions] Right direction ? Message-ID: <7AA05035-2AB7-4CB5-8FF0-A29A6D7455FF@nexttolast.com> Last night I began hacking on code.erl, code_server.erl, and looking to extend load_file(Module :: atom()) to include a load_file(Module, Url) function that would look for the module at the associated URL rather than searching for the file via abs path. It would then compare the sha256 hash of the file against the copy in cache and bail if they are different (no hash in cache adds it and uses as the baseline). I was wondering if there was a good way to verify that a .beam has not been modified since last load. For example: -module(my_mod) -require(daves_mod,"http://erlang.dloh.org/") Could then look for http://erlang.dloh.org/daves_mod.erl and download and compile a local beam. Once I have that beam, I can just load from cache, but what happens if the beam is modified after compilation? The other thing I would like to add is DNS TXT records that could be published sha256 hashes of each source module. http://erlang.dloh.org/daves_mod.erl 45663AFDA.... Adding a -signature("http://erlang.dlog.org/daves_mod.erl","45663AFDA....") Would allow a 3 part verification of the source: 1.) I can compute the source has the right hash 2.) I can look up that the module has the same published signature 3.) I can verify against the original at the specified URL In this scenario it is not enough to modify the source and rehash, nor enough to replace the upsteam file, but also replace the DNS entry without anyone noticing. Thoughts? Dave -=-=- dave@REDACTED -=-=- From magnus.klaar@REDACTED Wed Sep 21 20:18:51 2011 From: magnus.klaar@REDACTED (Magnus Klaar) Date: Wed, 21 Sep 2011 20:18:51 +0200 Subject: [erlang-questions] erlang gen_tcp:connect/3 not working with rpc:call/4, anyone knows why? In-Reply-To: References: Message-ID: Hi! The Erlang VM provides multiple strategies for name lookups, among them is a native lookup mechamism using getaddrinfo and an erlang client implemented by the inet_res module. The contents of the inet_db table on the detached node may also be worth looking into, does it differ? which strategy does it use?. You can access it using: rpc:call('loadtest@REDACTED', ets,tab2list,[inet_db]). You may also want to look into the mDNSResponder daemon that OSX uses as a system-wide resolver. this daemon appears to be an alternative to the nscd deamon found on GNU/Linux systems. Applications are expected to perform lookups against this daemon. MVH Magnus On Wed, Sep 21, 2011 at 7:03 PM, Allen Kim wrote: > It IS Mac OS X issue, definitely. > I ran it from CentOS box, and it works fine. > Don't know why it does not work on Mac OS X though. > > > On 11-09-21 12:35 PM, "Allen Kim" wrote: > > >$cat /etc/resolv.conf > ># > ># Mac OS X Notice > ># > ># This file is not used by the host name and address resolution > ># or the DNS query routing mechanisms used by most processes on > ># this Mac OS X system. > ># > ># This file is automatically generated. > ># > >nameserver 192.168.14.10nameserver 192.168.24.10 > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From allen.kim@REDACTED Wed Sep 21 20:48:57 2011 From: allen.kim@REDACTED (Allen Kim) Date: Wed, 21 Sep 2011 13:48:57 -0500 Subject: [erlang-questions] erlang gen_tcp:connect/3 not working with rpc:call/4, anyone knows why? In-Reply-To: Message-ID: I don't see any difference from outputs from the following commands. It both uses native lookup. Anyway, it is not a big deal since I do not use Mac for staging nor production. it's only for development. However, I hope it's better to be consistent on all kind of *nix environment. Or, it could be me only. Any Mac user who what to test this? $ssh allen@REDACTED 'erl -name loadtest@REDACTED -detached -setcookie loadtest' $erl -name allen@REDACTED -setcookie loadtest > net_adm:ping('loadtest@REDACTED'). > gen_tcp:connect("www.google.com",80,[]). >rpc:call('loadtest@REDACTED',gen_tcp,connect,["www.google.com",80,[]]). (allen@REDACTED)6> io:format("~p~n",[ets:tab2list(inet_db)]). [{res_resolv_conf_info,undefined}, {res_hosts_file_info,undefined}, {res_domain,"local"}, {res_hosts_file_tm,0}, {res_resolv_conf_tm,0}, {cache_refresh_interval,3600000}, {socks5_noproxy,[]}, {res_retry,3}, {res_search,["local"]}, {res_resolv_conf,"/etc/resolv.conf"}, {res_inet6,false}, {res_recurse,true}, {hostname,"allen"}, {res_udp_payload_size,1280}, {cache_size,100}, {sctp_module,inet_sctp}, {udp_module,inet_udp}, {tcp_module,inet_tcp}, {res_hosts_file,"/etc/hosts"}, {res_id,0}, {socks5_port,1080}, {res_timeout,2000}, {res_edns,false}, {res_alt_ns,[]}, {socks5_methods,[none]}, {res_ns,[{{192,168,14,10},53},{{192,168,24,10},53}]}, {res_usevc,false}, {socks5_server,[]}, {res_lookup,[native]}] ok (allen@REDACTED)7> io:format("~p~n",[rpc:call('loadtest@REDACTED', ets,tab2list,[inet_db])]). [{res_resolv_conf_info,undefined}, {res_hosts_file_info,undefined}, {res_domain,"local"}, {res_hosts_file_tm,0}, {res_resolv_conf_tm,0}, {cache_refresh_interval,3600000}, {socks5_noproxy,[]}, {res_retry,3}, {res_search,["local"]}, {res_resolv_conf,"/etc/resolv.conf"}, {res_inet6,false}, {res_recurse,true}, {hostname,"allen"}, {res_udp_payload_size,1280}, {cache_size,100}, {sctp_module,inet_sctp}, {udp_module,inet_udp}, {tcp_module,inet_tcp}, {res_hosts_file,"/etc/hosts"}, {res_id,0}, {socks5_port,1080}, {res_timeout,2000}, {res_edns,false}, {res_alt_ns,[]}, {socks5_methods,[none]}, {res_ns,[{{192,168,14,10},53},{{192,168,24,10},53}]}, {res_usevc,false}, {socks5_server,[]}, {res_lookup,[native]}] ok From: Magnus Klaar > Date: Wed, 21 Sep 2011 13:18:51 -0500 To: Allen Kim > Cc: Kaiduan Xie >, "erlang-questions@REDACTED" > Subject: Re: [erlang-questions] erlang gen_tcp:connect/3 not working with rpc:call/4, anyone knows why? Hi! The Erlang VM provides multiple strategies for name lookups, among them is a native lookup mechamism using getaddrinfo and an erlang client implemented by the inet_res module. The contents of the inet_db table on the detached node may also be worth looking into, does it differ? which strategy does it use?. You can access it using: rpc:call('loadtest@REDACTED', ets,tab2list,[inet_db]). You may also want to look into the mDNSResponder daemon that OSX uses as a system-wide resolver. this daemon appears to be an alternative to the nscd deamon found on GNU/Linux systems. Applications are expected to perform lookups against this daemon. MVH Magnus On Wed, Sep 21, 2011 at 7:03 PM, Allen Kim > wrote: It IS Mac OS X issue, definitely. I ran it from CentOS box, and it works fine. Don't know why it does not work on Mac OS X though. On 11-09-21 12:35 PM, "Allen Kim" > wrote: >$cat /etc/resolv.conf ># ># Mac OS X Notice ># ># This file is not used by the host name and address resolution ># or the DNS query routing mechanisms used by most processes on ># this Mac OS X system. ># ># This file is automatically generated. ># >nameserver 192.168.14.10nameserver 192.168.24.10 > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raould@REDACTED Thu Sep 22 01:00:45 2011 From: raould@REDACTED (Raoul Duke) Date: Wed, 21 Sep 2011 16:00:45 -0700 Subject: [erlang-questions] now fares node.js ideas vs. erlang otp ideas? Message-ID: i haven't been able to parse all the terminology yet. i always have to wonder when somebody has something new like node.js and then there's a band-wagon around it, how much is a new syntax for already-existing ideas? http://www.infoq.com/articles/multi-core-node-js From igarai@REDACTED Thu Sep 22 01:19:50 2011 From: igarai@REDACTED (=?UTF-8?B?ScOxYWtpIEdhcmF5?=) Date: Wed, 21 Sep 2011 16:19:50 -0700 Subject: [erlang-questions] now fares node.js ideas vs. erlang otp ideas? In-Reply-To: References: Message-ID: On Wed, Sep 21, 2011 at 16:00, Raoul Duke wrote: > i haven't been able to parse all the terminology yet. i always have to > wonder when somebody has something new like node.js and then there's a > band-wagon around it, how much is a new syntax for already-existing > ideas? This has been talked about some, jlouis has a good (IMHO) post about it [0], not all agree as always. Consensus is that node.js is nice but isn't about fault tolerance, which many people here care about much more. Others with more battle experience will have deeper insight. [0] http://jlouisramblings.blogspot.com/2011/05/nodejs-vs-erlang-is-wrong-battle.html good luck, I?aki Garay. From jesper.louis.andersen@REDACTED Thu Sep 22 02:38:23 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 22 Sep 2011 02:38:23 +0200 Subject: [erlang-questions] now fares node.js ideas vs. erlang otp ideas? In-Reply-To: References: Message-ID: On Thu, Sep 22, 2011 at 01:00, Raoul Duke wrote: > i haven't been able to parse all the terminology yet. i always have to > wonder when somebody has something new like node.js and then there's a > band-wagon around it, how much is a new syntax for already-existing > ideas? In Node.js case, it is a simple result of a number of moons aligning up. Basically, a number of things evolve, seemingly unrelated: The Apache webserver and the select() system call is evolved into the epoll() system call and web servers like nginx or lighttpd. Rather than a multi-process webserver, like Apache, these run a single web server process and handle everything in an epoll() loop. An embedded language for web page manipulation in Netscapes code base evolves over firefox and mozilla into a more full language. Then Google gets behind it fully with their Chrome browser and their V8 team. This provides the world with a world class javascript system. Event processing frameworks evolve by their use of continuations as a way to chain up events firing. Up pops frameworks like Eventmachine, Twisted, Netty, POE and so on. And then, since the building blocks are there, it is only a matter of time until someone has the audacity to merge them into one. Each part brings something to the table. Javascript is familarity and the ability to use the _same language_ at front and backend respectively. The event processing and epoll() parts provide the system an ability to run at quite large scale. Is it a new thing? Indeed it is. Like when The Prodigy sampled their way to "Smack my Bitch up": http://www.youtube.com/watch?v=eU5Dn-WaElI (The Sheila Chandra part is just beautiful in its own right btw). The mix of existing technologies provides something new. For one, it provides Javascript programmers who has mostly lived in web browsers a venue unto the server side, with something they already know and love. And the programming model is not too foreign if you already know some of the pieces. It is a very different mix of technologies compared to Erlang, but as I hint, it is another piece on the Chessboard. It is a stake that another model can be useful for a set of problems. Erlang substitutes a lot. The process model is fully preemptive and not cooperative as in Node.js. The language is functional, contrast to Javascripts prototypical OO imperative language. The focus is on writing robust programs in Erlang, an emphasis I have not found any other language which focus on. Note there is a subtle difference between robust and correct. A program is correct if it lives up to the frame of specification. A program is robust if it survives an event that was not part of the specification - outside the box :) And as such, the problems which Erlangs shines at is probably going to be different compared to the problems where Node.js will be known. Since the languages are so different in their basic nature, surely the problems they solve well will also be. -- J. From cgsmcmlxxv@REDACTED Thu Sep 22 10:40:54 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Thu, 22 Sep 2011 10:40:54 +0200 Subject: [erlang-questions] now fares node.js ideas vs. erlang otp ideas? In-Reply-To: References: Message-ID: There is nothing really new, except, maybe, for the idea itself. I've watched some of their presentations and they admitted that they didn't invent anything new, but just converging technologies. They use better the threading system (it's faster to reuse a thread than to create a new thread in linux, even if here many of you will jump on me - just try it before doing that), so the events seem to be processed when they occur, and not delayed by the time to spawn a thread. I do not deny the idea itself which I consider it magical. To have the same language back and frontend makes the life of a programmer much easier. Still, there are few disadvantages in programming under JavaScript, but that's not the point of this subject. Node.js is "young". By now they used the existing technology and that gave them a good start. Personally, I will follow their development with interest and I am wishing them to remain on the market and not to suffer the faith of the other attempts in that field. On Thu, Sep 22, 2011 at 1:00 AM, Raoul Duke wrote: > i haven't been able to parse all the terminology yet. i always have to > wonder when somebody has something new like node.js and then there's a > band-wagon around it, how much is a new syntax for already-existing > ideas? > > http://www.infoq.com/articles/multi-core-node-js > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Thu Sep 22 11:35:13 2011 From: erlangsiri@REDACTED (Siri Hansen) Date: Thu, 22 Sep 2011 11:35:13 +0200 Subject: [erlang-questions] Erlang release names and versions In-Reply-To: References: Message-ID: Hi Sam! After some research I agree there are some strange things here, and it might be a good idea to do some adjustments... >> The documentation for Releases >> (http://www.erlang.org/doc/design_principles/release_structure.html) >> shows a sample directory structure produced by make_tar, which >> includes this part: >> >> releases/A/start.boot >> releases/ch_rel-1.rel >> >> Where ch_rel-1 is the name of the release and A is the version of the >> release. To be specific, ch_rel-1 is really only the name of the release resource file, and not to be confused with the name of the release, which is stated inside this file: {release, {RelName, Vsn}, ....} I believe that a good recommendation could be to name the release resource file "RelName-Vsn.rel" (at least with the current handling by make_tar etc). >> Question 1: If the release file is placed in releases/ch_rel-1.rel, >> then isn't installing a second version of the release (say, "B") going >> to overwrite the file? Isn't this a problem? Is the addition of "-1" >> to the release name supposed to address this? Shouldn't the release >> name appear somewhere in the path? There are two situations where the .rel file is placed directly under the releases directory. First, it is done by systools:make_tar - and second (as a result of this) it is done by target_system.erl, which is used in the System Principles User's Guide as an example of how to create the first target system. As far as I can see, the reason for make_tar to store the .rel file directly under releases is that this makes it easier to extract only this file from the tar before unpacking all files (see release_handler:unpack_release). The version of the release is not known before this file is extracted - so it would be difficult to point out the file if it was stored in releases/Vsn ... There are of course other ways to go around this problem, but I guess this is the explanation for the strange behavior anyway. If you study release_handler:unpack_release you will also see that it later moves the .rel file to releases/Vsn... However, the very first release is of course not unpacked by release_handler:unpack_release, and if you follow the documentation/example then the .rel file of this release will be stored in the releases directory. And obviously you are correct - if the .rel file in the next release has the same name, it will be overwritten when the next release is unpacked (by release_handler:unpack_release) since it is extracted to the releases directory first before it is moved. I can only conclude that (at least) there is a bug in the documentation/example... >> Question 2: If another release, say "foo", is installed with the same >> version (say "A"), isn't it going to overwrite the start.boot file >> from release "ch_rel-1"? That would actually not be allowed by the release_handler - in a running system you can only install a release version once. The release_handler does not care about the name of the release, it only deals with the version here. However, you can of course unpack such a release manually and then you would have a problem. >> Question 3: Should I even be trying to install more than one release >> at once? Would I need a completely new Erlang install directory (or at >> least, releases directory, by using RELDIR)? Yes, the release_handler is intended for installing multiple releases under one root. It will work if you make sure that your .rel files have unique names (and you only have one release packet of a specific version). But I would also suggest that you move the .rel file of the first installation to releases/Vsn. This should be done by (your equivalence to) target_system:install/2. >> (I have, incidentally, seen somewhere that the placement of the .rel >> file directly under releases/ by make_tar is a bug. Is this so? It >> certainly doesn't seem to match the structure created by >> create_target.) I agree it is a bit confusing, but given the explanation above regarding make_tar, I don't think it is a bug that the file is stored under releases in the tar file. I rather think that the documentation and the target_system.erl has a bug, and we need to correct that. (What do you mean by 'create_target'?? Is it reltool:create_target/2?) I hope this was of some help! Thanks for pointing out this problem. The documentation and example will be improved in R15B. Regards /siri -------------- next part -------------- An HTML attachment was scrubbed... URL: From grimlog@REDACTED Thu Sep 22 11:54:03 2011 From: grimlog@REDACTED (Michael Schreckenbauer) Date: Thu, 22 Sep 2011 11:54:03 +0200 Subject: [erlang-questions] now fares node.js ideas vs. erlang otp ideas? In-Reply-To: References: Message-ID: <2509124.cr6imca3P0@pc> On Thursday, 22. September 2011 02:38:23 Jesper Louis Andersen wrote: > Is it a new thing? Indeed it is. Like when The Prodigy sampled their > way to "Smack my Bitch up": > > http://www.youtube.com/watch?v=eU5Dn-WaElI > > (The Sheila Chandra part is just beautiful in its own right btw). Wow! Many thanks for this link :) Best, Michael From kenji.rikitake@REDACTED Thu Sep 22 12:07:27 2011 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Thu, 22 Sep 2011 19:07:27 +0900 Subject: [erlang-questions] Fwd: ErlLounge after Erlang Workshop tomorrow on 23-SEP-2011 In-Reply-To: References: Message-ID: ErlLounge after Erlang Workshop 2011: 23-SEP-2011 7pm JST at Masuya Jimbocho (Izakaya) http://r.gnavi.co.jp/a683701/map/ (printable map in Japanese) Budget: JPY 5000 25 or more participants needed - so please join! :) (All Erlangers welcome) Regards, Kenji Rikitake From matthias@REDACTED Thu Sep 22 14:08:41 2011 From: matthias@REDACTED (Matthias Lang) Date: Thu, 22 Sep 2011 14:08:41 +0200 Subject: [erlang-questions] why can't _ be passed around? Message-ID: <20110922120841.GA17290@corelatus.se> Hi, Idle curiosity: why can't I pass or send _? I can write: f() -> receive _ -> ok after 500 -> exit(timeout) end. But I can't write g(Pattern) -> receive Pattern -> ok after 500 -> exit(timeout) end. f() -> g(_). Nor can I write f() -> g({my_atom, 3, _}). Ok, I realise I'm not allowed to because _ isn't bound. But _ can't ever be bound (right? or is there another way of thinking about _?). I've written g/1 as a macro. There doesn't seem to be a catch. Thoughts? Am I asking for patterns to be first class? Whoever wrote ets:match() must have had a similar train of thought. Matt From carlsson.richard@REDACTED Thu Sep 22 14:28:12 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 22 Sep 2011 14:28:12 +0200 Subject: [erlang-questions] why can't _ be passed around? In-Reply-To: <20110922120841.GA17290@corelatus.se> References: <20110922120841.GA17290@corelatus.se> Message-ID: <4E7B29DC.9@gmail.com> On 09/22/2011 02:08 PM, Matthias Lang wrote: > Hi, > > Idle curiosity: why can't I pass or send _? > > I can write: > > f() -> > receive _ -> ok > after 500 -> exit(timeout) end. > > But I can't write > > g(Pattern) -> > receive Pattern -> ok > after 500 -> exit(timeout) end. > > f() -> > g(_). > > Nor can I write > > f() -> > g({my_atom, 3, _}). > > Ok, I realise I'm not allowed to because _ isn't bound. But _ can't > ever be bound (right? or is there another way of thinking about _?). The way to think about _ is that it's a shorthand for "replace this underscore with some variable that doesn't occur anywhere else and will not be used again, not even for some other underscore". Which is really what the compiler will do with it (check out the generated Core Erlang code if you like: erlc +to_core foo.erl). > Thoughts? Am I asking for patterns to be first class? You're basically asking for Prolog. I.e., passing around uninstantiated variables ("holes" as first class objects), and letting the called code fill in the holes (using unification). The holes can occur within a partially instantiated structure, as in your {my_atom, 3, _}. And the bindings propagate back to the caller, so you could do: f() -> g({my_atom, 3, Hole}), Hole + 1. g({my_atom, 3, 4}) -> ok. and have f() return 5. (The above is a hypothetical example using Erlang syntax - it's not proper Prolog.) Cool and mindbending things can be done this way (difference lists: see e.g. http://bulba.sdsu.edu/prolog/diff_list/diff_list.htm), but I'm not sure I want that kind of code in a 100+KLOC system. /Richard From olopierpa@REDACTED Thu Sep 22 14:46:00 2011 From: olopierpa@REDACTED (Pierpaolo Bernardi) Date: Thu, 22 Sep 2011 14:46:00 +0200 Subject: [erlang-questions] why can't _ be passed around? In-Reply-To: <20110922120841.GA17290@corelatus.se> References: <20110922120841.GA17290@corelatus.se> Message-ID: On Thu, Sep 22, 2011 at 14:08, Matthias Lang wrote: > Hi, > > Idle curiosity: why can't I pass or send _? > > I can write: > > ?f() -> > ? ?receive _ -> ok > ? ?after 500 -> exit(timeout) end. > > But I can't write > > ?g(Pattern) -> > ? ?receive Pattern -> ok > ? ?after 500 -> exit(timeout) end. > > ?f() -> > ? ?g(_). > > Nor can I write > > ?f() -> > ? ?g({my_atom, 3, _}). > > Ok, I realise I'm not allowed to because _ isn't bound. But _ can't > ever be bound (right? or is there another way of thinking about _?). Pattern matching is syntax: it is just a very concise way to write a fixed (at compile time) cascade of nested case's. > I've written g/1 as a macro. There doesn't seem to be a catch. so, yes, macros would be the right tool, if Erlang had good macros. > Thoughts? Am I asking for patterns to be first class? It looks so to me. From watson.timothy@REDACTED Thu Sep 22 14:46:56 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 22 Sep 2011 13:46:56 +0100 Subject: [erlang-questions] erlang gen_tcp:connect/3 not working with rpc:call/4, anyone knows why? In-Reply-To: References: Message-ID: On 21 September 2011 19:48, Allen Kim wrote: > I don't see any difference from outputs from the following commands. > It both uses native lookup. > Anyway, it is not a big deal since I do not use Mac for staging nor > production. it's only for development. > > However, I hope it's better to be consistent on all kind of *nix > environment. > Or, it could be me only. > > Any Mac user who what to test this? > Yes I've noticed similar problems before: http://hyperthunk.wordpress.com/2010/05/18/things-i-always-forget-about-resolving-erlang-nodes-on-localhost/ I don't know if the issues are connected, but it does seem like OSX "badness" as on various linuxes and solaris it doesn't seem to crop up at all. -------------- next part -------------- An HTML attachment was scrubbed... URL: From magnus.klaar@REDACTED Thu Sep 22 16:53:45 2011 From: magnus.klaar@REDACTED (Magnus Klaar) Date: Thu, 22 Sep 2011 16:53:45 +0200 Subject: [erlang-questions] erlang gen_tcp:connect/3 not working with rpc:call/4, anyone knows why? In-Reply-To: References: Message-ID: Hi! If it uses native lookups we can assume that getaddrinfo is broken when it is called from the detached node. This narrows down the search for the cause of this issue to a more general problem with getaddrinfo on os-x, leading us to this thread: http://lists.apple.com/archives/unix-porting/2010/Jul/msg00001.html You could try just adding nohup to the command that you're executing using ssh. Although i have a hard time seeing how this would work. An erlang node started with -detached should ignore HUP signals (afaik). MVH Magnus On Wed, Sep 21, 2011 at 8:48 PM, Allen Kim wrote: > I don't see any difference from outputs from the following commands. > It both uses native lookup. > Anyway, it is not a big deal since I do not use Mac for staging nor > production. it's only for development. > > However, I hope it's better to be consistent on all kind of *nix > environment. > Or, it could be me only. > > Any Mac user who what to test this? > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Thu Sep 22 17:08:26 2011 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 22 Sep 2011 16:08:26 +0100 (BST) Subject: [erlang-questions] Erlounge or informal EUG meeting in Moscow Message-ID: I will be at the HighLoad++ conference in Moscow next weekend, 1-4/10, anyone for an Erlounge or maybe an informal Erlang user group meeting? From max.lapshin@REDACTED Thu Sep 22 17:10:19 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 22 Sep 2011 19:10:19 +0400 Subject: [erlang-questions] Erlounge or informal EUG meeting in Moscow In-Reply-To: References: Message-ID: Robert, it would be great to meet! On Thu, Sep 22, 2011 at 7:08 PM, Robert Virding wrote: > I will be at the HighLoad++ conference in Moscow next weekend, 1-4/10, anyone for an Erlounge or maybe an informal Erlang user group meeting? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From lemenkov@REDACTED Thu Sep 22 17:10:50 2011 From: lemenkov@REDACTED (Peter Lemenkov) Date: Thu, 22 Sep 2011 19:10:50 +0400 Subject: [erlang-questions] Erlounge or informal EUG meeting in Moscow In-Reply-To: References: Message-ID: 2011/9/22 Robert Virding : > I will be at the HighLoad++ conference in Moscow next weekend, 1-4/10, anyone for an Erlounge or maybe an informal Erlang user group meeting? +1 from me. Folks, anyone else? -- With best regards, Peter Lemenkov. From mabrek@REDACTED Thu Sep 22 17:35:06 2011 From: mabrek@REDACTED (mabrek) Date: Thu, 22 Sep 2011 19:35:06 +0400 Subject: [erlang-questions] Erlounge or informal EUG meeting in Moscow In-Reply-To: References: Message-ID: Me too. On Thu, Sep 22, 2011 at 7:10 PM, Peter Lemenkov wrote: > 2011/9/22 Robert Virding : >> I will be at the HighLoad++ conference in Moscow next weekend, 1-4/10, anyone for an Erlounge or maybe an informal Erlang user group meeting? > > +1 from me. > Folks, anyone else? > > -- > With best regards, Peter Lemenkov. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From max.lapshin@REDACTED Thu Sep 22 17:40:29 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 22 Sep 2011 19:40:29 +0400 Subject: [erlang-questions] Erlounge or informal EUG meeting in Moscow In-Reply-To: References: Message-ID: I don't know if my speech is accepted on Highload conference, but I will meet in any case. From maxtqm@REDACTED Thu Sep 22 17:49:01 2011 From: maxtqm@REDACTED (max tan) Date: Thu, 22 Sep 2011 23:49:01 +0800 Subject: [erlang-questions] Is it possible to align binary's byte array to cache line boundary? In-Reply-To: References: Message-ID: > I can maximize the processing throughput. > > I don't know of a dynamic language that gives you this control; be it PHP, > JavaScript, Haskell or Erlang. I just found Haskell has "Data.ByteString.Lazy", according to a paper titled "Rewriting Haskell Strings": A library for ByteStrings is implemented, providing a purely functional interface, which approaches the speed of low-level mutable arrays in C. From moxford@REDACTED Thu Sep 22 20:24:56 2011 From: moxford@REDACTED (Mike Oxford) Date: Thu, 22 Sep 2011 11:24:56 -0700 Subject: [erlang-questions] How erlang handle complicated data structure like C? In-Reply-To: References: Message-ID: You can pattern-match non-empty records against an "empty" record. receive_function(Packet = #my_record{}) -> do_something_with_my_packet(Packet); receive_function(Packet = #my_other_record{}) -> do_something_else_with_my_packet(Packet); receive_function(Packet) -> I_do_not_care. -mox On Tue, Sep 20, 2011 at 6:45 AM, Jesper Louis Andersen wrote: > On Mon, Sep 19, 2011 at 08:26, George Catalin Serbanut > wrote: >> Considering all variables integers, here is an example how to play with >> records: > > I'll move the quotes around a bit... > >> 7. How to use the test: >> receive_function(Packet) when is_my_record(Packet) == true -> >> do_something_with_my_packet; >> receive_function(Packet) when is_my_record(Packet) /= true -> I_do_not_care. > > You can't do this in Erlang. Guard expressions are limited so you can > be sure they terminate. From 8mayday@REDACTED Thu Sep 22 20:38:46 2011 From: 8mayday@REDACTED (Andrey Popp) Date: Thu, 22 Sep 2011 22:38:46 +0400 Subject: [erlang-questions] Erlounge or informal EUG meeting in Moscow In-Reply-To: References: Message-ID: <879CCFAD-C42D-4D7C-8CBA-DEB0875A42ED@gmail.com> On Sep 22, 2011, at 7:08 PM, Robert Virding wrote: > I will be at the HighLoad++ conference in Moscow next weekend, 1-4/10, anyone for an Erlounge or maybe an informal Erlang user group meeting? Definitely yes! From carlsson.richard@REDACTED Thu Sep 22 21:03:56 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 22 Sep 2011 21:03:56 +0200 Subject: [erlang-questions] sendmail library Message-ID: <4E7B869C.8020409@gmail.com> The following is a library we've been using at Klarna for sending mail via sendmail: https://github.com/richcarl/sendmail It's mainly a synthesis of two older libraries (by Klacke and Johan Bevemyr, respectively). It supports choosing content type (option {content_type, String}, or shorthand options 'text' and 'html'), and including a attachment (option {attached, Files}, where Files is a list of Filename or {Filename, ContentType, Data}). Attachments are sent as base64. Documentation is nonexistent, but it's pretty simple and works well enough. Feel free to use it and/or contribute. /Richard From allen.kim@REDACTED Thu Sep 22 21:53:46 2011 From: allen.kim@REDACTED (Allen Kim) Date: Thu, 22 Sep 2011 14:53:46 -0500 Subject: [erlang-questions] erlang gen_tcp:connect/3 not working with rpc:call/4, anyone knows why? In-Reply-To: Message-ID: Thanks Magnus, and that works BUT, it does not make sense to me. errrrr, Mac, why don't you work like others? ~$ssh allen@REDACTED 'nohup erl -name loadtest@REDACTED -detached -setcookie loadtest' ~$ps aux | grep loadtest allen 4137 0.5 0.1 2454316 10104 ?? S 3:49pm 0:00.12 /opt/local/lib/erlang/erts-5.8.4/bin/beam.smp -- -root /opt/local/lib/erlang -progname erl -- -home /Users/allen -- -name loadtest@REDACTED -noshell -noinput -setcookie loadtest allen 3219 0.0 0.0 2435328 1440 s002 S+ 2:46pm 0:00.17 ssh azoogle@REDACTED allen 4139 0.0 0.0 2435116 524 s005 S+ 3:49pm 0:00.00 grep loadtest ~$erl -name allen@REDACTED -setcookie loadtest Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.4 (abort with ^G) (allen@REDACTED)1> net_adm:ping('loadtest@REDACTED'). pong (allen@REDACTED)2> gen_tcp:connect("www.google.com",80,[]). {ok,#Port<0.633>} (allen@REDACTED)3> rpc:call('loadtest@REDACTED',gen_tcp,connect,["www.google.com",80,[]]). {ok,#Port<5895.487>} (allen@REDACTED)4> Allen Kim From: Magnus Klaar > Date: Thu, 22 Sep 2011 09:53:45 -0500 To: Allen Kim > Cc: Kaiduan Xie >, "erlang-questions@REDACTED" > Subject: Re: [erlang-questions] erlang gen_tcp:connect/3 not working with rpc:call/4, anyone knows why? Hi! If it uses native lookups we can assume that getaddrinfo is broken when it is called from the detached node. This narrows down the search for the cause of this issue to a more general problem with getaddrinfo on os-x, leading us to this thread: http://lists.apple.com/archives/unix-porting/2010/Jul/msg00001.html You could try just adding nohup to the command that you're executing using ssh. Although i have a hard time seeing how this would work. An erlang node started with -detached should ignore HUP signals (afaik). MVH Magnus On Wed, Sep 21, 2011 at 8:48 PM, Allen Kim > wrote: I don't see any difference from outputs from the following commands. It both uses native lookup. Anyway, it is not a big deal since I do not use Mac for staging nor production. it's only for development. However, I hope it's better to be consistent on all kind of *nix environment. Or, it could be me only. Any Mac user who what to test this? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Sep 22 22:46:24 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 22 Sep 2011 22:46:24 +0200 Subject: [erlang-questions] Is it possible to align binary's byte array to cache line boundary? In-Reply-To: References: Message-ID: On Thu, Sep 22, 2011 at 17:49, max tan wrote: > > I just found Haskell has "Data.ByteString.Lazy", according to a paper titled > "Rewriting Haskell Strings": > > ? A library for ByteStrings is implemented, ? providing a purely functional > interface, which approaches the speed of low-level mutable arrays in C. > _______________________________________________ This is true. In many ways, a Lazy ByteString in Haskell corresponds to a list of binaries in Erlang, which is a subset of the iolist() type in Erlang. If Haskells Lazy Bytestring implementation approaches the speed of low-level mutable arrays, so does binaries in Erlang. My experience is they perform at about the same speed. I don't know what happened in my brain when I called Haskell a dynamic language though. It is statically typed, and very much so. -- J. From jeffm@REDACTED Fri Sep 23 06:35:00 2011 From: jeffm@REDACTED (jm) Date: Fri, 23 Sep 2011 14:35:00 +1000 Subject: [erlang-questions] Rough thought on a P2P package distribution model for Erlang In-Reply-To: References: <4E700AFF.7090309@ghostgun.com> <4E79252C.9070005@ghostgun.com> Message-ID: <4E7C0C74.4080207@ghostgun.com> You assume speed is the only reason to use P2P. I'm more concerned with maintaining up to date package repositories, redundancy, and security. MIrrors that use ftp/http/rsync/etc have a tendency to fall out of date if not well maintained. These mirrors also require that up to date lists be maintained so the users (human or machine) can find active ones and how do you distribute those? CDNs do this through things like redirection and dynamic DNS records. The method that is used behind the scenes to distribute content would vary from provider to provider. P2P is just one possible technology (and that is all I'm claiming). Most likely, these providers use a mix of technologies or a hybrid approach for internal content distribution. For example, http://goanna.cs.rmit.edu.au/~xiaodong/mbc/Theses/jaison-minorThesis.pdf mentions P2P as one technology. and page 340 and page 341 of this paper has tables which list uses of P2P one sub table is devoted to "Content Publishing and Storage Systems" A Survey of Peer-to-Peer Content Distribution Technologies STEPHANOS ANDROUTSELLIS-THEOTOKIS AND DIOMIDIS SPINELLIS Athens University of Economics and Business Jeff. On 21/09/11 10:25 PM, Jesper Louis Andersen wrote: > Mainly because there is no need to do it. > > Data distribution systems like BitTorrent has the specific distinct > advantage that it can transfer data at high bandwidths, even if the > initial source is highly bandwidth constrained in its upstream. > Upstream bandwidth scales with demand in a BT network. Hence, the > applicability of BitTorrent (and like) protocols hinges on a need to > thwart an upstream bandwidth constraint. Example: You are Facebook and > need a 1 gigabyte image distributed quickly to 50.000 nodes from a > single deploy machine. > > But as soon as there is demand for a package distribution, and the > demand is high enough, mirrors form and mirrors have ample amounts of > upstream bandwidth available. Far more than what is needed. Thus, the > additional complexity of adding BitTorrent into the mix isn't needed. > > Add that HTTP transport is well-known and simple. You can say it is a > locally extreme value which is currently good enough. The proof is > Content Delivery Networks (CDNs) which does not use BitTorrent to > distribute content. > > From thomasl_erlang@REDACTED Fri Sep 23 11:11:25 2011 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Fri, 23 Sep 2011 02:11:25 -0700 (PDT) Subject: [erlang-questions] Rough thought on a P2P package distribution model for Erlang In-Reply-To: <4E7C0C74.4080207@ghostgun.com> References: <4E700AFF.7090309@ghostgun.com> <4E79252C.9070005@ghostgun.com> <4E7C0C74.4080207@ghostgun.com> Message-ID: <1316769085.18930.YahooMailNeo@web111406.mail.gq1.yahoo.com> Twitter uses p2p to deploy their services:? http://engineering.twitter.com/2010/07/murder-fast-datacenter-code-deploys.html https://github.com/lg/murder Nice judo, turning a big problem (that myriad of hosts) into a big advantage. Best, Thomas >________________________________ >From: jm >To: Jesper Louis Andersen >Cc: erlang-questions@REDACTED >Sent: Friday, September 23, 2011 6:35 AM >Subject: Re: [erlang-questions] Rough thought on a P2P package distribution model for Erlang > >You assume speed is the only reason to use P2P. I'm more concerned with maintaining up to date package repositories, redundancy, and security. MIrrors that use ftp/http/rsync/etc have a tendency? to fall out of date if not well maintained. These mirrors also require that up to date lists be maintained so the users (human or machine) can find active ones and how do you distribute those? CDNs do this through things like redirection and dynamic DNS records. The method that is used behind the scenes to distribute content would vary from provider to provider. P2P is just one possible technology (and that is all I'm claiming). Most likely, these providers use a mix of technologies or a hybrid approach for internal content distribution. > >For example, >http://goanna.cs.rmit.edu.au/~xiaodong/mbc/Theses/jaison-minorThesis.pdf mentions P2P as one technology. > >and page 340 and page 341 of this paper has tables which list uses of P2P one sub table is devoted to "Content Publishing and Storage Systems" >A Survey of Peer-to-Peer Content Distribution Technologies >STEPHANOS ANDROUTSELLIS-THEOTOKIS AND DIOMIDIS SPINELLIS >Athens University of Economics and Business > >Jeff. > >On 21/09/11 10:25 PM, Jesper Louis Andersen wrote: >> Mainly because there is no need to do it. >> >> Data distribution systems like BitTorrent has the specific distinct >> advantage that it can transfer data at high bandwidths, even if the >> initial source is highly bandwidth constrained in its upstream. >> Upstream bandwidth scales with demand in a BT network. Hence, the >> applicability of BitTorrent (and like) protocols hinges on a need to >> thwart an upstream bandwidth constraint. Example: You are Facebook and >> need a 1 gigabyte image distributed quickly to 50.000 nodes from a >> single deploy machine. >> >> But as soon as there is demand for a package distribution, and the >> demand is high enough, mirrors form and mirrors have ample amounts of >> upstream bandwidth available. Far more than what is needed. Thus, the >> additional complexity of adding BitTorrent into the mix isn't needed. >> >> Add that HTTP transport is well-known and simple. You can say it is a >> locally extreme value which is currently good enough. The proof is >> Content Delivery Networks (CDNs) which does not use BitTorrent to >> distribute content. >> >> > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions > > > From amir.fireflame@REDACTED Fri Sep 23 11:23:46 2011 From: amir.fireflame@REDACTED (Amir Almasi) Date: Fri, 23 Sep 2011 11:23:46 +0200 Subject: [erlang-questions] Jinterface Message-ID: Hi all, I am currently working on GUI by using Jinterface. I am try to send a message from an alive node made by Jinterface library in java to the alive normal node on shell. In this way it works and another module gets the message: OtpErlangAtom ok = new OtpErlangAtom("ok"); mbox.send("clientChat", "chat_c@" + computername, ok); The problem is, I need to pass the node() by message for the reply. Therefore, I call the function in this way, and there will be error notification!, *OtpNode myNode = new OtpNode("bar");* *mbox.send("clientChat", "chat_c@" + computername, myNode );* or *OtpNode myNode = new OtpNode("bar");* *mbox.send("clientChat", "chat_c@" + computername, myNode.node() );* Is there any hint or solution?? Thanks, -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Fri Sep 23 11:40:18 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 23 Sep 2011 11:40:18 +0200 Subject: [erlang-questions] erlang gen_tcp:connect/3 not working with rpc:call/4, anyone knows why? In-Reply-To: References: Message-ID: <20110923094018.GA31738@erix.ericsson.se> On Wed, Sep 21, 2011 at 01:48:57PM -0500, Allen Kim wrote: > I don't see any difference from outputs from the following commands. > It both uses native lookup. > Anyway, it is not a big deal since I do not use Mac for staging nor production. it's only for development. Debugging hints: run without the -detached flag and debug the native resolver's problems: $ ssh allen@REDACTED $ erl -name loadtest@REDACTED Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:8:8] [rq:8] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.4 (abort with ^G) (loadtest@REDACTED)1> inet_gethost_native:control({debug_level,9}). inet_gethost[3930] (DEBUG):debug_level = 9 ok (loadtest@REDACTED)2> inet:gethostbyname("www.google.com"). inet_gethost[3930] (DEBUG):Saved domainname .google.com. inet_gethost[3930] (DEBUG):Created worker[3948] with fd 3 inet_gethost[3930] (DEBUG):Saved domainname .google.com. inet_gethost[3948] (DEBUG):Worker got request, op = 1, proto = 1, data = www.google.com. inet_gethost[3948] (DEBUG):Starting gethostbyname(www.google.com) inet_gethost[3948] (DEBUG):gethostbyname OK {ok,{hostent,"www.l.google.com", ["www.google.com"], inet,4, [{173,194,32,52}, {173,194,32,48}, {173,194,32,49}, {173,194,32,50}, {173,194,32,51}]}} (loadtest@REDACTED)3> > > However, I hope it's better to be consistent on all kind of *nix environment. > Or, it could be me only. > > Any Mac user who what to test this? > > $ssh allen@REDACTED 'erl -name loadtest@REDACTED -detached -setcookie loadtest' > $erl -name allen@REDACTED -setcookie loadtest > > net_adm:ping('loadtest@REDACTED'). > > gen_tcp:connect("www.google.com",80,[]). > >rpc:call('loadtest@REDACTED',gen_tcp,connect,["www.google.com",80,[]]). > > > (allen@REDACTED)6> io:format("~p~n",[ets:tab2list(inet_db)]). > [{res_resolv_conf_info,undefined}, > {res_hosts_file_info,undefined}, > {res_domain,"local"}, > {res_hosts_file_tm,0}, > {res_resolv_conf_tm,0}, > {cache_refresh_interval,3600000}, > {socks5_noproxy,[]}, > {res_retry,3}, > {res_search,["local"]}, > {res_resolv_conf,"/etc/resolv.conf"}, > {res_inet6,false}, > {res_recurse,true}, > {hostname,"allen"}, > {res_udp_payload_size,1280}, > {cache_size,100}, > {sctp_module,inet_sctp}, > {udp_module,inet_udp}, > {tcp_module,inet_tcp}, > {res_hosts_file,"/etc/hosts"}, > {res_id,0}, > {socks5_port,1080}, > {res_timeout,2000}, > {res_edns,false}, > {res_alt_ns,[]}, > {socks5_methods,[none]}, > {res_ns,[{{192,168,14,10},53},{{192,168,24,10},53}]}, > {res_usevc,false}, > {socks5_server,[]}, > {res_lookup,[native]}] > ok > (allen@REDACTED)7> io:format("~p~n",[rpc:call('loadtest@REDACTED', ets,tab2list,[inet_db])]). > [{res_resolv_conf_info,undefined}, > {res_hosts_file_info,undefined}, > {res_domain,"local"}, > {res_hosts_file_tm,0}, > {res_resolv_conf_tm,0}, > {cache_refresh_interval,3600000}, > {socks5_noproxy,[]}, > {res_retry,3}, > {res_search,["local"]}, > {res_resolv_conf,"/etc/resolv.conf"}, > {res_inet6,false}, > {res_recurse,true}, > {hostname,"allen"}, > {res_udp_payload_size,1280}, > {cache_size,100}, > {sctp_module,inet_sctp}, > {udp_module,inet_udp}, > {tcp_module,inet_tcp}, > {res_hosts_file,"/etc/hosts"}, > {res_id,0}, > {socks5_port,1080}, > {res_timeout,2000}, > {res_edns,false}, > {res_alt_ns,[]}, > {socks5_methods,[none]}, > {res_ns,[{{192,168,14,10},53},{{192,168,24,10},53}]}, > {res_usevc,false}, > {socks5_server,[]}, > {res_lookup,[native]}] > ok > > > > From: Magnus Klaar > > Date: Wed, 21 Sep 2011 13:18:51 -0500 > To: Allen Kim > > Cc: Kaiduan Xie >, "erlang-questions@REDACTED" > > Subject: Re: [erlang-questions] erlang gen_tcp:connect/3 not working with rpc:call/4, anyone knows why? > > Hi! > > The Erlang VM provides multiple strategies for name lookups, among them is a native lookup mechamism using getaddrinfo and > an erlang client implemented by the inet_res module. > > The contents of the inet_db table on the detached node may also be worth looking into, does it differ? which strategy does it use?. You can access it using: > > rpc:call('loadtest@REDACTED', ets,tab2list,[inet_db]). > > You may also want to look into the mDNSResponder daemon that OSX uses as a system-wide resolver. this daemon appears to > be an alternative to the nscd deamon found on GNU/Linux systems. Applications are expected to perform lookups against this daemon. > > MVH Magnus > > On Wed, Sep 21, 2011 at 7:03 PM, Allen Kim > wrote: > It IS Mac OS X issue, definitely. > I ran it from CentOS box, and it works fine. > Don't know why it does not work on Mac OS X though. > > > On 11-09-21 12:35 PM, "Allen Kim" > wrote: > > >$cat /etc/resolv.conf > ># > ># Mac OS X Notice > ># > ># This file is not used by the host name and address resolution > ># or the DNS query routing mechanisms used by most processes on > ># this Mac OS X system. > ># > ># This file is automatically generated. > ># > >nameserver 192.168.14.10nameserver 192.168.24.10 > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From vladdu55@REDACTED Fri Sep 23 11:44:31 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 23 Sep 2011 11:44:31 +0200 Subject: [erlang-questions] Jinterface In-Reply-To: References: Message-ID: Hi, On Fri, Sep 23, 2011 at 11:23, Amir Almasi wrote: > I am currently working on GUI by using Jinterface. I am try to send a > message from an alive node made by Jinterface library in java to the alive > normal node on shell. > In this way it works and another module gets the message: > OtpErlangAtom ok = new OtpErlangAtom("ok"); > mbox.send("clientChat", "chat_c@" + computername, ok); > The problem is,?I need to pass the node() by message for the reply. > Therefore, I call the function in this way, and there will be error > notification!, > OtpNode myNode = new OtpNode("bar"); > mbox.send("clientChat", "chat_c@" + computername, myNode ); > or > OtpNode myNode = new OtpNode("bar"); > mbox.send("clientChat", "chat_c@" + computername, myNode.node() ); > Is there any hint or solution?? You don't have to send the node, but the pid of a mailbox to receive the answer (could be any mailbox). For example, mbox.send("clientChat", "chat_c@" + computername, mbox.self() ); regards, Vlad From lambdadmitry@REDACTED Fri Sep 23 14:09:12 2011 From: lambdadmitry@REDACTED (Dmitry Groshev) Date: Fri, 23 Sep 2011 05:09:12 -0700 (PDT) Subject: [erlang-questions] Erlounge or informal EUG meeting in Moscow In-Reply-To: References: Message-ID: <18192417.378.1316779752839.JavaMail.geo-discussion-forums@yqih9> Wonderful news! I will definitely be there if meeting will take place during weekend (1-2/10). -------------- next part -------------- An HTML attachment was scrubbed... URL: From fernando.benavides@REDACTED Fri Sep 23 17:34:15 2011 From: fernando.benavides@REDACTED (Fernando "Brujo" Benavides) Date: Fri, 23 Sep 2011 12:34:15 -0300 Subject: [erlang-questions] Dialyzer warning Message-ID: Hi everybody, I have a question that's mainly for Kostis, but maybe there're other out there that had faced similar problems before. I run dialyzer against the binaries of this project as it's shown in its Makefile and I get: > match_stream_user_mgr.erl:32: The specification for match_stream_user_mgr:init/1 states that the function might also return {'ok',{{'simple_one_for_one',100,1},[{'match_stream_user',{'match_stream_user','start_link',[]},'transient','brutal_kill','worker',['match_stream_user']}]}} but the inferred return is none() > match_stream_user_mgr.erl:33: Function init/1 has no local return You can check match_stream_user_mgr:init/1 code... it's fairly simple: > -spec init([]) -> {ok, {{simple_one_for_one, 100, 1}, [{match_stream_user, {match_stream_user, start_link, []}, transient, brutal_kill, worker, [match_stream_user]}]}}. > init([]) -> > {ok, {{simple_one_for_one, 100, 1}, > [{match_stream_user, {match_stream_user, start_link, []}, > transient, brutal_kill, worker, [match_stream_user]}]}}. Any clues? Thanks ____________________________ Fernando "Brujo" Benavides fernando.benavides@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Fri Sep 23 18:17:07 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 23 Sep 2011 19:17:07 +0300 Subject: [erlang-questions] Dialyzer warning In-Reply-To: References: Message-ID: <4E7CB103.7050007@it.uu.se> On 09/23/11 18:34, Fernando "Brujo" Benavides wrote: > Hi everybody, > > I have a question that's mainly for Kostis, but maybe there're other out > there that had faced similar problems before. > I run dialyzer against the binaries of this project > as it's shown in its Makefile > and I get: > >> match_stream_user_mgr.erl:32: The specification for >> match_stream_user_mgr:init/1 states that the function might also >> return >> {'ok',{{'simple_one_for_one',100,1},[{'match_stream_user',{'match_stream_user','start_link',[]},'transient','brutal_kill','worker',['match_stream_user']}]}} >> but the inferred return is none() >> match_stream_user_mgr.erl:33: Function init/1 has no local return > > You can check match_stream_user_mgr:init/1 > > code... it's fairly simple: > > >> -spec init([]) -> {ok, {{simple_one_for_one, 100, 1}, >> [{match_stream_user, {match_stream_user, start_link, []}, transient, >> brutal_kill, worker, [match_stream_user]}]}}. >> init([]) -> >> {ok, {{simple_one_for_one, 100, 1}, >> [{match_stream_user, {match_stream_user, start_link, []}, >> transient, brutal_kill, worker, [match_stream_user]}]}}. > > Any clues? Fernando, In the code of your file you are including file "match_stream.hrl" (unnecessarily? because it is not used anywhere) which has the following record declaration: -record(match_stream_user, {user_id :: match_stream:user_id(), visit_count = 1 :: pos_integer()}). This tells dialyzer that all 3-tuples tagged with the atom 'match_stream_user' are intended to have a match_stream:user_id() in the second element and a pos_integer() in the third element of the tuple. Still, in the file you are constructing the following 3-tuple: {match_stream_user, start_link, []} which violates the record definition (on the 3rd element). You cannot have both. You have three options: 1. Take out the include declaration 2. Use a different tag for the tuple that you construct in the file 3. Add the [] in the type of the visit_count field The first option is the easiest; the second is most probably the thing you want to do here. Kostis From erlang@REDACTED Fri Sep 23 18:19:36 2011 From: erlang@REDACTED (Oleg Chernyh) Date: Fri, 23 Sep 2011 20:19:36 +0400 Subject: [erlang-questions] Full text search in KV-storage. Message-ID: <20110923201936.29551qw15lcu7qu0@mail.udaff.com> I'm writing a forum engine that can sustain high amount of users making queries at the same time. I'm using a KV storage that stores erlang entities which is the way to kill overheads and make data accessible really fast (if we know the key, ofc). The problem I'm facing is full-text search, as you might guess. I don't really want to use any external indexers or databases because I want that piece of software to be scalable node-wise and don't want to add any more data storage entites to the system to prevent data duplication and to ensure consistency. So what I want to do is to "invent bicycle" by implementing a full text search on a key-value storage. Here I'll outline my plan for writing that feature and I'd be happy to hear criticism. A simplistic way to think of a full text search engine is a strict text search engine, when the result of a search is a list of messages (posts) that contain full search keywords (if we search for "abc" only "abc" words are matched, so that "abcd" won't match). In order to accomplish that I can think of the following things to do: 1) We make a set of all words W 2) For each w \in W we have an "index" I_w which returns a list of all posts that contain w When we add a pure full text search functionality (indeed, we might want "geology" to match "geologist") we do the following: 1) make a set of normalized words W_n 2) make a set of common prefixes P 3) make a set of common suffixes S 4) define a normalization function f that strips common suffixes and prefixes from a word 5) define a matching function m that maps f(w) to a list of words from the W_n set 5.1) 6) for each w in W_n we keep an "index" W_n that returns all the messages M for which \exists x \in M : m(f(x)) = w 7) for each word m in new message From fernando.benavides@REDACTED Fri Sep 23 18:23:11 2011 From: fernando.benavides@REDACTED (Fernando "Brujo" Benavides) Date: Fri, 23 Sep 2011 13:23:11 -0300 Subject: [erlang-questions] Dialyzer warning In-Reply-To: <4E7CB103.7050007@it.uu.se> References: <4E7CB103.7050007@it.uu.se> Message-ID: Oh! That was a tricky one! Thanks Kostis :) On 23/09/2011, at 13:17, Kostis Sagonas wrote: > On 09/23/11 18:34, Fernando "Brujo" Benavides wrote: >> Hi everybody, >> >> I have a question that's mainly for Kostis, but maybe there're other out >> there that had faced similar problems before. >> I run dialyzer against the binaries of this project >> as it's shown in its Makefile >> and I get: >> >>> match_stream_user_mgr.erl:32: The specification for >>> match_stream_user_mgr:init/1 states that the function might also >>> return >>> {'ok',{{'simple_one_for_one',100,1},[{'match_stream_user',{'match_stream_user','start_link',[]},'transient','brutal_kill','worker',['match_stream_user']}]}} >>> but the inferred return is none() >>> match_stream_user_mgr.erl:33: Function init/1 has no local return >> >> You can check match_stream_user_mgr:init/1 >> >> code... it's fairly simple: >> >> >>> -spec init([]) -> {ok, {{simple_one_for_one, 100, 1}, >>> [{match_stream_user, {match_stream_user, start_link, []}, transient, >>> brutal_kill, worker, [match_stream_user]}]}}. >>> init([]) -> >>> {ok, {{simple_one_for_one, 100, 1}, >>> [{match_stream_user, {match_stream_user, start_link, []}, >>> transient, brutal_kill, worker, [match_stream_user]}]}}. >> >> Any clues? > > Fernando, > > In the code of your file you are including file "match_stream.hrl" (unnecessarily? because it is not used anywhere) which has the following record declaration: > > -record(match_stream_user, {user_id :: match_stream:user_id(), > visit_count = 1 :: pos_integer()}). > > This tells dialyzer that all 3-tuples tagged with the atom 'match_stream_user' are intended to have a match_stream:user_id() in the second element and a pos_integer() in the third element of the tuple. > > Still, in the file you are constructing the following 3-tuple: > > {match_stream_user, start_link, []} > > which violates the record definition (on the 3rd element). > > You cannot have both. You have three options: > > 1. Take out the include declaration > 2. Use a different tag for the tuple that you construct in the file > 3. Add the [] in the type of the visit_count field > > The first option is the easiest; the second is most probably the thing you want to do here. > > Kostis > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jesper.louis.andersen@REDACTED Fri Sep 23 18:58:30 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 23 Sep 2011 18:58:30 +0200 Subject: [erlang-questions] Full text search in KV-storage. In-Reply-To: <20110923201936.29551qw15lcu7qu0@mail.udaff.com> References: <20110923201936.29551qw15lcu7qu0@mail.udaff.com> Message-ID: On Fri, Sep 23, 2011 at 18:19, Oleg Chernyh wrote: > 1) We make a set of all words W Two structures immediately suggest themselves: * Tries. That is store a digital tree on the word. "CAT" will search in a 26-way tree through the C-edge, the A-edge and the T-edge. If you need a base 62 or a base 36 match, it may be beneficial to compress nodes that are not totally full. An evil way is to use an ets table with the compressed flag set and call ets:lookup_element/3 on the beast. This is how a spelling checker often stores its dictionary. * Bloom filter based structures. Look at how postgresql does it with GiST indexes for instance: http://www.postgresql.org/docs/8.3/static/textsearch-indexes.html For each w \in document, you hash word, h(w) = n, and set the n'th bit in a bit string. Then you or all hashes together \/_{w \in D} h(w). This gives an approximate match, but hopefully it weeds out a lot of things you need to search and the amount of false positives is probably small. These bit strings can be stored as e.g., crit-bit trees (also called radix or patricia trees - another trie-structure, but this time binary). See for instance, http://cr.yp.to/critbit.html by Dan Bernstein. I did a crit-bit tree structure in Erlang some months ago, https://github.com/jlouis/bench-map/blob/master/src/patricia.erl and also one based on Hash-array Mapped Tries (HAMT, the basic data structure in Clojures dictionaries), https://github.com/jlouis/bench-map/blob/master/src/hamt.erl .. While they do not outperform dict, gb_trees and Robert Virdings RB trees for the workloads I've tried, the performance isn't too bad and they will probably pack much better for this kind of thing. Do note however that this is quite specialized and you can probably win some raw speed by using a pre-baked solution. My suggestions should take you up in speed by quite a lot, however. -- J. From erlang@REDACTED Fri Sep 23 19:42:43 2011 From: erlang@REDACTED (Oleg Chernyh) Date: Fri, 23 Sep 2011 21:42:43 +0400 Subject: [erlang-questions] Full text search in KV-storage. Message-ID: <20110923214243.17682w4tvjeovjmb@mail.udaff.com> I'm writing a forum engine that can sustain high amount of users making queries at the same time. I'm using a KV storage that stores erlang entities which is the way to kill overheads and make data accessible really fast (if we know the key, ofc). The problem I'm facing is full-text search, as you might guess. I don't really want to use any external indexers or databases because I want that piece of software to be scalable node-wise and don't want to add any more data storage entites to the system to prevent data duplication and to ensure consistency. So what I want to do is to "invent bicycle" by implementing a full text search on a key-value storage. Here I'll outline my plan for writing that feature and I'd be happy to hear criticism. A simplistic way to think of a full text search engine is a strict text search engine, when the result of a search is a list of messages (posts) that contain full search keywords (if we search for "abc" only "abc" words are matched, so that "abcd" won't match). In order to accomplish that I can think of the following things to do: 1) We make a set of all words W 2) For each w \in W we have an "index" I_w which returns a list of all posts that contain w When we add a pure full text search functionality (indeed, we might want "geology" to match "geologist") we do the following: 1) make a set of normalized words W_n 2) make a set of common prefixes P 3) make a set of common suffixes S 4) define a normalization function f that strips common suffixes and prefixes from a word 5) define a matching function m that maps f(w) to a list of words from the W_n set 5.1) if there is no adequate entry for f(w), m function adds f(w) to W_n 6) for each w in W_n we keep an "index" W_n that returns all the messages M for which \exists x \in M : m(f(x)) = w 7) for each word x in new message we asynchronously run m(f(x)) and update indexes appropriately 8) from time to time we re-index our W_n maps merging indexes for similar words What are the pitfalls of that approach? Any feedback will be appreciated. - Oleg From rzezeski@REDACTED Sat Sep 24 00:27:55 2011 From: rzezeski@REDACTED (Ryan Zezeski) Date: Fri, 23 Sep 2011 18:27:55 -0400 Subject: [erlang-questions] Full text search in KV-storage. In-Reply-To: <20110923201936.29551qw15lcu7qu0@mail.udaff.com> References: <20110923201936.29551qw15lcu7qu0@mail.udaff.com> Message-ID: Oleg, I know you said you don't want to use an external solution but perhaps Riak [1] is an exception given it's design to scale out and the fact that it's written in Erlang? It has full-text search capability [2]. Might be worth a look. -Ryan [1]: http://wiki.basho.com/Riak.html [2]: http://wiki.basho.com/Riak-Search.html On Fri, Sep 23, 2011 at 12:19 PM, Oleg Chernyh wrote: > I'm writing a forum engine that can sustain high amount of users making > queries at the same time. > I'm using a KV storage that stores erlang entities which is the way to kill > overheads and make data accessible really fast (if we know the key, ofc). > The problem I'm facing is full-text search, as you might guess. > I don't really want to use any external indexers or databases because I > want that piece of software to be scalable node-wise and don't want to add > any more data storage entites to the system to prevent data duplication and > to ensure consistency. > So what I want to do is to "invent bicycle" by implementing a full text > search on a key-value storage. > > Here I'll outline my plan for writing that feature and I'd be happy to hear > criticism. > > A simplistic way to think of a full text search engine is a strict text > search engine, when the result of a search is a list of messages (posts) > that contain full search keywords (if we search for "abc" only "abc" words > are matched, so that "abcd" won't match). > In order to accomplish that I can think of the following things to do: > 1) We make a set of all words W > 2) For each w \in W we have an "index" I_w which returns a list of all > posts that contain w > > When we add a pure full text search functionality (indeed, we might want > "geology" to match "geologist") we do the following: > 1) make a set of normalized words W_n > 2) make a set of common prefixes P > 3) make a set of common suffixes S > 4) define a normalization function f that strips common suffixes and > prefixes from a word > 5) define a matching function m that maps f(w) to a list of words from the > W_n set > 5.1) > 6) for each w in W_n we keep an "index" W_n that returns all the messages M > for which \exists x \in M : m(f(x)) = w > 7) for each word m in new message > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Sat Sep 24 01:11:41 2011 From: bob@REDACTED (Bob Ippolito) Date: Fri, 23 Sep 2011 16:11:41 -0700 Subject: [erlang-questions] Full text search in KV-storage. In-Reply-To: References: <20110923201936.29551qw15lcu7qu0@mail.udaff.com> Message-ID: And if you really wanted to do everything the hard way, you could probably manage to re-use some of the components that are in Riak. Would be far simpler to just use Riak as-is though. On Fri, Sep 23, 2011 at 3:27 PM, Ryan Zezeski wrote: > Oleg, > I know you said you don't want to use an external solution but perhaps Riak > [1] is an exception given it's design to scale out and the fact that it's > written in Erlang? ?It has full-text search capability [2]. ?Might be worth > a look. > -Ryan > > [1]:?http://wiki.basho.com/Riak.html > [2]:?http://wiki.basho.com/Riak-Search.html > On Fri, Sep 23, 2011 at 12:19 PM, Oleg Chernyh wrote: >> >> I'm writing a forum engine that can sustain high amount of users making >> queries at the same time. >> I'm using a KV storage that stores erlang entities which is the way to >> kill overheads and make data accessible really fast (if we know the key, >> ofc). >> The problem I'm facing is full-text search, as you might guess. >> I don't really want to use any external indexers or databases because I >> want that piece of software to be scalable node-wise and don't want to add >> any more data storage entites to the system to prevent data duplication and >> to ensure consistency. >> So what I want to do is to "invent bicycle" by implementing a full text >> search on a key-value storage. >> >> Here I'll outline my plan for writing that feature and I'd be happy to >> hear criticism. >> >> A simplistic way to think of a full text search engine is a strict text >> search engine, when the result of a search is a list of messages (posts) >> that contain full search keywords (if we search for "abc" only "abc" words >> are matched, so that "abcd" won't match). >> In order to accomplish that I can think of the following things to do: >> 1) We make a set of all words W >> 2) For each w \in W we have an "index" I_w which returns a list of all >> posts that contain w >> >> When we add a pure full text search functionality (indeed, we might want >> "geology" to match "geologist") we do the following: >> 1) make a set of normalized words W_n >> 2) make a set of common prefixes P >> 3) make a set of common suffixes S >> 4) define a normalization function f that strips common suffixes and >> prefixes from a word >> 5) define a matching function m that maps f(w) to a list of words from the >> W_n set >> 5.1) >> 6) for each w in W_n we keep an "index" W_n that returns all the messages >> M for which \exists x \in M : m(f(x)) = w >> 7) for each word m in new message >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From erlang@REDACTED Sat Sep 24 01:40:04 2011 From: erlang@REDACTED (=?utf-8?B?T2xlZyBDaGVybnlo?=) Date: Sat, 24 Sep 2011 02:40:04 +0300 Subject: [erlang-questions] =?utf-8?q?Full_text_search_in_KV-storage=2E?= Message-ID: <20110923234045.0F676B11FD9@mx2.eurocom.su> I actually had a look at Riak, I might even give it a try at some other project of mine. Thank you for your input anyway. ----- Reply message ----- From: "Ryan Zezeski" Date: Sat, Sep 24, 2011 1:27 am Subject: [erlang-questions] Full text search in KV-storage. To: "Oleg Chernyh" Cc: Oleg, I know you said you don't want to use an external solution but perhaps Riak [1] is an exception given it's design to scale out and the fact that it's written in Erlang? It has full-text search capability [2]. Might be worth a look. -Ryan [1]: http://wiki.basho.com/Riak.html [2]: http://wiki.basho.com/Riak-Search.html On Fri, Sep 23, 2011 at 12:19 PM, Oleg Chernyh wrote: > I'm writing a forum engine that can sustain high amount of users making > queries at the same time. > I'm using a KV storage that stores erlang entities which is the way to kill > overheads and make data accessible really fast (if we know the key, ofc). > The problem I'm facing is full-text search, as you might guess. > I don't really want to use any external indexers or databases because I > want that piece of software to be scalable node-wise and don't want to add > any more data storage entites to the system to prevent data duplication and > to ensure consistency. > So what I want to do is to "invent bicycle" by implementing a full text > search on a key-value storage. > > Here I'll outline my plan for writing that feature and I'd be happy to hear > criticism. > > A simplistic way to think of a full text search engine is a strict text > search engine, when the result of a search is a list of messages (posts) > that contain full search keywords (if we search for "abc" only "abc" words > are matched, so that "abcd" won't match). > In order to accomplish that I can think of the following things to do: > 1) We make a set of all words W > 2) For each w \in W we have an "index" I_w which returns a list of all > posts that contain w > > When we add a pure full text search functionality (indeed, we might want > "geology" to match "geologist") we do the following: > 1) make a set of normalized words W_n > 2) make a set of common prefixes P > 3) make a set of common suffixes S > 4) define a normalization function f that strips common suffixes and > prefixes from a word > 5) define a matching function m that maps f(w) to a list of words from the > W_n set > 5.1) > 6) for each w in W_n we keep an "index" W_n that returns all the messages M > for which \exists x \in M : m(f(x)) = w > 7) for each word m in new message > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Sat Sep 24 01:58:19 2011 From: erlang@REDACTED (=?utf-8?B?T2xlZyBDaGVybnlo?=) Date: Sat, 24 Sep 2011 02:58:19 +0300 Subject: [erlang-questions] =?utf-8?q?Full_text_search_in_KV-storage=2E?= Message-ID: <20110923235859.08552B9220E@mx2.eurocom.su> Jesper, thank you for the insights. I'm far from linguistics and full text search engines, do your input was particulary useful. And what about my idea that I have briefly described? I'm sure that it has many flaws such as m-lookup and re-indexing problem, but that's how I store non-full text search indexes at the moment such as tag searches and searches by author. So my mind was spinning around the idea of homogeneous O(1) indexes. To those who suggest using 3rd party storages - too much of the current logic relies on a "native" storage I currently use so I'd be glad to stick with it. ----- Reply message ----- From: "Jesper Louis Andersen" Date: Fri, Sep 23, 2011 7:58 pm Subject: [erlang-questions] Full text search in KV-storage. To: "Oleg Chernyh" Cc: On Fri, Sep 23, 2011 at 18:19, Oleg Chernyh wrote: > 1) We make a set of all words W Two structures immediately suggest themselves: * Tries. That is store a digital tree on the word. "CAT" will search in a 26-way tree through the C-edge, the A-edge and the T-edge. If you need a base 62 or a base 36 match, it may be beneficial to compress nodes that are not totally full. An evil way is to use an ets table with the compressed flag set and call ets:lookup_element/3 on the beast. This is how a spelling checker often stores its dictionary. * Bloom filter based structures. Look at how postgresql does it with GiST indexes for instance: http://www.postgresql.org/docs/8.3/static/textsearch-indexes.html For each w \in document, you hash word, h(w) = n, and set the n'th bit in a bit string. Then you or all hashes together \/_{w \in D} h(w). This gives an approximate match, but hopefully it weeds out a lot of things you need to search and the amount of false positives is probably small. These bit strings can be stored as e.g., crit-bit trees (also called radix or patricia trees - another trie-structure, but this time binary). See for instance, http://cr.yp.to/critbit.html by Dan Bernstein. I did a crit-bit tree structure in Erlang some months ago, https://github.com/jlouis/bench-map/blob/master/src/patricia.erl and also one based on Hash-array Mapped Tries (HAMT, the basic data structure in Clojures dictionaries), https://github.com/jlouis/bench-map/blob/master/src/hamt.erl .. While they do not outperform dict, gb_trees and Robert Virdings RB trees for the workloads I've tried, the performance isn't too bad and they will probably pack much better for this kind of thing. Do note however that this is quite specialized and you can probably win some raw speed by using a pre-baked solution. My suggestions should take you up in speed by quite a lot, however. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sat Sep 24 17:27:14 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 24 Sep 2011 17:27:14 +0200 Subject: [erlang-questions] Full text search in KV-storage. In-Reply-To: <20110923235859.08552B9220E@mx2.eurocom.su> References: <20110923235859.08552B9220E@mx2.eurocom.su> Message-ID: On Sat, Sep 24, 2011 at 01:58, Oleg Chernyh wrote: > I'm far from linguistics and full text search engines, do your input was > particulary useful. > And what about my idea that I have briefly described? I tried to be adept and avoid answering those parts because I know very little about it. It looks like you do some stemming of words to find their stem and then you index those. But I don't know what you do to achieve it. If you pick a language, like english there are probably two ways to go: 1. Form a set of rules and apply those rules to "normalize"/"canonicalize"/"extract-the-stem". 2. Play the google game: If you have enough data to mine statistically, figure out what the stems are via machine learning. Historically, judging by papers at AI/ML conferences, it looks like option 2 wins :P -- J. From bighostkim@REDACTED Sat Sep 24 19:02:07 2011 From: bighostkim@REDACTED (Allen Kim) Date: Sat, 24 Sep 2011 13:02:07 -0400 Subject: [erlang-questions] How to see debug messages of Message-ID: Hi I m having some issues of having 1% of httpc:request/4 ends up with {error, socket_closed_remotely}, 99 % are good with 200. I want to debug what's happening inside and I saw there are debug message already in code. What should I do to see those ?hcrd/2 messages? -module(httpc_handler). -behaviour(gen_server). ... connect_and_send_first_request(Address, #request{settings = Settings, headers = Headers, address = OrigAddress, scheme = Scheme} = Request, #state{options = Options} = State) -> * ?hcrd("connect",* * [{address, Address}, {request, Request}, {options, Options}]),* ..... Allen Kim -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Sat Sep 24 19:35:59 2011 From: jwatte@REDACTED (Jon Watte) Date: Sat, 24 Sep 2011 10:35:59 -0700 Subject: [erlang-questions] Is it possible to align binary's byte array to cache line boundary? In-Reply-To: References: Message-ID: Maybe LazyByteStrings and Erlang binaries approach the performance of randomly aligned heap blocks in C. But, if I do data streaming processing (say, transforming vertices for 3D graphcs, transcoding media in multimedia, detecting intersections in rigid body dynamics, etc) then the specific alignment and stride of my data matters. Neither Erlang nor Haskell allows me to specify that "the first byte of this byte string is always aligned on a multiple of 16 bytes" (for architecture-specific SIMD exploitation) or "the first byte of this byte string is always aligned on a multiple of 64 bytes" (for architecture-specific cache line exploitation) or "no word within this binary will share a L3 cache line with any other word allocated in the same manner" (for architecture-specific cache contention reduction for spinlocks). C allows you to control these things. That's why it's a systems programming language, and Erlang isn't (although it's a language for programming systems :-) Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Thu, Sep 22, 2011 at 1:46 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Thu, Sep 22, 2011 at 17:49, max tan wrote: > > > > > I just found Haskell has "Data.ByteString.Lazy", according to a paper > titled > > "Rewriting Haskell Strings": > > > > A library for ByteStrings is implemented, providing a purely > functional > > interface, which approaches the speed of low-level mutable arrays in C. > > _______________________________________________ > > This is true. In many ways, a Lazy ByteString in Haskell corresponds > to a list of binaries in Erlang, which is a subset of the iolist() > type in Erlang. If Haskells Lazy Bytestring implementation approaches > the speed of low-level mutable arrays, so does binaries in Erlang. > > My experience is they perform at about the same speed. I don't know > what happened in my brain when I called Haskell a dynamic language > though. It is statically typed, and very much so. > > > -- > J. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Sat Sep 24 19:41:00 2011 From: jwatte@REDACTED (Jon Watte) Date: Sat, 24 Sep 2011 10:41:00 -0700 Subject: [erlang-questions] Rough thought on a P2P package distribution model for Erlang In-Reply-To: <4E79252C.9070005@ghostgun.com> References: <4E700AFF.7090309@ghostgun.com> <4E79252C.9070005@ghostgun.com> Message-ID: > > why aren't we making better use of these technologies for things like this? Because, for most applications, it's not a solution that fits the problems people actually have. If you want high bandwidth, reliability, security and distribution, you sign up with Akamai, or you put your stuff in an S3 bucket, or you stick a Riak store in your datacenter, and 99% of the time, this solves the problem with a lot less effort and complication than a p2p system. Also, the difference between doing this only within a zone of trust (distribution of software to servers you control), and doing it on an insecure network (end-user peer-to-peer systems) is pretty significant! rsync trees for pushing updates to a large cluster of hosts are pretty easy to set up, if you have SSH keys for all the target hosts, and the hosts can just assume that the data they get is not tainted. (we do this to deploy web code to a cluster of many hundreds of servers, for example) In general, it's not how "smart" something is that matters to adoption, but how well it solves particular problems that many users have at some particular point in time. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Tue, Sep 20, 2011 at 4:43 PM, jm wrote: > I'm not saying it is any different. I'm just outlining a distribution > system for packages which is an alternative to the more tradition tiered ftp > or http site mirrors. Even in the sense I'm outlining it's not that novel. > One linux distribution, rubyx, used a similar model. Their package > distribution program was called "white water". The project appears to be > dead as the website seems to be unresponsive. > > As you point out these systems "have been around for a very long time" so > why aren't we making better use of these technologies for things like this? > > Jeff. > > > On 17/09/11 4:51 AM, Jon Watte wrote: > >> How is this different from the already-solved problem of peer-to-peer >> authenticated file distribution? >> >> Tracker-based systems like bittorrent, and fully peer-to-peer systems like >> freenet have been around for a very long time, and solve all of those >> problems, with different trade-offs for performance, security, >> susceptibility, etc. >> >> Sincerely, >> >> jw >> >> > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jwatte@REDACTED Sat Sep 24 19:45:09 2011 From: jwatte@REDACTED (Jon Watte) Date: Sat, 24 Sep 2011 10:45:09 -0700 Subject: [erlang-questions] Full text search in KV-storage. In-Reply-To: <20110923214243.17682w4tvjeovjmb@mail.udaff.com> References: <20110923214243.17682w4tvjeovjmb@mail.udaff.com> Message-ID: I implemented this on Redis with node.js last year. It was pretty easy, and worked OK, but it wasn't Amazon Product Search by any means. The mapping from word -> document is going to be pretty big in RAM, because the value for each key needs to be your unique identifier for each document -- I imagine you don't want to die when you reach 4 billion posts. (Actually, on Erlang, I think the limit for small ints is 28 bits, or about 256 million) Most sites that operate these things for real, though, just use some off-the-shelf index, like Apache Lucene or whatever (Solr wraps that pretty well). Yes, it's another system, but it's a system that already works and scales :-) Or they delegate to Google Site Search. Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Fri, Sep 23, 2011 at 10:42 AM, Oleg Chernyh wrote: > I'm writing a forum engine that can sustain high amount of users making > queries at the same time. > I'm using a KV storage that stores erlang entities which is the way to kill > overheads and make data accessible really fast (if we know the key, ofc). > The problem I'm facing is full-text search, as you might guess. > I don't really want to use any external indexers or databases because I > want that piece of software to be scalable node-wise and don't want to add > any more data storage entites to the system to prevent data duplication and > to ensure consistency. > So what I want to do is to "invent bicycle" by implementing a full text > search on a key-value storage. > > Here I'll outline my plan for writing that feature and I'd be happy to hear > criticism. > > A simplistic way to think of a full text search engine is a strict text > search engine, when the result of a search is a list of messages (posts) > that contain full search keywords (if we search for "abc" only "abc" words > are matched, so that "abcd" won't match). > In order to accomplish that I can think of the following things to do: > 1) We make a set of all words W > 2) For each w \in W we have an "index" I_w which returns a list of all > posts that contain w > > When we add a pure full text search functionality (indeed, we might want > "geology" to match "geologist") we do the following: > 1) make a set of normalized words W_n > 2) make a set of common prefixes P > 3) make a set of common suffixes S > 4) define a normalization function f that strips common suffixes and > prefixes from a word > 5) define a matching function m that maps f(w) to a list of words from the > W_n set > 5.1) if there is no adequate entry for f(w), m function adds f(w) to W_n > 6) for each w in W_n we keep an "index" W_n that returns all the messages M > for which \exists x \in M : m(f(x)) = w > 7) for each word x in new message we asynchronously run m(f(x)) and update > indexes appropriately > 8) from time to time we re-index our W_n maps merging indexes for similar > words > > What are the pitfalls of that approach? Any feedback will be appreciated. > - > Oleg > > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From freza@REDACTED Sat Sep 24 20:17:36 2011 From: freza@REDACTED (Jachym Holecek) Date: Sat, 24 Sep 2011 14:17:36 -0400 Subject: [erlang-questions] How to see debug messages of In-Reply-To: References: Message-ID: <20110924181736.GA20139@circlewave.net> # Allen Kim 2011-09-24: > I m having some issues of having 1% of httpc:request/4 ends up with > {error, socket_closed_remotely}, 99 % are good with 200. > > I want to debug what's happening inside and I saw there are debug message > already in code. > > What should I do to see those ?hcrd/2 messages? The macro ends up defined in terms of do-nothing inets:report_event/4 function -- its only purpose is to serve as target for native tracing facility built into BEAM (and exposed to user via dbg(3) module). There seem to be a couple of convenience functions that should make the whole process much more user-friendly than raw dbg(3) API: -export([enable_trace/2, enable_trace/3, disable_trace/0, set_trace/1, report_event/4]). Hopefully this should get you started -- I'm not using httpc(3) at all and won't pretend to be very fluent with dbg(3) either. BR, -- Jachym From cgsmcmlxxv@REDACTED Sat Sep 24 23:04:34 2011 From: cgsmcmlxxv@REDACTED (CGS) Date: Sat, 24 Sep 2011 23:04:34 +0200 Subject: [erlang-questions] Full text search in KV-storage. In-Reply-To: References: <20110923235859.08552B9220E@mx2.eurocom.su> Message-ID: Hi, Oleg, Your idea may work pretty well, but there are few points you should consider: 1. at high number of posts, you may want to switch from int (the numbering the posts) in something more roomy; 2. if you build such a system even for one language, the dictionary database will be enormous and some parts will contain no posts (therefore, you should start ranking your dictionary entries, so, the least important to be at the end and not necessary loaded in RAM); 3. such a mapping implies that you know that language pretty well (pretty close to the level of a linguist); 4. I know no language which doesn't have differences in between the literature language and the spoken language (therefore, you may need another mapping from spoken to literature language). And there are other problems coming from such an approach. As Jesper said, an AI/ML approach would help to improve your approach, if I understood his message correctly. That means, in a simplistic way of putting it, do not create a vocabulary from the beginning, but allow vocabulary entries with ranking on them. Of course, that will not help you at all in finding meaningful prefixes and suffixes, but it will help you in retrieving faster the posts as many people will hit the peak of your ranking system. This approach will also allow humans to decide what is more relevant and what is less relevant (let's not forget human languages are quite illogic and, therefore, inventing algorithms to mimic human languages is not so easy). I would also add a system to increment the post number which should allow expansion beyond the basic data types sizes (maybe pagination wouldn't be a bad idea). Well, that's my 2c opinion. I hope it will help you in thinking your idea. Good luck! CGS On Sat, Sep 24, 2011 at 5:27 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Sat, Sep 24, 2011 at 01:58, Oleg Chernyh wrote: > > > I'm far from linguistics and full text search engines, do your input was > > particulary useful. > > And what about my idea that I have briefly described? > > I tried to be adept and avoid answering those parts because I know > very little about it. It looks like you do some stemming of words to > find their stem and then you index those. But I don't know what you do > to achieve it. If you pick a language, like english there are probably > two ways to go: 1. Form a set of rules and apply those rules to > "normalize"/"canonicalize"/"extract-the-stem". 2. Play the google > game: If you have enough data to mine statistically, figure out what > the stems are via machine learning. > > Historically, judging by papers at AI/ML conferences, it looks like > option 2 wins :P > > -- > J. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From allen.kim@REDACTED Sun Sep 25 06:39:07 2011 From: allen.kim@REDACTED (Allen Kim) Date: Sat, 24 Sep 2011 23:39:07 -0500 Subject: [erlang-questions] How to see debug messages of In-Reply-To: Message-ID: got it. could not stop myself to find this command. inets:enable_trace(max,io). From: Allen Kim > Date: Sat, 24 Sep 2011 12:02:07 -0500 To: erlang-questions > Cc: Allen Kim > Subject: How to see debug messages of Hi I m having some issues of having 1% of httpc:request/4 ends up with {error, socket_closed_remotely}, 99 % are good with 200. I want to debug what's happening inside and I saw there are debug message already in code. What should I do to see those ?hcrd/2 messages? -module(httpc_handler). -behaviour(gen_server). ... connect_and_send_first_request(Address, #request{settings = Settings, headers = Headers, address = OrigAddress, scheme = Scheme} = Request, #state{options = Options} = State) -> ?hcrd("connect", [{address, Address}, {request, Request}, {options, Options}]), ..... Allen Kim -------------- next part -------------- An HTML attachment was scrubbed... URL: From bighostkim@REDACTED Sun Sep 25 08:53:11 2011 From: bighostkim@REDACTED (Allen Kim) Date: Sun, 25 Sep 2011 02:53:11 -0400 Subject: [erlang-questions] about socket_closed_remotely error on inets httpc:request/4 Message-ID: Hi, First, forgive me my question is stupid because I don't have deep knowledge of inets. I am just eager to learn it. I am building a erlang loadtest tool using inets, My problem is when I send httpc:request/4, to our web server, I see 5-10% of requests ends up with error, {error, socket_closed_remotely}. It only happens if I send more than 200 transactions per second to a web server. However, if I try other loadtest tool such as openload or tsung, I don't see any errors at all. So, I think our web server is fine, but the my loadtest client using inets is not. I see the following line on http_client/httpc_handler.erl, and I don't understand from this code is, when From(request sender) is a Pid, why it responds with error, socket_closed_remotely. deliver_answer(#request{id = Id, from = From} = Request) ?when is_pid(From) -> ? ?Response = httpc_response:error(Request, socket_closed_remotely), ? ??hcrd("deliver answer", [{id, Id}, {from, From}, {response, Response}]), ? ?httpc_response:send(From, Response); deliver_answer(Request) -> ? ??hcrd("skip deliver answer", [{request, Request}]), ? ?ok. I thought that error is server-side socket is closed before the client send a request to receive a message from the server. It does not look like that case for me from the code above. Am I wrong about this? Thanks in advance Allen From erlang@REDACTED Sun Sep 25 16:58:37 2011 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 25 Sep 2011 16:58:37 +0200 Subject: [erlang-questions] Full text search in KV-storage. In-Reply-To: <20110923201936.29551qw15lcu7qu0@mail.udaff.com> References: <20110923201936.29551qw15lcu7qu0@mail.udaff.com> Message-ID: Hello, I wrote a full-text index in Erlang a while back. It's part of elib1 which can be found at https://github.com/joearms/elib1 elib1 contain a lot-lot more than just an index so it may not be apparent how to use it - you have to first install elib1 to even read the documentation (which is probably a mistake). The indexer in elib1 is a more-or-less literal translation of the algorithms in the book "Managing Gigabytes" by I.A.Witten, A.Moffat and T.C. Bell. (and by the way, implementing a full-text indexer without having read the above is a foolish thing to do - and I should know since I wrote two full-text indexer before reading this :-) The tricky bits in an indexer are creating the inverted index (hint use the erlang module file_sorter.erl (in the standard distribution) this is blazingly fast and possible my favorite "module in the Erlang distribution that nobody has ever heard of but which is amazingly good". The you have to compress the index - I use gamma compression described in the above book and implemented in https://github.com/joearms/elib1/blob/master/lib/src/elib1_gamma.erl. Index entries are represented by a variable bit length code. The trickiest part of all is to decide which words in the document should be indexed - ie answering the question "what is an English word" - one solution is to create all know trigrams (ie combination of three letters in a row) from all words in the english language and apply this as a filter to recognise unknown words. There is code to do this in my Erlang book :-) But even this gives unsatisfactory results. The twitter idea of indexing strings starting with # is much easier and yields better results. One of these years I might break out the full-text indexer from elib1 as a stand-alone application to make it easier to integrate. A further path than might be worth investigating would be to make a lucene front-end. The lucene file format is well documented - one might let some standard jaava of c++ indexing engine performing the actual indexing and build the necessary index on disk, and let Erlang do the queries. This way you'll only have to unpack the disk data structures and interpret the results. Thinking out the appropriate disk data structures for a full-tet index is not easy. If you want to index several GBytes of data you will not be able to work in-memory and a lot of thought has to be given to how to cache things in memory an avoid unnecessary disk reads and writes. This has been studied for years. I'm a great fan of reinventing the wheel but reading "Managing Gigbytes" will give a lot of insights into how to do this. Cheers /Joe On Fri, Sep 23, 2011 at 6:19 PM, Oleg Chernyh wrote: > I'm writing a forum engine that can sustain high amount of users making > queries at the same time. > I'm using a KV storage that stores erlang entities which is the way to kill > overheads and make data accessible really fast (if we know the key, ofc). > The problem I'm facing is full-text search, as you might guess. > I don't really want to use any external indexers or databases because I want > that piece of software to be scalable node-wise and don't want to add any > more data storage entites to the system to prevent data duplication and to > ensure consistency. > So what I want to do is to "invent bicycle" by implementing a full text > search on a key-value storage. > > Here I'll outline my plan for writing that feature and I'd be happy to hear > criticism. > > A simplistic way to think of a full text search engine is a strict text > search engine, when the result of a search is a list of messages (posts) > that contain full search keywords (if we search for "abc" only "abc" words > are matched, so that "abcd" won't match). > In order to accomplish that I can think of the following things to do: > 1) We make a set of all words W > 2) For each w \in W we have an "index" I_w which returns a list of all posts > that contain w > > When we add a pure full text search functionality (indeed, we might want > "geology" to match "geologist") we do the following: > 1) make a set of normalized words W_n > 2) make a set of common prefixes P > 3) make a set of common suffixes S > 4) define a normalization function f that strips common suffixes and > prefixes from a word > 5) define a matching function m that maps f(w) to a list of words from the > W_n set > 5.1) > 6) for each w in W_n we keep an "index" W_n that returns all the messages M > for which \exists x \in M : m(f(x)) = w > 7) for each word m in new message > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From baliulia@REDACTED Mon Sep 26 01:20:32 2011 From: baliulia@REDACTED (=?UTF-8?B?SWduYXMgVnnFoW5pYXVza2Fz?=) Date: Mon, 26 Sep 2011 00:20:32 +0100 Subject: [erlang-questions] Different hostname from erl -sname and inet:gethostname() Message-ID: <4E7FB740.6040309@gmail.com> Hi erlang-questions, I have the following behaviour: [~]$ erl -sname foobar Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) (foobar@REDACTED)1> inet:gethostname(). {ok,"pufpuf"} Notice the different 'localhost' and 'pufpuf' hostnames. I am running Erlang R14B-2 on ArchLinux 3.0. [~]$ hostname pufpuf [~]$ cat /etc/hosts # 127.0.0.1 localhost.localdomain localhost pufpuf ::1 localhost.localdomain localhost Is it normal behaviour? I am using things like: {ok, Hostname} = inet:gethostname(), slave:start(Hostname, slave_node, []). which obviously fails under such circumstances. Is there a different approach to this? Is my system misconfigured? /Ignas From magnus.klaar@REDACTED Mon Sep 26 01:51:38 2011 From: magnus.klaar@REDACTED (Magnus Klaar) Date: Mon, 26 Sep 2011 01:51:38 +0200 Subject: [erlang-questions] Different hostname from erl -sname and inet:gethostname() In-Reply-To: <4E7FB740.6040309@gmail.com> References: <4E7FB740.6040309@gmail.com> Message-ID: Hi! The inet:gethostname() function returns the name of the system that the node is running on. This is the name that is printed by the hostname command. This persistent configuration data is not stored in the /etc/hosts, but in the /etc/hostname file. The presence of your hostname in the /etc/hosts file is to ensure that the local system is reachable at the loopback address, if needed. If you always want to start a node on localhost and use short names you can use use slave:start("localhost", slave_node, []). MVH Magnus 2011/9/26 Ignas Vy?niauskas > Hi erlang-questions, > > I have the following behaviour: > > [~]$ erl -sname foobar > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:4:4] [rq:4] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.1 (abort with ^G) > (foobar@REDACTED)1> inet:gethostname(). > {ok,"pufpuf"} > > Notice the different 'localhost' and 'pufpuf' hostnames. > > I am running Erlang R14B-2 on ArchLinux 3.0. > > [~]$ hostname > pufpuf > > [~]$ cat /etc/hosts > # > 127.0.0.1 localhost.localdomain localhost pufpuf > ::1 localhost.localdomain localhost > > Is it normal behaviour? I am using things like: > > {ok, Hostname} = inet:gethostname(), > slave:start(Hostname, slave_node, []). > > which obviously fails under such circumstances. > Is there a different approach to this? Is my system misconfigured? > > /Ignas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From magnus.klaar@REDACTED Mon Sep 26 02:14:40 2011 From: magnus.klaar@REDACTED (Magnus Klaar) Date: Mon, 26 Sep 2011 02:14:40 +0200 Subject: [erlang-questions] Different hostname from erl -sname and inet:gethostname() In-Reply-To: References: <4E7FB740.6040309@gmail.com> Message-ID: In hindsight, i missed your last question. If you do want pufpuf to be the name for 127.0.0.1 you can place it at the head of the list of names associated with an address in your hosts file. 127.0.0.1 pufpuf localhost.localdomain localhost ::1 pufpuf localhost.localdomain localhost -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Mon Sep 26 11:58:02 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 26 Sep 2011 11:58:02 +0200 Subject: [erlang-questions] Different hostname from erl -sname and inet:gethostname() In-Reply-To: References: <4E7FB740.6040309@gmail.com> Message-ID: <20110926095802.GA5618@erix.ericsson.se> On Mon, Sep 26, 2011 at 02:14:40AM +0200, Magnus Klaar wrote: > In hindsight, i missed your last question. If you do want pufpuf to be the > name for 127.0.0.1 you can place it at the head of the list of names > associated with an address in your hosts file. > > 127.0.0.1 pufpuf localhost.localdomain localhost > ::1 pufpuf localhost.localdomain localhost On a local network other machines may resolve 'pufpuf' to its external interface address, but 'pufpuf' itself resolves then name to the loopback address. This discrapancy can confuse NFS, automounter, etc... For the sake of Erlang/OTP you do not need to put the short hostname on the loopback address; if inet:gethostbyname fails to resolve the short hostname it will fall back to using the loopback address. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From vances@REDACTED Mon Sep 26 12:56:58 2011 From: vances@REDACTED (Vance Shipley) Date: Mon, 26 Sep 2011 16:26:58 +0530 Subject: [erlang-questions] Has Anybody noticed to_erl broken in R14B03 on OS X? Message-ID: <20110926105656.GR16095@aluminum.local> Is it just me or has R14B03 broken to_erl? -- -Vance From wiener.guy@REDACTED Mon Sep 26 13:38:19 2011 From: wiener.guy@REDACTED (Guy Wiener) Date: Mon, 26 Sep 2011 14:38:19 +0300 Subject: [erlang-questions] Erlang on ARM9? In-Reply-To: <0977DC71-22ED-43E0-8BD3-F2889C6FC370@isep.ipp.pt> References: <0977DC71-22ED-43E0-8BD3-F2889C6FC370@isep.ipp.pt> Message-ID: Looks like I'll have to compile from source - Embedded Erlang is not what I need because it includes its-own embedded Linux, and I need to install on an existing Linux. Can someone who has done this before be as kind as writing a minimal how-to for cross-compiling Erlang to ARM? Thanks again, Guy On Tue, Sep 20, 2011 at 5:16 PM, Paulo Ferreira wrote: > > On 2011/09/19, at 09:14, Guy Wiener wrote: > > > Hello Erlangers, > > I am checking an option to use Erlang to control a Linux-based device (a > programmable helicopter). Is there an efficient port of Erlang that works > on top of an ARM9 processor? If so, the usage scenario will be many > processes running on a single node, with frequent message-passing, some > access to binary files, and no shared data (no ETS, DETS or Mnesia). Is > this scenario likely to perform well on an embedded Erlang? > > > > Thanks, > > Guy Wiener, > > Dept. of Computer Science and Applied Math, > > Weizmann Institute for Science. > > > Well, just it just compiles from source well. > > I have experience using a S3C2440 board at 400 Mhz and on some informal > benchmarks ( ring ) it is just 7 times slower than a 2,2 Ghz core 2 Duo Mac > book. > > From other erlang embedded experiences: > > How do you control the hardware in C? Linux device drivers are very > slow, when compared to direct memory access. > Doing NIFs and dynamic libs may be hard. > ARM processors are usually supported by the latest gcc version, and > that is VERY good for Erlang porting. > If you try to compile Erlang with an older gcc version, you may get > all the kind of "strange and wonderful" errors. > Check the gcc version of your toolchain! > > > My best regards > > Paulo Ferreira > > P.S.: Please replace "strange and wonderful" by the suitable words in your > native language.... > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Mon Sep 26 13:48:05 2011 From: tony@REDACTED (Tony Rogvall) Date: Mon, 26 Sep 2011 13:48:05 +0200 Subject: [erlang-questions] Erlang on ARM9? In-Reply-To: References: <0977DC71-22ED-43E0-8BD3-F2889C6FC370@isep.ipp.pt> Message-ID: <75F7CC13-4F65-4F07-9E9E-2A2C2CF08E73@rogvall.se> On 26 sep 2011, at 13:38, Guy Wiener wrote: > Looks like I'll have to compile from source - Embedded Erlang is not what I need because it includes its-own embedded Linux, and I need to install on an existing Linux. > > Can someone who has done this before be as kind as writing a minimal how-to for cross-compiling Erlang to ARM? > I guess you found this link?: http://www.trapexit.org/Cross_compiling /Tony > Thanks again, > Guy > > On Tue, Sep 20, 2011 at 5:16 PM, Paulo Ferreira wrote: > > On 2011/09/19, at 09:14, Guy Wiener wrote: > > > Hello Erlangers, > > I am checking an option to use Erlang to control a Linux-based device (a programmable helicopter). Is there an efficient port of Erlang that works on top of an ARM9 processor? If so, the usage scenario will be many processes running on a single node, with frequent message-passing, some access to binary files, and no shared data (no ETS, DETS or Mnesia). Is this scenario likely to perform well on an embedded Erlang? > > > > Thanks, > > Guy Wiener, > > Dept. of Computer Science and Applied Math, > > Weizmann Institute for Science. > > > Well, just it just compiles from source well. > > I have experience using a S3C2440 board at 400 Mhz and on some informal benchmarks ( ring ) it is just 7 times slower than a 2,2 Ghz core 2 Duo Mac book. > > >From other erlang embedded experiences: > > How do you control the hardware in C? Linux device drivers are very slow, when compared to direct memory access. > Doing NIFs and dynamic libs may be hard. > ARM processors are usually supported by the latest gcc version, and that is VERY good for Erlang porting. > If you try to compile Erlang with an older gcc version, you may get all the kind of "strange and wonderful" errors. > Check the gcc version of your toolchain! > > > My best regards > > Paulo Ferreira > > P.S.: Please replace "strange and wonderful" by the suitable words in your native language.... > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From banibrata.dutta@REDACTED Mon Sep 26 14:22:22 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Mon, 26 Sep 2011 17:52:22 +0530 Subject: [erlang-questions] Erlang on ARM9? In-Reply-To: References: <0977DC71-22ED-43E0-8BD3-F2889C6FC370@isep.ipp.pt> Message-ID: This ARM9 in question would have some kind of a cross compile environment. For example, there's a rather popular (& cheap) ARM9 board called mini2440, and there are 2-3 different cross compilers... there's one from FriendlyARM, there's one from a co. called Pengutronix, then there's an old one from ARM itself, and some folks have rolled their own. On Mon, Sep 26, 2011 at 5:08 PM, Guy Wiener wrote: > Looks like I'll have to compile from source - Embedded Erlang is not what I > need because it includes its-own embedded Linux, and I need to install on an > existing Linux. > > Can someone who has done this before be as kind as writing a minimal how-to > for cross-compiling Erlang to ARM? > > Thanks again, > Guy > > > On Tue, Sep 20, 2011 at 5:16 PM, Paulo Ferreira wrote: > >> >> On 2011/09/19, at 09:14, Guy Wiener wrote: >> >> > Hello Erlangers, >> > I am checking an option to use Erlang to control a Linux-based device (a >> programmable helicopter). Is there an efficient port of Erlang that works >> on top of an ARM9 processor? If so, the usage scenario will be many >> processes running on a single node, with frequent message-passing, some >> access to binary files, and no shared data (no ETS, DETS or Mnesia). Is >> this scenario likely to perform well on an embedded Erlang? >> > >> > Thanks, >> > Guy Wiener, >> > Dept. of Computer Science and Applied Math, >> > Weizmann Institute for Science. >> >> >> Well, just it just compiles from source well. >> >> I have experience using a S3C2440 board at 400 Mhz and on some informal >> benchmarks ( ring ) it is just 7 times slower than a 2,2 Ghz core 2 Duo Mac >> book. >> >> From other erlang embedded experiences: >> >> How do you control the hardware in C? Linux device drivers are >> very slow, when compared to direct memory access. >> Doing NIFs and dynamic libs may be hard. >> ARM processors are usually supported by the latest gcc version, and >> that is VERY good for Erlang porting. >> If you try to compile Erlang with an older gcc version, you may get >> all the kind of "strange and wonderful" errors. >> Check the gcc version of your toolchain! >> >> >> My best regards >> >> Paulo Ferreira >> >> P.S.: Please replace "strange and wonderful" by the suitable words in your >> native language.... >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gustav.simonsson@REDACTED Mon Sep 26 14:39:56 2011 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Mon, 26 Sep 2011 13:39:56 +0100 (BST) Subject: [erlang-questions] Erlang on ARM9? In-Reply-To: <75F7CC13-4F65-4F07-9E9E-2A2C2CF08E73@rogvall.se> Message-ID: <0b58b591-dac9-4b51-8835-b941f5d6229a@knuth> The trapexit guide assumes you have a proper cross compilation toolchain. You can generate a ARM9 toolchain using OpenEmbedded either by following one of the guides online for creating a OE toolchain or by configuring OE for a ARM9 machine and building some app or system image with it, as that will also generate a ARM9 toolchain. Regards, Gustav Simonsson ----- Ursprungligt meddelande ----- Fr?n: "Tony Rogvall" Till: "Guy Wiener" Kopia: "Erlang Questions" Skickat: m?ndag, 26 sep 2011 13:48:05 ?mne: Re: [erlang-questions] Erlang on ARM9? On 26 sep 2011, at 13:38, Guy Wiener wrote: Looks like I'll have to compile from source - Embedded Erlang is not what I need because it includes its-own embedded Linux, and I need to install on an existing Linux. Can someone who has done this before be as kind as writing a minimal how-to for cross-compiling Erlang to ARM? I guess you found this link?: http://www.trapexit.org/Cross_compiling /Tony Thanks again, Guy On Tue, Sep 20, 2011 at 5:16 PM, Paulo Ferreira < pdf@REDACTED > wrote: On 2011/09/19, at 09:14, Guy Wiener wrote: > Hello Erlangers, > I am checking an option to use Erlang to control a Linux-based device (a programmable helicopter). Is there an efficient port of Erlang that works on top of an ARM9 processor? If so, the usage scenario will be many processes running on a single node, with frequent message-passing, some access to binary files, and no shared data (no ETS, DETS or Mnesia). Is this scenario likely to perform well on an embedded Erlang? > > Thanks, > Guy Wiener, > Dept. of Computer Science and Applied Math, > Weizmann Institute for Science. Well, just it just compiles from source well. I have experience using a S3C2440 board at 400 Mhz and on some informal benchmarks ( ring ) it is just 7 times slower than a 2,2 Ghz core 2 Duo Mac book. >From other erlang embedded experiences: How do you control the hardware in C? Linux device drivers are very slow, when compared to direct memory access. Doing NIFs and dynamic libs may be hard. ARM processors are usually supported by the latest gcc version, and that is VERY good for Erlang porting. If you try to compile Erlang with an older gcc version, you may get all the kind of "strange and wonderful" errors. Check the gcc version of your toolchain! My best regards Paulo Ferreira P.S.: Please replace "strange and wonderful" by the suitable words in your native language.... _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From bighostkim@REDACTED Mon Sep 26 16:48:47 2011 From: bighostkim@REDACTED (Allen Kim) Date: Mon, 26 Sep 2011 10:48:47 -0400 Subject: [erlang-questions] Parsing Url. Message-ID: For those who is interested in parsing url with a single function, -record(url,{protocol,user,password,host,port,path}). parse_url(Url) -> {ok,RE}= re:compile("^(http[s]?://)?([a-z0-9]+(:[^@]+)?@)?([a-z0-9\.\-]+(:[0-9]+)?)([/].*)?$",[caseless]), case re:run(Url, RE, [global, {capture, all, list}]) of {match,[[_Match,Scheme,UserPass,Password,Host]]} -> Protocol = re:replace(Scheme,"://","",[{return,list}]), User = re:replace(UserPass,Password++"@","",[{return,list}]), #url{protocol=Protocol,user=User,password=Password,host=Host,port="80",path=""}; {match,[[_Match,Scheme,UserPass,Password,UrlHost,UrlPort]]} -> Protocol = re:replace(Scheme,"://","",[{return,list}]), User = re:replace(UserPass,Password++"@","",[{return,list}]), Host = re:replace(UrlHost,":[0-9]+$","",[{return,list}]), Port = re:replace(UrlPort,":","",[{return,list}]), #url{protocol=Protocol,user=User,password=Password,host=Host,port=Port,path=""}; {match,[[_Match,Scheme,UserPass,Password,UrlHost,UrlPort,Path]]} -> Protocol = re:replace(Scheme,"://","",[{return,list}]), User = re:replace(UserPass,Password++"@","",[{return,list}]), Host = re:replace(UrlHost,":[0-9]+$","",[{return,list}]), Port = re:replace(UrlPort,":","",[{return,list}]), #url{protocol=Protocol,user=User,password=Password,host=Host,port=Port,path=Path}; nomatch -> #url{} end. Allen Kim From watson.timothy@REDACTED Mon Sep 26 17:43:06 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 26 Sep 2011 16:43:06 +0100 Subject: [erlang-questions] getting type info at runtime Message-ID: I'd like to be able to get type info declared using -spec annotations at runtime. Is this something that others would consider useful? Is it in the pipeline already, or is a patch required? Also, is there a recommended way to get the -spec data back using code available in existing R14 releases? My goal is to be able to locate things (e.g., modules and functions) by name and/or type signature. -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Mon Sep 26 18:00:58 2011 From: robert.virding@REDACTED (Robert Virding) Date: Mon, 26 Sep 2011 17:00:58 +0100 (BST) Subject: [erlang-questions] Erlounge or informal EUG meeting in Moscow In-Reply-To: Message-ID: <2b9ae5e3-a63f-4346-83ce-3827131c8296@knuth> I now have some more news. The Highload++ organisers have confirmed that we can use one of their rooms on Monday evening for an informal meeting AND that after 7pm they will allow "external" people (those not attending the actual conference) to enter. Thank you very much. This does not stop us holding something during the weekend as well. I like beer so at least an erlounge during the weekend would be welcome. I need help from someone who knows Moscow here. Robert ----- Original Message ----- > I will be at the HighLoad++ conference in Moscow next weekend, > 1-4/10, anyone for an Erlounge or maybe an informal Erlang user > group meeting? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From max.lapshin@REDACTED Mon Sep 26 18:10:55 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 26 Sep 2011 23:10:55 +0700 Subject: [erlang-questions] Erlounge or informal EUG meeting in Moscow In-Reply-To: <2b9ae5e3-a63f-4346-83ce-3827131c8296@knuth> References: <2b9ae5e3-a63f-4346-83ce-3827131c8296@knuth> Message-ID: On Mon, Sep 26, 2011 at 11:00 PM, Robert Virding wrote: > I now have some more news. > > The Highload++ organisers have confirmed that we can use one of their rooms on Monday evening for an informal meeting AND that after 7pm they will allow "external" people (those not attending the actual conference) to enter. Thank you very much. > > This does not stop us holding something during the weekend as well. I like beer so at least an erlounge during the weekend would be welcome. I need help from someone who knows Moscow here. > Of course, Robert. We have already discussed, where to invite you =) From lukas.larsson@REDACTED Mon Sep 26 18:13:24 2011 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Mon, 26 Sep 2011 17:13:24 +0100 (BST) Subject: [erlang-questions] getting type info at runtime In-Reply-To: Message-ID: <3bdebdb2-b536-4585-badf-4be5d4477054@knuth> Hi, You can get the raw data from beam_lib:chunks(Beam,[abstract_code]) and then go from there. There is afaik no other interface to get the type information in runtime, maybe some function within dialyzer? The beam file has to be compiled with debug_info though for the spec information to be made part of the beam file. Lukas ----- Original Message ----- From: "Tim Watson" To: erlang-questions@REDACTED Sent: Monday, September 26, 2011 5:43:06 PM Subject: Re: [erlang-questions] getting type info at runtime I'd like to be able to get type info declared using -spec annotations at runtime. Is this something that others would consider useful? Is it in the pipeline already, or is a patch required? Also, is there a recommended way to get the -spec data back using code available in existing R14 releases? My goal is to be able to locate things (e.g., modules and functions) by name and/or type signature. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From demeshchuk@REDACTED Mon Sep 26 18:16:41 2011 From: demeshchuk@REDACTED (Dmitry Demeshchuk) Date: Mon, 26 Sep 2011 20:16:41 +0400 Subject: [erlang-questions] Erlounge or informal EUG meeting in Moscow In-Reply-To: <2b9ae5e3-a63f-4346-83ce-3827131c8296@knuth> References: <2b9ae5e3-a63f-4346-83ce-3827131c8296@knuth> Message-ID: Robert, so do I understand correctly that you don't have any certain plans for the weekend, and you could take a Moscow tour (if we get decent weather, which will likely happen) with some of us? Will someone meet you at the airport, btw? On Mon, Sep 26, 2011 at 8:00 PM, Robert Virding wrote: > I now have some more news. > > The Highload++ organisers have confirmed that we can use one of their rooms on Monday evening for an informal meeting AND that after 7pm they will allow "external" people (those not attending the actual conference) to enter. Thank you very much. > > This does not stop us holding something during the weekend as well. I like beer so at least an erlounge during the weekend would be welcome. I need help from someone who knows Moscow here. > > Robert > > > > ----- Original Message ----- >> I will be at the HighLoad++ conference in Moscow next weekend, >> 1-4/10, anyone for an Erlounge or maybe an informal Erlang user >> group meeting? >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Best regards, Dmitry Demeshchuk From zabrane3@REDACTED Mon Sep 26 18:24:20 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Mon, 26 Sep 2011 18:24:20 +0200 Subject: [erlang-questions] Erlang and paypal integration for web application Message-ID: <3812C755-8144-49ED-9072-C6890BBCEE49@gmail.com> Hi guys, Is there any available Erlang package to simplify online payment with PayPal. The following module erlpay doesn't exist anymore on GoogleCode: http://code.google.com/p/erlpay/ And this one erlpaypal is empty: http://code.google.com/p/erlpaypal/ Regards, Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonas.boberg@REDACTED Mon Sep 26 18:28:00 2011 From: jonas.boberg@REDACTED (Jonas Boberg) Date: Mon, 26 Sep 2011 18:28:00 +0200 Subject: [erlang-questions] getting type info at runtime In-Reply-To: References: Message-ID: Hi, We solve this by having a parse transformation that adds additional functions to the module (at compile time). More specifically, we use this to generate functions that serialize and deserialize json objects to records, including type verification based on the spec annotations. Regards Jonas From atilla.erdodi@REDACTED Mon Sep 26 18:33:23 2011 From: atilla.erdodi@REDACTED (=?ISO-8859-2?Q?Atilla_Erd=F5di?=) Date: Mon, 26 Sep 2011 18:33:23 +0200 Subject: [erlang-questions] Erlang and paypal integration for web application In-Reply-To: <3812C755-8144-49ED-9072-C6890BBCEE49@gmail.com> References: <3812C755-8144-49ED-9072-C6890BBCEE49@gmail.com> Message-ID: Hi, There is one for Zotonic: http://modules.zotonic.com/page/341/mod-paypal Atilla 2011/9/26 Zabrane Mickael : > Hi guys, > Is there any available Erlang package to simplify online payment with > PayPal. > The following module erlpay doesn't exist anymore on GoogleCode: > http://code.google.com/p/erlpay/ > And this one erlpaypal is empty: > http://code.google.com/p/erlpaypal/ > Regards, > Zabrane > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From zabrane3@REDACTED Mon Sep 26 18:37:00 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Mon, 26 Sep 2011 18:37:00 +0200 Subject: [erlang-questions] Erlang and paypal integration for web application In-Reply-To: References: <3812C755-8144-49ED-9072-C6890BBCEE49@gmail.com> Message-ID: > There is one for Zotonic: http://modules.zotonic.com/page/341/mod-paypal Thanks Attila ;-) From daniel.goertzen@REDACTED Mon Sep 26 18:47:45 2011 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Mon, 26 Sep 2011 11:47:45 -0500 Subject: [erlang-questions] tracing very large terms Message-ID: I am struggling to debug functions that reference very large terms (thousands of lines to print). Crash reports and trace outputs are huge and finding the interesting bits is hard. This isn't a trace pattern problem; actual number of trace outputs is maybe a dozen or so. Just wondering what others do to manage this situation. I've looked for a text editor that can do parenthesis folding, but no luck so far. Ideas? Dan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Sep 26 18:51:44 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 26 Sep 2011 23:51:44 +0700 Subject: [erlang-questions] tracing very large terms In-Reply-To: References: Message-ID: I had to make such module: https://github.com/erlyvideo/erlyvideo/blob/master/apps/erlyvideo/src/core/io_lib_pretty_limited.erl for this fix: https://github.com/erlyvideo/erlyvideo/blob/master/apps/erlyvideo/src/core/gen_server_ems.erl#L747 Without this code one large process can kill whole VM while writing logs about his death. On Mon, Sep 26, 2011 at 11:47 PM, Daniel Goertzen wrote: > I am struggling to debug functions that reference very large terms > (thousands of lines to print). ?Crash reports and trace outputs are huge and > finding the interesting bits is hard. ?This isn't a trace pattern problem; > actual number of trace outputs is maybe a dozen or so. > Just wondering what others do to manage this situation. ?I've looked for a > text editor that can do parenthesis folding, but no luck so far. ?Ideas? > Dan. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From demeshchuk@REDACTED Mon Sep 26 18:53:03 2011 From: demeshchuk@REDACTED (Dmitry Demeshchuk) Date: Mon, 26 Sep 2011 20:53:03 +0400 Subject: [erlang-questions] tracing very large terms In-Reply-To: References: Message-ID: There are riak_error and lager for solving such issues. I'd recommend the latter, since lager does much more useful stuff. On Mon, Sep 26, 2011 at 8:47 PM, Daniel Goertzen wrote: > I am struggling to debug functions that reference very large terms > (thousands of lines to print). ?Crash reports and trace outputs are huge and > finding the interesting bits is hard. ?This isn't a trace pattern problem; > actual number of trace outputs is maybe a dozen or so. > Just wondering what others do to manage this situation. ?I've looked for a > text editor that can do parenthesis folding, but no luck so far. ?Ideas? > Dan. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Best regards, Dmitry Demeshchuk From erlang@REDACTED Mon Sep 26 19:44:30 2011 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 26 Sep 2011 19:44:30 +0200 Subject: [erlang-questions] tracing very large terms In-Reply-To: References: Message-ID: On Mon, Sep 26, 2011 at 6:47 PM, Daniel Goertzen wrote: > I am struggling to debug functions that reference very large terms > (thousands of lines to print). ?Crash reports and trace outputs are huge and > finding the interesting bits is hard. ?This isn't a trace pattern problem; > actual number of trace outputs is maybe a dozen or so. > Just wondering what others do to manage this situation. ?I've looked for a > text editor that can do parenthesis folding, but no luck so far. ?Ideas? I just pretty print the the term(s) to a file then fire up emacs :-) dump(File, Term) -> io:format("** dumping to ~s~n",[File]), {ok, S} = file:open(File, [write]), io:format(S, "~p.~n",[Term]), file:close(S). /Joe > Dan. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From max.lapshin@REDACTED Mon Sep 26 19:51:45 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 27 Sep 2011 00:51:45 +0700 Subject: [erlang-questions] tracing very large terms In-Reply-To: References: Message-ID: On Tue, Sep 27, 2011 at 12:44 AM, Joe Armstrong wrote: > ? ?io:format(S, "~p.~n",[Term]), This is most interesting place. It is ok, when your process, that is dumping term on disk can fit into 2-8 GB of RAM, but I have met situations, when 20 GB was not enough. Printing translate everything to lists =( From desired.mta@REDACTED Mon Sep 26 19:53:57 2011 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Mon, 26 Sep 2011 18:53:57 +0100 Subject: [erlang-questions] getting type info at runtime In-Reply-To: References: Message-ID: On Mon, Sep 26, 2011 at 17:28, Jonas Boberg wrote: > Hi, > > We solve this by having a parse transformation that adds additional > functions to the module (at compile time). More specifically, we use > this to generate functions that serialize and deserialize json objects > to records, including type verification based on the spec annotations. Hi, is this hosted somewhere? Motiejus Jak?tys From krab@REDACTED Mon Sep 26 19:58:59 2011 From: krab@REDACTED (Kresten Krab Thorup) Date: Mon, 26 Sep 2011 19:58:59 +0200 Subject: [erlang-questions] Full text search in KV-storage. In-Reply-To: <20110923201936.29551qw15lcu7qu0@mail.udaff.com> References: <20110923201936.29551qw15lcu7qu0@mail.udaff.com> Message-ID: <1E9A9CFF-43E0-4F95-A530-9148003339AC@trifork.com> You should take a look at https://github.com/basho/merge_index Kresten On 23/09/2011, at 18.20, "Oleg Chernyh" wrote: > I'm writing a forum engine that can sustain high amount of users > making queries at the same time. > I'm using a KV storage that stores erlang entities which is the way to > kill overheads and make data accessible really fast (if we know the > key, ofc). > The problem I'm facing is full-text search, as you might guess. > I don't really want to use any external indexers or databases because > I want that piece of software to be scalable node-wise and don't want > to add any more data storage entites to the system to prevent data > duplication and to ensure consistency. > So what I want to do is to "invent bicycle" by implementing a full > text search on a key-value storage. > > Here I'll outline my plan for writing that feature and I'd be happy to > hear criticism. > > A simplistic way to think of a full text search engine is a strict > text search engine, when the result of a search is a list of messages > (posts) that contain full search keywords (if we search for "abc" only > "abc" words are matched, so that "abcd" won't match). > In order to accomplish that I can think of the following things to do: > 1) We make a set of all words W > 2) For each w \in W we have an "index" I_w which returns a list of all > posts that contain w > > When we add a pure full text search functionality (indeed, we might > want "geology" to match "geologist") we do the following: > 1) make a set of normalized words W_n > 2) make a set of common prefixes P > 3) make a set of common suffixes S > 4) define a normalization function f that strips common suffixes and > prefixes from a word > 5) define a matching function m that maps f(w) to a list of words from > the W_n set > 5.1) > 6) for each w in W_n we keep an "index" W_n that returns all the > messages M for which \exists x \in M : m(f(x)) = w > 7) for each word m in new message > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From watson.timothy@REDACTED Mon Sep 26 22:13:23 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 26 Sep 2011 21:13:23 +0100 Subject: [erlang-questions] getting type info at runtime In-Reply-To: References: Message-ID: 2011/9/26 Motiejus Jak?tys > On Mon, Sep 26, 2011 at 17:28, Jonas Boberg > wrote: > > Hi, > > > > We solve this by having a parse transformation that adds additional > > functions to the module (at compile time). More specifically, we use > > this to generate functions that serialize and deserialize json objects > > to records, including type verification based on the spec annotations. > > Hi, > > is this hosted somewhere? > > Motiejus Jak?tys > Hi all, Thanks for the responses - I'll answer a few questions and clarify. I should have worded this question "getting type info from beam code compiled without debug_info" instead. I'd like to be able to get the -spec info for several reasons: 1. Discovery - think hoogle for Erlang packages 2. Static Analysis 3. Instrumentation The idea of searching based on type information has come up on the list before. Item (2) is obvious, and tools like dialyzer and QuickCheck/PropEr are already using this type information to great effect. I just haven't figured out how to unpick what they're doing properly. As for (3), there are amazing capabilities built into erlang thanks to the tracing capabilities in erts. I'm using these for diagnostic purposes, but I don't see them as something that I'd like to have *always on* in my production code. There are limitations to using a tracer process and of course writing to a file or socket is safe(er) but there are limited guarantees about what gets delivered and ordering and so on. Tracing feels to me like something I turn on for diagnostic purposes. The kind of instrumentation I'm after is more akin to something AOP, where I want to get the build system to do some work for me but keep a tight control of what code gets generated as it is now a proper part of my application. The kind of post processing on the beam code I'd like to do might insert things like logging statements or increment lightweight performance counters. I'd like to do this as part of my build, so that I can (a) pretty print the beam (using erl_pp) if I want to see what's been added (b) make sure that the instrumented code is what gets unit/integration/load tested In order to apply these modifications to my own source code, I can use a parse_transform and that's easy enough. I'd also like to have a declarative approach for defining where the instrumentation/code-weaving takes place, and type signatures seem like a good candidate for this. Let's say I want to put a performance counter around calls to any function in any module which has a name ending in _db, taking a #'myapp.query'{} record and an opaque connection_handle() type. I might define the target operations (in my own source code which is to be instrumented) by saying something like: around('call(*_log:*(#'myapp.query'{}, connection_handle()))', Handler) %% where Handler is an {M,F,A} or callback module conforming to a simple API Now I suppose this is somewhat akin to a match specification {'_', '_', ...} with some fancy stuff to handle the arguments such as {['$1', '$1', '$1'], [{is_record, '$1', 'myapp.query', record_info(size, 'myapp.query')}], .... So maybe match specifications would make a better language, but they are a bit hard to follow (no offense to the good OTP folks intended) and type specifications with wildcards for naming (modules and functions) seem a bit simpler to me, at least for my purposes. Now I *might* want to put a performance counter and/or logging statement (or whatever) around calls to _someone else's code_ such as calls to mochiweb_request or whatever, which I may not have the sources for and which may not have been compiled with debug_info (or used the "abstract_code" parse transform that's floating around). In those cases, it would be really nice if the compiler would keep hold of the type signatures in the beam, even when the abstract code isn't present. Of course in the meanwhile, if the abstract code is present I can get it out using beam_lib, but the clever people who wrote dialyzer/QuickCheck/PropEr must have solved this already. So I'm wondering whether 1. The type introspection code it could be "unpicked" from these tools easily - I will need pointing in the right direction though! 2. The compiler should preserve type information in the generated beam, and if so is this in any roadmap or is a contribution the way to go? 3. Have I got the wrong impression of the trace facility or not I see some kind of "type server" in both PropEr and Dialyzer and I'm pretty sure if this was abstracted out into a separate component that it could nicely form the core of a erlang-search web service or something of that ilk. I'd also be interested in this code you've mentioned Jonas, if it is open source. For matching type specifications against modules/functions in other people's code however, a parse_transform isn't what I need. Please do call out better approaches if you have them, but also bare in mind this is a mainly fun/educational exercise at the moment! Cheers, Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: From victor.sovetov@REDACTED Mon Sep 26 22:20:26 2011 From: victor.sovetov@REDACTED (Viktor Sovietov) Date: Mon, 26 Sep 2011 13:20:26 -0700 (PDT) Subject: [erlang-questions] Seminar on Erlang/OTP in Kiev, Oct 2nd Message-ID: <9982e7e4-e156-4540-b909-2a79a9ff7d7e@i28g2000yqn.googlegroups.com> Erlang Solutions sponsoring the 10th OSDN.Org.UA Conference, the meeting for the users and developers of Free Software, will be held in Kiev, Ukraine on October 1, 2011 We will also be holding an additional event: a four hour long seminar on Erlang held on October 2nd. This hands on tutorial will give you an introduction to the Erlang programming language. You will learn the basics of how to read, write and structure Erlang programs. We start with an insight into the theory and concepts behind sequential and concurrent Erlang, allowing you to get acquainted with the Erlang syntax and semantics. We conclude with an overview of the error handling mechanisms used to build fault tolerant systems with five nines availability. The tutorial is free of charge. Participants would need to bring their laptops with them with Erlang/OTP installed. An installation package can be found here ( http://www.erlang-solutions.com/section/132/erlang-otp-packages ). http://www.erlang-solutions.com/news/1/entry/1266 From watson.timothy@REDACTED Mon Sep 26 22:30:13 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 26 Sep 2011 21:30:13 +0100 Subject: [erlang-questions] logging from linked-in drivers In-Reply-To: References: Message-ID: > > > Yeah, this is what I do with debugging statements. I'd like to put > individual log lines/entries out to the file system rather than the console. > Given that posix requires stdio operations dealing with FILE* structures to > work atomically, do you think fprintf calls are a reasonable thing to do in > a multi-threaded driver?. > > >> This seems like an inane question. What I'm getting at is that I've seen plenty of driver code which writes to a file like this only when protected by a driver mutex. I'm guessing that this is because they are pushing multiple lines/writes out in one go and want to avoid interleaving the output when many threads simultaneously call the write/log function. Now I don't care about this interleaving as all my logging statements fit in a single (s)printf call and I couldn't care less about the order in which they appear in the log - the thread id is there to disseminate this information later on. I'm under the impression that posix compliant systems will ensure that the single write (and therefore individual log lines) are written atomically. Have I got this right and is this is reasonable assumption to make - it feels like no, but I'm being over cautious as I don't know the answer and don't want to write useless driver code. I'd also like to avoid excessive locking if it's not necessary and certainly avoid excessive communication between the driver and the emulator just for logging. Can anyone clear this up for me? Is there a Right Thing To Do (TM) in this situation? -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas.larsson@REDACTED Mon Sep 26 23:58:00 2011 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Mon, 26 Sep 2011 22:58:00 +0100 (BST) Subject: [erlang-questions] getting type info at runtime In-Reply-To: Message-ID: Hello! Regarding types in .beam files: That information is only part of the abtract code, so dialyzer, hipe and all other tools need the abtract code or the source code to do its magic. The reason why it is stripped out is because the .beam files should be as small as possible by default and also partly because you cannot trust a typespec to be correct. Regarding instrumenting, I think that would be really great to have. There are already tools which can achieve something like this (meck for instance), but less obtrusive tools are always good :) (Meck renames a module to M_meck and then generates a new module which calls the mocked one in some places and the mocking code in others. This method could be used for live instrumentation, but it would not be desirable in a live system as it is neither atomic, not very maintainable...) Describing where you want to put your instrumentation is harder to figure out though and to date no one really has managed to write a good wrapper to match specs. The best one (imo) is the dbg:fun2ms function Eg: dbg:fun2ms(fun([A,B,_]) when element(1,A) == record,element(2,B) == dict -> message(caller()),exception_trace(); ([_,B]) when B == debug -> return_trace() end). Though I'm not sure how much clearer than the below it is: [{['$1','$2','_'], [{'==',{element,1,'$1'},record}, {'==',{element,2,'$2'},dict}], [{message,{caller}},{exception_trace}]}, {['_','$1'],[{'==','$1',debug}],[{return_trace}]}] Using the improved -specs that will come in R14B04 it would be possible to write a spec which describes the same thing. Keeping things in Erlang has the advantage of it being easier to build tools around it though. Lukas ----- Original Message ----- From: "Tim Watson" To: "Motiejus Jak?tys" Cc: martynas@REDACTED, erlang-questions@REDACTED, cirka@REDACTED Sent: Monday, September 26, 2011 10:13:23 PM Subject: Re: [erlang-questions] getting type info at runtime 2011/9/26 Motiejus Jak?tys < desired.mta@REDACTED > On Mon, Sep 26, 2011 at 17:28, Jonas Boberg < jonas.boberg@REDACTED > wrote: > Hi, > > We solve this by having a parse transformation that adds additional > functions to the module (at compile time). More specifically, we use > this to generate functions that serialize and deserialize json objects > to records, including type verification based on the spec annotations. Hi, is this hosted somewhere? Motiejus Jak?tys Hi all, Thanks for the responses - I'll answer a few questions and clarify. I should have worded this question "getting type info from beam code compiled without debug_info" instead. I'd like to be able to get the -spec info for several reasons: 1. Discovery - think hoogle for Erlang packages 2. Static Analysis 3. Instrumentation The idea of searching based on type information has come up on the list before. Item (2) is obvious, and tools like dialyzer and QuickCheck/PropEr are already using this type information to great effect. I just haven't figured out how to unpick what they're doing properly. As for (3), there are amazing capabilities built into erlang thanks to the tracing capabilities in erts. I'm using these for diagnostic purposes, but I don't see them as something that I'd like to have *always on* in my production code. There are limitations to using a tracer process and of course writing to a file or socket is safe(er) but there are limited guarantees about what gets delivered and ordering and so on. Tracing feels to me like something I turn on for diagnostic purposes. The kind of instrumentation I'm after is more akin to something AOP, where I want to get the build system to do some work for me but keep a tight control of what code gets generated as it is now a proper part of my application. The kind of post processing on the beam code I'd like to do might insert things like logging statements or increment lightweight performance counters. I'd like to do this as part of my build, so that I can (a) pretty print the beam (using erl_pp) if I want to see what's been added (b) make sure that the instrumented code is what gets unit/integration/load tested In order to apply these modifications to my own source code, I can use a parse_transform and that's easy enough. I'd also like to have a declarative approach for defining where the instrumentation/code-weaving takes place, and type signatures seem like a good candidate for this. Let's say I want to put a performance counter around calls to any function in any module which has a name ending in _db, taking a #'myapp.query'{} record and an opaque connection_handle() type. I might define the target operations (in my own source code which is to be instrumented) by saying something like: around('call(*_log:*(#'myapp.query'{}, connection_handle()))', Handler) %% where Handler is an {M,F,A} or callback module conforming to a simple API Now I suppose this is somewhat akin to a match specification {'_', '_', ...} with some fancy stuff to handle the arguments such as {['$1', '$1', '$1'], [{is_record, '$1', 'myapp.query', record_info(size, 'myapp.query')}], .... So maybe match specifications would make a better language, but they are a bit hard to follow (no offense to the good OTP folks intended) and type specifications with wildcards for naming (modules and functions) seem a bit simpler to me, at least for my purposes. Now I *might* want to put a performance counter and/or logging statement (or whatever) around calls to _someone else's code_ such as calls to mochiweb_request or whatever, which I may not have the sources for and which may not have been compiled with debug_info (or used the "abstract_code" parse transform that's floating around). In those cases, it would be really nice if the compiler would keep hold of the type signatures in the beam, even when the abstract code isn't present. Of course in the meanwhile, if the abstract code is present I can get it out using beam_lib, but the clever people who wrote dialyzer/QuickCheck/PropEr must have solved this already. So I'm wondering whether 1. The type introspection code it could be "unpicked" from these tools easily - I will need pointing in the right direction though! 2. The compiler should preserve type information in the generated beam, and if so is this in any roadmap or is a contribution the way to go? 3. Have I got the wrong impression of the trace facility or not I see some kind of "type server" in both PropEr and Dialyzer and I'm pretty sure if this was abstracted out into a separate component that it could nicely form the core of a erlang-search web service or something of that ilk. I'd also be interested in this code you've mentioned Jonas, if it is open source. For matching type specifications against modules/functions in other people's code however, a parse_transform isn't what I need. Please do call out better approaches if you have them, but also bare in mind this is a mainly fun/educational exercise at the moment! Cheers, Tim _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From watson.timothy@REDACTED Tue Sep 27 00:57:09 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 26 Sep 2011 23:57:09 +0100 Subject: [erlang-questions] getting type info at runtime In-Reply-To: References: Message-ID: Hi Lukas, Thanks for this - some very helpful suggestions. > Regarding types in .beam files: That information is only part of the > abtract code, so dialyzer, hipe and all other tools need the abtract code or > the source code to do its magic. The reason why it is stripped out is > because the .beam files should be as small as possible by default and also > partly because you cannot trust a typespec to be correct. > Yes I can understand wanting to keep the beam small, but it would be nice to have the option of the compiler inserting the type specs and of course just because the type spec is in the beam file, doesn't mean it has to be brought into memory when loading the beam. :) If abstract_code is the only sane way to go then I can simply push out warnings for beams that don't have it and exclude those from the introspection. Describing where you want to put your instrumentation is harder to figure > out though and to date no one really has managed to write a good wrapper to > match specs. The best one (imo) is the dbg:fun2ms function Eg: > > Yes I've used this before. As you say though, it's not a great deal clearer than the match spec itself. > > Using the improved -specs that will come in R14B04 it would be possible to > write a spec which describes the same thing. Keeping things in Erlang has > the advantage of it being easier to build tools around it though. > > I haven't seen these, so I'll have a poke around and see how they work. In terms of actually writing the specification, I think you're right that having something pure erlang is good. I was pondering the approach I used for https://github.com/hyperthunk/hamcrest-erlang of using various functions as sugar. Ultimately though, I'd like to be able to put the instrumentation config used by the build into a config file (such as rebar.config) so it will probably end up either looking like a match spec or being a string that gets parsed. -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue Sep 27 02:00:08 2011 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 27 Sep 2011 02:00:08 +0200 Subject: [erlang-questions] Dialyzer and erlang:hibernate/3 Message-ID: <4E811208.9080200@dev-extend.eu> Hello! I'm having an issue with Dialyzer. I have a bunch of functions that call each other one after another and they all return the atom 'ok'. On one case I have a function clause that calls erlang:hibernate/3. When resuming from hibernate Erlang calls another function from that module that also returns 'ok'. So Dialyzer complains that erlang:hibernate/3 will never return, and obviously it doesn't. But does that mean that I must add the type no_return() to the returned values of all my functions? This doesn't feel too natural to me because hibernating is pretty much transparent and my code only returns the atom 'ok' at the end when unwrapping the stacktrace. Has anyone had this issue with Dialyzer before and is there a good way to handle it? Shouldn't Dialyzer have a special case to handle erlang:hibernate/3 to avoid this kind of issues? Would be glad to hear your thoughts about this. -- Lo?c Hoguin Dev:Extend From zabrane3@REDACTED Tue Sep 27 06:48:22 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 27 Sep 2011 06:48:22 +0200 Subject: [erlang-questions] Fwd: Erlang and paypal integration for web application References: Message-ID: I found mod_paypal a bit tighted to Zotonic. Any other pointer guys? Regards, Zabrane Begin forwarded message: > From: Zabrane Mickael > Date: September 26, 2011 6:37:00 PM GMT+02:00 > To: Atilla Erd?di > Cc: Erlang Questions > Subject: Re: [erlang-questions] Erlang and paypal integration for web application > >> There is one for Zotonic: http://modules.zotonic.com/page/341/mod-paypal > > Thanks Attila ;-) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kkcmpathi@REDACTED Tue Sep 27 07:41:08 2011 From: kkcmpathi@REDACTED (chamin madura) Date: Tue, 27 Sep 2011 11:11:08 +0530 Subject: [erlang-questions] Open Erlang SMPP library using problem? Message-ID: Hi all , I am trying to use OSERL ,according to it's user guide( http://oserl.sourceforge.net/oserl/index.html). try test it with submit_esme.erl $ erlc -W -I ../../include -pz ../../ebin submit_esme.erl arise many warnings ,see this is out put, ./submit_esme.erl:4: Warning: undefined callback function handle_accept/3 (behaviour 'gen_esme') ./submit_esme.erl:4: Warning: undefined callback function handle_alert_notification/2 (behaviour 'gen_esme') ./submit_esme.erl:4: Warning: undefined callback function handle_closed/2 (behaviour 'gen_esme') ./submit_esme.erl:4: Warning: undefined callback function handle_data_sm/3 (behaviour 'gen_esme') ./submit_esme.erl:4: Warning: undefined callback function handle_deliver_sm/3 (behaviour 'gen_esme') ./submit_esme.erl:4: Warning: undefined callback function handle_outbind/2 (behaviour 'gen_esme') ./submit_esme.erl:4: Warning: undefined callback function handle_req/4 (behaviour 'gen_esme') ./submit_esme.erl:4: Warning: undefined callback function handle_resp/3 (behaviour 'gen_esme') ./submit_esme.erl:4: Warning: undefined callback function start_link/4 (behaviour 'gen_esme') ./submit_esme.erl:165: Warning: variable 'From' is unused ./submit_esme.erl:165: Warning: variable 'Pdu' is unused ./submit_esme.erl:165: Warning: variable 'Session' is unused ./submit_esme.erl:189: Warning: variable 'From' is unused ./submit_esme.erl:189: Warning: variable 'Pdu' is unused ./submit_esme.erl:189: Warning: variable 'Session' is unused ./submit_esme.erl:224: Warning: variable 'From' is unused ./submit_esme.erl:224: Warning: variable 'Pdu' is unused ./submit_esme.erl:224: Warning: variable 'Session' is unused ./submit_esme.erl:257: Warning: variable 'From' is unused ./submit_esme.erl:257: Warning: variable 'Pdu' is unused ./submit_esme.erl:257: Warning: variable 'Session' is unused ./submit_esme.erl:306: Warning: variable 'From' is unused ./submit_esme.erl:324: Warning: integer/1 obsolete ./submit_esme.erl:353: Warning: variable 'Request' is unused ./submit_esme.erl:377: Warning: variable 'Info' is unused ./submit_esme.erl:392: Warning: variable 'S' is unused ./submit_esme.erl:394: Warning: variable 'Reason' is unused ./submit_esme.erl:409: Warning: variable 'Extra' is unused ./submit_esme.erl:409: Warning: variable 'OldVsn' is unused ./submit_esme.erl:443: Warning: function read_string/1 is unused then I try to run it according to user guide (after add compatible code path), 1>submit_esme:start_link(). ********************** output is =ERROR REPORT==== 27-Sep-2011::11:07:43 === ** Generic server <0.52.0> terminating ** Last message in was {'EXIT',<0.50.0>, {badarg, [{ets,insert, [cl_queue_tab, {<0.50.0>,<0.52.0>,<0.53.0>}]}, {cl_queue_tab,insert,2}, {gen_esme,init,1}, {gen_server,init_it,6}, {proc_lib,init_p_do_apply,3}]}} ** When Server state == {st,<0.50.0>,undefined,0,0,{1317,101863,441293}} ** Reason for termination == ** {badarg,[{ets,insert,[cl_queue_tab,{<0.50.0>,<0.52.0>,<0.53.0>}]}, {cl_queue_tab,insert,2}, {gen_esme,init,1}, {gen_server,init_it,6}, {proc_lib,init_p_do_apply,3}]} =ERROR REPORT==== 27-Sep-2011::11:07:43 === Error in process <0.53.0> with exit value: {badarg,[{ets,delete,[cl_queue_tab,<0.50.0>]},{cl_queue_tab,monitor_loop,4}]} ** exception exit: badarg in function ets:insert/2 called as ets:insert(cl_queue_tab,{<0.50.0>,<0.52.0>,<0.53.0>}) in call from cl_queue_tab:insert/2 in call from gen_esme:init/1 in call from gen_server:init_it/6 in call from proc_lib:init_p_do_apply/3 please,consider it. Regards, Chamin. -------------- next part -------------- An HTML attachment was scrubbed... URL: From aleksey@REDACTED Tue Sep 27 10:04:20 2011 From: aleksey@REDACTED (Aleksey Yeschenko) Date: Tue, 27 Sep 2011 11:04:20 +0300 Subject: [erlang-questions] Open Erlang SMPP library using problem? In-Reply-To: References: Message-ID: Try application:start(common_lib) first. -- Aleksey 2011/9/27 chamin madura > Hi all , > I am trying to use OSERL ,according to it's user guide( > http://oserl.sourceforge.net/oserl/index.html). > try test it with submit_esme.erl > > > > $ erlc -W -I ../../include -pz ../../ebin submit_esme.erl > > arise many warnings ,see this is out put, > > ./submit_esme.erl:4: Warning: undefined callback function handle_accept/3 (behaviour 'gen_esme') > > > ./submit_esme.erl:4: Warning: undefined callback function handle_alert_notification/2 (behaviour 'gen_esme') > ./submit_esme.erl:4: Warning: undefined callback function handle_closed/2 (behaviour 'gen_esme') > > > ./submit_esme.erl:4: Warning: undefined callback function handle_data_sm/3 (behaviour 'gen_esme') > ./submit_esme.erl:4: Warning: undefined callback function handle_deliver_sm/3 (behaviour 'gen_esme') > > > ./submit_esme.erl:4: Warning: undefined callback function handle_outbind/2 (behaviour 'gen_esme') > ./submit_esme.erl:4: Warning: undefined callback function handle_req/4 (behaviour 'gen_esme') > ./submit_esme.erl:4: Warning: undefined callback function handle_resp/3 (behaviour 'gen_esme') > > > ./submit_esme.erl:4: Warning: undefined callback function start_link/4 (behaviour 'gen_esme') > ./submit_esme.erl:165: Warning: variable 'From' is unused > ./submit_esme.erl:165: Warning: variable 'Pdu' is unused > > > ./submit_esme.erl:165: Warning: variable 'Session' is unused > ./submit_esme.erl:189: Warning: variable 'From' is unused > ./submit_esme.erl:189: Warning: variable 'Pdu' is unused > ./submit_esme.erl:189: Warning: variable 'Session' is unused > > > ./submit_esme.erl:224: Warning: variable 'From' is unused > ./submit_esme.erl:224: Warning: variable 'Pdu' is unused > ./submit_esme.erl:224: Warning: variable 'Session' is unused > ./submit_esme.erl:257: Warning: variable 'From' is unused > > > ./submit_esme.erl:257: Warning: variable 'Pdu' is unused > ./submit_esme.erl:257: Warning: variable 'Session' is unused > ./submit_esme.erl:306: Warning: variable 'From' is unused > ./submit_esme.erl:324: Warning: integer/1 obsolete > > > ./submit_esme.erl:353: Warning: variable 'Request' is unused > ./submit_esme.erl:377: Warning: variable 'Info' is unused > ./submit_esme.erl:392: Warning: variable 'S' is unused > ./submit_esme.erl:394: Warning: variable 'Reason' is unused > > > ./submit_esme.erl:409: Warning: variable 'Extra' is unused > ./submit_esme.erl:409: Warning: variable 'OldVsn' is unused > ./submit_esme.erl:443: Warning: function read_string/1 is unused > > then I try to run it according to user guide (after add compatible code path), > > > 1>submit_esme:start_link(). > ********************** output is > =ERROR REPORT==== 27-Sep-2011::11:07:43 === > ** Generic server <0.52.0> terminating > ** Last message in was {'EXIT',<0.50.0>, > > > {badarg, > [{ets,insert, > [cl_queue_tab, > {<0.50.0>,<0.52.0>,<0.53.0>}]}, > > > {cl_queue_tab,insert,2}, > {gen_esme,init,1}, > {gen_server,init_it,6}, > {proc_lib,init_p_do_apply,3}]}} > > > ** When Server state == {st,<0.50.0>,undefined,0,0,{1317,101863,441293}} > ** Reason for termination == > ** {badarg,[{ets,insert,[cl_queue_tab,{<0.50.0>,<0.52.0>,<0.53.0>}]}, > {cl_queue_tab,insert,2}, > > > {gen_esme,init,1}, > {gen_server,init_it,6}, > {proc_lib,init_p_do_apply,3}]} > > =ERROR REPORT==== 27-Sep-2011::11:07:43 === > Error in process <0.53.0> with exit value: {badarg,[{ets,delete,[cl_queue_tab,<0.50.0>]},{cl_queue_tab,monitor_loop,4}]} > > > ** exception exit: badarg > in function ets:insert/2 > called as ets:insert(cl_queue_tab,{<0.50.0>,<0.52.0>,<0.53.0>}) > in call from cl_queue_tab:insert/2 > in call from gen_esme:init/1 > > > in call from gen_server:init_it/6 > in call from proc_lib:init_p_do_apply/3 > > > please,consider it. > > Regards, > Chamin. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Tue Sep 27 11:47:22 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 27 Sep 2011 11:47:22 +0200 Subject: [erlang-questions] Right direction ? In-Reply-To: <7AA05035-2AB7-4CB5-8FF0-A29A6D7455FF@nexttolast.com> References: <7AA05035-2AB7-4CB5-8FF0-A29A6D7455FF@nexttolast.com> Message-ID: On Wed, Sep 21, 2011 at 7:40 PM, David Goehrig wrote: > Last night I began hacking on code.erl, code_server.erl, and looking to extend load_file(Module :: atom()) to include a load_file(Module, Url) function that would look for the module at the associated URL rather than searching for the file via abs path. It would then compare the sha256 hash of the file against the copy in cache and bail if they are different (no hash in cache adds it and uses as the baseline). > > I was wondering if there was a good way to verify that a .beam has not been modified since last load. > > For example: > > -module(my_mod) > -require(daves_mod,"http://erlang.dloh.org/") > > Could then look for http://erlang.dloh.org/daves_mod.erl and download and compile a local beam. Once I have that beam, I can just load from cache, but what happens if the beam is modified after compilation? I don't really understand. The only (legal) way to modify the beam is to change the source and recompile. I think you have to decide exactly what the semantics of require are. There are several possible meanings: a) We check the require targets *before* compilation with a separate program ie image a program: > check_dependencies *.erl This parses all erlang files (*.erl) extract the require information then it check the cache to see if it has all the necessary files Even this program could have two modes: a1) always check with the origonal source for new versions a2) check once every N days (or minutes or hours or something) b) we check at compile time c) we check at usage time. The first time we call daves_module and find it has not been loaded we check the cache and so on a) represents an early binding scenario, c) very-late binding If you are in a development scenario I'd favor a) because have code changing rapidly under my feet would worry me. (actually a) is the easiest to implement If I am in a deployment scenario I might choose c) I might even want to *push* changes rather than reply on polling or some other way of finding out that the code has changed. The point is that you have to have a clear idea of which of these particular problems you are solving. Doing a really good job on the a) scenario interest me - I'd just like to type "make" and be told - "foo123.erl on http:/..../ has changed from the cached version, do you want to update?" .... Only a) fits nicely with unit testing/type checking etc. delayering to load time makes testing difficult. If things can change under your feet without you knowing, life might become difficult. > > The other thing I would like to add is DNS TXT records that could be published sha256 hashes of each source module. > > http://erlang.dloh.org/daves_mod.erl 45663AFDA.... > > Adding a > > -signature("http://erlang.dlog.org/daves_mod.erl","45663AFDA....") > > Would allow a 3 part verification of the source: > > 1.) I can compute the source has the right hash > 2.) I can look up that the module has the same published signature > 3.) I can verify against the original at the specified URL > > In this scenario it is not enough to modify the source and rehash, nor enough to replace the upsteam file, but also replace the DNS entry without anyone noticing. > > Thoughts? Good stuff - needs some thought though. I was thinking of signing/validating the source with an RSA public/private keypair. I'd like to see this as part of the build process, if I did "make" I might like to see: $ make module foo123.erl is up to date. Written by joearms *validated* module bar23.erl has a newer version module bingo23.erl is up to date written by cleverperson *untrusted* ... etc. /Joe > > Dave > -=-=- dave@REDACTED -=-=- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From fred.hebert@REDACTED Tue Sep 27 17:37:42 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Tue, 27 Sep 2011 11:37:42 -0400 Subject: [erlang-questions] The importance of Basic Unicode Understanding in Erlang Message-ID: <4B939B9C-B0EF-4CCF-8F47-AE543F425BF1@erlang-solutions.com> Hi there everyone. I've recently done some work where, due to circumstances, unicode woes were had by everyone. It kind of got me by surprise, and I figure that if it hasn't bitten you yet, it might sooner or later. As such, I published a blog post on the issue yesterday: http://ferd.ca/will-the-real-unicode-wrangler-please-stand-up.html It's not going into advanced details, but it's about some very simple warnings. When dealing with strings, all the binary_to_list, list_to_binary, iolist_to_binary functions are to be avoided. The length function is no longer safe and neither are comparison operators. When using io:format, "~s" is no longer what we want all the time, but rather "~ts", etc. This partial support is weirder for countries and languages that depend on some unicode characters for their everyday use when the Erlang source files are always assumed to be latin-1, although the Erlang shell is fine with unicode. I'm no expert in i18n affairs, but we currently have no standard library way to do basic operations such as calculating the length of strings, splitting binaries or items by clusters, ways to perform normalisations, doing uppercase/lowercase/titlecase strings, comparing strings, reversing them, etc. We have to rely on external libraries. While these libraries are not bad, it is obvious that standard implementations are usually nicer for everyone. I'm also in no position to force people to implement libraries are need when I'm offering no money incentive myself. As such, I felt like having (yet another) discussion of the issues of unicode, and what we think would be the ideal way to solve the problem within Erlang. Any opinion? -- Fred H?bert http://www.erlang-solutions.com From daniel@REDACTED Tue Sep 27 17:53:37 2011 From: daniel@REDACTED (Daniel Luna) Date: Tue, 27 Sep 2011 11:53:37 -0400 Subject: [erlang-questions] getting type info at runtime In-Reply-To: References: Message-ID: Many years ago I wrote a runtime -spec checker, reading the data from abstract code (if I remember correctly). The checker will convert input to any function with a -spec declaration to a type, and validate that type against the -spec. This will then be done recursively for any function call originating from the checked function call. (As long as all modules in that chain are compiled with debug_info (since that is the only way to get access to the types).) All of this is done loading a spec checked version of the code, and rewriting all function calls with a magical prefix. Calls to unknown modules are then intercepted (by using a replacement error handler), and loads a modified version of the called module. Etc. The checking is all done in core erlang, which unfortunately ruins readability quite a bit. You can use and copy whatever you want from that project (https://github.com/dLuna/Erlang-type-checker). There are a few caveats: 1. I have not tested this in a long time, and I'm heavily depending on Dialyzer for the type parsing. If Dialyzer has been changed, my code might break. 2. Something changed in core erlang a few years back, so the code is broken as it stands today. Should be an easy fix for someone wiling to learn a bit about core erlang (or wants to do some io:format programming). I have marked the place of offending code. 3. Type checking using my current implementation is *slow*. The reason for this is that I type convert the full argument even if only a minor piece is necessary. (e.g. given the type [any()] my code would still convert a list with a thousand deeply nested elements into its fully expanded type) The fix for this is easy, but time consuming. Anyway. Even if it turns out that the core erlang parts are hard to reuse, the code server parts should be straight forward to copy. If anyone wants to know more detail just ask and I'll see what I can remember. Cheers, Daniel ps. Assume that it's under an MIT license, and I'll add the actual license statement later this week. On 26 September 2011 18:57, Tim Watson wrote: > Hi Lukas, > Thanks for this - some very helpful suggestions. > >> >> Regarding types in .beam files: That information is only part of the >> abtract code, so dialyzer, hipe and all other tools need the abtract code or >> the source code to do its magic. The reason why it is stripped out is >> because the .beam files should be as small as possible by default and also >> partly because you cannot trust a typespec to be correct. > > Yes I can understand wanting to keep the beam small, but it would be nice to > have the option of the compiler inserting the type specs and of course just > because the type spec is in the beam file, doesn't mean it has to be brought > into memory when loading the beam. :) > If abstract_code is the only sane way to go then I can simply push out > warnings for beams that don't have it and exclude those from the > introspection. >> >> Describing where you want to put your instrumentation is harder to figure >> out though and to date no one really has managed to write a good wrapper to >> match specs. The best one (imo) is the dbg:fun2ms function Eg: >> > > Yes I've used this before. As you say though, it's not a great deal clearer > than the match spec itself. > >> >> Using the improved -specs that will come in R14B04 it would be possible to >> write a spec which describes the same thing. Keeping things in Erlang has >> the advantage of it being easier to build tools around it though. >> > > I haven't seen these, so I'll have a poke around and see how they work. In > terms of actually writing the specification, I think you're right that > having something pure erlang is good. I was pondering the approach I used > for?https://github.com/hyperthunk/hamcrest-erlang of using various functions > as sugar. Ultimately though, I'd like to be able to put the instrumentation > config used by the build into a config file (such as rebar.config) so it > will probably end up either looking like a match spec or being a string that > gets parsed. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From watson.timothy@REDACTED Tue Sep 27 18:30:55 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 27 Sep 2011 17:30:55 +0100 Subject: [erlang-questions] getting type info at runtime In-Reply-To: References: Message-ID: On 27 September 2011 16:53, Daniel Luna wrote: > The checking is all done in core erlang, which unfortunately ruins > readability quite a bit. > > I've been meaning to learn core properly anyway. > You can use and copy whatever you want from that project > (https://github.com/dLuna/Erlang-type-checker). > > Awesome - thank you very much. > There are a few caveats: > > 1. I have not tested this in a long time, and I'm heavily depending on > Dialyzer for the type parsing. If Dialyzer has been changed, my code > might break. > > Ok that's perfect - I've struggled to get my head into which parts of Dialyzer can be reused in this way, so just having this as a head start is really helpful! > 2. Something changed in core erlang a few years back, so the code is > broken as it stands today. Should be an easy fix for someone wiling > to learn a bit about core erlang (or wants to do some io:format > programming). I have marked the place of offending code. > > Ok thanks - again, a good learning exercise. > 3. Type checking using my current implementation is *slow*. The > reason for this is that I type convert the full argument even if only > a minor piece is necessary. (e.g. given the type [any()] my code would > still convert a list with a thousand deeply nested elements into its > fully expanded type) The fix for this is easy, but time consuming. > > I'm not planning on doing real type checking anyway - just seeing whether a specification for a function matches the -spec for any known code without any widening conversion. For instrumentation purposes this is very easy, for search it's a bit harder. > Anyway. Even if it turns out that the core erlang parts are hard to > reuse, the code server parts should be straight forward to copy. If > anyone wants to know more detail just ask and I'll see what I can > remember. > > Brilliant - this should get me started playing with instrumentation and hopefully I can glue together the code server parts along with webmachine and a decent search index to make a hoogle-esque search service. Someone with a visually creative mind might then be willing/able to come along and actually make a reasonable looking front end for it then. ps. Assume that it's under an MIT license, and I'll add the actual > license statement later this week. > > Cool. I'll most likely license the instrumentation and code-search projects in the same way. Cheers! Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: From ebegumisa@REDACTED Tue Sep 27 18:31:02 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Wed, 28 Sep 2011 02:31:02 +1000 Subject: [erlang-questions] Right direction ? In-Reply-To: References: <7AA05035-2AB7-4CB5-8FF0-A29A6D7455FF@nexttolast.com> Message-ID: Hi there, Late on this and the preceding thread, but I've been *very* interested in something like this for a while now and raised it once... http://erlang.org/pipermail/erlang-questions/2011-May/058871.html http://erlang.org/pipermail/erlang-questions/2011-May/058904.html I personally would really like (c) below. Mainly because it would render packaging tools "less" useful and make deployment really easy. I think one can draw inspiration from others like Mozilla XULRunner for both locating code and how they make use of web-style deployment... * Support for hardcoded locations based on URIs... -location("http://erlang.org/foo.erl"). -location("file://./foo.erl"). * Support for centralising locations... -location("erlang://my_otp_app/foo.erl") where the actual location is picked up from "erlang.manifest" that maps "my_app" to some ebin dir similar to "chrome.manifest" in Mozilla. * Support for custom file streams... -location("riak://my_server/foo.erl") where loader would expect "riak.manifest" to tell it how to fetch the file (possibly an {M,F,A} entry) * Enforce security restrictions for unsigned code On that last point: I wonder for example how possible it would be to restrict code loaded from remote locations to the "same-source" rule. Also (probably more tricky), how difficult it would be enforce a rule that code coming from remote unsigned sources cannot have access to the local filesystem or erlang ports. Mozilla makes two broad categories for code loaded from URIs -- chrome:// (call it erlang:// for our case) that has access to everything, and everything else (http://, file://, etc) that mustn't access the (full) local file system or external processes. This would be nice to have, because code signing can get tedious. - Edmond - On Tue, 27 Sep 2011 19:47:22 +1000, Joe Armstrong wrote: > On Wed, Sep 21, 2011 at 7:40 PM, David Goehrig > wrote: >> Last night I began hacking on code.erl, code_server.erl, and looking to >> extend load_file(Module :: atom()) to include a load_file(Module, Url) >> function that would look for the module at the associated URL rather >> than searching for the file via abs path. It would then compare the >> sha256 hash of the file against the copy in cache and bail if they are >> different (no hash in cache adds it and uses as the baseline). >> >> I was wondering if there was a good way to verify that a .beam has not >> been modified since last load. >> >> For example: >> >> -module(my_mod) >> -require(daves_mod,"http://erlang.dloh.org/") >> >> Could then look for http://erlang.dloh.org/daves_mod.erl and download >> and compile a local beam. Once I have that beam, I can just load from >> cache, but what happens if the beam is modified after compilation? > > I don't really understand. The only (legal) way to modify the beam > is to change the source and recompile. I think you have to > decide exactly what the semantics of require are. There are several > possible meanings: > > a) We check the require targets *before* compilation with > a separate program > > ie image a program: > > > check_dependencies *.erl > > This parses all erlang files (*.erl) extract the require > information > then it check the cache to see if it has all the necessary files > > Even this program could have two modes: > a1) always check with the origonal source for new versions > a2) check once every N days (or minutes or hours or > something) > > b) we check at compile time > > c) we check at usage time. The first time we call daves_module and > find > it has not been loaded we check the cache and so on > > a) represents an early binding scenario, c) very-late binding > > If you are in a development scenario I'd favor a) because have code > changing rapidly under my feet would worry me. (actually a) is the > easiest to implement > > If I am in a deployment scenario I might choose c) I might even want to > *push* changes rather than reply on polling or some other way of > finding out that the code has changed. > > The point is that you have to have a clear idea of which of these > particular problems you are solving. > > Doing a really good job on the a) scenario interest me - I'd just like > to type "make" and be told - "foo123.erl on http:/..../ has changed from > the > cached version, do you want to update?" .... > > Only a) fits nicely with unit testing/type checking etc. delayering > to load time makes testing difficult. If things can change under your > feet > without you knowing, life might become difficult. > >> >> The other thing I would like to add is DNS TXT records that could be >> published sha256 hashes of each source module. >> >> http://erlang.dloh.org/daves_mod.erl 45663AFDA.... >> >> Adding a >> >> -signature("http://erlang.dlog.org/daves_mod.erl","45663AFDA....") >> >> Would allow a 3 part verification of the source: >> >> 1.) I can compute the source has the right hash >> 2.) I can look up that the module has the same published signature >> 3.) I can verify against the original at the specified URL >> >> In this scenario it is not enough to modify the source and rehash, nor >> enough to replace the upsteam file, but also replace the DNS entry >> without anyone noticing. >> >> Thoughts? > > Good stuff - needs some thought though. I was thinking of > signing/validating > the source with an RSA public/private keypair. > > I'd like to see this as part of the build process, if I did "make" I > might like to see: > > $ make > module foo123.erl is up to date. Written by joearms *validated* > module bar23.erl has a newer version > module bingo23.erl is up to date written by cleverperson *untrusted* > ... > etc. > > > /Joe > > > > >> >> Dave >> -=-=- dave@REDACTED -=-=- >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From matthias@REDACTED Tue Sep 27 19:03:39 2011 From: matthias@REDACTED (Matthias Lang) Date: Tue, 27 Sep 2011 19:03:39 +0200 Subject: [erlang-questions] Erlang on ARM9? In-Reply-To: <0b58b591-dac9-4b51-8835-b941f5d6229a@knuth> References: <75F7CC13-4F65-4F07-9E9E-2A2C2CF08E73@rogvall.se> <0b58b591-dac9-4b51-8835-b941f5d6229a@knuth> Message-ID: <20110927170338.GA3699@corelatus.se> On Monday, September 26, Gustav Simonsson wrote: > The trapexit guide assumes you have a proper cross compilation toolchain. Yes. And, if you want to compile R14B (i.e. the current Erlang version), it's best to ignore the trap exit guide, it's intended for R13B03 and earlier. (I wrote the trap exit guide for R10B, based on things I'd picked up since R7 or something. A bit after R13B04 came out, I changed the wiki page to say "don't bother reading this if you're using R13B04 or later" near the top. Maybe it's time for me to retire the whole page, R14B is firmly established.) > You can generate a ARM9 toolchain using OpenEmbedded either by following > one of the guides online for creating a OE toolchain or by configuring OE > for a ARM9 machine and building some app or system image with it, as that > will also generate a ARM9 toolchain. Another popular (free) toolchain is DENX ELDK http://www.denx.de/wiki/DULG/ELDK I haven't used OpenEmbedded, so I don't know if it's better or worse; just mentioning it as another alternative. Matt From ebegumisa@REDACTED Tue Sep 27 19:31:38 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Wed, 28 Sep 2011 03:31:38 +1000 Subject: [erlang-questions] Right direction ? In-Reply-To: References: <7AA05035-2AB7-4CB5-8FF0-A29A6D7455FF@nexttolast.com> Message-ID: Oh, two more things... On the question of to re-fetch or not re-fetch: I think pushing would work well for signed code, but polling would be better for unsigned code. Again, I would look to what browser technologies do - namely how extensions in Firefox are reloaded. You could have a special text file in the remote ebin directory that lists module updates, poll it every week, then have the local system inform the administrator (through SASL or something) that there's an upgrade worth considering (nothing is updated without consent). The local system would need to know if reloading a module is okay or if an app upgrade is needed. Secondly, you say compile .erl files. Are there any objections to fetching beams? Why not have -require("http://erlang.org/foo.erl") first check if there's a corresponding foo.beam at that location and only compile if there is not? - Edmond - On Wed, 28 Sep 2011 02:31:02 +1000, Edmond Begumisa wrote: > Hi there, > > Late on this and the preceding thread, but I've been *very* interested > in something like this for a while now and raised it once... > > http://erlang.org/pipermail/erlang-questions/2011-May/058871.html > http://erlang.org/pipermail/erlang-questions/2011-May/058904.html > > I personally would really like (c) below. Mainly because it would render > packaging tools "less" useful and make deployment really easy. > > I think one can draw inspiration from others like Mozilla XULRunner for > both locating code and how they make use of web-style deployment... > > * Support for hardcoded locations based on URIs... > -location("http://erlang.org/foo.erl"). > -location("file://./foo.erl"). > > * Support for centralising locations... > -location("erlang://my_otp_app/foo.erl") > where the actual location is picked up from "erlang.manifest" that > maps "my_app" to some ebin dir similar to "chrome.manifest" in Mozilla. > > * Support for custom file streams... > -location("riak://my_server/foo.erl") > where loader would expect "riak.manifest" to tell it how to fetch > the file (possibly an {M,F,A} entry) > > * Enforce security restrictions for unsigned code > > On that last point: I wonder for example how possible it would be to > restrict code loaded from remote locations to the "same-source" rule. > Also (probably more tricky), how difficult it would be enforce a rule > that code coming from remote unsigned sources cannot have access to the > local filesystem or erlang ports. > > Mozilla makes two broad categories for code loaded from URIs -- > chrome:// (call it erlang:// for our case) that has access to > everything, and everything else (http://, file://, etc) that mustn't > access the (full) local file system or external processes. This would be > nice to have, because code signing can get tedious. > > - Edmond - > > > On Tue, 27 Sep 2011 19:47:22 +1000, Joe Armstrong > wrote: > >> On Wed, Sep 21, 2011 at 7:40 PM, David Goehrig >> wrote: >>> Last night I began hacking on code.erl, code_server.erl, and looking >>> to extend load_file(Module :: atom()) to include a load_file(Module, >>> Url) function that would look for the module at the associated URL >>> rather than searching for the file via abs path. It would then compare >>> the sha256 hash of the file against the copy in cache and bail if they >>> are different (no hash in cache adds it and uses as the baseline). >>> >>> I was wondering if there was a good way to verify that a .beam has not >>> been modified since last load. >>> >>> For example: >>> >>> -module(my_mod) >>> -require(daves_mod,"http://erlang.dloh.org/") >>> >>> Could then look for http://erlang.dloh.org/daves_mod.erl and download >>> and compile a local beam. Once I have that beam, I can just load from >>> cache, but what happens if the beam is modified after compilation? >> >> I don't really understand. The only (legal) way to modify the beam >> is to change the source and recompile. I think you have to >> decide exactly what the semantics of require are. There are several >> possible meanings: >> >> a) We check the require targets *before* compilation with >> a separate program >> >> ie image a program: >> >> > check_dependencies *.erl >> >> This parses all erlang files (*.erl) extract the require >> information >> then it check the cache to see if it has all the necessary files >> >> Even this program could have two modes: >> a1) always check with the origonal source for new versions >> a2) check once every N days (or minutes or hours or >> something) >> >> b) we check at compile time >> >> c) we check at usage time. The first time we call daves_module and >> find >> it has not been loaded we check the cache and so on >> >> a) represents an early binding scenario, c) very-late binding >> >> If you are in a development scenario I'd favor a) because have code >> changing rapidly under my feet would worry me. (actually a) is the >> easiest to implement >> >> If I am in a deployment scenario I might choose c) I might even want to >> *push* changes rather than reply on polling or some other way of >> finding out that the code has changed. >> >> The point is that you have to have a clear idea of which of these >> particular problems you are solving. >> >> Doing a really good job on the a) scenario interest me - I'd just like >> to type "make" and be told - "foo123.erl on http:/..../ has changed >> from the >> cached version, do you want to update?" .... >> >> Only a) fits nicely with unit testing/type checking etc. delayering >> to load time makes testing difficult. If things can change under your >> feet >> without you knowing, life might become difficult. >> >>> >>> The other thing I would like to add is DNS TXT records that could be >>> published sha256 hashes of each source module. >>> >>> http://erlang.dloh.org/daves_mod.erl 45663AFDA.... >>> >>> Adding a >>> >>> -signature("http://erlang.dlog.org/daves_mod.erl","45663AFDA....") >>> >>> Would allow a 3 part verification of the source: >>> >>> 1.) I can compute the source has the right hash >>> 2.) I can look up that the module has the same published signature >>> 3.) I can verify against the original at the specified URL >>> >>> In this scenario it is not enough to modify the source and rehash, nor >>> enough to replace the upsteam file, but also replace the DNS entry >>> without anyone noticing. >>> >>> Thoughts? >> >> Good stuff - needs some thought though. I was thinking of >> signing/validating >> the source with an RSA public/private keypair. >> >> I'd like to see this as part of the build process, if I did "make" I >> might like to see: >> >> $ make >> module foo123.erl is up to date. Written by joearms *validated* >> module bar23.erl has a newer version >> module bingo23.erl is up to date written by cleverperson *untrusted* >> ... >> etc. >> >> >> /Joe >> >> >> >> >>> >>> Dave >>> -=-=- dave@REDACTED -=-=- >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From fred.hebert@REDACTED Tue Sep 27 20:47:54 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Tue, 27 Sep 2011 14:47:54 -0400 Subject: [erlang-questions] The importance of Basic Unicode Understanding in Erlang In-Reply-To: References: <4B939B9C-B0EF-4CCF-8F47-AE543F425BF1@erlang-solutions.com> Message-ID: <9C093AB9-3847-4A23-890E-CABB8E2553F3@erlang-solutions.com> I'm replying to this with erlang-questions in CC. On 2011-09-27, at 13:43 PM, Michael Uvarov wrote: > Hi, > > There is two ways to solve this problem: > use erlang for working with data; > use a native (c or c++) implimentation. > First variant is save, but slow. > Second variant is fast, but if we will choose it we will have problems > with concurency and stability of our application (nifs and drivers can > crash whole vm). > > Why is erlang implimentation slow? > Strings in Erlang are simple lists, 16 byte per char, no sequel form. You're forgetting the binary format, that lets you have any number of bytes per characters. There is currently a match type for utf8, utf16 and utf32, and it can support BOMs if I recall. > They are slow. If you work with advanced unicode algorithms > (normalization, collation, splitting) and i18n (locale-dependible > algorithms), you also needs global store for unidata and CLDR data. > > Second way is to use nifs and ICU. ICU is fast, well-tested. ICU > allows multithreads, you only need to have a copy of resourse for each > thread. > But ICU uses UTF-16, which is not nice formatted in the Erlang shell. > Also code of nif must be very simple and well-tested. > > Also ICU has API for processing dates and formating messages in the > third format (first and second are printf format and erlang's > io:format). But it is closer to gettext application. > > There are few ports ICU for Erlang. Basho has icu4e (nif for basic > functions, no locales). There is Starling driver also. And I am > writing my realization of nifs with locales. > > -- > ? ?????????, > ?????? ??????. > Best regards, > Uvarov Michael I must admit to not being knowledgeable enough for the rest of this post, but I find it instructive. You bring the point of locales, which is also pretty interesting. What's a smart way to handle locales? should they be VM-specific, process-specific? -- Fred H?bert http://www.erlang-solutions.com From steven.charles.davis@REDACTED Tue Sep 27 20:54:10 2011 From: steven.charles.davis@REDACTED (Steve Davis) Date: Tue, 27 Sep 2011 11:54:10 -0700 (PDT) Subject: [erlang-questions] The importance of Basic Unicode Understanding in Erlang In-Reply-To: <4B939B9C-B0EF-4CCF-8F47-AE543F425BF1@erlang-solutions.com> References: <4B939B9C-B0EF-4CCF-8F47-AE543F425BF1@erlang-solutions.com> Message-ID: Have said it before, will say it again. "Strings" are a really bad way of thinking about text. Not just in erlang but in any language/ platform. This is because the encoding is almost universally handled implicitly (but differently). As long as that implicit data remains consistent the problem with not being able to derive the character set encoding remains largely hidden - and so we sweep it under the carpet and bemoan the inconsistencies resulting as being the result of using a platform or language. But obviously, it isn't, as the issue is pandemic and deeply ingrained. /s From dave@REDACTED Wed Sep 28 03:25:42 2011 From: dave@REDACTED (David Goehrig) Date: Tue, 27 Sep 2011 21:25:42 -0400 Subject: [erlang-questions] Right direction ? In-Reply-To: References: <7AA05035-2AB7-4CB5-8FF0-A29A6D7455FF@nexttolast.com> Message-ID: > I don't really understand. The only (legal) way to modify the beam > is to change the source and recompile. I think you have to > decide exactly what the semantics of require are. I'm actually most concerned about the illegal way of modifying a beam: a.) Sysadmin gets clever an runs rsync patching the beam with a diff from one on another server (bad if that server doesn't have the right version) b.) Developer gets clever and uses source control as a deployment mechanism, "git push production master", overriding the version there c.) Nefarious type replaces beams with other beams that have been compiled with compromised security built in having a loader that can check at run time (late binding) > a) We check the require targets *before* compilation with > a separate program > rebar (http://github.com/basho/rebar) already does a pretty good job at this (as long as you list all your dependencies as git repos) and I've been making heavy use of this over the past year. It handles checking out and compiling all the dependencies, and you can specify which specific tags you depend upon. c) we check at usage time. The first time we call daves_module and find it has not been loaded we check the cache and so on > Right now I'm most worried about c.) in the context of lib/kernel/src/code_server.erl: try_load_module(Mod, Dir, Caller, St) -> File = filename:append(Dir, to_path(Mod) ++ objfile_extension()), case erl_prim_loader:get_file(File) of error -> {reply,error,St}; {ok,Binary,FName} -> try_load_module(absname(FName), Mod, Binary, Caller, St) end. Where the file pointed to by FName is now "trusted" and will then be read into memory and passed off to hipe. Part of the problem is I'm also introducing a new risk, because I'm replacing this load bit with code that can read a URL rather than just a filename, so I'd like a way to hook in to check that the file I've downloaded is the same as the signature I have on file in a dets store. > Good stuff - needs some thought though. I was thinking of > signing/validating > the source with an RSA public/private keypair. > I've thought about adding RSA public/private key signing, but that ultimately goes down the route of having a CA to form a trust network, and since CAs tend to prove to be unworthy of trust, I'm wary. Self publishing a RSA public key + signing a SHA hash of the source and putting both in DNS seems like a reasonable way of doing it, but can also be exploited to deny service by DNS cache poisoning. If one were to implement a pub/private key signature check, would it best be done in code_server.erl or somewhere else? That seems to be the first place the files are loaded into memory at run time. Dave -- -=-=-=-=-=-=-=-=-=-=- http://blog.dloh.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave@REDACTED Wed Sep 28 03:28:34 2011 From: dave@REDACTED (David Goehrig) Date: Tue, 27 Sep 2011 21:28:34 -0400 Subject: [erlang-questions] Right direction ? In-Reply-To: References: <7AA05035-2AB7-4CB5-8FF0-A29A6D7455FF@nexttolast.com> Message-ID: Hi Edmond, This is exactly what I'm working on with my patches. But as it is the first time I've poked around these particular parts of the guts of erlang, I am hopeful someone can confirm where the best place to do such dangerous things is :) Dave On Tue, Sep 27, 2011 at 12:31 PM, Edmond Begumisa < ebegumisa@REDACTED> wrote: > Hi there, > > Late on this and the preceding thread, but I've been *very* interested in > something like this for a while now and raised it once... > > http://erlang.org/pipermail/**erlang-questions/2011-May/**058871.html > http://erlang.org/pipermail/**erlang-questions/2011-May/**058904.html > > I personally would really like (c) below. Mainly because it would render > packaging tools "less" useful and make deployment really easy. > > I think one can draw inspiration from others like Mozilla XULRunner for > both locating code and how they make use of web-style deployment... > > * Support for hardcoded locations based on URIs... > -location("http://erlang.org/**foo.erl "). > -location("file://./foo.erl"). > > * Support for centralising locations... > -location("erlang://my_otp_**app/foo.erl") > where the actual location is picked up from "erlang.manifest" that maps > "my_app" to some ebin dir similar to "chrome.manifest" in Mozilla. > > * Support for custom file streams... > -location("riak://my_server/**foo.erl") > where loader would expect "riak.manifest" to tell it how to fetch the > file (possibly an {M,F,A} entry) > > * Enforce security restrictions for unsigned code > > On that last point: I wonder for example how possible it would be to > restrict code loaded from remote locations to the "same-source" rule. Also > (probably more tricky), how difficult it would be enforce a rule that code > coming from remote unsigned sources cannot have access to the local > filesystem or erlang ports. > > Mozilla makes two broad categories for code loaded from URIs -- chrome:// > (call it erlang:// for our case) that has access to everything, and > everything else (http://, file://, etc) that mustn't access the (full) > local file system or external processes. This would be nice to have, because > code signing can get tedious. > > - Edmond - > > > > On Tue, 27 Sep 2011 19:47:22 +1000, Joe Armstrong > wrote: > > On Wed, Sep 21, 2011 at 7:40 PM, David Goehrig >> wrote: >> >>> Last night I began hacking on code.erl, code_server.erl, and looking to >>> extend load_file(Module :: atom()) to include a load_file(Module, Url) >>> function that would look for the module at the associated URL rather than >>> searching for the file via abs path. It would then compare the sha256 hash >>> of the file against the copy in cache and bail if they are different (no >>> hash in cache adds it and uses as the baseline). >>> >>> I was wondering if there was a good way to verify that a .beam has not >>> been modified since last load. >>> >>> For example: >>> >>> -module(my_mod) >>> -require(daves_mod,"http://**erlang.dloh.org/ >>> ") >>> >>> Could then look for http://erlang.dloh.org/daves_**mod.erland download and compile a local beam. Once I have that beam, I can just >>> load from cache, but what happens if the beam is modified after compilation? >>> >> >> I don't really understand. The only (legal) way to modify the beam >> is to change the source and recompile. I think you have to >> decide exactly what the semantics of require are. There are several >> possible meanings: >> >> a) We check the require targets *before* compilation with >> a separate program >> >> ie image a program: >> >> > check_dependencies *.erl >> >> This parses all erlang files (*.erl) extract the require information >> then it check the cache to see if it has all the necessary files >> >> Even this program could have two modes: >> a1) always check with the origonal source for new versions >> a2) check once every N days (or minutes or hours or something) >> >> b) we check at compile time >> >> c) we check at usage time. The first time we call daves_module and >> find >> it has not been loaded we check the cache and so on >> >> a) represents an early binding scenario, c) very-late binding >> >> If you are in a development scenario I'd favor a) because have code >> changing rapidly under my feet would worry me. (actually a) is the >> easiest to implement >> >> If I am in a deployment scenario I might choose c) I might even want to >> *push* changes rather than reply on polling or some other way of >> finding out that the code has changed. >> >> The point is that you have to have a clear idea of which of these >> particular problems you are solving. >> >> Doing a really good job on the a) scenario interest me - I'd just like >> to type "make" and be told - "foo123.erl on http:/..../ has changed from >> the >> cached version, do you want to update?" .... >> >> Only a) fits nicely with unit testing/type checking etc. delayering >> to load time makes testing difficult. If things can change under your feet >> without you knowing, life might become difficult. >> >> >>> The other thing I would like to add is DNS TXT records that could be >>> published sha256 hashes of each source module. >>> >>> http://erlang.dloh.org/daves_**mod.erl45663AFDA.... >>> >>> Adding a >>> >>> -signature("http://erlang.**dlog.org/daves_mod.erl >>> ","**45663AFDA....") >>> >>> Would allow a 3 part verification of the source: >>> >>> 1.) I can compute the source has the right hash >>> 2.) I can look up that the module has the same published signature >>> 3.) I can verify against the original at the specified URL >>> >>> In this scenario it is not enough to modify the source and rehash, nor >>> enough to replace the upsteam file, but also replace the DNS entry without >>> anyone noticing. >>> >>> Thoughts? >>> >> >> Good stuff - needs some thought though. I was thinking of >> signing/validating >> the source with an RSA public/private keypair. >> >> I'd like to see this as part of the build process, if I did "make" I >> might like to see: >> >> $ make >> module foo123.erl is up to date. Written by joearms *validated* >> module bar23.erl has a newer version >> module bingo23.erl is up to date written by cleverperson *untrusted* >> ... >> etc. >> >> >> /Joe >> >> >> >> >> >>> Dave >>> -=-=- dave@REDACTED -=-=- >>> ______________________________**_________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/**listinfo/erlang-questions >>> >>> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> > > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > -- -=-=-=-=-=-=-=-=-=-=- http://blog.dloh.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bookjovi@REDACTED Wed Sep 28 04:57:41 2011 From: bookjovi@REDACTED (Jovi Zhang) Date: Wed, 28 Sep 2011 10:57:41 +0800 Subject: [erlang-questions] How to get pid number in NIF code? Message-ID: Hi, I'm writing some code need to get pid number in NIF library, so how to do it? I cannot found any nif function relate with it! Thanks very much. .jovi From bookjovi@REDACTED Wed Sep 28 09:10:16 2011 From: bookjovi@REDACTED (Jovi Zhang) Date: Wed, 28 Sep 2011 15:10:16 +0800 Subject: [erlang-questions] how to make binary in native thread created by enif_thread_create? Message-ID: Hi, This is a question about erlang NIF thread. As title, I cannot invoke enif_make_binary in native thread created by enif_thread_create. enif_make_binary need a env parameter, but it native thread, there don't have this env, of course I cannot use process bond environment, because those environment is invalid after NIF call finished. .jovi From paul.joseph.davis@REDACTED Wed Sep 28 09:28:47 2011 From: paul.joseph.davis@REDACTED (Paul Davis) Date: Wed, 28 Sep 2011 02:28:47 -0500 Subject: [erlang-questions] how to make binary in native thread created by enif_thread_create? In-Reply-To: References: Message-ID: Jovi, You can create environments with enif_alloc_env that are not bound to the scope of a NIF callback. See: http://www.erlang.org/doc/man/erl_nif.html#enif_alloc_env On Wed, Sep 28, 2011 at 2:10 AM, Jovi Zhang wrote: > Hi, > ? ?This is a question about erlang NIF thread. > > ? ?As title, I cannot invoke enif_make_binary in native thread > created by enif_thread_create. > > ? ?enif_make_binary need a env parameter, but it native thread, there > don't have this env, of course I cannot use process bond environment, > because those environment is invalid after > ? ?NIF call finished. > > .jovi > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From nox@REDACTED Wed Sep 28 09:31:19 2011 From: nox@REDACTED (Anthony Ramine) Date: Wed, 28 Sep 2011 09:31:19 +0200 Subject: [erlang-questions] How to get pid number in NIF code? In-Reply-To: References: Message-ID: Are you looking for enif_self() [1]? Regards, [1] http://www.erlang.org/doc/man/erl_nif.html#enif_self Le 28 sept. 2011 ? 04:57, Jovi Zhang a ?crit : > Hi, > I'm writing some code need to get pid number in NIF library, so > how to do it? I cannot found any nif function relate with it! > Thanks very much. > > .jovi > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Anthony Ramine / @nokusu Dev:Extend ? http://dev-extend.eu/ So as I pray, ?Unlimited Erlang Works? From bookjovi@REDACTED Wed Sep 28 09:53:48 2011 From: bookjovi@REDACTED (Jovi Zhang) Date: Wed, 28 Sep 2011 15:53:48 +0800 Subject: [erlang-questions] How to get pid number in NIF code? In-Reply-To: References: Message-ID: On Wed, Sep 28, 2011 at 3:31 PM, Anthony Ramine wrote: > Are you looking for enif_self() [1]? > > Regards, > > [1] http://www.erlang.org/doc/man/erl_nif.html#enif_self enif_self only return ErlNifPid, but I want to get pid number. like pid is <0.32.0>, what I want to get is 32, is there any way to get number? Thanks! ErlNifPid* enif_self(ErlNifEnv* caller_env, ErlNifPid* pid) From bookjovi@REDACTED Wed Sep 28 10:01:32 2011 From: bookjovi@REDACTED (Jovi Zhang) Date: Wed, 28 Sep 2011 16:01:32 +0800 Subject: [erlang-questions] how to make binary in native thread created by enif_thread_create? In-Reply-To: References: Message-ID: Thanks, it works correctly. On Wed, Sep 28, 2011 at 3:28 PM, Paul Davis wrote: > Jovi, > > You can create environments with enif_alloc_env that are not bound to > the scope of a NIF callback. See: > > ? ?http://www.erlang.org/doc/man/erl_nif.html#enif_alloc_env > > On Wed, Sep 28, 2011 at 2:10 AM, Jovi Zhang wrote: >> Hi, >> ? ?This is a question about erlang NIF thread. >> >> ? ?As title, I cannot invoke enif_make_binary in native thread >> created by enif_thread_create. >> >> ? ?enif_make_binary need a env parameter, but it native thread, there >> don't have this env, of course I cannot use process bond environment, >> because those environment is invalid after >> ? ?NIF call finished. >> >> .jovi >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > From tony@REDACTED Wed Sep 28 10:48:07 2011 From: tony@REDACTED (Tony Rogvall) Date: Wed, 28 Sep 2011 10:48:07 +0200 Subject: [erlang-questions] How to get pid number in NIF code? In-Reply-To: References: Message-ID: <6FF51D76-46C2-43C7-B291-A8AB7C155F0A@rogvall.se> I do not think there is a way to extract that number (except for a small hack, of course) Care to tell the reason behind why you need it ? /Tony On 28 sep 2011, at 09:53, Jovi Zhang wrote: > On Wed, Sep 28, 2011 at 3:31 PM, Anthony Ramine wrote: >> Are you looking for enif_self() [1]? >> >> Regards, >> >> [1] http://www.erlang.org/doc/man/erl_nif.html#enif_self > > enif_self only return ErlNifPid, but I want to get pid number. > like pid is <0.32.0>, what I want to get is 32, is there any way to > get number? Thanks! > > ErlNifPid* enif_self(ErlNifEnv* caller_env, ErlNifPid* pid) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From bookjovi@REDACTED Wed Sep 28 11:04:20 2011 From: bookjovi@REDACTED (Jovi Zhang) Date: Wed, 28 Sep 2011 17:04:20 +0800 Subject: [erlang-questions] How to get pid number in NIF code? In-Reply-To: <6FF51D76-46C2-43C7-B291-A8AB7C155F0A@rogvall.se> References: <6FF51D76-46C2-43C7-B291-A8AB7C155F0A@rogvall.se> Message-ID: On Wed, Sep 28, 2011 at 4:48 PM, Tony Rogvall wrote: > I do not think there is a way to extract that number (except for a small > hack, of course) > Care to tell the reason behind why you need it ? > /Tony > Hmm, I'm writing a NIF library to let erlang process communicate with external hand. we call that as "hand", it is similar with erlang process, but with different number reference, hand is range from 0-65535. So in this NIF library I must remember the mapping of "process<->hand", actually I only run this in one node, with process like <0.X.0>, which I only care about process number, that's why I want to get process number, not whole pid structure. Of course I can store all "ErlNifPid<->hand index", but that will consume a lot of memory. I think erts should provide API or macro for get process number for some use case. From sverker@REDACTED Wed Sep 28 11:18:20 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Wed, 28 Sep 2011 11:18:20 +0200 Subject: [erlang-questions] how to make binary in native thread created by enif_thread_create? In-Reply-To: References: Message-ID: <4E82E65C.3060508@erix.ericsson.se> Jovi Zhang wrote: > Hi, > This is a question about erlang NIF thread. > > As title, I cannot invoke enif_make_binary in native thread > created by enif_thread_create. > > enif_make_binary need a env parameter, but it native thread, there > don't have this env, of course I cannot use process bond environment, > because those environment is invalid after > NIF call finished. > > What's wrong with enif_alloc_env()? /Sverker, Erlang/OTP From erlang@REDACTED Wed Sep 28 12:28:59 2011 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 28 Sep 2011 12:28:59 +0200 Subject: [erlang-questions] Right direction ? In-Reply-To: References: <7AA05035-2AB7-4CB5-8FF0-A29A6D7455FF@nexttolast.com> Message-ID: Ok - I Let's talk about how to ensure we get the correct beam code. I guess you know how code gets loaded, but I'll summarize it, since it's probably not that well known There are several ways to load code 1) Method 1 You call foo:bar(1,2,3) If the module foo has not been loaded the system converts this to the call error_handler:undefined_function(foo, bar, [1,2,3]) this is in .../kernel/src/error_handler.erl the function undefined_function deligates the problem of actually finding the code to the code_server, which is supposed to know how to find the code. One the code has been found and loaded, which is the job of the code_server undefined_function can then evaluates apply(foo, bar, [1,2,3]) 2) The code is pre-loaded in a boot file and loaded by init - I'm not sure if the boot file has a list of filesnames or the content of the beam files, so there might be a security problem here. 3) Some programs (actually any program) evaluates the BIF erlang:load_module(Mod, Bin) This BIF assumes Bin contains the beam code for Mod - this is potentially dangerous since *any* process can execute this. Note: you might have to do purge_module(Mod) before calling load_module/2 (because we can run up to two different version of the module at the same time). When you compile a program in the shell method 3) is used to reload the code. Running compiled programs usually autoloads the code using method 1) A simple security measure would be to modify the code server to make sure it authenticates the code before loading. The problem is that any program can a type 3) operation So the only way to be really secure is to hack the emulator so that load_module will fail if the code in Bin has not been correctly signed. You could possibly run a two node distributed system, where one node is guaranteed to be secure and the other untrusted. Then you highly restrict which processes are running. Thinking out loud - the root problem is that "any" processes can evaluate erlang:load_module/2 I guess we could hack the system so that *only* the registered process code_server can call this .. this sounds doable I'll ask around a bit to see. /Joe On Wed, Sep 28, 2011 at 3:25 AM, David Goehrig wrote: > > >> >> I don't really understand. The only (legal) way to modify the beam >> is to change the source and recompile. I think you have to >> decide exactly what the semantics of require are. > > I'm actually most concerned about the illegal way of modifying a beam: > a.) Sysadmin gets clever an runs rsync patching the beam with a diff from > one on another server (bad if that server doesn't have the right version) > b.) Developer gets clever and uses source control as a deployment mechanism, > "git push production master", overriding the version there > c.) Nefarious type replaces beams with other beams that have been compiled > with compromised security built in > having a loader that can check at run time (late binding) > >> >> ? a) We check the require targets *before* compilation with >> ? ? ? ?a separate program > > rebar (http://github.com/basho/rebar) already does a pretty good job at this > (as long as you list all your dependencies as git repos) and I've been > making heavy use of this over the past year. ?It handles checking out and > compiling all the dependencies, and you can specify which specific tags you > depend upon. >> >> ? ? c) we check at usage time. The first time we call daves_module and >> find >> >> ? ? ? ? it has not been loaded we check the cache and so on > > > Right now I'm most worried about c.) in the context > of?lib/kernel/src/code_server.erl: > > try_load_module(Mod, Dir, Caller, St) -> > ? ? File = filename:append(Dir, to_path(Mod) ++ > ? ? ? ? ? ? ? ? ? ? ? ? ? ?objfile_extension()), > ? ? case erl_prim_loader:get_file(File) of > ? ? ? ? error -> > ? ? ? ? ? ? {reply,error,St}; > ? ? ? ? {ok,Binary,FName} -> > ? ? ? ? ? ? try_load_module(absname(FName), Mod, Binary, Caller, St) > ? ? end. > > Where the file pointed to by FName is now "trusted" and will then be read > into memory and passed off to hipe. > Part of the problem is I'm also introducing a new risk, because I'm > replacing this load bit with code that can read a URL rather than just a > filename, so I'd like a way to hook in to check that the file I've > downloaded is the same as the signature I have on file in a dets store. > >> >> Good stuff - needs some thought though. I was thinking of >> signing/validating >> the source with an RSA public/private keypair. > > I've thought about adding RSA public/private key signing, but that > ultimately goes down the route of having a CA to form a trust network, and > since CAs tend to prove to be unworthy of trust, I'm wary. ?Self publishing > a RSA public key + signing a SHA hash of the source and putting both in DNS > seems like a reasonable way of doing it, but can also be exploited to deny > service by DNS cache poisoning. > > If one were to implement a pub/private key signature check, would it best be > done in code_server.erl or somewhere else? ?That seems to be the first place > the files are loaded into memory at run time. > Dave > > -- > -=-=-=-=-=-=-=-=-=-=- http://blog.dloh.org/ > From robert.virding@REDACTED Wed Sep 28 15:15:08 2011 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 28 Sep 2011 14:15:08 +0100 (BST) Subject: [erlang-questions] How to get pid number in NIF code? In-Reply-To: Message-ID: <5dac7d4e-c8e9-4ee4-b37f-0ac9f56d1ab5@knuth> Someone must correct if I am wrong here but I don't think that an ErlNifPid takes much space. As I understand it is not the whole process structure, just a reference to it. From erl_nif.h: #ifdef HALFWORD_HEAP_EMULATOR # define ERL_NIF_VM_VARIANT "beam.halfword" typedef unsigned int ERL_NIF_TERM; #else # define ERL_NIF_VM_VARIANT "beam.vanilla" typedef unsigned long ERL_NIF_TERM; #endif ... typedef struct { ERL_NIF_TERM pid; /* internal, may change */ }ErlNifPid; But someone more knowledgeable should confirm this. Robert ----- Original Message ----- > On Wed, Sep 28, 2011 at 4:48 PM, Tony Rogvall > wrote: > > I do not think there is a way to extract that number (except for a > > small > > hack, of course) > > Care to tell the reason behind why you need it ? > > /Tony > > > > Hmm, I'm writing a NIF library to let erlang process communicate with > external hand. we call that as "hand", it is similar with erlang > process, but with different number reference, hand is range from > 0-65535. > > So in this NIF library I must remember the mapping of > "process<->hand", actually I only run this in one node, with process > like <0.X.0>, > which I only care about process number, that's why I want to get > process number, not whole pid structure. > > Of course I can store all "ErlNifPid<->hand index", but that will > consume a lot of memory. > > I think erts should provide API or macro for get process number for > some use case. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From sverker@REDACTED Wed Sep 28 20:35:11 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Wed, 28 Sep 2011 20:35:11 +0200 Subject: [erlang-questions] How to get pid number in NIF code? In-Reply-To: <5dac7d4e-c8e9-4ee4-b37f-0ac9f56d1ab5@knuth> References: <5dac7d4e-c8e9-4ee4-b37f-0ac9f56d1ab5@knuth> Message-ID: <4E8368DF.7050108@erix.ericsson.se> Yes, ErlNifPid is typically the size of one word. You can type cast it to an integer type of equal size to get a "process number". /Sverker, Erlang/OTP Robert Virding wrote: > Someone must correct if I am wrong here but I don't think that an ErlNifPid takes much space. As I understand it is not the whole process structure, just a reference to it. From erl_nif.h: > > #ifdef HALFWORD_HEAP_EMULATOR > # define ERL_NIF_VM_VARIANT "beam.halfword" > typedef unsigned int ERL_NIF_TERM; > #else > # define ERL_NIF_VM_VARIANT "beam.vanilla" > typedef unsigned long ERL_NIF_TERM; > #endif > > ... > > typedef struct > { > ERL_NIF_TERM pid; /* internal, may change */ > }ErlNifPid; > > But someone more knowledgeable should confirm this. > > Robert > > ----- Original Message ----- > >> On Wed, Sep 28, 2011 at 4:48 PM, Tony Rogvall >> wrote: >> >>> I do not think there is a way to extract that number (except for a >>> small >>> hack, of course) >>> Care to tell the reason behind why you need it ? >>> /Tony >>> >>> >> Hmm, I'm writing a NIF library to let erlang process communicate with >> external hand. we call that as "hand", it is similar with erlang >> process, but with different number reference, hand is range from >> 0-65535. >> >> So in this NIF library I must remember the mapping of >> "process<->hand", actually I only run this in one node, with process >> like <0.X.0>, >> which I only care about process number, that's why I want to get >> process number, not whole pid structure. >> >> Of course I can store all "ErlNifPid<->hand index", but that will >> consume a lot of memory. >> >> I think erts should provide API or macro for get process number for >> some use case. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From carlsson.richard@REDACTED Wed Sep 28 23:14:39 2011 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 28 Sep 2011 23:14:39 +0200 Subject: [erlang-questions] The importance of Basic Unicode Understanding in Erlang In-Reply-To: <4B939B9C-B0EF-4CCF-8F47-AE543F425BF1@erlang-solutions.com> References: <4B939B9C-B0EF-4CCF-8F47-AE543F425BF1@erlang-solutions.com> Message-ID: <4E838E3F.5040705@gmail.com> On 09/27/2011 05:37 PM, Fr?d?ric Trottier-H?bert wrote: > I've recently done some work where, due to circumstances, unicode > woes were had by everyone. It kind of got me by surprise, and I > figure that if it hasn't bitten you yet, it might sooner or later. As > such, I published a blog post on the issue yesterday: > http://ferd.ca/will-the-real-unicode-wrangler-please-stand-up.html While I definitely agree with the sentiment that standard library support for unicode string normalisation, collation, etc., is needed, there are some things in your blog post I want to point out: - The "good old length and comparison functions" are not broken, they just answer much simpler questions than what you're asking. length(S) tells you how many code points are in string S, no more, no less. Not glyphs, not graphemes, not abstract characters. Code points. Similar for comparisons. And for some applications, this is all you need. It's only when you want to apply "human" ideas of order and visual appearance that you need to use special library functions - and if you do this, you should _know_ that this is what you're doing; not hope that a primitive function like length(S) will guess what kind of information you want it to compute. - There's nothing strange about having to use ~ts instead of ~s in format strings: similar changes have to be made in C code to handle wide characters and multibyte encodings. Backwards compatibility with the existing codebase is simply a necessary thing. Yes, you have to update your source code if you want to make it work on Unicode. - If you're taking a Unicode string that starts with a combining character, and then append this onto a base character, as in your [$a, U1, U2] example, you're doing something odd (though not illegal), and I don't see - at least not after just a brief glance - that the Unicode specification suggests a way of handling this in a standard way. I guess that one could insert a zero-width space base character each time one concatenates two strings, but that assumes that you _didn't_ actually want the effect that you got, and who is to know that? Or you could sanitize your input so that you don't have strings that start with combining characters. - Your example io:format("~ts~n",[binary_to_list(<<208,...,33>>)]) doesn't work because the input to ~ts is expected to be of type unicode:chardata(), which is an iolist-like mixture of lists of integers and binary fragments, where integers are expected to be _Unicode code points_ while binaries are interpreted as being in UTF-8. Hence, your first example that passed the binary directly works fine, but when you convert it to a list of bytes using binary_to_list, that list is not what ~ts expects, because it's still in UTF-8. If you do io:format("~ts~n",[unicode:characters_to_list(<<208,...,33>>)]) instead, it works fine, and is equivalent to io:format("~ts~n",[[1055,1088,1080,1074,1077,1090,33]]). - list_to_binary crashes on input that is outside the range 0..255 precisely so that you may catch errors early. If you suddenly start to feed Unicode strings to an existing program, and it silently does something like truncating to bytes upon writing to file, you could end up with a lot more damage. It's better to be told right away that your code requires 8-bit data at that point, so you have a chance of figuring out who should do the encoding and where. /Richard From ok@REDACTED Thu Sep 29 00:52:16 2011 From: ok@REDACTED (Richard O'Keefe) Date: Thu, 29 Sep 2011 11:52:16 +1300 Subject: [erlang-questions] The importance of Basic Unicode Understanding in Erlang In-Reply-To: <4E838E3F.5040705@gmail.com> References: <4B939B9C-B0EF-4CCF-8F47-AE543F425BF1@erlang-solutions.com> <4E838E3F.5040705@gmail.com> Message-ID: <8070F9E5-A8F9-4C26-A8F3-5172E261B30A@cs.otago.ac.nz> On 29/09/2011, at 10:14 AM, Richard Carlsson wrote: > - The "good old length and comparison functions" are not broken, they just answer much simpler questions than what you're asking. length(S) tells you how many code points are in string S, no more, no less. Not glyphs, not graphemes, not abstract characters. Code points. I should point out that the question "how many characters are there" is locale-dependent. My mother's father, looking at the place name "?ub?ana" would have seen 7 letters. I see 9. (There are in fact 7 Unicode code points. Who said one code point couldn't count as more than one letter?) Looking at my Father's middle name: "?neas", I see 5 letters. (Unicode agrees with me.) Other people see 6. This means that there is no such thing as a "unicode" function grapheme_length :: String ? Integer but only a function grapheme_length :: String ? Locale ? Integer This is only the beginning of the problems! > Similar for comparisons. And again, similar for comparisons. From bookjovi@REDACTED Thu Sep 29 03:41:56 2011 From: bookjovi@REDACTED (Jovi Zhang) Date: Thu, 29 Sep 2011 09:41:56 +0800 Subject: [erlang-questions] How to get pid number in NIF code? In-Reply-To: <4E8368DF.7050108@erix.ericsson.se> References: <5dac7d4e-c8e9-4ee4-b37f-0ac9f56d1ab5@knuth> <4E8368DF.7050108@erix.ericsson.se> Message-ID: Hmm, understood, Thanks very much, I will store the ErlNifPid instead of process number. On Thu, Sep 29, 2011 at 2:35 AM, Sverker Eriksson wrote: > Yes, ErlNifPid is typically the size of one word. You can type cast it to an > integer type of equal size to get a "process number". > > /Sverker, Erlang/OTP > > Robert Virding wrote: >> >> Someone must correct if I am wrong here but I don't think that an >> ErlNifPid takes much space. As I understand it is not the whole process >> structure, just a reference to it. From erl_nif.h: >> >> #ifdef HALFWORD_HEAP_EMULATOR >> # ?define ERL_NIF_VM_VARIANT "beam.halfword" typedef unsigned int >> ERL_NIF_TERM; >> #else >> # ?define ERL_NIF_VM_VARIANT "beam.vanilla" typedef unsigned long >> ERL_NIF_TERM; >> #endif >> >> ... >> >> typedef struct >> { >> ? ?ERL_NIF_TERM pid; ?/* internal, may change */ >> }ErlNifPid; >> >> But someone more knowledgeable should confirm this. >> >> Robert >> >> ----- Original Message ----- >> >>> >>> On Wed, Sep 28, 2011 at 4:48 PM, Tony Rogvall >>> wrote: >>> >>>> >>>> I do not think there is a way to extract that number (except for a >>>> small >>>> hack, of course) >>>> Care to tell the reason behind why you need it ? >>>> /Tony >>>> >>>> >>> >>> Hmm, I'm writing a NIF library to let erlang process communicate with >>> external hand. we call that as "hand", it is similar with erlang >>> process, but with different number reference, hand is range from >>> 0-65535. >>> >>> So in this NIF library I must remember the mapping of >>> "process<->hand", actually I only run this in one node, with process >>> like <0.X.0>, >>> which I only care about process number, that's why I want to get >>> process number, not whole pid structure. >>> >>> Of course I can store all "ErlNifPid<->hand index", but that will >>> consume a lot of memory. >>> >>> I think erts should provide API or macro for get process number for >>> some use case. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > From jwatte@REDACTED Thu Sep 29 04:01:21 2011 From: jwatte@REDACTED (Jon Watte) Date: Wed, 28 Sep 2011 19:01:21 -0700 Subject: [erlang-questions] Right direction ? In-Reply-To: References: <7AA05035-2AB7-4CB5-8FF0-A29A6D7455FF@nexttolast.com> Message-ID: I've thought about adding RSA public/private key signing, but that ultimately goes down the route of having a CA to form a trust network, and since CAs tend to prove to be unworthy of trust, I'm wary. While it's certainly true that certificate authorities are a racket (at a minimum until such time as governments get in on it, for voting and taxation, but perhaps even after that) there is very little other choice. But if "nefarious type" has access to your machine, presumably he can patch the kernel to patch erlang to patch your beam to accept even the wrong checksum, so I would not worry about trying to protect against someone who can subvert your file system protection. Use proper permissions and proper account access keys for your production server for this -- you *can't* defend against that kind of intrusion. Even external verifiers are vulnerable, because if you "ask" the compromised machine for its "opinion" (anything from "what code is running?" to "what is the signature?") then, it being compromised, it can already lie and tell you the answers you want to hear. Trusted Hardware Platform, anyone? :-) Sincerely, jw -- Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. On Tue, Sep 27, 2011 at 6:25 PM, David Goehrig wrote: > > > >> I don't really understand. The only (legal) way to modify the beam >> is to change the source and recompile. I think you have to >> decide exactly what the semantics of require are. > > > I'm actually most concerned about the illegal way of modifying a beam: > > a.) Sysadmin gets clever an runs rsync patching the beam with a diff from > one on another server (bad if that server doesn't have the right version) > b.) Developer gets clever and uses source control as a deployment > mechanism, "git push production master", overriding the version there > c.) Nefarious type replaces beams with other beams that have been compiled > with compromised security built in > > having a loader that can check at run time (late binding) > > >> a) We check the require targets *before* compilation with >> a separate program >> > > rebar (http://github.com/basho/rebar) already does a pretty good job at > this (as long as you list all your dependencies as git repos) and I've been > making heavy use of this over the past year. It handles checking out and > compiling all the dependencies, and you can specify which specific tags you > depend upon. > > c) we check at usage time. The first time we call daves_module and find > > it has not been loaded we check the cache and so on >> > > Right now I'm most worried about c.) in the context > of lib/kernel/src/code_server.erl: > > try_load_module(Mod, Dir, Caller, St) -> > File = filename:append(Dir, to_path(Mod) ++ > objfile_extension()), > case erl_prim_loader:get_file(File) of > error -> > {reply,error,St}; > {ok,Binary,FName} -> > try_load_module(absname(FName), Mod, Binary, Caller, St) > end. > > > Where the file pointed to by FName is now "trusted" and will then be read > into memory and passed off to hipe. > > Part of the problem is I'm also introducing a new risk, because I'm > replacing this load bit with code that can read a URL rather than just a > filename, so I'd like a way to hook in to check that the file I've > downloaded is the same as the signature I have on file in a dets store. > > > >> Good stuff - needs some thought though. I was thinking of >> signing/validating >> the source with an RSA public/private keypair. >> > > I've thought about adding RSA public/private key signing, but that > ultimately goes down the route of having a CA to form a trust network, and > since CAs tend to prove to be unworthy of trust, I'm wary. Self publishing > a RSA public key + signing a SHA hash of the source and putting both in DNS > seems like a reasonable way of doing it, but can also be exploited to deny > service by DNS cache poisoning. > > If one were to implement a pub/private key signature check, would it best > be done in code_server.erl or somewhere else? That seems to be the first > place the files are loaded into memory at run time. > > Dave > > -- > -=-=-=-=-=-=-=-=-=-=- http://blog.dloh.org/ > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Sep 29 09:19:32 2011 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 29 Sep 2011 09:19:32 +0200 Subject: [erlang-questions] Right direction ? In-Reply-To: References: <7AA05035-2AB7-4CB5-8FF0-A29A6D7455FF@nexttolast.com> Message-ID: On Thu, Sep 29, 2011 at 4:01 AM, Jon Watte wrote: > I've thought about adding RSA public/private key signing, but that > ultimately goes down the route of having a CA to form a trust network, and > since CAs tend to prove to be unworthy of trust, I'm wary. > While it's certainly true that certificate authorities are a racket (at a > minimum until such time as governments get in on it, for voting and > taxation, but perhaps even after that) there is very little other choice. > But if "nefarious type" has access to your machine, presumably he can patch > the kernel to patch erlang to patch your beam to accept even the wrong > checksum, so I would not worry about trying to protect against someone who > can subvert your file system protection. Use proper permissions and proper > account access keys for your production server for this -- you *can't* > defend against that kind of intrusion. > Even external verifiers are vulnerable, because if you "ask" the compromised > machine for its "opinion" (anything from "what code is running?" to "what is > the signature?") then, it being compromised, it can already lie and tell you > the answers you want to hear. > Trusted Hardware Platform, anyone? :-) I have always assumed that the problem was trust and not security. If I trust the source I'll accept the certificate - if the trust is betrayed I loose. I don't think it makes sense to ask "is the system secure?" - we need to ask "how much effort will an attacker have to expend to break the security". I guess protecting against script-kiddies who try to guess passwords is one thing - but protecting against agencies who install password sniffers in the firmware of your keyboard is a far more difficult. The level of protection that interests me is: a) - protection against shooting yourself in the foot (ie non-malevolent mistakes) - such as rsyncing the wrong version of the code b) - protection against simple malevolent attacks (password guessing, exposing plain text passwords etc.) If the system is insecure then I would hope it could resist the attack of a "professional" cracker for several weeks - so they would give up. Actually a) should come "out of the box" but is pretty difficult to acheive most security violations are "obvious" after they have been exposed, but non-evident up to then. /Joe > Sincerely, > jw > > -- > Americans might object: there is no way we would sacrifice our living > standards for the benefit of people in the rest of the world. Nevertheless, > whether we get there willingly or not, we shall soon have lower consumption > rates, because our present rates are unsustainable. > > > > On Tue, Sep 27, 2011 at 6:25 PM, David Goehrig wrote: >> >> >>> >>> I don't really understand. The only (legal) way to modify the beam >>> is to change the source and recompile. I think you have to >>> decide exactly what the semantics of require are. >> >> I'm actually most concerned about the illegal way of modifying a beam: >> a.) Sysadmin gets clever an runs rsync patching the beam with a diff from >> one on another server (bad if that server doesn't have the right version) >> b.) Developer gets clever and uses source control as a deployment >> mechanism, "git push production master", overriding the version there >> c.) Nefarious type replaces beams with other beams that have been compiled >> with compromised security built in >> having a loader that can check at run time (late binding) >> >>> >>> ? a) We check the require targets *before* compilation with >>> ? ? ? ?a separate program >> >> rebar (http://github.com/basho/rebar) already does a pretty good job at >> this (as long as you list all your dependencies as git repos) and I've been >> making heavy use of this over the past year. ?It handles checking out and >> compiling all the dependencies, and you can specify which specific tags you >> depend upon. >>> >>> ? ? c) we check at usage time. The first time we call daves_module and >>> find >>> >>> ? ? ? ? it has not been loaded we check the cache and so on >> >> >> Right now I'm most worried about c.) in the context >> of?lib/kernel/src/code_server.erl: >> >> try_load_module(Mod, Dir, Caller, St) -> >> ? ? File = filename:append(Dir, to_path(Mod) ++ >> ? ? ? ? ? ? ? ? ? ? ? ? ? ?objfile_extension()), >> ? ? case erl_prim_loader:get_file(File) of >> ? ? ? ? error -> >> ? ? ? ? ? ? {reply,error,St}; >> ? ? ? ? {ok,Binary,FName} -> >> ? ? ? ? ? ? try_load_module(absname(FName), Mod, Binary, Caller, St) >> ? ? end. >> >> Where the file pointed to by FName is now "trusted" and will then be read >> into memory and passed off to hipe. >> Part of the problem is I'm also introducing a new risk, because I'm >> replacing this load bit with code that can read a URL rather than just a >> filename, so I'd like a way to hook in to check that the file I've >> downloaded is the same as the signature I have on file in a dets store. >> >>> >>> Good stuff - needs some thought though. I was thinking of >>> signing/validating >>> the source with an RSA public/private keypair. >> >> I've thought about adding RSA public/private key signing, but that >> ultimately goes down the route of having a CA to form a trust network, and >> since CAs tend to prove to be unworthy of trust, I'm wary. ?Self publishing >> a RSA public key + signing a SHA hash of the source and putting both in DNS >> seems like a reasonable way of doing it, but can also be exploited to deny >> service by DNS cache poisoning. >> >> If one were to implement a pub/private key signature check, would it best >> be done in code_server.erl or somewhere else? ?That seems to be the first >> place the files are loaded into memory at run time. >> Dave >> >> -- >> -=-=-=-=-=-=-=-=-=-=- http://blog.dloh.org/ >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > From jonas.boberg@REDACTED Thu Sep 29 10:55:02 2011 From: jonas.boberg@REDACTED (Jonas Boberg) Date: Thu, 29 Sep 2011 10:55:02 +0200 Subject: [erlang-questions] Mnesia does not detect netsplit Message-ID: Hi, We found a case where mnesia does not detect a netsplit. Let's say we are running two mnesia nodes, A and B: At startup, node A can't connect to node B (specified in the mnesia config parameter extra_db_nodes). In this case node B is actually running, but because of a temporary network issue, or node B being heavily loaded, net_kernel:connect fails. When node A and B eventually are connected (for example due to a non-mnesia process sending a message between the nodes), mnesia does not detect the split, and the two isles continue to run separately. Note that when we say that mnesia does not detect the netsplit, we mean that mnesia does not generate any 'inconsistent_database' event. How to reproduce. * In this example we simulate a network problem (net_kernel:connect failure) by having the two nodes use different cookies. ------------------ $ erl -name test1@REDACTED -mnesia schema_location ram -mnesia extra_db_nodes "['test2@REDACTED']" -setcookie a (test1@REDACTED)1> application:start(mnesia), mnesia:subscribe(system), mnesia:create_table(my_table, []). $ erl -name test2@REDACTED -mnesia schema_location ram -mnesia extra_db_nodes "['test1@REDACTED']" -setcookie b (test2@REDACTED)1> application:start(mnesia), mnesia:subscribe(system), mnesia:create_table(my_other_table, []). %% Connect nodes (test1@REDACTED)2> erlang:set_cookie(node(), b), net_kernel:connect('test2@REDACTED'). (test1@REDACTED)3> nodes(). ['test2@REDACTED'] (test1@REDACTED)4> mnesia:info(). ... running db nodes = ['test1@REDACTED'] stopped db nodes = ['test2@REDACTED'] ... ------------------ Expected behaviour: subscriber gets a 'inconsistent_database' event Actual behaviour: subscriber does not get any event. Compare to this case, where mnesia correctly detects a inconsistent database: ------------------ $ erl -name test1@REDACTED -mnesia schema_location ram -mnesia extra_db_nodes "['test2@REDACTED']" -setcookie a (test1@REDACTED)1> application:start(mnesia), mnesia:subscribe(system), mnesia:create_table(my_table, []). $ erl -name test2@REDACTED -@REDACTED -mnesia schema_location ram -mnesia extra_db_nodes "['test1@REDACTED']" -setcookie a (test2@REDACTED)1> application:start(mnesia), mnesia:subscribe(system), mnesia:create_table(my_other_table, []). (test2@REDACTED)2> net_kernel:disconnect('test1@REDACTED'). (test2@REDACTED)3> net_kernel:connect('test1@REDACTED'). (test2@REDACTED)4> flush(). Shell got {mnesia_system_event,{mnesia_down,'test1@REDACTED'}} Shell got {mnesia_system_event, {inconsistent_database,running_partitioned_network, 'test1@REDACTED'}} We found that the mnesia code that detects netsplits is in mnesia_monitor. It uses net_kernel:monitor_nodes(true), to monitor nodes going up and down. In the problematic scenario, when the mnesia_monitor gets the the 'nodeup', it seems to ignore it since a node down has not been seen. Trace: (<0.53.0>) call mnesia_monitor:handle_info({nodeup,'test1@REDACTED'},{state,<0.52.0>,[],[],true,[],undefined,[]}) (<0.53.0>) call mnesia_recover:has_mnesia_down('test1@REDACTED') (<0.53.0>) returned from mnesia_recover:has_mnesia_down/1 -> false Does anyone have an idea about how we could work around this issue? If we would detect the split ourselves, is there anyway we could get mnesia to reconnect the nodes? Regards Jonas From norton@REDACTED Thu Sep 29 11:03:52 2011 From: norton@REDACTED (Joseph Norton) Date: Thu, 29 Sep 2011 18:03:52 +0900 Subject: [erlang-questions] Mnesia does not detect netsplit In-Reply-To: References: Message-ID: <042D3854-BEA1-449F-A7F4-646DD18850C5@lovely.email.ne.jp> FYI. I posted a suggestion on the mailing list for a network partition detector application. http://erlang.org/pipermail/erlang-questions/2011-August/060702.html If you have any questions, please send to me off list. thanks, Joseph Norton norton@REDACTED On Sep 29, 2011, at 5:55 PM, Jonas Boberg wrote: > Hi, > > We found a case where mnesia does not detect a netsplit. > > Let's say we are running two mnesia nodes, A and B: > At startup, node A can't connect to node B (specified in the mnesia > config parameter extra_db_nodes). In this case node B is actually > running, but because of a temporary network issue, or node B being > heavily loaded, net_kernel:connect fails. When node A and B eventually > are connected (for example due to a non-mnesia process sending a > message between the nodes), mnesia does not detect the split, and the > two isles continue to run separately. > > Note that when we say that mnesia does not detect the netsplit, we > mean that mnesia does not generate any 'inconsistent_database' event. > > How to reproduce. > * In this example we simulate a network problem (net_kernel:connect > failure) by having the two nodes use different cookies. > ------------------ > $ erl -name test1@REDACTED -mnesia schema_location ram -mnesia > extra_db_nodes "['test2@REDACTED']" -setcookie a > (test1@REDACTED)1> application:start(mnesia), > mnesia:subscribe(system), mnesia:create_table(my_table, []). > $ erl -name test2@REDACTED -mnesia schema_location ram -mnesia > extra_db_nodes "['test1@REDACTED']" -setcookie b > (test2@REDACTED)1> application:start(mnesia), > mnesia:subscribe(system), mnesia:create_table(my_other_table, []). > %% Connect nodes > (test1@REDACTED)2> erlang:set_cookie(node(), b), > net_kernel:connect('test2@REDACTED'). > (test1@REDACTED)3> nodes(). > ['test2@REDACTED'] > (test1@REDACTED)4> mnesia:info(). > ... > running db nodes = ['test1@REDACTED'] > stopped db nodes = ['test2@REDACTED'] > ... > > ------------------ > Expected behaviour: subscriber gets a 'inconsistent_database' event > Actual behaviour: subscriber does not get any event. > > Compare to this case, where mnesia correctly detects a inconsistent database: > ------------------ > $ erl -name test1@REDACTED -mnesia schema_location ram -mnesia > extra_db_nodes "['test2@REDACTED']" -setcookie a > (test1@REDACTED)1> application:start(mnesia), > mnesia:subscribe(system), mnesia:create_table(my_table, []). > $ erl -name test2@REDACTED -@REDACTED -mnesia schema_location ram > -mnesia extra_db_nodes "['test1@REDACTED']" -setcookie a > (test2@REDACTED)1> application:start(mnesia), > mnesia:subscribe(system), mnesia:create_table(my_other_table, []). > (test2@REDACTED)2> net_kernel:disconnect('test1@REDACTED'). > (test2@REDACTED)3> net_kernel:connect('test1@REDACTED'). > (test2@REDACTED)4> flush(). > Shell got {mnesia_system_event,{mnesia_down,'test1@REDACTED'}} > Shell got {mnesia_system_event, > {inconsistent_database,running_partitioned_network, > 'test1@REDACTED'}} > > We found that the mnesia code that detects netsplits is in > mnesia_monitor. It uses net_kernel:monitor_nodes(true), to monitor > nodes going up and down. In the problematic scenario, when the > mnesia_monitor gets the the 'nodeup', it seems to ignore it since a > node down has not been seen. > Trace: > (<0.53.0>) call > mnesia_monitor:handle_info({nodeup,'test1@REDACTED'},{state,<0.52.0>,[],[],true,[],undefined,[]}) > (<0.53.0>) call mnesia_recover:has_mnesia_down('test1@REDACTED') > (<0.53.0>) returned from mnesia_recover:has_mnesia_down/1 -> false > > Does anyone have an idea about how we could work around this issue? If > we would detect the split ourselves, is there anyway we could get > mnesia to reconnect the nodes? > > Regards > Jonas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From fred.hebert@REDACTED Thu Sep 29 13:20:01 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Thu, 29 Sep 2011 07:20:01 -0400 Subject: [erlang-questions] The importance of Basic Unicode Understanding in Erlang In-Reply-To: <4E838E3F.5040705@gmail.com> References: <4B939B9C-B0EF-4CCF-8F47-AE543F425BF1@erlang-solutions.com> <4E838E3F.5040705@gmail.com> Message-ID: <1E9093AE-B49A-432C-815B-6B2E54E91C65@erlang-solutions.com> On 2011-09-28, at 17:14 PM, Richard Carlsson wrote: > On 09/27/2011 05:37 PM, Fr?d?ric Trottier-H?bert wrote: >> I've recently done some work where, due to circumstances, unicode >> woes were had by everyone. It kind of got me by surprise, and I >> figure that if it hasn't bitten you yet, it might sooner or later. As >> such, I published a blog post on the issue yesterday: >> http://ferd.ca/will-the-real-unicode-wrangler-please-stand-up.html > > While I definitely agree with the sentiment that standard library support for unicode string normalisation, collation, etc., is needed, there are some things in your blog post I want to point out: > > - The "good old length and comparison functions" are not broken, they just answer much simpler questions than what you're asking. length(S) tells you how many code points are in string S, no more, no less. Not glyphs, not graphemes, not abstract characters. Code points. Similar for comparisons. And for some applications, this is all you need. It's only when you want to apply "human" ideas of order and visual appearance that you need to use special library functions - and if you do this, you should _know_ that this is what you're doing; not hope that a primitive function like length(S) will guess what kind of information you want it to compute. That is a good point > > - There's nothing strange about having to use ~ts instead of ~s in format strings: similar changes have to be made in C code to handle wide characters and multibyte encodings. Backwards compatibility with the existing codebase is simply a necessary thing. Yes, you have to update your source code if you want to make it work on Unicode. It is an annoyance when you compare it to languages where strings can contain information about their own encoding. You print stuff and that's it, no questions asked. It is not strange, though, and I should have worded it better. > > - If you're taking a Unicode string that starts with a combining character, and then append this onto a base character, as in your [$a, U1, U2] example, you're doing something odd (though not illegal), and I don't see - at least not after just a brief glance - that the Unicode specification suggests a way of handling this in a standard way. I guess that one could insert a zero-width space base character each time one concatenates two strings, but that assumes that you _didn't_ actually want the effect that you got, and who is to know that? Or you could sanitize your input so that you don't have strings that start with combining characters. When you're writing libraries or certain applications, you don't get to choose what input you get, sadly. We've had issues in socket.io-erlang based on that. Part of the problem is that the encoded strings we get depend on the length of the string on whatever base Javascript is used, and not codepoints. The problem is that they use some kind of weird framing of the form "~Length~~m~Message~Length~~m~OtherMessage". As such, when a control character is submitted at the beginning or the end of a message, the length you have is different from the length that was counted earlier. You have to be extra-careful about splitting your strings into independent fragments before extracting messages. This means you can't just take the length and split away (even with unicode-compliant functions), you have to actually take a look at what's at the end of the 'Message' in order to do it right. As far as I know, there is no way around it. > > - Your example io:format("~ts~n",[binary_to_list(<<208,...,33>>)]) doesn't work because the input to ~ts is expected to be of type unicode:chardata(), which is an iolist-like mixture of lists of integers and binary fragments, where integers are expected to be _Unicode code points_ while binaries are interpreted as being in UTF-8. Hence, your first example that passed the binary directly works fine, but when you convert it to a list of bytes using binary_to_list, that list is not what ~ts expects, because it's still in UTF-8. If you do io:format("~ts~n",[unicode:characters_to_list(<<208,...,33>>)]) instead, it works fine, and is equivalent to io:format("~ts~n",[[1055,1088,1080,1074,1077,1090,33]]). Right. That's what I tried to explain in the text. Obviously the string is output, but wrong. That's when I get to discuss the unicode module with some examples, and add: "if you get <<101,204,129>> and convert it to a string using binary_to_list, you'll get the list [101,204,129], which is an entirely different unicode string. Using the unicode module, you'll get the right thing back" > > - list_to_binary crashes on input that is outside the range 0..255 precisely so that you may catch errors early. If you suddenly start to feed Unicode strings to an existing program, and it silently does something like truncating to bytes upon writing to file, you could end up with a lot more damage. It's better to be told right away that your code requires 8-bit data at that point, so you have a chance of figuring out who should do the encoding and where. True. It's not a bad thing, and I agree that letting it crash is a decent choice there. > > /Richard > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From masklinn@REDACTED Thu Sep 29 13:35:59 2011 From: masklinn@REDACTED (Masklinn) Date: Thu, 29 Sep 2011 13:35:59 +0200 Subject: [erlang-questions] The importance of Basic Unicode Understanding in Erlang In-Reply-To: <4E838E3F.5040705@gmail.com> References: <4B939B9C-B0EF-4CCF-8F47-AE543F425BF1@erlang-solutions.com> <4E838E3F.5040705@gmail.com> Message-ID: On 2011-09-28, at 23:14 , Richard Carlsson wrote: > > - The "good old length and comparison functions" are not broken, they just answer much simpler questions than what you're asking. length(S) tells you how many code points are in string S, no more, no less. Not glyphs, not graphemes, not abstract characters. Code points. Similar for comparisons. And for some applications, this is all you need. It's only when you want to apply "human" ideas of order and visual appearance that you need to use special library functions I'm not sure I agree about that: let's imagine you send a name to a third-party tool, and that tool happens to have very precise ideas about normalization (e.g. it's an OSX API and it *will* manipulate only NFD strings). You send an NFC UTF-8 bytestring, you get an NDF UTF-8 bytestring, you decode to Unicode codepoints. The two unicode sequences are canonically equivalent, but not equal. This has little to do with "human ideas of order and visual appearance" now does it? > - and if you do this, you should _know_ that this is what you're doing; not hope that a primitive function like length(S) will guess what kind of information you want it to compute. I really don't agree. A good API should make doing the common and right thing *easy* (and fool-proof), and the uncommon (and usually wrong) thing harder. Most string APIs do the exact opposite re. unicode, that's bonkers. How often do you need the codepoints-length of a unicode sequence? The length of a UTF-encoded binary stream yes, the number of grapheme clusters (for elision or length cutoff of some character field), but the unicode sequence? I don't think I've ever had this need. The number of grapheme clusters for elision or length cutoff yes, but definitely not the number of codepoints. > - There's nothing strange about having to use ~ts instead of ~s in format strings: similar changes have to be made in C code to handle wide characters and multibyte encodings. Backwards compatibility with the existing codebase is simply a necessary thing. Yes, you have to update your source code if you want to make it work on Unicode. On the other hand, Erlang is not C, it's not like I have to do pointer arithmetics or consider collection ownership to know when to manually release my arrays. From cbenac@REDACTED Thu Sep 29 15:06:04 2011 From: cbenac@REDACTED (Clara Benac Earle) Date: Thu, 29 Sep 2011 15:06:04 +0200 Subject: [erlang-questions] [ANN] Madrid Erlounge 4th of October Message-ID: <4E846D3C.9010909@fi.upm.es> Dear all We will have a Madrid Erlounge next Tuesday 4th of October at 18:30. I still need a confirmation for the meeting place but it will be somewhere in central Madrid. Samuel Rivas, previously at the company LambdaStream and currently CTO of Interoud (http://www.interoud.com/) will be giving the following talk. We've been developing an Erlang based solution for embedded devices to control the interaction between web-based interfaces and video processing and streaming hardware. The result is an Erlang middleware with two interfaces, one to control low level drivers written in C and other to control an embedded browser where GUI programmers can develop apps and user interfaces using standard javascript techniques. The target of the project is a consumer electronics device for home entertainment, as a new step forward in our ongoing plan to have one Erlang VM in each living room. After the talk we will continue with some tapas. More details on the meeting place soon! Clara From dave@REDACTED Thu Sep 29 15:18:51 2011 From: dave@REDACTED (David Goehrig) Date: Thu, 29 Sep 2011 09:18:51 -0400 Subject: [erlang-questions] Right direction ? In-Reply-To: References: <7AA05035-2AB7-4CB5-8FF0-A29A6D7455FF@nexttolast.com> Message-ID: On Wed, Sep 28, 2011 at 6:28 AM, Joe Armstrong wrote: 3) Some programs (actually any program) evaluates the BIF > > erlang:load_module(Mod, Bin) > > I'm curious, this looks like one place in the entire system that I'd love to keep an active trace log of over time. Since we can load arbitrary byte code here, it is the most likely place for abuse (both good and bad). This makes me feel like it might be a half decent idea to implement http_code_server.erl that uses erlang:load_module(Mod,Bin) to directly load modules from remote sites, and leave code_server.erl alone for the time being. This way, my default erlang can be "as secure as it ever was" and I can introduce all sorts of horrific hacks at this layer. I would still want to extend the module syntax with the -location() bit, but have it fall back to the http_code_server for loading. Thoughts? Dave -- -=-=-=-=-=-=-=-=-=-=- http://blog.dloh.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave@REDACTED Thu Sep 29 15:30:09 2011 From: dave@REDACTED (David Goehrig) Date: Thu, 29 Sep 2011 09:30:09 -0400 Subject: [erlang-questions] Right direction ? In-Reply-To: References: <7AA05035-2AB7-4CB5-8FF0-A29A6D7455FF@nexttolast.com> Message-ID: On Thu, Sep 29, 2011 at 3:19 AM, Joe Armstrong wrote: > > > The level of protection that interests me is: > > a) - protection against shooting yourself in the foot > (ie non-malevolent mistakes) - such as rsyncing the > wrong version of the code > b) - protection against simple malevolent attacks (password guessing, > exposing plain text passwords etc.) > a.) also requires that it be EASY for developers to support it. Requiring a $1000 CA cert just to publish some open source code is a sure way to shoot yourself in the foot. For a.) to work we need: * A way for a developer to easily publish a SIGNATURE so that all people everywhere can download and verify the code against it * A way for a user to easily verify that a particular bit of code matches the SIGNATURE from the location where it was acquired Both of these typically are done by sticking a MD5 sum or SHA1 has in the directory in a file along side the original file, so that you can verify that the file has not been modified from the version at that location. The problem here is that if both the package and the signature are tampered with, you don't necessarily know. For these sorts of problems, DNS seems like a good idea in so far as it is a distributed database that is publicly accessible, and there exist tools for detecting when someone is messing with it. A dets table with a set of SHA1 hashes of every module loaded that simply refuses to load the module if the signature changes can be useful. A process that periodically checks DNS for 3rd party modules to notify you of changes can be really useful, as it becomes a way for a developer to notify you of security patches, bug fixes, deprecations, and all sorts of other maintenance issues that will threaten your project over time. I'm almost done with a proof of concept of such a service. Maybe next week I'll release a public beta. Dave -- -=-=-=-=-=-=-=-=-=-=- http://blog.dloh.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Sep 29 16:18:41 2011 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 29 Sep 2011 16:18:41 +0200 Subject: [erlang-questions] Right direction ? In-Reply-To: References: <7AA05035-2AB7-4CB5-8FF0-A29A6D7455FF@nexttolast.com> Message-ID: On Thu, Sep 29, 2011 at 3:18 PM, David Goehrig wrote: > > > On Wed, Sep 28, 2011 at 6:28 AM, Joe Armstrong wrote: >> >> 3) Some programs (actually any program) evaluates the BIF >> >> ? ?erlang:load_module(Mod, Bin) >> > > I'm curious, this looks like one place in the entire system that I'd love to > keep an active trace log of over time. ?Since we can load arbitrary byte > code here, it is the most likely place for abuse (both good and bad). > This makes me feel like it might be a half decent idea to implement > http_code_server.erl that uses erlang:load_module(Mod,Bin) to directly load > modules from remote sites, and leave code_server.erl alone for the time > being. ?This way, my default erlang can be "as secure as it ever was" and I > can introduce all sorts of horrific hacks at this layer. > I would still want to extend the module syntax with the -location() bit, but > have it fall back to the http_code_server for loading. > Thoughts? Problem is the http_code_server.erl might load some beam code Z.beam and Z.beam calls erlang:load_module/2 - I think it is possible to analyse the beam code to see if this is true. /Joe > Dave > -- > -=-=-=-=-=-=-=-=-=-=- http://blog.dloh.org/ > From erlang@REDACTED Thu Sep 29 16:29:15 2011 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 29 Sep 2011 16:29:15 +0200 Subject: [erlang-questions] Right direction ? In-Reply-To: References: <7AA05035-2AB7-4CB5-8FF0-A29A6D7455FF@nexttolast.com> Message-ID: On Thu, Sep 29, 2011 at 3:30 PM, David Goehrig wrote: > > > On Thu, Sep 29, 2011 at 3:19 AM, Joe Armstrong wrote: >> >> The level of protection that interests me is: >> >> ? ? a) - protection against shooting yourself in the foot >> ? ? ? ?(ie non-malevolent mistakes) - such as rsyncing the >> ? ? ? ?wrong version of the code >> ? ? ?b) - protection against simple malevolent attacks (password guessing, >> ? ? ? ? exposing plain text passwords etc.) > > > a.) also requires that it be EASY for developers to support it. ?Requiring a > $1000 CA cert just to publish some open source code is a sure way to shoot > yourself in the foot. For a.) to work we need: > * A way for a developer to easily publish a SIGNATURE so that all people > everywhere can download and verify the code against it > * A way for a user to easily verify that a particular bit of code matches > the SIGNATURE from the location where it was acquired > Both of these typically are done by sticking a MD5 sum or SHA1 has in the > directory in a file along side the original file, so that you can verify > that the file has not been modified from the version at that location. ?The > problem here is that if both the package and the signature are tampered > with, you don't necessarily know. You could sign the MD5 sum of the beam with (say) an RSA private key and this could then be validated by anybody who has your public key. Then you only have to distribute your public key :-) /Joe > For these sorts of problems, DNS seems like a good idea in so far as it is a > distributed database that is publicly accessible, and there exist tools for > detecting when someone is messing with it. ?A dets table with a set of SHA1 > hashes of every module loaded that simply refuses to load the module if the > signature changes can be useful. ?A process that periodically checks DNS for > 3rd party modules to notify you of changes can be really useful, as it > becomes a way for a developer to notify you of security patches, bug fixes, > deprecations, and all sorts of other maintenance issues that will threaten > your project over time. > I'm almost done with a proof of concept of such a service. ?Maybe next week > I'll release a public beta. > Dave > -- > -=-=-=-=-=-=-=-=-=-=- http://blog.dloh.org/ > From robert.virding@REDACTED Thu Sep 29 17:49:01 2011 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 29 Sep 2011 16:49:01 +0100 (BST) Subject: [erlang-questions] Right direction ? In-Reply-To: Message-ID: ----- Original Message ----- > On Wed, Sep 28, 2011 at 6:28 AM, Joe Armstrong < erlang@REDACTED > > wrote: > > 3) Some programs (actually any program) evaluates the BIF > > > erlang:load_module(Mod, Bin) > > I'm curious, this looks like one place in the entire system that I'd > love to keep an active trace log of over time. Since we can load > arbitrary byte code here, it is the most likely place for abuse > (both good and bad). > This makes me feel like it might be a half decent idea to implement > http_code_server.erl that uses erlang:load_module(Mod,Bin) to > directly load modules from remote sites, and leave code_server.erl > alone for the time being. This way, my default erlang can be "as > secure as it ever was" and I can introduce all sorts of horrific > hacks at this layer. If you intend that http_code_server to run instead of code_server.erl or together with it? If together then it probably be better if http_code_server used code_server to do the actual code management in the node; having to code servers will definitely lead to trouble. > I would still want to extend the module syntax with the -location() > bit, but have it fall back to the http_code_server for loading. > Thoughts? > Dave Robert -------------- next part -------------- An HTML attachment was scrubbed... URL: From yrashk@REDACTED Thu Sep 29 18:31:12 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Thu, 29 Sep 2011 09:31:12 -0700 Subject: [erlang-questions] Vancouver Erlang Meetup: Oct 6th (One Machine to Rule Them All) Message-ID: Hi, Next Vancouver Erlang meetup will be held in a week, on 6th of October and we will be talking about PropEr's state machine testing: http://www.meetup.com/erlang-vancouver/events/34655632/ If you're still not on meetup.com and therefore missed our meetups in the past, you have no more excuses. Join us next Thursday and learn about statem testing before orks do! Yurii. From james@REDACTED Thu Sep 29 20:27:02 2011 From: james@REDACTED (James Aimonetti) Date: Thu, 29 Sep 2011 11:27:02 -0700 Subject: [erlang-questions] dialyzer -Wunderspecs and the binary() type Message-ID: <4E84B876.8040207@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 We use -Wunderspecs a lot in our Dialyzing and the only minor complaint I have is when I get warnings that the success typing is more specfic with the binary() type. We see a lot of: foo.erl:123: Type specfication foo:a_fun(BinVal) -> 'ok' when is_subtype(BinVal, binary()) is a supertype of the success typing: foo:a_fun(<<_:8,_:_*8>>) -> 'ok'. In foo.erl, a_fun/1 is defined as: - -spec a_fun(BinVal) -> 'ok' when BinVal :: binary(). a_fun(BinVal) -> %% do something with BinVal ok. This is obviously a simplified, trivial example. Just curious if its possible to have Dialyzer *not* output the -Wunderspecs warnings for binaries? Or if there's a better way to specify binary inputs/outputs (since not all of our functions using binary() in the spec have this issue). This is in R14B03, if that makes a difference. Thanks, James - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOhLh2AAoJENc77s1OYoGg9m8H/iiILocPGlrCEwpg1xdVsQEO E6AZbrvjJHSpieJUfCLmuSUH7K3qQxdDPNA1l2S9QIzCm+TmeBSj37GVdW5gj5i3 SGTQoCYEgUmDqJISK7QgmGePms+3PmpN1HDw18Y1QwddXcbznSl05bCgGOhyFKee At7zjLDbuhONzU8gB8aoCdzetSzUxbs0CzNggi7y1qdzLY7POuCVOa6+g8Oo7HY/ kd0490D7By68vlp2ALGcW1fSVZk8Zb5MFd+dA3TFgyXzfEPtKx6xvX/lTzi2COdc 5T3hk1jxCeUkTsvLcUq2ZtrjXzwWhPrDjPoThPMWr3DiO++y6j8TCwV3F/e1tJA= =kdk0 -----END PGP SIGNATURE----- From andrew.pennebaker@REDACTED Thu Sep 29 20:25:36 2011 From: andrew.pennebaker@REDACTED (Andrew Pennebaker) Date: Thu, 29 Sep 2011 14:25:36 -0400 Subject: [erlang-questions] erl, escript, shebangs and syntax errors Message-ID: Please instruct erl to ignore shebangs. $ cat hello.erl #!/usr/bin/env escript -module(hello). -export([main/1]). main(_) -> io:format("Hello World!~n", []).wonko:Desktop andrew$ ./hello.erl Hello World! wonko:Desktop andrew$ erl Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.4 (abort with ^G) 1> c(hello). ./hello.erl:1: syntax error before: '#' ./hello.erl:4: no module definition error Cheers, Andrew Pennebaker www.yellosoft.us -------------- next part -------------- An HTML attachment was scrubbed... URL: From zerthurd@REDACTED Fri Sep 30 03:33:02 2011 From: zerthurd@REDACTED (Maxim Treskin) Date: Fri, 30 Sep 2011 08:33:02 +0700 Subject: [erlang-questions] erl, escript, shebangs and syntax errors In-Reply-To: References: Message-ID: You should not compile this script, just run. Remove first line If you want compile it as regular erlang module. On 30 September 2011 01:25, Andrew Pennebaker wrote: > Please instruct erl to ignore shebangs. > > $ cat hello.erl > #!/usr/bin/env escript > > -module(hello). > -export([main/1]). > > main(_) -> io:format("Hello World!~n", []).wonko:Desktop andrew$ > ./hello.erl > Hello World! > wonko:Desktop andrew$ erl > Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.4 (abort with ^G) > 1> c(hello). > ./hello.erl:1: syntax error before: '#' > ./hello.erl:4: no module definition > error > > Cheers, > > Andrew Pennebaker > www.yellosoft.us > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Maxim Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From zerthurd@REDACTED Fri Sep 30 04:32:08 2011 From: zerthurd@REDACTED (Maxim Treskin) Date: Fri, 30 Sep 2011 09:32:08 +0700 Subject: [erlang-questions] erl, escript, shebangs and syntax errors In-Reply-To: References: Message-ID: Programming is not an intuitive job, people are not programmed from birth, so there is different ways to be intuitive for different people. For example I see nothing intuitive to mix escript code and code for OTP. If you want to use common library in OTP application and escript, put following line after shebang line: %%! -pa path/to/library On 30 September 2011 09:07, Andrew Pennebaker wrote: > No. erl should ignore shebang lines, because that's easy to do in a parser, > and it makes the language more intuitive to program in. > > > Cheers, > > Andrew Pennebaker > www.yellosoft.us > > On Thu, Sep 29, 2011 at 9:33 PM, Maxim Treskin wrote: > >> You should not compile this script, just run. Remove first line If you >> want compile it as regular erlang module. >> >> On 30 September 2011 01:25, Andrew Pennebaker < >> andrew.pennebaker@REDACTED> wrote: >> >>> Please instruct erl to ignore shebangs. >>> >>> $ cat hello.erl >>> #!/usr/bin/env escript >>> >>> -module(hello). >>> -export([main/1]). >>> >>> main(_) -> io:format("Hello World!~n", []).wonko:Desktop andrew$ >>> ./hello.erl >>> Hello World! >>> wonko:Desktop andrew$ erl >>> Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] >>> [async-threads:0] [hipe] [kernel-poll:false] >>> >>> Eshell V5.8.4 (abort with ^G) >>> 1> c(hello). >>> ./hello.erl:1: syntax error before: '#' >>> ./hello.erl:4: no module definition >>> error >>> >>> Cheers, >>> >>> Andrew Pennebaker >>> www.yellosoft.us >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> >> -- >> Maxim Treskin >> > > -- Maxim Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenji.rikitake@REDACTED Fri Sep 30 08:10:30 2011 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Fri, 30 Sep 2011 15:10:30 +0900 Subject: [erlang-questions] FreeBSD PCRE Bugfix Re: FreeBSD patches In-Reply-To: <20110916095213.GA11461@erix.ericsson.se> References: <20110916095213.GA11461@erix.ericsson.se> Message-ID: <20110930061030.GA13216@k2r.org> The CC option sequence change was for correctly linking the PCRE library, I remember. This has been around for at least for a few years and I suggest including this to R15 will really help. See also: https://github.com/jj1bdx/otp/commit/d362e677b5bd75f1aae7798f3b1b923d33cb0dc0 http://erlang.2086793.n4.nabble.com/Make-Erlang-compile-on-recent-FreeBSD-releases-long-td2119832.html Kenji Rikitake ++> Raimo Niskanen [2011-09-16 11:51:50 +0200]: > # cat files/patch-erts-emulator-Makefile.in > > $FreeBSD: ports/lang/erlang/files/patch-erts-emulator-Makefile.in,v 1.3 2008/11/05 23:32:22 olgeni Exp $ > > --- erts/emulator/Makefile.in.orig > +++ erts/emulator/Makefile.in > @@ -544,7 +546,7 @@ > endif > > $(OBJDIR)/%.o: beam/%.c > - $(CC) $(subst -O2, $(GEN_OPT_FLGS), $(CFLAGS)) $(INCLUDES) -c $< -o $@ > + $(CC) $(INCLUDES) $(subst -O2, $(GEN_OPT_FLGS), $(CFLAGS)) -c $< -o $@ > > else From kenji.rikitake@REDACTED Fri Sep 30 08:33:02 2011 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Fri, 30 Sep 2011 15:33:02 +0900 Subject: [erlang-questions] [ANN] Erlang Workshop 2011 Proceeding on ACM Digital Library Message-ID: <20110930063302.GB13216@k2r.org> Proceedings of the 10th ACM SIGPLAN workshop on Erlang is now available in the ACM Digital Library at: http://dl.acm.org/citation.cfm?id=2034654&picked=prox&CFID=51530338&CFTOKEN=66006286 The Workshop had 37 attendants. I appreciate all who participated in the event. ICFP 2012 will be held in Copenhagen, Denmark; so will Erlang Workshop 2012 be. Regards, Kenji Rikitake Erlang Workshop 2011 General Chair From kostis@REDACTED Fri Sep 30 08:41:54 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 30 Sep 2011 09:41:54 +0300 Subject: [erlang-questions] dialyzer -Wunderspecs and the binary() type In-Reply-To: <4E84B876.8040207@2600hz.com> References: <4E84B876.8040207@2600hz.com> Message-ID: <4E8564B2.8090304@cs.ntua.gr> On 09/29/11 21:27, James Aimonetti wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > We use -Wunderspecs a lot in our Dialyzing and the only minor complaint > I have is when I get warnings that the success typing is more specfic > with the binary() type. > > We see a lot of: > > foo.erl:123: Type specfication foo:a_fun(BinVal) -> 'ok' when > is_subtype(BinVal, binary()) is a supertype of the success typing: > foo:a_fun(<<_:8,_:_*8>>) -> 'ok'. > > In foo.erl, a_fun/1 is defined as: > > - -spec a_fun(BinVal) -> 'ok' when > BinVal :: binary(). > a_fun(BinVal) -> > %% do something with BinVal > ok. > > This is obviously a simplified, trivial example. It's not just a simplified example. It's an example that is incomplete because it does not show the problem. Here is one that does: -module(foo). -export([a_fun/1]). -spec a_fun(binary()) -> 'ok'. a_fun(B) -> do(B), ok. do(<<_>>) -> ok; do(<<_,R/binary>>) -> do(R). Note that the first clause of do/1 has a non-empty binary there (there is an underscore). From this clause dialyzer will infer that the do/1 function only returns if its argument is a non-empty binary of at least one byte. > Just curious if its > possible to have Dialyzer *not* output the -Wunderspecs warnings for > binaries? Why on earth would we want to special case this? > Or if there's a better way to specify binary inputs/outputs > (since not all of our functions using binary() in the spec have this issue). As I explained, the warning is related to the fact that these functions only accept non-empty binaries. (Whether this is intentional or not in your code I will let you decide.) To suppress these warnings I recommend the following: define a non-empty binary type -type ne_binary() :: <<_:8,_:_*8>>. and change the spec of your a_fun above to: -spec a_fun(ne_binary()) -> 'ok'. (Aside: I do not see the need to use 'when' for type variables that only appear once in a spec. The non when form is shorter and nicer.) Kostis From zabrane3@REDACTED Fri Sep 30 11:38:25 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Fri, 30 Sep 2011 11:38:25 +0200 Subject: [erlang-questions] Yaws installer for Windows? Message-ID: <9C6C2717-0C20-4767-B220-726385814E15@gmail.com> Hi Claes & Steve, Could you please explain how did you proceed to create an installer for Yaws (yaws-xyz.exe) on Windows : http://yaws.hyber.org/download/ Is the process documented somewhere? Thanks in advance. Regards, Zabrane From ola.a.andersson@REDACTED Fri Sep 30 13:57:42 2011 From: ola.a.andersson@REDACTED (Ola Andersson A) Date: Fri, 30 Sep 2011 13:57:42 +0200 Subject: [erlang-questions] Common Test Hooks Message-ID: Hi CT users, I wonder if anyone else has tried out the relatively recent addition of Common Test Hooks in Common Test? In the project I'm working in we have used it successfully for about six months now. I think it's a very powerful tool for creating behaviours that are common to multiple test suites. We also use CT Hooks to start up and supervise resources and to perform cleanup and recovery actions after failed test cases. It saves the testers a lot of work. The CT Hooks functionality is still in alpha vsn according to OTP R14B03 but I think it's stable enough to go ahead and use it. We have had no problems at all during these six months. If anyone else out there has used CT Hooks I would appreciate if you could share your experiences and maybe provide some new ideas about creative usage of the feature. BR, /OLA. From seand-erlang@REDACTED Fri Sep 30 14:17:52 2011 From: seand-erlang@REDACTED (SeanD) Date: Fri, 30 Sep 2011 13:17:52 +0100 Subject: [erlang-questions] Query about services created by Erlsrv Message-ID: <20110930121751.GA17574@mryaffle.seand.me.uk> Hi all, I am writing an application that is deployed on Windows environments using erlsrv which works very nicely. However, the service that Erlsrv creates has a dependency on the Windows Workstation service. If I manually remove this dependency and disable the workstation service, everything seems to run fine, however I imagine the dependency was put in for a reason. There are no comments in the code to suggest why this may be the case, and I was wondering if anyone here knew of any scenarios where things might start to fail because the Workstation service isn't running. (My app isn't using SMB or similar for any of it's communications) Thanks in advance, Sean From lukas@REDACTED Fri Sep 30 14:48:08 2011 From: lukas@REDACTED (Lukas Larsson) Date: Fri, 30 Sep 2011 14:48:08 +0200 Subject: [erlang-questions] Common Test Hooks In-Reply-To: References: Message-ID: <4E85BA88.3090500@erlang.org> Hi Ola! Just letting you and everyone else know that there will be a small interface change (return value of init callback) in R14B04 with CTHs in order to allow priority inbetween CTHs. I have also planned to extend the priority feature to be more flexible duing execution of a suite so that one could run hooks in one order during init_per_* and another during end_per_*. Besides this change nothing more is planned right now, though suggestions on improvements are more than welcome! CTH will remain alpha until the R15 release and then be put in final version. Regarding creative uses of hooks, we at OTP use them to compile C code in SUITE_data directories which are then used to test various interface functionality within the erlang emulator. Lukas Erlang/OTP team On 30/09/11 13:57, Ola Andersson A wrote: > Hi CT users, > > I wonder if anyone else has tried out the relatively recent addition of Common Test Hooks in Common Test? > In the project I'm working in we have used it successfully for about six months now. > I think it's a very powerful tool for creating behaviours that are common to multiple test suites. > We also use CT Hooks to start up and supervise resources and to perform cleanup and recovery actions after > failed test cases. It saves the testers a lot of work. > > The CT Hooks functionality is still in alpha vsn according to OTP R14B03 but I think it's stable enough > to go ahead and use it. We have had no problems at all during these six months. > > If anyone else out there has used CT Hooks I would appreciate if you could share your experiences and maybe > provide some new ideas about creative usage of the feature. > BR, > /OLA. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From erlang@REDACTED Fri Sep 30 15:04:35 2011 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 30 Sep 2011 15:04:35 +0200 Subject: [erlang-questions] crypto in erlang and javascript Message-ID: Hello, I'm looking for "pairs" of crypto algorithms. I'm making a web authentication framework and I want crypto algorithms written Javascript running in the browser to interact with Erlang versions written in the server. So far I have got RSA and MD5 running in both JS and Erlang I now want a decent symmetric encryption algorithm. Any ideas? I want both side to be reasonably efficient with non-restrictive licenses. /Joe From wmacgyver@REDACTED Fri Sep 30 15:08:56 2011 From: wmacgyver@REDACTED (Wilson MacGyver) Date: Fri, 30 Sep 2011 09:08:56 -0400 Subject: [erlang-questions] crypto in erlang and javascript In-Reply-To: References: Message-ID: <4144331E-90C7-4A6F-8956-FCF8B2CB449D@gmail.com> Try blowfish http://dren.ch/js_blowfish/ On Sep 30, 2011, at 9:04 AM, Joe Armstrong wrote: > Hello, > > I'm looking for "pairs" of crypto algorithms. I'm making a > web authentication framework and I want > crypto algorithms written Javascript running in the browser > to interact with Erlang versions written in the server. > > So far I have got RSA and MD5 running in both JS and Erlang > > I now want a decent symmetric encryption algorithm. > > Any ideas? > > I want both side to be reasonably efficient with non-restrictive > licenses. > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dave@REDACTED Fri Sep 30 16:26:45 2011 From: dave@REDACTED (David Goehrig) Date: Fri, 30 Sep 2011 10:26:45 -0400 Subject: [erlang-questions] crypto in erlang and javascript In-Reply-To: References: Message-ID: <11F74076-2209-489E-8568-9E77B3BB9D32@nexttolast.com> Crypto-js is something I've personally used in production for 3 years: http://code.google.com/p/crypto-js/ Has aes and DES and supports one way block cyphers like HMAC256 which is great for request signing. Dave -=-=- dave@REDACTED -=-=- On Sep 30, 2011, at 9:04 AM, Joe Armstrong wrote: > Hello, > > I'm looking for "pairs" of crypto algorithms. I'm making a > web authentication framework and I want > crypto algorithms written Javascript running in the browser > to interact with Erlang versions written in the server. > > So far I have got RSA and MD5 running in both JS and Erlang > > I now want a decent symmetric encryption algorithm. > > Any ideas? > > I want both side to be reasonably efficient with non-restrictive > licenses. > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ali.sabil@REDACTED Fri Sep 30 16:34:10 2011 From: ali.sabil@REDACTED (Ali Sabil) Date: Fri, 30 Sep 2011 16:34:10 +0200 Subject: [erlang-questions] crypto in erlang and javascript In-Reply-To: <11F74076-2209-489E-8568-9E77B3BB9D32@nexttolast.com> References: <11F74076-2209-489E-8568-9E77B3BB9D32@nexttolast.com> Message-ID: This could be an interesting read on Javascript cryptography: http://www.matasano.com/articles/javascript-cryptography/ On Fri, Sep 30, 2011 at 4:26 PM, David Goehrig wrote: > Crypto-js is something I've personally used in production for 3 years: > > http://code.google.com/p/crypto-js/ > > Has aes and DES and supports one way block cyphers like HMAC256 which is great for request signing. > > Dave > > -=-=- dave@REDACTED -=-=- > > On Sep 30, 2011, at 9:04 AM, Joe Armstrong wrote: > >> Hello, >> >> I'm looking for "pairs" of crypto algorithms. I'm making a >> web authentication framework and I want >> crypto algorithms written Javascript running in the browser >> to interact with Erlang versions written in the server. >> >> So far I have got RSA and MD5 running in both JS and Erlang >> >> I now want a decent symmetric encryption algorithm. >> >> Any ideas? >> >> I want both side to be reasonably efficient with non-restrictive >> licenses. >> >> /Joe >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ola.a.andersson@REDACTED Fri Sep 30 16:41:00 2011 From: ola.a.andersson@REDACTED (Ola Andersson A) Date: Fri, 30 Sep 2011 16:41:00 +0200 Subject: [erlang-questions] Common Test Hooks In-Reply-To: <4E85BA88.3090500@erlang.org> References: <4E85BA88.3090500@erlang.org> Message-ID: Thanks for the info Lukas! We will probably go directly for R14B04 at the next upgrade so that's good to know. By the way, we also recently added a hook that compiles C code. In our case it's test cases that are executed on the target node. The code is compiled and transferred to the node with ftp, then executed from the erlang suite with some hidden signalling. All magically provided by the CTH. /OLA. > -----Original Message----- > From: erlang-questions-bounces@REDACTED > [mailto:erlang-questions-bounces@REDACTED] On Behalf Of > Lukas Larsson > Sent: den 30 september 2011 14:48 > To: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Common Test Hooks > > Hi Ola! > > Just letting you and everyone else know that there will be a > small interface change (return value of init callback) in > R14B04 with CTHs in order to allow priority inbetween CTHs. I > have also planned to extend the priority feature to be more > flexible duing execution of a suite so that one could run > hooks in one order during init_per_* and another during > end_per_*. Besides this change nothing more is planned right > now, though suggestions on improvements are more than welcome! > > CTH will remain alpha until the R15 release and then be put > in final version. > > Regarding creative uses of hooks, we at OTP use them to > compile C code in SUITE_data directories which are then used > to test various interface functionality within the erlang emulator. > > Lukas > Erlang/OTP team > > On 30/09/11 13:57, Ola Andersson A wrote: > > Hi CT users, > > > > I wonder if anyone else has tried out the relatively recent > addition of Common Test Hooks in Common Test? > > In the project I'm working in we have used it successfully > for about six months now. > > I think it's a very powerful tool for creating behaviours > that are common to multiple test suites. > > We also use CT Hooks to start up and supervise resources and to > > perform cleanup and recovery actions after failed test > cases. It saves the testers a lot of work. > > > > The CT Hooks functionality is still in alpha vsn according to OTP > > R14B03 but I think it's stable enough to go ahead and use > it. We have had no problems at all during these six months. > > > > If anyone else out there has used CT Hooks I would > appreciate if you > > could share your experiences and maybe provide some new > ideas about creative usage of the feature. > > BR, > > /OLA. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From thomasl_erlang@REDACTED Fri Sep 30 17:14:11 2011 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Fri, 30 Sep 2011 08:14:11 -0700 (PDT) Subject: [erlang-questions] "Erlang ETS tables and software transactional memory: how transactions make ETS tables more like ordinary actors" Message-ID: <1317395651.61440.YahooMailNeo@web111406.mail.gq1.yahoo.com> This paper from this year's Erlang Workshop sounds kind of interesting: Patrik Nyblom, Erlang ETS tables and software transactional memory: how transactions make ETS tables more like ordinary actors is there an ungated version available somewhere? Thanks. Best regards, Thomas From filippo.pacini@REDACTED Fri Sep 30 17:49:00 2011 From: filippo.pacini@REDACTED (filippo pacini) Date: Fri, 30 Sep 2011 17:49:00 +0200 Subject: [erlang-questions] crypto in erlang and javascript In-Reply-To: References: Message-ID: Check also this one: http://crypto.stanford.edu/sjcl/ filippo On Fri, Sep 30, 2011 at 3:04 PM, Joe Armstrong wrote: > Hello, > > I'm looking for "pairs" of crypto algorithms. I'm making a > web authentication framework and I want > crypto algorithms written Javascript running in the browser > to interact with Erlang versions written in the server. > > So far I have got RSA and MD5 running in both JS and Erlang > > I now want a decent symmetric encryption algorithm. > > Any ideas? > > I want both side to be reasonably efficient with non-restrictive > licenses. > > /Joe From erlang@REDACTED Fri Sep 30 17:57:50 2011 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 30 Sep 2011 17:57:50 +0200 Subject: [erlang-questions] crypto in erlang and javascript In-Reply-To: References: Message-ID: On Fri, Sep 30, 2011 at 4:38 PM, wrote: > Joe, > > On Sep 30, 2011, at 9:04 AM, Joe Armstrong wrote: > >> Hello, >> >> I'm looking for "pairs" of crypto algorithms. I'm making a >> web authentication framework and I want >> crypto algorithms written Javascript running in the browser >> to interact with Erlang versions written in the server. >> >> So far I have got RSA and MD5 running in both JS and Erlang >> >> I now want a decent symmetric encryption algorithm. > > How will you deliver the secret key to the browser such that the JS can encrypt securely for some period of time? I won't - The following seems ok 1) the browser gets the RSA public key of the server. This is hard wired or "well known" 2) the browser generates a random session key and encrypts it with the server's public key. 3) the encrypted session key is sent to the server 4) Only the server can decrypt this key 5) both sides use the session key > If you trust the server to deliver crypto code + key, why not trust the server to do > SSL/TLS which will require less new code? Because I haven't implemented SSL myself :-) - it's an opportunity to learn a bit more about number theory. /Joe > > - John > >> >> Any ideas? >> >> I want both side to be reasonably efficient with non-restrictive >> licenses. >> >> /Joe >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From john@REDACTED Fri Sep 30 18:10:18 2011 From: john@REDACTED (John Kemp) Date: Fri, 30 Sep 2011 12:10:18 -0400 Subject: [erlang-questions] crypto in erlang and javascript In-Reply-To: References: Message-ID: <82626698-72FB-4DFF-AF4F-5FF1B5C9EA20@jkemp.net> Joe, On Sep 30, 2011, at 11:57 AM, ext Joe Armstrong wrote: >> >> How will you deliver the secret key to the browser such that the JS can encrypt securely for some period of time? > > I won't - The following seems ok > > 1) the browser gets the RSA public key of the server. This is hard wired > or "well known" > > 2) the browser generates a random session key and encrypts it with > the server's public key. > > 3) the encrypted session key is sent to the server > > 4) Only the server can decrypt this key > > 5) both sides use the session key > >> If you trust the server to deliver crypto code + key, why not trust the server to do >> SSL/TLS which will require less new code? > > Because I haven't implemented SSL myself :-) Well, that is what you're doing, based on the steps you write above, but presumably with just the key parts, and no CAs or certs involved ;) > - it's an opportunity to > learn a bit more > about number theory. As long as you _want_ to re-implement SSL/TLS, then? enjoy! Cheers, - John > > /Joe > > > >> >> - John >> >>> >>> Any ideas? >>> >>> I want both side to be reasonably efficient with non-restrictive >>> licenses. >>> >>> /Joe >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> From dave@REDACTED Fri Sep 30 19:07:46 2011 From: dave@REDACTED (David Goehrig) Date: Fri, 30 Sep 2011 13:07:46 -0400 Subject: [erlang-questions] crypto in erlang and javascript In-Reply-To: References: <11F74076-2209-489E-8568-9E77B3BB9D32@nexttolast.com> Message-ID: <8AB16167-0025-4A37-8B20-3BD89FD7AD44@nexttolast.com> I've read this but disagree with the basic premise. The author assumes the purpose of crypto is to secure the system. It is not. The purpose of crypto is to increase the cost beyond the point of reasonable benefit for the effort. Yes I may be able to inject JS to defeat your crypto scheme, but so what! If I can inject JS code I can grab all of your keystrokes and log them to my server. I on the otherhand don't want to store your cats name in my DB, and store it all over my server logs. You know that cats name you use on all your accounts. -=-=- dave@REDACTED -=-=- On Sep 30, 2011, at 10:34 AM, Ali Sabil wrote: > This could be an interesting read on Javascript cryptography: > http://www.matasano.com/articles/javascript-cryptography/ > > On Fri, Sep 30, 2011 at 4:26 PM, David Goehrig wrote: >> Crypto-js is something I've personally used in production for 3 years: >> >> http://code.google.com/p/crypto-js/ >> >> Has aes and DES and supports one way block cyphers like HMAC256 which is great for request signing. >> >> Dave >> >> -=-=- dave@REDACTED -=-=- >> >> On Sep 30, 2011, at 9:04 AM, Joe Armstrong wrote: >> >>> Hello, >>> >>> I'm looking for "pairs" of crypto algorithms. I'm making a >>> web authentication framework and I want >>> crypto algorithms written Javascript running in the browser >>> to interact with Erlang versions written in the server. >>> >>> So far I have got RSA and MD5 running in both JS and Erlang >>> >>> I now want a decent symmetric encryption algorithm. >>> >>> Any ideas? >>> >>> I want both side to be reasonably efficient with non-restrictive >>> licenses. >>> >>> /Joe >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From vinoski@REDACTED Fri Sep 30 19:36:47 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Fri, 30 Sep 2011 13:36:47 -0400 Subject: [erlang-questions] Yaws installer for Windows? In-Reply-To: <9C6C2717-0C20-4767-B220-726385814E15@gmail.com> References: <9C6C2717-0C20-4767-B220-726385814E15@gmail.com> Message-ID: On Fri, Sep 30, 2011 at 5:38 AM, Zabrane Mickael wrote: > Hi Claes & Steve, > > Could you please explain how did you proceed to create an installer for Yaws (yaws-xyz.exe) on Windows : > http://yaws.hyber.org/download/ > > Is the process documented somewhere? Hi Mickael, I stay completely away from Windows and Klacke handles all the Yaws releases, so he'll have to give you the details. --steve From zabrane3@REDACTED Fri Sep 30 20:04:26 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Fri, 30 Sep 2011 20:04:26 +0200 Subject: [erlang-questions] Yaws installer for Windows? In-Reply-To: References: <9C6C2717-0C20-4767-B220-726385814E15@gmail.com> Message-ID: <74C99D37-6767-4E58-9BA4-B7978B97C64F@gmail.com> Hi guys, On Sep 30, 2011, at 7:36 PM, Steve Vinoski wrote: > On Fri, Sep 30, 2011 at 5:38 AM, Zabrane Mickael wrote: >> Hi Claes & Steve, >> >> Could you please explain how did you proceed to create an installer for Yaws (yaws-xyz.exe) on Windows : >> http://yaws.hyber.org/download/ >> >> Is the process documented somewhere? > > Hi Mickael, I stay completely away from Windows I understand ;-) > and Klacke handles all > the Yaws releases, so he'll have to give you the details. Ok, I'll wait his answer. Regards, Zabrane From john.hughes@REDACTED Fri Sep 30 20:10:44 2011 From: john.hughes@REDACTED (John Hughes) Date: Fri, 30 Sep 2011 20:10:44 +0200 Subject: [erlang-questions] Nitrogen under Windows Message-ID: Does anyone actually have Nitrogen running under Windows? My son would like to use it for a school project, but it seems dashed hard to get it work, and Googling only seems to find people having problems... John -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Fri Sep 30 20:45:33 2011 From: comptekki@REDACTED (Wes James) Date: Fri, 30 Sep 2011 12:45:33 -0600 Subject: [erlang-questions] wx on lion with erlang-solutions version? Message-ID: On this site: http://www.erlang-solutions.com/section/132/erlang-otp-packages it has different erlang distros available. It mentions wx not being available on windows, but it doesn't say that for OS X Lion. I installed the Lion version and tried wx:demo(), but it failed with a no driver error. I didn't think there was a wx for Lion, yet. Does someone else have wx working on Lion? If so, how did you get it installed? Thanks, -wes