From erlang@REDACTED Mon Apr 1 09:48:43 2013 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 1 Apr 2013 09:48:43 +0200 Subject: [erlang-questions] New Feature added to Erlang Message-ID: http://joearms.github.com/2013/04/01/too-big-to-fail.html /Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Apr 1 10:16:06 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 1 Apr 2013 12:16:06 +0400 Subject: [erlang-questions] New Feature added to Erlang In-Reply-To: References: Message-ID: Nice joke, of course, but still the main reason for erlang application to fail is error_logger process. Every blog post about erlang starts from word "it is rock solid, everything is restarted", but when buggy error_logger is killed after eating 10 Gig of RAM, then whole erlang system is down. I don't understand what is the problem in restarting error_logger -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon@REDACTED Mon Apr 1 10:19:33 2013 From: jon@REDACTED (Jon Schneider) Date: Mon, 01 Apr 2013 09:19:33 +0100 Subject: [erlang-questions] New Feature added to Erlang In-Reply-To: References: Message-ID: Thanks Joe, I feel I should point out that this feature will work particularly well on Linux where the OOM killer already targets random small processes rather than the obviously errant one. Jon Joe Armstrong wrote: >http://joearms.github.com/2013/04/01/too-big-to-fail.html > >/Joe > > >------------------------------------------------------------------------ > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From rtrlists@REDACTED Mon Apr 1 10:51:14 2013 From: rtrlists@REDACTED (Robert Raschke) Date: Mon, 1 Apr 2013 09:51:14 +0100 Subject: [erlang-questions] New Feature added to Erlang In-Reply-To: References: Message-ID: Hi Max, it's the sasl logging that includes compete state info on a process crash that can cause the out of memory situation. The error_logger itself is not at fault. Of course, the too_big_to_fail flag will get around the sasl deficiency. Roll on R18. Robby On Apr 1, 2013 9:16 AM, "Max Lapshin" wrote: > Nice joke, of course, but still the main reason for erlang application to > fail is error_logger process. > > Every blog post about erlang starts from word "it is rock solid, > everything is restarted", but when buggy error_logger is killed after > eating 10 Gig of RAM, then whole erlang system is down. > > I don't understand what is the problem in restarting error_logger > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Apr 1 10:56:40 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 1 Apr 2013 12:56:40 +0400 Subject: [erlang-questions] New Feature added to Erlang In-Reply-To: References: Message-ID: I don't even start sasl application =) -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Apr 1 10:57:27 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 1 Apr 2013 12:57:27 +0400 Subject: [erlang-questions] New Feature added to Erlang In-Reply-To: References: Message-ID: Problem is that system goes down if error_logger is killed. Why? Why not just to restart it? -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Apr 1 11:23:01 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 1 Apr 2013 13:23:01 +0400 Subject: [erlang-questions] Dynamically add +A threads Message-ID: In some cases, I need +A 100, in some cases it is not required. Is it possible to change this number in runtime? -------------- next part -------------- An HTML attachment was scrubbed... URL: From wrr@REDACTED Mon Apr 1 13:21:50 2013 From: wrr@REDACTED (Jan Wrobel) Date: Mon, 1 Apr 2013 13:21:50 +0200 Subject: [erlang-questions] New Feature added to Erlang In-Reply-To: References: Message-ID: Help! I've patched this and run on a small Amazon ec2 cluster. Everything worked fine until it started spreading. It somehow got access to the Amazon management console and is allocating new machines. It is already at 1000 Extra Large instances with monthly cost of 370k$! I need to stop this, but it changed the management console password! Wait it is now on my laptop! And my phone! How can I kill this, please he... On Mon, Apr 1, 2013 at 9:48 AM, Joe Armstrong wrote: > http://joearms.github.com/2013/04/01/too-big-to-fail.html > > /Joe > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From dmkolesnikov@REDACTED Mon Apr 1 13:27:21 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Mon, 1 Apr 2013 14:27:21 +0300 Subject: [erlang-questions] New Feature added to Erlang In-Reply-To: References: Message-ID: Sure! Here is an instruction: http://devopsreactions.tumblr.com/post/39647530743/kill-9-1 - Dmitry On Apr 1, 2013, at 2:21 PM, Jan Wrobel wrote: > Help! > > I've patched this and run on a small Amazon ec2 cluster. Everything > worked fine until it started spreading. It somehow got access to the > Amazon management console and is allocating new machines. It is > already at 1000 Extra Large instances with monthly cost of 370k$! I > need to stop this, but it changed the management console password! > Wait it is now on my laptop! And my phone! How can I kill this, please > he... > > On Mon, Apr 1, 2013 at 9:48 AM, Joe Armstrong wrote: >> http://joearms.github.com/2013/04/01/too-big-to-fail.html >> >> /Joe >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mats.westin@REDACTED Mon Apr 1 16:17:29 2013 From: mats.westin@REDACTED (Mats) Date: Mon, 1 Apr 2013 16:17:29 +0200 Subject: [erlang-questions] New Feature added to Erlang In-Reply-To: References: Message-ID: It's a clear violation of the isolation principle! Nodes will be happy until they learn about network latency and then they will stop using memory from TMALR(The Memory Allocator of Last Resort). Already affected nodes will free it back to TMALR and a lot of memory will be unused. To make the IT buyer happy, data structures will be made bigger in next release. Everyone will think it's working until they notice memory bandwidth is saturated (processes start using wheelbarrow to just get a diod flashing). All this mess could be avoided if we start using binarycoin. On Mon, Apr 1, 2013 at 1:27 PM, Dmitry Kolesnikov wrote: > Sure! > > Here is an instruction: > http://devopsreactions.tumblr.com/post/39647530743/kill-9-1 > > - Dmitry > > On Apr 1, 2013, at 2:21 PM, Jan Wrobel wrote: > > > Help! > > > > I've patched this and run on a small Amazon ec2 cluster. Everything > > worked fine until it started spreading. It somehow got access to the > > Amazon management console and is allocating new machines. It is > > already at 1000 Extra Large instances with monthly cost of 370k$! I > > need to stop this, but it changed the management console password! > > Wait it is now on my laptop! And my phone! How can I kill this, please > > he... > > > > On Mon, Apr 1, 2013 at 9:48 AM, Joe Armstrong wrote: > >> http://joearms.github.com/2013/04/01/too-big-to-fail.html > >> > >> /Joe > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Mon Apr 1 16:30:51 2013 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 1 Apr 2013 10:30:51 -0400 Subject: [erlang-questions] New Feature added to Erlang In-Reply-To: References: Message-ID: <20130401143050.GA97906@ferdmbp.local> I would recommend making a better use of the network in such cases. There are various options available: use the socket buffers as a storage mechanism. Each port can be mapped to specific memory addresses and pushing data on there (and subsequently reading and pushing back data on there) can be used to offload some state. Moreover, I would recommend each Erlang node has a 'ping' interface, or a delayed ping interface. Overloading memory can be sent over the network that way, and the delayed ping will make use of other nodes' buffers and also the actual cables and connections between computers as an 'in-flight' memory storage. This is similar to old lamps from back in the day where you had a given time interval worth of data to store in the electrical loop going through the lamp. In this day and age with the cloud and multi-datacenter connections, it becomes excessively cheap to achieve high latencies between data centers. Using this, and computers purposedly placed far away as to create latency, it is possible to store data in flight over the network. I would recommend writing a new RFC that has ISPs and other providers placing Internet nodes at various strategic points whose sole purpose will be to mirror back data so that various services can overflow their memory to the network. Regards, Fred. On 04/01, Joe Armstrong wrote: > http://joearms.github.com/2013/04/01/too-big-to-fail.html > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From lee.sylvester@REDACTED Mon Apr 1 22:05:47 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Mon, 1 Apr 2013 21:05:47 +0100 Subject: [erlang-questions] Secure Tokens Message-ID: <57A08917-A253-44AF-9865-440651D8C131@gmail.com> Hey guys, So, I'd like to create secure tokens in Erlang. This can either be a simple UUID generator which I then store with user credentials or a way to encode a string, such as JSON, as an encrypted token. In Golang, I would do this with fernet, but I need an Erlang solution :-) I know Erlang isn't best used for such tasks, but does anyone out there know of something usable for this purpose? Thanks loads, Lee From jeremy@REDACTED Mon Apr 1 22:07:56 2013 From: jeremy@REDACTED (Jeremy Ong) Date: Mon, 1 Apr 2013 13:07:56 -0700 Subject: [erlang-questions] Secure Tokens In-Reply-To: <57A08917-A253-44AF-9865-440651D8C131@gmail.com> References: <57A08917-A253-44AF-9865-440651D8C131@gmail.com> Message-ID: Look at bcrypt or crypto. On Mon, Apr 1, 2013 at 1:05 PM, Lee Sylvester wrote: > Hey guys, > > So, I'd like to create secure tokens in Erlang. This can either be a simple UUID generator which I then store with user credentials or a way to encode a string, such as JSON, as an encrypted token. In Golang, I would do this with fernet, but I need an Erlang solution :-) > > I know Erlang isn't best used for such tasks, but does anyone out there know of something usable for this purpose? > > Thanks loads, > Lee > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From lee.sylvester@REDACTED Mon Apr 1 22:08:47 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Mon, 1 Apr 2013 21:08:47 +0100 Subject: [erlang-questions] Secure Tokens In-Reply-To: References: <57A08917-A253-44AF-9865-440651D8C131@gmail.com> Message-ID: Wow, thank you Jeremy, that was fast!!! Thank you, I will. :-) Lee On 1 Apr 2013, at 21:07, Jeremy Ong wrote: > Look at bcrypt or crypto. > > On Mon, Apr 1, 2013 at 1:05 PM, Lee Sylvester wrote: >> Hey guys, >> >> So, I'd like to create secure tokens in Erlang. This can either be a simple UUID generator which I then store with user credentials or a way to encode a string, such as JSON, as an encrypted token. In Golang, I would do this with fernet, but I need an Erlang solution :-) >> >> I know Erlang isn't best used for such tasks, but does anyone out there know of something usable for this purpose? >> >> Thanks loads, >> Lee >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From bob@REDACTED Mon Apr 1 22:15:34 2013 From: bob@REDACTED (Bob Ippolito) Date: Mon, 1 Apr 2013 13:15:34 -0700 Subject: [erlang-questions] Secure Tokens In-Reply-To: <57A08917-A253-44AF-9865-440651D8C131@gmail.com> References: <57A08917-A253-44AF-9865-440651D8C131@gmail.com> Message-ID: There's something similar to your requirements in here: https://github.com/mochi/mochiweb/blob/master/src/mochiweb_session.erl On Mon, Apr 1, 2013 at 1:05 PM, Lee Sylvester wrote: > Hey guys, > > So, I'd like to create secure tokens in Erlang. This can either be a > simple UUID generator which I then store with user credentials or a way to > encode a string, such as JSON, as an encrypted token. In Golang, I would > do this with fernet, but I need an Erlang solution :-) > > I know Erlang isn't best used for such tasks, but does anyone out there > know of something usable for this purpose? > > Thanks loads, > Lee > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan@REDACTED Tue Apr 2 00:14:36 2013 From: stefan@REDACTED (Stefan Jahn) Date: Tue, 2 Apr 2013 00:14:36 +0200 Subject: [erlang-questions] ssh eow In-Reply-To: References: <8d026d1d272a84accac74056e8bfbb2d.squirrel@service.rules.org> <4f9c5ac539ef1c35325a412a2a9c46ec.squirrel@service.rules.org> <36a480137a631a0cb8b65c60864e6fac.squirrel@service.rules.org> Message-ID: hello, when I try ssh_connection_manager:request(State#state.cm, State#state.channel, "eow@REDACTED", false, [?boolean(false)], 0), after all data delivered, I get a data integrity error from scp. This is not the correct way to introduce new messages, isn't it? Thank you in advance, Stefan. On Wed, March 27, 2013 2:13 am, Stefan Jahn wrote: > hello, > > while implementing a scp daemon i wonder how to send the eow event > to the client in order to tell the scp command to finish expecting > data from my scp daemon... > > see in PROTOCOL (from openssh): > 2.1. connection: Channel write close extension "eow@REDACTED" > > is there a function in ssh api of erlang how i can send this protocol > item?? > > thank you in advance, > stefan. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From dronnikov@REDACTED Tue Apr 2 07:09:19 2013 From: dronnikov@REDACTED (Vladimir Dronnikov) Date: Tue, 2 Apr 2013 09:09:19 +0400 Subject: [erlang-questions] Secure Tokens In-Reply-To: References: <57A08917-A253-44AF-9865-440651D8C131@gmail.com> Message-ID: I drive https://github.com/dvv/termit for this. Feel free to feedback/blame :) On Tue, Apr 2, 2013 at 12:15 AM, Bob Ippolito wrote: > There's something similar to your requirements in here: > https://github.com/mochi/mochiweb/blob/master/src/mochiweb_session.erl > > > On Mon, Apr 1, 2013 at 1:05 PM, Lee Sylvester wrote: > >> Hey guys, >> >> So, I'd like to create secure tokens in Erlang. This can either be a >> simple UUID generator which I then store with user credentials or a way to >> encode a string, such as JSON, as an encrypted token. In Golang, I would >> do this with fernet, but I need an Erlang solution :-) >> >> I know Erlang isn't best used for such tasks, but does anyone out there >> know of something usable for this purpose? >> >> Thanks loads, >> Lee >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvg.mailing@REDACTED Tue Apr 2 07:22:10 2013 From: rvg.mailing@REDACTED (Rudolph van Graan) Date: Tue, 02 Apr 2013 06:22:10 +0100 Subject: [erlang-questions] Erlang Type specifications and throw Message-ID: <675F7D33-1F95-4EEA-9842-04BD45D8C881@me.com> Hi, I want to write a type spec for a function that throws exceptions, something like this: > -spec where(G :: atom()) -> pid() | throw({no_name,atom()}). I don't want to use an exit signal, because this behaviour is intentional and the calling process will catch this and take specific action. But I can't write this using the -spec syntax. The documentation says: > Some functions in Erlang are not meant to return; either because they define servers or because they are used to throw exceptions as the function below: > my_error(Err) -> erlang:throw({error, Err}). > For such functions we recommend the use of the special no_return() type for their "return", via a contract of the form: > > -spec my_error(term()) -> no_return(). no_return() is completely the wrong thing to use and just leaving out the throw(?) declaration is also wrong in my eyes. How do I do this? Thanks, Rudolph van Graan -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Tue Apr 2 07:37:01 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 2 Apr 2013 06:37:01 +0100 Subject: [erlang-questions] Secure Tokens In-Reply-To: References: <57A08917-A253-44AF-9865-440651D8C131@gmail.com> Message-ID: <639C85DA-F075-455E-ACD0-930EE2F9A7BD@gmail.com> Wow, a fernet like impl for Erlang!!! Perfect!!! Thank you very much. This will make my life so much easier :-) Regards, Lee On 2 Apr 2013, at 06:09, Vladimir Dronnikov wrote: > I drive https://github.com/dvv/termit for this. Feel free to feedback/blame :) > > > On Tue, Apr 2, 2013 at 12:15 AM, Bob Ippolito wrote: > There's something similar to your requirements in here: > https://github.com/mochi/mochiweb/blob/master/src/mochiweb_session.erl > > > On Mon, Apr 1, 2013 at 1:05 PM, Lee Sylvester wrote: > Hey guys, > > So, I'd like to create secure tokens in Erlang. This can either be a simple UUID generator which I then store with user credentials or a way to encode a string, such as JSON, as an encrypted token. In Golang, I would do this with fernet, but I need an Erlang solution :-) > > I know Erlang isn't best used for such tasks, but does anyone out there know of something usable for this purpose? > > Thanks loads, > Lee > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Tue Apr 2 07:50:29 2013 From: bob@REDACTED (Bob Ippolito) Date: Mon, 1 Apr 2013 22:50:29 -0700 Subject: [erlang-questions] Secure Tokens In-Reply-To: <639C85DA-F075-455E-ACD0-930EE2F9A7BD@gmail.com> References: <57A08917-A253-44AF-9865-440651D8C131@gmail.com> <639C85DA-F075-455E-ACD0-930EE2F9A7BD@gmail.com> Message-ID: No reason to celebrate just yet, termit has a broken cryptosystem. Here's two things I noticed after a quick glance: * The IV is derived from the secret key. The IV must be unpredictable at encryption time in CBC mode. This is VERY VERY bad. * Verification of the signature isn't constant-time, so it's susceptible to timing attacks. This is still bad, but probably harder to exploit. It would be unwise to use this implementation. I don't claim that the mochiweb code is perfect, and I'm not a cryptograph expert, but I have audited it and I didn't find any obvious flaws (other than the timing attack that I fixed). On Mon, Apr 1, 2013 at 10:37 PM, Lee Sylvester wrote: > Wow, a fernet like impl for Erlang!!! Perfect!!! Thank you very much. > This will make my life so much easier :-) > > Regards, > Lee > > > > On 2 Apr 2013, at 06:09, Vladimir Dronnikov wrote: > > I drive https://github.com/dvv/termit for this. Feel free to > feedback/blame :) > > > On Tue, Apr 2, 2013 at 12:15 AM, Bob Ippolito wrote: > >> There's something similar to your requirements in here: >> https://github.com/mochi/mochiweb/blob/master/src/mochiweb_session.erl >> >> >> On Mon, Apr 1, 2013 at 1:05 PM, Lee Sylvester wrote: >> >>> Hey guys, >>> >>> So, I'd like to create secure tokens in Erlang. This can either be a >>> simple UUID generator which I then store with user credentials or a way to >>> encode a string, such as JSON, as an encrypted token. In Golang, I would >>> do this with fernet, but I need an Erlang solution :-) >>> >>> I know Erlang isn't best used for such tasks, but does anyone out there >>> know of something usable for this purpose? >>> >>> Thanks loads, >>> Lee >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dronnikov@REDACTED Tue Apr 2 07:58:51 2013 From: dronnikov@REDACTED (Vladimir Dronnikov) Date: Tue, 2 Apr 2013 09:58:51 +0400 Subject: [erlang-questions] Secure Tokens In-Reply-To: References: <57A08917-A253-44AF-9865-440651D8C131@gmail.com> <639C85DA-F075-455E-ACD0-930EE2F9A7BD@gmail.com> Message-ID: Hi, Bob! Thank you for feedback. Am not an expert in crypto domain, so I wonder if you could help me amend termit' cryptosystem. TIA, --Vladimir On Tue, Apr 2, 2013 at 9:50 AM, Bob Ippolito wrote: > No reason to celebrate just yet, termit has a broken cryptosystem. Here's > two things I noticed after a quick glance: > > * The IV is derived from the secret key. The IV must be unpredictable at > encryption time in CBC mode. This is VERY VERY bad. > * Verification of the signature isn't constant-time, so it's susceptible > to timing attacks. This is still bad, but probably harder to exploit. > > It would be unwise to use this implementation. I don't claim that the > mochiweb code is perfect, and I'm not a cryptograph expert, but I have > audited it and I didn't find any obvious flaws (other than the timing > attack that I fixed). > > > On Mon, Apr 1, 2013 at 10:37 PM, Lee Sylvester wrote: > >> Wow, a fernet like impl for Erlang!!! Perfect!!! Thank you very much. >> This will make my life so much easier :-) >> >> Regards, >> Lee >> >> >> >> On 2 Apr 2013, at 06:09, Vladimir Dronnikov wrote: >> >> I drive https://github.com/dvv/termit for this. Feel free to >> feedback/blame :) >> >> >> On Tue, Apr 2, 2013 at 12:15 AM, Bob Ippolito wrote: >> >>> There's something similar to your requirements in here: >>> https://github.com/mochi/mochiweb/blob/master/src/mochiweb_session.erl >>> >>> >>> On Mon, Apr 1, 2013 at 1:05 PM, Lee Sylvester wrote: >>> >>>> Hey guys, >>>> >>>> So, I'd like to create secure tokens in Erlang. This can either be a >>>> simple UUID generator which I then store with user credentials or a way to >>>> encode a string, such as JSON, as an encrypted token. In Golang, I would >>>> do this with fernet, but I need an Erlang solution :-) >>>> >>>> I know Erlang isn't best used for such tasks, but does anyone out there >>>> know of something usable for this purpose? >>>> >>>> Thanks loads, >>>> Lee >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Tue Apr 2 08:00:06 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 2 Apr 2013 07:00:06 +0100 Subject: [erlang-questions] Secure Tokens In-Reply-To: References: <57A08917-A253-44AF-9865-440651D8C131@gmail.com> <639C85DA-F075-455E-ACD0-930EE2F9A7BD@gmail.com> Message-ID: <19F84F4D-C2A2-4BBA-A76D-52900D9DB627@gmail.com> Thanks for the heads up, Bob. Luckily, I simply need to mimick Fernet while my portal interface is being built; then I can scrap my tokening altogether. However, I'll take your points onboard for any production impl. I'm sure, tho, that your raised points will help improve Termit. Regards, Lee Sent from my iPhone On 2 Apr 2013, at 06:50, Bob Ippolito wrote: > No reason to celebrate just yet, termit has a broken cryptosystem. Here's two things I noticed after a quick glance: > > * The IV is derived from the secret key. The IV must be unpredictable at encryption time in CBC mode. This is VERY VERY bad. > * Verification of the signature isn't constant-time, so it's susceptible to timing attacks. This is still bad, but probably harder to exploit. > > It would be unwise to use this implementation. I don't claim that the mochiweb code is perfect, and I'm not a cryptograph expert, but I have audited it and I didn't find any obvious flaws (other than the timing attack that I fixed). > > > On Mon, Apr 1, 2013 at 10:37 PM, Lee Sylvester wrote: >> Wow, a fernet like impl for Erlang!!! Perfect!!! Thank you very much. This will make my life so much easier :-) >> >> Regards, >> Lee >> >> >> >> On 2 Apr 2013, at 06:09, Vladimir Dronnikov wrote: >> >>> I drive https://github.com/dvv/termit for this. Feel free to feedback/blame :) >>> >>> >>> On Tue, Apr 2, 2013 at 12:15 AM, Bob Ippolito wrote: >>>> There's something similar to your requirements in here: >>>> https://github.com/mochi/mochiweb/blob/master/src/mochiweb_session.erl >>>> >>>> >>>> On Mon, Apr 1, 2013 at 1:05 PM, Lee Sylvester wrote: >>>>> Hey guys, >>>>> >>>>> So, I'd like to create secure tokens in Erlang. This can either be a simple UUID generator which I then store with user credentials or a way to encode a string, such as JSON, as an encrypted token. In Golang, I would do this with fernet, but I need an Erlang solution :-) >>>>> >>>>> I know Erlang isn't best used for such tasks, but does anyone out there know of something usable for this purpose? >>>>> >>>>> Thanks loads, >>>>> Lee >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Tue Apr 2 08:01:57 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 2 Apr 2013 07:01:57 +0100 Subject: [erlang-questions] Secure Tokens In-Reply-To: References: <57A08917-A253-44AF-9865-440651D8C131@gmail.com> <639C85DA-F075-455E-ACD0-930EE2F9A7BD@gmail.com> Message-ID: <4A6B4D59-D396-4AB4-BC22-5E7291690EBE@gmail.com> Hi Vladimir, I'd certainly help if I can. I'm no expert, either, but I find offloading handling of token decryption to be useful, especially across unlinked nodes. So, I'm sure others will see the usefulness of this. Best, Lee Sent from my iPhone On 2 Apr 2013, at 06:58, Vladimir Dronnikov wrote: > Hi, Bob! > Thank you for feedback. Am not an expert in crypto domain, so I wonder if you could help me amend termit' cryptosystem. > TIA, > --Vladimir > > > On Tue, Apr 2, 2013 at 9:50 AM, Bob Ippolito wrote: >> No reason to celebrate just yet, termit has a broken cryptosystem. Here's two things I noticed after a quick glance: >> >> * The IV is derived from the secret key. The IV must be unpredictable at encryption time in CBC mode. This is VERY VERY bad. >> * Verification of the signature isn't constant-time, so it's susceptible to timing attacks. This is still bad, but probably harder to exploit. >> >> It would be unwise to use this implementation. I don't claim that the mochiweb code is perfect, and I'm not a cryptograph expert, but I have audited it and I didn't find any obvious flaws (other than the timing attack that I fixed). >> >> >> On Mon, Apr 1, 2013 at 10:37 PM, Lee Sylvester wrote: >>> Wow, a fernet like impl for Erlang!!! Perfect!!! Thank you very much. This will make my life so much easier :-) >>> >>> Regards, >>> Lee >>> >>> >>> >>> On 2 Apr 2013, at 06:09, Vladimir Dronnikov wrote: >>> >>>> I drive https://github.com/dvv/termit for this. Feel free to feedback/blame :) >>>> >>>> >>>> On Tue, Apr 2, 2013 at 12:15 AM, Bob Ippolito wrote: >>>>> There's something similar to your requirements in here: >>>>> https://github.com/mochi/mochiweb/blob/master/src/mochiweb_session.erl >>>>> >>>>> >>>>> On Mon, Apr 1, 2013 at 1:05 PM, Lee Sylvester wrote: >>>>>> Hey guys, >>>>>> >>>>>> So, I'd like to create secure tokens in Erlang. This can either be a simple UUID generator which I then store with user credentials or a way to encode a string, such as JSON, as an encrypted token. In Golang, I would do this with fernet, but I need an Erlang solution :-) >>>>>> >>>>>> I know Erlang isn't best used for such tasks, but does anyone out there know of something usable for this purpose? >>>>>> >>>>>> Thanks loads, >>>>>> Lee >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Tue Apr 2 08:29:54 2013 From: bob@REDACTED (Bob Ippolito) Date: Mon, 1 Apr 2013 23:29:54 -0700 Subject: [erlang-questions] Secure Tokens In-Reply-To: References: <57A08917-A253-44AF-9865-440651D8C131@gmail.com> <639C85DA-F075-455E-ACD0-930EE2F9A7BD@gmail.com> Message-ID: If you take a look at the implementation in mochiweb_session, you'll see that it's very similar in structure to termit but the IV is generated with random bytes in encrypt_data/2 and there's an eq/2 function to do constant time comparison of signatures. If you're interested in learning more about crypto I highly recommend Dan Boneh's class on Coursera https://www.coursera.org/course/crypto On Mon, Apr 1, 2013 at 10:58 PM, Vladimir Dronnikov wrote: > Hi, Bob! > Thank you for feedback. Am not an expert in crypto domain, so I wonder if > you could help me amend termit' cryptosystem. > TIA, > --Vladimir > > > On Tue, Apr 2, 2013 at 9:50 AM, Bob Ippolito wrote: > >> No reason to celebrate just yet, termit has a broken cryptosystem. Here's >> two things I noticed after a quick glance: >> >> * The IV is derived from the secret key. The IV must be unpredictable at >> encryption time in CBC mode. This is VERY VERY bad. >> * Verification of the signature isn't constant-time, so it's susceptible >> to timing attacks. This is still bad, but probably harder to exploit. >> >> It would be unwise to use this implementation. I don't claim that the >> mochiweb code is perfect, and I'm not a cryptograph expert, but I have >> audited it and I didn't find any obvious flaws (other than the timing >> attack that I fixed). >> >> >> On Mon, Apr 1, 2013 at 10:37 PM, Lee Sylvester wrote: >> >>> Wow, a fernet like impl for Erlang!!! Perfect!!! Thank you very much. >>> This will make my life so much easier :-) >>> >>> Regards, >>> Lee >>> >>> >>> >>> On 2 Apr 2013, at 06:09, Vladimir Dronnikov wrote: >>> >>> I drive https://github.com/dvv/termit for this. Feel free to >>> feedback/blame :) >>> >>> >>> On Tue, Apr 2, 2013 at 12:15 AM, Bob Ippolito wrote: >>> >>>> There's something similar to your requirements in here: >>>> https://github.com/mochi/mochiweb/blob/master/src/mochiweb_session.erl >>>> >>>> >>>> On Mon, Apr 1, 2013 at 1:05 PM, Lee Sylvester wrote: >>>> >>>>> Hey guys, >>>>> >>>>> So, I'd like to create secure tokens in Erlang. This can either be a >>>>> simple UUID generator which I then store with user credentials or a way to >>>>> encode a string, such as JSON, as an encrypted token. In Golang, I would >>>>> do this with fernet, but I need an Erlang solution :-) >>>>> >>>>> I know Erlang isn't best used for such tasks, but does anyone out >>>>> there know of something usable for this purpose? >>>>> >>>>> Thanks loads, >>>>> Lee >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gumm@REDACTED Tue Apr 2 09:14:32 2013 From: gumm@REDACTED (Jesse Gumm) Date: Tue, 2 Apr 2013 02:14:32 -0500 Subject: [erlang-questions] Erlang Type specifications and throw In-Reply-To: <675F7D33-1F95-4EEA-9842-04BD45D8C881@me.com> References: <675F7D33-1F95-4EEA-9842-04BD45D8C881@me.com> Message-ID: (including the ML with this one) Hi Rudolph, With a situation like this, the more canonical Erlang approach is to do something like: on success, return {ok, Pid} and on failure return the atom 'error'. Then the spec would simply be: -spec where(G::atom()) -> {ok, pid()} | error. Then you can trivially use it in a case statement and use pattern matching. Using a combination of throw and try blocks as intended behavior for control flow is a recipe for confusion. -Jesse On Apr 2, 2013 12:22 AM, "Rudolph van Graan" wrote: > Hi, > > I want to write a type spec for a function that throws exceptions, > something like this: > > -spec where(G :: atom()) -> pid() | throw({no_name,atom()}). > > > I don't want to use an exit signal, because this behaviour is intentional > and the calling process will catch this and take specific action. But I > can't write this using the -spec syntax. The documentation says: > > Some functions in Erlang are not meant to return; either because they > define servers or because they are used to throw exceptions as the function > below: > > my_error(Err) -> erlang:throw({error, Err}). > > For such functions we recommend the use of the special no_return() type > for their "return", via a contract of the form: > > -spec my_error(term()) -> no_return(). > > > no_return() is completely the wrong thing to use and just leaving out the > throw(?) declaration is also wrong in my eyes. How do I do this? > > Thanks, > > Rudolph van Graan > > > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From 249505968@REDACTED Tue Apr 2 09:20:47 2013 From: 249505968@REDACTED (=?gb18030?B?99L30Q==?=) Date: Tue, 2 Apr 2013 15:20:47 +0800 Subject: [erlang-questions] How to get more than one clause from one record ? Message-ID: How to get more than one clause from one record ? I just know this one: -record(r,{c1 = 1,c2 = 2}). #r{c1 = FirstC, c2 = SecondC} = #r{c1 = 1,c2 = 2}. Then FirstC == 1, SecondC == 2. It doesn't looks so well. Because we assign the value from left to right. Is there any way to write a better-look code? -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Tue Apr 2 10:02:57 2013 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 02 Apr 2013 11:02:57 +0300 Subject: [erlang-questions] Erlang Type specifications and throw In-Reply-To: <675F7D33-1F95-4EEA-9842-04BD45D8C881@me.com> References: <675F7D33-1F95-4EEA-9842-04BD45D8C881@me.com> Message-ID: <515A90B1.6000306@cs.ntua.gr> On 04/02/2013 08:22 AM, Rudolph van Graan wrote: > Hi, > > I want to write a type spec for a function that throws exceptions, > something like this: > >> -spec where(G :: atom()) -> pid() | throw({no_name,atom()}). > > I don't want to use an exit signal, because this behaviour is > intentional and the calling process will catch this and take specific > action. But I can't write this using the -spec syntax. The documentation > says: > >> Some functions in Erlang are not meant to return; either because they >> define servers or because they are used to throw exceptions as the >> function below: >> my_error(Err) -> erlang:throw({error, Err}). >> >> For such functions we recommend the use of the special no_return() >> type for their "return", via a contract of the form: >> >> -spec my_error(term()) -> no_return(). > > no_return() is completely the wrong thing to use and just leaving out > the throw(?) declaration is also wrong in my eyes. You are right that the current language of function specifications does not allow specifying exceptions that a function may throw. This could be done either by extending the spec syntax to include this info, i.e. allow one to write in your example something like: -spec where(G :: atom()) -> pid() throws {no_name,atom()}. or by introducing a new attribute -throws(...) or even -exceptions(...) to document the exception behavior of functions. However, neither of these alternatives is currently implemented. But note that when functions throw exceptions, nothing gets returned. In this respect, the no_return() return type that the documentation currently suggests to use is correct. Kostis PS. I will not comment on the general use of throw in Erlang programming as I realize that your function(s) may be more complicated than the simple example you show us. However, in that one, simply returning {no_name,atom()} or simply 'no_such_process' is arguably a better alternative. From bengt.kleberg@REDACTED Tue Apr 2 10:13:48 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 2 Apr 2013 10:13:48 +0200 Subject: [erlang-questions] How to get more than one clause from one record ? In-Reply-To: References: Message-ID: <1364890428.5966.11.camel@sekic1152.rnd.ki.sw.ericsson.se> Greetings, Is your goal to assign/bind the value of more then one member in a record on a single line of code? Then you can use the way you have in your example: #r{c1=C1, c2=C2} = R. This is assuming that R is a #r{}. Otherwise you will get a bad match exception. You mention "clause" in your question. Is that because you want to have an example with a function clause or a case clause? bengt On Tue, 2013-04-02 at 15:20 +0800, ?? wrote: > How to get more than one clause from one record ? > > > I just know this one: > -record(r,{c1 = 1,c2 = 2}). > #r{c1 = FirstC, c2 = SecondC} = #r{c1 = 1,c2 = 2}. > > > Then FirstC == 1, SecondC == 2. > > > It doesn't looks so well. > Because we assign the value from left to right. > > > Is there any way to write a better-look code? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From colanderman@REDACTED Tue Apr 2 03:13:51 2013 From: colanderman@REDACTED (colanderman@REDACTED) Date: Mon, 1 Apr 2013 18:13:51 -0700 (PDT) Subject: [erlang-questions] oe_obj? (& mailing list issues) Message-ID: Hi, I'm playing around with IC and the c_client backend. All the generated stubs seem to want an "oe_obj" parameter of the type of my interface (which is typedef'd to CORBA_Object). My assumption is that this is analogous to the OE_THIS parameter required by the stubs generated by the erl_genserver backend (i.e. it's the gen_server's PID), but: 1) I see no function (analogous to oe_create/0,1,2) capable of generating such a value, unless I explicitly declare one in another interface (which of course has the same problem), and 2) this value seems to be unused by all the generated code. The c_client example seems to pass in NULL for this argument, instead obtaining the PID manually and storing it manually in the CORBA_Environment structure. This seems... odd. What am I missing? Why does oe_obj not do what I expect? Thanks, Chris P.S.: I tried e-mailing to erlang-questions@REDACTED directly (after signing up), but my message has been "awaiting moderator approval" for several days now. What's going on? (I'm using Google Groups for the time being.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Tue Apr 2 11:45:55 2013 From: ingela.andin@REDACTED (Ingela Andin) Date: Tue, 2 Apr 2013 11:45:55 +0200 Subject: [erlang-questions] ssl_upgrade_failure with particular SSL certificate In-Reply-To: References: Message-ID: Hi! This is because of some software does not use the PKCS-standard oids but alternative ISO-oids. In the upcoming version of public_key and ssl this is solved by handling these alternative oids see commit: 006f45a738a6612958381b2fcbf48586c008d911 Regards Ingela Erlang/OTP team - Ericsson AB 2013/3/29, Scott Baldwin : > I am trying to configure SSL for connections to my RabbitMQ broker. I > realize that this is not the RabbitMQ mailing list, but I think that my > problem is related specifically to Erlang's SSL implementation. I was able > to get it working with a certificate/key pair created directly with > OpenSSL; however, when I converted a certificate made with makecert.exe to > PEM format and try to use that, the client fails to connect and the server > logs an ssl_upgrade_failure. It seems that there is something about my > certificate that Erlang doesn't like. > > I am using Erlang R16B. > > Here is my certificate: > > -----BEGIN CERTIFICATE----- > MIIDTzCCAjugAwIBAgIQYuux7Ob2BL5PUnDLgT/igTAJBgUrDgMCHQUAMCgxJjAk > BgNVBAMTHUVsbGtheSBTdGFnaW5nIFJvb3QgQXV0aG9yaXR5MB4XDTEyMDgxNTE1 > MTMzN1oXDTM5MTIzMTIzNTk1OVowMzExMC8GA1UEAx4oACoALgBsAGsAYwBsAG8A > dQBkAHMAdABhAGcAaQBuAGcALgBjAG8AbTCCASIwDQYJKoZIhvcNAQEBBQADggEP > ADCCAQoCggEBANhryzuSNbDOUVqD7Oby/z+JNjICGemlpP0qmcAZ8JbE7ci/l5eu > BYwIyKy/LvjYYV6Z8ZlMKIbzmEgKxGCmSZjTcg08QXxG7CXpJfls/1ycv8Le7Tz0 > ep2mzBnFhkOCNDQz2zAOiI/K6gwB0D2tv3O+j3ytnME8w+To5epzZSnfGHRIutQ4 > jC7rVz8T1oLixYynQ39tG6L5ALmu5u1DZTRYmzaIbF16c6dy1m8OCqAvQ3LnykZq > rukjjaLDlJT6ZbUUXaZeGS2avf8ZM0f+HlrdDR+IFC/CxipxzHa6kStc+1dZVgqj > jT7ql9nEQ/8DaXmF4C749ELbtWOlSB/ElwUCAwEAAaNyMHAwEwYDVR0lBAwwCgYI > KwYBBQUHAwEwWQYDVR0BBFIwUIAQx8ryGLLGJ2Qr6NrWGYDWT6EqMCgxJjAkBgNV > BAMTHUVsbGtheSBTdGFnaW5nIFJvb3QgQXV0aG9yaXR5ghAu7ZXj5fLAu0CXveR3 > xHi0MAkGBSsOAwIdBQADggEBACiAPScOR/DViwY4ZDVSxeGFqezh6ubWt4aqrYlt > h6ODWF1T0uUjf/VKksPtXlAxAz1F7IHmf80VAGPY18ZmH9JvnVz67PdGcKi6RMHY > vpBT79vbv0/+9TXxdIl2+qafuVb5ckmSlq1pIslnlZszt32pwrSYDvLihfRLStvV > MzKtUGRsug/eUeuCQBAalAHmuNh77bC6Bnp2ZMg/7HEb0bqXQS1mOupiN3Ylpe/y > r3pT7+xLzyzX4NY7GyYVO2VPnz2kvNbrTsTPWO7y1NQc3tDbRIwQeCqpYditByVN > cS/zgODqcpH1NipIfL/JTMFvA5O0jlgpSQDbRxiQELjJ9ms= > -----END CERTIFICATE----- > > Here is the relevant part of the log from RabbitMQ: > > =INFO REPORT==== 28-Mar-2013::20:46:52 === > accepting AMQP connection <0.301.0> (192.168.51.234:50804 -> > 192.168.51.153:5671) > > =ERROR REPORT==== 28-Mar-2013::20:46:52 === > ** State machine <0.302.0> terminating > ** Last message in was {tcp,#Port<0.15153>, > > <<22,3,0,0,53,1,0,0,49,3,0,81,84,228,150,220,41, > > 203,120,104,165,175,147,215,108,167,136,54,238, > > 178,50,70,122,181,212,166,114,251,121,27,202,52, > 143,0,0,10,0,5,0,10,0,19,0,4,0,255,1,0>>} > ** When State == hello > ** Data == {state,server, > {#Ref<0.0.0.1972>,<0.301.0>}, > gen_tcp,tcp,tcp_closed,tcp_error,"localhost",5671, > #Port<0.15153>, > {ssl_options,[],verify_none, > {#Fun,[]}, > false,false,undefined,1, > > <<"C:/Users/ScottB/AppData/Roaming/RabbitMQ/lkcloudstaging_cer.pem">>, > undefined, > > <<"C:/Users/ScottB/AppData/Roaming/RabbitMQ/server/key.pem">>, > undefined,undefined,undefined,<<>>,undefined, > undefined, > [<<0,107>>, > <<0,106>>, > <<0,61>>, > <<0,103>>, > <<0,64>>, > <<0,60>>, > <<0,57>>, > <<0,56>>, > <<0,53>>, > <<0,22>>, > <<0,19>>, > <<0,10>>, > <<0,51>>, > <<0,50>>, > <<0,47>>, > <<0,5>>, > <<0,4>>, > <<0,21>>, > <<0,9>>], > > #Fun,true,268435456,false,undefined, > undefined,false,undefined,undefined}, > {socket_options,binary,0,0,0,false}, > {connection_states, > {connection_state, > {security_parameters, > <<0,0>>, > 0,0,0,0,0,0,0,0,0,0,0,undefined,undefined, > undefined,undefined}, > undefined,undefined,undefined,0,undefined, > undefined,undefined}, > {connection_state, > {security_parameters,undefined,0,undefined, > undefined,undefined,undefined,undefined, > undefined,undefined,undefined,undefined, > undefined,undefined,undefined, > > <<81,84,228,124,31,218,166,3,48,108,125,182, > > 121,180,129,153,59,55,16,200,98,117,189,183, > 170,169,208,189,111,61,67,162>>, > undefined}, > undefined,undefined,undefined,undefined, > undefined,undefined,undefined}, > {connection_state, > {security_parameters, > <<0,0>>, > 0,0,0,0,0,0,0,0,0,0,0,undefined,undefined, > undefined,undefined}, > undefined,undefined,undefined,0,undefined, > undefined,undefined}, > {connection_state, > {security_parameters,undefined,0,undefined, > undefined,undefined,undefined,undefined, > undefined,undefined,undefined,undefined, > undefined,undefined,undefined, > > <<81,84,228,124,31,218,166,3,48,108,125,182, > > 121,180,129,153,59,55,16,200,98,117,189,183, > 170,169,208,189,111,61,67,162>>, > undefined}, > undefined,undefined,undefined,undefined, > undefined,undefined,undefined}}, > [],<<>>,<<>>, > {[],[]}, > [],311374, > {session,undefined,undefined, > > <<48,130,3,79,48,130,2,59,160,3,2,1,2,2,16,98,235,177, > > 236,230,246,4,190,79,82,112,203,129,63,226,129,48,9, > > 6,5,43,14,3,2,29,5,0,48,40,49,38,48,36,6,3,85,4,3, > > 19,29,69,108,108,107,97,121,32,83,116,97,103,105, > > 110,103,32,82,111,111,116,32,65,117,116,104,111,114, > > 105,116,121,48,30,23,13,49,50,48,56,49,53,49,53,49, > > 51,51,55,90,23,13,51,57,49,50,51,49,50,51,53,57,53, > > 57,90,48,51,49,49,48,47,6,3,85,4,3,30,40,0,42,0,46, > > 0,108,0,107,0,99,0,108,0,111,0,117,0,100,0,115,0, > > 116,0,97,0,103,0,105,0,110,0,103,0,46,0,99,0,111,0, > > 109,48,130,1,34,48,13,6,9,42,134,72,134,247,13,1,1, > > 1,5,0,3,130,1,15,0,48,130,1,10,2,130,1,1,0,216,107, > > 203,59,146,53,176,206,81,90,131,236,230,242,255,63, > > 137,54,50,2,25,233,165,164,253,42,153,192,25,240, > > 150,196,237,200,191,151,151,174,5,140,8,200,172,191, > > 46,248,216,97,94,153,241,153,76,40,134,243,152,72, > > 10,196,96,166,73,152,211,114,13,60,65,124,70,236,37, > > 233,37,249,108,255,92,156,191,194,222,237,60,244, > > 122,157,166,204,25,197,134,67,130,52,52,51,219,48, > > 14,136,143,202,234,12,1,208,61,173,191,115,190,143, > > 124,173,156,193,60,195,228,232,229,234,115,101,41, > > 223,24,116,72,186,212,56,140,46,235,87,63,19,214, > > 130,226,197,140,167,67,127,109,27,162,249,0,185,174, > > 230,237,67,101,52,88,155,54,136,108,93,122,115,167, > > 114,214,111,14,10,160,47,67,114,231,202,70,106,174, > > 233,35,141,162,195,148,148,250,101,181,20,93,166,94, > > 25,45,154,189,255,25,51,71,254,30,90,221,13,31,136, > > 20,47,194,198,42,113,204,118,186,145,43,92,251,87, > > 89,86,10,163,141,62,234,151,217,196,67,255,3,105, > > 121,133,224,46,248,244,66,219,181,99,165,72,31,196, > > 151,5,2,3,1,0,1,163,114,48,112,48,19,6,3,85,29,37,4, > > 12,48,10,6,8,43,6,1,5,5,7,3,1,48,89,6,3,85,29,1,4, > > 82,48,80,128,16,199,202,242,24,178,198,39,100,43, > > 232,218,214,25,128,214,79,161,42,48,40,49,38,48,36, > > 6,3,85,4,3,19,29,69,108,108,107,97,121,32,83,116,97, > > 103,105,110,103,32,82,111,111,116,32,65,117,116,104, > > 111,114,105,116,121,130,16,46,237,149,227,229,242, > > 192,187,64,151,189,228,119,196,120,180,48,9,6,5,43, > > 14,3,2,29,5,0,3,130,1,1,0,40,128,61,39,14,71,240, > > 213,139,6,56,100,53,82,197,225,133,169,236,225,234, > > 230,214,183,134,170,173,137,109,135,163,131,88,93, > > 83,210,229,35,127,245,74,146,195,237,94,80,49,3,61, > > 69,236,129,230,127,205,21,0,99,216,215,198,102,31, > > 210,111,157,92,250,236,247,70,112,168,186,68,193, > > 216,190,144,83,239,219,219,191,79,254,245,53,241, > > 116,137,118,250,166,159,185,86,249,114,73,146,150, > > 173,105,34,201,103,149,155,51,183,125,169,194,180, > > 152,14,242,226,133,244,75,74,219,213,51,50,173,80, > > 100,108,186,15,222,81,235,130,64,16,26,148,1,230, > > 184,216,123,237,176,186,6,122,118,100,200,63,236, > > 113,27,209,186,151,65,45,102,58,234,98,55,118,37, > > 165,239,242,175,122,83,239,236,75,207,44,215,224, > > 214,59,27,38,21,59,101,79,159,61,164,188,214,235,78, > > 196,207,88,238,242,212,212,28,222,208,219,68,140,16, > > 120,42,169,97,216,173,7,37,77,113,47,243,128,224, > > 234,114,145,245,54,42,72,124,191,201,76,193,111,3, > > 147,180,142,88,41,73,0,219,71,24,144,16,184,201,246, > 107>>, > undefined,undefined,undefined,new,63531722812}, > 323665,ssl_session_cache,undefined,undefined,false, > undefined,undefined,undefined, > {'RSAPrivateKey','two-prime', > > 25091000490399564416382733665912293706281236323287507449391018333858706088067104372951637210440828548699801793107621328582247328739957168356535343760898421117596223923057958675108280840952652110424468556362893842108742460936250265912296002218912760264533284800177616747391132407486580757942725318853670784742540298023139943942002078742079335138046822007139070167779479715409389988021492873379536675527198388004784204705449619014967663111341423672277165259908002197645143645833929707716094821495848245665580802072300300901995696081299311434728567907957618159230597695337971845318310069905698028328520007565703331606819, > 65537, > > 12532291835951284642352753464759952731760837234028003552929880741268762456120795803045590924921343389430997938501684187097537025786559622030041471881063352256944852432936802405831735737793065202597533511207149656340503466992496089298764016305810310122514496309703131156584850210212028846765905833153120519214366483351036620512028360903366902227866159233021509892771286294064778569099266243884082209785268720465970929381008430443130075496396131177443808450873061131440124680376808011317874020764946935204300278562787258089499308485762628408971801392792765876969493808892573747399158232707154902628249712310347508330481, > > 164613524625768478096728511491146234379950805547018160443402940694931123301226530314268605486708880647658162742710176890755691202467149416112553065729831746391569481381229328262217225008710581122456985360175690217141752754366597025760074826970126144030433840076718674219450293036228318089528491377991378917023, > > 152423687831490839453627602007609954938806264385151113997291723876694061058672531571680491904693205860873313947735180318401018227463103944680073963443527347105243646402511993135691316201430837009543216841366727950952917475175355759283610454988240555587842851002909990207473661609226206434152468235025307200253, > > 103984547751379971996375538203182369609466154978729646218112491292391375460388439026510307132524542623745369476562226118076733144497574174552444945117251391868174999766567175194585209852993108440859312097378784492720927449807326399887717438420071901928924585277569562140638458907286206884483421800776127924467, > > 39507777060187907438527428403852332339678380351718296130002815409515266417499584872791499702229633458331247753638059539934359165508273901891762155988452310073344428665326017782260225343145179490686339388197454990354108505894437772295812911773276810317388444847741459078907412450309375905167279214922484907925, > > 140777917719684893441642072243040594921813463059778562021367548768326948139714681618402000290527139618053328133891840461484222782830228667641262369743730585486629970714763524415800836168519782394433537656246543908266747427470739521793087643652694808980372432733634387874662999415574210646072641560865328049441, > asn1_NOVALUE}, > {'DHParameter', > > 179769313486231590770839156793787453197860296048756011706444423684197180216158519368947833795864925541502180565485980503646440548199239100050792877003355816639229553136239076508735759914822574862575007425302077447712589550957937778424442426617334727629299387668709205606050270810842907692932019128194467627007, > 2,asn1_NOVALUE}, > undefined,undefined,315471,#Ref<0.0.0.1974>,undefined, > <<>>,true, > {false,first}, > {<0.301.0>,#Ref<0.0.0.1971>}, > #Ref<0.0.0.1980>, > {[],[]}, > false,true,false,undefined} > ** Reason for termination = > ** {{badmatch, > {error, > {asn1, > {'Type not compatible with table constraint', > {{component,'Type'}, > {value,{5,<<>>}}, > {unique_name_and_value,id,{1,3,14,3,2,29}}}}}}}, > [{public_key,pkix_decode_cert,2,[{file,"public_key.erl"},{line,218}]}, > {ssl_cipher,filter,2,[{file,"ssl_cipher.erl"},{line,484}]}, > > {ssl_handshake,select_session,8,[{file,"ssl_handshake.erl"},{line,654}]}, > {ssl_handshake,hello,4,[{file,"ssl_handshake.erl"},{line,178}]}, > {ssl_connection,hello,2,[{file,"ssl_connection.erl"},{line,413}]}, > > {ssl_connection,next_state,4,[{file,"ssl_connection.erl"},{line,2001}]}, > {gen_fsm,handle_msg,7,[{file,"gen_fsm.erl"},{line,494}]}, > {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]} > > =ERROR REPORT==== 28-Mar-2013::20:46:52 === > error on AMQP connection <0.301.0>: {ssl_upgrade_failure, > {{{badmatch, > {error, > {asn1, > {'Type not compatible with table > constraint', > {{component,'Type'}, > {value,{5,<<>>}}, > {unique_name_and_value,id, > {1,3,14,3,2,29}}}}}}}, > [{public_key,pkix_decode_cert,2, > > [{file,"public_key.erl"},{line,218}]}, > {ssl_cipher,filter,2, > > [{file,"ssl_cipher.erl"},{line,484}]}, > {ssl_handshake,select_session,8, > [{file,"ssl_handshake.erl"}, > {line,654}]}, > {ssl_handshake,hello,4, > [{file,"ssl_handshake.erl"}, > {line,178}]}, > {ssl_connection,hello,2, > [{file,"ssl_connection.erl"}, > {line,413}]}, > {ssl_connection,next_state,4, > [{file,"ssl_connection.erl"}, > {line,2001}]}, > {gen_fsm,handle_msg,7, > > [{file,"gen_fsm.erl"},{line,494}]}, > {proc_lib,init_p_do_apply,3, > > [{file,"proc_lib.erl"},{line,239}]}]}, > {gen_fsm,sync_send_all_state_event, > [<0.302.0>,{start,5000},infinity]}}} > > Note that this certificate/key pair was created for testing purposes only, > but I am concerned that our production certificate/key pair will fail in > similar fashion. > > Thanks, > Scott > From ingela.andin@REDACTED Tue Apr 2 12:32:40 2013 From: ingela.andin@REDACTED (Ingela Andin) Date: Tue, 2 Apr 2013 12:32:40 +0200 Subject: [erlang-questions] ssh eow In-Reply-To: References: <8d026d1d272a84accac74056e8bfbb2d.squirrel@service.rules.org> <4f9c5ac539ef1c35325a412a2a9c46ec.squirrel@service.rules.org> <36a480137a631a0cb8b65c60864e6fac.squirrel@service.rules.org> Message-ID: Hi! It looks like Openssh has extended the possible channel request and ssh_connection.erl needs to be extended to also handle that extension. Regards Ingela Erlang/OTP team - Ericsson AB 2013/4/2, Stefan Jahn : > hello, > > when I try > > ssh_connection_manager:request(State#state.cm, State#state.channel, > "eow@REDACTED", false, > [?boolean(false)], 0), > > after all data delivered, I get a data integrity error from scp. > This is not the correct way to introduce new messages, isn't it? > > Thank you in advance, > Stefan. > > On Wed, March 27, 2013 2:13 am, Stefan Jahn wrote: >> hello, >> >> while implementing a scp daemon i wonder how to send the eow event >> to the client in order to tell the scp command to finish expecting >> data from my scp daemon... >> >> see in PROTOCOL (from openssh): >> 2.1. connection: Channel write close extension "eow@REDACTED" >> >> is there a function in ssh api of erlang how i can send this protocol >> item?? >> >> thank you in advance, >> stefan. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From naikvin@REDACTED Tue Apr 2 13:47:52 2013 From: naikvin@REDACTED (Vineet Naik) Date: Tue, 2 Apr 2013 17:17:52 +0530 Subject: [erlang-questions] Mysql Query builder lib in Erlang? Message-ID: Hi, Is there a mysql query builder library for Erlang? On some searching, I came across boss_db[1] which is an ORM. While I am open to ORMs and planning to give it a try, I would prefer a simple query builder [1]: https://github.com/evanmiller/boss_db Regards, Vineet -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Tue Apr 2 13:59:57 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Tue, 2 Apr 2013 14:59:57 +0300 Subject: [erlang-questions] Mysql Query builder lib in Erlang? In-Reply-To: References: Message-ID: <2F08EF6A-C7B8-437A-ADA4-2C357CFFB3A7@gmail.com> Hi, I've made Erlang port of whiskers.js template library https://github.com/fogfish/swirl as a side effect is become a client-side SQL query template builder, which I've started to use for my project. Here is a small example: swirl:c(mysql, "SELECT * FROM Users WHERE id={id}"). mysql:render([{id, "xxx"}]). "SELECT * FROM Users WHERE id=xxx" - Dmitry On Apr 2, 2013, at 2:47 PM, Vineet Naik wrote: > Hi, > > Is there a mysql query builder library for Erlang? On some searching, > I came across boss_db[1] which is an ORM. While I am open to ORMs and > planning to give it a try, I would prefer a simple query builder > > [1]: https://github.com/evanmiller/boss_db > > Regards, > Vineet > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue Apr 2 14:00:50 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 02 Apr 2013 14:00:50 +0200 Subject: [erlang-questions] Mysql Query builder lib in Erlang? In-Reply-To: <2F08EF6A-C7B8-437A-ADA4-2C357CFFB3A7@gmail.com> References: <2F08EF6A-C7B8-437A-ADA4-2C357CFFB3A7@gmail.com> Message-ID: <515AC872.2000609@ninenines.eu> mysql:render([{id, "'' or 1=1 --"}]). On 04/02/2013 01:59 PM, Dmitry Kolesnikov wrote: > Hi, > > I've made Erlang port of whiskers.js template library > https://github.com/fogfish/swirl > > as a side effect is become a client-side SQL query template builder, > which I've started to use for my project. > > Here is a small example: > > swirl:c(mysql, "SELECT * FROM Users WHERE id={id}"). > mysql:render([{id, "xxx"}]). > "SELECT * FROM Users WHERE id=xxx" > > - Dmitry > > > On Apr 2, 2013, at 2:47 PM, Vineet Naik > wrote: > >> Hi, >> >> Is there a mysql query builder library for Erlang? On some searching, >> I came across boss_db[1] which is an ORM. While I am open to ORMs and >> planning to give it a try, I would prefer a simple query builder >> >> [1]: https://github.com/evanmiller/boss_db >> >> Regards, >> Vineet >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From dmkolesnikov@REDACTED Tue Apr 2 14:06:55 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Tue, 2 Apr 2013 15:06:55 +0300 Subject: [erlang-questions] Mysql Query builder lib in Erlang? In-Reply-To: <515AC872.2000609@ninenines.eu> References: <2F08EF6A-C7B8-437A-ADA4-2C357CFFB3A7@gmail.com> <515AC872.2000609@ninenines.eu> Message-ID: <0DEE6735-8750-4C08-9FA7-94167BD55C1A@gmail.com> You are welcome to patch it!!! as I said this is Erlang port of whiskers.js template library. all aspects of SQL injection must be handled before *:render is called. - Dmitry On Apr 2, 2013, at 3:00 PM, Lo?c Hoguin wrote: > mysql:render([{id, "'' or 1=1 --"}]). > > On 04/02/2013 01:59 PM, Dmitry Kolesnikov wrote: >> Hi, >> >> I've made Erlang port of whiskers.js template library >> https://github.com/fogfish/swirl >> >> as a side effect is become a client-side SQL query template builder, >> which I've started to use for my project. >> >> Here is a small example: >> >> swirl:c(mysql, "SELECT * FROM Users WHERE id={id}"). >> mysql:render([{id, "xxx"}]). >> "SELECT * FROM Users WHERE id=xxx" >> >> - Dmitry >> >> >> On Apr 2, 2013, at 2:47 PM, Vineet Naik > > wrote: >> >>> Hi, >>> >>> Is there a mysql query builder library for Erlang? On some searching, >>> I came across boss_db[1] which is an ORM. While I am open to ORMs and >>> planning to give it a try, I would prefer a simple query builder >>> >>> [1]: https://github.com/evanmiller/boss_db >>> >>> Regards, >>> Vineet >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu From naikvin@REDACTED Tue Apr 2 14:19:51 2013 From: naikvin@REDACTED (Vineet Naik) Date: Tue, 2 Apr 2013 17:49:51 +0530 Subject: [erlang-questions] Mysql Query builder lib in Erlang? In-Reply-To: <2F08EF6A-C7B8-437A-ADA4-2C357CFFB3A7@gmail.com> References: <2F08EF6A-C7B8-437A-ADA4-2C357CFFB3A7@gmail.com> Message-ID: On Tue, Apr 2, 2013 at 5:29 PM, Dmitry Kolesnikov wrote: > Hi, > > I've made Erlang port of whiskers.js template library > https://github.com/fogfish/swirl > > as a side effect is become a client-side SQL query template builder, which > I've started to use for my project. > > Here is a small example: > > swirl:c(mysql, "SELECT * FROM Users WHERE id={id}"). > mysql:render([{id, "xxx"}]). > "SELECT * FROM Users WHERE id=xxx" > This is an interesting approach but not exactly what I am looking for. I am searching for an alternative to writing raw queries in code. I have one idea ie. to express the query may be using tuples and lists. and have a set of functions transform it to sql. Not sure if this will make things more complicated though :-) but it's just an idea. Thanks, vineet > > - Dmitry > > > On Apr 2, 2013, at 2:47 PM, Vineet Naik wrote: > > Hi, > > Is there a mysql query builder library for Erlang? On some searching, > I came across boss_db[1] which is an ORM. While I am open to ORMs and > planning to give it a try, I would prefer a simple query builder > > [1]: https://github.com/evanmiller/boss_db > > Regards, > Vineet > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Tue Apr 2 14:30:22 2013 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 2 Apr 2013 08:30:22 -0400 Subject: [erlang-questions] Mysql Query builder lib in Erlang? In-Reply-To: <0DEE6735-8750-4C08-9FA7-94167BD55C1A@gmail.com> References: <2F08EF6A-C7B8-437A-ADA4-2C357CFFB3A7@gmail.com> <515AC872.2000609@ninenines.eu> <0DEE6735-8750-4C08-9FA7-94167BD55C1A@gmail.com> Message-ID: <20130402123021.GA70308@ferdair.local> Then I would not advertise it as a way to template SQL. This is dangerous for any reader who felt the urge to use the whikers.js port to handle SQL, as there is absolutely no protection against one of the biggest attack vectors out there. On 04/02, Dmitry Kolesnikov wrote: > You are welcome to patch it!!! > > as I said this is Erlang port of whiskers.js template library. > all aspects of SQL injection must be handled before *:render is called. > > - Dmitry > > On Apr 2, 2013, at 3:00 PM, Lo?c Hoguin wrote: > > > mysql:render([{id, "'' or 1=1 --"}]). > > > > On 04/02/2013 01:59 PM, Dmitry Kolesnikov wrote: > >> Hi, > >> > >> I've made Erlang port of whiskers.js template library > >> https://github.com/fogfish/swirl > >> > >> as a side effect is become a client-side SQL query template builder, > >> which I've started to use for my project. > >> > >> Here is a small example: > >> > >> swirl:c(mysql, "SELECT * FROM Users WHERE id={id}"). > >> mysql:render([{id, "xxx"}]). > >> "SELECT * FROM Users WHERE id=xxx" > >> > >> - Dmitry > >> > >> > >> On Apr 2, 2013, at 2:47 PM, Vineet Naik >> > wrote: > >> > >>> Hi, > >>> > >>> Is there a mysql query builder library for Erlang? On some searching, > >>> I came across boss_db[1] which is an ORM. While I am open to ORMs and > >>> planning to give it a try, I would prefer a simple query builder > >>> > >>> [1]: https://github.com/evanmiller/boss_db > >>> > >>> Regards, > >>> Vineet > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >> > >> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > > > > > -- > > Lo?c Hoguin > > Erlang Cowboy > > Nine Nines > > http://ninenines.eu > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From kostis@REDACTED Tue Apr 2 14:55:58 2013 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 02 Apr 2013 15:55:58 +0300 Subject: [erlang-questions] [ann]: PropEr v1.1 Message-ID: <515AD55E.5050306@cs.ntua.gr> We are happy to announce the availability of a new version (v1.1) of PropEr, a QuickCheck-inspired Property-Based Testing Tool for Erlang. The code and its manual are accessible from the tool's home page. http://proper.softlab.ntua.gr/ This is the second version of PropEr that has been "officially" released. It contains bug fixes, additions and changes done over the last one and a half years. We sincerely thank all users who sent us pull requests, suggestions for improvements, bug reports and comments. For projects that pull proper directly from its github repository: https://github.com/manopapad/proper the new version has been tagged with its version number (v1.1). Enjoy! Kostis Sagonas (on behalf of the PropEr developers) From nx@REDACTED Tue Apr 2 14:52:20 2013 From: nx@REDACTED (nx) Date: Tue, 2 Apr 2013 08:52:20 -0400 Subject: [erlang-questions] Mysql Query builder lib in Erlang? In-Reply-To: References: Message-ID: I remember coming across erlsql while checking out the erlyweb project a while back: https://github.com/yariv/erlyweb/blob/master/src/erlsql/erlsql.erl. On Tue, Apr 2, 2013 at 7:47 AM, Vineet Naik wrote: > Hi, > > Is there a mysql query builder library for Erlang? On some searching, > I came across boss_db[1] which is an ORM. While I am open to ORMs and > planning to give it a try, I would prefer a simple query builder > > [1]: https://github.com/evanmiller/boss_db > > Regards, > Vineet > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Tue Apr 2 14:55:12 2013 From: ingela.andin@REDACTED (Ingela Andin) Date: Tue, 2 Apr 2013 14:55:12 +0200 Subject: [erlang-questions] scp subsystem In-Reply-To: <2feadbbc5cbca5a1450d5c293011a04e.squirrel@service.rules.org> References: <8d026d1d272a84accac74056e8bfbb2d.squirrel@service.rules.org> <4f6727f775395e32810aeb661470a04d.squirrel@service.rules.org> <2feadbbc5cbca5a1450d5c293011a04e.squirrel@service.rules.org> Message-ID: Hi! >>> 2) save path handling (resolve symlinks); I leave this out, because >>> I've seen in sftpd implementation similar thing. Probably we should >>> share functions across scpd and sftpd? >>> Sounds like a good idea, maybe it would be application internal API!? If you have prototype branch we could take a look and maybe have a design discussion of list. Regards Ingela Erlang/OTP team - Ericssson AB From stefan@REDACTED Tue Apr 2 15:02:23 2013 From: stefan@REDACTED (Stefan Jahn) Date: Tue, 2 Apr 2013 15:02:23 +0200 Subject: [erlang-questions] scp subsystem In-Reply-To: <2feadbbc5cbca5a1450d5c293011a04e.squirrel@service.rules.org> References: <8d026d1d272a84accac74056e8bfbb2d.squirrel@service.rules.org> <4f6727f775395e32810aeb661470a04d.squirrel@service.rules.org> <2feadbbc5cbca5a1450d5c293011a04e.squirrel@service.rules.org> Message-ID: <9062e2cfe0236bb0ef6ad4b97fac70a0.squirrel@service.rules.org> Hello! In the meanwhile I was able to fix 4). In source mode I missed a final protocol zero... that's why communication was hanging. "echo $?" delivers now "0" =) This also means I do not need "eow" from openssh necessarily for this particular purpose. In sink mode file transfer works as well. But "echo $?" still delivers "1"... Probably another protocol item missing. To be investigated. Best regards, Stefan. On Thu, March 28, 2013 11:04 pm, Stefan Jahn wrote: > Hello Attila, > > thank you again =D works like charm also for privileged ports... > > Thus, only 2) ... 4) still to be done. > > Best regards, Stefan. > > On Thu, March 28, 2013 1:25 pm, Attila Rajmund Nohl wrote: >> Hello! >> >> There is already an {fd, FD} option in ssh:daemon, so bind first, then >> pass the file descriptor. >> >> 2013/3/28 Stefan Jahn : >>> Hello Ingela, >>> >>> after some experimentation I have an almost completed scpd server >>> implementattion at hand. >>> >>> Some issues still remain: >>> >>> 1) for privileged ports for ssh:daemon() I suggest to have a callback >>> to the ssh options to obtain listen() handle from e.g. fd_server. >>> What do you think? >>> >>> 2) save path handling (resolve symlinks); I leave this out, because >>> I've seen in sftpd implementation similar thing. Probably we should >>> share functions across scpd and sftpd? >>> >>> 3) fine grained file permissions per user base: r/w for list of >>> directories, I leave it out for now, because not necessery so far >>> for my own purposes >>> >>> 4) ssh channel flushing/eof/close on the last transferred file in >>> source mode of the scpd server. needs some more investigations... >>> that why I wrote the "scp eow" topic on the list. >>> >>> 1) until 3) is not that big problem, but 4) need to be solved. After >>> that I would be happy to contribute if you are interested... >>> >>> What do you think? >>> >>> BR, Stefan. >>> >>> On Tue, March 19, 2013 3:04 pm, Ingela Andin wrote: >>>> Hello again, >>>> >>>> If you do a general scp client/server implementation you should >>>> consider contributing it. >>>> We would be interested. >>>> >>>> Regards Ingela Erlang/OTP team - Ericsson AB >>>> >>>> 2013/3/19, Ingela Andin : >>>>> Hi! >>>>> >>>>> 2013/3/19, Stefan Jahn : >>>>>> dear erlang'ers, >>>>>> >>>>>> after reading some documentation on ssh otp system I ended up with: >>>>>> >>>>>> ssh:daemon({0,0,0,0}, 45678, [{pwdfun, fun auth/2}, >>>>>> %% create server keys: >>>>>> %% ssh-keygen -f /tmp/ssh/ssh_host_rsa_key -N '' -t rsa >>>>>> %% ssh-keygen -f /tmp/ssh/ssh_host_dsa_key -N '' -t dsa >>>>>> {system_dir, "/tmp/ssh"}, >>>>>> {user_dir, "/tmp/ssh"}, >>>>>> % {ssh_cli,{ssh_cli, {ssh_scpd, []}}}, >>>>>> {auth_methods, >>>>>> "keyboard-interactive,password"}, >>>>>> {subsystems, [ >>>>>> ssh_scpd:subsystem_spec([]) >>>>>> % >>>>>> ssh_sftpd:subsystem_spec([{vsn, 3}]) >>>>>> ]}]). >>>>>> >>>>>> whereas the ssh_scpd module is base on the ssh_sftpd module from >>>>>> otp, >>>>>> i.e. >>>>>> >>>>>> -module(ssh_scpd). >>>>>> %-behaviour(ssh_daemon_channel). >>>>>> -behaviour(ssh_channel). >>>>>> >>>>>> -define(UINT32(X), X:32/unsigned-big-integer). >>>>>> >>>>>> %% External exports >>>>>> -export([subsystem_spec/1]). >>>>>> >>>>>> %% Callbacks >>>>>> -export([init/1, handle_ssh_msg/2, handle_msg/2, terminate/2, >>>>>> code_change/3]). >>>>>> >>>>>> implementing the callbacks similar to what we have in ssh_sftpd... >>>>>> >>>>>> now, when I start >>>>>> >>>>>> $ scp -P 45678 README ssh@REDACTED:README >>>>>> >>>>>> authentification works fine as implemented in auth/2, but somehow >>>>>> the ssh channel behaviour is not used at all. >>>>>> >>>>>> ssh@REDACTED's password: >>>>>> {error,{1,erl_parse,["syntax error before: ",[]]}} >>>>>> $ Received disconnect from 127.0.0.1: 11: Application shutdown >>>>>> >>>>>> It seems like scp commands are send into the standard erlang shell >>>>>> (which >>>>>> I did not specify) and not into the channel. >>>>>> >>>>>> I am using R15B2. >>>>>> >>>>>> Could please someone help out here and give me some hint where to >>>>>> proceed reading? >>>>>> >>>>>> Also an abstract description of the difference between the purposes >>>>>> of >>>>>> ssh_cli and subsystems options would be appriciated. >>>>> >>>>> ssh_cli is to customize the shell I do not think you want to use >>>>> ssh_cli at all. You should write your scp-deamon as subsystem and >>>>> then >>>>> use an existing scp client to connect to it or >>>>> write your own erlang client that opens an ssh connection, requests >>>>> the ssh-scp subsystem and then sends scp commands on the channel >>>>> along >>>>> the lines; >>>>> >>>>> ssh:connect ... >>>>> ssh_connection:session_channel... >>>>> ssh_connection:subsystem... >>>>> ssh_connection:send... >>>>> >>>>> The client can use the ssh_channel behavior. >>>>> >>>>> Regards Ingela Erlang/OTP team Ericsson AB >>>>> >>>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From siraaj@REDACTED Tue Apr 2 15:05:43 2013 From: siraaj@REDACTED (Siraaj Khandkar) Date: Tue, 2 Apr 2013 09:05:43 -0400 Subject: [erlang-questions] scp subsystem In-Reply-To: References: <8d026d1d272a84accac74056e8bfbb2d.squirrel@service.rules.org> <4f6727f775395e32810aeb661470a04d.squirrel@service.rules.org> <2feadbbc5cbca5a1450d5c293011a04e.squirrel@service.rules.org> Message-ID: On Apr 2, 2013, at 8:55, Ingela Andin wrote: > Hi! > >>>> 2) save path handling (resolve symlinks); I leave this out, because >>>> I've seen in sftpd implementation similar thing. Probably we should >>>> share functions across scpd and sftpd? >>>> > > Sounds like a good idea, maybe it would be application internal API!? > If you have prototype branch we could take a look and maybe have a > design discussion of list. I, for one, am very interested in this discussion. It'd be awesome if you could keep it on list. From ingela.andin@REDACTED Tue Apr 2 15:24:03 2013 From: ingela.andin@REDACTED (Ingela Andin) Date: Tue, 2 Apr 2013 15:24:03 +0200 Subject: [erlang-questions] scp subsystem In-Reply-To: References: <8d026d1d272a84accac74056e8bfbb2d.squirrel@service.rules.org> <4f6727f775395e32810aeb661470a04d.squirrel@service.rules.org> <2feadbbc5cbca5a1450d5c293011a04e.squirrel@service.rules.org> Message-ID: Hi! 2013/4/2, Siraaj Khandkar : > I, for one, am very interested in this discussion. It'd be awesome if you > could keep it on list. Well I do not mind discussing it here if there is such an interest, I just felt that sometimes discussions can be more effective if they are more private. Also a more private discussion does not necessarily rule out further post here on the topic later on in the discussion has come a bit further. Regards Ingela Erlang/OTP team - Ericsson AB From magnus@REDACTED Tue Apr 2 17:53:48 2013 From: magnus@REDACTED (Magnus Henoch) Date: Tue, 02 Apr 2013 16:53:48 +0100 Subject: [erlang-questions] Mnesia: Whether it is good for my application! help? In-Reply-To: (aman mangal's message of "Fri, 29 Mar 2013 23:14:16 +0530") References: Message-ID: aman mangal writes: > Hi everyone, > > This is my first post on this forum! I am new to erlang, just started using > it a month ago or so. Since I have come across erlang, I have become such a > big fan of it that I have started working on a product in erlang for a > company. > > Coming to the point, here is my question. I want to make > a distributive application. There are multiple processes running on a node. > The processes are divided in logical groups. Each group serves a specific > purpose. Each group has some data(few tables) shared among the processes > inside the group but not outside the group. Now my question is that whether > to use mnesia or not to store the data? If I don't use mnesia, I'll store > the data as state in a gen server processes running in the group. In that > case I don't know how will I will keep back up of the data for the case > when any node goes down. It sounds like Mnesia would be a good fit. You would get persistence for free by creating your tables with disc_copies. The only potential issue I see is table names: each Mnesia table must be named by an atom. Are your process groups created dynamically? If so, you'd need to come up with a scheme to get the correct table names from within the processes that need to access the tables. You'll also need to use the record_name option when creating the tables, as by default Mnesia requires you to use records with the same name as the table. > Also my database can grow as large as thousand tables each having hudred > entries on a node. Is mnesia a good idea to use for such a large (I'm not > sure whether it is large :P ) database? Is there any free alternative to > mnesia? Please note that there is no inter node communication in order to > get data except with the back up node! Assuming each entry takes no more than a kilobyte, you're looking at roughly 100 megabytes of data in total. When using the disc_copies table storage type, Mnesia will store the entire database in memory as well as on disk, but I presume that you have enough RAM that 100 megabytes isn't a big problem. Could you elaborate on your backup node setup? If you want your data to be distributed to protect against disk failure on the main node, then Mnesia distribution is probably the way to go, but depending on your requirements you might be able to get away with a simpler solution. Hope this helps, Magnus From olivier.boudeville@REDACTED Tue Apr 2 18:08:24 2013 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Tue, 2 Apr 2013 18:08:24 +0200 Subject: [erlang-questions] On node disconnections Message-ID: Hi all, I had a few questions about node disconnections. Currently I have a distributed application that must resist the crash of at least some of its hosts. I test the whole feature first by using ten remote virtual machines which I software-disconnect from the user host at random moments of the application execution, thanks to a merciless 'ifconfig down' of the relevant network interface. It works great, insofar as the communications then freeze immediately (no surprise). My intent was to monitor, from a specific process on the user node, each of these 10 worker nodes (using net_kernel:monitor_nodes/2), to receive the corresponding 'nodedown' messages and (if monitoring does not prevent 'noconnection' to be triggered) then to issue a disconnect_node/1 for each of them, so that my user node can resist these losses. However, most of the time I cannot intercept the 'nodedown' information soon enough (or at all), and the whole program crashes and burns, with a message: ** Node 'N' not responding ** ** Removing (timedout) connection ** {"init terminating in do_boot",noconnection} So, my question: how can I prevent this noconnection to wreak havoc, as it seems to ruin our ability to resist node losses? If I understand well, as these reliability messages shall be managed "out of band", there is always a race condition between their receiving and the telling to all processes to stop interacting with the lost node(s). So if there were no way of at least temporarily resisting 'noconnection' (as whatever we do there *will* be processes that will attempt to send a message to a lost node), the whole purpose of the approach would be defeated. Unless I misunderstood something? A related question is that, apparently, increasing the kernel net tick time (say, from 60 to 300) does not seem to increase accordingly the noconnection time-out that must exist somewhere. As a result, I think that by design the node monitoring can only fail then (a noconnection will always happen before the monitoring messages have a chance to kick in). Thanks in advance for any hint! Best regards, Olivier. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjarvstrand@REDACTED Tue Apr 2 18:36:00 2013 From: tjarvstrand@REDACTED (=?ISO-8859-1?Q?Thomas_J=E4rvstrand?=) Date: Tue, 2 Apr 2013 18:36:00 +0200 Subject: [erlang-questions] introductory book In-Reply-To: <269821a6b224cb58a54c0de734cb3ecf@bosqueviejo.net> References: <269821a6b224cb58a54c0de734cb3ecf@bosqueviejo.net> Message-ID: If you have some previous programming experience, I would really like to recommend *Erlang and OTP in Action * by Martin Logan, Eric Merritt and Richard Carlsson. It's really to-the-point while still being very informative and shows you how to build a real-world, useful application. T 2013/3/24 Manuel A. Rubio "Bombadil" > Hi, > > El 2013-03-22 19:39, washington3@REDACTED escribi?: > > Can you suggest an introductory book on Erlang? >> > > You can read this for more information about erlang books :-) > > http://www.erlang.org/faq/**obtaining.html#id50627 > > Regards. > Manuel Rubio. > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hq@REDACTED Tue Apr 2 21:34:16 2013 From: hq@REDACTED (Adam Rutkowski) Date: Tue, 2 Apr 2013 21:34:16 +0200 Subject: [erlang-questions] xmerl/xpath alternatives? Message-ID: <5C89CE9B-A94F-4A07-87DA-6C4DA49A0301@mtod.org> Hi List, I need to provide a configurable set of validation rules for arbitrary user-generated XML documents. XPath would be just fine. i.e. {"/foo/bar[1]/baz", 'in', [1,2,3], 'accept'} But there's a catch - xmerl produces atoms so malicious user could take my VM down without trying too hard. I'm thinking of two ugly hacks as I need to ship soon: 1) Bootstrapping a separate Erlang VM with heart enabled only for the sake of dealing with XPath predicates. 2) Forking the simple parser found here [1] and making it compatible with jsonpath [2], as I'm already using the latter for arbitrary JSON documents, which I need to handle as well. Solutions above are far from being right... Suggestions appreciated. [1] https://github.com/maxlapshin/parsexml [2] https://github.com/GeneStevens/jsonpath -- Adam From lee.sylvester@REDACTED Tue Apr 2 22:08:34 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 2 Apr 2013 21:08:34 +0100 Subject: [erlang-questions] NineNines Bullet Message-ID: <19C75ED6-E427-4099-B4A7-75FC382345F9@gmail.com> Hey guys, Does anyone here know of a good tutorial or chat example of NineNines Bullet handler for Cowboy? Thanks loads, Lee From essen@REDACTED Tue Apr 2 22:23:21 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 02 Apr 2013 22:23:21 +0200 Subject: [erlang-questions] NineNines Bullet In-Reply-To: <19C75ED6-E427-4099-B4A7-75FC382345F9@gmail.com> References: <19C75ED6-E427-4099-B4A7-75FC382345F9@gmail.com> Message-ID: <515B3E39.6010009@ninenines.eu> On 04/02/2013 10:08 PM, Lee Sylvester wrote: > Hey guys, > > Does anyone here know of a good tutorial or chat example of NineNines Bullet handler for Cowboy? Have you looked at the clock example that comes with it? Building a chat on top of that is just making connection processes talk to each other using a registry like gproc or a gen_server that dispatches messages. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From lee.sylvester@REDACTED Tue Apr 2 22:27:47 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 2 Apr 2013 21:27:47 +0100 Subject: [erlang-questions] NineNines Bullet In-Reply-To: <515B3E39.6010009@ninenines.eu> References: <19C75ED6-E427-4099-B4A7-75FC382345F9@gmail.com> <515B3E39.6010009@ninenines.eu> Message-ID: <75D865DE-95F6-4582-B90F-664F6808DF2B@gmail.com> Hi Lo?c, Yes, I spotted that. It just doesn't seem very explanatory. I was hoping for something a little bit more extensive. Preferably something that used the concept of rooms. The NineNines support page for Bullet is also a little thin? :-S I guess I'm too much of a noob. Thanks, Lee On 2 Apr 2013, at 21:23, Lo?c Hoguin wrote: > On 04/02/2013 10:08 PM, Lee Sylvester wrote: >> Hey guys, >> >> Does anyone here know of a good tutorial or chat example of NineNines Bullet handler for Cowboy? > > Have you looked at the clock example that comes with it? Building a chat on top of that is just making connection processes talk to each other using a registry like gproc or a gen_server that dispatches messages. > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu From essen@REDACTED Tue Apr 2 22:35:27 2013 From: essen@REDACTED (=?windows-1252?Q?Lo=EFc_Hoguin?=) Date: Tue, 02 Apr 2013 22:35:27 +0200 Subject: [erlang-questions] NineNines Bullet In-Reply-To: <75D865DE-95F6-4582-B90F-664F6808DF2B@gmail.com> References: <19C75ED6-E427-4099-B4A7-75FC382345F9@gmail.com> <515B3E39.6010009@ninenines.eu> <75D865DE-95F6-4582-B90F-664F6808DF2B@gmail.com> Message-ID: <515B410F.2080704@ninenines.eu> Rooms is a bit out of scope. I agree it could use some more documentation though. The basic idea is that when your process connects its pid is added to a list somewhere (gproc, gen_server, anything else). When a message is sent to the room, all pids in the list receive it. You can also broadcast when someone enters or leave the room. And you remove the pid from the list when it's gone (using monitors for example). On 04/02/2013 10:27 PM, Lee Sylvester wrote: > Hi Lo?c, > > Yes, I spotted that. It just doesn't seem very explanatory. I was hoping for something a little bit more extensive. Preferably something that used the concept of rooms. The NineNines support page for Bullet is also a little thin? :-S I guess I'm too much of a noob. > > Thanks, > Lee > > > > On 2 Apr 2013, at 21:23, Lo?c Hoguin wrote: > >> On 04/02/2013 10:08 PM, Lee Sylvester wrote: >>> Hey guys, >>> >>> Does anyone here know of a good tutorial or chat example of NineNines Bullet handler for Cowboy? >> >> Have you looked at the clock example that comes with it? Building a chat on top of that is just making connection processes talk to each other using a registry like gproc or a gen_server that dispatches messages. >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> http://ninenines.eu > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From lee.sylvester@REDACTED Tue Apr 2 23:00:09 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 2 Apr 2013 22:00:09 +0100 Subject: [erlang-questions] NineNines Bullet In-Reply-To: <515B410F.2080704@ninenines.eu> References: <19C75ED6-E427-4099-B4A7-75FC382345F9@gmail.com> <515B3E39.6010009@ninenines.eu> <75D865DE-95F6-4582-B90F-664F6808DF2B@gmail.com> <515B410F.2080704@ninenines.eu> Message-ID: Yes, that's how I was seeing it. Mentally at least. I've got a similar thing built in Go which I want to port to Erlang using Bullet. Go's websocket implementation is quite far from complete and has been a headache to work with. I'm hoping to have something a lot more robust with Erlang. So, let me get this straight? init handles the connection setup and terminate handles the teardown. So, I can make calls in those two functions to add the user to a room collection or removing her. What are the differences between stream and info? When do those come into play? I'm assuming one (stream?) is invoked when data is sent from the end user. Is that right? Thanks, Lee On 2 Apr 2013, at 21:35, Lo?c Hoguin wrote: > Rooms is a bit out of scope. I agree it could use some more documentation though. > > The basic idea is that when your process connects its pid is added to a list somewhere (gproc, gen_server, anything else). When a message is sent to the room, all pids in the list receive it. You can also broadcast when someone enters or leave the room. And you remove the pid from the list when it's gone (using monitors for example). > > On 04/02/2013 10:27 PM, Lee Sylvester wrote: >> Hi Lo?c, >> >> Yes, I spotted that. It just doesn't seem very explanatory. I was hoping for something a little bit more extensive. Preferably something that used the concept of rooms. The NineNines support page for Bullet is also a little thin? :-S I guess I'm too much of a noob. >> >> Thanks, >> Lee >> >> >> >> On 2 Apr 2013, at 21:23, Lo?c Hoguin wrote: >> >>> On 04/02/2013 10:08 PM, Lee Sylvester wrote: >>>> Hey guys, >>>> >>>> Does anyone here know of a good tutorial or chat example of NineNines Bullet handler for Cowboy? >>> >>> Have you looked at the clock example that comes with it? Building a chat on top of that is just making connection processes talk to each other using a registry like gproc or a gen_server that dispatches messages. >>> >>> -- >>> Lo?c Hoguin >>> Erlang Cowboy >>> Nine Nines >>> http://ninenines.eu >> > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu From lee.sylvester@REDACTED Tue Apr 2 23:09:25 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 2 Apr 2013 22:09:25 +0100 Subject: [erlang-questions] NineNines Bullet In-Reply-To: References: <19C75ED6-E427-4099-B4A7-75FC382345F9@gmail.com> <515B3E39.6010009@ninenines.eu> <75D865DE-95F6-4582-B90F-664F6808DF2B@gmail.com> <515B410F.2080704@ninenines.eu> Message-ID: <14D83CB9-EC20-4D40-B4D6-9457507EE7BA@gmail.com> Another thing; I'm guessing returning {reply,?} from the stream function sends a reply to the user for that connection and returning {ok,?} doesn't. How, though, do I send a message to some other specific user, providing I have their Pid? Thanks, Lee On 2 Apr 2013, at 22:00, Lee Sylvester wrote: > Yes, that's how I was seeing it. Mentally at least. I've got a similar thing built in Go which I want to port to Erlang using Bullet. Go's websocket implementation is quite far from complete and has been a headache to work with. I'm hoping to have something a lot more robust with Erlang. > > So, let me get this straight? init handles the connection setup and terminate handles the teardown. So, I can make calls in those two functions to add the user to a room collection or removing her. > > What are the differences between stream and info? When do those come into play? I'm assuming one (stream?) is invoked when data is sent from the end user. Is that right? > > Thanks, > Lee > > > > On 2 Apr 2013, at 21:35, Lo?c Hoguin wrote: > >> Rooms is a bit out of scope. I agree it could use some more documentation though. >> >> The basic idea is that when your process connects its pid is added to a list somewhere (gproc, gen_server, anything else). When a message is sent to the room, all pids in the list receive it. You can also broadcast when someone enters or leave the room. And you remove the pid from the list when it's gone (using monitors for example). >> >> On 04/02/2013 10:27 PM, Lee Sylvester wrote: >>> Hi Lo?c, >>> >>> Yes, I spotted that. It just doesn't seem very explanatory. I was hoping for something a little bit more extensive. Preferably something that used the concept of rooms. The NineNines support page for Bullet is also a little thin? :-S I guess I'm too much of a noob. >>> >>> Thanks, >>> Lee >>> >>> >>> >>> On 2 Apr 2013, at 21:23, Lo?c Hoguin wrote: >>> >>>> On 04/02/2013 10:08 PM, Lee Sylvester wrote: >>>>> Hey guys, >>>>> >>>>> Does anyone here know of a good tutorial or chat example of NineNines Bullet handler for Cowboy? >>>> >>>> Have you looked at the clock example that comes with it? Building a chat on top of that is just making connection processes talk to each other using a registry like gproc or a gen_server that dispatches messages. >>>> >>>> -- >>>> Lo?c Hoguin >>>> Erlang Cowboy >>>> Nine Nines >>>> http://ninenines.eu >>> >> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> http://ninenines.eu > From essen@REDACTED Tue Apr 2 23:09:31 2013 From: essen@REDACTED (=?windows-1252?Q?Lo=EFc_Hoguin?=) Date: Tue, 02 Apr 2013 23:09:31 +0200 Subject: [erlang-questions] NineNines Bullet In-Reply-To: References: <19C75ED6-E427-4099-B4A7-75FC382345F9@gmail.com> <515B3E39.6010009@ninenines.eu> <75D865DE-95F6-4582-B90F-664F6808DF2B@gmail.com> <515B410F.2080704@ninenines.eu> Message-ID: <515B490B.8020302@ninenines.eu> That's the idea. As for the rest: stream = message from the browser, info = message from erlang On 04/02/2013 11:00 PM, Lee Sylvester wrote: > Yes, that's how I was seeing it. Mentally at least. I've got a similar thing built in Go which I want to port to Erlang using Bullet. Go's websocket implementation is quite far from complete and has been a headache to work with. I'm hoping to have something a lot more robust with Erlang. > > So, let me get this straight? init handles the connection setup and terminate handles the teardown. So, I can make calls in those two functions to add the user to a room collection or removing her. > > What are the differences between stream and info? When do those come into play? I'm assuming one (stream?) is invoked when data is sent from the end user. Is that right? > > Thanks, > Lee > > > > On 2 Apr 2013, at 21:35, Lo?c Hoguin wrote: > >> Rooms is a bit out of scope. I agree it could use some more documentation though. >> >> The basic idea is that when your process connects its pid is added to a list somewhere (gproc, gen_server, anything else). When a message is sent to the room, all pids in the list receive it. You can also broadcast when someone enters or leave the room. And you remove the pid from the list when it's gone (using monitors for example). >> >> On 04/02/2013 10:27 PM, Lee Sylvester wrote: >>> Hi Lo?c, >>> >>> Yes, I spotted that. It just doesn't seem very explanatory. I was hoping for something a little bit more extensive. Preferably something that used the concept of rooms. The NineNines support page for Bullet is also a little thin? :-S I guess I'm too much of a noob. >>> >>> Thanks, >>> Lee >>> >>> >>> >>> On 2 Apr 2013, at 21:23, Lo?c Hoguin wrote: >>> >>>> On 04/02/2013 10:08 PM, Lee Sylvester wrote: >>>>> Hey guys, >>>>> >>>>> Does anyone here know of a good tutorial or chat example of NineNines Bullet handler for Cowboy? >>>> >>>> Have you looked at the clock example that comes with it? Building a chat on top of that is just making connection processes talk to each other using a registry like gproc or a gen_server that dispatches messages. >>>> >>>> -- >>>> Lo?c Hoguin >>>> Erlang Cowboy >>>> Nine Nines >>>> http://ninenines.eu >>> >> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> http://ninenines.eu > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From essen@REDACTED Tue Apr 2 23:10:25 2013 From: essen@REDACTED (=?windows-1252?Q?Lo=EFc_Hoguin?=) Date: Tue, 02 Apr 2013 23:10:25 +0200 Subject: [erlang-questions] NineNines Bullet In-Reply-To: <14D83CB9-EC20-4D40-B4D6-9457507EE7BA@gmail.com> References: <19C75ED6-E427-4099-B4A7-75FC382345F9@gmail.com> <515B3E39.6010009@ninenines.eu> <75D865DE-95F6-4582-B90F-664F6808DF2B@gmail.com> <515B410F.2080704@ninenines.eu> <14D83CB9-EC20-4D40-B4D6-9457507EE7BA@gmail.com> Message-ID: <515B4941.2020002@ninenines.eu> Pid ! {you_got_mail, TheMessage} and receive it in info: info({you_got_mail, TheMessage}, ...) -> {reply, TheMessage, ...}; On 04/02/2013 11:09 PM, Lee Sylvester wrote: > Another thing; I'm guessing returning {reply,?} from the stream function sends a reply to the user for that connection and returning {ok,?} doesn't. How, though, do I send a message to some other specific user, providing I have their Pid? > > Thanks, > Lee > > > > > On 2 Apr 2013, at 22:00, Lee Sylvester wrote: > >> Yes, that's how I was seeing it. Mentally at least. I've got a similar thing built in Go which I want to port to Erlang using Bullet. Go's websocket implementation is quite far from complete and has been a headache to work with. I'm hoping to have something a lot more robust with Erlang. >> >> So, let me get this straight? init handles the connection setup and terminate handles the teardown. So, I can make calls in those two functions to add the user to a room collection or removing her. >> >> What are the differences between stream and info? When do those come into play? I'm assuming one (stream?) is invoked when data is sent from the end user. Is that right? >> >> Thanks, >> Lee >> >> >> >> On 2 Apr 2013, at 21:35, Lo?c Hoguin wrote: >> >>> Rooms is a bit out of scope. I agree it could use some more documentation though. >>> >>> The basic idea is that when your process connects its pid is added to a list somewhere (gproc, gen_server, anything else). When a message is sent to the room, all pids in the list receive it. You can also broadcast when someone enters or leave the room. And you remove the pid from the list when it's gone (using monitors for example). >>> >>> On 04/02/2013 10:27 PM, Lee Sylvester wrote: >>>> Hi Lo?c, >>>> >>>> Yes, I spotted that. It just doesn't seem very explanatory. I was hoping for something a little bit more extensive. Preferably something that used the concept of rooms. The NineNines support page for Bullet is also a little thin? :-S I guess I'm too much of a noob. >>>> >>>> Thanks, >>>> Lee >>>> >>>> >>>> >>>> On 2 Apr 2013, at 21:23, Lo?c Hoguin wrote: >>>> >>>>> On 04/02/2013 10:08 PM, Lee Sylvester wrote: >>>>>> Hey guys, >>>>>> >>>>>> Does anyone here know of a good tutorial or chat example of NineNines Bullet handler for Cowboy? >>>>> >>>>> Have you looked at the clock example that comes with it? Building a chat on top of that is just making connection processes talk to each other using a registry like gproc or a gen_server that dispatches messages. >>>>> >>>>> -- >>>>> Lo?c Hoguin >>>>> Erlang Cowboy >>>>> Nine Nines >>>>> http://ninenines.eu >>>> >>> >>> >>> -- >>> Lo?c Hoguin >>> Erlang Cowboy >>> Nine Nines >>> http://ninenines.eu >> > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From lee.sylvester@REDACTED Tue Apr 2 23:10:57 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 2 Apr 2013 22:10:57 +0100 Subject: [erlang-questions] NineNines Bullet In-Reply-To: <515B490B.8020302@ninenines.eu> References: <19C75ED6-E427-4099-B4A7-75FC382345F9@gmail.com> <515B3E39.6010009@ninenines.eu> <75D865DE-95F6-4582-B90F-664F6808DF2B@gmail.com> <515B410F.2080704@ninenines.eu> <515B490B.8020302@ninenines.eu> Message-ID: <26A834F9-1111-4E9D-86E4-32045584A1D5@gmail.com> I see, so info handles messages I've sent straight to the Pid using standard erlang messaging. If I return {reply,?} to that function, does it send the returned data to the user associated with that info function? Thanks, Lee On 2 Apr 2013, at 22:09, Lo?c Hoguin wrote: > That's the idea. > > As for the rest: > stream = message from the browser, info = message from erlang > > On 04/02/2013 11:00 PM, Lee Sylvester wrote: >> Yes, that's how I was seeing it. Mentally at least. I've got a similar thing built in Go which I want to port to Erlang using Bullet. Go's websocket implementation is quite far from complete and has been a headache to work with. I'm hoping to have something a lot more robust with Erlang. >> >> So, let me get this straight? init handles the connection setup and terminate handles the teardown. So, I can make calls in those two functions to add the user to a room collection or removing her. >> >> What are the differences between stream and info? When do those come into play? I'm assuming one (stream?) is invoked when data is sent from the end user. Is that right? >> >> Thanks, >> Lee >> >> >> >> On 2 Apr 2013, at 21:35, Lo?c Hoguin wrote: >> >>> Rooms is a bit out of scope. I agree it could use some more documentation though. >>> >>> The basic idea is that when your process connects its pid is added to a list somewhere (gproc, gen_server, anything else). When a message is sent to the room, all pids in the list receive it. You can also broadcast when someone enters or leave the room. And you remove the pid from the list when it's gone (using monitors for example). >>> >>> On 04/02/2013 10:27 PM, Lee Sylvester wrote: >>>> Hi Lo?c, >>>> >>>> Yes, I spotted that. It just doesn't seem very explanatory. I was hoping for something a little bit more extensive. Preferably something that used the concept of rooms. The NineNines support page for Bullet is also a little thin? :-S I guess I'm too much of a noob. >>>> >>>> Thanks, >>>> Lee >>>> >>>> >>>> >>>> On 2 Apr 2013, at 21:23, Lo?c Hoguin wrote: >>>> >>>>> On 04/02/2013 10:08 PM, Lee Sylvester wrote: >>>>>> Hey guys, >>>>>> >>>>>> Does anyone here know of a good tutorial or chat example of NineNines Bullet handler for Cowboy? >>>>> >>>>> Have you looked at the clock example that comes with it? Building a chat on top of that is just making connection processes talk to each other using a registry like gproc or a gen_server that dispatches messages. >>>>> >>>>> -- >>>>> Lo?c Hoguin >>>>> Erlang Cowboy >>>>> Nine Nines >>>>> http://ninenines.eu >>>> >>> >>> >>> -- >>> Lo?c Hoguin >>> Erlang Cowboy >>> Nine Nines >>> http://ninenines.eu >> > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu From lee.sylvester@REDACTED Tue Apr 2 23:11:27 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 2 Apr 2013 22:11:27 +0100 Subject: [erlang-questions] NineNines Bullet In-Reply-To: <515B4941.2020002@ninenines.eu> References: <19C75ED6-E427-4099-B4A7-75FC382345F9@gmail.com> <515B3E39.6010009@ninenines.eu> <75D865DE-95F6-4582-B90F-664F6808DF2B@gmail.com> <515B410F.2080704@ninenines.eu> <14D83CB9-EC20-4D40-B4D6-9457507EE7BA@gmail.com> <515B4941.2020002@ninenines.eu> Message-ID: <137E5D91-84C0-4117-B0E6-7835F1049A2D@gmail.com> Perfect :-) Thanks Lo?c. Lee On 2 Apr 2013, at 22:10, Lo?c Hoguin wrote: > Pid ! {you_got_mail, TheMessage} and receive it in info: > > info({you_got_mail, TheMessage}, ...) -> > {reply, TheMessage, ...}; > > On 04/02/2013 11:09 PM, Lee Sylvester wrote: >> Another thing; I'm guessing returning {reply,?} from the stream function sends a reply to the user for that connection and returning {ok,?} doesn't. How, though, do I send a message to some other specific user, providing I have their Pid? >> >> Thanks, >> Lee >> >> >> >> >> On 2 Apr 2013, at 22:00, Lee Sylvester wrote: >> >>> Yes, that's how I was seeing it. Mentally at least. I've got a similar thing built in Go which I want to port to Erlang using Bullet. Go's websocket implementation is quite far from complete and has been a headache to work with. I'm hoping to have something a lot more robust with Erlang. >>> >>> So, let me get this straight? init handles the connection setup and terminate handles the teardown. So, I can make calls in those two functions to add the user to a room collection or removing her. >>> >>> What are the differences between stream and info? When do those come into play? I'm assuming one (stream?) is invoked when data is sent from the end user. Is that right? >>> >>> Thanks, >>> Lee >>> >>> >>> >>> On 2 Apr 2013, at 21:35, Lo?c Hoguin wrote: >>> >>>> Rooms is a bit out of scope. I agree it could use some more documentation though. >>>> >>>> The basic idea is that when your process connects its pid is added to a list somewhere (gproc, gen_server, anything else). When a message is sent to the room, all pids in the list receive it. You can also broadcast when someone enters or leave the room. And you remove the pid from the list when it's gone (using monitors for example). >>>> >>>> On 04/02/2013 10:27 PM, Lee Sylvester wrote: >>>>> Hi Lo?c, >>>>> >>>>> Yes, I spotted that. It just doesn't seem very explanatory. I was hoping for something a little bit more extensive. Preferably something that used the concept of rooms. The NineNines support page for Bullet is also a little thin? :-S I guess I'm too much of a noob. >>>>> >>>>> Thanks, >>>>> Lee >>>>> >>>>> >>>>> >>>>> On 2 Apr 2013, at 21:23, Lo?c Hoguin wrote: >>>>> >>>>>> On 04/02/2013 10:08 PM, Lee Sylvester wrote: >>>>>>> Hey guys, >>>>>>> >>>>>>> Does anyone here know of a good tutorial or chat example of NineNines Bullet handler for Cowboy? >>>>>> >>>>>> Have you looked at the clock example that comes with it? Building a chat on top of that is just making connection processes talk to each other using a registry like gproc or a gen_server that dispatches messages. >>>>>> >>>>>> -- >>>>>> Lo?c Hoguin >>>>>> Erlang Cowboy >>>>>> Nine Nines >>>>>> http://ninenines.eu >>>>> >>>> >>>> >>>> -- >>>> Lo?c Hoguin >>>> Erlang Cowboy >>>> Nine Nines >>>> http://ninenines.eu >>> >> > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu From ok@REDACTED Tue Apr 2 23:31:17 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 3 Apr 2013 10:31:17 +1300 Subject: [erlang-questions] 2 Erlang articles @news.yc: 1) Armstrong - Right Problem. 2) Go vs Erlang In-Reply-To: References: <1364426760.13181.140661210075249.056372B4@webmail.messagingengine.com> Message-ID: I've just started reading http://joearms.github.com. I always thought the right word to describe Erlang was "lagom". From brisa.jimenez.f@REDACTED Wed Apr 3 01:09:02 2013 From: brisa.jimenez.f@REDACTED (=?ISO-8859-1?Q?Brisa_Jim=E9nez?=) Date: Tue, 2 Apr 2013 17:09:02 -0600 Subject: [erlang-questions] how to know who is the parent pid of a child pid? Message-ID: Hi everyone, I have a question: How to know who is the parent pid of a child pid? Thank you very much! -------------- next part -------------- An HTML attachment was scrubbed... URL: From brisa.jimenez.f@REDACTED Wed Apr 3 01:10:12 2013 From: brisa.jimenez.f@REDACTED (=?ISO-8859-1?Q?Brisa_Jim=E9nez?=) Date: Tue, 2 Apr 2013 17:10:12 -0600 Subject: [erlang-questions] how to know who is the caller id from an httpc:request? Message-ID: Hi, Another question: How to know who is the caller id from an httpc:request? Thank again! -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Wed Apr 3 01:21:52 2013 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 2 Apr 2013 19:21:52 -0400 Subject: [erlang-questions] how to know who is the parent pid of a child pid? In-Reply-To: References: Message-ID: <20130402232151.GA96512@ferdmbp.local> Short answer: There is no way. You need to have the parent give it to the child through a message or as an argument when spawning it. Slightly longer answer: OTP hides the parent in the process dictionary (useful when handling shutdown messages from supervisors and trapping exits). OTP processes thus have the parent accessible. Long answer: Do not use OTP internals to do your bidding in there. They belong to OTP libraries and you should stay out of its way (similarly, you shouldn't play with links set by OTP or other pdict values). For debugging purposes, however, given it's read only and short-lived in terms of how much you depend on the information, using the process dictionary's value is fine. You can do it by calling `process_info(Pid, dictionary)` and hoping nobody shoved a GB of data in there. For most production purposes outside of debugging, if you need to pass a parent, do it as an argument to a function that spawns the process. This will make it easier to test things and the dependency will be explicit, so fewer people get surprise breakage when they 'give away' the process to a new adoptive parent (think of how people do it with ports, sockets, or ETS tables). Being explicit is usually the safest option, and the most readable one too. Regards, Fred. On 04/02, Brisa Jim?nez wrote: > Hi everyone, > I have a question: > How to know who is the parent pid of a child pid? > Thank you very much! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Wed Apr 3 01:31:07 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 3 Apr 2013 12:31:07 +1300 Subject: [erlang-questions] how to know who is the parent pid of a child pid? In-Reply-To: References: Message-ID: <872ACFA7-1134-4AE7-8B26-8E2BD0E6714D@cs.otago.ac.nz> On 3/04/2013, at 12:09 PM, Brisa Jim?nez wrote: > How to know who is the parent pid of a child pid? It isn't clear whether you are asking "if process A spawned process B, can process B discover who A was?" or "if process A spawned process B and process C somehow becomes aquainted with B, can C discover who A was?" In either case, the obvious question is "why do you want to know this?" From john@REDACTED Wed Apr 3 01:47:34 2013 From: john@REDACTED (John Kemp) Date: Tue, 2 Apr 2013 19:47:34 -0400 Subject: [erlang-questions] how to know who is the caller id from an httpc:request? In-Reply-To: References: Message-ID: <9A45DD0A-3338-404E-8319-BA9E83D109F8@jkemp.net> On Apr 2, 2013, at 7:10 PM, Brisa Jim?nez wrote: > Hi, > Another question: > How to know who is the caller id from an httpc:request? If you are using a (generic) HTTP server to receive the request, you could set, in the httpc request, the User-Agent HTTP header to a known value and then extract the vaue of the User-Agent HTTP header at your server. Another option would be to set a cookie from the server to client upon first connection, and then send the cookie on all subsequent requests from httpc. Depending on what HTTP server technology you're using, there may also be other mechanisms available. What HTTP server are you using? Cheers, JohnK > Thank again! > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From naikvin@REDACTED Wed Apr 3 04:44:55 2013 From: naikvin@REDACTED (Vineet Naik) Date: Wed, 3 Apr 2013 08:14:55 +0530 Subject: [erlang-questions] Mysql Query builder lib in Erlang? In-Reply-To: References: Message-ID: erlsql.erl looks good. I will give it a try. Thanks On Tue, Apr 2, 2013 at 6:22 PM, nx wrote: > I remember coming across erlsql while checking out the erlyweb project a > while back: > https://github.com/yariv/erlyweb/blob/master/src/erlsql/erlsql.erl. > > > On Tue, Apr 2, 2013 at 7:47 AM, Vineet Naik wrote: > >> Hi, >> >> Is there a mysql query builder library for Erlang? On some searching, >> I came across boss_db[1] which is an ORM. While I am open to ORMs and >> planning to give it a try, I would prefer a simple query builder >> >> [1]: https://github.com/evanmiller/boss_db >> >> Regards, >> Vineet >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- Vineet Naik -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Apr 3 09:58:38 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 3 Apr 2013 11:58:38 +0400 Subject: [erlang-questions] Need help with async disk IO and thread pool on many devices (more than 1 Gbit/s ) Message-ID: Hi. Situation is following: erlyvideo is running on Xeon E5 server with 16 SATA HDD attached. RAID and other nightmare devices are removed, only raw disks are mounted in Linux. 2500 clients are watching video on speed about 1600 Mbit/s. They are downloading chunks of video via HTTP. Each chunk requires two disk pread calls: video and audio chunks, so disk is read by contiguous blocks of about 300-700 Kb. Everything is ok, response time is lower than 50 ms. But if video is uploaded to one HDD, it becomes unresponsible and whole server becomes unresponsible. I think that all 160 async threads gets blocked by pread calls to this device and thus whole system becomes unresponsible. So, problem is not in raw throughput and RPS, there is not more than 400 RPS. Problem is as usual: how to handle errors and make system very soft and responsive. Webserver must not fail with 500 error timeout after 5 seconds of waiting, it should respond 503 after several milliseconds. The problem, I've met is not specific for video, it may be common for any web server. I don't see any reasons why erlang should be slower than nginx, however I see some minor problems that doesn't allow to reach the same predictiveness. There are following ideas to solve this problem: 1) spawn separate node per each device and route all disk requests to it. Communicate via TCP. It is dumb, but it is a solution. 2) spawn pool of separate file reader process for each opened file. It may be even better than previous choice, because OS will fully schedule them. 3) add async disk io from linux to erlang VM. Question is: why it isn't already done? Maybe there are some problems with it? 4) add some affinity between ports and async threads. But frankly speaking it will require adding feature that allows dynamic changing size of this thread pool. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Wed Apr 3 10:08:26 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 3 Apr 2013 09:08:26 +0100 Subject: [erlang-questions] Distributed comms and Mnesia Message-ID: <80720DF1-239B-4C8D-B6EC-0D358E4B1793@gmail.com> Hey guys, So, based on my posts yesterday; I would like my websocket platform to be distributed. To help with this, I figured I'd use Mnesia. I can opt for the "in memory" option, as if a node goes down, the connections will be lost, anyway. Using Mnesia, I can tag users with their pid, so I can message them directly rather than sending messages to every node, whether or not a recipient exists there. Okay, so that's the theory out the door. Now for the impl. When sending messages, a user should be allowed to message a single person or all connected users in the "room". Rooms belong to an application (I might want use this framework for real time events, for example), so selecting users to message is based on the tuple {ApplicationId, RoomId, UserId}. This is my primary key. My question is, is it efficient to have such keys in Mnesia? I guess I could set the table up to be ordered, which would speed up requests for records. To get users by RoomId, I'd need to search using the key {ApplicationId, RoomId, _}. Is this the best way to handle this? Thanks, Lee From pan@REDACTED Wed Apr 3 10:24:00 2013 From: pan@REDACTED (Patrik Nyblom) Date: Wed, 3 Apr 2013 10:24:00 +0200 Subject: [erlang-questions] Dynamically add +A threads In-Reply-To: References: Message-ID: <515BE720.7080307@erlang.org> On 04/01/2013 11:23 AM, Max Lapshin wrote: > In some cases, I need +A 100, in some cases it is not required. > > Is it possible to change this number in runtime? No, you have to set it with the start up parameter. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Cheers, /Patrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon@REDACTED Wed Apr 3 10:50:54 2013 From: jon@REDACTED (Jon Schneider) Date: Wed, 3 Apr 2013 09:50:54 +0100 Subject: [erlang-questions] Need help with async disk IO and thread pool on many devices (more than 1 Gbit/s ) In-Reply-To: References: Message-ID: <2622371cef8fadc7a8a34abe609ba782.squirrel@mail.jschneider.net> > But if video is uploaded to one HDD, it becomes unresponsible and whole > server becomes unresponsible. It could be that your hardware simply cannot do uploads at the desired speed given optimal software on top. Perhaps you could regulate uploads according to current measured latency. Jon From max.lapshin@REDACTED Wed Apr 3 10:56:39 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 3 Apr 2013 12:56:39 +0400 Subject: [erlang-questions] Need help with async disk IO and thread pool on many devices (more than 1 Gbit/s ) In-Reply-To: <2622371cef8fadc7a8a34abe609ba782.squirrel@mail.jschneider.net> References: <2622371cef8fadc7a8a34abe609ba782.squirrel@mail.jschneider.net> Message-ID: You are trying to solve another problem. Hardware can _always_ be bad. But whole system must not go down if one disk is slow. Currently, whole erlang is becoming useless if only one hard drive from 16 doesn't respond. This is the problem. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pan@REDACTED Wed Apr 3 11:19:14 2013 From: pan@REDACTED (Patrik Nyblom) Date: Wed, 3 Apr 2013 11:19:14 +0200 Subject: [erlang-questions] Need help with async disk IO and thread pool on many devices (more than 1 Gbit/s ) In-Reply-To: References: <2622371cef8fadc7a8a34abe609ba782.squirrel@mail.jschneider.net> Message-ID: <515BF412.1080108@erlang.org> On 04/03/2013 10:56 AM, Max Lapshin wrote: > You are trying to solve another problem. > > Hardware can _always_ be bad. But whole system must not go down if one > disk is slow. > > Currently, whole erlang is becoming useless if only one hard drive > from 16 doesn't respond. > > This is the problem. Wouldn't you rather need to have one async pool per device? Or the option to bind certain files to specific async threads? The thing with dynamic async threads is that the implementation relies on requests for the same file descriptor being routed to the same thread for all of the descriptors lifespan. If the async thread hangs, other file descriptors already routed to that thread have to wait. You would need to have the "good" mapping between threads and devices from the beginning. Something like open_a_file(Name) -> PoolID = get_pool_id_from_name(Name), {ok, FD} = file:open(Name,{async_pool_id, PoolId}), FD. and get_pool_id_from_name/1 will look at the file name, figure out the device and return a pool id that is only used for that particular device? It could even be a specific thread id - you could always find out how many async threads you have, know how many devices you have and round robin files between the threads that should handle that particular device. In that case you would only need to add an interface where you can specify the thread when opening the file (and remenber which thread is connected to each port). Sounds like a fairly feasible hack... On a side note, are you using sendfile? That might at least make the ongoing connections run smooth while another device is blocked... Not that it addresses the real problem, but just wondering if you have tried using it - it might be good for performance in your application. Another, less compelling, solution would be to route only the writes through another process, so that you open a small "writer program" and pump the data to it through the spawn driver. Not a general solution either, but much simpler than to hack the file driver... The async thread pool leaves a lot to desire - we have planned to replace it with something called "dirty schedulers" (internal name...), which is in our roadmap for R17. In the end they should solve a lot of the problems with the current implementation. Not that that helps you now, but it makes me wonder if not a "dirty" solution might be the right way to solve it for the moment, as another solution is in the pipe... > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Cheers, /Patrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Apr 3 11:25:13 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 3 Apr 2013 13:25:13 +0400 Subject: [erlang-questions] Need help with async disk IO and thread pool on many devices (more than 1 Gbit/s ) In-Reply-To: <515BF412.1080108@erlang.org> References: <2622371cef8fadc7a8a34abe609ba782.squirrel@mail.jschneider.net> <515BF412.1080108@erlang.org> Message-ID: sendfile doesn't work for me, because I need to postprocess data from file. thread pools can really help, because I can read major_device from read_file_info and route all requests to this pool id. and right now it seems that I need to launch dozens of external workers that just read and reply with block from opened file. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pan@REDACTED Wed Apr 3 11:39:26 2013 From: pan@REDACTED (Patrik Nyblom) Date: Wed, 3 Apr 2013 11:39:26 +0200 Subject: [erlang-questions] Need help with async disk IO and thread pool on many devices (more than 1 Gbit/s ) In-Reply-To: References: <2622371cef8fadc7a8a34abe609ba782.squirrel@mail.jschneider.net> <515BF412.1080108@erlang.org> Message-ID: <515BF8CE.70004@erlang.org> On 04/03/2013 11:25 AM, Max Lapshin wrote: > sendfile doesn't work for me, because I need to postprocess data from > file. > > thread pools can really help, because I can read major_device from > read_file_info and route all requests to this pool id. > > and right now it seems that I need to launch dozens of external > workers that just read and reply with block from opened file. Yes, right, writers in separate processes doesn't help - you have to isolate the reader threads for that device, my mind slipped there :) You need to have either a new driver that reads the files or a separate process if you're not to patch the efile_driver and prim_file.erl... Cheers, /Patrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Apr 3 11:47:32 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 3 Apr 2013 13:47:32 +0400 Subject: [erlang-questions] Need help with async disk IO and thread pool on many devices (more than 1 Gbit/s ) In-Reply-To: <515BF8CE.70004@erlang.org> References: <2622371cef8fadc7a8a34abe609ba782.squirrel@mail.jschneider.net> <515BF412.1080108@erlang.org> <515BF8CE.70004@erlang.org> Message-ID: Yes, there are two good ways to go: 1) add async IO for those platforms that can (FreeBSD, Linux), it will remove requirement to use async threads 2) lock driver to async thread pool -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Wed Apr 3 11:51:38 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 3 Apr 2013 11:51:38 +0200 Subject: [erlang-questions] using inet_gethost_native even if disabled Message-ID: Hi, I compiled Erlang with --disable-native-libs --disable-options but it is still trying to use it on load.. Did I miss an option? - beno?t From z@REDACTED Wed Apr 3 11:57:57 2013 From: z@REDACTED (Danil Zagoskin) Date: Wed, 3 Apr 2013 13:57:57 +0400 Subject: [erlang-questions] using inet_gethost_native even if disabled In-Reply-To: References: Message-ID: Hello. Try to start your VM with custom inetrc. Just set ERL_INETRC environment variable to inet config file path. In config file write: {lookup, [file, dns]}. This will tell erlang's resolver to use it's own DNS client, not system one. More info at http://www.erlang.org/doc/apps/erts/inet_cfg.html 2013/4/3 Benoit Chesneau > Hi, > > I compiled Erlang with --disable-native-libs --disable-options but it > is still trying to use it on load.. Did I miss an option? > > - beno?t > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- --------------------------------------------- ????? ???????? | +7 906 064 20 47 | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenneth.lundin@REDACTED Wed Apr 3 12:00:27 2013 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 3 Apr 2013 12:00:27 +0200 Subject: [erlang-questions] using inet_gethost_native even if disabled In-Reply-To: References: Message-ID: the use of inet_gethost_native for making DNS requests the native OS way is not related at all with the configure option "diable-native-libs" which controls if the erlang libs stdlib, kernel etc should be native compiled (with the Hipe compiler) or not. /Kenneth, Erlang/OTP Ericsson On Wed, Apr 3, 2013 at 11:51 AM, Benoit Chesneau wrote: > Hi, > > I compiled Erlang with --disable-native-libs --disable-options but it > is still trying to use it on load.. Did I miss an option? > > - beno?t > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Wed Apr 3 12:00:40 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Wed, 3 Apr 2013 12:00:40 +0200 Subject: [erlang-questions] using inet_gethost_native even if disabled In-Reply-To: References: Message-ID: <79C76474-9A71-40AD-8C84-976F4351382A@gmail.com> Hello Benoit, The configure flag --enable-native-libs natively compiles stdlib and its friends with HiPE and has nothing to do with the thing you are trying to disable. Regards, -- Anthony Ramine Le 3 avr. 2013 ? 11:51, Benoit Chesneau a ?crit : > Hi, > > I compiled Erlang with --disable-native-libs --disable-options but it > is still trying to use it on load.. Did I miss an option? > > - beno?t > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From pan@REDACTED Wed Apr 3 12:01:46 2013 From: pan@REDACTED (Patrik Nyblom) Date: Wed, 3 Apr 2013 12:01:46 +0200 Subject: [erlang-questions] Need help with async disk IO and thread pool on many devices (more than 1 Gbit/s ) In-Reply-To: References: <2622371cef8fadc7a8a34abe609ba782.squirrel@mail.jschneider.net> <515BF412.1080108@erlang.org> <515BF8CE.70004@erlang.org> Message-ID: <515BFE0A.3040800@erlang.org> On 04/03/2013 11:47 AM, Max Lapshin wrote: > Yes, there are two good ways to go: > > 1) add async IO for those platforms that can (FreeBSD, Linux), it will > remove requirement to use async threads > 2) lock driver to async thread pool If I were you, I would try the second approach. Aio would more or less require a new driver. The upside of Aio, if you get it to work properly, would be that you do not need to get a patch accepted, you could have the aio driver solution in your own codebase. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Wed Apr 3 12:04:44 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 3 Apr 2013 12:04:44 +0200 Subject: [erlang-questions] using inet_gethost_native even if disabled In-Reply-To: References: Message-ID: On Wed, Apr 3, 2013 at 12:00 PM, Kenneth Lundin wrote: > the use of inet_gethost_native for making DNS requests the native OS way is > not related at all > with the configure option "diable-native-libs" which controls if the erlang > libs stdlib, kernel etc should be native compiled (with the Hipe compiler) > or not. > > /Kenneth, Erlang/OTP Ericsson > > I tried that because putthing `{lookup, [file,dns]}` in erl_inetrc is apparently not enough to disable it. I'm not sure why actually :/ - beno?t From mrtndimitrov@REDACTED Wed Apr 3 12:58:55 2013 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Wed, 03 Apr 2013 13:58:55 +0300 Subject: [erlang-questions] starting eprof for all processes Message-ID: <515C0B6F.5030503@gmail.com> Hi, I try 1> eprof:start(). {ok,<0.146.0>} 2> eprof:start_profiling(processes()). error How can I start profiling all processes? Thank you, Martin From erlang@REDACTED Wed Apr 3 13:25:07 2013 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 3 Apr 2013 13:25:07 +0200 Subject: [erlang-questions] how to know who is the parent pid of a child pid? In-Reply-To: References: Message-ID: Wise sonnes they be in very deede, That knowe their Parents who did them breede. ?A shorte dictionarie for yonge begynners? [John Withals, 1584] It's not possible. The parent has to pass this information to the child. Something like: make_child() -> S = self(), spawn(fun() -> start_child(S) end). start_child(Parent) -> ... /Joe On Wed, Apr 3, 2013 at 1:09 AM, Brisa Jim?nez wrote: > Hi everyone, > I have a question: > How to know who is the parent pid of a child pid? > Thank you very much! > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From egil@REDACTED Wed Apr 3 13:42:16 2013 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Wed, 3 Apr 2013 13:42:16 +0200 Subject: [erlang-questions] starting eprof for all processes In-Reply-To: <515C0B6F.5030503@gmail.com> References: <515C0B6F.5030503@gmail.com> Message-ID: <515C1598.1090701@erlang.org> On 2013-04-03 12:58, Martin Dimitrov wrote: > Hi, > > I try > > 1> eprof:start(). > {ok,<0.146.0>} > 2> eprof:start_profiling(processes()). > error > > How can I start profiling all processes? 1> {ok, P} = eprof:start(). {ok,<0.146.0>} 2> eprof:start_profiling(processes() -- [P]). profiling It will not trace itself. // Bj?rn-Egil > > Thank you, > > Martin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From bchesneau@REDACTED Wed Apr 3 14:47:35 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 3 Apr 2013 14:47:35 +0200 Subject: [erlang-questions] using inet_gethost_native even if disabled In-Reply-To: References: Message-ID: On Wed, Apr 3, 2013 at 11:57 AM, Danil Zagoskin wrote: > Hello. > > Try to start your VM with custom inetrc. Just set ERL_INETRC environment > variable to inet config file path. > In config file write: > {lookup, [file, dns]}. > > This will tell erlang's resolver to use it's own DNS client, not system one. > More info at http://www.erlang.org/doc/apps/erts/inet_cfg.html > mmm thanks, I will try to set the env I was using the option `-kernel inetrc /path/to/erl_inetrc` for now but it doesn't seem to work . - beno?t From bchesneau@REDACTED Wed Apr 3 14:50:29 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 3 Apr 2013 14:50:29 +0200 Subject: [erlang-questions] using inet_gethost_native even if disabled In-Reply-To: References: Message-ID: So I tried to use the environment variable and it is still using the native resolving: Here is the trace: I/ErlangThread( 1447): default inetrc path: /data/data/com.example.app/erlang/bin/erl_inetrc V/ErlangThread( 1447): {error_logger,{{2013,4,3},{14,45,27}},"~s~n",["Error in process <0.17.0> with exit value: {epipe,[{erts_internal,port_command,[#Port<0.54>,[<<6 bytes>>,\"localhost\",0],[]],[]},{erlang,port_command,2,[]},{inet_gethost_native,run_once,0,[{file,\"inet_gethost_native.erl\"},{line,152}]}]}\n"]} The first line display what is passed to setenv. And when I read the erl_inetrc content on the device I have: # cat /data/data/com.daleharvey.mobilefuton/erlang/bin/erl_inetrc %% Use erlang's internal resolver. Native DNS on android is annoying {lookup, [file,dns]}. not sure what's the problem there... - beno?t On Wed, Apr 3, 2013 at 2:47 PM, Benoit Chesneau wrote: > On Wed, Apr 3, 2013 at 11:57 AM, Danil Zagoskin wrote: >> Hello. >> >> Try to start your VM with custom inetrc. Just set ERL_INETRC environment >> variable to inet config file path. >> In config file write: >> {lookup, [file, dns]}. >> >> This will tell erlang's resolver to use it's own DNS client, not system one. >> More info at http://www.erlang.org/doc/apps/erts/inet_cfg.html >> > > mmm thanks, I will try to set the env I was using the option `-kernel > inetrc /path/to/erl_inetrc` for now but it doesn't seem to work . > > - beno?t From paul.cager@REDACTED Wed Apr 3 14:18:12 2013 From: paul.cager@REDACTED (Paul Cager) Date: Wed, 3 Apr 2013 05:18:12 -0700 (PDT) Subject: [erlang-questions] Using public_key:sign/3 gives error badkey (crypto.erl, line 852) Message-ID: <43abb24b-655c-4941-9fee-e36a2397cf9d@googlegroups.com> I can successfully RSA sign a message using lower-level functions such as crypto:mpint, crypto:rsa_private_encrypt (e.g. as in the samples at http://lj.rossia.org/users/eacher/2013/01/07/). However I'd prefer to use what looks like higher-level functions in the public_key module but I'm not sure what I'm doing wrong. The following sample program fails with "badkey" - any ideas what is wrong? (Erlang R16B) -module(test_crypto). -compile(export_all). -include_lib("public_key/include/public_key.hrl"). -include_lib("eunit/include/eunit.hrl"). -define(TEST_DATA, <<"Test">>). sign_test() -> application:start(crypto), [Entry] = public_key:pem_decode(test_priv_key()), PrivateKey = public_key:pem_entry_decode(Entry), ?assert(is_record(PrivateKey, 'RSAPrivateKey')), #'RSAPrivateKey'{ modulus = N, publicExponent = E, privateExponent = D} = PrivateKey, io:format("Decoded: ~p~n N=~p~n E=~p~n D=~p~n", [PrivateKey, N, E, D]), Signed = public_key:sign(?TEST_DATA, md5, PrivateKey), io:format("signed: ~p~n", [Signed]), error(force_failure). test_priv_key() -> iolist_to_binary([ "-----BEGIN RSA PRIVATE KEY-----", $\n, "MIGqAgEAAiEAsfQgn015/7zGm2Hy0wh28voJTcDM5vXbQIb47IRvPkECAwEAAQIg", $\n, "Ibi0Udu5qaeZLyqxfCIXgATS+x7mDA0tMJXtheJCkUECEQDWLuQfQZJEJOZq05eM", $\n, "EFypAhEA1LJxrD0/ZVA8u7FSlV/72QIRALr5BkXO3RvFxqv8+K/Z3LkCEBQpJeUx", $\n, "mQS9akRQNoe6JWECEDR94PUjkOWSkZrCwpsXbbg=", $\n, "-----END RSA PRIVATE KEY-----", $\n]). test_pub_key() -> iolist_to_binary([ "-----BEGIN PUBLIC KEY-----", $\n, "MDwwDQYJKoZIhvcNAQEBBQADKwAwKAIhALH0IJ9Nef+8xpth8tMIdvL6CU3AzOb1", $\n, "20CG+OyEbz5BAgMBAAE=", $\n, "-----END PUBLIC KEY-----", $\n]). -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlo.bertoldi@REDACTED Wed Apr 3 15:11:42 2013 From: carlo.bertoldi@REDACTED (Carlo Bertoldi) Date: Wed, 03 Apr 2013 15:11:42 +0200 Subject: [erlang-questions] Compiling R16B on Mac OS X Mountain Lion In-Reply-To: <51473633.4070507@erlang.org> References: <513DE820.1060900@ubiquity.it> <51405EC3.10502@erlang.org> <51418FD6.1070709@ubiquity.it> <5141A5DB.7070307@ubiquity.it> <51473633.4070507@erlang.org> Message-ID: <515C2A8E.7010507@ubiquity.it> On 18/03/13 16:43, Patrik Nyblom wrote: > Hi! > > Are you using macports gcc or apples llvm-wrapper in this case? If > macports, did you install the +universal variant? > gcc45 @4.5.4_6+universal Just for the records, I've finally managed to find the problem. During the various try the memsup was compiled for a 32bit architecture. I've ran make clean many times, but that didn't deleted the file memsup.o. I discovered this since I removed the whole source folder and started from scratch. Eventually, the compilation went fine. Cheers, Carlo From jani.j.hakala@REDACTED Wed Apr 3 16:46:12 2013 From: jani.j.hakala@REDACTED (Jani Hakala) Date: Wed, 03 Apr 2013 17:46:12 +0300 Subject: [erlang-questions] oe_obj? In-Reply-To: (Chris King's message of "Sat, 30 Mar 2013 19:26:06 -0400") References: Message-ID: <87d2ubu0y3.fsf@pingviini.dyndns.org> Chris King writes: > All the generated stubs seem to want an "oe_obj" parameter of the type of > my interface (which is typedef'd to CORBA_Object). My assumption is that > this is analogous to the OE_THIS parameter required by the stubs generated > by the erl_genserver backend (i.e. it's the gen_server's PID), but: > My guess is that oe_obj is what is usually named user_data (in ic.h there is typedef void* CORBA_Object). Jani Hakala From bombadil@REDACTED Wed Apr 3 18:32:38 2013 From: bombadil@REDACTED (Manuel A. Rubio "Bombadil") Date: Wed, 03 Apr 2013 18:32:38 +0200 Subject: [erlang-questions] erlang:now() vs os:timestamp() Message-ID: Hi, I saw a keynote[1] from Rick Reed (WhatsApp) and he said: "Prefer os:timestamp to erlang:now"... I did a benchmark to test it[2], and get the following results: os:timestamp = 1.44 1.38 1.38 erlang:now = 1.01 1.00 1.00 I know that he says "prefer" and not "better"... but the benchmark says that he should be say the opposite :-P What do you think? [1] http://www.erlang-factory.com/upload/presentations/558/efsf2012-whatsapp-scaling.pdf [2] https://gist.github.com/manuel-rubio/5302361 Regards. Manuel Rubio. From mononcqc@REDACTED Wed Apr 3 18:41:02 2013 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 3 Apr 2013 12:41:02 -0400 Subject: [erlang-questions] erlang:now() vs os:timestamp() In-Reply-To: References: Message-ID: <20130403164038.GA83547@ferdair.local> Many things are missing in the benchmark, one of which is concurrency and parallelism. The calls you make are purely sequential, and you'll have pretty much no lock contention on there. now() is a monotonic function that always increases and thus requires a global lock to read. It's generally very fast, but calling it hundreds of times per second over thousands of processes is going to be worse than calling os:timestamp() assuming your system doesn't add such a lock to the os:timestamp() call. I'd have to agree with the advice -- in practice, I tend to avoid using now() and use os:timestamp(), unless I know I want my time to be monotonic (and unique, albeit predictable). Regards, Fred. On 04/03, Manuel A. Rubio "Bombadil" wrote: > Hi, > > I saw a keynote[1] from Rick Reed (WhatsApp) and he said: "Prefer > os:timestamp to erlang:now"... I did a benchmark to test it[2], and > get the following results: > > os:timestamp = 1.44 1.38 1.38 > erlang:now = 1.01 1.00 1.00 > > I know that he says "prefer" and not "better"... but the benchmark > says that he should be say the opposite :-P > > What do you think? > > [1] http://www.erlang-factory.com/upload/presentations/558/efsf2012-whatsapp-scaling.pdf > [2] https://gist.github.com/manuel-rubio/5302361 > > Regards. > Manuel Rubio. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From uvelner@REDACTED Wed Apr 3 16:16:03 2013 From: uvelner@REDACTED (Uri Velner) Date: Wed, 3 Apr 2013 14:16:03 +0000 Subject: [erlang-questions] Problem with Erlide Message-ID: <3D0492343D4AAE48879AA6F0CBBFC6624B3925CA@ISRTLVEXMX2.interwise.com> Hi, I'm new to Erlang developing ... I've tried the Erlide for eclipse. But, it seems not to work so well. I'm not sure the installation went well. During installation I got this warning: [cid:part1.08010701.05070400@REDACTED] Although, I can see the Erlide in the Eclipse Marketplace -> installed. When I'm trying to create new Erlang project in Eclipse I get the follows: [cid:part2.03080308.03030302@REDACTED] I've searched for this error. But, didn't get real solution for my problem. Can you explain me what went wrong ? Thanks in Advance. Uri -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 26699 bytes Desc: image001.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.png Type: image/png Size: 23878 bytes Desc: image002.png URL: From magnus@REDACTED Wed Apr 3 18:57:54 2013 From: magnus@REDACTED (Magnus Henoch) Date: Wed, 03 Apr 2013 17:57:54 +0100 Subject: [erlang-questions] erlang:now() vs os:timestamp() In-Reply-To: (Manuel A. Rubio's message of "Wed, 03 Apr 2013 18:32:38 +0200") References: Message-ID: "Manuel A. Rubio \"Bombadil\"" writes: > Hi, > > I saw a keynote[1] from Rick Reed (WhatsApp) and he said: "Prefer > os:timestamp to erlang:now"... I did a benchmark to test it[2], and > get the following results: > > os:timestamp = 1.44 1.38 1.38 > erlang:now = 1.01 1.00 1.00 > > I know that he says "prefer" and not "better"... but the benchmark > says that he should be say the opposite :-P > > What do you think? I added -mode(compile) to your script: https://gist.github.com/legoscia/5303050 and now I get completely different results: os:timestamp = 0.36 0.15 0.17 erlang:now = 1.00 1.00 1.00 Regards, Magnus From bombadil@REDACTED Wed Apr 3 19:05:18 2013 From: bombadil@REDACTED (Manuel A. Rubio "Bombadil") Date: Wed, 03 Apr 2013 19:05:18 +0200 Subject: [erlang-questions] erlang:now() vs os:timestamp() In-Reply-To: <20130403164038.GA83547@ferdair.local> References: <20130403164038.GA83547@ferdair.local> Message-ID: <8a21db05413780c34b669843daacfc6e@bosqueviejo.net> Hi Fred, yes, you're right. I did the test again[1] and get these results: os:timestamp = 9.00 9.19 9.42 erlang:now = 25.01 25.00 25.00 Having 1,000 processes doing 25,000 executions, looks like that os:timestamp is better for concurrency that erlang:now. Thanks for the point :-) [1] https://gist.github.com/manuel-rubio/5303075 Regards. Manuel Rubio. From bombadil@REDACTED Wed Apr 3 19:07:31 2013 From: bombadil@REDACTED (Manuel A. Rubio "Bombadil") Date: Wed, 03 Apr 2013 19:07:31 +0200 Subject: [erlang-questions] erlang:now() vs os:timestamp() In-Reply-To: References: Message-ID: good point! I don't know about -mode(compile), thanks Magnus. El 2013-04-03 18:57, Magnus Henoch escribi?: > "Manuel A. Rubio \"Bombadil\"" writes: > >> Hi, >> >> I saw a keynote[1] from Rick Reed (WhatsApp) and he said: "Prefer >> os:timestamp to erlang:now"... I did a benchmark to test it[2], and >> get the following results: >> >> os:timestamp = 1.44 1.38 1.38 >> erlang:now = 1.01 1.00 1.00 >> >> I know that he says "prefer" and not "better"... but the benchmark >> says that he should be say the opposite :-P >> >> What do you think? > > I added -mode(compile) to your script: > > https://gist.github.com/legoscia/5303050 > > and now I get completely different results: > > os:timestamp = 0.36 0.15 0.17 > erlang:now = 1.00 1.00 1.00 > > > Regards, > Magnus From magnus@REDACTED Wed Apr 3 19:18:18 2013 From: magnus@REDACTED (Magnus Henoch) Date: Wed, 03 Apr 2013 18:18:18 +0100 Subject: [erlang-questions] erlang:now() vs os:timestamp() In-Reply-To: (Magnus Henoch's message of "Wed, 03 Apr 2013 17:57:54 +0100") References: Message-ID: Magnus Henoch writes: > https://gist.github.com/legoscia/5303050 > > and now I get completely different results: > > os:timestamp = 0.36 0.15 0.17 > erlang:now = 1.00 1.00 1.00 The very even results for erlang:now made me suspicious. If you call now() one million times, then that will obviously force the monotonic clock to advance one million ticks, i.e. exactly one second. I replaced the now() call in epoch/0 with os:timestamp, and now the results are much more even, though slightly in favor of os:timestamp: os:timestamp = 0.33 0.15 0.17 erlang:now = 0.31 0.26 0.30 I guess that's the lesson to learn: if you want unique values, use now(), but if you want to know the actual time, use os:timestamp :) Regards, Magnus From garry@REDACTED Wed Apr 3 19:59:47 2013 From: garry@REDACTED (Garry Hodgson) Date: Wed, 03 Apr 2013 13:59:47 -0400 Subject: [erlang-questions] low level packet access from erlang Message-ID: <515C6E13.1050406@research.att.com> i am building an experimental firewall of sorts, and need to be able to access incoming packets directly, so i can muck around with low level src/dst/ports/etc information. it looks like the standard modules handle the low level things for me, such that i by the time i see an incoming message, the low level details are lost. how can i arrange access to the lower level information (ignoring performance issues for now)? i see like mentioned a way in an old (2001) thread: http://www.trapexit.org/forum/viewtopic.php?p=4258&sid=4469db61020efe9100e1e1c2504bfc8c but the link to bluetail where his code was doesn't exist anymore. i've read the ei/pcap approach presented here: http://blog.listincomprehension.com/2009/12/erlang-packet-sniffer-using-ei-and.html but i don't want to just sniff packets, but intercept them. i'd appreciate any insights into how to tackle this. -- Garry Hodgson AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." From nem@REDACTED Wed Apr 3 20:06:14 2013 From: nem@REDACTED (Geoff Cant) Date: Wed, 3 Apr 2013 11:06:14 -0700 Subject: [erlang-questions] low level packet access from erlang In-Reply-To: <515C6E13.1050406@research.att.com> References: <515C6E13.1050406@research.att.com> Message-ID: Hi there - I'm slowly building a TAP interface based library for networking at https://github.com/archaelus/enet -- some of the code might save you some time. You can send and receive packets with it, but it's missing a lot of mechanisms and infrastructure for making that easy. I've used the packet parsing code a bunch for pcap file analysis, and the generation code only a little to try and build an IP stack. The docs are non-existent but I'm more than happy to answer questions about the code. -Geoff On 2013-04-03, at 10:59 , Garry Hodgson wrote: > i am building an experimental firewall of sorts, > and need to be able to access incoming packets > directly, so i can muck around with low level > src/dst/ports/etc information. it looks like the standard > modules handle the low level things for me, such that > i by the time i see an incoming message, the low level > details are lost. > > how can i arrange access to the lower level information > (ignoring performance issues for now)? > > i see like mentioned a way in an old (2001) thread: > http://www.trapexit.org/forum/viewtopic.php?p=4258&sid=4469db61020efe9100e1e1c2504bfc8c > but the link to bluetail where his code was doesn't exist anymore. > > i've read the ei/pcap approach presented here: > http://blog.listincomprehension.com/2009/12/erlang-packet-sniffer-using-ei-and.html > > but i don't want to just sniff packets, but intercept them. > > i'd appreciate any insights into how to tackle this. > > -- > Garry Hodgson > AT&T Chief Security Office (CSO) From mattevans123@REDACTED Wed Apr 3 20:08:39 2013 From: mattevans123@REDACTED (Matthew Evans) Date: Wed, 3 Apr 2013 14:08:39 -0400 Subject: [erlang-questions] low level packet access from erlang In-Reply-To: References: <515C6E13.1050406@research.att.com>, Message-ID: You could also look at procket: http://blog.listincomprehension.com/ https://github.com/msantos/procket > From: nem@REDACTED > Date: Wed, 3 Apr 2013 11:06:14 -0700 > To: garry@REDACTED > CC: erlang-questions@REDACTED > Subject: Re: [erlang-questions] low level packet access from erlang > > Hi there - I'm slowly building a TAP interface based library for networking at https://github.com/archaelus/enet -- some of the code might save you some time. You can send and receive packets with it, but it's missing a lot of mechanisms and infrastructure for making that easy. > > I've used the packet parsing code a bunch for pcap file analysis, and the generation code only a little to try and build an IP stack. > > The docs are non-existent but I'm more than happy to answer questions about the code. > > -Geoff > > On 2013-04-03, at 10:59 , Garry Hodgson wrote: > > > i am building an experimental firewall of sorts, > > and need to be able to access incoming packets > > directly, so i can muck around with low level > > src/dst/ports/etc information. it looks like the standard > > modules handle the low level things for me, such that > > i by the time i see an incoming message, the low level > > details are lost. > > > > how can i arrange access to the lower level information > > (ignoring performance issues for now)? > > > > i see like mentioned a way in an old (2001) thread: > > http://www.trapexit.org/forum/viewtopic.php?p=4258&sid=4469db61020efe9100e1e1c2504bfc8c > > but the link to bluetail where his code was doesn't exist anymore. > > > > i've read the ei/pcap approach presented here: > > http://blog.listincomprehension.com/2009/12/erlang-packet-sniffer-using-ei-and.html > > > > but i don't want to just sniff packets, but intercept them. > > > > i'd appreciate any insights into how to tackle this. > > > > -- > > Garry Hodgson > > AT&T Chief Security Office (CSO) > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremy@REDACTED Wed Apr 3 20:20:16 2013 From: jeremy@REDACTED (Jeremy Ong) Date: Wed, 3 Apr 2013 11:20:16 -0700 Subject: [erlang-questions] Compiling R16B on Mac OS X Mountain Lion In-Reply-To: <515C2A8E.7010507@ubiquity.it> References: <513DE820.1060900@ubiquity.it> <51405EC3.10502@erlang.org> <51418FD6.1070709@ubiquity.it> <5141A5DB.7070307@ubiquity.it> <51473633.4070507@erlang.org> <515C2A8E.7010507@ubiquity.it> Message-ID: +1 for ditching macports. On Wed, Apr 3, 2013 at 6:11 AM, Carlo Bertoldi wrote: > On 18/03/13 16:43, Patrik Nyblom wrote: > >> Hi! >> >> > Are you using macports gcc or apples llvm-wrapper in this case? If >> macports, did you install the +universal variant? >> >> gcc45 @4.5.4_6+universal > > Just for the records, I've finally managed to find the problem. > During the various try the memsup was compiled for a 32bit architecture. > > I've ran make clean many times, but that didn't deleted the file memsup.o. > I discovered this since I removed the whole source folder and started from > scratch. Eventually, the compilation went fine. > > > Cheers, > Carlo > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthias@REDACTED Wed Apr 3 20:56:02 2013 From: matthias@REDACTED (Matthias Lang) Date: Wed, 3 Apr 2013 20:56:02 +0200 Subject: [erlang-questions] low level packet access from erlang In-Reply-To: References: <515C6E13.1050406@research.att.com> Message-ID: <20130403185602.GA25259@corelatus.se> I have a copy of Luke's TUN/TAP code. I successfully used it many years ago for some experiments, IIRC to study what TCP did on a lossy network. Worked fine for me. Luke's code is a thin layer over TUN/TAP, it's just one C file and one .erl file. It lets you do two things: receive all packets on an ethernet interface and send packets on an ethernet interface. It comes with an example to get you started. After a _quick_ look at Geoff's code, I think it's _functionally_ a superset of Luke's. Geoff's "enet_eth_iface.erl" does pretty much the same thing as Luke's "tuntap.erl". The rest of Geoff's code seems to be more of the IP stack to let you do more complicated decoding, e.g. TCP. Here's Luke's code, it'll be there until next time I clean up: http://corelatus.se/~matthias/luke_tuntap.tgz Matt -------------------- On Wednesday, April 03, Matthew Evans wrote: > You could also look at procket: > http://blog.listincomprehension.com/ > https://github.com/msantos/procket > > > From: nem@REDACTED > > Date: Wed, 3 Apr 2013 11:06:14 -0700 > > To: garry@REDACTED > > CC: erlang-questions@REDACTED > > Subject: Re: [erlang-questions] low level packet access from erlang > > > > Hi there - I'm slowly building a TAP interface based library for networking at https://github.com/archaelus/enet -- some of the code might save you some time. You can send and receive packets with it, but it's missing a lot of mechanisms and infrastructure for making that easy. > > > > I've used the packet parsing code a bunch for pcap file analysis, and the generation code only a little to try and build an IP stack. > > > > The docs are non-existent but I'm more than happy to answer questions about the code. > > > > -Geoff > > > > On 2013-04-03, at 10:59 , Garry Hodgson wrote: > > > > > i am building an experimental firewall of sorts, > > > and need to be able to access incoming packets > > > directly, so i can muck around with low level > > > src/dst/ports/etc information. it looks like the standard > > > modules handle the low level things for me, such that > > > i by the time i see an incoming message, the low level > > > details are lost. > > > > > > how can i arrange access to the lower level information > > > (ignoring performance issues for now)? > > > > > > i see like mentioned a way in an old (2001) thread: > > > http://www.trapexit.org/forum/viewtopic.php?p=4258&sid=4469db61020efe9100e1e1c2504bfc8c > > > but the link to bluetail where his code was doesn't exist anymore. > > > > > > i've read the ei/pcap approach presented here: > > > http://blog.listincomprehension.com/2009/12/erlang-packet-sniffer-using-ei-and.html > > > > > > but i don't want to just sniff packets, but intercept them. > > > > > > i'd appreciate any insights into how to tackle this. > > > > > > -- > > > Garry Hodgson > > > AT&T Chief Security Office (CSO) > > > > > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From nem@REDACTED Wed Apr 3 21:31:19 2013 From: nem@REDACTED (Geoff Cant) Date: Wed, 3 Apr 2013 12:31:19 -0700 Subject: [erlang-questions] low level packet access from erlang In-Reply-To: <20130403185602.GA25259@corelatus.se> References: <515C6E13.1050406@research.att.com> <20130403185602.GA25259@corelatus.se> Message-ID: It's a bit of a rewrite I think - you get a port program enet_tap that opens the tap device and then does {packet, 2} to/from erlang for forwarding the frames. It uses libevent, but I haven't worked out how fast/slow it goes. It works on OSX and linux. enet_tap is the erlang module responsible for opening the port and decoding the 'running' vs {'frame', Data} messages. (The 'running' message lets you know when you can ifconfig the device). enet_eth_iface is my current crazy attempt at representing an ethernet interface in erlang with pubsub for subscribing to ethernet frame types. You don't need to use it to send/receive frames. It's overly complicated right now imo. -G On 2013-04-03, at 11:56 , Matthias Lang wrote: > > I have a copy of Luke's TUN/TAP code. I successfully used it many > years ago for some experiments, IIRC to study what TCP did on a lossy > network. Worked fine for me. > > Luke's code is a thin layer over TUN/TAP, it's just one C file and one > .erl file. It lets you do two things: receive all packets on an > ethernet interface and send packets on an ethernet interface. It comes > with an example to get you started. > > After a _quick_ look at Geoff's code, I think it's _functionally_ a > superset of Luke's. Geoff's "enet_eth_iface.erl" does pretty much the > same thing as Luke's "tuntap.erl". The rest of Geoff's code seems to > be more of the IP stack to let you do more complicated decoding, e.g. TCP. > > Here's Luke's code, it'll be there until next time I clean up: > > http://corelatus.se/~matthias/luke_tuntap.tgz > > Matt From garry@REDACTED Wed Apr 3 21:48:27 2013 From: garry@REDACTED (Garry Hodgson) Date: Wed, 03 Apr 2013 15:48:27 -0400 Subject: [erlang-questions] low level packet access from erlang In-Reply-To: References: <515C6E13.1050406@research.att.com> <20130403185602.GA25259@corelatus.se> Message-ID: <515C878B.5050209@research.att.com> thanks to all for your suggestions. i was hoping the answer would be "just run inet:get_all_the_info_garry_needs( Sock )" oh well, looks like i have some studying to do. On 04/03/2013 03:31 PM, Geoff Cant wrote: > It's a bit of a rewrite I think - you get a port program enet_tap that opens the tap device and then does {packet, 2} to/from erlang for forwarding the frames. It uses libevent, but I haven't worked out how fast/slow it goes. It works on OSX and linux. > > enet_tap is the erlang module responsible for opening the port and decoding the 'running' vs {'frame', Data} messages. (The 'running' message lets you know when you can ifconfig the device). > > enet_eth_iface is my current crazy attempt at representing an ethernet interface in erlang with pubsub for subscribing to ethernet frame types. You don't need to use it to send/receive frames. It's overly complicated right now imo. > > -G > > On 2013-04-03, at 11:56 , Matthias Lang wrote: > >> I have a copy of Luke's TUN/TAP code. I successfully used it many >> years ago for some experiments, IIRC to study what TCP did on a lossy >> network. Worked fine for me. >> >> Luke's code is a thin layer over TUN/TAP, it's just one C file and one >> .erl file. It lets you do two things: receive all packets on an >> ethernet interface and send packets on an ethernet interface. It comes >> with an example to get you started. >> >> After a _quick_ look at Geoff's code, I think it's _functionally_ a >> superset of Luke's. Geoff's "enet_eth_iface.erl" does pretty much the >> same thing as Luke's "tuntap.erl". The rest of Geoff's code seems to >> be more of the IP stack to let you do more complicated decoding, e.g. TCP. >> >> Here's Luke's code, it'll be there until next time I clean up: >> >> http://corelatus.se/~matthias/luke_tuntap.tgz >> >> Matt > -- Garry Hodgson AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." From jbothma@REDACTED Wed Apr 3 22:04:12 2013 From: jbothma@REDACTED (JD Bothma) Date: Wed, 3 Apr 2013 22:04:12 +0200 Subject: [erlang-questions] low level packet access from erlang In-Reply-To: <515C6E13.1050406@research.att.com> References: <515C6E13.1050406@research.att.com> Message-ID: It might be worth looking into how https://www.erlang-solutions.com/products/openflow does it. IIRC it works at the ethernet level so there's lots of info available. On 3 April 2013 19:59, Garry Hodgson wrote: > i am building an experimental firewall of sorts, > and need to be able to access incoming packets > directly, so i can muck around with low level > src/dst/ports/etc information. it looks like the standard > modules handle the low level things for me, such that > i by the time i see an incoming message, the low level > details are lost. > > how can i arrange access to the lower level information > (ignoring performance issues for now)? > > i see like mentioned a way in an old (2001) thread: > http://www.trapexit.org/forum/**viewtopic.php?p=4258&sid=** > 4469db61020efe9100e1e1c2504bfc**8c > but the link to bluetail where his code was doesn't exist anymore. > > i've read the ei/pcap approach presented here: > http://blog.**listincomprehension.com/2009/**12/erlang-packet-sniffer-** > using-ei-and.html > > but i don't want to just sniff packets, but intercept them. > > i'd appreciate any insights into how to tackle this. > > -- > Garry Hodgson > AT&T Chief Security Office (CSO) > > "This e-mail and any files transmitted with it are AT&T property, are > confidential, and are intended solely for the use of the individual or > entity to whom this e-mail is addressed. If you are not one of the named > recipient(s) or otherwise have reason to believe that you have received > this message in error, please notify the sender and delete this message > immediately from your computer. Any other use, retention, dissemination, > forwarding, printing, or copying of this e-mail is strictly prohibited." > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.santos@REDACTED Wed Apr 3 23:24:07 2013 From: michael.santos@REDACTED (Michael Santos) Date: Wed, 3 Apr 2013 17:24:07 -0400 Subject: [erlang-questions] low level packet access from erlang In-Reply-To: <515C6E13.1050406@research.att.com> References: <515C6E13.1050406@research.att.com> Message-ID: <20130403212407.GA25006@ioctl> On Wed, Apr 03, 2013 at 01:59:47PM -0400, Garry Hodgson wrote: > i am building an experimental firewall of sorts, > and need to be able to access incoming packets > directly, so i can muck around with low level > src/dst/ports/etc information. it looks like the standard > modules handle the low level things for me, such that > i by the time i see an incoming message, the low level > details are lost. > > how can i arrange access to the lower level information > (ignoring performance issues for now)? > > i see like mentioned a way in an old (2001) thread: > http://www.trapexit.org/forum/viewtopic.php?p=4258&sid=4469db61020efe9100e1e1c2504bfc8c > but the link to bluetail where his code was doesn't exist anymore. > > i've read the ei/pcap approach presented here: > http://blog.listincomprehension.com/2009/12/erlang-packet-sniffer-using-ei-and.html > > but i don't want to just sniff packets, but intercept them. > > i'd appreciate any insights into how to tackle this. It really depends on what level and which platforms you want to work on. There are BSD raw sockets, the Linux PF_PACKET interface, BPF for BSD and LSF for Linux, divert sockets, tun devices, tap devices ... procket can handle all of those but assuming you just want to manipulate the IP headers and don't want to worry about the ethernet framing, maybe using a tun device would be the easiest way. I wrote an Erlang tun/tap interface on top of procket: https://github.com/msantos/tunctl I've tested the code on Linux, Mac OS X and FreeBSD. Supporting other BSDs shouldn't be a problem. Adding Windows support is on my TODO list. Once you've created an interface and set up the routing table, you can simply read/write frames to the device. Here is an example of using a tap device to create the most insecure VPN ever over Erlang distribution: https://github.com/msantos/tunctl/blob/master/examples/vpwn.erl Another example using a tun device: https://github.com/msantos/sut sut sets up an RFC 4213 IPv6 over IPv4 tunnel. There is an example of creating a basic stateless firewall here: https://github.com/msantos/sut/blob/master/examples/basic_firewall.erl From mediocregopher@REDACTED Wed Apr 3 23:38:05 2013 From: mediocregopher@REDACTED (Brian Picciano) Date: Wed, 3 Apr 2013 17:38:05 -0400 Subject: [erlang-questions] Problem sending responses back from cnode Message-ID: I have the following code acting as a c node for an erlang instance: https://gist.github.com/mediocregopher/5305479 (a lot of it is copied straight from the tutorial). Previously I had the c node connecting to the erlang instance and everything worked fine, however I changed the code to listen for a connection from an erlang node and work from there to make deployment easier. Unfortunately now whenever I try to send back a response to the erlang node using erl_send nothing is sent back to erlang, instead it looks like the message to be sent is output to the terminal. For example, if I do: {any, 'cnode@REDACTED'} ! {get_all_keys}. in the erlang node with the c node running, the c node should send back the atom get_all_keys. However nothing gets sent back, and the following is output to the console where the c node is running: :phadgdnode1@REDACTED'd get_all_keys Does anyone know why this would be happening? -------------- next part -------------- An HTML attachment was scrubbed... URL: From brisa.jimenez.f@REDACTED Thu Apr 4 02:58:43 2013 From: brisa.jimenez.f@REDACTED (=?ISO-8859-1?Q?Brisa_Jim=E9nez?=) Date: Wed, 3 Apr 2013 18:58:43 -0600 Subject: [erlang-questions] simulate a transaction Message-ID: I want to simulate a transaction. I want to know when a riak operation happens. I know that riak isn't a relational database, but it is important for me in particular cases to undo operations when something goes wrong. I'm reading about how to use dbg library to trace functions, I hope that it will be useful to use inside a module. If any have suggestions about I want to do, I will be very grateful. Regards. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brisa.jimenez.f@REDACTED Thu Apr 4 03:00:37 2013 From: brisa.jimenez.f@REDACTED (=?ISO-8859-1?Q?Brisa_Jim=E9nez?=) Date: Wed, 3 Apr 2013 19:00:37 -0600 Subject: [erlang-questions] dbg Message-ID: Match specification is for evaluate function params Am I right? How can I get the result of a function for doing something later? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Thu Apr 4 07:11:16 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 4 Apr 2013 07:11:16 +0200 Subject: [erlang-questions] dbg In-Reply-To: References: Message-ID: <1365052276.4777.2.camel@sekic1152.rnd.ki.sw.ericsson.se> Greetings, This 3:rd argument to dbg:tpl/3 (instead of []) will display the return value: dbg:fun2ms( fun(_)-> return_trace() end ). bengt On Wed, 2013-04-03 at 19:00 -0600, Brisa Jim?nez wrote: > Match specification is for evaluate function params Am I right? > How can I get the result of a function for doing something later? > Thanks. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mangalaman93@REDACTED Thu Apr 4 04:49:29 2013 From: mangalaman93@REDACTED (aman mangal) Date: Thu, 4 Apr 2013 08:19:29 +0530 Subject: [erlang-questions] Mnesia: Whether it is good for my application! help? In-Reply-To: References: Message-ID: Hi, Thank you so much for your response! On one side I tend to use mnesia because it is free and can be used on distributive nodes, on the other hand I am not very much aware of the shortcoming of mnesia. So there is always a question in front of me whether to what extend can mneisa handle things easily. I tried reading some article on internet like learnyousomeerlang.com, looked at erlang documentation but couldn't understand the limitations of mnesia. It will be very nice if you can give me some pointers where I can understand the internals of mnesia properly and use it accordingly. About my back up node, the process group has some specific name or id. I'll prefix the table names with the unique id. Each process group serves a specific group of people(say). I am using back up node for failover case so that I don't lose data. It is important not to lose data. Are you saying that saving it on the disk on the same node is enough? No need to store on the other node? disk failure is rare and in that case I've data in RAM(mnesia). The capacity of servers like RAM, disk space can be assumed as much as we want. Aman Mangal 3rd year Undergraduate Department of Computer Science & Engineering IIT Bombay www.cse.iitb.ac.in/~amanmangal On Tue, Apr 2, 2013 at 9:23 PM, Magnus Henoch wrote: > aman mangal writes: > > > Hi everyone, > > > > This is my first post on this forum! I am new to erlang, just started > using > > it a month ago or so. Since I have come across erlang, I have become > such a > > big fan of it that I have started working on a product in erlang for a > > company. > > > > Coming to the point, here is my question. I want to make > > a distributive application. There are multiple processes running on a > node. > > The processes are divided in logical groups. Each group serves a specific > > purpose. Each group has some data(few tables) shared among the processes > > inside the group but not outside the group. Now my question is that > whether > > to use mnesia or not to store the data? If I don't use mnesia, I'll store > > the data as state in a gen server processes running in the group. In that > > case I don't know how will I will keep back up of the data for the case > > when any node goes down. > > It sounds like Mnesia would be a good fit. You would get persistence > for free by creating your tables with disc_copies. The only potential > issue I see is table names: each Mnesia table must be named by an atom. > Are your process groups created dynamically? If so, you'd need to come > up with a scheme to get the correct table names from within the > processes that need to access the tables. You'll also need to use the > record_name option when creating the tables, as by default Mnesia > requires you to use records with the same name as the table. > > > Also my database can grow as large as thousand tables each having hudred > > entries on a node. Is mnesia a good idea to use for such a large (I'm not > > sure whether it is large :P ) database? Is there any free alternative to > > mnesia? Please note that there is no inter node communication in order to > > get data except with the back up node! > > Assuming each entry takes no more than a kilobyte, you're looking at > roughly 100 megabytes of data in total. When using the disc_copies > table storage type, Mnesia will store the entire database in memory as > well as on disk, but I presume that you have enough RAM that 100 > megabytes isn't a big problem. > > Could you elaborate on your backup node setup? If you want your data to > be distributed to protect against disk failure on the main node, then > Mnesia distribution is probably the way to go, but depending on your > requirements you might be able to get away with a simpler solution. > > Hope this helps, > Magnus > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hirotnkg@REDACTED Thu Apr 4 09:48:24 2013 From: hirotnkg@REDACTED (Yoshihiro Tanaka) Date: Thu, 4 Apr 2013 00:48:24 -0700 Subject: [erlang-questions] dbg In-Reply-To: <1365052276.4777.2.camel@sekic1152.rnd.ki.sw.ericsson.se> References: <1365052276.4777.2.camel@sekic1152.rnd.ki.sw.ericsson.se> Message-ID: Hi, You can also use dbg as following: dbg:tracer(). dbg:p(all, [call, timestamp]). dbg:tpl(module_name,func_name,[]). dbg:tpl(module_name,func_name,x). This will show the parameters to module_name:func_name when it gets called, and return value of it with timestamp. The meaning of the built-in alias is written here: http://www.erlang.org/doc/man/dbg.html Yoshihiro On Wed, Apr 3, 2013 at 10:11 PM, Bengt Kleberg wrote: > Greetings, > > This 3:rd argument to dbg:tpl/3 (instead of []) will display the return > value: > dbg:fun2ms( fun(_)-> return_trace() end ). > > > bengt > > On Wed, 2013-04-03 at 19:00 -0600, Brisa Jim?nez wrote: > > Match specification is for evaluate function params Am I right? > > How can I get the result of a function for doing something later? > > Thanks. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Apr 4 10:23:06 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 4 Apr 2013 10:23:06 +0200 Subject: [erlang-questions] Sending a large Erlang content to a set of remote nodes In-Reply-To: References: Message-ID: <545FA80A-DD08-4B89-A3AF-53616D6335DD@erlang-solutions.com> On Mar 28, 2013, at 6:58 PM, Bob Ippolito wrote: > > You may even be able to repurpose etorrent for this kind of purpose, I'm sure it'd make it easier to put together a great prototype if nothing else. > Yes, that is a possibility, although it may be a bit overkill as a system. I think it is possible to find a simpler protocol when you have a less dynamic environment and you can trust the other clients. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen From tereskin@REDACTED Thu Apr 4 12:24:19 2013 From: tereskin@REDACTED (Andrew Tereskin) Date: Thu, 4 Apr 2013 14:24:19 +0400 Subject: [erlang-questions] Socket acceptor strategy: pooling or async Message-ID: Hi everyone, I'm trying to what is the best strategy for tcp socket acceptors. Just after brief research I found 2 approaches: 1. Pool of acceptors that can work in parallel. According to documentation ListenSocket can be used not only in the process where this socket was created. Example code (from http://learnyousomeerlang.com/buckets-of-sockets): %%% The supervisor in charge of all the socket acceptors. -module(sockserv_sup). -behaviour(supervisor). init([]) -> ..... {ok, ListenSocket} = gen_tcp:listen(Port, [{active,once}, {packet,line}]), spawn_link(fun empty_listeners/0), {ok, {{simple_one_for_one, 60, 3600}, [{socket, {sockserv_serv, start_link, [ListenSocket]}, % pass the socket! temporary, 1000, worker, [sockserv_serv]} ]}}. start_socket() -> supervisor:start_child(?MODULE, []). empty_listeners() -> [start_socket() || _ <- lists:seq(1,20)], ok. %%%%%%%%%%%%%%%%%% %% socket acceptor -module(sockserv_serv). -behaviour(gen_server). init(Socket) -> .... gen_server:cast(self(), accept), {ok, #state{socket=Socket}}. handle_call(_E, _From, State) -> {noreply, State}. handle_cast(accept, S = #state{socket=ListenSocket}) -> {ok, AcceptSocket} = gen_tcp:accept(ListenSocket), sockserv_sup:start_socket(), % a new acceptor is born, praise the lord {noreply, S#state{socket=AcceptSocket, next=name}}; --------------------------------- As I examined code of mochiweb ( https://github.com/mochi/mochiweb/blob/master/src/mochiweb_socket_server.erl, https://github.com/mochi/mochiweb/blob/master/src/mochiweb_acceptor.erl) and ranch (https://github.com/extend/ranch) they use the same approach. 2. Using undocumented function prim_inet:async_accept Example code (from http://www.trapexit.org/Building_a_Non-blocking_TCP_server_using_OTP_principles ) init([Port, Module]) -> process_flag(trap_exit, true), Opts = [binary, {packet, 2}, {reuseaddr, true}, {keepalive, true}, {backlog, 30}, {active, false}], case gen_tcp:listen(Port, Opts) of {ok, Listen_socket} -> %%Create first accepting process {ok, Ref} = prim_inet:async_accept(Listen_socket, -1), {ok, #state{listener = Listen_socket, acceptor = Ref, module = Module}}; {error, Reason} -> {stop, Reason} end. handle_info({inet_async, ListSock, Ref, {ok, CliSocket}}, #state{listener=ListSock, acceptor=Ref, module=Module} = State) -> try case set_sockopt(ListSock, CliSocket) of ok -> ok; {error, Reason} -> exit({set_sockopt, Reason}) end, %% New client connected - spawn a new process using the simple_one_for_one %% supervisor. {ok, Pid} = tcp_server_app:start_client(), gen_tcp:controlling_process(CliSocket, Pid), %% Instruct the new FSM that it owns the socket. Module:set_socket(Pid, CliSocket), %% Signal the network driver that we are ready to accept another connection case prim_inet:async_accept(ListSock, -1) of {ok, NewRef} -> ok; {error, NewRef} -> exit({async_accept, inet:format_error(NewRef)}) end, {noreply, State#state{acceptor=NewRef}} catch exit:Why -> error_logger:error_msg("Error in async accept: ~p.\n", [Why]), {stop, Why, State} end; It is used by erlyvideo ( https://github.com/erlyvideo/erlyvideo/blob/master/apps/erlyvideo/src/core/gen_listener.erl ). Can you give some pros and cons of both approaches? With best wishes, Andrey Tereskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu Apr 4 12:30:05 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 4 Apr 2013 14:30:05 +0400 Subject: [erlang-questions] Socket acceptor strategy: pooling or async In-Reply-To: References: Message-ID: This old approach is not used by erlyvideo, there is ranch now with pool of acceptors. Pool allows you to accept connections on all cores. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Thu Apr 4 12:46:31 2013 From: bourinov@REDACTED (Max Bourinov) Date: Thu, 4 Apr 2013 12:46:31 +0200 Subject: [erlang-questions] Socket acceptor strategy: pooling or async In-Reply-To: References: Message-ID: +1 for ranch Best regards, Max On Thu, Apr 4, 2013 at 12:30 PM, Max Lapshin wrote: > This old approach is not used by erlyvideo, there is ranch now with pool > of acceptors. Pool allows you to accept connections on all cores. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From seglingskungen@REDACTED Thu Apr 4 16:23:06 2013 From: seglingskungen@REDACTED (Seglings Kungen) Date: Thu, 4 Apr 2013 16:23:06 +0200 Subject: [erlang-questions] Web UI testing? Message-ID: Hi Are there any test tools for that automatically can test Web UIs that is written in erlang ie some thing like selenium? erl_selenuim does not seam to have been updated on quite awhile...... Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: From garry@REDACTED Thu Apr 4 19:58:59 2013 From: garry@REDACTED (Garry Hodgson) Date: Thu, 04 Apr 2013 13:58:59 -0400 Subject: [erlang-questions] low level packet access from erlang In-Reply-To: <20130403212407.GA25006@ioctl> References: <515C6E13.1050406@research.att.com> <20130403212407.GA25006@ioctl> Message-ID: <515DBF63.8090504@research.att.com> wow. this looks great. you've built up a nice set of layers here. i was just tinkering with procket when i saw this message. guess i need to keep on tinkering. thanks On 04/03/2013 05:24 PM, Michael Santos wrote: > On Wed, Apr 03, 2013 at 01:59:47PM -0400, Garry Hodgson wrote: >> i am building an experimental firewall of sorts, >> and need to be able to access incoming packets >> directly, so i can muck around with low level >> src/dst/ports/etc information. it looks like the standard >> modules handle the low level things for me, such that >> i by the time i see an incoming message, the low level >> details are lost. >> >> how can i arrange access to the lower level information >> (ignoring performance issues for now)? >> >> i see like mentioned a way in an old (2001) thread: >> http://www.trapexit.org/forum/viewtopic.php?p=4258&sid=4469db61020efe9100e1e1c2504bfc8c >> but the link to bluetail where his code was doesn't exist anymore. >> >> i've read the ei/pcap approach presented here: >> http://blog.listincomprehension.com/2009/12/erlang-packet-sniffer-using-ei-and.html >> >> but i don't want to just sniff packets, but intercept them. >> >> i'd appreciate any insights into how to tackle this. > It really depends on what level and which platforms you want to work > on. There are BSD raw sockets, the Linux PF_PACKET interface, BPF for > BSD and LSF for Linux, divert sockets, tun devices, tap devices ... > > procket can handle all of those but assuming you just want to manipulate > the IP headers and don't want to worry about the ethernet framing, > maybe using a tun device would be the easiest way. > > I wrote an Erlang tun/tap interface on top of procket: > > https://github.com/msantos/tunctl > > I've tested the code on Linux, Mac OS X and FreeBSD. Supporting other > BSDs shouldn't be a problem. Adding Windows support is on my TODO list. > > Once you've created an interface and set up the routing table, you can > simply read/write frames to the device. Here is an example of using a tap > device to create the most insecure VPN ever over Erlang distribution: > > https://github.com/msantos/tunctl/blob/master/examples/vpwn.erl > > Another example using a tun device: > > https://github.com/msantos/sut > > sut sets up an RFC 4213 IPv6 over IPv4 tunnel. There is an example of > creating a basic stateless firewall here: > > https://github.com/msantos/sut/blob/master/examples/basic_firewall.erl > -- Garry Hodgson AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." From ok@REDACTED Fri Apr 5 05:10:08 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 5 Apr 2013 16:10:08 +1300 Subject: [erlang-questions] Parallelism lesson for an idiot Message-ID: The idiot in question was me. A colleague handed me a book about parallel programming, and I thought, well, maybe in the concurrent programming paper I should show the students a parallel sorting algorithm using POSIX threads. So I wrote one. In C. I was ever so pleased. It worked first time. The problem is that it's 1.06x to 2.61x _slower_ than the sequential sorting algorithm it uses as a base, for _any_ number of threads I tried, on a 4-core machine. A few minutes' thought told me why. In order to get a time that was worth speeding up, I gave it a lot of data to sort. And that meant that the memory bus was being saturated. And since there's only one of _that_, there was no way that having multiple cores was going to make anything faster. The thing is that I've known for a couple of decades that managing *memory* is the key to good performance. I think I forgot to think about _what_ I was parallelising and whether that was _everything_ that needed to be parallelised. This will be shown to the concurrent programming students after all, but as a dreadful warning. One of the things I like about Erlang is that it gently leads me away from the illusion that shared memory is cheap. From jon@REDACTED Fri Apr 5 08:03:00 2013 From: jon@REDACTED (Jon Schneider) Date: Fri, 05 Apr 2013 07:03:00 +0100 Subject: [erlang-questions] Parallelism lesson for an idiot In-Reply-To: References: Message-ID: <99eab5fc-95c7-4215-9c65-a5e7e64b1369@email.android.com> There could be another reason people often miss. Manipulating data within a cache line concurrently from more than one core absolutely destroys caching. I would imagine this will get many sorting algorithms. Jon -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Fri Apr 5 09:37:23 2013 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 5 Apr 2013 09:37:23 +0200 Subject: [erlang-questions] Sending a large Erlang content to a set of remote nodes In-Reply-To: <545FA80A-DD08-4B89-A3AF-53616D6335DD@erlang-solutions.com> References: <545FA80A-DD08-4B89-A3AF-53616D6335DD@erlang-solutions.com> Message-ID: I agree - there should be a bit-torrent like thing that does not do the "only give you data if you give me data" thing (If you see what I mean) an "altruistic" bit torrent that gives data to it's peers even if they give nothing in return. A flood-fill algorithm /Joe On Thu, Apr 4, 2013 at 10:23 AM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Mar 28, 2013, at 6:58 PM, Bob Ippolito wrote: > > > > > You may even be able to repurpose etorrent for this kind of purpose, I'm > sure it'd make it easier to put together a great prototype if nothing else. > > > > Yes, that is a possibility, although it may be a bit overkill as a system. > I think it is possible to find a simpler protocol when you have a less > dynamic environment and you can trust the other clients. > > > Jesper Louis Andersen > Erlang Solutions Ltd., Copenhagen > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dch@REDACTED Fri Apr 5 09:40:20 2013 From: dch@REDACTED (Dave Cottlehuber) Date: Fri, 5 Apr 2013 09:40:20 +0200 Subject: [erlang-questions] Sending a large Erlang content to a set of remote nodes In-Reply-To: References: <545FA80A-DD08-4B89-A3AF-53616D6335DD@erlang-solutions.com> Message-ID: This is interesting http://libswift.org/ but I didn't start on writing it yet. On 5 April 2013 09:37, Joe Armstrong wrote: > I agree - there should be a bit-torrent like thing that does not do the > "only give you data if you give me data" thing (If you see what I mean) an > "altruistic" bit torrent that gives data to it's peers even if they give > nothing in return. A flood-fill algorithm > > /Joe > > > On Thu, Apr 4, 2013 at 10:23 AM, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > >> >> On Mar 28, 2013, at 6:58 PM, Bob Ippolito wrote: >> >> > >> > You may even be able to repurpose etorrent for this kind of purpose, >> I'm sure it'd make it easier to put together a great prototype if nothing >> else. >> > >> >> Yes, that is a possibility, although it may be a bit overkill as a >> system. I think it is possible to find a simpler protocol when you have a >> less dynamic environment and you can trust the other clients. >> >> >> Jesper Louis Andersen >> Erlang Solutions Ltd., Copenhagen >> >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Apr 5 10:38:35 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 5 Apr 2013 12:38:35 +0400 Subject: [erlang-questions] high_watermark and high_msgq_watermark: how to use them? Message-ID: Hi. There are two new options: high_watermark and high_msgq_watermark I want client to block at 4 Mb of unfetched data and unblock when his queue is empty. Is it enough just to use high_watermark and low_watermark? Is it possible that I set 4 Mb of high_watermark, but sender is blocked due to msgq overfill? -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Sat Apr 6 19:23:25 2013 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Sat, 6 Apr 2013 19:23:25 +0200 Subject: [erlang-questions] reltool: sys-level {mod_cond, derived} Message-ID: As discussed previously[1] it was decided to make the default rebar reltool.config template safer to use by removing[2] the non-default sys-level mod_cond. While this doesn't produce releases with potentially missing modules, that one-line patch also pulls in many seemingly unused applications resulting in a release that's at least twice as big: With sys-level {mod_cond, derived} (aka non-default settings): $ ls -1 rel/exemplar/lib compiler-4.9.1 crypto-2.3 erts-5.10.1 exemplar hipe-3.10.1 kernel-2.16.1 sasl-2.3.1 stdlib-1.19.1 tools-2.6.10 Without sys-level {mod_cond, derived} (aka default settings): $ g diff diff --git a/rel/reltool.config b/rel/reltool.config index 2dd47be..e65c53c 100644 --- a/rel/reltool.config +++ b/rel/reltool.config @@ -19,7 +19,6 @@ {boot_rel, "exemplar"}, {profile, embedded}, {incl_cond, derived}, - {mod_cond, derived}, {excl_archive_filters, [".*"]}, %% Do not archive built libs {excl_sys_filters, ["^bin/.*", "^erts.*/bin/(dialyzer|typer)", "^erts.*/(doc|info|include|lib|man|src)"]}, $ ls -1 rel/exemplar/lib asn1-2.0.1 compiler-4.9.1 crypto-2.3 edoc-0.7.12 erts-5.10.1 et-1.4.4.3 exemplar gs-1.5.15.2 hipe-3.10.1 inets-5.9.4 kernel-2.16.1 mnesia-4.8 observer-1.3 public_key-0.18 runtime_tools-1.8.10 sasl-2.3.1 ssl-5.2.1 stdlib-1.19.1 syntax_tools-1.6.11 tools-2.6.10 webtool-0.8.9.2 wx-1.0 xmerl-1.3.3 To reproduce, follow the steps found in https://github.com/rebar/rebar/wiki/Release-handling and make sure to delete the sys-level mod_cond. If you want to keep the sys-level {mod_cond, derived} and use the nodetool escript, remember to add the following line: {app, compiler, [{mod_cond, app}]}, Otherwise reltool might not include[3] some compiler modules which are used due to the `-mode(compile)` directive. As I don't think the nodetool escript has anything to gain by being compiled, I've posted a patch to delete[4] the directive. So, the question is, is this expected reltool behaviour? If so, is this really desired or not possibly including much more than is actually used. Shouldn't the sys-level {incl_cond, derived} make sure only used applications are included (regardless of mod_cond)? [1] http://erlang.org/pipermail/erlang-questions/2012-December/071135.html [2] https://github.com/rebar/rebar/pull/43 [3] https://github.com/rebar/rebar/issues/64#issuecomment-15998750 [4] https://github.com/rebar/rebar/pull/80 From co7eb@REDACTED Sat Apr 6 21:11:20 2013 From: co7eb@REDACTED (=?iso-8859-1?Q?Ivan_Carmenates_Garc=EDa?=) Date: Sat, 6 Apr 2013 14:11:20 -0500 Subject: [erlang-questions] I am looking for an Erlang job Message-ID: <006c01ce32fa$8dc27fb0$a9477f10$@frcuba.co.cu> Hi all, first of all I want to apologize for any disgust. I?m looking for a job in Erlang, I think I?m ready to program hard in Erlang. I accept any proposition you have. Greetings, Ivan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From francesco@REDACTED Sat Apr 6 21:44:58 2013 From: francesco@REDACTED (Francesco Cesarini) Date: Sat, 06 Apr 2013 20:44:58 +0100 Subject: [erlang-questions] I am looking for an Erlang job In-Reply-To: <006c01ce32fa$8dc27fb0$a9477f10$@frcuba.co.cu> References: <006c01ce32fa$8dc27fb0$a9477f10$@frcuba.co.cu> Message-ID: We have a job board on Erlang Central. A few more jobs will be added on Monday. http://erlangcentral.org/jobs/ Francesco "Ivan Carmenates Garc?a" wrote: >Hi all, first of all I want to apologize for any disgust. > > > >I?m looking for a job in Erlang, I think I?m ready to program hard in >Erlang. > >I accept any proposition you have. > > > >Greetings, > >Ivan. > > > >------------------------------------------------------------------------ > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -- Erlang Solutions Ltd -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierrefenoll@REDACTED Sat Apr 6 22:34:33 2013 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Sat, 6 Apr 2013 22:34:33 +0200 Subject: [erlang-questions] =?utf-8?b?cm91bmQoMS4wZTIzKSDiiaAgMTDCssKz?= Message-ID: Hi, I have notice that on both my R16B 64bit machines (Archlinux & OSX), round(1.0e23) = 99999999999999991611392 Instead of 10?? as it should. I think it's a matter of float to int. Do you guys have any idea? Am I doing something wrong? Cheers, Pierre -------------- next part -------------- An HTML attachment was scrubbed... URL: From eriksoe@REDACTED Sat Apr 6 23:00:02 2013 From: eriksoe@REDACTED (=?ISO-8859-1?Q?Erik_S=F8e_S=F8rensen?=) Date: Sat, 6 Apr 2013 23:00:02 +0200 Subject: [erlang-questions] =?utf-8?b?cm91bmQoMS4wZTIzKSDiiaAgMTDCssKz?= In-Reply-To: References: Message-ID: Hi there, There are certain common misconceptions around floating-point numbers. The key thing to keep in mind is that a) they have limited precision, as they are represented in a fixed number of bits, and b) they are represented in base 2, so numbers which look nice in base 10 and "feel" as if they should not suffer from precision loss, may surprise by doing so. In your case, notice that for 10?? to be represented accurately takes >3*23 = 69 bits -- and Erlang floats are represented in 64 bits (IEEE 754 double precision / C's "double") -- which means that even if all 64 bits were used for the mantissa, the number could not be represented accurately. In fact, there are only 52+1 bits for the mantissa, so the problem starts already at around 10^16: round(10000000000000001.0) => 10000000000000000. Hoping this helps, /Erik 2013/4/6 Pierre Fenoll > Hi, > > I have notice that on both my R16B 64bit machines (Archlinux & OSX), > round(1.0e23) = 99999999999999991611392 > Instead of 10?? as it should. > > I think it's a matter of float to int. > Do you guys have any idea? Am I doing something wrong? > > Cheers, > Pierre > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From seth@REDACTED Sun Apr 7 07:09:25 2013 From: seth@REDACTED (Seth Falcon) Date: Sat, 6 Apr 2013 22:09:25 -0700 Subject: [erlang-questions] Querying JSON, team chat In-Reply-To: References: Message-ID: I like the simple interface in jsonq. If you end up needing to set or delete values, or to validate the structure of incoming JSON based on a "jterms" like spec, checkout ej (shameless plug) https://github.com/seth/ej + seth On Wed, Mar 27, 2013 at 9:51 AM, Andrei Soroker wrote: > Greetings! > > I've had to deal with a lot of JSON over the last year or so and I > wholeheartedly recommend Jiffy [1] for encoding and decoding JSON. One > problem I had was querying the decoded JSON (we call the resulting > Erlang terms jterms). To address this, I wrote a very simple > XPath-inspired query library [2]. Perhaps others would find it useful. > > In the spirit of attempting to embrace the subject of self-promotion > in the Erlang community, I'd like to self-promote my new project - > LeChat [3]. It's a team chat service that emphasizes history search, > humane copy-pasting of code, and reasonable UI. The backend is handled > by Cowboy and Postgres, the fronted is knockout.js, and the two are > connected by websockets. > > We're charging $1/user/month after a 30 day trial, but it's free for > use in schools (universities, etc). > > Andrei > > [1] https://github.com/davisp/jiffy > [2] https://github.com/lechat-im/jsonq > [3] http://lechat.im > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Seth Falcon | @sfalcon | http://userprimary.net/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Sun Apr 7 10:52:41 2013 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Sun, 7 Apr 2013 10:52:41 +0200 Subject: [erlang-questions] Problem with Erlide In-Reply-To: <3D0492343D4AAE48879AA6F0CBBFC6624B3925CA@ISRTLVEXMX2.interwise.com> References: <3D0492343D4AAE48879AA6F0CBBFC6624B3925CA@ISRTLVEXMX2.interwise.com> Message-ID: Hi Uri, Sorry for the delay, I was on vacation. The first dialog you can ignore (if you trust us, that is), it tells that the plugins are not signed. I don't think I saw any signed plugins except those from the eclipse foundation. For the other problem, please locate and send me (off list) the logs from /erlide.log and /.metadata/.log. regards, Vlad On Wed, Apr 3, 2013 at 4:16 PM, Uri Velner wrote: > Hi,**** > > I'm new to Erlang developing ?**** > > I've tried the Erlide for eclipse. But, it seems not to work so well. **** > > I'm not sure the installation went well. During installation I got this > warning:**** > > [image: cid:part1.08010701.05070400@REDACTED]**** > > Although, I can see the Erlide in the Eclipse Marketplace ?> installed.*** > * > > **** > > When I'm trying to create new Erlang project in Eclipse I get the follows: > **** > > [image: cid:part2.03080308.03030302@REDACTED]**** > > I've searched for this error. But, didn't get real solution for my problem. > **** > > Can you explain me what went wrong ?**** > > ** ** > > Thanks in Advance.**** > > Uri**** > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.png Type: image/png Size: 23878 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 26699 bytes Desc: not available URL: From tony@REDACTED Sun Apr 7 11:37:30 2013 From: tony@REDACTED (Tony Rogvall) Date: Sun, 7 Apr 2013 11:37:30 +0200 Subject: [erlang-questions] =?utf-8?b?cm91bmQoMS4wZTIzKSDiiaAgMTDCssKz?= In-Reply-To: References: Message-ID: To help? understand how float works here is a code fragment implementing trunc in erlang: Here you can see both the representation and how to handle the exponent (for 64 bit IEEE floats). -module(float). -compile(export_all). trunc(X) when is_float(X) -> <> = <>, Exp = E - 1023, Sign = if S =:= 0 -> 1; true -> -1 end, if Exp >= 0 -> if Exp >= 52 -> Sign*(((1 bsl 52)+F) bsr (52-Exp)); true -> Sign*(((1 bsl 52)+F) bsl (Exp-52)) end; true -> 0 end. Some examples: > float:trunc(1.0e22). 10000000000000000000000 > float:trunc(1.0e23). 99999999999999991611392 > float:trunc(0.9999999999999999). 0 > float:trunc(0.99999999999999999). 1 ... > float:trunc(float(1 bsl 127)). 170141183460469231731687303715884105728 > (1 bsl 127). 170141183460469231731687303715884105728 /Tony On 6 apr 2013, at 23:00, Erik S?e S?rensen wrote: > Hi there, > > There are certain common misconceptions around floating-point numbers. > The key thing to keep in mind is that a) they have limited precision, as they are represented in a fixed number of bits, and b) they are represented in base 2, so numbers which look nice in base 10 and "feel" as if they should not suffer from precision loss, may surprise by doing so. > > In your case, notice that for 10?? to be represented accurately takes >3*23 = 69 bits -- and Erlang floats are represented in 64 bits (IEEE 754 double precision / C's "double") -- which means that even if all 64 bits were used for the mantissa, the number could not be represented accurately. In fact, there are only 52+1 bits for the mantissa, so the problem starts already at around 10^16: > round(10000000000000001.0) => 10000000000000000. > > > Hoping this helps, > /Erik > > > 2013/4/6 Pierre Fenoll > Hi, > > I have notice that on both my R16B 64bit machines (Archlinux & OSX), > round(1.0e23) = 99999999999999991611392 > Instead of 10?? as it should. > > I think it's a matter of float to int. > Do you guys have any idea? Am I doing something wrong? > > Cheers, > Pierre > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From helge@REDACTED Sun Apr 7 12:58:32 2013 From: helge@REDACTED (Helge Rausch) Date: Sun, 07 Apr 2013 12:58:32 +0200 Subject: [erlang-questions] [ANN] Axiom 0.1.0 released Message-ID: <51615158.9070507@rausch.io> Hello Erlang people! I released Axiom 0.1.0 today! For some it is a micro-framework for developing web applications in Erlang (similar tu Ruby's Sinatra, by which it is inspired), for others it's a convenience layer on top of Cowboy. I also heard quite a few times that it was great for Erlang beginners to get into the language and have something to play around with. Find it here: https://github.com/tsujigiri/axiom Cheers, Helge -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: OpenPGP digital signature URL: From dave@REDACTED Sun Apr 7 16:58:35 2013 From: dave@REDACTED (David Goehrig) Date: Sun, 7 Apr 2013 10:58:35 -0400 Subject: [erlang-questions] Adding new operators? Message-ID: <2DCB80CB-8712-4A53-91AB-76643E84E7D1@nexttolast.com> So I've been playing with a couple operators that compliment existing Erlang. -=-=- dave@REDACTED -=-=- From seth@REDACTED Sun Apr 7 17:58:09 2013 From: seth@REDACTED (Seth Falcon) Date: Sun, 7 Apr 2013 08:58:09 -0700 Subject: [erlang-questions] Querying JSON, team chat In-Reply-To: References: Message-ID: (hope you don't mind that I put this back on the list) On Sat, Apr 6, 2013 at 11:04 PM, Andrei Soroker wrote: > Whoa, I had no idea ej existed! Why did you pick a tuple vs. a list for > the path? > To be honest, I don't think this was a very good design choice on my part and I would go with a list if I was starting over. I wanted to be able to support accessing array elements by index and if I'm remembering my thought process, I was worried about the ambiguity of passing a bare string accidentally instead of a list. In ej, you can do: J = [<<"a1">>, <<"a2">>, <<"a3">>]. ej:get({2}, J). <<"a2">> If the path argument was a list, and bare integers were still the way to ask for an array element by index, then the code would allow you to say this: ej:get("accident", J) That would probably end up failing anyway, just not straight away. Given how infrequently I've ended up with a need for array access by index, I think a better choice would have been to stick with list for the path and then tag indices like {idx, 5}. > I am also somewhat partial to the analogy with XPath, which only addresses > reading and not writing. > Horses for courses :) We end up using the set and delete quite a bit for normalizing incoming data before validating with ej:valid. + seth -- Seth Falcon | @sfalcon | http://userprimary.net/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Sun Apr 7 23:55:02 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 8 Apr 2013 09:55:02 +1200 Subject: [erlang-questions] =?utf-8?b?cm91bmQoMS4wZTIzKSDiiaAgMTDCssKz?= In-Reply-To: References: Message-ID: <18B6829A-130D-45E6-B64E-D4366CE5FCB7@cs.otago.ac.nz> On 7/04/2013, at 8:34 AM, Pierre Fenoll wrote: > Hi, > > I have notice that on both my R16B 64bit machines (Archlinux & OSX), > round(1.0e23) = 99999999999999991611392 > Instead of 10?? as it should. WHy do you think that? > > I think it's a matter of float to int. > Do you guys have any idea? Am I doing something wrong? Yes I do and yes you are. (10 raisedToInteger: 23) asDouble rounded => 99999999999999991611392 in VisualWorks Smalltalk. There's nothing wrong with round(): 1.0e23 *is* 99999999999999991611392.0. This has nothing to do with R16B, nothing to do with Erlang, and nothing to do with whether your machine is 32-bit or 64-bit or what operating system it is running. It's all about the precision of IEEE doubles. Let's see what's happening. (10 raisedToInteger: 23) printStringRadix: 16 => '152D02C7E14AF6800000' 99999999999999991611392 printStringRadix: 16 => '152D02C7E14AF6000000' Yep, 10**23 has *just* too many bits between its most significant 1 and its least significant 1 to fit into an IEEE double. This number needs a 54-bit significand, but IEEE doubles offer only 53. From dave@REDACTED Mon Apr 8 02:12:40 2013 From: dave@REDACTED (David Goehrig) Date: Sun, 7 Apr 2013 20:12:40 -0400 Subject: [erlang-questions] Adding new operators? In-Reply-To: <2DCB80CB-8712-4A53-91AB-76643E84E7D1@nexttolast.com> References: <2DCB80CB-8712-4A53-91AB-76643E84E7D1@nexttolast.com> Message-ID: <5B558EDF-A921-475C-B33A-B2F41C5045A2@nexttolast.com> > So I've been playing with a couple operators that compliment existing Erlang. And to continue the thought, these operators are currently added via source preprocessing and translated to some module calls. They aren't terribly fast, but have shown themselves to be terribly useful. @ - the f-script inspired parallel send which either sends to a list of pids $ - like @ but matches the registered process names against a pattern/regex in parallel While both of these are implementable using ordinary Erlang, having explicitly parallel message send operators makes multicall & friends much easier to reason about. One of the use cases I'm particularly interested in is applying these operators to run many parallel NLP and machine learning applications to a stream of incoming messages. Next step is integrating into the stack better. But the question remains, what is the best way to add a new operator. Dave -=-=- dave@REDACTED -=-=- On Apr 7, 2013, at 10:58 AM, David Goehrig wrote: > > > -=-=- dave@REDACTED -=-=- From n.oxyde@REDACTED Mon Apr 8 08:21:17 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Mon, 8 Apr 2013 08:21:17 +0200 Subject: [erlang-questions] Adding new operators? In-Reply-To: <5B558EDF-A921-475C-B33A-B2F41C5045A2@nexttolast.com> References: <2DCB80CB-8712-4A53-91AB-76643E84E7D1@nexttolast.com> <5B558EDF-A921-475C-B33A-B2F41C5045A2@nexttolast.com> Message-ID: Hello David, A fancy way to extend the Erlang syntax would be to have something like Camlp4, a powerful and extensible preprocessor for OCaml. It has a powerful quotation system which it itself uses to implements its parser construct, to extend OCaml's syntax at compile-time. I have began writing a quotation system for Erlang but didn't have the time finish it yet [1]. Regards, [1] https://github.com/nox/otp/tree/brackets -- Anthony Ramine Le 8 avr. 2013 ? 02:12, David Goehrig a ?crit : > Next step is integrating into the stack better. But the question remains, what is the best way to add a new operator. From zabrane3@REDACTED Mon Apr 8 11:17:09 2013 From: zabrane3@REDACTED (Zabrane Mickael) Date: Mon, 8 Apr 2013 11:17:09 +0200 Subject: [erlang-questions] Text crunching ... help needed Message-ID: Hi guys, I'm facing a nice problem in order to accelerate words search in a fairly large file. * Problem: get a list of [{offset, size} | ...] for each word in a text file. * Baseline: For the purpose of the exercise, I'm using an online version of "King James Bible". $ wget http://printkjv.ifbweb.com/AV_txt.zip $ unzip -a AV_txt.zip $ erl > {ok, Bin} = file:read_file("AV1611Bible.txt"). > {ok, MP} = re:compile("\\w+"). > {match, L} = re:run(Bin, MP, [global]). > length(L). 839979 When timing this version on my machine, I got: > timer:tc( fun() -> re:run(Bin, MP, [global]) end ). 2002007 us., which is OK. But can we do better? And how fast we can go? The word separators for this problem are: $\s, $\t, $\n, and $\r. You can use anything you'd like to accelerate the solution (binary matching, os:cmd(), open_port, NIF, LinkedIn driver). The only one constraint is to get back a list of [{offset, size} | ...] as a result. Waiting your hacks ... thanks!!! Regards, Zab From james@REDACTED Mon Apr 8 11:30:12 2013 From: james@REDACTED (James Aimonetti) Date: Mon, 8 Apr 2013 11:30:12 +0200 Subject: [erlang-questions] Text crunching ... help needed In-Reply-To: References: Message-ID: <51628E24.4050507@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 You may want to research the widefinder project that Tim Bray did a while ago; I think there were quite a few submissions in Erlang that had interesting optimizations for text processing. On 04/08/2013 11:17 AM, Zabrane Mickael wrote: > Hi guys, > > I'm facing a nice problem in order to accelerate words search in a > fairly large file. > > * Problem: get a list of [{offset, size} | ...] for each word in a > text file. > > * Baseline: For the purpose of the exercise, I'm using an online > version of "King James Bible". > > $ wget http://printkjv.ifbweb.com/AV_txt.zip $ unzip -a AV_txt.zip > > $ erl >> {ok, Bin} = file:read_file("AV1611Bible.txt"). {ok, MP} = >> re:compile("\\w+"). {match, L} = re:run(Bin, MP, [global]). >> length(L). > 839979 > > When timing this version on my machine, I got: >> timer:tc( fun() -> re:run(Bin, MP, [global]) end ). > > 2002007 us., which is OK. But can we do better? And how fast we can > go? > > The word separators for this problem are: $\s, $\t, $\n, and $\r. > You can use anything you'd like to accelerate the solution (binary > matching, os:cmd(), open_port, NIF, LinkedIn driver). > > The only one constraint is to get back a list of [{offset, size} | > ...] as a result. > > Waiting your hacks ... thanks!!! > > Regards, Zab > > > > _______________________________________________ erlang-questions > mailing list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > - -- James Aimonetti Distributed Systems Engineer / DJ MC_ 2600hz | http://2600hz.com sip:james@REDACTED tel: 415.886.7905 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJRYo4kAAoJENc77s1OYoGgYX8H/2wgYJGINGrseQpw7gYpuy7W lAQ2dHBHFUcn2V8kTKBZpCIxC62DRgeuNmJQ6wiHn5KuNonLtUcFSN2Nh2Z2KKNi TbbOJOJggswpgLK2Cop+pk2+9811x1bhCPDm5cLtrK2m3D8lMjgCLdvycEfbaAd5 oAKZgIJ7PD2syccMoMcyn6UI6Jf0DrEG+U0r6Aiy81rSZv1WqEtzWT1M4rfb6LYO wFUwPWBhgcDAYUA+6A1I+uaav/IU+lGmQ/gie5ZXHvIbrTl2uD++4fyZU6QKD5Ck VkJhOO/PPS3XccE5m+tgWm38ZnLiikREiVtOZYQxNA0RCue9Bwoq4B5awy1MgBA= =jtwl -----END PGP SIGNATURE----- From zabrane3@REDACTED Mon Apr 8 11:45:27 2013 From: zabrane3@REDACTED (Zabrane Mikael) Date: Mon, 8 Apr 2013 11:45:27 +0200 Subject: [erlang-questions] Text crunching ... help needed In-Reply-To: <51628E24.4050507@2600hz.com> References: <51628E24.4050507@2600hz.com> Message-ID: Thanks James. Already checked that. /Zab 2013/4/8 James Aimonetti > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > You may want to research the widefinder project that Tim Bray did a > while ago; I think there were quite a few submissions in Erlang that > had interesting optimizations for text processing. > > > On 04/08/2013 11:17 AM, Zabrane Mickael wrote: > > Hi guys, > > > > I'm facing a nice problem in order to accelerate words search in a > > fairly large file. > > > > * Problem: get a list of [{offset, size} | ...] for each word in a > > text file. > > > > * Baseline: For the purpose of the exercise, I'm using an online > > version of "King James Bible". > > > > $ wget http://printkjv.ifbweb.com/AV_txt.zip $ unzip -a AV_txt.zip > > > > $ erl > >> {ok, Bin} = file:read_file("AV1611Bible.txt"). {ok, MP} = > >> re:compile("\\w+"). {match, L} = re:run(Bin, MP, [global]). > >> length(L). > > 839979 > > > > When timing this version on my machine, I got: > >> timer:tc( fun() -> re:run(Bin, MP, [global]) end ). > > > > 2002007 us., which is OK. But can we do better? And how fast we can > > go? > > > > The word separators for this problem are: $\s, $\t, $\n, and $\r. > > You can use anything you'd like to accelerate the solution (binary > > matching, os:cmd(), open_port, NIF, LinkedIn driver). > > > > The only one constraint is to get back a list of [{offset, size} | > > ...] as a result. > > > > Waiting your hacks ... thanks!!! > > > > Regards, Zab > > > > > > > > _______________________________________________ erlang-questions > > mailing list erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > - -- > James Aimonetti > Distributed Systems Engineer / DJ MC_ > > 2600hz | http://2600hz.com > sip:james@REDACTED > tel: 415.886.7905 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.11 (GNU/Linux) > Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ > > iQEcBAEBAgAGBQJRYo4kAAoJENc77s1OYoGgYX8H/2wgYJGINGrseQpw7gYpuy7W > lAQ2dHBHFUcn2V8kTKBZpCIxC62DRgeuNmJQ6wiHn5KuNonLtUcFSN2Nh2Z2KKNi > TbbOJOJggswpgLK2Cop+pk2+9811x1bhCPDm5cLtrK2m3D8lMjgCLdvycEfbaAd5 > oAKZgIJ7PD2syccMoMcyn6UI6Jf0DrEG+U0r6Aiy81rSZv1WqEtzWT1M4rfb6LYO > wFUwPWBhgcDAYUA+6A1I+uaav/IU+lGmQ/gie5ZXHvIbrTl2uD++4fyZU6QKD5Ck > VkJhOO/PPS3XccE5m+tgWm38ZnLiikREiVtOZYQxNA0RCue9Bwoq4B5awy1MgBA= > =jtwl > -----END PGP SIGNATURE----- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Regards Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Mon Apr 8 12:00:08 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 8 Apr 2013 12:00:08 +0200 Subject: [erlang-questions] how to debug the erlang boot? Message-ID: Hi, I have an application that load Erlang application in a thread (on android) using a patched version of Erlang [1]. I'm using application by passsing following parameter to erl_start: ["beam", "-K", "true", "--", "-noinput", "-boot_var", "APK", path + "/apk.ez", "-kernel", "inetrc", "\""+ path + "/erlang/bin/erl_inetrc\"", "-native_lib_path", path + "/lib", "-sasl", "errlog_type", "all", "-boot", path + "/erlang/bin/start", "-root", path + "/apk.ez/assets", "-eval", "R = application:start(somapp), io:format(\"~w~n\",[R])."] The thread start but the application isn't launched. for the boots script i'm using the minimal one created during the bootstrap. Any idea how I could log what happen on boot. The thread is just running without any error published. I can open more code if needed. - benoit [1] https://github.com/benoitc/otp/tree/OTP_R16B_ANDROID From hm@REDACTED Mon Apr 8 12:40:02 2013 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Mon, 8 Apr 2013 12:40:02 +0200 Subject: [erlang-questions] how to debug the erlang boot? In-Reply-To: References: Message-ID: Try -init_debug (and possible -loader_debug) to get more more info about what is going on during boot. /H?kan On Mon, Apr 8, 2013 at 12:00 PM, Benoit Chesneau wrote: > Hi, > > I have an application that load Erlang application in a thread (on > android) using a patched version of Erlang [1]. I'm using application > by passsing following parameter to erl_start: > > > ["beam", "-K", "true", "--", > "-noinput", > "-boot_var", "APK", path + "/apk.ez", > "-kernel", "inetrc", "\""+ path + "/erlang/bin/erl_inetrc\"", > "-native_lib_path", path + "/lib", > "-sasl", "errlog_type", "all", > "-boot", path + "/erlang/bin/start", > "-root", path + "/apk.ez/assets", > "-eval", "R = application:start(somapp), io:format(\"~w~n\",[R])."] > > The thread start but the application isn't launched. for the boots > script i'm using the minimal one created during the bootstrap. > > Any idea how I could log what happen on boot. The thread is just > running without any error published. I can open more code if needed. > > > - benoit > > [1] https://github.com/benoitc/otp/tree/OTP_R16B_ANDROID > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Mon Apr 8 13:32:12 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 8 Apr 2013 13:32:12 +0200 Subject: [erlang-questions] how to debug the erlang boot? In-Reply-To: References: Message-ID: On Mon, Apr 8, 2013 at 12:40 PM, H?kan Mattsson wrote: > Try -init_debug (and possible -loader_debug) to get more more info about > what is going on during boot. Thanks, trying it now. - beno[it > > /H?kan > > On Mon, Apr 8, 2013 at 12:00 PM, Benoit Chesneau > wrote: >> >> Hi, >> >> I have an application that load Erlang application in a thread (on >> android) using a patched version of Erlang [1]. I'm using application >> by passsing following parameter to erl_start: >> >> >> ["beam", "-K", "true", "--", >> "-noinput", >> "-boot_var", "APK", path + "/apk.ez", >> "-kernel", "inetrc", "\""+ path + "/erlang/bin/erl_inetrc\"", >> "-native_lib_path", path + "/lib", >> "-sasl", "errlog_type", "all", >> "-boot", path + "/erlang/bin/start", >> "-root", path + "/apk.ez/assets", >> "-eval", "R = application:start(somapp), io:format(\"~w~n\",[R])."] >> >> The thread start but the application isn't launched. for the boots >> script i'm using the minimal one created during the bootstrap. >> >> Any idea how I could log what happen on boot. The thread is just >> running without any error published. I can open more code if needed. >> >> >> - benoit >> >> [1] https://github.com/benoitc/otp/tree/OTP_R16B_ANDROID >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > From vincent.dephily@REDACTED Mon Apr 8 13:32:58 2013 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Mon, 08 Apr 2013 13:32:58 +0200 Subject: [erlang-questions] [whishlist] Adjustable erlang:now() resync speed Message-ID: <2140176.9RAfQ0mVpy@moltowork> Hi list, I keep wishing for this feature, and because I am very naive and not doing the work myself, I assume it wouldn't be all that hard to implement : It would be great to have an emulator argument to specify how fast the VM time adjusts to system time. Basically "+c" but with a number appended. For example defined as the number of millisecond per second that the VM is allowed to add/subtract to converge to system time : * "+C 10" would correspond to the current behaviour. * "+C 1000" would take an hour to catch up an hour. * "+C 99999999999999" would be equivalent to "+c". * "+C 0" would not try to do any sync. Or a fancyer definition to make big jumps at first, and smaller ones at the end : Add max(10, Percent * TimeDifference) millisecond per second to catch up the TimeDifference : * "+C 0.01" whould catch up a second in 100 seconds, a minute in 8 minutes, an hour in 15 minutes, and a day in 20 minutes. * "+C 0.1" would catch up a day in 140 seconds; the first jump being a hefty 2.4 hours, but jumping just 15 seconds after a minute, and less than a second 16 seconds later (see script below). * "+C 1.0" would be equivalent to "+c". Why ? * How far timers are allowed to jump is very application-dependant. A cache TTL for a website is much less sensitive than a sound processing pipeline. They should have different settings. * The current behaviour is a pain for devices that sleep frequently. I put my work laptop to sleep every evening, and every morning I have to restart all my erlang VMs because they are otherwise unusable (more than a month to sync). On a system like Android that sleeps willy-nilly, it would be even worse. * The existing "+c" argument is too drastic for most cases; if you're using that maybe you should be using system time in your code instead. Slow sync is a great feature. Does that sound like a good idea to ye ? #!/usr/bin/env escript main([Percent]) -> main([Percent, "3600000", "10"]); main([Percent, Delta]) -> main([Percent, Delta, "10"]); main([Percent, Delta, Min]) -> P = list_to_float(Percent), D = list_to_integer(Delta), M = list_to_integer(Min), io:format("Converged in ~p seconds.~n", [converge(0, P, 0, D, M)]); main(_) -> io:format("Usage: converge_now.es speed [delta] [min]~n"). converge(N,P,Vm,Sys,M) when Sys > Vm -> D = max(M,round((Sys-Vm)*P)), io:format("~p ~p ~p ~p~n", [N,D,Vm,Sys]), converge(N + 1, P, Vm + 1000 + D, Sys + 1000, M); converge(N,_,_,_,_) -> N. -- Vincent de Phily From zabrane3@REDACTED Mon Apr 8 15:18:04 2013 From: zabrane3@REDACTED (Zabrane Mikael) Date: Mon, 8 Apr 2013 15:18:04 +0200 Subject: [erlang-questions] Text crunching ... help needed In-Reply-To: References: <51628E24.4050507@2600hz.com> Message-ID: Here's a faster solution for the problem: ------------------------------------------------- -module(words). -export([go/1]). -define(WHITESPACE(C), ((C == $\s) orelse (C == $\n) orelse (C == $\t) orelse (C == $\f) orelse (C == $\r) orelse (C == $\v))). go(<<>>) -> [{0, 0}]; go(Bin) -> go(Bin, 0, 0, []). go(<>, Pos, Sz, Acc) -> case ?WHITESPACE(C) of true when Sz > 0 -> go(Rest, Pos + Sz + 1, 0, [{Pos, Sz} | Acc]); true -> go(Rest, Pos + Sz + 1, 0, Acc); false -> go(Rest, Pos, Sz + 1, Acc) end; go(<<>>, _, 0, Acc) -> lists:reverse(Acc); go(<<>>, Pos, Sz, Acc) -> lists:reverse([{Pos, Sz} | Acc]). ------------------------------------------------- $ erlc words.erl $ erl > {ok, Bin} = file:read_file("AV1611Bible.txt"). > {, L} = timer:tc(fun() -> words:go(Bin) end). 764842 us. Almost 2.6 times faster than the "regular expression" based solution. > length(L). 840245 We also found more words (equiv. to: $ wc -w AV1611Bible.txt). So guys, can we do better than that? Please advice/help/hack! regards /Zab 2013/4/8 Zabrane Mikael > Thanks James. Already checked that. > > /Zab > > > 2013/4/8 James Aimonetti > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> You may want to research the widefinder project that Tim Bray did a >> while ago; I think there were quite a few submissions in Erlang that >> had interesting optimizations for text processing. >> >> >> On 04/08/2013 11:17 AM, Zabrane Mickael wrote: >> > Hi guys, >> > >> > I'm facing a nice problem in order to accelerate words search in a >> > fairly large file. >> > >> > * Problem: get a list of [{offset, size} | ...] for each word in a >> > text file. >> > >> > * Baseline: For the purpose of the exercise, I'm using an online >> > version of "King James Bible". >> > >> > $ wget http://printkjv.ifbweb.com/AV_txt.zip $ unzip -a AV_txt.zip >> > >> > $ erl >> >> {ok, Bin} = file:read_file("AV1611Bible.txt"). {ok, MP} = >> >> re:compile("\\w+"). {match, L} = re:run(Bin, MP, [global]). >> >> length(L). >> > 839979 >> > >> > When timing this version on my machine, I got: >> >> timer:tc( fun() -> re:run(Bin, MP, [global]) end ). >> > >> > 2002007 us., which is OK. But can we do better? And how fast we can >> > go? >> > >> > The word separators for this problem are: $\s, $\t, $\n, and $\r. >> > You can use anything you'd like to accelerate the solution (binary >> > matching, os:cmd(), open_port, NIF, LinkedIn driver). >> > >> > The only one constraint is to get back a list of [{offset, size} | >> > ...] as a result. >> > >> > Waiting your hacks ... thanks!!! >> > >> > Regards, Zab >> > >> > >> > >> > _______________________________________________ erlang-questions >> > mailing list erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> >> >> - -- >> James Aimonetti >> Distributed Systems Engineer / DJ MC_ >> >> 2600hz | http://2600hz.com >> sip:james@REDACTED >> tel: 415.886.7905 >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1.4.11 (GNU/Linux) >> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ >> >> iQEcBAEBAgAGBQJRYo4kAAoJENc77s1OYoGgYX8H/2wgYJGINGrseQpw7gYpuy7W >> lAQ2dHBHFUcn2V8kTKBZpCIxC62DRgeuNmJQ6wiHn5KuNonLtUcFSN2Nh2Z2KKNi >> TbbOJOJggswpgLK2Cop+pk2+9811x1bhCPDm5cLtrK2m3D8lMjgCLdvycEfbaAd5 >> oAKZgIJ7PD2syccMoMcyn6UI6Jf0DrEG+U0r6Aiy81rSZv1WqEtzWT1M4rfb6LYO >> wFUwPWBhgcDAYUA+6A1I+uaav/IU+lGmQ/gie5ZXHvIbrTl2uD++4fyZU6QKD5Ck >> VkJhOO/PPS3XccE5m+tgWm38ZnLiikREiVtOZYQxNA0RCue9Bwoq4B5awy1MgBA= >> =jtwl >> -----END PGP SIGNATURE----- >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Regards > Zabrane > -- Regards Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From zabrane3@REDACTED Mon Apr 8 15:25:05 2013 From: zabrane3@REDACTED (Zabrane Mikael) Date: Mon, 8 Apr 2013 15:25:05 +0200 Subject: [erlang-questions] Text crunching ... help needed In-Reply-To: References: <51628E24.4050507@2600hz.com> Message-ID: By using native compilation, I got these numbers: $ erlc +native words.erl $ erl > {ok, Bin} = file:read_file("AV1611Bible.txt"). > f(L), {_, L} = timer:tc(fun() -> words:go(Bin) end). > length(L). 98643 us., this is 20x faster than the baseline solution. But again ... can we do better than that? /Zab 2013/4/8 Zabrane Mikael > Here's a faster solution for the problem: > > ------------------------------------------------- > -module(words). > > -export([go/1]). > > -define(WHITESPACE(C), ((C == $\s) orelse (C == $\n) orelse (C == $\t) > orelse (C == $\f) orelse (C == $\r) orelse (C == $\v))). > > go(<<>>) -> > [{0, 0}]; > go(Bin) -> > go(Bin, 0, 0, []). > > go(<>, Pos, Sz, Acc) -> > case ?WHITESPACE(C) of > true when Sz > 0 -> > go(Rest, Pos + Sz + 1, 0, [{Pos, Sz} | Acc]); > true -> > go(Rest, Pos + Sz + 1, 0, Acc); > false -> > go(Rest, Pos, Sz + 1, Acc) > end; > go(<<>>, _, 0, Acc) -> > lists:reverse(Acc); > go(<<>>, Pos, Sz, Acc) -> > lists:reverse([{Pos, Sz} | Acc]). > ------------------------------------------------- > > $ erlc words.erl > > $ erl > > {ok, Bin} = file:read_file("AV1611Bible.txt"). > > {, L} = timer:tc(fun() -> words:go(Bin) end). > > 764842 us. Almost 2.6 times faster than the "regular expression" based > solution. > > > length(L). > 840245 > > We also found more words (equiv. to: $ wc -w AV1611Bible.txt). > > So guys, can we do better than that? Please advice/help/hack! > > regards > /Zab > > > > 2013/4/8 Zabrane Mikael > >> Thanks James. Already checked that. >> >> /Zab >> >> >> 2013/4/8 James Aimonetti >> >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> You may want to research the widefinder project that Tim Bray did a >>> while ago; I think there were quite a few submissions in Erlang that >>> had interesting optimizations for text processing. >>> >>> >>> On 04/08/2013 11:17 AM, Zabrane Mickael wrote: >>> > Hi guys, >>> > >>> > I'm facing a nice problem in order to accelerate words search in a >>> > fairly large file. >>> > >>> > * Problem: get a list of [{offset, size} | ...] for each word in a >>> > text file. >>> > >>> > * Baseline: For the purpose of the exercise, I'm using an online >>> > version of "King James Bible". >>> > >>> > $ wget http://printkjv.ifbweb.com/AV_txt.zip $ unzip -a AV_txt.zip >>> > >>> > $ erl >>> >> {ok, Bin} = file:read_file("AV1611Bible.txt"). {ok, MP} = >>> >> re:compile("\\w+"). {match, L} = re:run(Bin, MP, [global]). >>> >> length(L). >>> > 839979 >>> > >>> > When timing this version on my machine, I got: >>> >> timer:tc( fun() -> re:run(Bin, MP, [global]) end ). >>> > >>> > 2002007 us., which is OK. But can we do better? And how fast we can >>> > go? >>> > >>> > The word separators for this problem are: $\s, $\t, $\n, and $\r. >>> > You can use anything you'd like to accelerate the solution (binary >>> > matching, os:cmd(), open_port, NIF, LinkedIn driver). >>> > >>> > The only one constraint is to get back a list of [{offset, size} | >>> > ...] as a result. >>> > >>> > Waiting your hacks ... thanks!!! >>> > >>> > Regards, Zab >>> > >>> > >>> > >>> > _______________________________________________ erlang-questions >>> > mailing list erlang-questions@REDACTED >>> > http://erlang.org/mailman/listinfo/erlang-questions >>> > >>> >>> >>> - -- >>> James Aimonetti >>> Distributed Systems Engineer / DJ MC_ >>> >>> 2600hz | http://2600hz.com >>> sip:james@REDACTED >>> tel: 415.886.7905 >>> -----BEGIN PGP SIGNATURE----- >>> Version: GnuPG v1.4.11 (GNU/Linux) >>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ >>> >>> iQEcBAEBAgAGBQJRYo4kAAoJENc77s1OYoGgYX8H/2wgYJGINGrseQpw7gYpuy7W >>> lAQ2dHBHFUcn2V8kTKBZpCIxC62DRgeuNmJQ6wiHn5KuNonLtUcFSN2Nh2Z2KKNi >>> TbbOJOJggswpgLK2Cop+pk2+9811x1bhCPDm5cLtrK2m3D8lMjgCLdvycEfbaAd5 >>> oAKZgIJ7PD2syccMoMcyn6UI6Jf0DrEG+U0r6Aiy81rSZv1WqEtzWT1M4rfb6LYO >>> wFUwPWBhgcDAYUA+6A1I+uaav/IU+lGmQ/gie5ZXHvIbrTl2uD++4fyZU6QKD5Ck >>> VkJhOO/PPS3XccE5m+tgWm38ZnLiikREiVtOZYQxNA0RCue9Bwoq4B5awy1MgBA= >>> =jtwl >>> -----END PGP SIGNATURE----- >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> >> -- >> Regards >> Zabrane >> > > > > -- > Regards > Zabrane > -- Regards Zabrane -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Mon Apr 8 16:54:36 2013 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 8 Apr 2013 16:54:36 +0200 Subject: [erlang-questions] using inet_gethost_native even if disabled In-Reply-To: References: Message-ID: <20130408145436.GA19356@erix.ericsson.se> On Wed, Apr 03, 2013 at 02:50:29PM +0200, Benoit Chesneau wrote: > So I tried to use the environment variable and it is still using the > native resolving: > > Here is the trace: > > I/ErlangThread( 1447): default inetrc path: > /data/data/com.example.app/erlang/bin/erl_inetrc > V/ErlangThread( 1447): > {error_logger,{{2013,4,3},{14,45,27}},"~s~n",["Error in process > <0.17.0> with exit value: > {epipe,[{erts_internal,port_command,[#Port<0.54>,[<<6 > bytes>>,\"localhost\",0],[]],[]},{erlang,port_command,2,[]},{inet_gethost_native,run_once,0,[{file,\"inet_gethost_native.erl\"},{line,152}]}]}\n"]} > Have you got more information on when this happens and on what OS. It appears to be Android, and both the Pid <0.17.0> and the inet_gethost_native:run_once() call indicates that this is early in the start of the node. Can you confirm. If so it might be that early in the start inet_gethost_native is called before [file,dns] has been enforced and thus can be regarded a bug. Is the node started with short or long hostnames? More information about exactly how the node is started is needed. It might also be that this is a problem that the Android porter has run into. > The first line display what is passed to setenv. > > And when I read the erl_inetrc content on the device I have: > > # cat /data/data/com.daleharvey.mobilefuton/erlang/bin/erl_inetrc > %% Use erlang's internal resolver. Native DNS on android is annoying > {lookup, [file,dns]}. > > > not sure what's the problem there... > > - beno?t > > > On Wed, Apr 3, 2013 at 2:47 PM, Benoit Chesneau wrote: > > On Wed, Apr 3, 2013 at 11:57 AM, Danil Zagoskin wrote: > >> Hello. > >> > >> Try to start your VM with custom inetrc. Just set ERL_INETRC environment > >> variable to inet config file path. > >> In config file write: > >> {lookup, [file, dns]}. > >> > >> This will tell erlang's resolver to use it's own DNS client, not system one. > >> More info at http://www.erlang.org/doc/apps/erts/inet_cfg.html > >> > > > > mmm thanks, I will try to set the env I was using the option `-kernel > > inetrc /path/to/erl_inetrc` for now but it doesn't seem to work . > > > > - beno?t > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From kvc@REDACTED Sat Apr 6 01:14:45 2013 From: kvc@REDACTED (Kevin Carosso) Date: Fri, 5 Apr 2013 16:14:45 -0700 Subject: [erlang-questions] clustering with SSL (-proto_dist inet_tls) Message-ID: We've run into a couple of issues using R15B03-1 and enabling SSL on Erlang distribution protocol between cluster members: (1) We're used to being able to restrict the range of ports used between the cluster members via the -kernel parameters inet_dist_listen_min and inet_dist_listen_max. These seem to be ignored by -proto_dist inet_tls Is this expected? Is there an alternative mechanism for specifying a range for coordination with firewalls? (2) Everytime a BEAM that's been clustered using dist inet_tls exits, it produces what appears to be debug output to the console, of the form: close called #Port<0.1074> [{inet_tls_dist,close,1, [{file,"inet_tls_dist.erl"},{line,99}]}, {lists,foreach,2,[{file,"lists.erl"},{line,1262}]}, {net_kernel,terminate,2, [{file,"net_kernel.erl"},{line,570}]}, {gen_server,terminate,6, [{file,"gen_server.erl"},{line,722}]}, {proc_lib,init_p_do_apply,3, [{file,"proc_lib.erl"},{line,227}]}] which is coming from inet_tls_dist.erl right here: close(Socket) -> try erlang:error(foo) catch _:_ -> io:format("close called ~p ~p~n",[Socket, erlang:get_stacktrace()]) end, gen_tcp:close(Socket), ok. Looks like left-over debug code, or similar. Is this a bug in R15B? Thanks much, /Kevin Carosso From lee.sylvester@REDACTED Tue Apr 9 00:06:42 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Mon, 8 Apr 2013 23:06:42 +0100 Subject: [erlang-questions] Mnesia with Records Confusion Message-ID: Hey guys, So, I have a record that looks a little like this -record(user, {pid, username}). That's a simplified version. Anyhoo, I was trying to add it to and remove it from a table in Mnesia, but kept getting a badarg error. I'd set the table up like so: mnesia:create_table(user, [ {type,set}, {record_name, user}, {index, [pid]}, {attributes, record_info(fields, user)} ]), This error had me baffled for a while as I was able to add records to the table, but couldn't read them with index_read. I eventually found that the issue was to do with the index of the pid property in the record and, by adding an id property as the first field, all my problems disappeared. Can someone please explain why this is so? I mean, I'm glad it works now, but I'd love to know why I had to do this. Thanks loads, Lee From ok@REDACTED Tue Apr 9 00:13:57 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 9 Apr 2013 10:13:57 +1200 Subject: [erlang-questions] Text crunching ... help needed In-Reply-To: References: Message-ID: <09FE44EC-7F0C-4215-822F-D4FD4DF59CC6@cs.otago.ac.nz> On 8/04/2013, at 9:17 PM, Zabrane Mickael wrote: > Hi guys, > > I'm facing a nice problem in order to accelerate words search in a fairly large file. > > * Problem: > get a list of [{offset, size} | ...] for each word in a text file. > > * Baseline: > For the purpose of the exercise, I'm using an online version of "King James Bible". > > $ wget http://printkjv.ifbweb.com/AV_txt.zip > $ unzip -a AV_txt.zip There are some oddities about that file. For example, it appears to be a document with CR+LF line endings compressed as binary, so that -a does not remove the CRs. There are stray "#" characters for no apparent reason. And above all, verses are written one per line with leading verse *numbers*, except for Philemon, where the first two verses are in a single line. The chapter and verse numbers are part of the *structure* of the document, not part of the *contents*. There _are_ as it happens some numbers in the text (but not in the actual Bible, in the preamble) which one might want to search for. > $ erl >> {ok, Bin} = file:read_file("AV1611Bible.txt"). >> {ok, MP} = re:compile("\\w+"). >> {match, L} = re:run(Bin, MP, [global]). >> length(L). > 839979 > > When timing this version on my machine, I got: >> timer:tc( fun() -> re:run(Bin, MP, [global]) end ). > > 2002007 us., which is OK. > But can we do better? And how fast we can go? > > The word separators for this problem are: $\s, $\t, $\n, and $\r. Not counting \n and \r, I get this list: 806901 ' ' 72022 ',' 26593 '.' 21533 ']' 21533 '[' 12748 ':' 10312 ';' 3360 '?' 2950 '#' 2105 ''' 896 '-' 315 '!' 306 ')' 306 '(' 141 '"' 2 '>' 2 '<' 1 '&' One <> pair comes from "b" and another from "". For your purposes, you probably want to discard those characters completely. The use of the square brackets in the preamble is distinctly odd. I suspect that whoever prepared this text intended [] to mark the use of italics. You might well want to treat "--" and "-" differently: someone looking for "Bethlehem" might be disappointed to miss "Beth-lehem". > You can use anything you'd like to accelerate the solution (binary matching, os:cmd(), open_port, NIF, LinkedIn driver). > > The only one constraint is to get back a list of [{offset, size} | ...] as a result. If you have an n-core machine, you could simply chop the binary into n pieces, find words in each of them, adjust the results (a bit like how you do parallel prefix on a coarse-grained machine), and paste the results together. In fact, keeping them separate might do even better, depending on what you want to DO with the results. m% ctime mawk -f idct.awk AV1611Bible.txt 804918 occurrences of 13579 distinct words. 0.570 user + 0.070 system = 0.640 total in 1.074 real seconds. This ignores alphabetic case. For many text processing purposes, a list of words is better than a list of byte slices. One possible representation of a document is - a tuple of binaries (or strings, or atoms, or what you please) saying what the distinct words _are_ (and if that tuple is not in alphabetic order, you might want to have a hash table inverting it) - a list (or tuple, or tree, or what you please) of small integers indexing into the dictionary, saying which words are where. It is certainly going to be a lot quicker to search for a phrase in such a representation. > > Waiting your hacks ... thanks!!! From zabrane3@REDACTED Tue Apr 9 00:32:20 2013 From: zabrane3@REDACTED (Zabrane Mickael) Date: Tue, 9 Apr 2013 00:32:20 +0200 Subject: [erlang-questions] Text crunching ... help needed In-Reply-To: <09FE44EC-7F0C-4215-822F-D4FD4DF59CC6@cs.otago.ac.nz> References: <09FE44EC-7F0C-4215-822F-D4FD4DF59CC6@cs.otago.ac.nz> Message-ID: <5CACC54B-7E13-4DC4-9615-2649D61D18DA@gmail.com> Hi Richard, Thanks for the analysis. KJB was simply choosen to simplify the problem. Maybe you can suggest a better one as long as it'll exceed +4MB size (and up to 100MB). My main concern now is to find out a speedy way for words detection (i.e [{offset, length} | ...]). Regards, Zabrane > There are some oddities about that file. For example, it appears to be > a document with CR+LF line endings compressed as binary, so that -a does > not remove the CRs. There are stray "#" characters for no apparent > reason. And above all, verses are written one per line with leading > verse *numbers*, except for Philemon, where the first two verses are in > a single line. The chapter and verse numbers are part of the *structure* > of the document, not part of the *contents*. There _are_ as it happens > some numbers in the text (but not in the actual Bible, in the preamble) > which one might want to search for. > >> $ erl >>> {ok, Bin} = file:read_file("AV1611Bible.txt"). >>> {ok, MP} = re:compile("\\w+"). >>> {match, L} = re:run(Bin, MP, [global]). >>> length(L). >> 839979 >> >> When timing this version on my machine, I got: >>> timer:tc( fun() -> re:run(Bin, MP, [global]) end ). >> >> 2002007 us., which is OK. >> But can we do better? And how fast we can go? >> >> The word separators for this problem are: $\s, $\t, $\n, and $\r. > > Not counting \n and \r, I get this list: > 806901 ' ' > 72022 ',' > 26593 '.' > 21533 ']' > 21533 '[' > 12748 ':' > 10312 ';' > 3360 '?' > 2950 '#' > 2105 ''' > 896 '-' > 315 '!' > 306 ')' > 306 '(' > 141 '"' > 2 '>' > 2 '<' > 1 '&' > > One <> pair comes from "b" and another from "". > For your purposes, you probably want to discard those characters completely. > The use of the square brackets in the preamble is distinctly odd. > I suspect that whoever prepared this text intended [] to mark the use of italics. > You might well want to treat "--" and "-" differently: someone looking for > "Bethlehem" might be disappointed to miss "Beth-lehem". > >> You can use anything you'd like to accelerate the solution (binary matching, os:cmd(), open_port, NIF, LinkedIn driver). >> >> The only one constraint is to get back a list of [{offset, size} | ...] as a result. > > If you have an n-core machine, you could simply chop the binary into n pieces, > find words in each of them, adjust the results (a bit like how you do parallel > prefix on a coarse-grained machine), and paste the results together. In fact, > keeping them separate might do even better, depending on what you want to DO > with the results. > > m% ctime mawk -f idct.awk AV1611Bible.txt > 804918 occurrences of 13579 distinct words. > 0.570 user + 0.070 system = 0.640 total in 1.074 real seconds. > This ignores alphabetic case. > > For many text processing purposes, a list of words is better than a > list of byte slices. One possible representation of a document is > - a tuple of binaries (or strings, or atoms, or what you please) > saying what the distinct words _are_ > (and if that tuple is not in alphabetic order, you might want to > have a hash table inverting it) > - a list (or tuple, or tree, or what you please) of small integers > indexing into the dictionary, saying which words are where. > It is certainly going to be a lot quicker to search for a phrase in > such a representation. >> >> Waiting your hacks ... thanks!!! > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Tue Apr 9 00:39:28 2013 From: jose.valim@REDACTED (=?ISO-8859-1?Q?Jos=E9_Valim?=) Date: Mon, 8 Apr 2013 16:39:28 -0600 Subject: [erlang-questions] The BEAM Community was accepted in the Google Summer of Code 2013 Message-ID: Hello everyone, I am very happy to announce that the BEAM Community has been accepted into Google Summer of Code 2013! The Google Summer of Code (GSoC) is an annual program that allows students to do paid work during the summer on their Open Source Project of choice: http://www.google-melange.com/gsoc/homepage/google/gsoc2013 For this year, we have decided to create an organization called the BEAM Community that hosts different projects that run on the Erlang VM (examples: ejabberd, disco, elixir, zotonic, etc). This is a great opportunity to bring more students to our community while improving many projects! Getting accepted was just the first step and we need your help to move forward. You can help in two ways: *1. Get the word out!* Right now we need to get students interested in the BEAM Community, so let everyone know (twitter, mailing list, etc): Website: http://beamcommunity.github.io/ Project's list: https://github.com/beamcommunity/beamcommunity.github.com/wiki Students should pick their favorite projects and write proposals to work with them during the summer. *2. Apply as a mentor* If you are interested in any of the projects and you would like to participate as a mentor, reach us in the BEAM Community mailing list or the #beam-community channel on freenode! If you have any questions, please let us know! * Jos? Valim www.plataformatec.com.br Founder and Lead Developer * -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Tue Apr 9 02:47:12 2013 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 9 Apr 2013 01:47:12 +0100 (BST) Subject: [erlang-questions] Mnesia with Records Confusion In-Reply-To: Message-ID: <25446857.51292719.1365468432897.JavaMail.root@erlang-solutions.com> Mnesia uses the first record field, 'pid' in your case, as the *primary* index of a table. This is the key which is used when you do a straight read(Tab, Key). Adding an explicit {index,Keys} element to the table definition creates *extra* index tables for the table using that the fields in the Keys list. In your case 'pid''. Again. I am guessing that having a field as both the primary key and an extra key causes it to fail. So when you insert a new first field 'id' this then becomes the primary key and you extra key 'pid' is no longer the primary key so it works. Actually in this case explicitly defining the the first record field as a key is totally unnecessary as it it is already the primary key. Even if it worked. :-) And adding an extra first field 'id' just so you can can use the 'pid' field as an index and read it with index_read/3 is duplicating the work and is unnecessary. If it is as you described it. Robert ----- Original Message ----- > From: "Lee Sylvester" > To: "erlang-questions@REDACTED Questions" > Sent: Tuesday, 9 April, 2013 12:06:42 AM > Subject: [erlang-questions] Mnesia with Records Confusion > > Hey guys, > > So, I have a record that looks a little like this > > -record(user, {pid, username}). > > That's a simplified version. Anyhoo, I was trying to add it to and > remove it from a table in Mnesia, but kept getting a badarg error. > I'd set the table up like so: > > mnesia:create_table(user, > [ > {type,set}, > {record_name, user}, > {index, [pid]}, > {attributes, record_info(fields, user)} > ]), > > This error had me baffled for a while as I was able to add records to > the table, but couldn't read them with index_read. I eventually > found that the issue was to do with the index of the pid property in > the record and, by adding an id property as the first field, all my > problems disappeared. > > Can someone please explain why this is so? I mean, I'm glad it works > now, but I'd love to know why I had to do this. > > Thanks loads, > Lee > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ok@REDACTED Tue Apr 9 04:21:37 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 9 Apr 2013 14:21:37 +1200 Subject: [erlang-questions] Text crunching ... help needed In-Reply-To: <5CACC54B-7E13-4DC4-9615-2649D61D18DA@gmail.com> References: <09FE44EC-7F0C-4215-822F-D4FD4DF59CC6@cs.otago.ac.nz> <5CACC54B-7E13-4DC4-9615-2649D61D18DA@gmail.com> Message-ID: <629CA6A3-50B2-457A-B371-42F266E63378@cs.otago.ac.nz> On 9/04/2013, at 10:32 AM, Zabrane Mickael wrote: > KJB was simply choosen to simplify the problem. Maybe you can suggest a better one as long as it'll exceed +4MB size (and up to 100MB). > > My main concern now is to find out a speedy way for words detection (i.e [{offset, length} | ...]). I may have been a bit too subtle. My point is basically "there is no point in doing the wrong thing faster." I can't think of anything you might want to do with this text *as* text that will treat "(so" and "devised--namely" as *words*. I can't think of anything you might want to do with this text *as* text for which a list of {byte offset,byte length} pairs is a good data structure. I can't think of anything you might want to do with this text *as* text that doesn't involve some kind of normalisation so that at least "Yet" and "yet" are recognised as the same word. In a situation where you have a *structured* text (books, chapters, verses) it seems a little odd to completely disregard that structure. The nearest I ever came to it was "find the longest continuous passage that is the same in the AV and the Book of Mormon", but even that needed to report in book/chapter/verse form *where* that passage was. I mean, if you want to find words or phrases, distributing the books or chapters across different Erlang processes, so that each process has about the same about of text, lets you run searches in parallel. If you don't have enough *uses* of the text for that kind of thing to pay off, then it doesn't really matter how long it takes to build the index; fast enough is fast enough. So *everything* hinges on the part you are not telling us, which is "what do you actually want do *DO* with the text?" There is no point in doing the wrong thing faster. As for other texts, Project Gutenberg has over 42,000 books for free. Downloading the five Edgar Rice Burroughs "Mars" books they have as plain text gave me 2MB of data, and there are other texts in great abundance. From mangalaman93@REDACTED Mon Apr 8 17:37:00 2013 From: mangalaman93@REDACTED (aman mangal) Date: Mon, 8 Apr 2013 21:07:00 +0530 Subject: [erlang-questions] integrating nodejs with erlang code Message-ID: Hi, What is the most efficient way to pass events/data from erlang processes to nodejs and vice versa? Setting up a tcp socket may be slower or inefficient. I was thinking of doing something native. Both erlang and nodejs provide libraries(NIF) to call C functions but none of them provide an efficient implementation for calling erlang/nodejs functions from C or C++ as far as I know. Please suggest some ideas! Thanks in advance. Aman Mangal 3rd year Undergraduate Department of Computer Science & Engineering IIT Bombay www.cse.iitb.ac.in/~amanmangal -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Tue Apr 9 06:46:25 2013 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Tue, 9 Apr 2013 05:46:25 +0100 Subject: [erlang-questions] integrating nodejs with erlang code In-Reply-To: References: Message-ID: You should try the TCP implementation first. It will give you the cleanest code, and a good degree of separation/isolation between the two systems which is quite useful to have. Premature optimisation is evil. The only question you have to ask yourself is whether it is fast enough. cheers Chandru On 8 April 2013 16:37, aman mangal wrote: > > Hi, > > What is the most efficient way to pass events/data from erlang processes > to nodejs and vice versa? Setting up a tcp socket may be slower or > inefficient. I was thinking of doing something native. Both erlang and > nodejs provide libraries(NIF) to call C functions but none of them provide > an efficient implementation for calling erlang/nodejs functions from C or > C++ as far as I know. Please suggest some ideas! > > Thanks in advance. > > Aman Mangal > 3rd year Undergraduate > Department of Computer Science & Engineering > IIT Bombay > www.cse.iitb.ac.in/~amanmangal > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Tue Apr 9 08:36:24 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Tue, 9 Apr 2013 07:36:24 +0100 Subject: [erlang-questions] Mnesia with Records Confusion In-Reply-To: <25446857.51292719.1365468432897.JavaMail.root@erlang-solutions.com> References: <25446857.51292719.1365468432897.JavaMail.root@erlang-solutions.com> Message-ID: <0922DD38-77D3-4002-888F-2B80AA1EA409@gmail.com> Thank you. Now I understand. However, removing the index attribute and the I'd field causes everything to fail. I end up with the same error mentioned before. So confusing :-( Thanks, Lee On 9 Apr 2013, at 01:47, Robert Virding wrote: > Mnesia uses the first record field, 'pid' in your case, as the *primary* index of a table. This is the key which is used when you do a straight read(Tab, Key). Adding an explicit {index,Keys} element to the table definition creates *extra* index tables for the table using that the fields in the Keys list. In your case 'pid''. Again. > > I am guessing that having a field as both the primary key and an extra key causes it to fail. So when you insert a new first field 'id' this then becomes the primary key and you extra key 'pid' is no longer the primary key so it works. > > Actually in this case explicitly defining the the first record field as a key is totally unnecessary as it it is already the primary key. Even if it worked. :-) And adding an extra first field 'id' just so you can can use the 'pid' field as an index and read it with index_read/3 is duplicating the work and is unnecessary. > > If it is as you described it. > > Robert > > ----- Original Message ----- >> From: "Lee Sylvester" >> To: "erlang-questions@REDACTED Questions" >> Sent: Tuesday, 9 April, 2013 12:06:42 AM >> Subject: [erlang-questions] Mnesia with Records Confusion >> >> Hey guys, >> >> So, I have a record that looks a little like this >> >> -record(user, {pid, username}). >> >> That's a simplified version. Anyhoo, I was trying to add it to and >> remove it from a table in Mnesia, but kept getting a badarg error. >> I'd set the table up like so: >> >> mnesia:create_table(user, >> [ >> {type,set}, >> {record_name, user}, >> {index, [pid]}, >> {attributes, record_info(fields, user)} >> ]), >> >> This error had me baffled for a while as I was able to add records to >> the table, but couldn't read them with index_read. I eventually >> found that the issue was to do with the index of the pid property in >> the record and, by adding an id property as the first field, all my >> problems disappeared. >> >> Can someone please explain why this is so? I mean, I'm glad it works >> now, but I'd love to know why I had to do this. >> >> Thanks loads, >> Lee >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From robert.virding@REDACTED Tue Apr 9 10:28:27 2013 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 9 Apr 2013 09:28:27 +0100 (BST) Subject: [erlang-questions] Mnesia with Records Confusion In-Reply-To: <0922DD38-77D3-4002-888F-2B80AA1EA409@gmail.com> Message-ID: <1103932915.51952087.1365496107800.JavaMail.root@erlang-solutions.com> Are you calling read/2 or index_read/3? Robert ----- Original Message ----- > From: "Lee Sylvester" > To: "Robert Virding" > Cc: "erlang-questions@REDACTED Questions" > Sent: Tuesday, 9 April, 2013 8:36:24 AM > Subject: Re: [erlang-questions] Mnesia with Records Confusion > > Thank you. Now I understand. However, removing the index attribute > and the I'd field causes everything to fail. I end up with the same > error mentioned before. > > So confusing :-( > > Thanks, > Lee > > > On 9 Apr 2013, at 01:47, Robert Virding > wrote: > > > Mnesia uses the first record field, 'pid' in your case, as the > > *primary* index of a table. This is the key which is used when you > > do a straight read(Tab, Key). Adding an explicit {index,Keys} > > element to the table definition creates *extra* index tables for > > the table using that the fields in the Keys list. In your case > > 'pid''. Again. > > > > I am guessing that having a field as both the primary key and an > > extra key causes it to fail. So when you insert a new first field > > 'id' this then becomes the primary key and you extra key 'pid' is > > no longer the primary key so it works. > > > > Actually in this case explicitly defining the the first record > > field as a key is totally unnecessary as it it is already the > > primary key. Even if it worked. :-) And adding an extra first > > field 'id' just so you can can use the 'pid' field as an index and > > read it with index_read/3 is duplicating the work and is > > unnecessary. > > > > If it is as you described it. > > > > Robert > > > > ----- Original Message ----- > >> From: "Lee Sylvester" > >> To: "erlang-questions@REDACTED Questions" > >> > >> Sent: Tuesday, 9 April, 2013 12:06:42 AM > >> Subject: [erlang-questions] Mnesia with Records Confusion > >> > >> Hey guys, > >> > >> So, I have a record that looks a little like this > >> > >> -record(user, {pid, username}). > >> > >> That's a simplified version. Anyhoo, I was trying to add it to > >> and > >> remove it from a table in Mnesia, but kept getting a badarg error. > >> I'd set the table up like so: > >> > >> mnesia:create_table(user, > >> [ > >> {type,set}, > >> {record_name, user}, > >> {index, [pid]}, > >> {attributes, record_info(fields, user)} > >> ]), > >> > >> This error had me baffled for a while as I was able to add records > >> to > >> the table, but couldn't read them with index_read. I eventually > >> found that the issue was to do with the index of the pid property > >> in > >> the record and, by adding an id property as the first field, all > >> my > >> problems disappeared. > >> > >> Can someone please explain why this is so? I mean, I'm glad it > >> works > >> now, but I'd love to know why I had to do this. > >> > >> Thanks loads, > >> Lee > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > From erlang@REDACTED Tue Apr 9 11:34:04 2013 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 9 Apr 2013 11:34:04 +0200 Subject: [erlang-questions] How to connect android to raspberry pi Message-ID: Dear Aunty Erlang List, I would like to write a simple Android Application. What I want to do is build an Android client running on my phone talking to an Erlang server running on my raspberry pi. I know how to do the server bit on the raspberry pi, that's really really easy, but do not know how to do the Android bit. Can anybody refer me to a simple( see footnote [1]) and *minimal* [2] example code for Android that does (say) the following: - Puts a single button on the screen of my phone with the word "click" - When I press click sends a GET or POST request to a web server with a known IP address and port (this will be my raspberry pi) [1] When I say *simple* I mean: - All the code is made in emacs by editing text files - There is a makefile - There are some instructions (that work) which tell me how to move the application from my development machine to my phone What I do not want is: - flexible cross-platform responsive designs - integrated IDE cross-platform - no stupid IDE that asks me questions that I do not understand - one click deployment - powerful API with bells and whistles - built-in security, authentication ... - ... - designed for non-programmers - no programming necessary - make your first application and earn $$$ with one click [2] Minimal means as few (text) files as possible - and *with* comments I want the good old fashion hard method. Sweat blood to make it work, "read and understand *every* line of code in every file" - no dependence on libraries - write the code in emacs - automate with "make." I know I'm a Dinosaur, and I should use Eclipse, but I always get stuck at the first question when it asks me questions that I do not understand. I am seeking enlightenment, not ease of programming. Cheers /Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.james.barry@REDACTED Tue Apr 9 11:42:53 2013 From: paul.james.barry@REDACTED (Paul Barry) Date: Tue, 9 Apr 2013 10:42:53 +0100 Subject: [erlang-questions] Fwd: How to connect android to raspberry pi In-Reply-To: References: Message-ID: Forgot to CC this the Erlang list... my reply to Joe's request. ---------- Forwarded message ---------- From: Paul Barry Date: 9 April 2013 10:42 Subject: Re: [erlang-questions] How to connect android to raspberry pi To: Joe Armstrong Hi Joe. Please see my Python for Android article on Linux Journal: http://www.linuxjournal.com/article/10940 - you'll need to install the Android SDK, but not all the Eclispe stuff to get it to work. My little example app is not far off what you need/want re: communicating with a web server running on some device somewhere. Other examples are in Chapter 8 of my Head First Python book: http://shop.oreilly.com/product/0636920003434.do Main point: you can do what you need on your Android device in a couple lines of Python. :-) Hope this helps. Paul. On 9 April 2013 10:34, Joe Armstrong wrote: > Dear Aunty Erlang List, > > I would like to write a simple Android Application. > > What I want to do is build an Android client running on my phone > talking to an Erlang server running on my raspberry pi. > > I know how to do the server bit on the raspberry pi, that's really really > easy, > but do not know how to do the Android bit. > > Can anybody refer me to a simple( see footnote [1]) and *minimal* [2] > example code for Android that does (say) the following: > > - Puts a single button on the screen of my phone with the word "click" > - When I press click sends a GET or POST request to a > web server with a known IP address and port (this will be my > raspberry pi) > > [1] When I say *simple* I mean: > > - All the code is made in emacs by editing text files > - There is a makefile > - There are some instructions (that work) which tell me how to > move the application from my development machine to my phone > > What I do not want is: > > - flexible cross-platform responsive designs > - integrated IDE cross-platform > - no stupid IDE that asks me questions that I do not understand > - one click deployment > - powerful API with bells and whistles > - built-in security, authentication ... > - ... > - designed for non-programmers > - no programming necessary > - make your first application and earn $$$ with one click > > [2] Minimal means > > as few (text) files as possible - and *with* comments > > I want the good old fashion hard method. Sweat blood to make it work, > "read and understand *every* line of code in every file" - no > dependence on libraries - write the code in emacs - automate with > "make." > > I know I'm a Dinosaur, and I should use Eclipse, but I always get > stuck at the first question when it asks me questions that I do not > understand. > > I am seeking enlightenment, not ease of programming. > > Cheers > > /Joe > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul.barry@REDACTED Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. -- Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul.barry@REDACTED Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon@REDACTED Tue Apr 9 11:50:18 2013 From: jon@REDACTED (Jonathan Schneider) Date: Tue, 9 Apr 2013 10:50:18 +0100 Subject: [erlang-questions] Arbitrary old versions feature Message-ID: I've been looking into what would be required to maintain old versions of modules. It doesn't look terribly onerous. Would it be useful to work on ? Is somebody else already working on it ? Jon From kenneth.lundin@REDACTED Tue Apr 9 15:10:24 2013 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Tue, 9 Apr 2013 15:10:24 +0200 Subject: [erlang-questions] Arbitrary old versions feature In-Reply-To: References: Message-ID: Hmm? I don't understand what you mean but I would like to do that. Please explain the use case and perhaps also a possible implementation. /Kenneth Erlang/OTP, Ericsson On Tue, Apr 9, 2013 at 11:50 AM, Jonathan Schneider wrote: > I've been looking into what would be required to maintain old versions of > modules. It doesn't look terribly onerous. > > Would it be useful to work on ? Is somebody else already working on it ? > > Jon > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon@REDACTED Tue Apr 9 15:47:37 2013 From: jon@REDACTED (Jon Schneider) Date: Tue, 9 Apr 2013 14:47:37 +0100 Subject: [erlang-questions] Arbitrary old versions feature Message-ID: The (admittedly rather contrived) use case is where you have a large number of mobile clients maintaining TCP (perhaps SSL) connections. They pay a silly amount for mobile data so it would be best not to cut them off. (Obviously the client software would re-connect anyway but it's the principle here.) There are two upgrade requirements in succession. Maybe it's a workaround to deal with some nasty distributed denial of service that's just popped up (and again something I've rather dreamt up). The memory cost of keeping many copies of even large modules is insignificant by modern standards. I notice the old bits of the Module structure have changed since the source (R15 ?) I looked at earlier so my thoughts on implementation are somewhat invalid. Serves me right for not fetching latest code. Maybe whoever changed that did so as a precursor to this feature. Commit 166032bddf9a14f3ea6252724532039a1113612d specifically. Jon From simon@REDACTED Tue Apr 9 17:44:31 2013 From: simon@REDACTED (Simon MacMullen) Date: Tue, 09 Apr 2013 16:44:31 +0100 Subject: [erlang-questions] Unrecognised OID -> fail to decode printableString Message-ID: <5164375F.4080302@rabbitmq.com> With the attached certificate (provided by a user, apparently generated by "Red Hat Certificate System 7.3") and R16B: 1> rr(public_key). [...] 2> {ok, File} = file:read_file("/path/to/cert.pem"). {ok,<<"-----BEGIN CERTIFICATE-----"...>>} 3> [{'Certificate', Cert, _}] = public_key:pem_decode(File). [{'Certificate',<<48,130,3,178,...>>, not_encrypted}] 4> #'OTPCertificate'{tbsCertificate = #'OTPTBSCertificate'{subject = Subject}} = public_key:pkix_decode_cert(Cert, otp). #'OTPCertificate'{...} 5> Subject. {rdnSequence, [[#'AttributeTypeAndValue'{type = {2,5,4,6},value = "ES"}], [#'AttributeTypeAndValue'{ type = {2,5,4,10}, value = {printableString,"OMEL"}}], [#'AttributeTypeAndValue'{ type = {2,5,4,11}, value = {printableString,"OM"}}], [#'AttributeTypeAndValue'{ type = {2,5,4,3}, value = {printableString,"DSI PRUEBAS"}}], [#'AttributeTypeAndValue'{ type = {0,9,2342,19200300,100,1,1}, value = <<19,9,68,83,73,83,79,70,84,57,57>>}]]} Umm, why has the last value not been decoded? According to my very very slack knowledge of BER, 19 = printableString, 9 is the length, and the rest is the string. And decoding it by hand gives the expected result. So why does this value not get decoded? Cheers, Simon -- Simon MacMullen RabbitMQ, VMware -------------- next part -------------- -----BEGIN CERTIFICATE----- MIIDsjCCApqgAwIBAgICBr0wDQYJKoZIhvcNAQEFBQAwKzEYMBYGA1UEChMPT01F TCBQS0kgRG9tYWluMQ8wDQYDVQQDEwZvbWVsQ0EwHhcNMTMwNDA4MTAzODI1WhcN MTMwNDMwMTAzODI1WjBaMQswCQYDVQQGEwJFUzENMAsGA1UEChMET01FTDELMAkG A1UECxMCT00xFDASBgNVBAMTC0RTSSBQUlVFQkFTMRkwFwYKCZImiZPyLGQBARMJ RFNJU09GVDk5MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwQsrz7gJ cP2kxxtLu4ZozYL/FtIuFG2MqH0jSbztbSnE2ylyOo0sN/yXzeb5kM/gsN+gBgqo ox4g+2Q13bKUCy/NeCm2O1n/ETf9+H4PDE6/h6NueAv1vSBWZxhGFzAGaRVlid+P qnja+Jm4fdIl9un2oDcZE3got0KjbGwljl/NiMGlLG52J3+eUiuG+DaAq4toLAlU KWcuy266M2W1GAVqptA+kh35FNvhiBW5fQhp16hwDevY/91AdCrGu9htUYNmiDVU KVzZcijiZGDHKYFmYP4L/2Fszz5Pp1NH04kQGU9/O/HYibmhZFd7m2CaV+4RIuFv Yde5SYJCKrIG9wIDAQABo4GwMIGtMB8GA1UdIwQYMBaAFNbKR4oHY8Fw92anG0vW rEFeebR4MEAGCCsGAQUFBwEBBDQwMjAwBggrBgEFBQcwAYYkaHR0cDovL0NFUlQx LU9NLk9NRUwuRVM6OTA4MC9jYS9vY3NwMA4GA1UdDwEB/wQEAwIF4DAdBgNVHSUE FjAUBggrBgEFBQcDAgYIKwYBBQUHAwQwGQYDVR0RBBIwEIEORFNJU09GVEBQUkIu RVMwDQYJKoZIhvcNAQEFBQADggEBAHG8Yl27yGjNoZzap5PtEwpRIuBgoiFwyVDX BL1m7SbgD1bVdwEfCwM9dMZy2kSpxiDgRoh/6M3xQAhU3Z/mppOeBtckTKXWnlkH HOoE92XwOmkjuz4PSkxrinc50n/NiKPRvxt6K07NAKm+t2n/LopKkiKCYgKIgOi1 U9ONhvUBuP7tAnXjhTMqTl1bg9sNxt+ceI7ucx7NZeAtqnYlh+SrPpMVhWBAZpKs /kQ0D1VpsyM6qq0+mCCTAHQAZsCErjPb9tjsVKZfOdRAHCD4ASjqaUJos2+f2dzG P/JUhE/BewPJvWgADBBKwj/IKSLrYYYigzqLlqnmCEDt6Jtrt5Q= -----END CERTIFICATE----- From edgurgel@REDACTED Tue Apr 9 14:11:49 2013 From: edgurgel@REDACTED (Eduardo Gurgel) Date: Tue, 9 Apr 2013 09:11:49 -0300 Subject: [erlang-questions] integrating nodejs with erlang code In-Reply-To: References: Message-ID: On Mon, Apr 8, 2013 at 12:37 PM, aman mangal wrote: > > Hi, > > What is the most efficient way to pass events/data from erlang processes > to nodejs and vice versa? Setting up a tcp socket may be slower or > inefficient. I was thinking of doing something native. Both erlang and > nodejs provide libraries(NIF) to call C functions but none of them provide > an efficient implementation for calling erlang/nodejs functions from C or > C++ as far as I know. Please suggest some ideas! > > You could take a look on BERT RPC: https://github.com/rtomayko/node-bertrpc https://github.com/mojombo/bert.erl Right now, bert-rpc.org is returning 404, but it was up two days ago: http://bert-rpc.org/ Maybe checking cached webpage on bert-rpc.org may help you. -- Eduardo -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Tue Apr 9 18:31:57 2013 From: ivan@REDACTED (Ivan Uemlianin) Date: Tue, 09 Apr 2013 17:31:57 +0100 Subject: [erlang-questions] queue:split/2 unsafe! Message-ID: <5164427D.7090400@llaisdy.com> Dear All I've just discovered that queue:split(N, Queue) causes a crash if N is larger than Queue has items: 1> self(). <0.37.0> 2> Q1 = queue:new(). {[],[]} 3> Q2 = queue:in(a, Q1). {[a],[]} 4> Q3 = queue:in(b, Q2). {[b],[a]} 5> Q4 = queue:in(c, Q3). {[c,b],[a]} 6> queue:split(4, Q4). ** exception error: bad argument in function queue:split/2 called as queue:split(4,{[c,b],[a]}) 7> self(). <0.45.0> I've written a safe split: queue_safe_split(N, Q) -> case queue:len(Q) >= N of true -> queue:split(N,Q); false -> {Q, queue:new()} end. Thanks Ivan -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From mononcqc@REDACTED Tue Apr 9 19:25:16 2013 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 9 Apr 2013 13:25:16 -0400 Subject: [erlang-questions] queue:split/2 unsafe! In-Reply-To: <5164427D.7090400@llaisdy.com> References: <5164427D.7090400@llaisdy.com> Message-ID: <20130409172515.GA16954@ferdair.local> I would prefer to keep the current behaviour. Note that it's similar to the one used for different data structures, such as lists or binaries: 1> lists:split(4, [1,2,3]). ** exception error: bad argument in function lists:split/2 called as lists:split(4,[1,2,3]) 2> erlang:split_binary(<<1,2,3>>, 4). ** exception error: bad argument in function split_binary/2 called as split_binary(<<1,2,3>>,4) Going out of bound on where you want to split is, in all senses of the an error. It's not the same as say, functions where you split in two lists according to a predicate, a pattern, or a regular expression, which as far I'm aware, are the only splitting functions that allow to have empty elements one side or the other. Regards, Fred. On 04/09, Ivan Uemlianin wrote: > Dear All > > I've just discovered that queue:split(N, Queue) causes a crash if N > is larger than Queue has items: > > 1> self(). > <0.37.0> > 2> Q1 = queue:new(). > {[],[]} > 3> Q2 = queue:in(a, Q1). > {[a],[]} > 4> Q3 = queue:in(b, Q2). > {[b],[a]} > 5> Q4 = queue:in(c, Q3). > {[c,b],[a]} > 6> queue:split(4, Q4). > ** exception error: bad argument > in function queue:split/2 > called as queue:split(4,{[c,b],[a]}) > 7> self(). > <0.45.0> > > I've written a safe split: > > queue_safe_split(N, Q) -> > case queue:len(Q) >= N of > true -> > queue:split(N,Q); > false -> > {Q, queue:new()} > end. > > Thanks > > Ivan > > > -- > ============================================================ > Ivan A. Uemlianin PhD > Llaisdy > Speech Technology Research and Development > > ivan@REDACTED > www.llaisdy.com > llaisdy.wordpress.com > github.com/llaisdy > www.linkedin.com/in/ivanuemlianin > > festina lente > ============================================================ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bchesneau@REDACTED Tue Apr 9 19:44:52 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Tue, 9 Apr 2013 19:44:52 +0200 Subject: [erlang-questions] How to connect android to raspberry pi In-Reply-To: References: Message-ID: This link is probably what you're ;looking for: http://developer.android.com/tools/projects/projects-cmdline.html For emacs I found : https://code.google.com/p/android-emacs-toolkit/ I personnaly use the vim android plugin: https://github.com/bpowell/vim-android On Tue, Apr 9, 2013 at 11:34 AM, Joe Armstrong wrote: > Dear Aunty Erlang List, > > I would like to write a simple Android Application. > > What I want to do is build an Android client running on my phone > talking to an Erlang server running on my raspberry pi. > > I know how to do the server bit on the raspberry pi, that's really really > easy, > but do not know how to do the Android bit. > > Can anybody refer me to a simple( see footnote [1]) and *minimal* [2] > example code for Android that does (say) the following: > > - Puts a single button on the screen of my phone with the word "click" > - When I press click sends a GET or POST request to a > web server with a known IP address and port (this will be my > raspberry pi) > > [1] When I say *simple* I mean: > > - All the code is made in emacs by editing text files > - There is a makefile > - There are some instructions (that work) which tell me how to > move the application from my development machine to my phone > > What I do not want is: > > - flexible cross-platform responsive designs > - integrated IDE cross-platform > - no stupid IDE that asks me questions that I do not understand > - one click deployment > - powerful API with bells and whistles > - built-in security, authentication ... > - ... > - designed for non-programmers > - no programming necessary > - make your first application and earn $$$ with one click > > [2] Minimal means > > as few (text) files as possible - and *with* comments > > I want the good old fashion hard method. Sweat blood to make it work, > "read and understand *every* line of code in every file" - no > dependence on libraries - write the code in emacs - automate with > "make." > > I know I'm a Dinosaur, and I should use Eclipse, but I always get > stuck at the first question when it asks me questions that I do not > understand. > > I am seeking enlightenment, not ease of programming. > > Cheers > > /Joe > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ok@REDACTED Wed Apr 10 02:21:58 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 10 Apr 2013 12:21:58 +1200 Subject: [erlang-questions] queue:split/2 unsafe! In-Reply-To: <20130409172515.GA16954@ferdair.local> References: <5164427D.7090400@llaisdy.com> <20130409172515.GA16954@ferdair.local> Message-ID: <7AEFFA2A-05F7-4063-A421-4F37B57F38EA@cs.otago.ac.nz> There are actually two different things that someone might want to do: (X) I want *exactly* N items from this sequence: n -> seq -> (pref, suff) where pref++suff == seq && #pref == n (Y) I want *at most* N items from this sequence: n -> seq -> (pref, suff) where pref++suff == seq && #pref = min(#seq, n) If you want exactly N items and there are not N items available, that's an error. If you want at most N items and there are not N items available, that is *not* an error, it's something you had in mind when you asked. I agree with Fred Herbert that existing functions designed for use case (X) should not be changed; that guarantee that you got as much as you asked for is too precious to lose. I also agree with Ivan Uemlianin that use case (Y) is also important enough to support (think of Unix read(2)). I disagree with him that 'safe_split' is a good name for use case (Y). There are plenty of situations where (X) is the safe alternative. So why not add .. in lists.erl .. %% split_at_most(N, L) returns {P, S} %% such that L = P++S and length(P) == min(N, length(L)). %% If you need length(P) == N exactly, you must use split/2. %% This is for use when you are happy with a short result. -spec(split_at_most/2 :: (non_neg_integer(),[T]) -> {[T],[T]}). split_at_most(N, L) when is_integer(N), N >= 0, is_list(L) -> split_at_most(N, L, []). split_at_most(N, [H|T], PR) when N > 0 -> split_at_most(N-1, T, [H|PR]); split_at_most(_, S, PR) -> {reverse(PR), S}. Does anyone else find it confusing that it's "listS.erl" (plural) but "queue.erl" (singular)? Is there only one queue? %% split_at_most(N, Q) returns {P, S} %% such that P and S are queues and concatenating S after P %% gives you a queue with the same elements as Q in the same %% logical order, and length(P) == min(N, length(Q)). %% If you need length(P) == N exactly, you must use split/2. %% This is for use when you are happy with a short result. -spec(split_at_most/2 :: (non_neg_integer(),{[T],[T]}) -> {{[T],[T]},{[T],[T]}}). split_at_most(N, {Back,Front}) when is_integer(N), N >= 0, is_list(Back), is_list(Front) -> LF = length(Front), if LF >= N -> {PF, SF} = lists:split_at_most(N, Front), {f2r(PF), {Back,SF}} ; true -> {PB, SB} = lists:split_at_most(N-LF, lists:reverse(Back)), {f2r(Front++PB), f2r(SB)} end. This code has been tested. However, looking through queue.erl has convinced me, the next time I need queues, to write my own module. From mononcqc@REDACTED Wed Apr 10 03:29:03 2013 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 9 Apr 2013 21:29:03 -0400 Subject: [erlang-questions] queue:split/2 unsafe! In-Reply-To: <7AEFFA2A-05F7-4063-A421-4F37B57F38EA@cs.otago.ac.nz> References: <5164427D.7090400@llaisdy.com> <20130409172515.GA16954@ferdair.local> <7AEFFA2A-05F7-4063-A421-4F37B57F38EA@cs.otago.ac.nz> Message-ID: <20130410012902.GA50979@ferdmbp.local> On 04/10, Richard A. O'Keefe wrote: > > Does anyone else find it confusing that it's "listS.erl" (plural) > but "queue.erl" (singular)? Is there only one queue? > That kind of inconsistency is repeated in many places. "gb_trees", "dict", "array", "sets", "digraph", and so on. There just isn't much consistency when it comes to pluralization of data structures. If anything "queue" would benefit from being plural given it implements three different APIs over the same functional queue structure! > However, looking through queue.erl has convinced me, the next > time I need queues, to write my own module. > Generally speaking, I haven't had issues with the module regarding its amortized O(1) time (ignoring repeated cases that would need laziness/memoization according to Okasaki). The most annoying thing about it is possibly the reversed-spelling API (Okasaki API) which contains 'snoc' for 'cons', 'deah' for 'head', but has 'last' spelled correctly (because it was available), and contains 'lial' for 'tail', but still has the mispelling for 'lait' as a bad reversal of 'tail'. I'm guessing deprecation would make sense there. But yeah, all in all, it seems to respect the queues described by Okasaki rather well. What's your main problem with it? Regards, Fred. From felixgallo@REDACTED Tue Apr 9 22:16:47 2013 From: felixgallo@REDACTED (Felix Gallo) Date: Tue, 9 Apr 2013 13:16:47 -0700 Subject: [erlang-questions] reltool/rebar vs. ssl certificates Message-ID: I am seeing a (probably intentional) difference between the way that the ssl module resolves certificates when run by hand from a command line emulator, and the way that it resolves certificates when run via a rebar-generated start script. I would like to understand the underlying mechanism and how to change its behavior. In particular, my OTP application, which uses httpc, ssl and public_key, when run from the reltool- and rebar-generated application start script, emits: =ERROR REPORT==== 9-Apr-2013::20:20:59 === SSL: certify: ssl_handshake.erl:263:Fatal error: certificate unknown =ERROR REPORT==== 9-Apr-2013::20:20:59 === Error in process <0.77.0> on node 'message_queue_worker@REDACTED' with exit value: {{badmatch,{error,{failed_connect,[{to_address,{" go.urbanairship.com",443}},{inet,[inet],{tls_alert,"certificate unknown"}}]}}}, but when run via erl it works fine. I suspect/think/am trying to deduce that the rebar-generated start script is attempting to sandbox/clean-start the environment so that minimal system dependencies are injected, which would be a good general practice for things like rebar/reltool. However, this app will only ever be installed on unix systems with properly installed certs in the normative openssl directories so I'd like to bypass that, or work around it with proper inclusions. Unfortunately the documentation is a little sparse in this area and it doesn't appear that my situation is super common. Thanks in advance for any clues / tips. F. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tristan.sloughter@REDACTED Wed Apr 10 05:25:08 2013 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Tue, 9 Apr 2013 22:25:08 -0500 Subject: [erlang-questions] reltool/rebar vs. ssl certificates In-Reply-To: References: Message-ID: Are you sure both times, the release and running manually yourself, it is using the same version of erts? This is an error message a number of us have seen when trying to use code that worked in R15 but now gives this error in R16. Tristan On Tue, Apr 9, 2013 at 3:16 PM, Felix Gallo wrote: > I am seeing a (probably intentional) difference between the way that the > ssl module resolves certificates when run by hand from a command line > emulator, and the way that it resolves certificates when run via a > rebar-generated start script. I would like to understand the underlying > mechanism and how to change its behavior. > > In particular, my OTP application, which uses httpc, ssl and public_key, > when run from the reltool- and rebar-generated application start script, > emits: > > =ERROR REPORT==== 9-Apr-2013::20:20:59 === > SSL: certify: ssl_handshake.erl:263:Fatal error: certificate unknown > > =ERROR REPORT==== 9-Apr-2013::20:20:59 === > Error in process <0.77.0> on node 'message_queue_worker@REDACTED' with > exit value: {{badmatch,{error,{failed_connect,[{to_address,{" > go.urbanairship.com",443}},{inet,[inet],{tls_alert,"certificate > unknown"}}]}}}, > > but when run via erl it works fine. > > I suspect/think/am trying to deduce that the rebar-generated start script > is attempting to sandbox/clean-start the environment so that minimal system > dependencies are injected, which would be a good general practice for > things like rebar/reltool. However, this app will only ever be installed > on unix systems with properly installed certs in the normative openssl > directories so I'd like to bypass that, or work around it with proper > inclusions. > > Unfortunately the documentation is a little sparse in this area and it > doesn't appear that my situation is super common. > > Thanks in advance for any clues / tips. > > F. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Wed Apr 10 05:34:47 2013 From: felixgallo@REDACTED (Felix Gallo) Date: Tue, 9 Apr 2013 20:34:47 -0700 Subject: [erlang-questions] reltool/rebar vs. ssl certificates In-Reply-To: References: Message-ID: Hi Tristan -- Interesting theory. I located the 'erl' script in the generated build, and ran it directly, and it reports itself as Erlang R16B (erts-5.10.1), just like my system /usr/local/bin/erl does. However, and this is a new data point, if I run the generated build's erl, and start my application manually under that erl, then I get the unknown certificate error message, whereas if I run /usr/local/bin/erl and start my application manually, I don't. There must be some initialization or configuration option which is different. F. On Tue, Apr 9, 2013 at 8:25 PM, Tristan Sloughter < tristan.sloughter@REDACTED> wrote: > Are you sure both times, the release and running manually yourself, it is > using the same version of erts? > > This is an error message a number of us have seen when trying to use code > that worked in R15 but now gives this error in R16. > > Tristan > > > On Tue, Apr 9, 2013 at 3:16 PM, Felix Gallo wrote: > >> I am seeing a (probably intentional) difference between the way that the >> ssl module resolves certificates when run by hand from a command line >> emulator, and the way that it resolves certificates when run via a >> rebar-generated start script. I would like to understand the underlying >> mechanism and how to change its behavior. >> >> In particular, my OTP application, which uses httpc, ssl and public_key, >> when run from the reltool- and rebar-generated application start script, >> emits: >> >> =ERROR REPORT==== 9-Apr-2013::20:20:59 === >> SSL: certify: ssl_handshake.erl:263:Fatal error: certificate unknown >> >> =ERROR REPORT==== 9-Apr-2013::20:20:59 === >> Error in process <0.77.0> on node 'message_queue_worker@REDACTED' with >> exit value: {{badmatch,{error,{failed_connect,[{to_address,{" >> go.urbanairship.com",443}},{inet,[inet],{tls_alert,"certificate >> unknown"}}]}}}, >> >> but when run via erl it works fine. >> >> I suspect/think/am trying to deduce that the rebar-generated start script >> is attempting to sandbox/clean-start the environment so that minimal system >> dependencies are injected, which would be a good general practice for >> things like rebar/reltool. However, this app will only ever be installed >> on unix systems with properly installed certs in the normative openssl >> directories so I'd like to bypass that, or work around it with proper >> inclusions. >> >> Unfortunately the documentation is a little sparse in this area and it >> doesn't appear that my situation is super common. >> >> Thanks in advance for any clues / tips. >> >> F. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Apr 10 07:27:43 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 10 Apr 2013 17:27:43 +1200 Subject: [erlang-questions] queue:split/2 unsafe! In-Reply-To: <20130410012902.GA50979@ferdmbp.local> References: <5164427D.7090400@llaisdy.com> <20130409172515.GA16954@ferdair.local> <7AEFFA2A-05F7-4063-A421-4F37B57F38EA@cs.otago.ac.nz> <20130410012902.GA50979@ferdmbp.local> Message-ID: <4E865AD1-715D-4641-912B-AEFA0D3D3888@cs.otago.ac.nz> On 10/04/2013, at 1:29 PM, Fred Hebert wrote: > What's your main problem with it? I have two problems, and I think both of them boil down to the same thing. Once there was a queue.erl which _just_ implemented queues. The interface was not what I'd have made it, but it was smallish, and the underlying representation was adequate to the job. Now there is a queue.erl which provides deques, and has a mashup of at least three different interfaces. It is possible to implement deques tolerably efficiently in a functional language. However, this module still uses a data structure which is basically only adequate, and tweaks it lightly to be less inadequate for deques. But still not adequate. The claims of O(1) amortised time are, sadly, *bogus*. Try it for yourself. Make a benchmarks that creates a queue of 4N elements, and then times a loop that repeats drop(drop(drop_r(drop_r(Q)))) until the queue is empty. If the operations are O(1) amortised time, the cost should scale linearly with N. By actual measurement, the cost is O(N**2), which means that the operations are O(N) amortised time, not O(1). The code is much more complicated than code for simple queues. There are extra overheads making it slower than code for simple queues when used as simple queues. For a trivial example, converting a list to a simple queue is an O(1) operation, but queue.erl takes O(|list|) time. We're paying constant factor overheads in order to gain asymptotic performance that we don't actually _get_, for operations that, if we just want queues, we don't use. I suggest that having *separate* 'queues' and 'deques' modules would be a good idea. From erlang@REDACTED Wed Apr 10 08:40:31 2013 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 10 Apr 2013 08:40:31 +0200 Subject: [erlang-questions] ANNOUNCE: 2'nd edition of Programming Erlang now available Message-ID: The 2'nd edition of programming Erlang is now available as an e-book from http://pragprog.com/book/jaerlang2/programming-erlang Cheers /Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Apr 10 08:58:12 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 10 Apr 2013 10:58:12 +0400 Subject: [erlang-questions] =?utf-8?q?Preallocated_binary_=E2=80=94_save_m?= =?utf-8?q?emory?= Message-ID: There is a problem in erlyvideo with generating mpeg-ts. I have list of video frames, they are binaries of 4K - 100K bytes size with some additional metadata. I need to split all these binaries in chunks of about 184 bytes and create lots of such chunks. When mpeg-ts transformation is written in C, large binary is allocated (size = sum(frame_size)*C) and all frames with additional data are written to this block chunk by chunk. Only one memory allocation is done and only one copy is done. When I write it in erlang, lots (thousands) of small binaries are allocated and iolist_to_binary makes another copy with another allocation. May it be a good idea to add nif function that returns large preallocated binary and then generate such data with <> so that no reallocation is done? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Wed Apr 10 09:49:51 2013 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 10 Apr 2013 09:49:51 +0200 Subject: [erlang-questions] reltool/rebar vs. ssl certificates In-Reply-To: References: Message-ID: Hi! In R16 error handling was changed to avoid hiding information that we acctually have so you proably would get another error in R15. It however seems strange that that R15 would accept a cert that R16 does not. The error below will typically occurs when there are problems with ASN-1 or decoding or perhaps interpreation of ASN-1 decoded data. Have you tried to apply the following commit to R16 from the maint branch? 006f45a738a6612958381b2fcbf48586c008d911 Regards Ingela Erlang/OTP tema - Ericsson AB 2013/4/10, Felix Gallo : > Hi Tristan -- > > Interesting theory. I located the 'erl' script in the generated build, and > ran it directly, and it reports itself as Erlang R16B (erts-5.10.1), just > like my system /usr/local/bin/erl does. > > However, and this is a new data point, if I run the generated build's erl, > and start my application manually under that erl, then I get the unknown > certificate error message, whereas if I run /usr/local/bin/erl and start my > application manually, I don't. There must be some initialization or > configuration option which is different. > > F. > > > On Tue, Apr 9, 2013 at 8:25 PM, Tristan Sloughter < > tristan.sloughter@REDACTED> wrote: > >> Are you sure both times, the release and running manually yourself, it is >> using the same version of erts? >> >> This is an error message a number of us have seen when trying to use code >> that worked in R15 but now gives this error in R16. >> >> Tristan >> >> >> On Tue, Apr 9, 2013 at 3:16 PM, Felix Gallo wrote: >> >>> I am seeing a (probably intentional) difference between the way that the >>> ssl module resolves certificates when run by hand from a command line >>> emulator, and the way that it resolves certificates when run via a >>> rebar-generated start script. I would like to understand the underlying >>> mechanism and how to change its behavior. >>> >>> In particular, my OTP application, which uses httpc, ssl and public_key, >>> when run from the reltool- and rebar-generated application start script, >>> emits: >>> >>> =ERROR REPORT==== 9-Apr-2013::20:20:59 === >>> SSL: certify: ssl_handshake.erl:263:Fatal error: certificate unknown >>> >>> =ERROR REPORT==== 9-Apr-2013::20:20:59 === >>> Error in process <0.77.0> on node 'message_queue_worker@REDACTED' with >>> exit value: {{badmatch,{error,{failed_connect,[{to_address,{" >>> go.urbanairship.com",443}},{inet,[inet],{tls_alert,"certificate >>> unknown"}}]}}}, >>> >>> but when run via erl it works fine. >>> >>> I suspect/think/am trying to deduce that the rebar-generated start >>> script >>> is attempting to sandbox/clean-start the environment so that minimal >>> system >>> dependencies are injected, which would be a good general practice for >>> things like rebar/reltool. However, this app will only ever be >>> installed >>> on unix systems with properly installed certs in the normative openssl >>> directories so I'd like to bypass that, or work around it with proper >>> inclusions. >>> >>> Unfortunately the documentation is a little sparse in this area and it >>> doesn't appear that my situation is super common. >>> >>> Thanks in advance for any clues / tips. >>> >>> F. >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > From qoocku@REDACTED Wed Apr 10 10:04:32 2013 From: qoocku@REDACTED (=?UTF-8?B?RGFtaWFuIERvYnJvY3p5xYRza2k=?=) Date: Wed, 10 Apr 2013 10:04:32 +0200 Subject: [erlang-questions] ANNOUNCE: 2'nd edition of Programming Erlang now available In-Reply-To: References: Message-ID: <51651D10.2090106@gmail.com> Hi Joe, I've searched through available text and noticed some "maps", like the following: #{cmd ~ fill_div, id ~ clock, txt ~ current_time()} What is it, by the way? Where can I find them? -- D. W dniu 10.04.2013 08:40, Joe Armstrong pisze: > The 2'nd edition of programming Erlang is now available as an e-book from > > http://pragprog.com/book/jaerlang2/programming-erlang > > Cheers > > /Joe > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Wed Apr 10 10:13:34 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Wed, 10 Apr 2013 10:13:34 +0200 Subject: [erlang-questions] =?windows-1252?q?Preallocated_binary_=97_save_?= =?windows-1252?q?memory?= In-Reply-To: References: Message-ID: <63BF624E-847B-4363-89F5-0ECAA385E4C1@gmail.com> Hello Max, There already exists a BEAM instruction that preallocates a binary: bs_init_writable. From what I understand it is only used when compiling bit string comprehensions when their body expression and binary generators' patterns only have sized fields and they have no list generators or filters. Could you rewrite your decoder in such a bit string comprehension? Regards, -- Anthony Ramine Le 10 avr. 2013 ? 08:58, Max Lapshin a ?crit : > There is a problem in erlyvideo with generating mpeg-ts. > > I have list of video frames, they are binaries of 4K - 100K bytes size with some additional metadata. I need to split all these binaries in chunks of about 184 bytes and create lots of such chunks. > > When mpeg-ts transformation is written in C, large binary is allocated (size = sum(frame_size)*C) and all frames with additional data are written to this block chunk by chunk. > > Only one memory allocation is done and only one copy is done. > > > When I write it in erlang, lots (thousands) of small binaries are allocated and iolist_to_binary makes another copy with another allocation. > > May it be a good idea to add nif function that returns large preallocated binary and then generate such data with <> so that no reallocation is done? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ingela.andin@REDACTED Wed Apr 10 10:32:04 2013 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 10 Apr 2013 10:32:04 +0200 Subject: [erlang-questions] clustering with SSL (-proto_dist inet_tls) In-Reply-To: References: Message-ID: Hi! 2013/4/6, Kevin Carosso : > We've run into a couple of issues using R15B03-1 and enabling SSL on > Erlang distribution protocol between cluster members: > > (1) We're used to being able to restrict the range of ports used > between the cluster members via the -kernel parameters > inet_dist_listen_min and inet_dist_listen_max. > > These seem to be ignored by -proto_dist inet_tls > Is this expected? Is there an alternative mechanism for specifying > a range for coordination with firewalls? This is proably a bug we will look into it. > (2) Everytime a BEAM that's been clustered using dist inet_tls exits, it > produces what appears to be debug output to the console, of the form: > > close called #Port<0.1074> [{inet_tls_dist,close,1, > > [{file,"inet_tls_dist.erl"},{line,99}]}, > > {lists,foreach,2,[{file,"lists.erl"},{line,1262}]}, > {net_kernel,terminate,2, > > [{file,"net_kernel.erl"},{line,570}]}, > {gen_server,terminate,6, > > [{file,"gen_server.erl"},{line,722}]}, > {proc_lib,init_p_do_apply,3, > > [{file,"proc_lib.erl"},{line,227}]}] > > which is coming from inet_tls_dist.erl right here: > > close(Socket) -> > try > erlang:error(foo) > catch _:_ -> > io:format("close called ~p ~p~n",[Socket, > erlang:get_stacktrace()]) > end, > gen_tcp:close(Socket), > ok. > > Looks like left-over debug code, or similar. > Is this a bug in R15B? > We will remove that debug code thanks for alerting us. Regards Ingela Erlang/OTP team - Ericsson AB From max.lapshin@REDACTED Wed Apr 10 10:58:45 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 10 Apr 2013 12:58:45 +0400 Subject: [erlang-questions] =?utf-8?q?Preallocated_binary_=E2=80=94_save_m?= =?utf-8?q?emory?= In-Reply-To: <63BF624E-847B-4363-89F5-0ECAA385E4C1@gmail.com> References: <63BF624E-847B-4363-89F5-0ECAA385E4C1@gmail.com> Message-ID: I don't exactly understand, how to enable it. I want to find the way to hint BEAM that I'm going to fill 2 megabyte binary with small chunks of 1 to 184 bytes and I can predict size of this binary. Should I write in beam assembly for it? =)) -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Apr 10 11:08:52 2013 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 10 Apr 2013 11:08:52 +0200 Subject: [erlang-questions] ANNOUNCE: 2'nd edition of Programming Erlang now available In-Reply-To: <51651D10.2090106@gmail.com> References: <51651D10.2090106@gmail.com> Message-ID: These are the new maps that will be available in R17 of Erlang (and in pre-releases before this) The are like records only you don't have to define the keys in advance, and keys can be any valid Erlang terms. /Joe On Wed, Apr 10, 2013 at 10:04 AM, Damian Dobroczy?ski wrote: > Hi Joe, > > I've searched through available text and noticed some "maps", like the > following: > > #{cmd ~ fill_div, id ~ clock, txt ~ current_time()} > > What is it, by the way? Where can I find them? > > -- D. > > > W dniu 10.04.2013 08:40, Joe Armstrong pisze: > > The 2'nd edition of programming Erlang is now available as an e-book from > > http://pragprog.com/book/jaerlang2/programming-erlang > > Cheers > > /Joe > > > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eriksoe@REDACTED Wed Apr 10 11:12:39 2013 From: eriksoe@REDACTED (=?ISO-8859-1?Q?Erik_S=F8e_S=F8rensen?=) Date: Wed, 10 Apr 2013 11:12:39 +0200 Subject: [erlang-questions] =?windows-1252?q?Preallocated_binary_=97_save_?= =?windows-1252?q?memory?= In-Reply-To: References: Message-ID: > May it be a good idea to add nif function that returns large preallocated binary and then generate such data with <> so that no reallocation is done? How about if you do that, without preallocating, just starting with <<>>? That would at least lower the number of allocations to O(log(number_of_chunks)). And if that doesn't cut it, sure - do the same except hand-craft a BEAM function which does the preallocation... it sounds like that is possible without resorting to NIFs :-) 2013/4/10 Max Lapshin > There is a problem in erlyvideo with generating mpeg-ts. > > I have list of video frames, they are binaries of 4K - 100K bytes size > with some additional metadata. I need to split all these binaries in chunks > of about 184 bytes and create lots of such chunks. > > When mpeg-ts transformation is written in C, large binary is allocated > (size = sum(frame_size)*C) and all frames with additional data are written > to this block chunk by chunk. > > Only one memory allocation is done and only one copy is done. > > > When I write it in erlang, lots (thousands) of small binaries are > allocated and iolist_to_binary makes another copy with another allocation. > > May it be a good idea to add nif function that returns large preallocated > binary and then generate such data with <> so > that no reallocation is done? > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Apr 10 11:14:19 2013 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 10 Apr 2013 11:14:19 +0200 Subject: [erlang-questions] integrating nodejs with erlang code In-Reply-To: References: Message-ID: Just curious, but what is the *least efficient* way of connecting node.js to Erlang. If I were doing this I'd start with the least efficient method, build my app, measure, then optimize if necessary. If the optimization is not necessary you have saved yourself a whole lot of work. Alternatively, write the code and if it is not fast enough wait ten years. Cheers /Joe On Tue, Apr 9, 2013 at 2:11 PM, Eduardo Gurgel wrote: > > > > On Mon, Apr 8, 2013 at 12:37 PM, aman mangal wrote: > >> >> Hi, >> >> What is the most efficient way to pass events/data from erlang processes >> to nodejs and vice versa? Setting up a tcp socket may be slower or >> inefficient. I was thinking of doing something native. Both erlang and >> nodejs provide libraries(NIF) to call C functions but none of them provide >> an efficient implementation for calling erlang/nodejs functions from C or >> C++ as far as I know. Please suggest some ideas! >> >> > You could take a look on BERT RPC: > > https://github.com/rtomayko/node-bertrpc > > https://github.com/mojombo/bert.erl > > Right now, bert-rpc.org is returning 404, but it was up two days ago: > > http://bert-rpc.org/ > > Maybe checking cached webpage on bert-rpc.org may help you. > > -- > > Eduardo > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From olivier.boudeville@REDACTED Wed Apr 10 11:27:52 2013 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Wed, 10 Apr 2013 11:27:52 +0200 Subject: [erlang-questions] RE On node disconnections In-Reply-To: Message-ID: Hi, To answer my own question (why the loss of a remote node was making a monitoring node crash with an untrapable 'noconnection' error), it was due to the fact that this monitoring node was run with "-eval": launching the same application, this time from the Erlang shell, exhibited no crash dump, and the node loss was correctly detected. However, often an application must be run from a script (not from an interactive shell; ex: in order to run on a cluster), and apparently none of the solutions to do so with the 'init' module (namely "-eval", "-run" and "-s") allows to resist to these kinds of runtime failures. So: what could be the solution to run a VM as batch (without the interactive shell) while still being able to overcome errors (such as 'noconnection')? Thanks in advance for any hint! Best regards, Olivier. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 Olivier BOUDEVILLE/IMA/DER/EDFGDF/FR Envoy? par : Olivier BOUDEVILLE/A/EDF/FR 02/04/2013 18:08 A erlang-questions@REDACTED cc Objet On node disconnections Hi all, I had a few questions about node disconnections. Currently I have a distributed application that must resist the crash of at least some of its hosts. I test the whole feature first by using ten remote virtual machines which I software-disconnect from the user host at random moments of the application execution, thanks to a merciless 'ifconfig down' of the relevant network interface. It works great, insofar as the communications then freeze immediately (no surprise). My intent was to monitor, from a specific process on the user node, each of these 10 worker nodes (using net_kernel:monitor_nodes/2), to receive the corresponding 'nodedown' messages and (if monitoring does not prevent 'noconnection' to be triggered) then to issue a disconnect_node/1 for each of them, so that my user node can resist these losses. However, most of the time I cannot intercept the 'nodedown' information soon enough (or at all), and the whole program crashes and burns, with a message: ** Node 'N' not responding ** ** Removing (timedout) connection ** {"init terminating in do_boot",noconnection} So, my question: how can I prevent this noconnection to wreak havoc, as it seems to ruin our ability to resist node losses? If I understand well, as these reliability messages shall be managed "out of band", there is always a race condition between their receiving and the telling to all processes to stop interacting with the lost node(s). So if there were no way of at least temporarily resisting 'noconnection' (as whatever we do there *will* be processes that will attempt to send a message to a lost node), the whole purpose of the approach would be defeated. Unless I misunderstood something? A related question is that, apparently, increasing the kernel net tick time (say, from 60 to 300) does not seem to increase accordingly the noconnection time-out that must exist somewhere. As a result, I think that by design the node monitoring can only fail then (a noconnection will always happen before the monitoring messages have a chance to kick in). Thanks in advance for any hint! Best regards, Olivier. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eriksoe@REDACTED Wed Apr 10 11:28:31 2013 From: eriksoe@REDACTED (=?ISO-8859-1?Q?Erik_S=F8e_S=F8rensen?=) Date: Wed, 10 Apr 2013 11:28:31 +0200 Subject: [erlang-questions] Bounded length() Message-ID: A pro pos split_at_most(): I've at times wished for a guard function 'lists:length_at_most(L,N)' - something which determines min(length(L),N) in the non-stupid way, i.e. counting the elements of the list but stopping when it reaches N. >From time to time, I run into situations where I just need to know "is this lists 'large'?", and it feels wasteful then to measure its entire length when you know it might *be* quite large. And pattern-matching for [_,_,_,_, ... ,_|_] isn't nice (or efficient) either when the threshold length is, say, a hundred. Is this just me, or has anyone else run into such cases? 2013/4/10 Richard A. O'Keefe > There are actually two different things that someone might want to do: > > (X) I want *exactly* N items from this sequence: > n -> seq -> (pref, suff) where pref++suff == seq && #pref == n > > (Y) I want *at most* N items from this sequence: > n -> seq -> (pref, suff) where pref++suff == seq && > #pref = min(#seq, n) > [snip] -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Wed Apr 10 11:32:59 2013 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 10 Apr 2013 11:32:59 +0200 Subject: [erlang-questions] Unrecognised OID -> fail to decode printableString In-Reply-To: <5164375F.4080302@rabbitmq.com> References: <5164375F.4080302@rabbitmq.com> Message-ID: Hi! The "otp" strategy decodes as much as possible of the certificate that it recognizes it recurses and decodes "manually" more than the standard ASN-1 decoding "pkix" mode does. But proably it does not know about this oid. Maybe the ASN-1 compiler could behave differently but someone else needs to answer that. Regards Ingela Erlang/OTP team - Ericsson AB 2013/4/9, Simon MacMullen : > With the attached certificate (provided by a user, apparently generated > by "Red Hat Certificate System 7.3") and R16B: > > 1> rr(public_key). > [...] > > 2> {ok, File} = file:read_file("/path/to/cert.pem"). > {ok,<<"-----BEGIN CERTIFICATE-----"...>>} > > 3> [{'Certificate', Cert, _}] = public_key:pem_decode(File). > [{'Certificate',<<48,130,3,178,...>>, not_encrypted}] > > 4> #'OTPCertificate'{tbsCertificate = #'OTPTBSCertificate'{subject = > Subject}} = public_key:pkix_decode_cert(Cert, otp). > #'OTPCertificate'{...} > > 5> Subject. > {rdnSequence, > [[#'AttributeTypeAndValue'{type = {2,5,4,6},value = "ES"}], > [#'AttributeTypeAndValue'{ > type = {2,5,4,10}, > value = {printableString,"OMEL"}}], > [#'AttributeTypeAndValue'{ > type = {2,5,4,11}, > value = {printableString,"OM"}}], > [#'AttributeTypeAndValue'{ > type = {2,5,4,3}, > value = {printableString,"DSI PRUEBAS"}}], > [#'AttributeTypeAndValue'{ > type = {0,9,2342,19200300,100,1,1}, > value = <<19,9,68,83,73,83,79,70,84,57,57>>}]]} > > Umm, why has the last value not been decoded? According to my very very > slack knowledge of BER, 19 = printableString, 9 is the length, and the > rest is the string. And decoding it by hand gives the expected result. > > So why does this value not get decoded? > > Cheers, Simon > > -- > Simon MacMullen > RabbitMQ, VMware > From vances@REDACTED Wed Apr 10 11:34:11 2013 From: vances@REDACTED (Vance Shipley) Date: Wed, 10 Apr 2013 15:04:11 +0530 Subject: [erlang-questions] RE On node disconnections In-Reply-To: References: Message-ID: <20130410093411.GA99642@aluminium.local> On Wed, Apr 10, 2013 at 11:27:52AM +0200, Olivier BOUDEVILLE wrote: } So: what could be the solution to run a VM as batch (without the } interactive shell) while still being able to overcome errors (such as } 'noconnection')? http://www.erlang.org/doc/embedded/users_guide.html http://www.erlang.org/doc/man/run_erl.html -- -Vance From ok@REDACTED Wed Apr 10 11:35:55 2013 From: ok@REDACTED (ok@REDACTED) Date: Wed, 10 Apr 2013 21:35:55 +1200 Subject: [erlang-questions] Bounded length() In-Reply-To: References: Message-ID: <31b2e0b08a92e5e5239bde17a37ba4f4.squirrel@chasm.otago.ac.nz> > A pro pos split_at_most(): > I've at times wished for a guard function 'lists:length_at_most(L,N)' > > Is this just me, or has anyone else run into such cases? It's not just you, and for most uses of length/2 that I've seen in guards, length_at_most/2 would be a better choice, giving you O(N) testing instead of O(|L|) time. So I'd say, we don't just want that function, we want it usable in guards. From max.lapshin@REDACTED Wed Apr 10 11:43:28 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 10 Apr 2013 13:43:28 +0400 Subject: [erlang-questions] =?utf-8?q?Preallocated_binary_=E2=80=94_save_m?= =?utf-8?q?emory?= In-Reply-To: References: Message-ID: Yes, of course nif is a very bad way to solve problem =( But we are speaking about many, many useless reallocations of memory. 1 megabyte will be reallocated about 10 (?) times? Ok, I need to try it first. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Wed Apr 10 12:46:55 2013 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 10 Apr 2013 12:46:55 +0200 Subject: [erlang-questions] ANNOUNCE: 2'nd edition of Programming Erlang now available In-Reply-To: References: <51651D10.2090106@gmail.com> Message-ID: <153E7B77-6F4D-4E88-9C96-8313EEC9FF0B@feuerlabs.com> How do they sort? As tuples? BR, Ulf W On 10 Apr 2013, at 11:08, Joe Armstrong wrote: > These are the new maps that will be available in R17 of Erlang (and in pre-releases before this) > > The are like records only you don't have to define the keys in advance, and keys can be any valid > Erlang terms. > > /Joe > > > On Wed, Apr 10, 2013 at 10:04 AM, Damian Dobroczy?ski wrote: > Hi Joe, > > I've searched through available text and noticed some "maps", like the following: > > #{cmd ~ fill_div, id ~ clock, txt ~ current_time()} > > What is it, by the way? Where can I find them? > > -- D. > > > W dniu 10.04.2013 08:40, Joe Armstrong pisze: >> The 2'nd edition of programming Erlang is now available as an e-book from >> >> http://pragprog.com/book/jaerlang2/programming-erlang >> >> Cheers >> >> /Joe >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From egil@REDACTED Wed Apr 10 12:56:49 2013 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Wed, 10 Apr 2013 12:56:49 +0200 Subject: [erlang-questions] ANNOUNCE: 2'nd edition of Programming Erlang now available In-Reply-To: <153E7B77-6F4D-4E88-9C96-8313EEC9FF0B@feuerlabs.com> References: <51651D10.2090106@gmail.com> <153E7B77-6F4D-4E88-9C96-8313EEC9FF0B@feuerlabs.com> Message-ID: <51654571.7060601@erlang.org> On 2013-04-10 12:46, Ulf Wiger wrote: > > How do they sort? > > As tuples? Yes, first by #associations, then by keys, then by vaues in key order. Much like frames (and thus tuples), i.e. like {{K1, K2, .., Kn}, {V1,V2, .., Vn}}. EEP will be out shortly. I've been saying within a week for a couple of weeks now. // Bj?rn-Egil > > BR, > Ulf W > > On 10 Apr 2013, at 11:08, Joe Armstrong wrote: > >> These are the new maps that will be available in R17 of Erlang (and >> in pre-releases before this) >> >> The are like records only you don't have to define the keys in >> advance, and keys can be any valid >> Erlang terms. >> >> /Joe >> >> >> On Wed, Apr 10, 2013 at 10:04 AM, Damian Dobroczyn'ski >> > wrote: >> >> Hi Joe, >> >> I've searched through available text and noticed some "maps", >> like the following: >> >> #{cmd ~ fill_div, id ~ clock, txt ~ current_time()} >> >> What is it, by the way? Where can I find them? >> >> -- D. >> >> >> W dniu 10.04.2013 08:40, Joe Armstrong pisze: >>> The 2'nd edition of programming Erlang is now available as an >>> e-book from >>> >>> http://pragprog.com/book/jaerlang2/programming-erlang >>> >>> Cheers >>> >>> /Joe >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Wed Apr 10 14:40:23 2013 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 10 Apr 2013 14:40:23 +0200 Subject: [erlang-questions] ANNOUNCE: 2'nd edition of Programming Erlang now available In-Reply-To: <51654571.7060601@erlang.org> References: <51651D10.2090106@gmail.com> <153E7B77-6F4D-4E88-9C96-8313EEC9FF0B@feuerlabs.com> <51654571.7060601@erlang.org> Message-ID: <2B98C27F-090B-4CAC-818A-4166C56E47DE@feuerlabs.com> One of my reasons for asking is that I started thinking about how to modify sext so that it can support maps in a backward-compatible way. In my infinite stupidity, I chose adjacent tags for the different data types, which makes it quite hard to squeeze in a new data type in the middle (since the tags must be in the order corresponding to the ordering between types). BR, Ulf W On 10 Apr 2013, at 12:56, Bj?rn-Egil Dahlberg wrote: > On 2013-04-10 12:46, Ulf Wiger wrote: >> >> How do they sort? >> >> As tuples? > Yes, first by #associations, then by keys, then by vaues in key order. > > Much like frames (and thus tuples), i.e. like {{K1, K2, .., Kn}, {V1,V2, .., Vn}}. > > EEP will be out shortly. I've been saying within a week for a couple of weeks now. > > // Bj?rn-Egil > > >> >> BR, >> Ulf W >> >> On 10 Apr 2013, at 11:08, Joe Armstrong wrote: >> >>> These are the new maps that will be available in R17 of Erlang (and in pre-releases before this) >>> >>> The are like records only you don't have to define the keys in advance, and keys can be any valid >>> Erlang terms. >>> >>> /Joe >>> >>> >>> On Wed, Apr 10, 2013 at 10:04 AM, Damian Dobroczy?ski wrote: >>> Hi Joe, >>> >>> I've searched through available text and noticed some "maps", like the following: >>> >>> #{cmd ~ fill_div, id ~ clock, txt ~ current_time()} >>> >>> What is it, by the way? Where can I find them? >>> >>> -- D. >>> >>> >>> W dniu 10.04.2013 08:40, Joe Armstrong pisze: >>>> The 2'nd edition of programming Erlang is now available as an e-book from >>>> >>>> http://pragprog.com/book/jaerlang2/programming-erlang >>>> >>>> Cheers >>>> >>>> /Joe >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >> http://feuerlabs.com >> >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Wed Apr 10 14:51:34 2013 From: ivan@REDACTED (Ivan Uemlianin) Date: Wed, 10 Apr 2013 13:51:34 +0100 Subject: [erlang-questions] queue:split/2 unsafe! In-Reply-To: <7AEFFA2A-05F7-4063-A421-4F37B57F38EA@cs.otago.ac.nz> References: <5164427D.7090400@llaisdy.com> <20130409172515.GA16954@ferdair.local> <7AEFFA2A-05F7-4063-A421-4F37B57F38EA@cs.otago.ac.nz> Message-ID: <51656056.4040003@llaisdy.com> Dear All Thanks for your comments. (Y) is exactly my use case. You are right 'split_at_most' is a better name. Best wishes Ivan On 10/04/2013 01:21, Richard A. O'Keefe wrote: > There are actually two different things that someone might want to do: > > (X) I want *exactly* N items from this sequence: > n -> seq -> (pref, suff) where pref++suff == seq && #pref == n > > (Y) I want *at most* N items from this sequence: > n -> seq -> (pref, suff) where pref++suff == seq && > #pref = min(#seq, n) > > If you want exactly N items and there are not N items available, > that's an error. > If you want at most N items and there are not N items available, > that is *not* an error, it's something you had in mind when you asked. > > I agree with Fred Herbert that existing functions designed for use > case (X) should not be changed; that guarantee that you got as much > as you asked for is too precious to lose. > > I also agree with Ivan Uemlianin that use case (Y) is also important > enough to support (think of Unix read(2)). > > I disagree with him that 'safe_split' is a good name for use case (Y). > There are plenty of situations where (X) is the safe alternative. > > So why not add > > .. in lists.erl .. > > %% split_at_most(N, L) returns {P, S} > %% such that L = P++S and length(P) == min(N, length(L)). > %% If you need length(P) == N exactly, you must use split/2. > %% This is for use when you are happy with a short result. > > -spec(split_at_most/2 :: (non_neg_integer(),[T]) -> {[T],[T]}). > > split_at_most(N, L) when is_integer(N), N >= 0, is_list(L) -> > split_at_most(N, L, []). > > split_at_most(N, [H|T], PR) when N > 0 -> > split_at_most(N-1, T, [H|PR]); > split_at_most(_, S, PR) -> > {reverse(PR), S}. > > Does anyone else find it confusing that it's "listS.erl" (plural) > but "queue.erl" (singular)? Is there only one queue? > > %% split_at_most(N, Q) returns {P, S} > %% such that P and S are queues and concatenating S after P > %% gives you a queue with the same elements as Q in the same > %% logical order, and length(P) == min(N, length(Q)). > %% If you need length(P) == N exactly, you must use split/2. > %% This is for use when you are happy with a short result. > > -spec(split_at_most/2 :: (non_neg_integer(),{[T],[T]}) -> > {{[T],[T]},{[T],[T]}}). > > split_at_most(N, {Back,Front}) > when is_integer(N), N >= 0, is_list(Back), is_list(Front) -> > LF = length(Front), > if LF >= N -> > {PF, SF} = lists:split_at_most(N, Front), > {f2r(PF), {Back,SF}} > ; true -> > {PB, SB} = lists:split_at_most(N-LF, lists:reverse(Back)), > {f2r(Front++PB), f2r(SB)} > end. > > This code has been tested. > > However, looking through queue.erl has convinced me, the next > time I need queues, to write my own module. > -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From pablo.platt@REDACTED Wed Apr 10 14:59:07 2013 From: pablo.platt@REDACTED (pablo platt) Date: Wed, 10 Apr 2013 15:59:07 +0300 Subject: [erlang-questions] Using the half-word emulator with the erlang-solutions ubuntu package Message-ID: Hi, The page on the erlang-solutions website says that the packages support the half-word emulator: https://www.erlang-solutions.com/downloads "These have included HiPE enabled, 32 and 64 bit configurations, alongside the half-word emulator" I see [hipe] when starting the shell so half-word is probably not enabled. I'm not using hipe so there is no problem disabling it. Do I need to set a flag when installing the deb package? Do I need to use a flag when starting erlang or creating a release with rebar in the vm.args? Are there reasons not to use half-word besides the 4GB process memory limit? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From olivier.boudeville@REDACTED Wed Apr 10 15:38:26 2013 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Wed, 10 Apr 2013 15:38:26 +0200 Subject: [erlang-questions] RE On node disconnections In-Reply-To: <20130410093411.GA99642@aluminium.local> Message-ID: Hi Vance, Thanks for your answer! Indeed by running: run_erl -daemon /tmp/my-pipe . "exec erl [...]" Then waiting for the pipes to be created, then running: echo "my_module:my_function()." >> /tmp/my-pipe.w I could run a code that resists 'no_connection' exceptions and does not need the user to type anything in the shell. Adding a "tail -f erlang.log.1" pretty recreates what the 'init' module gives out-of-the-box (but with an exception-safe VM). A minor question: in run-erl.log a line tells "Client expected on FIFO /tmp/my-pipe.r, but can't open"; I assume this is not a real problem? I would have imagined that I could read this pipe (with a tail or a cat) to monitor the program console outputs, possibly instead of relying on erlang.log.1 for that. Maybe run_erl prevents that. Actually a tee on this pipe could have been useful, as for longer outputs the newer logs will be written in erlang.log.2, which makes a console monitoring tricky ("tail -f erlang.log.1" having to become "tail -f erlang.log.2" etc.) Thanks again, Best regards, Olivier. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 vances@REDACTED 10/04/2013 11:34 A olivier.boudeville@REDACTED cc erlang-questions@REDACTED Objet Re: [erlang-questions] RE On node disconnections On Wed, Apr 10, 2013 at 11:27:52AM +0200, Olivier BOUDEVILLE wrote: } So: what could be the solution to run a VM as batch (without the } interactive shell) while still being able to overcome errors (such as } 'noconnection')? http://www.erlang.org/doc/embedded/users_guide.html http://www.erlang.org/doc/man/run_erl.html -- -Vance Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ddosia@REDACTED Wed Apr 10 16:11:29 2013 From: ddosia@REDACTED (Daniil Churikov) Date: Wed, 10 Apr 2013 07:11:29 -0700 (PDT) Subject: [erlang-questions] simulate a transaction In-Reply-To: References: Message-ID: <0269860b-e8f0-46ca-a861-6cdfd9d8e2de@googlegroups.com> We use riak since 0.14 version, till now and learn one thing: if you need transactions on riak, take postgres and dont try to invent it over riak -------------- next part -------------- An HTML attachment was scrubbed... URL: From calleja.justin@REDACTED Wed Apr 10 16:41:41 2013 From: calleja.justin@REDACTED (Justin Calleja) Date: Wed, 10 Apr 2013 16:41:41 +0200 Subject: [erlang-questions] Adding new operators? In-Reply-To: References: <2DCB80CB-8712-4A53-91AB-76643E84E7D1@nexttolast.com> <5B558EDF-A921-475C-B33A-B2F41C5045A2@nexttolast.com> Message-ID: Hi David, You should be able to do that with a parse_transform/2. Look up parse transform: you basically get to implement a function which the compiler calls at compilation time passing it a formal representation of the source code it's compiling (a.k.a abstract forms... though I'm not entirely sure about the terminology). You can then use the erl_syntax module to manipulate these forms. cheers, Justin On 8 April 2013 08:21, Anthony Ramine wrote: > Hello David, > > A fancy way to extend the Erlang syntax would be to have something like > Camlp4, a powerful and extensible preprocessor for OCaml. It has a powerful > quotation system which it itself uses to implements its parser construct, > to extend OCaml's syntax at compile-time. I have began writing a quotation > system for Erlang but didn't have the time finish it yet [1]. > > Regards, > > [1] https://github.com/nox/otp/tree/brackets > > -- > Anthony Ramine > > Le 8 avr. 2013 ? 02:12, David Goehrig a ?crit : > > > Next step is integrating into the stack better. But the question > remains, what is the best way to add a new operator. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Apr 10 17:18:15 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 10 Apr 2013 19:18:15 +0400 Subject: [erlang-questions] =?utf-8?q?Preallocated_binary_=E2=80=94_save_m?= =?utf-8?q?emory?= In-Reply-To: References: Message-ID: How can I reproduce that bs_init_writable? I'm trying to make a smallest example of erlang code, make assembly out of it, try to change it and compile back. Currently all my tests show that binary is reallocated. -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Wed Apr 10 17:46:34 2013 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 10 Apr 2013 16:46:34 +0100 (BST) Subject: [erlang-questions] integrating nodejs with erlang code In-Reply-To: Message-ID: <1492815702.54665218.1365608794384.JavaMail.root@erlang-solutions.com> The most efficient way of connecting Erlang with node.js would be, of course, to implement JS and hence node.js in Erlang. Maybe not the easiest way, but definitely the most efficient connection. :-) Robert ----- Original Message ----- > From: "Joe Armstrong" > To: "Eduardo Gurgel" > Cc: "Ali Attarwala" , "Shailesh Tyagi" > , "wncc iitb" , > "Erlang" , "chirag jain" > > Sent: Wednesday, 10 April, 2013 11:14:19 AM > Subject: Re: [erlang-questions] integrating nodejs with erlang code > Just curious, but what is the *least efficient* way of connecting > node.js to Erlang. > If I were doing this I'd start with the least efficient method, build > my app, measure, then optimize if > necessary. > If the optimization is not necessary you have saved yourself a whole > lot of work. > Alternatively, write the code and if it is not fast enough wait ten > years. > Cheers > /Joe > On Tue, Apr 9, 2013 at 2:11 PM, Eduardo Gurgel < edgurgel@REDACTED > > wrote: > > On Mon, Apr 8, 2013 at 12:37 PM, aman mangal < > > mangalaman93@REDACTED > > > wrote: > > > > Hi, > > > > > > What is the most efficient way to pass events/data from erlang > > > processes to nodejs and vice versa? Setting up a tcp socket may > > > be > > > slower or inefficient. I was thinking of doing something native. > > > Both erlang and nodejs provide libraries(NIF) to call C functions > > > but none of them provide an efficient implementation for calling > > > erlang/nodejs functions from C or C++ as far as I know. Please > > > suggest some ideas! > > > > > You could take a look on BERT RPC: > > > https://github.com/rtomayko/node-bertrpc > > > https://github.com/mojombo/bert.erl > > > Right now, bert-rpc.org is returning 404, but it was up two days > > ago: > > > http://bert-rpc.org/ > > > Maybe checking cached webpage on bert-rpc.org may help you. > > > -- > > > Eduardo > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From hd2010@REDACTED Wed Apr 10 18:23:36 2013 From: hd2010@REDACTED (Henning Diedrich) Date: Wed, 10 Apr 2013 18:23:36 +0200 Subject: [erlang-questions] integrating nodejs with erlang code In-Reply-To: <1492815702.54665218.1365608794384.JavaMail.root@erlang-solutions.com> References: <1492815702.54665218.1365608794384.JavaMail.root@erlang-solutions.com> Message-ID: <81F4AA36-5E19-438D-982B-036FEC84C549@eonblast.com> Right AND it's certainly the *least efficient*, TOO, as seen from the Python programmer's time mantra! So it's both in one. Satisfies Joe's AND Robert's recommendations. Could you object against that advise? Cheers, Henning On Apr 10, 2013, at 5:46 PM, Robert Virding wrote: > The most efficient way of connecting Erlang with node.js would be, of course, to implement JS and hence node.js in Erlang. Maybe not the easiest way, but definitely the most efficient connection. :-) > > Robert > > From: "Joe Armstrong" > To: "Eduardo Gurgel" > Cc: "Ali Attarwala" , "Shailesh Tyagi" , "wncc iitb" , "Erlang" , "chirag jain" > Sent: Wednesday, 10 April, 2013 11:14:19 AM > Subject: Re: [erlang-questions] integrating nodejs with erlang code > > > Just curious, but what is the *least efficient* way of connecting node.js to Erlang. > > If I were doing this I'd start with the least efficient method, build my app, measure, then optimize if > necessary. > > If the optimization is not necessary you have saved yourself a whole lot of work. > > Alternatively, write the code and if it is not fast enough wait ten years. > > > Cheers > > /Joe > > > > > On Tue, Apr 9, 2013 at 2:11 PM, Eduardo Gurgel wrote: > > > > On Mon, Apr 8, 2013 at 12:37 PM, aman mangal wrote: > > Hi, > > What is the most efficient way to pass events/data from erlang processes to nodejs and vice versa? Setting up a tcp socket may be slower or inefficient. I was thinking of doing something native. Both erlang and nodejs provide libraries(NIF) to call C functions but none of them provide an efficient implementation for calling erlang/nodejs functions from C or C++ as far as I know. Please suggest some ideas! > > > You could take a look on BERT RPC: > > https://github.com/rtomayko/node-bertrpc > > https://github.com/mojombo/bert.erl > > Right now, bert-rpc.org is returning 404, but it was up two days ago: > > http://bert-rpc.org/ > > Maybe checking cached webpage on bert-rpc.org may help you. > > -- > > Eduardo > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Wed Apr 10 18:29:28 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Wed, 10 Apr 2013 18:29:28 +0200 Subject: [erlang-questions] =?windows-1252?q?Preallocated_binary_=97_save_?= =?windows-1252?q?memory?= In-Reply-To: References: Message-ID: The following is an example of binary comprehension which is optimized to use a preallocated binary. $ cat bc.erl -module(bc). -compile(export_all). bc(Input) -> << <<0, 1, 2, Bin:42/binary>> || <> <= Input >>. $ erlc +to_core bc.erl Look at the generated bc.core file, you don't need to tweak the assembly directly as the mutable primops are introduced when compiling to Erlang Code. Basically, it uses bs_init_writable instead of bs_init and bs_private_append instead of bs_append and if it can, it uses a sensible size for the initial buffer. You can see what these opcodes mean on Erlang on Xen's website [1]. Regards, [1] http://erlangonxen.org/more/beam -- Anthony Ramine Le 10 avr. 2013 ? 17:18, Max Lapshin a ?crit : > How can I reproduce that bs_init_writable? > > I'm trying to make a smallest example of erlang code, make assembly out of it, try to change it and compile back. > > Currently all my tests show that binary is reallocated. From n.oxyde@REDACTED Wed Apr 10 18:31:28 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Wed, 10 Apr 2013 18:31:28 +0200 Subject: [erlang-questions] Adding new operators? In-Reply-To: References: <2DCB80CB-8712-4A53-91AB-76643E84E7D1@nexttolast.com> <5B558EDF-A921-475C-B33A-B2F41C5045A2@nexttolast.com> Message-ID: That wouldn't work because you need one new token per operator you want to add to the language, parse transforms work on abstract syntax trees which are returned by the parser, a level higher than tokens. On the other hand he can also abuse existing expressions by using them in ways which are illegal in Erlang. For example he could write a parse transform for a parallel send operator like this: +[Pids/parallel] ! Message. This is beyond ugly but at least it's quite obvious that some spooky action at the distance is going on. Regards, -- Anthony Ramine Le 10 avr. 2013 ? 16:41, Justin Calleja a ?crit : > Hi David, > > You should be able to do that with a parse_transform/2. Look up parse transform: you basically get to implement a function which the compiler calls at compilation time passing it a formal representation of the source code it's compiling (a.k.a abstract forms... though I'm not entirely sure about the terminology). > > You can then use the erl_syntax module to manipulate these forms. > > cheers, > Justin > > > On 8 April 2013 08:21, Anthony Ramine wrote: > Hello David, > > A fancy way to extend the Erlang syntax would be to have something like Camlp4, a powerful and extensible preprocessor for OCaml. It has a powerful quotation system which it itself uses to implements its parser construct, to extend OCaml's syntax at compile-time. I have began writing a quotation system for Erlang but didn't have the time finish it yet [1]. > > Regards, > > [1] https://github.com/nox/otp/tree/brackets > > -- > Anthony Ramine > > Le 8 avr. 2013 ? 02:12, David Goehrig a ?crit : > > > Next step is integrating into the stack better. But the question remains, what is the best way to add a new operator. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From simon@REDACTED Wed Apr 10 18:40:52 2013 From: simon@REDACTED (Simon MacMullen) Date: Wed, 10 Apr 2013 17:40:52 +0100 Subject: [erlang-questions] Unrecognised OID -> fail to decode printableString In-Reply-To: References: <5164375F.4080302@rabbitmq.com> Message-ID: <51659614.7030507@rabbitmq.com> Hi, thanks. It doesn't know about this OID, certainly (maybe it should, I don't think it's that uncommon?) But I thought ASN.1 was self-describing enough that it could continue to recurse and unpack even in the face of unrecognised OIDs? Cheers, Simon On 10/04/13 10:32, Ingela Andin wrote: > Hi! > > The "otp" strategy decodes as much as possible of the certificate that > it recognizes > it recurses and decodes "manually" more than the standard ASN-1 > decoding "pkix" mode does. But proably it does not know about this > oid. Maybe the ASN-1 compiler could > behave differently but someone else needs to answer that. > > Regards Ingela Erlang/OTP team - Ericsson AB > > > 2013/4/9, Simon MacMullen : >> With the attached certificate (provided by a user, apparently generated >> by "Red Hat Certificate System 7.3") and R16B: >> >> 1> rr(public_key). >> [...] >> >> 2> {ok, File} = file:read_file("/path/to/cert.pem"). >> {ok,<<"-----BEGIN CERTIFICATE-----"...>>} >> >> 3> [{'Certificate', Cert, _}] = public_key:pem_decode(File). >> [{'Certificate',<<48,130,3,178,...>>, not_encrypted}] >> >> 4> #'OTPCertificate'{tbsCertificate = #'OTPTBSCertificate'{subject = >> Subject}} = public_key:pkix_decode_cert(Cert, otp). >> #'OTPCertificate'{...} >> >> 5> Subject. >> {rdnSequence, >> [[#'AttributeTypeAndValue'{type = {2,5,4,6},value = "ES"}], >> [#'AttributeTypeAndValue'{ >> type = {2,5,4,10}, >> value = {printableString,"OMEL"}}], >> [#'AttributeTypeAndValue'{ >> type = {2,5,4,11}, >> value = {printableString,"OM"}}], >> [#'AttributeTypeAndValue'{ >> type = {2,5,4,3}, >> value = {printableString,"DSI PRUEBAS"}}], >> [#'AttributeTypeAndValue'{ >> type = {0,9,2342,19200300,100,1,1}, >> value = <<19,9,68,83,73,83,79,70,84,57,57>>}]]} >> >> Umm, why has the last value not been decoded? According to my very very >> slack knowledge of BER, 19 = printableString, 9 is the length, and the >> rest is the string. And decoding it by hand gives the expected result. >> >> So why does this value not get decoded? >> >> Cheers, Simon >> >> -- >> Simon MacMullen >> RabbitMQ, VMware >> -- Simon MacMullen RabbitMQ, VMware From max.lapshin@REDACTED Wed Apr 10 18:44:20 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 10 Apr 2013 20:44:20 +0400 Subject: [erlang-questions] =?utf-8?q?Preallocated_binary_=E2=80=94_save_m?= =?utf-8?q?emory?= In-Reply-To: References: Message-ID: Oh, I was looking at erlc +S, will check core erlang, thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbothma@REDACTED Wed Apr 10 19:00:09 2013 From: jbothma@REDACTED (JD Bothma) Date: Wed, 10 Apr 2013 19:00:09 +0200 Subject: [erlang-questions] simulate a transaction In-Reply-To: References: Message-ID: how much guarantee do you need that the rollback happens and when do you need that to be the case? Do you need ACID or do you just want to clean up most of the time, eventually, unless several things fail around the same time and it's ok to forget to clean up sometimes? For certain data models and usage patterns something similar to meshup's riak integration could be useful. However, I agree with Daniil - if you want a global consistent view, just go with something made for that. On 4 April 2013 02:58, Brisa Jim?nez wrote: > I want to simulate a transaction. > I want to know when a riak operation happens. > I know that riak isn't a relational database, but it is important for > me in > particular cases to undo operations when something goes wrong. > I'm reading about how to use dbg library to trace functions, I hope > that it > will be useful to use inside a module. > If any have suggestions about I want to do, I will be very grateful. > Regards. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From witeman.g@REDACTED Wed Apr 10 19:12:54 2013 From: witeman.g@REDACTED (Zheng Zhibin) Date: Thu, 11 Apr 2013 01:12:54 +0800 Subject: [erlang-questions] integrating nodejs with erlang code In-Reply-To: <1492815702.54665218.1365608794384.JavaMail.root@erlang-solutions.com> References: <1492815702.54665218.1365608794384.JavaMail.root@erlang-solutions.com> Message-ID: beam.js would be one way to build node.js into Erlang :) Cheers Ben ? 2013-4-10???11:46?Robert Virding ??? > The most efficient way of connecting Erlang with node.js would be, of course, to implement JS and hence node.js in Erlang. Maybe not the easiest way, but definitely the most efficient connection. :-) > > Robert > > From: "Joe Armstrong" > To: "Eduardo Gurgel" > Cc: "Ali Attarwala" , "Shailesh Tyagi" , "wncc iitb" , "Erlang" , "chirag jain" > Sent: Wednesday, 10 April, 2013 11:14:19 AM > Subject: Re: [erlang-questions] integrating nodejs with erlang code > > > Just curious, but what is the *least efficient* way of connecting node.js to Erlang. > > If I were doing this I'd start with the least efficient method, build my app, measure, then optimize if > necessary. > > If the optimization is not necessary you have saved yourself a whole lot of work. > > Alternatively, write the code and if it is not fast enough wait ten years. > > > Cheers > > /Joe > > > > > On Tue, Apr 9, 2013 at 2:11 PM, Eduardo Gurgel wrote: >> >> >> >> On Mon, Apr 8, 2013 at 12:37 PM, aman mangal wrote: >>> >>> Hi, >>> >>> What is the most efficient way to pass events/data from erlang processes to nodejs and vice versa? Setting up a tcp socket may be slower or inefficient. I was thinking of doing something native. Both erlang and nodejs provide libraries(NIF) to call C functions but none of them provide an efficient implementation for calling erlang/nodejs functions from C or C++ as far as I know. Please suggest some ideas! >> >> You could take a look on BERT RPC: >> >> https://github.com/rtomayko/node-bertrpc >> >> https://github.com/mojombo/bert.erl >> >> Right now, bert-rpc.org is returning 404, but it was up two days ago: >> >> http://bert-rpc.org/ >> >> Maybe checking cached webpage on bert-rpc.org may help you. >> >> -- >> >> Eduardo >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ddosia@REDACTED Wed Apr 10 19:58:27 2013 From: ddosia@REDACTED (Daniil Churikov) Date: Wed, 10 Apr 2013 10:58:27 -0700 (PDT) Subject: [erlang-questions] simulate a transaction In-Reply-To: References: Message-ID: Also in certain situations at first glance you think that you need transactions, but you can: a) scratch your data model to place more information into single object (depends on how you fetch your data, denormalization is your friend) b) some data schemes with additional design could eventually heal them selfs in case of previous errors (e.g on next write previous rubbish cleaned up from data) -------------- next part -------------- An HTML attachment was scrubbed... URL: From konrad.gadek@REDACTED Wed Apr 10 20:36:48 2013 From: konrad.gadek@REDACTED (Konrad =?utf-8?Q?G=C4=85dek?=) Date: Wed, 10 Apr 2013 19:36:48 +0100 (BST) Subject: [erlang-questions] Using the half-word emulator with the erlang-solutions ubuntu package In-Reply-To: Message-ID: <771586514.54915014.1365619008849.JavaMail.root@erlang-solutions.com> Hi, As far as I'm aware, we don't have half-word emulator enabled in our packages, so this seems like an erroneous statement on ESL website -- I'll investigate it more tomorrow. Both disabling HiPE and enabling half-word emulator are done while building packages. Some more info: http://stackoverflow.com/a/1185490/547223 /Konrad G?dek ----- Original Message ----- From: "pablo platt" To: erlang-questions@REDACTED Sent: Wednesday, April 10, 2013 2:59:07 PM Subject: [erlang-questions] Using the half-word emulator with the erlang-solutions ubuntu package Hi, The page on the erlang-solutions website says that the packages support the half-word emulator: https://www.erlang-solutions.com/downloads "These have included HiPE enabled, 32 and 64 bit configurations, alongside the half-word emulator" I see [hipe] when starting the shell so half-word is probably not enabled. I'm not using hipe so there is no problem disabling it. Do I need to set a flag when installing the deb package? Do I need to use a flag when starting erlang or creating a release with rebar in the vm.args? Are there reasons not to use half-word besides the 4GB process memory limit? Thanks _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From mangalaman93@REDACTED Wed Apr 10 20:49:03 2013 From: mangalaman93@REDACTED (aman mangal) Date: Thu, 11 Apr 2013 00:19:03 +0530 Subject: [erlang-questions] integrating nodejs with erlang code In-Reply-To: References: <1492815702.54665218.1365608794384.JavaMail.root@erlang-solutions.com> Message-ID: Hi, How can I go about implementing JS in erlang? This seems interesting to me. Please tell me what do you mean by implementing JS or node.js in erlang? Aman Mangal 3rd year Undergraduate Department of Computer Science & Engineering IIT Bombay www.cse.iitb.ac.in/~amanmangal On Wed, Apr 10, 2013 at 10:42 PM, Zheng Zhibin wrote: > beam.js would be one way to build node.js into Erlang :) > > Cheers > Ben > > ? 2013-4-10???11:46?Robert Virding > ??? > > The most efficient way of connecting Erlang with node.js would be, of > course, to implement JS and hence node.js in Erlang. Maybe not the easiest > way, but definitely the most efficient connection. :-) > > Robert > > ------------------------------ > > *From: *"Joe Armstrong" > *To: *"Eduardo Gurgel" > *Cc: *"Ali Attarwala" , "Shailesh Tyagi" < > shailesh@REDACTED>, "wncc iitb" , "Erlang" > , "chirag jain" > *Sent: *Wednesday, 10 April, 2013 11:14:19 AM > *Subject: *Re: [erlang-questions] integrating nodejs with erlang code > > > Just curious, but what is the *least efficient* way of connecting node.js > to Erlang. > > If I were doing this I'd start with the least efficient method, build my > app, measure, then optimize if > necessary. > > If the optimization is not necessary you have saved yourself a whole lot > of work. > > Alternatively, write the code and if it is not fast enough wait ten years. > > > Cheers > > /Joe > > > > > On Tue, Apr 9, 2013 at 2:11 PM, Eduardo Gurgel wrote: > >> >> >> >> On Mon, Apr 8, 2013 at 12:37 PM, aman mangal wrote: >> >>> >>> Hi, >>> >>> What is the most efficient way to pass events/data from erlang processes >>> to nodejs and vice versa? Setting up a tcp socket may be slower or >>> inefficient. I was thinking of doing something native. Both erlang and >>> nodejs provide libraries(NIF) to call C functions but none of them provide >>> an efficient implementation for calling erlang/nodejs functions from C or >>> C++ as far as I know. Please suggest some ideas! >>> >>> >> You could take a look on BERT RPC: >> >> https://github.com/rtomayko/node-bertrpc >> >> https://github.com/mojombo/bert.erl >> >> Right now, bert-rpc.org is returning 404, but it was up two days ago: >> >> http://bert-rpc.org/ >> >> Maybe checking cached webpage on bert-rpc.org may help you. >> >> -- >> >> Eduardo >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dizzyd@REDACTED Wed Apr 10 20:56:05 2013 From: dizzyd@REDACTED (Dave Smith) Date: Wed, 10 Apr 2013 12:56:05 -0600 Subject: [erlang-questions] integrating nodejs with erlang code In-Reply-To: References: Message-ID: There is "most efficient" and "pragmatic". My "pragmatic" suggestion would be to run nodejs as a port (e.g. open_port) and use stdio to interact with it. Then you can let Erlang supervise a pool of nodejs VMs. If you want to be fancy, you can implement a ping/pong protocol for detecting when the nodejs system has gone off the deep end chasing down endless chains of callbacks. :) I may or may not have had to do this recently. :) D. On Mon, Apr 8, 2013 at 9:37 AM, aman mangal wrote: > > Hi, > > What is the most efficient way to pass events/data from erlang processes > to nodejs and vice versa? Setting up a tcp socket may be slower or > inefficient. I was thinking of doing something native. Both erlang and > nodejs provide libraries(NIF) to call C functions but none of them provide > an efficient implementation for calling erlang/nodejs functions from C or > C++ as far as I know. Please suggest some ideas! > > Thanks in advance. > > Aman Mangal > 3rd year Undergraduate > Department of Computer Science & Engineering > IIT Bombay > www.cse.iitb.ac.in/~amanmangal > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Apr 10 20:59:40 2013 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 10 Apr 2013 20:59:40 +0200 Subject: [erlang-questions] erlounge in nyc Message-ID: Hello, Somebody, and I forget who asked me, and I forget when, to drop them a line if I was every in New York. I'll be in New York from 10 - 14 sept, if anybody is interested in a meetup. /Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablo.platt@REDACTED Wed Apr 10 21:02:48 2013 From: pablo.platt@REDACTED (pablo platt) Date: Wed, 10 Apr 2013 22:02:48 +0300 Subject: [erlang-questions] Using the half-word emulator with the erlang-solutions ubuntu package In-Reply-To: <771586514.54915014.1365619008849.JavaMail.root@erlang-solutions.com> References: <771586514.54915014.1365619008849.JavaMail.root@erlang-solutions.com> Message-ID: Is it possible to add a 64bit with half-word enabled and HiPE disabled package in addition to the 32bit and 64bit packages? Is there a chance HiPE and half-word will work together in a future release? Thank you for the packages. They are extremely helpful. On Wed, Apr 10, 2013 at 9:36 PM, Konrad G?dek < konrad.gadek@REDACTED> wrote: > Hi, > > As far as I'm aware, we don't have half-word emulator enabled in our > packages, so this seems like an erroneous statement on ESL website -- I'll > investigate it more tomorrow. > > Both disabling HiPE and enabling half-word emulator are done while > building packages. Some more info: > http://stackoverflow.com/a/1185490/547223 > > /Konrad G?dek > > ----- Original Message ----- > From: "pablo platt" > To: erlang-questions@REDACTED > Sent: Wednesday, April 10, 2013 2:59:07 PM > Subject: [erlang-questions] Using the half-word emulator with the > erlang-solutions ubuntu package > > > > > > > > Hi, > > The page on the erlang-solutions website says that the packages support > the half-word emulator: > https://www.erlang-solutions.com/downloads > "These have included HiPE enabled, 32 and 64 bit configurations, alongside > the half-word emulator" > > I see [hipe] when starting the shell so half-word is probably not enabled. > I'm not using hipe so there is no problem disabling it. > > Do I need to set a flag when installing the deb package? > Do I need to use a flag when starting erlang or creating a release with > rebar in the vm.args? > > > Are there reasons not to use half-word besides the 4GB process memory > limit? > > > > Thanks > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hd2010@REDACTED Wed Apr 10 21:07:58 2013 From: hd2010@REDACTED (Henning Diedrich) Date: Wed, 10 Apr 2013 21:07:58 +0200 Subject: [erlang-questions] integrating nodejs with erlang code In-Reply-To: References: Message-ID: <74DA1156-8225-46F4-A74D-5A76BBEA91BB@eonblast.com> Maybe use that tip, Aman, if that's what Dave found to make sense. I understood Robert as half-ways joking. Implementing other languages in Erlang is what he does himself, for the fun of it. See LFE and Luerl. And Joe was illustrating the lesson that you should never optimize more than you need, what's more: not before you know what (more) you need. Henning On Apr 10, 2013, at 8:56 PM, Dave Smith wrote: > There is "most efficient" and "pragmatic". > > My "pragmatic" suggestion would be to run nodejs as a port (e.g. open_port) and use stdio to interact with it. Then you can let Erlang supervise a pool of nodejs VMs. If you want to be fancy, you can implement a ping/pong protocol for detecting when the nodejs system has gone off the deep end chasing down endless chains of callbacks. :) > > I may or may not have had to do this recently. :) > > D. > > > On Mon, Apr 8, 2013 at 9:37 AM, aman mangal wrote: > > Hi, > > What is the most efficient way to pass events/data from erlang processes to nodejs and vice versa? Setting up a tcp socket may be slower or inefficient. I was thinking of doing something native. Both erlang and nodejs provide libraries(NIF) to call C functions but none of them provide an efficient implementation for calling erlang/nodejs functions from C or C++ as far as I know. Please suggest some ideas! > > Thanks in advance. > > Aman Mangal > 3rd year Undergraduate > Department of Computer Science & Engineering > IIT Bombay > www.cse.iitb.ac.in/~amanmangal > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From siraaj@REDACTED Wed Apr 10 21:12:40 2013 From: siraaj@REDACTED (Siraaj Khandkar) Date: Wed, 10 Apr 2013 15:12:40 -0400 Subject: [erlang-questions] erlounge in nyc In-Reply-To: References: Message-ID: <5165B9A8.6000601@khandkar.net> On 04/10/2013 02:59 PM, Joe Armstrong wrote: > Hello, > > Somebody, and I forget who asked me, and I forget when, to drop them a line > if I was every in New York. > > I'll be in New York from 10 - 14 sept, if anybody is interested in a meetup. > Awesome! I'll have the red carpet ready! :) From erlang@REDACTED Wed Apr 10 21:17:57 2013 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 10 Apr 2013 21:17:57 +0200 Subject: [erlang-questions] integrating nodejs with erlang code In-Reply-To: <74DA1156-8225-46F4-A74D-5A76BBEA91BB@eonblast.com> References: <74DA1156-8225-46F4-A74D-5A76BBEA91BB@eonblast.com> Message-ID: Robert never jokes about implementing languages :-) /Joe On Wed, Apr 10, 2013 at 9:07 PM, Henning Diedrich wrote: > Maybe use that tip, Aman, if that's what Dave found to make sense. > > I understood Robert as half-ways joking. Implementing other languages in > Erlang is what he does himself, for the fun of it. See LFE and Luerl. > > And Joe was illustrating the lesson that you should never optimize more > than you need, what's more: not before you know what (more) you need. > > Henning > > > On Apr 10, 2013, at 8:56 PM, Dave Smith wrote: > > There is "most efficient" and "pragmatic". > > My "pragmatic" suggestion would be to run nodejs as a port (e.g. > open_port) and use stdio to interact with it. Then you can let Erlang > supervise a pool of nodejs VMs. If you want to be fancy, you can implement > a ping/pong protocol for detecting when the nodejs system has gone off the > deep end chasing down endless chains of callbacks. :) > > I may or may not have had to do this recently. :) > > D. > > > On Mon, Apr 8, 2013 at 9:37 AM, aman mangal wrote: > >> >> Hi, >> >> What is the most efficient way to pass events/data from erlang processes >> to nodejs and vice versa? Setting up a tcp socket may be slower or >> inefficient. I was thinking of doing something native. Both erlang and >> nodejs provide libraries(NIF) to call C functions but none of them provide >> an efficient implementation for calling erlang/nodejs functions from C or >> C++ as far as I know. Please suggest some ideas! >> >> Thanks in advance. >> >> Aman Mangal >> 3rd year Undergraduate >> Department of Computer Science & Engineering >> IIT Bombay >> www.cse.iitb.ac.in/~amanmangal >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Wed Apr 10 21:29:04 2013 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 10 Apr 2013 21:29:04 +0200 Subject: [erlang-questions] integrating nodejs with erlang code In-Reply-To: <74DA1156-8225-46F4-A74D-5A76BBEA91BB@eonblast.com> References: <74DA1156-8225-46F4-A74D-5A76BBEA91BB@eonblast.com> Message-ID: Just to qualify my earlier remark. The first step in implementing any language is the parser for that language. This is usually one of the trickiest parts (except for LISP, and forth :-) - the problem is most languages are designed to be easy(ish) to write, which means they have horrible grammars and a odd mix of techniques to actually parse the stuff. Getting a parse tree for real languages that accepts the entire language (and not a subset) is pretty difficult. The next bit - writing an interpretor for the abstract syntax is far easier - and you'll often get a long way with a subset. Cheers /Joe On Wed, Apr 10, 2013 at 9:07 PM, Henning Diedrich wrote: > Maybe use that tip, Aman, if that's what Dave found to make sense. > > I understood Robert as half-ways joking. Implementing other languages in > Erlang is what he does himself, for the fun of it. See LFE and Luerl. > > And Joe was illustrating the lesson that you should never optimize more > than you need, what's more: not before you know what (more) you need. > > Henning > > > On Apr 10, 2013, at 8:56 PM, Dave Smith wrote: > > There is "most efficient" and "pragmatic". > > My "pragmatic" suggestion would be to run nodejs as a port (e.g. > open_port) and use stdio to interact with it. Then you can let Erlang > supervise a pool of nodejs VMs. If you want to be fancy, you can implement > a ping/pong protocol for detecting when the nodejs system has gone off the > deep end chasing down endless chains of callbacks. :) > > I may or may not have had to do this recently. :) > > D. > > > On Mon, Apr 8, 2013 at 9:37 AM, aman mangal wrote: > >> >> Hi, >> >> What is the most efficient way to pass events/data from erlang processes >> to nodejs and vice versa? Setting up a tcp socket may be slower or >> inefficient. I was thinking of doing something native. Both erlang and >> nodejs provide libraries(NIF) to call C functions but none of them provide >> an efficient implementation for calling erlang/nodejs functions from C or >> C++ as far as I know. Please suggest some ideas! >> >> Thanks in advance. >> >> Aman Mangal >> 3rd year Undergraduate >> Department of Computer Science & Engineering >> IIT Bombay >> www.cse.iitb.ac.in/~amanmangal >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Apr 10 21:59:06 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 10 Apr 2013 23:59:06 +0400 Subject: [erlang-questions] =?utf-8?q?Preallocated_binary_=E2=80=94_save_m?= =?utf-8?q?emory?= In-Reply-To: References: Message-ID: Ok, I can tell about first results: I made packing two times faster by switching whole algorithm from making dozens of small chunks to appending to binary. Just plain binary with reallocating. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dizzyd@REDACTED Wed Apr 10 22:43:01 2013 From: dizzyd@REDACTED (Dave Smith) Date: Wed, 10 Apr 2013 14:43:01 -0600 Subject: [erlang-questions] MySQL driver Message-ID: Hello list, I'm curious what libraries people are currently using for MySQL access in Erlang. I have an old library on my github that I've kept basically working, but am wondering if there are any preferred/production libraries out there that people like better. I ask because I'm considering retrofitting the pooling system in the driver I maintain to use poolboy, but figure it was worth checking if there are better alternatives out there. :) After all, the best code to maintain is the code you don't maintain. D. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremy@REDACTED Wed Apr 10 22:46:07 2013 From: jeremy@REDACTED (Jeremy Ong) Date: Wed, 10 Apr 2013 13:46:07 -0700 Subject: [erlang-questions] MySQL driver In-Reply-To: References: Message-ID: I've been using emysql https://github.com/Eonblast/Emysql which has connection pools and supports storing and executing procedures in addition to arbitrary sql queries. On Wed, Apr 10, 2013 at 1:43 PM, Dave Smith wrote: > Hello list, > > I'm curious what libraries people are currently using for MySQL access in > Erlang. I have an old library on my github that I've kept basically > working, but am wondering if there are any preferred/production libraries > out there that people like better. > > I ask because I'm considering retrofitting the pooling system in the > driver I maintain to use poolboy, but figure it was worth checking if there > are better alternatives out there. :) > > After all, the best code to maintain is the code you don't maintain. > > D. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Wed Apr 10 22:49:54 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Wed, 10 Apr 2013 23:49:54 +0300 Subject: [erlang-questions] MySQL driver In-Reply-To: References: Message-ID: Hello, Thanks for asking? I am using your library at production with small enhancement, which is part of pull request. There are couple of minor concerns and issues with that library, especially in area of connection supervision but in-general is good. Best Regards, Dmitry On Apr 10, 2013, at 11:43 PM, Dave Smith wrote: > Hello list, > > I'm curious what libraries people are currently using for MySQL access in Erlang. I have an old library on my github that I've kept basically working, but am wondering if there are any preferred/production libraries out there that people like better. > > I ask because I'm considering retrofitting the pooling system in the driver I maintain to use poolboy, but figure it was worth checking if there are better alternatives out there. :) > > After all, the best code to maintain is the code you don't maintain. > > D. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Wed Apr 10 22:50:12 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 11 Apr 2013 00:50:12 +0400 Subject: [erlang-questions] =?utf-8?q?Preallocated_binary_=E2=80=94_save_m?= =?utf-8?q?emory?= In-Reply-To: References: Message-ID: I've looked at erlc -S bc.erl: {function, bc, 1, 2}. {label,1}. {line,[{location,"bc.erl",3}]}. {func_info,{atom,bc},{atom,bc},1}. {label,2}. {allocate_zero,1,1}. {line,[{location,"bc.erl",3}]}. {gc_bif,bit_size,{f,0},1,[{x,0}],{x,1}}. {line,[]}. {gc_bif,'div',{f,0},2,[{x,1},{integer,336}],{x,1}}. {line,[]}. {gc_bif,'*',{f,0},2,[{x,1},{integer,45}],{x,1}}. {move,{x,0},{y,0}}. {move,{x,1},{x,0}}. bs_init_writable. {move,{x,0},{x,1}}. {move,{y,0},{x,0}}. {call_last,2,{f,8},1}. It is really cool! Erlang understands that resulting binary will be exactly 45 * (size(Input) div 42) bytes and preallocates it. This is a very important reason to use list and binary comprehensions. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hd2010@REDACTED Wed Apr 10 23:00:50 2013 From: hd2010@REDACTED (Henning Diedrich) Date: Wed, 10 Apr 2013 23:00:50 +0200 Subject: [erlang-questions] MySQL driver In-Reply-To: References: Message-ID: <51E1C638-0C5F-4C7F-AB32-7A51123378E8@eonblast.com> Hi Dave, if I remember right, you were using the driver that came from Yxa via Yarif's erlang-emysql-driver https://github.com/dizzyd/erlang-mysql-driver ? (Here is the history of that. https://github.com/eonblast/Emysql#History) It was found less than perfect when Nick and Jacob gave it something to chew building Rupture. I never spotted the problems myself I should add. That's purely from Nick Gerakine's and Jacob Vorreuter's words. But they report that with much effort they could not get the races out of it. That's why they started emysql, which I later started maintaining, and which Jeremy just mentioned, https://github.com/eonblast/Emysql Considerable work went into getting the pooling system of Emysql right. (What I don't know is when exactly it got buggy, it may have been after Jacob's Emysql fork that now lies dormant.) But I worked a while, with help from others to get all races out of the queuing. However, the best fork of Emysql, I *think* is currently that of Chris Rempel at https://github.com/csrl/emysql as I got behind while making Erlvolt, the VoltDB driver, modeled in parts on Emysql. Chris did some impressive clean up. Now that Erlvolt is out, I'll turn around and catch up asap. But that's not a New York asap. An alternative may be to go back to a more original Yxa version, like the one possibly still to be found in ejabberd. That may be more stable than Yariv's branch that you started out with. The "history" I linked to above has the reasons, why that branch may be not quite clean and hard to debug. Best, Henning On Apr 10, 2013, at 10:43 PM, Dave Smith wrote: > Hello list, > > I'm curious what libraries people are currently using for MySQL access in Erlang. I have an old library on my github that I've kept basically working, but am wondering if there are any preferred/production libraries out there that people like better. > > I ask because I'm considering retrofitting the pooling system in the driver I maintain to use poolboy, but figure it was worth checking if there are better alternatives out there. :) > > After all, the best code to maintain is the code you don't maintain. > > D. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dizzyd@REDACTED Wed Apr 10 23:59:20 2013 From: dizzyd@REDACTED (Dave Smith) Date: Wed, 10 Apr 2013 15:59:20 -0600 Subject: [erlang-questions] MySQL driver In-Reply-To: <51E1C638-0C5F-4C7F-AB32-7A51123378E8@eonblast.com> References: <51E1C638-0C5F-4C7F-AB32-7A51123378E8@eonblast.com> Message-ID: Hi Henning, Thanks for the pointers. I must admit that I find the level of snark in the emysql history delightful, considering that it's at least as forked as erlang-mysql-driver now. :) There are a number of good fixes for the emysql driver, but they are on 20 different forks across Github. Looking over csrl's, fixes, I'm pretty concerned those aren't in mainline since there are some non-trivial pooling problems he's addressed (would you like some irony with that irony? :)). Do you know if eonblast is planning to incorporate all the work from those different forks? I think we need to build some momentum behind a driver and I'm certainly happy to direct people towards a maintained one. IMO, it would also be handy if the driver wasn't pooled, but used poolboy since getting the pooling right is rather difficult. (I know that poolboy has been tested with some of Quviq's concurrent checking stuff and a number of race conditions addressed). Thanks! D. On Wed, Apr 10, 2013 at 3:00 PM, Henning Diedrich wrote: > Hi Dave, > > if I remember right, you were using the driver that came from Yxa via > Yarif's erlang-emysql-driver https://github.com/dizzyd/erlang-mysql-driver? (Here is the history of that. > https://github.com/eonblast/Emysql#History) > > It was found less than perfect when Nick and Jacob gave it something to > chew building Rupture. I never spotted the problems myself I should add. > That's purely from Nick Gerakine's and Jacob Vorreuter's words. > > But they report that with much effort they could not get the races out of > it. That's why they started emysql, which I later started maintaining, and > which Jeremy just mentioned, https://github.com/eonblast/Emysql > > Considerable work went into getting the pooling system of Emysql right. > (What I don't know is when exactly it got buggy, it may have been after > Jacob's Emysql fork that now lies dormant.) But I worked a while, with help > from others to get all races out of the queuing. > > However, the best fork of Emysql, I *think* is currently that of Chris > Rempel at https://github.com/csrl/emysql as I got behind while making > Erlvolt, the VoltDB driver, modeled in parts on Emysql. Chris did some > impressive clean up. Now that Erlvolt is out, I'll turn around and catch up > asap. But that's not a New York asap. > > An alternative may be to go back to a more original Yxa version, like the > one possibly still to be found in ejabberd. That may be more stable than > Yariv's branch that you started out with. The "history" I linked to above > has the reasons, why that branch may be not quite clean and hard to debug. > > Best, > > Henning > > On Apr 10, 2013, at 10:43 PM, Dave Smith wrote: > > > Hello list, > > > > I'm curious what libraries people are currently using for MySQL access > in Erlang. I have an old library on my github that I've kept basically > working, but am wondering if there are any preferred/production libraries > out there that people like better. > > > > I ask because I'm considering retrofitting the pooling system in the > driver I maintain to use poolboy, but figure it was worth checking if there > are better alternatives out there. :) > > > > After all, the best code to maintain is the code you don't maintain. > > > > D. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hd2010@REDACTED Thu Apr 11 00:21:40 2013 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 11 Apr 2013 00:21:40 +0200 Subject: [erlang-questions] MySQL driver In-Reply-To: References: <51E1C638-0C5F-4C7F-AB32-7A51123378E8@eonblast.com> Message-ID: <5F319A2C-95E0-448C-A48F-E6575347D21C@eonblast.com> Hi Dave, that would be great if you'd join the effort there with the driver! As I said, I can't wait to bring the mainline up to date, and I'll do it, it was delayed by the focus on Erlvolt recently. (Please check it out: https://github.com/Eonblast/Erlvolt -- http://blog.voltdb.com/877000-tps-with-erlang-and-voltdb/) I'll take a look at poolboy. However, I've gone through the pool again and again, and so did some people smarter than me, I think it's solid. What I came to believe about erlang-mysql-driver was that the concerns of its three different layers had been violated on the way and that will have made it hard to spot errors. I also have a good orientation about the Emysql forks as they are and which ones are more valuable, as preparation for catching up. It's quite some work there to re-unite them, but additions and corrections are minor, with the exception of two competing approaches to transactions, where I'd love to get your opinion. Yes it grew to be the same "convincing case for the github Network graphs" :-D Henning On Apr 10, 2013, at 11:59 PM, Dave Smith wrote: > Hi Henning, > > Thanks for the pointers. I must admit that I find the level of snark in the emysql history delightful, considering that it's at least as forked as erlang-mysql-driver now. :) > > There are a number of good fixes for the emysql driver, but they are on 20 different forks across Github. Looking over csrl's, fixes, I'm pretty concerned those aren't in mainline since there are some non-trivial pooling problems he's addressed (would you like some irony with that irony? :)). > > Do you know if eonblast is planning to incorporate all the work from those different forks? I think we need to build some momentum behind a driver and I'm certainly happy to direct people towards a maintained one. > > IMO, it would also be handy if the driver wasn't pooled, but used poolboy since getting the pooling right is rather difficult. (I know that poolboy has been tested with some of Quviq's concurrent checking stuff and a number of race conditions addressed). > > Thanks! > > D. > > > On Wed, Apr 10, 2013 at 3:00 PM, Henning Diedrich wrote: > Hi Dave, > > if I remember right, you were using the driver that came from Yxa via Yarif's erlang-emysql-driver https://github.com/dizzyd/erlang-mysql-driver ? (Here is the history of that. https://github.com/eonblast/Emysql#History) > > It was found less than perfect when Nick and Jacob gave it something to chew building Rupture. I never spotted the problems myself I should add. That's purely from Nick Gerakine's and Jacob Vorreuter's words. > > But they report that with much effort they could not get the races out of it. That's why they started emysql, which I later started maintaining, and which Jeremy just mentioned, https://github.com/eonblast/Emysql > > Considerable work went into getting the pooling system of Emysql right. (What I don't know is when exactly it got buggy, it may have been after Jacob's Emysql fork that now lies dormant.) But I worked a while, with help from others to get all races out of the queuing. > > However, the best fork of Emysql, I *think* is currently that of Chris Rempel at https://github.com/csrl/emysql as I got behind while making Erlvolt, the VoltDB driver, modeled in parts on Emysql. Chris did some impressive clean up. Now that Erlvolt is out, I'll turn around and catch up asap. But that's not a New York asap. > > An alternative may be to go back to a more original Yxa version, like the one possibly still to be found in ejabberd. That may be more stable than Yariv's branch that you started out with. The "history" I linked to above has the reasons, why that branch may be not quite clean and hard to debug. > > Best, > > Henning > > On Apr 10, 2013, at 10:43 PM, Dave Smith wrote: > > > Hello list, > > > > I'm curious what libraries people are currently using for MySQL access in Erlang. I have an old library on my github that I've kept basically working, but am wondering if there are any preferred/production libraries out there that people like better. > > > > I ask because I'm considering retrofitting the pooling system in the driver I maintain to use poolboy, but figure it was worth checking if there are better alternatives out there. :) > > > > After all, the best code to maintain is the code you don't maintain. > > > > D. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Thu Apr 11 00:25:45 2013 From: g@REDACTED (Garrett Smith) Date: Wed, 10 Apr 2013 17:25:45 -0500 Subject: [erlang-questions] MySQL driver In-Reply-To: References: <51E1C638-0C5F-4C7F-AB32-7A51123378E8@eonblast.com> Message-ID: Someday someone will write a book on the history of the Erlang MySQL libraries. Not a great read, but it'd have to be scholarly. I've modified the Eonblast library with some connection timeout tweaks (as in, supporting them): https://github.com/gar1t/Emysql On Wed, Apr 10, 2013 at 4:59 PM, Dave Smith wrote: > Hi Henning, > > Thanks for the pointers. I must admit that I find the level of snark in the > emysql history delightful, considering that it's at least as forked as > erlang-mysql-driver now. :) > > There are a number of good fixes for the emysql driver, but they are on 20 > different forks across Github. Looking over csrl's, fixes, I'm pretty > concerned those aren't in mainline since there are some non-trivial pooling > problems he's addressed (would you like some irony with that irony? :)). > > Do you know if eonblast is planning to incorporate all the work from those > different forks? I think we need to build some momentum behind a driver and > I'm certainly happy to direct people towards a maintained one. > > IMO, it would also be handy if the driver wasn't pooled, but used poolboy > since getting the pooling right is rather difficult. (I know that poolboy > has been tested with some of Quviq's concurrent checking stuff and a number > of race conditions addressed). > > Thanks! > > D. > > > On Wed, Apr 10, 2013 at 3:00 PM, Henning Diedrich > wrote: >> >> Hi Dave, >> >> if I remember right, you were using the driver that came from Yxa via >> Yarif's erlang-emysql-driver https://github.com/dizzyd/erlang-mysql-driver ? >> (Here is the history of that. https://github.com/eonblast/Emysql#History) >> >> It was found less than perfect when Nick and Jacob gave it something to >> chew building Rupture. I never spotted the problems myself I should add. >> That's purely from Nick Gerakine's and Jacob Vorreuter's words. >> >> But they report that with much effort they could not get the races out of >> it. That's why they started emysql, which I later started maintaining, and >> which Jeremy just mentioned, https://github.com/eonblast/Emysql >> >> Considerable work went into getting the pooling system of Emysql right. >> (What I don't know is when exactly it got buggy, it may have been after >> Jacob's Emysql fork that now lies dormant.) But I worked a while, with help >> from others to get all races out of the queuing. >> >> However, the best fork of Emysql, I *think* is currently that of Chris >> Rempel at https://github.com/csrl/emysql as I got behind while making >> Erlvolt, the VoltDB driver, modeled in parts on Emysql. Chris did some >> impressive clean up. Now that Erlvolt is out, I'll turn around and catch up >> asap. But that's not a New York asap. >> >> An alternative may be to go back to a more original Yxa version, like the >> one possibly still to be found in ejabberd. That may be more stable than >> Yariv's branch that you started out with. The "history" I linked to above >> has the reasons, why that branch may be not quite clean and hard to debug. >> >> Best, >> >> Henning >> >> On Apr 10, 2013, at 10:43 PM, Dave Smith wrote: >> >> > Hello list, >> > >> > I'm curious what libraries people are currently using for MySQL access >> > in Erlang. I have an old library on my github that I've kept basically >> > working, but am wondering if there are any preferred/production libraries >> > out there that people like better. >> > >> > I ask because I'm considering retrofitting the pooling system in the >> > driver I maintain to use poolboy, but figure it was worth checking if there >> > are better alternatives out there. :) >> > >> > After all, the best code to maintain is the code you don't maintain. >> > >> > D. >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From dizzyd@REDACTED Thu Apr 11 00:41:10 2013 From: dizzyd@REDACTED (Dave Smith) Date: Wed, 10 Apr 2013 16:41:10 -0600 Subject: [erlang-questions] MySQL driver In-Reply-To: References: <51E1C638-0C5F-4C7F-AB32-7A51123378E8@eonblast.com> Message-ID: Re: pooling -- there's always bugs...ALWAYS BUGS. We were using poolboy at Basho and thought that it was so simple/basic that bugs should be easy to sort out...until Dr. Hughes walked into the room and pulled a Concurrent Debuggerator 3000 from his black bag (aka QuickCheck) and found 5 significant concurrency bugs in as many minutes...and this was on a library that only does pooling and nothing else. So...unless you tell me that Dr. Hughes has been over this pooling code with the aforementioned tool, I will eye the pooling code with a raised eyebrow. :) Broadly, I would suggest looking at these forks as being useful to merge in: csrl - pooling fixes, edge cases, etc. (underscores my point about pooling, but I digress) jlarky - looks like a opscode branch that has some good bug fixes taybin - adds better UTF support (it's a big patch though, so probably needs some work) rflynn - bug fix for float encoding gar1t - bug fixes + polish on API What are the competing approaches on transactions? D. On Wed, Apr 10, 2013 at 4:25 PM, Garrett Smith wrote: > Someday someone will write a book on the history of the Erlang MySQL > libraries. Not a great read, but it'd have to be scholarly. > > I've modified the Eonblast library with some connection timeout tweaks > (as in, supporting them): > > https://github.com/gar1t/Emysql > > > > On Wed, Apr 10, 2013 at 4:59 PM, Dave Smith wrote: > > Hi Henning, > > > > Thanks for the pointers. I must admit that I find the level of snark in > the > > emysql history delightful, considering that it's at least as forked as > > erlang-mysql-driver now. :) > > > > There are a number of good fixes for the emysql driver, but they are on > 20 > > different forks across Github. Looking over csrl's, fixes, I'm pretty > > concerned those aren't in mainline since there are some non-trivial > pooling > > problems he's addressed (would you like some irony with that irony? :)). > > > > Do you know if eonblast is planning to incorporate all the work from > those > > different forks? I think we need to build some momentum behind a driver > and > > I'm certainly happy to direct people towards a maintained one. > > > > IMO, it would also be handy if the driver wasn't pooled, but used poolboy > > since getting the pooling right is rather difficult. (I know that poolboy > > has been tested with some of Quviq's concurrent checking stuff and a > number > > of race conditions addressed). > > > > Thanks! > > > > D. > > > > > > On Wed, Apr 10, 2013 at 3:00 PM, Henning Diedrich > > wrote: > >> > >> Hi Dave, > >> > >> if I remember right, you were using the driver that came from Yxa via > >> Yarif's erlang-emysql-driver > https://github.com/dizzyd/erlang-mysql-driver ? > >> (Here is the history of that. > https://github.com/eonblast/Emysql#History) > >> > >> It was found less than perfect when Nick and Jacob gave it something to > >> chew building Rupture. I never spotted the problems myself I should add. > >> That's purely from Nick Gerakine's and Jacob Vorreuter's words. > >> > >> But they report that with much effort they could not get the races out > of > >> it. That's why they started emysql, which I later started maintaining, > and > >> which Jeremy just mentioned, https://github.com/eonblast/Emysql > >> > >> Considerable work went into getting the pooling system of Emysql right. > >> (What I don't know is when exactly it got buggy, it may have been after > >> Jacob's Emysql fork that now lies dormant.) But I worked a while, with > help > >> from others to get all races out of the queuing. > >> > >> However, the best fork of Emysql, I *think* is currently that of Chris > >> Rempel at https://github.com/csrl/emysql as I got behind while making > >> Erlvolt, the VoltDB driver, modeled in parts on Emysql. Chris did some > >> impressive clean up. Now that Erlvolt is out, I'll turn around and > catch up > >> asap. But that's not a New York asap. > >> > >> An alternative may be to go back to a more original Yxa version, like > the > >> one possibly still to be found in ejabberd. That may be more stable than > >> Yariv's branch that you started out with. The "history" I linked to > above > >> has the reasons, why that branch may be not quite clean and hard to > debug. > >> > >> Best, > >> > >> Henning > >> > >> On Apr 10, 2013, at 10:43 PM, Dave Smith wrote: > >> > >> > Hello list, > >> > > >> > I'm curious what libraries people are currently using for MySQL access > >> > in Erlang. I have an old library on my github that I've kept basically > >> > working, but am wondering if there are any preferred/production > libraries > >> > out there that people like better. > >> > > >> > I ask because I'm considering retrofitting the pooling system in the > >> > driver I maintain to use poolboy, but figure it was worth checking if > there > >> > are better alternatives out there. :) > >> > > >> > After all, the best code to maintain is the code you don't maintain. > >> > > >> > D. > >> > _______________________________________________ > >> > erlang-questions mailing list > >> > erlang-questions@REDACTED > >> > http://erlang.org/mailman/listinfo/erlang-questions > >> > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hd2010@REDACTED Thu Apr 11 00:41:26 2013 From: hd2010@REDACTED (Henning Diedrich) Date: Thu, 11 Apr 2013 00:41:26 +0200 Subject: [erlang-questions] MySQL driver In-Reply-To: References: <51E1C638-0C5F-4C7F-AB32-7A51123378E8@eonblast.com> Message-ID: <5638AEFC-E7D7-4CB2-ABCF-250CE2882FE8@eonblast.com> Hi Garrett, great, I'll check out your connection time outs, thanks a lot! It was quite fun to find Jacob, Nick and Fredrik to write this [1] up. It certainly felt more like historical forensics from some point on although I was throughout only trying to find out which driver version made most sense. Which in the end came to light in a way I could not read out of the source. If someone would like to share the story of his fork I'll be happy to include it, for the fun of it. Somehow, this all fits MySQL. Henning [1] https://github.com/eonblast/Emysql#History On Apr 11, 2013, at 12:25 AM, Garrett Smith wrote: > Someday someone will write a book on the history of the Erlang MySQL > libraries. Not a great read, but it'd have to be scholarly. > > I've modified the Eonblast library with some connection timeout tweaks > (as in, supporting them): > > https://github.com/gar1t/Emysql > > > > On Wed, Apr 10, 2013 at 4:59 PM, Dave Smith wrote: >> Hi Henning, >> >> Thanks for the pointers. I must admit that I find the level of snark in the >> emysql history delightful, considering that it's at least as forked as >> erlang-mysql-driver now. :) >> >> There are a number of good fixes for the emysql driver, but they are on 20 >> different forks across Github. Looking over csrl's, fixes, I'm pretty >> concerned those aren't in mainline since there are some non-trivial pooling >> problems he's addressed (would you like some irony with that irony? :)). >> >> Do you know if eonblast is planning to incorporate all the work from those >> different forks? I think we need to build some momentum behind a driver and >> I'm certainly happy to direct people towards a maintained one. >> >> IMO, it would also be handy if the driver wasn't pooled, but used poolboy >> since getting the pooling right is rather difficult. (I know that poolboy >> has been tested with some of Quviq's concurrent checking stuff and a number >> of race conditions addressed). >> >> Thanks! >> >> D. >> >> >> On Wed, Apr 10, 2013 at 3:00 PM, Henning Diedrich >> wrote: >>> >>> Hi Dave, >>> >>> if I remember right, you were using the driver that came from Yxa via >>> Yarif's erlang-emysql-driver https://github.com/dizzyd/erlang-mysql-driver ? >>> (Here is the history of that. https://github.com/eonblast/Emysql#History) >>> >>> It was found less than perfect when Nick and Jacob gave it something to >>> chew building Rupture. I never spotted the problems myself I should add. >>> That's purely from Nick Gerakine's and Jacob Vorreuter's words. >>> >>> But they report that with much effort they could not get the races out of >>> it. That's why they started emysql, which I later started maintaining, and >>> which Jeremy just mentioned, https://github.com/eonblast/Emysql >>> >>> Considerable work went into getting the pooling system of Emysql right. >>> (What I don't know is when exactly it got buggy, it may have been after >>> Jacob's Emysql fork that now lies dormant.) But I worked a while, with help >>> from others to get all races out of the queuing. >>> >>> However, the best fork of Emysql, I *think* is currently that of Chris >>> Rempel at https://github.com/csrl/emysql as I got behind while making >>> Erlvolt, the VoltDB driver, modeled in parts on Emysql. Chris did some >>> impressive clean up. Now that Erlvolt is out, I'll turn around and catch up >>> asap. But that's not a New York asap. >>> >>> An alternative may be to go back to a more original Yxa version, like the >>> one possibly still to be found in ejabberd. That may be more stable than >>> Yariv's branch that you started out with. The "history" I linked to above >>> has the reasons, why that branch may be not quite clean and hard to debug. >>> >>> Best, >>> >>> Henning >>> >>> On Apr 10, 2013, at 10:43 PM, Dave Smith wrote: >>> >>>> Hello list, >>>> >>>> I'm curious what libraries people are currently using for MySQL access >>>> in Erlang. I have an old library on my github that I've kept basically >>>> working, but am wondering if there are any preferred/production libraries >>>> out there that people like better. >>>> >>>> I ask because I'm considering retrofitting the pooling system in the >>>> driver I maintain to use poolboy, but figure it was worth checking if there >>>> are better alternatives out there. :) >>>> >>>> After all, the best code to maintain is the code you don't maintain. >>>> >>>> D. >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From norton@REDACTED Thu Apr 11 02:15:45 2013 From: norton@REDACTED (=?utf-8?B?44OO44O844OI44OzIOOCuOODp+ODvOOCu+ODlSDjgqbjgqfjgqQg?= =?utf-8?B?44Oz?=) Date: Thu, 11 Apr 2013 09:15:45 +0900 Subject: [erlang-questions] integrating nodejs with erlang code In-Reply-To: References: <1492815702.54665218.1365608794384.JavaMail.root@erlang-solutions.com> Message-ID: Aman - Hello. I'm currently building a Scheme interpreter for the Erlang VM (https://github.com/the-concurrent-schemer). I drafted a data model that maps Scheme to Erlang and then built a Scheme token scanner, a Scheme datum parser, and the basics for a Scheme interpreter. I don't know much about JS (or Scheme for that matter) but feel free to contact me directly if you have questions or find any of this relevant to your interest about implementing JS in Erlang. Good luck! Cheers, Joe N On Apr 11, 2013, at 03:49 , aman mangal wrote: > Hi, > > How can I go about implementing JS in erlang? This seems interesting to me. Please tell me what do you mean by implementing JS or node.js in erlang? > > Aman Mangal > 3rd year Undergraduate > Department of Computer Science & Engineering > IIT Bombay > www.cse.iitb.ac.in/~amanmangal > > > On Wed, Apr 10, 2013 at 10:42 PM, Zheng Zhibin wrote: > beam.js would be one way to build node.js into Erlang :) > > Cheers > Ben > > ? 2013-4-10???11:46?Robert Virding ??? > >> The most efficient way of connecting Erlang with node.js would be, of course, to implement JS and hence node.js in Erlang. Maybe not the easiest way, but definitely the most efficient connection. :-) >> >> Robert >> >> From: "Joe Armstrong" >> To: "Eduardo Gurgel" >> Cc: "Ali Attarwala" , "Shailesh Tyagi" , "wncc iitb" , "Erlang" , "chirag jain" >> Sent: Wednesday, 10 April, 2013 11:14:19 AM >> Subject: Re: [erlang-questions] integrating nodejs with erlang code >> >> >> Just curious, but what is the *least efficient* way of connecting node.js to Erlang. >> >> If I were doing this I'd start with the least efficient method, build my app, measure, then optimize if >> necessary. >> >> If the optimization is not necessary you have saved yourself a whole lot of work. >> >> Alternatively, write the code and if it is not fast enough wait ten years. >> >> >> Cheers >> >> /Joe >> >> >> >> >> On Tue, Apr 9, 2013 at 2:11 PM, Eduardo Gurgel wrote: >> >> >> >> On Mon, Apr 8, 2013 at 12:37 PM, aman mangal wrote: >> >> Hi, >> >> What is the most efficient way to pass events/data from erlang processes to nodejs and vice versa? Setting up a tcp socket may be slower or inefficient. I was thinking of doing something native. Both erlang and nodejs provide libraries(NIF) to call C functions but none of them provide an efficient implementation for calling erlang/nodejs functions from C or C++ as far as I know. Please suggest some ideas! >> >> >> You could take a look on BERT RPC: >> >> https://github.com/rtomayko/node-bertrpc >> >> https://github.com/mojombo/bert.erl >> >> Right now, bert-rpc.org is returning 404, but it was up two days ago: >> >> http://bert-rpc.org/ >> >> Maybe checking cached webpage on bert-rpc.org may help you. >> >> -- >> >> Eduardo >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From esente@REDACTED Thu Apr 11 05:29:50 2013 From: esente@REDACTED (Son Tran-Nguyen) Date: Wed, 10 Apr 2013 22:29:50 -0500 Subject: [erlang-questions] integrating nodejs with erlang code In-Reply-To: References: <1492815702.54665218.1365608794384.JavaMail.root@erlang-solutions.com> Message-ID: Hi list, I just want to chime in my 2 cent. I have been thinking about the communication between Erlang and other languages for a while, and of course you can write port to do so but there are problems with them. So I was thinking, why don't we just use a message queue system like RabbitMQ to act as the broker in between languages. Most message queue systems have driver for most languages. We just need to implement an RPC over messages. And we have total control over them. I'm probably not familiar enough with Erlang but I think it maybe a better choice. Sincerely, Son Tran-Nguyen On Wed, Apr 10, 2013 at 7:15 PM, ???? ????? ??? ? wrote: > > Aman - > > Hello. I'm currently building a Scheme interpreter for the Erlang VM ( > https://github.com/the-concurrent-schemer). I drafted a data model that > maps Scheme to Erlang and then built a Scheme token scanner, a Scheme datum > parser, and the basics for a Scheme interpreter. > > I don't know much about JS (or Scheme for that matter) but feel free to > contact me directly if you have questions or find any of this relevant to > your interest about implementing JS in Erlang. Good luck! > > Cheers, > > Joe N > > On Apr 11, 2013, at 03:49 , aman mangal wrote: > > Hi, > > How can I go about implementing JS in erlang? This seems interesting to > me. Please tell me what do you mean by implementing JS or node.js in erlang? > > Aman Mangal > 3rd year Undergraduate > Department of Computer Science & Engineering > IIT Bombay > www.cse.iitb.ac.in/~amanmangal > > > On Wed, Apr 10, 2013 at 10:42 PM, Zheng Zhibin wrote: > >> beam.js would be one way to build node.js into Erlang :) >> >> Cheers >> Ben >> >> ? 2013-4-10???11:46?Robert Virding >> ??? >> >> The most efficient way of connecting Erlang with node.js would be, of >> course, to implement JS and hence node.js in Erlang. Maybe not the easiest >> way, but definitely the most efficient connection. :-) >> >> Robert >> >> ------------------------------ >> >> *From: *"Joe Armstrong" >> *To: *"Eduardo Gurgel" >> *Cc: *"Ali Attarwala" , "Shailesh Tyagi" < >> shailesh@REDACTED>, "wncc iitb" , >> "Erlang" , "chirag jain" < >> chirag.jain@REDACTED> >> *Sent: *Wednesday, 10 April, 2013 11:14:19 AM >> *Subject: *Re: [erlang-questions] integrating nodejs with erlang code >> >> >> Just curious, but what is the *least efficient* way of connecting node.js >> to Erlang. >> >> If I were doing this I'd start with the least efficient method, build my >> app, measure, then optimize if >> necessary. >> >> If the optimization is not necessary you have saved yourself a whole lot >> of work. >> >> Alternatively, write the code and if it is not fast enough wait ten years. >> >> >> Cheers >> >> /Joe >> >> >> >> >> On Tue, Apr 9, 2013 at 2:11 PM, Eduardo Gurgel wrote: >> >>> >>> >>> >>> On Mon, Apr 8, 2013 at 12:37 PM, aman mangal wrote: >>> >>>> >>>> Hi, >>>> >>>> What is the most efficient way to pass events/data from erlang >>>> processes to nodejs and vice versa? Setting up a tcp socket may be slower >>>> or inefficient. I was thinking of doing something native. Both erlang and >>>> nodejs provide libraries(NIF) to call C functions but none of them provide >>>> an efficient implementation for calling erlang/nodejs functions from C or >>>> C++ as far as I know. Please suggest some ideas! >>>> >>>> >>> You could take a look on BERT RPC: >>> >>> https://github.com/rtomayko/node-bertrpc >>> >>> https://github.com/mojombo/bert.erl >>> >>> Right now, bert-rpc.org is returning 404, but it was up two days ago: >>> >>> http://bert-rpc.org/ >>> >>> Maybe checking cached webpage on bert-rpc.org may help you. >>> >>> -- >>> >>> Eduardo >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Thu Apr 11 09:52:29 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Thu, 11 Apr 2013 09:52:29 +0200 Subject: [erlang-questions] =?windows-1252?q?Preallocated_binary_=97_save_?= =?windows-1252?q?memory?= In-Reply-To: References: Message-ID: Hello Max. I put together some BEAM assembly to write a function that just use bs_init_writable. Do you have any idea which size the initial buffer should be in your code? Could you try using prealloc_bin:new/1[1] and see if it improve things? https://gist.github.com/nox/5359459/raw/0b8615/prealloc_bin.S Regards, -- Anthony Ramine Le 10 avr. 2013 ? 22:50, Max Lapshin a ?crit : > I've looked at erlc -S bc.erl: > > {function, bc, 1, 2}. > {label,1}. > {line,[{location,"bc.erl",3}]}. > {func_info,{atom,bc},{atom,bc},1}. > {label,2}. > {allocate_zero,1,1}. > {line,[{location,"bc.erl",3}]}. > {gc_bif,bit_size,{f,0},1,[{x,0}],{x,1}}. > {line,[]}. > {gc_bif,'div',{f,0},2,[{x,1},{integer,336}],{x,1}}. > {line,[]}. > {gc_bif,'*',{f,0},2,[{x,1},{integer,45}],{x,1}}. > {move,{x,0},{y,0}}. > {move,{x,1},{x,0}}. > bs_init_writable. > {move,{x,0},{x,1}}. > {move,{y,0},{x,0}}. > {call_last,2,{f,8},1}. > > > > It is really cool! Erlang understands that resulting binary will be exactly 45 * (size(Input) div 42) bytes and preallocates it. > This is a very important reason to use list and binary comprehensions. > > From n.oxyde@REDACTED Thu Apr 11 09:58:15 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Thu, 11 Apr 2013 09:58:15 +0200 Subject: [erlang-questions] =?windows-1252?q?Preallocated_binary_=97_save_?= =?windows-1252?q?memory?= In-Reply-To: References: Message-ID: <153493C3-C99B-4EEA-9A48-8518D48D2984@gmail.com> Correct URL: https://gist.github.com/nox/5359459/raw/0b86154804b43b9043a3fed00debe284f4702f10/prealloc_bin.S -- Anthony Ramine Le 11 avr. 2013 ? 09:52, Anthony Ramine a ?crit : > Hello Max. > > I put together some BEAM assembly to write a function that just use bs_init_writable. Do you have any idea which size the initial buffer should be in your code? Could you try using prealloc_bin:new/1[1] and see if it improve things? > > https://gist.github.com/nox/5359459/raw/0b8615/prealloc_bin.S > > Regards, > > -- > Anthony Ramine > > Le 10 avr. 2013 ? 22:50, Max Lapshin a ?crit : > >> I've looked at erlc -S bc.erl: >> >> {function, bc, 1, 2}. >> {label,1}. >> {line,[{location,"bc.erl",3}]}. >> {func_info,{atom,bc},{atom,bc},1}. >> {label,2}. >> {allocate_zero,1,1}. >> {line,[{location,"bc.erl",3}]}. >> {gc_bif,bit_size,{f,0},1,[{x,0}],{x,1}}. >> {line,[]}. >> {gc_bif,'div',{f,0},2,[{x,1},{integer,336}],{x,1}}. >> {line,[]}. >> {gc_bif,'*',{f,0},2,[{x,1},{integer,45}],{x,1}}. >> {move,{x,0},{y,0}}. >> {move,{x,1},{x,0}}. >> bs_init_writable. >> {move,{x,0},{x,1}}. >> {move,{y,0},{x,0}}. >> {call_last,2,{f,8},1}. >> >> >> >> It is really cool! Erlang understands that resulting binary will be exactly 45 * (size(Input) div 42) bytes and preallocates it. >> This is a very important reason to use list and binary comprehensions. >> >> > From Andreas.Hellstrom@REDACTED Thu Apr 11 10:55:53 2013 From: Andreas.Hellstrom@REDACTED (Andreas.Hellstrom@REDACTED) Date: Thu, 11 Apr 2013 08:55:53 +0000 Subject: [erlang-questions] Missing module create_certs in ssl-5.2.1 Message-ID: <6CA60E3A41E34D4A851816F9BA44F76A3419B1D4@EXMB13TSTRZ2.tcad.telia.se> Hi! According to the current SSL library documentation there should exist an example module for generating certificates: "The following module create_certs is used by the Erlang/OTP SSL application for generating certificates to be used in tests. The source code is also found in ssl-X.Y.Z/examples/certs/src." There's no module there (and not in previous version ssl-4.1.6 either). Can I find the module somewhere else? http://www.erlang.org/documentation/doc-5.7.5/lib/ssl-3.10.8/doc/html/create_certs.html /Andreas -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu Apr 11 13:38:01 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 11 Apr 2013 15:38:01 +0400 Subject: [erlang-questions] =?utf-8?q?Preallocated_binary_=E2=80=94_save_m?= =?utf-8?q?emory?= In-Reply-To: <153493C3-C99B-4EEA-9A48-8518D48D2984@gmail.com> References: <153493C3-C99B-4EEA-9A48-8518D48D2984@gmail.com> Message-ID: I've made following: {function, bin, 1, 2}. {label,1}. {line,[{location,"alloc.erl",4}]}. {func_info,{atom,alloc},{atom,bin},1}. {label,2}. {line,[{location,"alloc.erl",5}]}. bs_init_writable. return. It shows excelent results on syntetic tests, but when I run real code, binary doesn't remain the same, but I'm only appending. Perhaps, this zerocopy magic will work only if compiler generates bs_private_append instructions? -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu Apr 11 13:42:54 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 11 Apr 2013 15:42:54 +0400 Subject: [erlang-questions] =?utf-8?q?Preallocated_binary_=E2=80=94_save_m?= =?utf-8?q?emory?= In-Reply-To: References: <153493C3-C99B-4EEA-9A48-8518D48D2984@gmail.com> Message-ID: I'm looking at erlang:process_info(self(),binary) and assuming that first number is a binary identifier, that must not change after all changes. -------------- next part -------------- An HTML attachment was scrubbed... URL: From konrad.gadek@REDACTED Thu Apr 11 14:03:34 2013 From: konrad.gadek@REDACTED (Konrad =?utf-8?Q?G=C4=85dek?=) Date: Thu, 11 Apr 2013 13:03:34 +0100 (BST) Subject: [erlang-questions] Using the half-word emulator with the erlang-solutions ubuntu package In-Reply-To: Message-ID: <2071360021.56439943.1365681814045.JavaMail.root@erlang-solutions.com> I consulted the half-word emulator option: we shall have such packages (and others) in future, but this is not a priority and, I must admit, Karol and I have some other work to do now, so there's no deadline for that unfortunately. In the meantime, if you need half-word emulator now, I'd suggest using kerl[1] to build your custom Erlang -- it's like rvm/virtualenv but for Erlang and it's easy to use (yes! 2 or so commands to build&install it!). You shall add --enable-m64-build --disable-hipe --enable-halfword-emulator --with-ssl to KERL_CONFIGURE_OPTIONS (some more info here[2] and in the kerl readme[3]; ~/.kerlrc file I use for my projects is here: [4]; in ESL we don't use kerl). Take a note that it's a known issue for kerl[5] to silently build Erlang without crypto - and probably everyone needs it (at least for rebar) - so keep an eye on that. In case of any problems, don't hesitate to ask :) [1] https://github.com/spawngrid/kerl [2] http://www.erlang.org/doc/installation_guide/INSTALL.html#id62320 [3] https://github.com/spawngrid/kerl/blob/master/README.md [4] http://pastebin.com/qWVC6rHr [5] https://github.com/spawngrid/kerl/issues/31 /Konrad G?dek ----- Original Message ----- From: "pablo platt" To: "Konrad G?dek" Cc: erlang-questions@REDACTED Sent: Wednesday, April 10, 2013 9:02:48 PM Subject: Re: [erlang-questions] Using the half-word emulator with the erlang-solutions ubuntu package Is it possible to add a 64bit with half-word enabled and HiPE disabled package in addition to the 32bit and 64bit packages? Is there a chance HiPE and half-word will work together in a future release? Thank you for the packages. They are extremely helpful. On Wed, Apr 10, 2013 at 9:36 PM, Konrad G?dek < konrad.gadek@REDACTED > wrote: Hi, As far as I'm aware, we don't have half-word emulator enabled in our packages, so this seems like an erroneous statement on ESL website -- I'll investigate it more tomorrow. Both disabling HiPE and enabling half-word emulator are done while building packages. Some more info: http://stackoverflow.com/a/1185490/547223 /Konrad G?dek ----- Original Message ----- From: "pablo platt" < pablo.platt@REDACTED > To: erlang-questions@REDACTED Sent: Wednesday, April 10, 2013 2:59:07 PM Subject: [erlang-questions] Using the half-word emulator with the erlang-solutions ubuntu package Hi, The page on the erlang-solutions website says that the packages support the half-word emulator: https://www.erlang-solutions.com/downloads "These have included HiPE enabled, 32 and 64 bit configurations, alongside the half-word emulator" I see [hipe] when starting the shell so half-word is probably not enabled. I'm not using hipe so there is no problem disabling it. Do I need to set a flag when installing the deb package? Do I need to use a flag when starting erlang or creating a release with rebar in the vm.args? Are there reasons not to use half-word besides the 4GB process memory limit? Thanks _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From n.oxyde@REDACTED Thu Apr 11 14:34:55 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Thu, 11 Apr 2013 14:34:55 +0200 Subject: [erlang-questions] =?windows-1252?q?Preallocated_binary_=97_save_?= =?windows-1252?q?memory?= In-Reply-To: References: <153493C3-C99B-4EEA-9A48-8518D48D2984@gmail.com> Message-ID: No, it should work anyway, bs_private_append is the same thing as bs_append except that it won't check whether the binary is writable or not. -- Anthony Ramine Le 11 avr. 2013 ? 13:38, Max Lapshin a ?crit : > Perhaps, this zerocopy magic will work only if compiler generates bs_private_append instructions? From hd2010@REDACTED Fri Apr 12 03:59:17 2013 From: hd2010@REDACTED (Henning Diedrich) Date: Fri, 12 Apr 2013 03:59:17 +0200 Subject: [erlang-questions] Daffodils Message-ID: The daffodils sang, giants, when you write a tool what is 'fast enough'. From hd2010@REDACTED Fri Apr 12 06:53:37 2013 From: hd2010@REDACTED (Henning Diedrich) Date: Fri, 12 Apr 2013 06:53:37 +0200 Subject: [erlang-questions] [ANN]: New Erlvolt Message-ID: Hi list, an all-new iteration of the VoltDB driver Erlvolt has been released. It's easy to use, provides strong connection pooling, plus a broad array of options. It is optimized for a central node architecture, and maximal throughput. VoltDB is a scalable, Open Source, high-performance data base for online transactions -- typically small data sets sent to a server at rapid pace and in high volume. Volt keeps SQL and ACID and scales pretty much linearly up to silly cluster sizes. I did a benchmark using AWS EC2 recently, where I got 877,000 transactions/second, equalling 3.5M operations/second (3 reads, 1 write + bonus: 2 materialized view updates). For details see: http://blog.voltdb.com/877000-tps-with-erlang-and-voltdb/ The Erlvolt driver is the result of the hunt for a better database for heavy duty online-game servers. I experienced first hand what a pain it could be to try to scale MySQL and found VoltDB better suited for the requirements of more complex game worlds. Better than any other database in fact for games with free player interaction. To be able to use it with Erlang, I started creating the Erlang driver for VoltDB. Giving it a whirl takes seconds, plus a few short minutes build time. Simply execute these lines to clone, build, and run a simple benchmark: git clone https://github.com/VoltDB/voltdb.git voltdb git clone https://github.com/Eonblast/Erlvolt.git erlvolt cd voltdb && ant && cd examples/voter && ./run.sh & cd erlvolt && make profile bench Hello World ---------------- This is a hello world program. See the README.md for more. ( https://github.com/Eonblast/Erlvolt#Samples ) -module(hello). -export([run/0]). -include("erlvolt.hrl"). run() -> crypto:start(), application:start(erlvolt), erlvolt:add_pool(hello_pool, [{"localhost", 21212}]), erlvolt:call_procedure(hello_pool, "Insert", ["Hej", "v?rlden", "Swedish"]), Result = erlvolt:call_procedure(hello_pool, "Select", ["Swedish"]), Table = erlvolt:get_table(Result, 1), Row = erlvolt:get_row(Table, 1), Hello = erlvolt:get_string(Row, Table, "HELLO"), World = erlvolt:get_string(Row, Table, "WORLD"), io:format("~n~s ~s!~n~n", [Hello, World]), erlvolt:close_pool(hello_pool). Execution ------------- The driver can serve multiple processes, using multiple connections and connection pools. Erlvolt can serve multiple pools in parallel. Queries that are not implemented as stored procedures, so-called 'ad-hoc queries' are easily available with this driver, for tests and non-performance sensitive experimentation: R = erlvolt:call_procedure(mypool, "@AdHoc", ["SELECT COUNT(*) AS cnt FROM votes"]), If you find time, please let me know your feedback! Thanks, Henning From dmkolesnikov@REDACTED Fri Apr 12 10:32:00 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Fri, 12 Apr 2013 11:32:00 +0300 Subject: [erlang-questions] [ANN]: New Erlvolt In-Reply-To: References: Message-ID: <247A94FA-7337-47BF-8B1F-E04FA14F5353@gmail.com> Hello, Thanks for sharing it? Just one issue pop-up in my mind, I was struggling with it in mysql drivers as well. The issue is here: erlvolt:add_pool(hello_pool, [{"localhost", 21212}]), From my perspective, the "connection" pool has to be supervised by the client application (not by the library). The client application might have a different fault recovery principles while library makes it generic. I think that the supervisor tree is a good place for app to set-up/tear-down connection pools. I would make it something like this: %% -module(erlvolt). -spec(start_link/2 :: (atom(), list()) -> {ok, pid} | {error, any()}). then client application might implement what every fault management principles it like * use it like in you example * use it through supervisor e.g. %% %% -module(my_sup). -behaviour(supervisor). -export([start_link/0, init/1]). start_link() -> supervisor:start_link({local, ?MODULE}, ?MODULE, []). %% %% init([]) -> {ok, { {one_for_one, 10, 900}, [pool()] } }. pool() -> { pool, {erlvolt, start_link, [...]}, permanent, 1000, worker, dynamic }. I'll appreciate if you share your thought on your design decision. - Dmitry On Apr 12, 2013, at 7:53 AM, Henning Diedrich wrote: > Hi list, > > an all-new iteration of the VoltDB driver Erlvolt has been released. > > It's easy to use, provides strong connection pooling, plus a broad array of options. It is optimized for a central node architecture, and maximal throughput. > > VoltDB is a scalable, Open Source, high-performance data base for online transactions -- typically small data sets sent to a server at rapid pace and in high volume. Volt keeps SQL and ACID and scales pretty much linearly up to silly cluster sizes. I did a benchmark using AWS EC2 recently, where I got 877,000 transactions/second, equalling 3.5M operations/second (3 reads, 1 write + bonus: 2 materialized view updates). For details see: http://blog.voltdb.com/877000-tps-with-erlang-and-voltdb/ > > The Erlvolt driver is the result of the hunt for a better database for heavy duty online-game servers. I experienced first hand what a pain it could be to try to scale MySQL and found VoltDB better suited for the requirements of more complex game worlds. Better than any other database in fact for games with free player interaction. To be able to use it with Erlang, I started creating the Erlang driver for VoltDB. > > Giving it a whirl takes seconds, plus a few short minutes build time. Simply execute these lines to clone, build, and run a simple benchmark: > > git clone https://github.com/VoltDB/voltdb.git voltdb > git clone https://github.com/Eonblast/Erlvolt.git erlvolt > cd voltdb && ant && cd examples/voter && ./run.sh & > cd erlvolt && make profile bench > > > Hello World > ---------------- > > This is a hello world program. See the README.md for more. ( https://github.com/Eonblast/Erlvolt#Samples ) > > -module(hello). > -export([run/0]). > > -include("erlvolt.hrl"). > > run() -> > > crypto:start(), > application:start(erlvolt), > > erlvolt:add_pool(hello_pool, [{"localhost", 21212}]), > > erlvolt:call_procedure(hello_pool, "Insert", ["Hej", "v?rlden", "Swedish"]), > > Result = erlvolt:call_procedure(hello_pool, "Select", ["Swedish"]), > > Table = erlvolt:get_table(Result, 1), > Row = erlvolt:get_row(Table, 1), > Hello = erlvolt:get_string(Row, Table, "HELLO"), > World = erlvolt:get_string(Row, Table, "WORLD"), > > io:format("~n~s ~s!~n~n", [Hello, World]), > > erlvolt:close_pool(hello_pool). > > > Execution > ------------- > > The driver can serve multiple processes, using multiple connections and connection pools. Erlvolt can serve multiple pools in parallel. > > Queries that are not implemented as stored procedures, so-called 'ad-hoc queries' are easily available with this driver, for tests and non-performance sensitive experimentation: > > R = erlvolt:call_procedure(mypool, "@AdHoc", ["SELECT COUNT(*) AS cnt FROM votes"]), > > > If you find time, please let me know your feedback! > > Thanks, > > Henning > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablo.platt@REDACTED Fri Apr 12 11:53:42 2013 From: pablo.platt@REDACTED (pablo platt) Date: Fri, 12 Apr 2013 12:53:42 +0300 Subject: [erlang-questions] Using the half-word emulator with the erlang-solutions ubuntu package In-Reply-To: <2071360021.56439943.1365681814045.JavaMail.root@erlang-solutions.com> References: <2071360021.56439943.1365681814045.JavaMail.root@erlang-solutions.com> Message-ID: kerl looks great but I only need one release and having to activate Erlang on every shell can be annoying. A package also takes care of dependencies for me avoiding issues like the missing libssl headers you mentioned. Thanks On Thu, Apr 11, 2013 at 3:03 PM, Konrad G?dek < konrad.gadek@REDACTED> wrote: > I consulted the half-word emulator option: we shall have such packages > (and others) in future, but this is not a priority and, I must admit, Karol > and I have some other work to do now, so there's no deadline for that > unfortunately. > > In the meantime, if you need half-word emulator now, I'd suggest using > kerl[1] to build your custom Erlang -- it's like rvm/virtualenv but for > Erlang and it's easy to use (yes! 2 or so commands to build&install it!). > You shall add --enable-m64-build --disable-hipe --enable-halfword-emulator > --with-ssl to KERL_CONFIGURE_OPTIONS (some more info here[2] and in the > kerl readme[3]; ~/.kerlrc file I use for my projects is here: [4]; in ESL > we don't use kerl). > Take a note that it's a known issue for kerl[5] to silently build Erlang > without crypto - and probably everyone needs it (at least for rebar) - so > keep an eye on that. > > In case of any problems, don't hesitate to ask :) > > > [1] https://github.com/spawngrid/kerl > [2] http://www.erlang.org/doc/installation_guide/INSTALL.html#id62320 > [3] https://github.com/spawngrid/kerl/blob/master/README.md > [4] http://pastebin.com/qWVC6rHr > [5] https://github.com/spawngrid/kerl/issues/31 > > > > /Konrad G?dek > > ----- Original Message ----- > From: "pablo platt" > To: "Konrad G?dek" > Cc: erlang-questions@REDACTED > Sent: Wednesday, April 10, 2013 9:02:48 PM > Subject: Re: [erlang-questions] Using the half-word emulator with the > erlang-solutions ubuntu package > > > > > > Is it possible to add a 64bit with half-word enabled and HiPE disabled > package in addition to the 32bit and 64bit packages? > > > Is there a chance HiPE and half-word will work together in a future > release? > > > Thank you for the packages. > They are extremely helpful. > > > > > On Wed, Apr 10, 2013 at 9:36 PM, Konrad G?dek < > konrad.gadek@REDACTED > wrote: > > > Hi, > > As far as I'm aware, we don't have half-word emulator enabled in our > packages, so this seems like an erroneous statement on ESL website -- I'll > investigate it more tomorrow. > > Both disabling HiPE and enabling half-word emulator are done while > building packages. Some more info: > http://stackoverflow.com/a/1185490/547223 > > /Konrad G?dek > > > > ----- Original Message ----- > From: "pablo platt" < pablo.platt@REDACTED > > To: erlang-questions@REDACTED > Sent: Wednesday, April 10, 2013 2:59:07 PM > Subject: [erlang-questions] Using the half-word emulator with the > erlang-solutions ubuntu package > > > > > > > > Hi, > > The page on the erlang-solutions website says that the packages support > the half-word emulator: > https://www.erlang-solutions.com/downloads > "These have included HiPE enabled, 32 and 64 bit configurations, alongside > the half-word emulator" > > I see [hipe] when starting the shell so half-word is probably not enabled. > I'm not using hipe so there is no problem disabling it. > > Do I need to set a flag when installing the deb package? > Do I need to use a flag when starting erlang or creating a release with > rebar in the vm.args? > > > Are there reasons not to use half-word besides the 4GB process memory > limit? > > > > Thanks > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hq@REDACTED Fri Apr 12 14:15:55 2013 From: hq@REDACTED (Adam Rutkowski) Date: Fri, 12 Apr 2013 14:15:55 +0200 Subject: [erlang-questions] Using the half-word emulator with the erlang-solutions ubuntu package In-Reply-To: References: <2071360021.56439943.1365681814045.JavaMail.root@erlang-solutions.com> Message-ID: <37D9D4D2-749F-4CB4-BF51-677D5DC95CDC@mtod.org> On 12 Apr 2013, at 11:53, pablo platt wrote: > kerl looks great but I only need one release and having to activate Erlang on every shell can be annoying. Activate in your shell startup scripts. A. From mrtndimitrov@REDACTED Fri Apr 12 16:38:19 2013 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Fri, 12 Apr 2013 17:38:19 +0300 Subject: [erlang-questions] eheap_alloc: Cannot allocate 1272748 bytes of memory (of type "heap"). Message-ID: <51681C5B.30105@gmail.com> Hello, I perform performance tests on our app and I manage to crash the VM every time with the above message. What bothers me is: 1. the allocation request is low: 1-4 MB 2. the total amount of used memory is also low: 300-500 MB (the test machine has 8GB of RAM) 3. all processes have low consumption of memory (highest 3 MB) but the app creates many(?) of them: about 2 500 4. there is no Running process, one is Garbing and it is started by YAWS. About 100 processes are Scheduled and all the rest are Waiting. This state repeats on every run. The test machine is Windows 8, 4 cores, 8 GB Ram, running Erlang R16B and YAWS 1.6. Thank you very much, Martin From rtrlists@REDACTED Fri Apr 12 17:04:11 2013 From: rtrlists@REDACTED (Robert Raschke) Date: Fri, 12 Apr 2013 16:04:11 +0100 Subject: [erlang-questions] eheap_alloc: Cannot allocate 1272748 bytes of memory (of type "heap"). In-Reply-To: <51681C5B.30105@gmail.com> References: <51681C5B.30105@gmail.com> Message-ID: Hi Martin, not sure I'm reading that correctly, but 2500 * ~2 MB is ~5GB. So I don't know how you are measuring 500MB overall usage. Robby On Apr 12, 2013 3:38 PM, "Martin Dimitrov" wrote: > Hello, > > I perform performance tests on our app and I manage to crash the VM > every time with the above message. What bothers me is: > 1. the allocation request is low: 1-4 MB > 2. the total amount of used memory is also low: 300-500 MB (the test > machine has 8GB of RAM) > 3. all processes have low consumption of memory (highest 3 MB) but the > app creates many(?) of them: about 2 500 > 4. there is no Running process, one is Garbing and it is started by > YAWS. About 100 processes are Scheduled and all the rest are Waiting. > This state repeats on every run. > > The test machine is Windows 8, 4 cores, 8 GB Ram, running Erlang R16B > and YAWS 1.6. > > Thank you very much, > > Martin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrtndimitrov@REDACTED Fri Apr 12 17:12:03 2013 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Fri, 12 Apr 2013 18:12:03 +0300 Subject: [erlang-questions] eheap_alloc: Cannot allocate 1272748 bytes of memory (of type "heap"). In-Reply-To: References: <51681C5B.30105@gmail.com> Message-ID: <51682443.7090207@gmail.com> The highest amount of memory per process is 3MB. There are only 10-15 of them. The rest have lower amount of used memory. Some - few hundred bytes. The overall usage I see through crashdump_viewer under Memory Information -> Memory. Martin On 4/12/2013 6:04 PM, Robert Raschke wrote: > Hi Martin, > > not sure I'm reading that correctly, but 2500 * ~2 MB is ~5GB. So I don't > know how you are measuring 500MB overall usage. > > Robby > On Apr 12, 2013 3:38 PM, "Martin Dimitrov" wrote: > >> Hello, >> >> I perform performance tests on our app and I manage to crash the VM >> every time with the above message. What bothers me is: >> 1. the allocation request is low: 1-4 MB >> 2. the total amount of used memory is also low: 300-500 MB (the test >> machine has 8GB of RAM) >> 3. all processes have low consumption of memory (highest 3 MB) but the >> app creates many(?) of them: about 2 500 >> 4. there is no Running process, one is Garbing and it is started by >> YAWS. About 100 processes are Scheduled and all the rest are Waiting. >> This state repeats on every run. >> >> The test machine is Windows 8, 4 cores, 8 GB Ram, running Erlang R16B >> and YAWS 1.6. >> >> Thank you very much, >> >> Martin >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > From tristan.sloughter@REDACTED Fri Apr 12 18:49:11 2013 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Fri, 12 Apr 2013 11:49:11 -0500 Subject: [erlang-questions] Gittip Message-ID: I just learned about https://www.gittip.com/ It would be awesome if as many of us as possible started funding, even just a $1 a week, to help our favorite Erlang projects. I know Loic is on, https://www.gittip.com/essen/ -- so if you use Cowboy, Ranch or Bullet, that would be a good place to start :) Would be great if people also gave suggestions of whom people should donate to, even if they themselves can't afford it or if it is their project and they know they could do more with some donations. Tristan -------------- next part -------------- An HTML attachment was scrubbed... URL: From hd2010@REDACTED Sat Apr 13 01:15:40 2013 From: hd2010@REDACTED (Henning Diedrich) Date: Sat, 13 Apr 2013 01:15:40 +0200 Subject: [erlang-questions] Delegating supervision - was Re: [ANN]: New Erlvolt In-Reply-To: <247A94FA-7337-47BF-8B1F-E04FA14F5353@gmail.com> References: <247A94FA-7337-47BF-8B1F-E04FA14F5353@gmail.com> Message-ID: <0CB849ED-0899-4BE8-875A-7ECD4382A557@eonblast.com> Hi Dmitry, I think you have a good point. The more if you encountered the real world necessity to take control. And I wondered about this too at some point but found no good generic answer. My first argument for why it is right as it is, would be that an application should have one top node. You probably don't want to have two to have a robust model. List, how do other applications do it? Henning On Apr 12, 2013, at 10:32 AM, Dmitry Kolesnikov wrote: > Hello, > > Thanks for sharing it? > > Just one issue pop-up in my mind, I was struggling with it in mysql drivers as well. > The issue is here: > erlvolt:add_pool(hello_pool, [{"localhost", 21212}]), > From my perspective, the "connection" pool has to be supervised by the client application (not by the library). > The client application might have a different fault recovery principles while library makes it generic. > I think that the supervisor tree is a good place for app to set-up/tear-down connection pools. > > I would make it something like this: > > %% > -module(erlvolt). > > -spec(start_link/2 :: (atom(), list()) -> {ok, pid} | {error, any()}). > > then client application might implement what every fault management principles it like > * use it like in you example > * use it through supervisor > > e.g. > > %% > %% > -module(my_sup). > -behaviour(supervisor). > -export([start_link/0, init/1]). > > start_link() -> > supervisor:start_link({local, ?MODULE}, ?MODULE, []). > > %% > %% > init([]) -> > {ok, > { > {one_for_one, 10, 900}, > [pool()] > } > }. > > pool() -> > { > pool, > {erlvolt, start_link, [...]}, > permanent, 1000, worker, dynamic > }. > > > I'll appreciate if you share your thought on your design decision. > > - Dmitry > > > > On Apr 12, 2013, at 7:53 AM, Henning Diedrich wrote: > >> Hi list, >> >> an all-new iteration of the VoltDB driver Erlvolt has been released. >> >> It's easy to use, provides strong connection pooling, plus a broad array of options. It is optimized for a central node architecture, and maximal throughput. >> >> VoltDB is a scalable, Open Source, high-performance data base for online transactions -- typically small data sets sent to a server at rapid pace and in high volume. Volt keeps SQL and ACID and scales pretty much linearly up to silly cluster sizes. I did a benchmark using AWS EC2 recently, where I got 877,000 transactions/second, equalling 3.5M operations/second (3 reads, 1 write + bonus: 2 materialized view updates). For details see: http://blog.voltdb.com/877000-tps-with-erlang-and-voltdb/ >> >> The Erlvolt driver is the result of the hunt for a better database for heavy duty online-game servers. I experienced first hand what a pain it could be to try to scale MySQL and found VoltDB better suited for the requirements of more complex game worlds. Better than any other database in fact for games with free player interaction. To be able to use it with Erlang, I started creating the Erlang driver for VoltDB. >> >> Giving it a whirl takes seconds, plus a few short minutes build time. Simply execute these lines to clone, build, and run a simple benchmark: >> >> git clone https://github.com/VoltDB/voltdb.git voltdb >> git clone https://github.com/Eonblast/Erlvolt.git erlvolt >> cd voltdb && ant && cd examples/voter && ./run.sh & >> cd erlvolt && make profile bench >> >> >> Hello World >> ---------------- >> >> This is a hello world program. See the README.md for more. ( https://github.com/Eonblast/Erlvolt#Samples ) >> >> -module(hello). >> -export([run/0]). >> >> -include("erlvolt.hrl"). >> >> run() -> >> >> crypto:start(), >> application:start(erlvolt), >> >> erlvolt:add_pool(hello_pool, [{"localhost", 21212}]), >> >> erlvolt:call_procedure(hello_pool, "Insert", ["Hej", "v?rlden", "Swedish"]), >> >> Result = erlvolt:call_procedure(hello_pool, "Select", ["Swedish"]), >> >> Table = erlvolt:get_table(Result, 1), >> Row = erlvolt:get_row(Table, 1), >> Hello = erlvolt:get_string(Row, Table, "HELLO"), >> World = erlvolt:get_string(Row, Table, "WORLD"), >> >> io:format("~n~s ~s!~n~n", [Hello, World]), >> >> erlvolt:close_pool(hello_pool). >> >> >> Execution >> ------------- >> >> The driver can serve multiple processes, using multiple connections and connection pools. Erlvolt can serve multiple pools in parallel. >> >> Queries that are not implemented as stored procedures, so-called 'ad-hoc queries' are easily available with this driver, for tests and non-performance sensitive experimentation: >> >> R = erlvolt:call_procedure(mypool, "@AdHoc", ["SELECT COUNT(*) AS cnt FROM votes"]), >> >> >> If you find time, please let me know your feedback! >> >> Thanks, >> >> Henning >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sat Apr 13 07:46:19 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 13 Apr 2013 09:46:19 +0400 Subject: [erlang-questions] Delegating supervision - was Re: [ANN]: New Erlvolt In-Reply-To: <0CB849ED-0899-4BE8-875A-7ECD4382A557@eonblast.com> References: <247A94FA-7337-47BF-8B1F-E04FA14F5353@gmail.com> <0CB849ED-0899-4BE8-875A-7ECD4382A557@eonblast.com> Message-ID: There is a good idea in ranch: you may either start process under ranch_sup, either take supervisor spec and launch it under your own supervisor. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Sat Apr 13 10:10:46 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Sat, 13 Apr 2013 11:10:46 +0300 Subject: [erlang-questions] Delegating supervision - was Re: [ANN]: New Erlvolt In-Reply-To: References: <247A94FA-7337-47BF-8B1F-E04FA14F5353@gmail.com> <0CB849ED-0899-4BE8-875A-7ECD4382A557@eonblast.com> Message-ID: <457B4B90-896A-466A-AD8F-EB3867337D6E@gmail.com> the functionality to take "supervisor spec" and return the process for supervision are equivalent. but, start_link is much cleaner and reusable at both contexts, IMHO - Dmitry On Apr 13, 2013, at 8:46 AM, Max Lapshin wrote: > There is a good idea in ranch: you may either start process under ranch_sup, either take supervisor spec and launch it under your own supervisor. From calleja.justin@REDACTED Sat Apr 13 12:07:15 2013 From: calleja.justin@REDACTED (Justin Calleja) Date: Sat, 13 Apr 2013 12:07:15 +0200 Subject: [erlang-questions] Adding new operators? In-Reply-To: References: <2DCB80CB-8712-4A53-91AB-76643E84E7D1@nexttolast.com> <5B558EDF-A921-475C-B33A-B2F41C5045A2@nexttolast.com> Message-ID: ahh yes Anthony's right, my bad. Ok, so this is the first thing that comes to mind then, don't know if it'll help: http://www.scribd.com/doc/22451864/Hacking-Erlang Regards, Justin On 10 April 2013 18:31, Anthony Ramine wrote: > That wouldn't work because you need one new token per operator you want to > add to the language, parse transforms work on abstract syntax trees which > are returned by the parser, a level higher than tokens. > > On the other hand he can also abuse existing expressions by using them in > ways which are illegal in Erlang. > > For example he could write a parse transform for a parallel send operator > like this: > > +[Pids/parallel] ! Message. > > This is beyond ugly but at least it's quite obvious that some spooky > action at the distance is going on. > > Regards, > > -- > Anthony Ramine > > Le 10 avr. 2013 ? 16:41, Justin Calleja a > ?crit : > > > Hi David, > > > > You should be able to do that with a parse_transform/2. Look up parse > transform: you basically get to implement a function which the compiler > calls at compilation time passing it a formal representation of the source > code it's compiling (a.k.a abstract forms... though I'm not entirely sure > about the terminology). > > > > You can then use the erl_syntax module to manipulate these forms. > > > > cheers, > > Justin > > > > > > On 8 April 2013 08:21, Anthony Ramine wrote: > > Hello David, > > > > A fancy way to extend the Erlang syntax would be to have something like > Camlp4, a powerful and extensible preprocessor for OCaml. It has a powerful > quotation system which it itself uses to implements its parser construct, > to extend OCaml's syntax at compile-time. I have began writing a quotation > system for Erlang but didn't have the time finish it yet [1]. > > > > Regards, > > > > [1] https://github.com/nox/otp/tree/brackets > > > > -- > > Anthony Ramine > > > > Le 8 avr. 2013 ? 02:12, David Goehrig a ?crit : > > > > > Next step is integrating into the stack better. But the question > remains, what is the best way to add a new operator. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sat Apr 13 13:07:43 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 13 Apr 2013 15:07:43 +0400 Subject: [erlang-questions] Need help with async disk IO and thread pool on many devices (more than 1 Gbit/s ) In-Reply-To: <515BFE0A.3040800@erlang.org> References: <2622371cef8fadc7a8a34abe609ba782.squirrel@mail.jschneider.net> <515BF412.1080108@erlang.org> <515BF8CE.70004@erlang.org> <515BFE0A.3040800@erlang.org> Message-ID: I've read sources and found new ideas. efile_drv.c is using driver_async: DRIVER_ASYNC(d->level, desc, d->invoke, void_ptr=d, d->free); desc is a pointer to file structure. It means, that each request to one opened file descriptor will be executed on single async thread. In fact, I need to make an erlang-level job queue for reading from disk. There is a long queue of disk requests. I can tag them by their "major device id" or any other tag. I can track, how many requests are opened and not finished for one tag. If there is more than N/M requests (where N is async threads size and M is number of different tags), than request is queued or failed. So, I think that I don't need to write it in C, I can do it in erlang, but make the back pressure not on C level, but on Erlang level. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave@REDACTED Sat Apr 13 20:00:39 2013 From: dave@REDACTED (David Goehrig) Date: Sat, 13 Apr 2013 14:00:39 -0400 Subject: [erlang-questions] Adding new operators? In-Reply-To: References: <2DCB80CB-8712-4A53-91AB-76643E84E7D1@nexttolast.com> <5B558EDF-A921-475C-B33A-B2F41C5045A2@nexttolast.com> Message-ID: Thanks for that deck, I hadn't seen it before. Funnily enough, I've played with erl_scan and erl_parse, but then got sucked into reading the main loop of the beam and started playing around with adding the low level support there, which felt right. I got new opcodes working faster than I got any of the scanner/parser bits working, mostly because the simple C was easier to decypher without knowing the compiler infrastructure. Dave -=-=- dave@REDACTED -=-=- On Apr 13, 2013, at 6:07 AM, Justin Calleja wrote: > ahh yes Anthony's right, my bad. > > Ok, so this is the first thing that comes to mind then, don't know if it'll help: > > http://www.scribd.com/doc/22451864/Hacking-Erlang > > Regards, > Justin > > > On 10 April 2013 18:31, Anthony Ramine wrote: >> That wouldn't work because you need one new token per operator you want to add to the language, parse transforms work on abstract syntax trees which are returned by the parser, a level higher than tokens. >> >> On the other hand he can also abuse existing expressions by using them in ways which are illegal in Erlang. >> >> For example he could write a parse transform for a parallel send operator like this: >> >> +[Pids/parallel] ! Message. >> >> This is beyond ugly but at least it's quite obvious that some spooky action at the distance is going on. >> >> Regards, >> >> -- >> Anthony Ramine >> >> Le 10 avr. 2013 ? 16:41, Justin Calleja a ?crit : >> >> > Hi David, >> > >> > You should be able to do that with a parse_transform/2. Look up parse transform: you basically get to implement a function which the compiler calls at compilation time passing it a formal representation of the source code it's compiling (a.k.a abstract forms... though I'm not entirely sure about the terminology). >> > >> > You can then use the erl_syntax module to manipulate these forms. >> > >> > cheers, >> > Justin >> > >> > >> > On 8 April 2013 08:21, Anthony Ramine wrote: >> > Hello David, >> > >> > A fancy way to extend the Erlang syntax would be to have something like Camlp4, a powerful and extensible preprocessor for OCaml. It has a powerful quotation system which it itself uses to implements its parser construct, to extend OCaml's syntax at compile-time. I have began writing a quotation system for Erlang but didn't have the time finish it yet [1]. >> > >> > Regards, >> > >> > [1] https://github.com/nox/otp/tree/brackets >> > >> > -- >> > Anthony Ramine >> > >> > Le 8 avr. 2013 ? 02:12, David Goehrig a ?crit : >> > >> > > Next step is integrating into the stack better. But the question remains, what is the best way to add a new operator. >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave@REDACTED Sat Apr 13 21:44:10 2013 From: dave@REDACTED (David Goehrig) Date: Sat, 13 Apr 2013 15:44:10 -0400 Subject: [erlang-questions] Adding new operators? In-Reply-To: References: <2DCB80CB-8712-4A53-91AB-76643E84E7D1@nexttolast.com> <5B558EDF-A921-475C-B33A-B2F41C5045A2@nexttolast.com> Message-ID: On Wed, Apr 10, 2013 at 12:31 PM, Anthony Ramine wrote: > > For example he could write a parse transform for a parallel send operator > like this: > > +[Pids/parallel] ! Message. > > This is beyond ugly but at least it's quite obvious that some spooky > action at the distance is going on. > This isn't as ugly as one might think.. Say erl_scan:string(" Pids @! Message "). yields {ok,[{var,1,'Pids'},{'@',1},{'!',1},{var,1,'Message'}],1} and erl_scan:string(" Pids #! Message "). yields {ok,[{var,1,'Pids'},{'#',1},{'!',1},{var,1,'Message'}],1} these I can transform into something reasonable ... and @! as parallel array send and #! and parallel hash send have a sort of Perlish beauty to them :) AFAICT neither @! nor #! mean anything in existing erlang code as both are syntax errors (which in this case is a good thing). Dave -=-=-=-=-=-=-=-=-=-=- http://blog.dloh.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Sun Apr 14 17:28:30 2013 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 14 Apr 2013 17:28:30 +0200 Subject: [erlang-questions] symlink error (I think) Message-ID: I have a troublesome symlink that puts one of my programs into a loop If I list the files in my troublesome directory I see this: ls -l /Users/joe/projects/fuse/fuserl-2.0.5/fw total 8 drwxr-xr-x@ 6 joe staff 204 Jul 23 2009 bin drwxr-xr-x@ 3 joe staff 102 Jul 23 2009 build drwxr-xr-x@ 19 joe staff 646 Jul 23 2009 m4 drwxr-xr-x@ 4 joe staff 136 Jul 23 2009 package drwxr-xr-x@ 5 joe staff 170 Jul 23 2009 revision drwxr-xr-x@ 9 joe staff 306 Jul 23 2009 sh lrwxr-xr-x 1 joe staff 2 Oct 16 2011 share -> .. drwxr-xr-x@ 7 joe staff 238 Jul 23 2009 template So share is a link to '..' But file:read_file_info thinks 'share' is a directory -module(bug). -compile(export_all). -include_lib("kernel/include/file.hrl"). test() -> {ok, X} = file:read_file_info("/Users/joe/projects/fuse/fuserl-2.0.5/fw/share"), X#file_info.type. Running this > bug:test() directory. I'd expected this to return symlink This is on a macbook OS X 10.8.3 Is this a bug, or have I misunderstood something Cheers /Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon@REDACTED Sun Apr 14 18:26:21 2013 From: jon@REDACTED (Jonathan Schneider) Date: Sun, 14 Apr 2013 17:26:21 +0100 Subject: [erlang-questions] : symlink error (I think) References: Message-ID: Quick test here with 15B03 also produces "directory" instead of symlink which I would not expect. Could be today's warmth affecting low temperature Snow Leopard. Also if I create a finder alias called share it's "regular" which I guess it is at some level. There is a more general question of how various filesystem oddities should be treated. For example NTFS has more than one kind of link-type-thing contrary to popular myth. Jon On 14 Apr 2013, at 16:28, Joe Armstrong wrote: > -module(bug). > -compile(export_all). > > -include_lib("kernel/include/file.hrl"). > > test() -> > {ok, X} = file:read_file_info("/Users/joe/projects/fuse/fuserl-2.0.5/fw/share"), > X#file_info.type. -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangprogram@REDACTED Sun Apr 14 19:01:28 2013 From: erlangprogram@REDACTED (S X) Date: Sun, 14 Apr 2013 13:01:28 -0400 Subject: [erlang-questions] diameter callback module Message-ID: Hello, Based on the erlang diameter library and the sample code, I want to start multiple diameter client processes in one erlang node(one client IP), and the client needs to define a diameter_app callback module for certain application, for example: -define(SERVICE(Name), [{'Origin-Host', ?L(Name) ++ ".example.com"}, {'Origin-Realm', "example.com"}, {'Vendor-Id', 193}, {'Product-Name', "Client"}, {'Auth-Application-Id', [?DIAMETER_APP_ID_COMMON, ?DIAMETER_APP_ID_CCRA]}, {application, [{alias, ?APP_CCR_ALIAS}, {dictionary, ?DIAMETER_DICT_CCRA}, {module, client_cb_ccra}]}]). First question: how the diameter library handles this situation? Will all diameter client processes share one single diameter_app callback module "client_cb_ccra" or it will automatically attach different instance of the callback module (process) to the different client process by using spawn_monitor? So from the callback handle_answer in "client_cb_ccra" I can notify the proper client proce by just calling client:notify() something. Second question: Notice that diameter:call allows to set extra arguments, so I was wondering I could set some data like: diameter:call(Name, ?APP_CCR_ALIAS, CCR, [{extra, [self()]}]). which sets the client process ID and I hope to deliver to the callback module "client_cb_ccra" and when the callback module knows which client process sends request and response accordingly, for example using the client process ID in the callback function handle_answer. However, I don't want pack the extra arguments into the diameter packet since the diameter server doesn't know what they are and the extra ones are not part of standard diameter packet. Now I changed the pick_peer callback signatures to allow extra arguments, pick_peer([Peer | _], _, _SvcName, _State, A) But I got encoding error in the callback prepare_request =ERROR REPORT==== 14-Apr-2013::11:40:41 === Error in process <0.175.0> with exit value: {undef,[{client_cb_ccra,prepare_request,[{diameter_packet,{diameter_header,1,undefined,undefined,undefined,3958849953,3958849953,undefined,undefined,undefined,undefined},undefined,{diameter_rfc4006_cc_CCR,["who",";","142745567... {error,encode} In overall, I read the online documents, which have limited information on how to use the extra arguments in the library and don't quite get how to utilize the extra arguments to do something tricky, Are there any suggestions on how to deal with multiple client processes? Thanks a lot! Samuel -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave@REDACTED Sun Apr 14 21:10:18 2013 From: dave@REDACTED (Dave Peticolas) Date: Sun, 14 Apr 2013 12:10:18 -0700 Subject: [erlang-questions] symlink error (I think) In-Reply-To: References: Message-ID: 2013/4/14 Joe Armstrong > I have a troublesome symlink that puts one of my programs into a loop > > If I list the files in my troublesome directory I see this: > > ls -l /Users/joe/projects/fuse/fuserl-2.0.5/fw > total 8 > drwxr-xr-x@ 6 joe staff 204 Jul 23 2009 bin > drwxr-xr-x@ 3 joe staff 102 Jul 23 2009 build > drwxr-xr-x@ 19 joe staff 646 Jul 23 2009 m4 > drwxr-xr-x@ 4 joe staff 136 Jul 23 2009 package > drwxr-xr-x@ 5 joe staff 170 Jul 23 2009 revision > drwxr-xr-x@ 9 joe staff 306 Jul 23 2009 sh > lrwxr-xr-x 1 joe staff 2 Oct 16 2011 share -> .. > drwxr-xr-x@ 7 joe staff 238 Jul 23 2009 template > > So share is a link to '..' > > But file:read_file_info thinks 'share' is a directory > > > -module(bug). > -compile(export_all). > > -include_lib("kernel/include/file.hrl"). > > test() -> > {ok, X} = > file:read_file_info("/Users/joe/projects/fuse/fuserl-2.0.5/fw/share"), > X#file_info.type. > > Running this > > > bug:test() > directory. > > I'd expected this to return symlink > > This is on a macbook OS X 10.8.3 > > Is this a bug, or have I misunderstood something > I believe file:read_file_info/1,2 will "follow" symlinks and give you information about whatever the link is pointing to. To get information about the link itself use file:read_link_info/1,2 instead. Despite the name, that function will also work on non-links, in which case you get the same information as file:read_file_info/1,2 > Cheers > > /Joe > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- --Dave Peticolas -------------- next part -------------- An HTML attachment was scrubbed... URL: From edgurgel@REDACTED Sun Apr 14 21:40:53 2013 From: edgurgel@REDACTED (Eduardo Gurgel) Date: Sun, 14 Apr 2013 16:40:53 -0300 Subject: [erlang-questions] integrating nodejs with erlang code In-Reply-To: References: <1492815702.54665218.1365608794384.JavaMail.root@erlang-solutions.com> Message-ID: What's the problem with http://bert-rpc.org/ ? -- Eduardo -------------- next part -------------- An HTML attachment was scrubbed... URL: From jozsef.berces@REDACTED Mon Apr 15 05:08:29 2013 From: jozsef.berces@REDACTED (=?iso-8859-1?Q?J=F3zsef_B=E9rces?=) Date: Mon, 15 Apr 2013 03:08:29 +0000 Subject: [erlang-questions] n_fragments Message-ID: <7460EBDDCF52084A849D0F271CE059B80FA4AF@ESESSMB101.ericsson.se> What is the maximum value of mnesia n_fragments? Is it OS dependent or mnesia has a limit? Thanks, Jozsef -------------- next part -------------- An HTML attachment was scrubbed... URL: From mangalaman93@REDACTED Mon Apr 15 05:27:15 2013 From: mangalaman93@REDACTED (aman mangal) Date: Mon, 15 Apr 2013 08:57:15 +0530 Subject: [erlang-questions] [WnCC] Re: integrating nodejs with erlang code In-Reply-To: <0b7c93ea-bdc3-421b-98da-9f9cdee6d460@googlegroups.com> References: <1492815702.54665218.1365608794384.JavaMail.root@erlang-solutions.com> <0b7c93ea-bdc3-421b-98da-9f9cdee6d460@googlegroups.com> Message-ID: Hi, Thanks everyone for their time to reply on this post. Anil, TCP/IP sockets are slow(not really actually) compared to erlang message passing in an erlang cluster. The more important thing is if I use erlang message passing and other distributive features of the language, it will avoid writing the code myself, will be more fault tolerant and more reliable to use. As I cannot do everything in erlang, I am using node.js on the front end side (erlang is on back end side). My application is running in a cluster where any server may have to talk to other for information, I thought that doing the nodejs to erlang communication locally and use message passing among the nodes in the cluster will be better. Thanks once again :) I will try some of the ideas given here and will experiment and choose the one that fits to my requirements. Aman Mangal 3rd year Undergraduate Department of Computer Science & Engineering IIT Bombay www.cse.iitb.ac.in/~amanmangal On Thu, Apr 11, 2013 at 8:33 PM, Anil wrote: > Can you highlight more on why you think its 'slow and inefficient' ? Also > probably give more details on what exactly erlang process and nodejs > process is doing ? > > As Tapan pointed out - using message brokers like ZeroMQ or RabbitMQ is > according to me best way out.. > > On Thursday, April 11, 2013 2:18:15 PM UTC+5:30, tapan pandita wrote: > >> Hey Aman, >> >> Have you considered using ZeroMQ ? >> You can use it over tcp/ip or IPC(if processes are local) and it's pretty >> efficient. It has bindings for many languages including erlang, node.js, >> C++, C, etc. >> >> >> On Thu, Apr 11, 2013 at 12:19 AM, aman mangal wrote: >> >>> Hi, >>> >>> How can I go about implementing JS in erlang? This seems interesting to >>> me. Please tell me what do you mean by implementing JS or node.js in erlang? >>> >>> Aman Mangal >>> 3rd year Undergraduate >>> Department of Computer Science & Engineering >>> IIT Bombay >>> www.cse.iitb.ac.in/~amanmangal >>> >>> >>> On Wed, Apr 10, 2013 at 10:42 PM, Zheng Zhibin wrote: >>> >>>> beam.js would be one way to build node.js into Erlang :) >>>> >>>> Cheers >>>> Ben >>>> >>>> ? 2013-4-10???11:46?Robert Virding >>>> ??? >>>> >>>> The most efficient way of connecting Erlang with node.js would be, of >>>> course, to implement JS and hence node.js in Erlang. Maybe not the easiest >>>> way, but definitely the most efficient connection. :-) >>>> >>>> Robert >>>> >>>> ------------------------------ >>>> >>>> *From: *"Joe Armstrong" >>>> *To: *"Eduardo Gurgel" >>>> *Cc: *"Ali Attarwala" , "Shailesh Tyagi" < >>>> shai...@REDACTED>, "wncc iitb" , "Erlang" >>>> , "chirag jain" >>>> >>>> *Sent: *Wednesday, 10 April, 2013 11:14:19 AM >>>> *Subject: *Re: [erlang-questions] integrating nodejs with erlang code >>>> >>>> >>>> Just curious, but what is the *least efficient* way of connecting >>>> node.js to Erlang. >>>> >>>> If I were doing this I'd start with the least efficient method, build >>>> my app, measure, then optimize if >>>> necessary. >>>> >>>> If the optimization is not necessary you have saved yourself a whole >>>> lot of work. >>>> >>>> Alternatively, write the code and if it is not fast enough wait ten >>>> years. >>>> >>>> >>>> Cheers >>>> >>>> /Joe >>>> >>>> >>>> >>>> >>>> On Tue, Apr 9, 2013 at 2:11 PM, Eduardo Gurgel wrote: >>>> >>>>> >>>>> >>>>> >>>>> On Mon, Apr 8, 2013 at 12:37 PM, aman mangal wrote: >>>>> >>>>>> >>>>>> Hi, >>>>>> >>>>>> What is the most efficient way to pass events/data from erlang >>>>>> processes to nodejs and vice versa? Setting up a tcp socket may be slower >>>>>> or inefficient. I was thinking of doing something native. Both erlang and >>>>>> nodejs provide libraries(NIF) to call C functions but none of them provide >>>>>> an efficient implementation for calling erlang/nodejs functions from C or >>>>>> C++ as far as I know. Please suggest some ideas! >>>>>> >>>>>> >>>>> You could take a look on BERT RPC: >>>>> >>>>> https://github.com/rtomayko/**node-bertrpc >>>>> >>>>> https://github.com/mojombo/**bert.erl >>>>> >>>>> Right now, bert-rpc.org is returning 404, but it was up two days ago: >>>>> >>>>> http://bert-rpc.org/ >>>>> >>>>> Maybe checking cached webpage on bert-rpc.org may help you. >>>>> >>>>> -- >>>>> >>>>> Eduardo >>>>> >>>>> >>>>> >>>>> ______________________________**_________________ >>>>> erlang-questions mailing list >>>>> erlang-q...@REDACTED >>>>> http://erlang.org/mailman/**listinfo/erlang-questions >>>>> >>>>> >>>> >>>> ______________________________**_________________ >>>> erlang-questions mailing list >>>> erlang-q...@REDACTED >>>> http://erlang.org/mailman/**listinfo/erlang-questions >>>> >>>> >>>> ______________________________**_________________ >>>> erlang-questions mailing list >>>> erlang-q...@REDACTED >>>> http://erlang.org/mailman/**listinfo/erlang-questions >>>> >>>> >>>> ______________________________**_________________ >>>> erlang-questions mailing list >>>> erlang-q...@REDACTED >>>> http://erlang.org/mailman/**listinfo/erlang-questions >>>> >>>> >>> -- >>> -- >>> The website for the club is http://stab-iitb.org/wncc >>> To post to this group, send email to wncc...@REDACTED >>> >>> >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Web and Coding Club IIT Bombay" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to wncc_iitb+...@**googlegroups.com. >>> >>> For more options, visit https://groups.google.com/**groups/opt_out >>> . >>> >>> >>> >> >> -- > -- > The website for the club is http://stab-iitb.org/wncc > To post to this group, send email to wncc_iitb@REDACTED > > --- > You received this message because you are subscribed to the Google Groups > "Web and Coding Club IIT Bombay" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to wncc_iitb+unsubscribe@REDACTED > For more options, visit https://groups.google.com/groups/opt_out. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stas@REDACTED Mon Apr 15 06:15:19 2013 From: stas@REDACTED (Stanislav Sedov) Date: Sun, 14 Apr 2013 21:15:19 -0700 Subject: [erlang-questions] symlink error (I think) In-Reply-To: References: Message-ID: <2589932D-3166-460F-8278-50B5FF138B68@freebsd.org> On Apr 14, 2013, at 12:10 PM, Dave Peticolas wrote: > I believe file:read_file_info/1,2 will "follow" symlinks and give you information > about whatever the link is pointing to. To get information about the link itself > use file:read_link_info/1,2 instead. Despite the name, that function will also > work on non-links, in which case you get the same information as > file:read_file_info/1,2 > I think you are right. It's basically the same situation as with stat(2)/lstat(2). read_file_info/read_link_infi functions mimic what stat(2) and lstat(2) do. -- ST4096-RIPE From erlangsiri@REDACTED Mon Apr 15 11:49:19 2013 From: erlangsiri@REDACTED (Siri Hansen) Date: Mon, 15 Apr 2013 11:49:19 +0200 Subject: [erlang-questions] reltool: sys-level {mod_cond, derived} In-Reply-To: References: Message-ID: I think this only reflects the application dependencies as they actually look sasl uses tools tools uses webtool (cover) and inets webtool uses observer inets uses mnesia, runtime_tools, ssl, observer uses gs, wx, et ssl uses crypto and public_key ... Adding {mod_cond,derived} is the way to limit the way reltool calculates the application dependencies. Regards /siri 2013/4/6 Tuncer Ayaz > As discussed previously[1] it was decided to make the default > rebar reltool.config template safer to use by removing[2] the > non-default sys-level mod_cond. > > While this doesn't produce releases with potentially missing > modules, that one-line patch also pulls in many seemingly unused > applications resulting in a release that's at least twice as big: > > With sys-level {mod_cond, derived} (aka non-default settings): > $ ls -1 rel/exemplar/lib > compiler-4.9.1 > crypto-2.3 > erts-5.10.1 > exemplar > hipe-3.10.1 > kernel-2.16.1 > sasl-2.3.1 > stdlib-1.19.1 > tools-2.6.10 > > Without sys-level {mod_cond, derived} (aka default settings): > $ g diff > diff --git a/rel/reltool.config b/rel/reltool.config > index 2dd47be..e65c53c 100644 > --- a/rel/reltool.config > +++ b/rel/reltool.config > @@ -19,7 +19,6 @@ > {boot_rel, "exemplar"}, > {profile, embedded}, > {incl_cond, derived}, > - {mod_cond, derived}, > {excl_archive_filters, [".*"]}, %% Do not archive built libs > {excl_sys_filters, ["^bin/.*", "^erts.*/bin/(dialyzer|typer)", > "^erts.*/(doc|info|include|lib|man|src)"]}, > > $ ls -1 rel/exemplar/lib > asn1-2.0.1 > compiler-4.9.1 > crypto-2.3 > edoc-0.7.12 > erts-5.10.1 > et-1.4.4.3 > exemplar > gs-1.5.15.2 > hipe-3.10.1 > inets-5.9.4 > kernel-2.16.1 > mnesia-4.8 > observer-1.3 > public_key-0.18 > runtime_tools-1.8.10 > sasl-2.3.1 > ssl-5.2.1 > stdlib-1.19.1 > syntax_tools-1.6.11 > tools-2.6.10 > webtool-0.8.9.2 > wx-1.0 > xmerl-1.3.3 > > To reproduce, follow the steps found in > https://github.com/rebar/rebar/wiki/Release-handling > and make sure to delete the sys-level mod_cond. > > If you want to keep the sys-level {mod_cond, derived} and use the > nodetool escript, remember to add the following line: > {app, compiler, [{mod_cond, app}]}, > Otherwise reltool might not include[3] some compiler modules which are > used due to the `-mode(compile)` directive. As I don't think the > nodetool escript has anything to gain by being compiled, I've posted a > patch to delete[4] the directive. > > So, the question is, is this expected reltool behaviour? If so, > is this really desired or not possibly including much more than > is actually used. Shouldn't the sys-level {incl_cond, derived} > make sure only used applications are included (regardless of > mod_cond)? > > [1] http://erlang.org/pipermail/erlang-questions/2012-December/071135.html > [2] https://github.com/rebar/rebar/pull/43 > [3] https://github.com/rebar/rebar/issues/64#issuecomment-15998750 > [4] https://github.com/rebar/rebar/pull/80 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon@REDACTED Mon Apr 15 11:51:43 2013 From: jon@REDACTED (Jon Schneider) Date: Mon, 15 Apr 2013 10:51:43 +0100 Subject: [erlang-questions] symlink error (I think) In-Reply-To: References: Message-ID: <983ea445c246a66fdcca2f5b7d998b19.squirrel@mail.jschneider.net> > I believe file:read_file_info/1,2 will "follow" symlinks and give you > information Clearly there's a job to decide what these functions are _supposed_ to do exactly and fix code and documentation. Jon From erlang@REDACTED Mon Apr 15 12:17:28 2013 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 15 Apr 2013 12:17:28 +0200 Subject: [erlang-questions] file finders Message-ID: Hello, Has anybody written a decent "file finder" in Erlang. Something like the unix 'find' command. I want something that: - traverse the file system recursively - applies a function to any filename that matches a regexp - correctly handles symlinks - should work on all platforms I'd like to follow symlinks, but not when they result in a circular path. I thought I had one, but it turns out not to correctly handle symlinks /Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Mon Apr 15 12:28:15 2013 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 15 Apr 2013 12:28:15 +0200 Subject: [erlang-questions] symlink error (I think) In-Reply-To: <983ea445c246a66fdcca2f5b7d998b19.squirrel@mail.jschneider.net> References: <983ea445c246a66fdcca2f5b7d998b19.squirrel@mail.jschneider.net> Message-ID: I tried this on ubuntu as well: test() -> File = "/home/ejoearm/work/bugs/share", {ok, X1} = file:read_file_info(File), {ok, X2} = file:read_link_info(File), {X1#file_info.type, X2#file_info.type}. > bug:test(). {directory,symlink} So read_file_info does follow symlinks. But the man page says it can return type symlink. Is this a bug in the manual page? Or have I misunderstood something. /Joe On Mon, Apr 15, 2013 at 11:51 AM, Jon Schneider wrote: > > I believe file:read_file_info/1,2 will "follow" symlinks and give you > > information > > Clearly there's a job to decide what these functions are _supposed_ to do > exactly and fix code and documentation. > > Jon > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Mon Apr 15 12:28:45 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 15 Apr 2013 12:28:45 +0200 Subject: [erlang-questions] file finders In-Reply-To: References: Message-ID: <1366021726.4764.21.camel@sekic1152.rnd.ki.sw.ericsson.se> Greetings, How does the existing filelib:fold_files/5 fail your requirements? bengt On Mon, 2013-04-15 at 12:17 +0200, Joe Armstrong wrote: > Hello, > > > Has anybody written a decent "file finder" in Erlang. Something like > the unix 'find' command. > > > I want something that: > > > - traverse the file system recursively > - applies a function to any filename that matches a regexp > - correctly handles symlinks > - should work on all platforms > > > I'd like to follow symlinks, but not when they result in a circular > path. > > > I thought I had one, but it turns out not to correctly handle symlinks > > > /Joe > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bengt.kleberg@REDACTED Mon Apr 15 12:31:50 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 15 Apr 2013 12:31:50 +0200 Subject: [erlang-questions] symlink error (I think) In-Reply-To: <983ea445c246a66fdcca2f5b7d998b19.squirrel@mail.jschneider.net> References: <983ea445c246a66fdcca2f5b7d998b19.squirrel@mail.jschneider.net> Message-ID: <1366021910.4764.23.camel@sekic1152.rnd.ki.sw.ericsson.se> Greetings, The existing documentation seems to be clear on the subject. Differences in the behaviour of the code are bugs. bengt On Mon, 2013-04-15 at 10:51 +0100, Jon Schneider wrote: > > I believe file:read_file_info/1,2 will "follow" symlinks and give you > > information > > Clearly there's a job to decide what these functions are _supposed_ to do > exactly and fix code and documentation. > > Jon > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Mon Apr 15 12:42:43 2013 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 15 Apr 2013 12:42:43 +0200 Subject: [erlang-questions] file finders In-Reply-To: <1366021726.4764.21.camel@sekic1152.rnd.ki.sw.ericsson.se> References: <1366021726.4764.21.camel@sekic1152.rnd.ki.sw.ericsson.se> Message-ID: On Mon, Apr 15, 2013 at 12:28 PM, Bengt Kleberg wrote: > Greetings, > > How does the existing filelib:fold_files/5 fail your requirements? > Because it loops. I have a symlink in a directory pointing to '..' $ ls -l bugs total 12 -rw-r--r-- 1 ejoearm rnd 1168 Apr 15 12:37 bug.beam -rw-r--r-- 1 ejoearm rnd 514 Apr 15 12:36 bug.erl -rw-r--r-- 1 ejoearm rnd 184 Apr 15 12:21 bug.erl~ lrwxrwxrwx 1 ejoearm rnd 2 Apr 15 12:20 share -> .. I tried this test1() -> filelib:fold_files("/home/ejoearm/work", ".*", true, fun visit/2, 0). visit(F, N) -> io:format("~p: ~p~n",[N, F]), N+1. So then I run this I see: 730: "/home/ejoearm/work/bugs/share/bugs/share/bugs/share/bugs/share/bugs/bug.erl~" 731: "/home/ejoearm/work/bugs/share/bugs/share/bugs/share/bugs/share/bugs/bug.beam" 732: "/home/ejoearm/work/bugs/share/bugs/share/bugs/share/bugs/share/rules.erl" 733: "/home/ejoearm/work/bugs/share/bugs/share/bugs/share/bugs/share/test.erl" I only want to follow the symlink once - not multiple times /Joe > > > bengt > > On Mon, 2013-04-15 at 12:17 +0200, Joe Armstrong wrote: > > Hello, > > > > > > Has anybody written a decent "file finder" in Erlang. Something like > > the unix 'find' command. > > > > > > I want something that: > > > > > > - traverse the file system recursively > > - applies a function to any filename that matches a regexp > > - correctly handles symlinks > > - should work on all platforms > > > > > > I'd like to follow symlinks, but not when they result in a circular > > path. > > > > > > I thought I had one, but it turns out not to correctly handle symlinks > > > > > > /Joe > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Mon Apr 15 12:56:23 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 15 Apr 2013 12:56:23 +0200 Subject: [erlang-questions] symlink error (I think) In-Reply-To: References: <983ea445c246a66fdcca2f5b7d998b19.squirrel@mail.jschneider.net> Message-ID: <1366023384.4764.29.camel@sekic1152.rnd.ki.sw.ericsson.se> Greetings, This is a case of "read all of the documentation to understand a part of it". For read_link_info/1-2 it says that it is different from read_file_info/1-2 because read_link_info/1-2 can get "the type field of the record will be set to symlink". If read_file_info/1-2 could have symlink then there would be no difference. It would definitely be better if read_file_info/1-2 also mentioned this. bengt On Mon, 2013-04-15 at 12:28 +0200, Joe Armstrong wrote: > > > I tried this on ubuntu as well: > > test() -> > File = "/home/ejoearm/work/bugs/share", > {ok, X1} = file:read_file_info(File), > {ok, X2} = file:read_link_info(File), > {X1#file_info.type, X2#file_info.type}. > > > > bug:test(). > {directory,symlink} > > > So read_file_info does follow symlinks. But the man page says it can > return type > symlink. Is this a bug in the manual page? Or have I misunderstood > something. > > > /Joe > > > > > > > > > > > On Mon, Apr 15, 2013 at 11:51 AM, Jon Schneider > wrote: > > I believe file:read_file_info/1,2 will "follow" symlinks and > give you > > information > > > Clearly there's a job to decide what these functions are > _supposed_ to do > exactly and fix code and documentation. > > Jon > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bengt.kleberg@REDACTED Mon Apr 15 12:58:25 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 15 Apr 2013 12:58:25 +0200 Subject: [erlang-questions] file finders In-Reply-To: References: <1366021726.4764.21.camel@sekic1152.rnd.ki.sw.ericsson.se> Message-ID: <1366023505.4764.31.camel@sekic1152.rnd.ki.sw.ericsson.se> Would it not be better to file a bug report on filelib:fold_files/5? Maybe even accompanied by a patch? bengt On Mon, 2013-04-15 at 12:42 +0200, Joe Armstrong wrote: > > > > > > On Mon, Apr 15, 2013 at 12:28 PM, Bengt Kleberg > wrote: > Greetings, > > How does the existing filelib:fold_files/5 fail your > requirements? > > > Because it loops. > > > I have a symlink in a directory pointing to '..' > > > $ ls -l bugs > total 12 > -rw-r--r-- 1 ejoearm rnd 1168 Apr 15 12:37 bug.beam > -rw-r--r-- 1 ejoearm rnd 514 Apr 15 12:36 bug.erl > -rw-r--r-- 1 ejoearm rnd 184 Apr 15 12:21 bug.erl~ > lrwxrwxrwx 1 ejoearm rnd 2 Apr 15 12:20 share -> .. > > > I tried this > > > test1() -> > filelib:fold_files("/home/ejoearm/work", > ".*", > true, > fun visit/2, > 0). > > > visit(F, N) -> > io:format("~p: ~p~n",[N, F]), > N+1. > > > So then I run this I see: > > > > > 730: > "/home/ejoearm/work/bugs/share/bugs/share/bugs/share/bugs/share/bugs/bug.erl~" > 731: > "/home/ejoearm/work/bugs/share/bugs/share/bugs/share/bugs/share/bugs/bug.beam" > 732: > "/home/ejoearm/work/bugs/share/bugs/share/bugs/share/bugs/share/rules.erl" > 733: > "/home/ejoearm/work/bugs/share/bugs/share/bugs/share/bugs/share/test.erl" > > > I only want to follow the symlink once - not multiple times > > > /Joe > > > > > > bengt > > On Mon, 2013-04-15 at 12:17 +0200, Joe Armstrong wrote: > > Hello, > > > > > > Has anybody written a decent "file finder" in Erlang. > Something like > > the unix 'find' command. > > > > > > I want something that: > > > > > > - traverse the file system recursively > > - applies a function to any filename that matches a > regexp > > - correctly handles symlinks > > - should work on all platforms > > > > > > I'd like to follow symlinks, but not when they result in a > circular > > path. > > > > > > I thought I had one, but it turns out not to correctly > handle symlinks > > > > > > /Joe > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From jeffschultz@REDACTED Mon Apr 15 15:19:05 2013 From: jeffschultz@REDACTED (Jeff Schultz) Date: Mon, 15 Apr 2013 23:19:05 +1000 Subject: [erlang-questions] symlink error (I think) In-Reply-To: References: <983ea445c246a66fdcca2f5b7d998b19.squirrel@mail.jschneider.net> Message-ID: <516BFE49.9000700@lavabit.com> On 15/04/2013 20:28, Joe Armstrong wrote: > I tried this on ubuntu as well: > > test() -> > File = "/home/ejoearm/work/bugs/share", > {ok, X1} = file:read_file_info(File), > {ok, X2} = file:read_link_info(File), > {X1#file_info.type, X2#file_info.type}. > >> bug:test(). > {directory,symlink} > > So read_file_info does follow symlinks. But the man page says it can return > type > symlink. Is this a bug in the manual page? Or have I misunderstood > something. Just a guess, but it may return symlink if the symlink's target doesn't exist, and the target's type otherwise. If so, that's probably worth pointing out in the documentation. Jeff Schultz From robert.virding@REDACTED Mon Apr 15 17:26:18 2013 From: robert.virding@REDACTED (Robert Virding) Date: Mon, 15 Apr 2013 16:26:18 +0100 (BST) Subject: [erlang-questions] Erlounge in/around Tallahassee next week? In-Reply-To: <518233447.65189176.1366039490618.JavaMail.root@erlang-solutions.com> Message-ID: <1940409300.65191980.1366039578523.JavaMail.root@erlang-solutions.com> I will be in Tallahassee, Fl next week, 21-26/4, to hold a course. Are there any Erlangers in the area that feel up to an informal Erlounge in a suitable pub? Please let me know. Robert From robert.virding@REDACTED Mon Apr 15 17:44:18 2013 From: robert.virding@REDACTED (Robert Virding) Date: Mon, 15 Apr 2013 16:44:18 +0100 (BST) Subject: [erlang-questions] integrating nodejs with erlang code In-Reply-To: Message-ID: <2117436062.65217249.1366040658102.JavaMail.root@erlang-solutions.com> You have two problems, well many problems really but two major problems: - the syntax of JS is "not trivial". It's one of those that look simple until you really examining it. PHP is another case of this. - implementing shared mutable data in Erlang which is rather straight forward but again is "not trivial" to get it efficient. To get some idea of the problem look at my Luerl, Lua implemented in Erlang. First version an interpreter but I have a compiler now. https://github.com/rvirding/luerl Robert ----- Original Message ----- > From: "aman mangal" > To: "Zheng Zhibin" > Cc: "Robert Virding" , "Ali > Attarwala" , "Shailesh Tyagi" > , "wncc iitb" , > "Erlang" , "chirag jain" > > Sent: Wednesday, 10 April, 2013 8:49:03 PM > Subject: Re: [erlang-questions] integrating nodejs with erlang code > Hi, > How can I go about implementing JS in erlang? This seems interesting > to me. Please tell me what do you mean by implementing JS or node.js > in erlang? > Aman Mangal > 3rd year Undergraduate > Department of Computer Science & Engineering > IIT Bombay > www.cse.iitb.ac.in/~amanmangal > On Wed, Apr 10, 2013 at 10:42 PM, Zheng Zhibin < witeman.g@REDACTED > > wrote: > > beam.js would be one way to build node.js into Erlang :) > > > Cheers > > > Ben > > > ? 2013-4-10???11:46?Robert Virding < > > robert.virding@REDACTED > ??? > > > > The most efficient way of connecting Erlang with node.js would > > > be, > > > of > > > course, to implement JS and hence node.js in Erlang. Maybe not > > > the > > > easiest way, but definitely the most efficient connection. :-) > > > > > > Robert > > > > > > > From: "Joe Armstrong" < erlang@REDACTED > > > > > > > > > > > To: "Eduardo Gurgel" < edgurgel@REDACTED > > > > > > > > > > > Cc: "Ali Attarwala" < ali.attar@REDACTED >, "Shailesh Tyagi" > > > > < > > > > shailesh@REDACTED >, "wncc iitb" < > > > > wncc_iitb@REDACTED > > > > >, > > > > "Erlang" < erlang-questions@REDACTED >, "chirag jain" < > > > > chirag.jain@REDACTED > > > > > > > > > > > Sent: Wednesday, 10 April, 2013 11:14:19 AM > > > > > > > > > > Subject: Re: [erlang-questions] integrating nodejs with erlang > > > > code > > > > > > > > > > Just curious, but what is the *least efficient* way of > > > > connecting > > > > node.js to Erlang. > > > > > > > > > > If I were doing this I'd start with the least efficient method, > > > > build > > > > my app, measure, then optimize if > > > > > > > > > > necessary. > > > > > > > > > > If the optimization is not necessary you have saved yourself a > > > > whole > > > > lot of work. > > > > > > > > > > Alternatively, write the code and if it is not fast enough wait > > > > ten > > > > years. > > > > > > > > > > Cheers > > > > > > > > > > /Joe > > > > > > > > > > On Tue, Apr 9, 2013 at 2:11 PM, Eduardo Gurgel < > > > > edgurgel@REDACTED > > > > > > > > > wrote: > > > > > > > > > > > On Mon, Apr 8, 2013 at 12:37 PM, aman mangal < > > > > > mangalaman93@REDACTED > > > > > > wrote: > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > What is the most efficient way to pass events/data from > > > > > > erlang > > > > > > processes to nodejs and vice versa? Setting up a tcp socket > > > > > > may > > > > > > be > > > > > > slower or inefficient. I was thinking of doing something > > > > > > native. > > > > > > Both erlang and nodejs provide libraries(NIF) to call C > > > > > > functions > > > > > > but none of them provide an efficient implementation for > > > > > > calling > > > > > > erlang/nodejs functions from C or C++ as far as I know. > > > > > > Please > > > > > > suggest some ideas! > > > > > > > > > > > > > > > > > > > > You could take a look on BERT RPC: > > > > > > > > > > > > > > > https://github.com/rtomayko/node-bertrpc > > > > > > > > > > > > > > > https://github.com/mojombo/bert.erl > > > > > > > > > > > > > > > Right now, bert-rpc.org is returning 404, but it was up two > > > > > days > > > > > ago: > > > > > > > > > > > > > > > http://bert-rpc.org/ > > > > > > > > > > > > > > > Maybe checking cached webpage on bert-rpc.org may help you. > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > Eduardo > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > > > > > > > > erlang-questions mailing list > > > > > > > > > > > > > > > erlang-questions@REDACTED > > > > > > > > > > > > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > > > erlang-questions mailing list > > > > > > > > > > erlang-questions@REDACTED > > > > > > > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > _______________________________________________ > > > > > > erlang-questions mailing list > > > > > > erlang-questions@REDACTED > > > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Mon Apr 15 17:50:04 2013 From: comptekki@REDACTED (Wes James) Date: Mon, 15 Apr 2013 09:50:04 -0600 Subject: [erlang-questions] symlink error (I think) In-Reply-To: <516BFE49.9000700@lavabit.com> References: <983ea445c246a66fdcca2f5b7d998b19.squirrel@mail.jschneider.net> <516BFE49.9000700@lavabit.com> Message-ID: I created a file zblah.txt and did ln -s zblah.txt zblahblah this is what I get after deleting zblah.txt 1> file:read_file_info("zblahblah"). {error,enoent} 2> file:read_link_info("zblahblah"). {ok,{file_info,9,symlink,none, {{2013,4,15},{9,46,0}}, {{2013,4,15},{9,45,59}}, {{2013,4,15},{9,45,59}}, 41471,1,2053,0,5505639,1000,1000}} wes On Mon, Apr 15, 2013 at 7:19 AM, Jeff Schultz wrote: > On 15/04/2013 20:28, Joe Armstrong wrote: > >> I tried this on ubuntu as well: >> >> test() -> >> File = "/home/ejoearm/work/bugs/**share", >> {ok, X1} = file:read_file_info(File), >> {ok, X2} = file:read_link_info(File), >> {X1#file_info.type, X2#file_info.type}. >> >> bug:test(). >>> >> {directory,symlink} >> >> So read_file_info does follow symlinks. But the man page says it can >> return >> type >> symlink. Is this a bug in the manual page? Or have I misunderstood >> something. >> > > Just a guess, but it may return symlink if the symlink's target doesn't > exist, and the target's type otherwise. > > If so, that's probably worth pointing out in the documentation. > > > Jeff Schultz > > > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangprogram@REDACTED Mon Apr 15 23:02:38 2013 From: erlangprogram@REDACTED (S X) Date: Mon, 15 Apr 2013 17:02:38 -0400 Subject: [erlang-questions] diameter callback module In-Reply-To: References: Message-ID: I think the first question is clear. The diameter library maintains the diameter_app callback module properly with each client calling process. In erlang, each process behaves like an object comparing with the OO programming language. On the other hand, in order to deliver variables, has to pass the values through a series of function calls or through message box. Sometimes it is bit difficult to organize the code nicely. But how to establish a nice notification mechanism, i.e. when the diameter_app callback module "client_cb_ccra" receives the response from the diameter server, how to notify the master process? By using ets table or some generic behaviours? Thanks, Samuel On Sun, Apr 14, 2013 at 1:01 PM, S X wrote: > Hello, > > Based on the erlang diameter library and the sample code, I want to start > multiple diameter client processes in one erlang node(one client IP), and > the client needs to define a diameter_app callback module for certain > application, for example: > > -define(SERVICE(Name), [{'Origin-Host', ?L(Name) ++ ".example.com"}, > {'Origin-Realm', "example.com"}, > {'Vendor-Id', 193}, > {'Product-Name', "Client"}, > {'Auth-Application-Id', [?DIAMETER_APP_ID_COMMON, > ?DIAMETER_APP_ID_CCRA]}, > {application, [{alias, ?APP_CCR_ALIAS}, > {dictionary, ?DIAMETER_DICT_CCRA}, > {module, client_cb_ccra}]}]). > > > First question: > how the diameter library handles this situation? Will all diameter client > processes share one single diameter_app callback module "client_cb_ccra" or > it will automatically attach different instance of the callback module > (process) to the different client process by using spawn_monitor? So from > the callback handle_answer in "client_cb_ccra" I can notify the proper > client proce by just calling client:notify() something. > > Second question: > Notice that diameter:call allows to set extra arguments, so I was > wondering I could set some data like: > diameter:call(Name, ?APP_CCR_ALIAS, CCR, [{extra, [self()]}]). > which sets the client process ID and I hope to deliver to the callback > module "client_cb_ccra" and when the callback module knows which client > process sends request and response accordingly, for example using the > client process ID in the callback function handle_answer. > However, I don't want pack the extra arguments into the diameter packet > since the diameter server doesn't know what they are and the extra ones are > not part of standard diameter packet. > > Now I changed the pick_peer callback signatures to allow extra arguments, > pick_peer([Peer | _], _, _SvcName, _State, A) > > But I got encoding error in the callback prepare_request > > =ERROR REPORT==== 14-Apr-2013::11:40:41 === > Error in process <0.175.0> with exit value: > {undef,[{client_cb_ccra,prepare_request,[{diameter_packet,{diameter_header,1,undefined,undefined,undefined,3958849953,3958849953,undefined,undefined,undefined,undefined},undefined,{diameter_rfc4006_cc_CCR,["who",";","142745567... > > > {error,encode} > > In overall, I read the online documents, which have limited information on > how to use the extra arguments in the library and don't quite get how to > utilize the extra arguments to do something tricky, > > Are there any suggestions on how to deal with multiple client processes? > > Thanks a lot! > > Samuel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangprogram@REDACTED Tue Apr 16 05:51:02 2013 From: erlangprogram@REDACTED (S X) Date: Mon, 15 Apr 2013 23:51:02 -0400 Subject: [erlang-questions] diameter callback module In-Reply-To: References: Message-ID: Maybe using registered process PID can handle the notifications across different modules. Not sue it is a good practice or not. On Mon, Apr 15, 2013 at 5:02 PM, S X wrote: > I think the first question is clear. The diameter library maintains the > diameter_app callback module properly with each client calling process. In > erlang, each process behaves like an object comparing with the OO > programming language. On the other hand, in order to deliver variables, has > to pass the values through a series of function calls or through message > box. Sometimes it is bit difficult to organize the code nicely. > > But how to establish a nice notification mechanism, i.e. when the > diameter_app callback module "client_cb_ccra" receives the response from > the diameter server, how to notify the master process? By using ets table > or some generic behaviours? > > Thanks, > > Samuel > > > On Sun, Apr 14, 2013 at 1:01 PM, S X wrote: > >> Hello, >> >> Based on the erlang diameter library and the sample code, I want to start >> multiple diameter client processes in one erlang node(one client IP), and >> the client needs to define a diameter_app callback module for certain >> application, for example: >> >> -define(SERVICE(Name), [{'Origin-Host', ?L(Name) ++ ".example.com"}, >> {'Origin-Realm', "example.com"}, >> {'Vendor-Id', 193}, >> {'Product-Name', "Client"}, >> {'Auth-Application-Id', [?DIAMETER_APP_ID_COMMON, >> ?DIAMETER_APP_ID_CCRA]}, >> {application, [{alias, ?APP_CCR_ALIAS}, >> {dictionary, ?DIAMETER_DICT_CCRA}, >> {module, client_cb_ccra}]}]). >> >> >> First question: >> how the diameter library handles this situation? Will all diameter client >> processes share one single diameter_app callback module "client_cb_ccra" or >> it will automatically attach different instance of the callback module >> (process) to the different client process by using spawn_monitor? So from >> the callback handle_answer in "client_cb_ccra" I can notify the proper >> client proce by just calling client:notify() something. >> >> Second question: >> Notice that diameter:call allows to set extra arguments, so I was >> wondering I could set some data like: >> diameter:call(Name, ?APP_CCR_ALIAS, CCR, [{extra, [self()]}]). >> which sets the client process ID and I hope to deliver to the callback >> module "client_cb_ccra" and when the callback module knows which client >> process sends request and response accordingly, for example using the >> client process ID in the callback function handle_answer. >> However, I don't want pack the extra arguments into the diameter packet >> since the diameter server doesn't know what they are and the extra ones are >> not part of standard diameter packet. >> >> Now I changed the pick_peer callback signatures to allow extra arguments, >> pick_peer([Peer | _], _, _SvcName, _State, A) >> >> But I got encoding error in the callback prepare_request >> >> =ERROR REPORT==== 14-Apr-2013::11:40:41 === >> Error in process <0.175.0> with exit value: >> {undef,[{client_cb_ccra,prepare_request,[{diameter_packet,{diameter_header,1,undefined,undefined,undefined,3958849953,3958849953,undefined,undefined,undefined,undefined},undefined,{diameter_rfc4006_cc_CCR,["who",";","142745567... >> >> >> {error,encode} >> >> In overall, I read the online documents, which have limited information >> on how to use the extra arguments in the library and don't quite get how >> to utilize the extra arguments to do something tricky, >> >> Are there any suggestions on how to deal with multiple client processes? >> >> Thanks a lot! >> >> Samuel >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenneth.lundin@REDACTED Tue Apr 16 10:05:28 2013 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Tue, 16 Apr 2013 10:05:28 +0200 Subject: [erlang-questions] Unrecognised OID -> fail to decode printableString In-Reply-To: <5164375F.4080302@rabbitmq.com> References: <5164375F.4080302@rabbitmq.com> Message-ID: In the ASN.1 specs relevant to this standard there is an information object set which defines the supported Attribute and Type combinations. For a given Object Identifier there is a certain type of value associated. The value is encoded as an Open Type which is an anonymous sequence of bytes. Given tha mapping these bytes can be decoded in a second step. In this case the oid to type of value mapping is unknown so we can only return a number of bytes i.e a binary. It happens to be some encoded ASN.1 data in thoose bytes but we cannot know that. For the other values in the rdnSequence there is a mapping between OID and type of value and it is typically the type X520name or some other type prefixed X520. The typical for these types is that they are CHOICE types like this: X520name ::= CHOICE { teletexString TeletexString (SIZE (1..ub-name)), printableString PrintableString (SIZE (1..ub-name)), ... } This is why the tuple {printableString,"SomeString"} is returned, the mapping of a CHOICE to Erlang terms is a tuple with the choice alternative name , the value. Without tha mapping between OID and type of value we have no chance of knowing that the Open Type should be interpreted as for example a X520Name, we could only guess and probably do this correct most of the times but not for sure. If we just decode the Open Type we see the tag for PrintableString but we don't know that it is a CHOICE. /Kenneth, Erlang/OTP, Ericsson On Tue, Apr 9, 2013 at 5:44 PM, Simon MacMullen wrote: > With the attached certificate (provided by a user, apparently generated by > "Red Hat Certificate System 7.3") and R16B: > > 1> rr(public_key). > [...] > > 2> {ok, File} = file:read_file("/path/to/cert.**pem"). > {ok,<<"-----BEGIN CERTIFICATE-----"...>>} > > 3> [{'Certificate', Cert, _}] = public_key:pem_decode(File). > [{'Certificate',<<48,130,3,**178,...>>, not_encrypted}] > > 4> #'OTPCertificate'{**tbsCertificate = #'OTPTBSCertificate'{subject = > Subject}} = public_key:pkix_decode_cert(**Cert, otp). > #'OTPCertificate'{...} > > 5> Subject. > {rdnSequence, > [[#'AttributeTypeAndValue'{**type = {2,5,4,6},value = "ES"}], > [#'AttributeTypeAndValue'{ > type = {2,5,4,10}, > value = {printableString,"OMEL"}}], > [#'AttributeTypeAndValue'{ > type = {2,5,4,11}, > value = {printableString,"OM"}}], > [#'AttributeTypeAndValue'{ > type = {2,5,4,3}, > value = {printableString,"DSI PRUEBAS"}}], > [#'AttributeTypeAndValue'{ > type = {0,9,2342,19200300,100,1,1}, > value = <<19,9,68,83,73,83,79,70,84,**57,57>>}]]} > > Umm, why has the last value not been decoded? According to my very very > slack knowledge of BER, 19 = printableString, 9 is the length, and the rest > is the string. And decoding it by hand gives the expected result. > > So why does this value not get decoded? > > Cheers, Simon > > -- > Simon MacMullen > RabbitMQ, VMware > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andre.graf@REDACTED Tue Apr 16 10:39:51 2013 From: andre.graf@REDACTED (=?ISO-8859-1?Q?Andr=E9_Graf?=) Date: Tue, 16 Apr 2013 10:39:51 +0200 Subject: [erlang-questions] diameter callback module In-Reply-To: References: Message-ID: We use a registered gen_server process for that. On 16 April 2013 05:51, S X wrote: > Maybe using registered process PID can handle the notifications across > different modules. Not sue it is a good practice or not. > > > > > On Mon, Apr 15, 2013 at 5:02 PM, S X wrote: >> >> I think the first question is clear. The diameter library maintains the >> diameter_app callback module properly with each client calling process. In >> erlang, each process behaves like an object comparing with the OO >> programming language. On the other hand, in order to deliver variables, has >> to pass the values through a series of function calls or through message >> box. Sometimes it is bit difficult to organize the code nicely. >> >> But how to establish a nice notification mechanism, i.e. when the >> diameter_app callback module "client_cb_ccra" receives the response from the >> diameter server, how to notify the master process? By using ets table or >> some generic behaviours? >> >> Thanks, >> >> Samuel >> >> >> On Sun, Apr 14, 2013 at 1:01 PM, S X wrote: >>> >>> Hello, >>> >>> Based on the erlang diameter library and the sample code, I want to start >>> multiple diameter client processes in one erlang node(one client IP), and >>> the client needs to define a diameter_app callback module for certain >>> application, for example: >>> >>> -define(SERVICE(Name), [{'Origin-Host', ?L(Name) ++ ".example.com"}, >>> {'Origin-Realm', "example.com"}, >>> {'Vendor-Id', 193}, >>> {'Product-Name', "Client"}, >>> {'Auth-Application-Id', [?DIAMETER_APP_ID_COMMON, >>> ?DIAMETER_APP_ID_CCRA]}, >>> {application, [{alias, ?APP_CCR_ALIAS}, >>> {dictionary, ?DIAMETER_DICT_CCRA}, >>> {module, client_cb_ccra}]}]). >>> >>> >>> First question: >>> how the diameter library handles this situation? Will all diameter client >>> processes share one single diameter_app callback module "client_cb_ccra" or >>> it will automatically attach different instance of the callback module >>> (process) to the different client process by using spawn_monitor? So from >>> the callback handle_answer in "client_cb_ccra" I can notify the proper >>> client proce by just calling client:notify() something. >>> >>> Second question: >>> Notice that diameter:call allows to set extra arguments, so I was >>> wondering I could set some data like: >>> diameter:call(Name, ?APP_CCR_ALIAS, CCR, [{extra, [self()]}]). >>> which sets the client process ID and I hope to deliver to the callback >>> module "client_cb_ccra" and when the callback module knows which client >>> process sends request and response accordingly, for example using the client >>> process ID in the callback function handle_answer. >>> However, I don't want pack the extra arguments into the diameter packet >>> since the diameter server doesn't know what they are and the extra ones are >>> not part of standard diameter packet. >>> >>> Now I changed the pick_peer callback signatures to allow extra arguments, >>> pick_peer([Peer | _], _, _SvcName, _State, A) >>> >>> But I got encoding error in the callback prepare_request >>> >>> =ERROR REPORT==== 14-Apr-2013::11:40:41 === >>> Error in process <0.175.0> with exit value: >>> {undef,[{client_cb_ccra,prepare_request,[{diameter_packet,{diameter_header,1,undefined,undefined,undefined,3958849953,3958849953,undefined,undefined,undefined,undefined},undefined,{diameter_rfc4006_cc_CCR,["who",";","142745567... >>> >>> {error,encode} >>> >>> In overall, I read the online documents, which have limited information >>> on how to use the extra arguments in the library and don't quite get how to >>> utilize the extra arguments to do something tricky, >>> >>> Are there any suggestions on how to deal with multiple client processes? >>> >>> Thanks a lot! >>> >>> Samuel >>> >> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From jesper.louis.andersen@REDACTED Tue Apr 16 12:06:21 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 16 Apr 2013 12:06:21 +0200 Subject: [erlang-questions] MySQL driver In-Reply-To: <5638AEFC-E7D7-4CB2-ABCF-250CE2882FE8@eonblast.com> References: <51E1C638-0C5F-4C7F-AB32-7A51123378E8@eonblast.com> <5638AEFC-E7D7-4CB2-ABCF-250CE2882FE8@eonblast.com> Message-ID: <36183CD0-CC60-4547-BDA1-B56693307668@erlang-solutions.com> I may be interested in helping out on this thing as well. I have some work to do changing driver infrastructure to Emysql and when that comes around (in a couple of days/weeks), i'll probably also try to look into some merging of patches if possible. Please keep me in the loop :) Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen On Apr 11, 2013, at 12:41 AM, Henning Diedrich wrote: > Hi Garrett, > > great, I'll check out your connection time outs, thanks a lot! > > It was quite fun to find Jacob, Nick and Fredrik to write this [1] up. It certainly felt more like historical forensics from some point on although I was throughout only trying to find out which driver version made most sense. Which in the end came to light in a way I could not read out of the source. > > If someone would like to share the story of his fork I'll be happy to include it, for the fun of it. Somehow, this all fits MySQL. > > Henning > > [1] https://github.com/eonblast/Emysql#History > > > On Apr 11, 2013, at 12:25 AM, Garrett Smith wrote: > >> Someday someone will write a book on the history of the Erlang MySQL >> libraries. Not a great read, but it'd have to be scholarly. >> >> I've modified the Eonblast library with some connection timeout tweaks >> (as in, supporting them): >> >> https://github.com/gar1t/Emysql >> >> >> >> On Wed, Apr 10, 2013 at 4:59 PM, Dave Smith wrote: >>> Hi Henning, >>> >>> Thanks for the pointers. I must admit that I find the level of snark in the >>> emysql history delightful, considering that it's at least as forked as >>> erlang-mysql-driver now. :) >>> >>> There are a number of good fixes for the emysql driver, but they are on 20 >>> different forks across Github. Looking over csrl's, fixes, I'm pretty >>> concerned those aren't in mainline since there are some non-trivial pooling >>> problems he's addressed (would you like some irony with that irony? :)). >>> >>> Do you know if eonblast is planning to incorporate all the work from those >>> different forks? I think we need to build some momentum behind a driver and >>> I'm certainly happy to direct people towards a maintained one. >>> >>> IMO, it would also be handy if the driver wasn't pooled, but used poolboy >>> since getting the pooling right is rather difficult. (I know that poolboy >>> has been tested with some of Quviq's concurrent checking stuff and a number >>> of race conditions addressed). >>> >>> Thanks! >>> >>> D. >>> >>> >>> On Wed, Apr 10, 2013 at 3:00 PM, Henning Diedrich >>> wrote: >>>> >>>> Hi Dave, >>>> >>>> if I remember right, you were using the driver that came from Yxa via >>>> Yarif's erlang-emysql-driver https://github.com/dizzyd/erlang-mysql-driver ? >>>> (Here is the history of that. https://github.com/eonblast/Emysql#History) >>>> >>>> It was found less than perfect when Nick and Jacob gave it something to >>>> chew building Rupture. I never spotted the problems myself I should add. >>>> That's purely from Nick Gerakine's and Jacob Vorreuter's words. >>>> >>>> But they report that with much effort they could not get the races out of >>>> it. That's why they started emysql, which I later started maintaining, and >>>> which Jeremy just mentioned, https://github.com/eonblast/Emysql >>>> >>>> Considerable work went into getting the pooling system of Emysql right. >>>> (What I don't know is when exactly it got buggy, it may have been after >>>> Jacob's Emysql fork that now lies dormant.) But I worked a while, with help >>>> from others to get all races out of the queuing. >>>> >>>> However, the best fork of Emysql, I *think* is currently that of Chris >>>> Rempel at https://github.com/csrl/emysql as I got behind while making >>>> Erlvolt, the VoltDB driver, modeled in parts on Emysql. Chris did some >>>> impressive clean up. Now that Erlvolt is out, I'll turn around and catch up >>>> asap. But that's not a New York asap. >>>> >>>> An alternative may be to go back to a more original Yxa version, like the >>>> one possibly still to be found in ejabberd. That may be more stable than >>>> Yariv's branch that you started out with. The "history" I linked to above >>>> has the reasons, why that branch may be not quite clean and hard to debug. >>>> >>>> Best, >>>> >>>> Henning >>>> >>>> On Apr 10, 2013, at 10:43 PM, Dave Smith wrote: >>>> >>>>> Hello list, >>>>> >>>>> I'm curious what libraries people are currently using for MySQL access >>>>> in Erlang. I have an old library on my github that I've kept basically >>>>> working, but am wondering if there are any preferred/production libraries >>>>> out there that people like better. >>>>> >>>>> I ask because I'm considering retrofitting the pooling system in the >>>>> driver I maintain to use poolboy, but figure it was worth checking if there >>>>> are better alternatives out there. :) >>>>> >>>>> After all, the best code to maintain is the code you don't maintain. >>>>> >>>>> D. >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From me@REDACTED Tue Apr 16 12:11:03 2013 From: me@REDACTED (just me) Date: Tue, 16 Apr 2013 11:11:03 +0100 Subject: [erlang-questions] [ANN] Contract job offer Message-ID: (forwarding for a friend) Good morning all, Firstly I apologise for my approach to this mailing list, however if one needs, one must ask. I am currently working with a very renowned and innovative mobile/network provider in the UK, on a 6 month contract based in central London. The client are looking for Erlang Developers (4 in total). I would be very interested to hear from any such developers who are interested in a fantastic project with handsome rates. Please do feel free to contact me directly on 0207 928 2525 or hugo.deburghmound@REDACTED / hdeburghmound@REDACTED Again apologise for this approach. Kindest regards, Hugo -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Apr 16 13:18:29 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 16 Apr 2013 13:18:29 +0200 Subject: [erlang-questions] integrating nodejs with erlang code In-Reply-To: References: Message-ID: On Apr 10, 2013, at 8:56 PM, Dave Smith wrote: > There is "most efficient" and "pragmatic". > > My "pragmatic" suggestion would be to run nodejs as a port (e.g. open_port) and use stdio to interact with it. Then you can let Erlang supervise a pool of nodejs VMs. If you want to be fancy, you can implement a ping/pong protocol for detecting when the nodejs system has gone off the deep end chasing down endless chains of callbacks. :) > > I may or may not have had to do this recently. :) > This would be my approach as well. In addition one can define a simple framing format like {packet, 4} and then pass JSON structures to node.js, which it has an easy time working with. Alternatively, you could lean the other way and pass the Erlang term format directly and write a parser for it in node.js. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen From anders.otp@REDACTED Tue Apr 16 13:22:03 2013 From: anders.otp@REDACTED (Anders Svensson) Date: Tue, 16 Apr 2013 13:22:03 +0200 Subject: [erlang-questions] diameter callback module In-Reply-To: References: Message-ID: On Sun, Apr 14, 2013 at 7:01 PM, S X wrote: > Hello, > > Based on the erlang diameter library and the sample code, I want to start > multiple diameter client processes in one erlang node(one client IP), and > the client needs to define a diameter_app callback module for certain > application, for example: > > -define(SERVICE(Name), [{'Origin-Host', ?L(Name) ++ ".example.com"}, > {'Origin-Realm', "example.com"}, > {'Vendor-Id', 193}, > {'Product-Name', "Client"}, > {'Auth-Application-Id', [?DIAMETER_APP_ID_COMMON, > ?DIAMETER_APP_ID_CCRA]}, > {application, [{alias, ?APP_CCR_ALIAS}, > {dictionary, ?DIAMETER_DICT_CCRA}, > {module, client_cb_ccra}]}]). > > > First question: > how the diameter library handles this situation? Will all diameter client > processes share one single diameter_app callback module "client_cb_ccra" or > it will automatically attach different instance of the callback module > (process) to the different client process by using spawn_monitor? So from > the callback handle_answer in "client_cb_ccra" I can notify the proper > client proce by just calling client:notify() something. diameter doesn't know anything about your client process. If you want a callback to be able to contact the client process associated with the service in question (ie. the service whose name the callback gets as an argument) then you need to give the callback the means to do so. One way would be to map the service name to your process (eg. it's the registered name of your client process), another would be to pass some identification as extra arguments to the callbacks. (Eg. {module, [client_cb_ccra, X]} in the config above.) > Second question: > Notice that diameter:call allows to set extra arguments, so I was wondering > I could set some data like: > diameter:call(Name, ?APP_CCR_ALIAS, CCR, [{extra, [self()]}]). > which sets the client process ID and I hope to deliver to the callback > module "client_cb_ccra" and when the callback module knows which client > process sends request and response accordingly, for example using the client > process ID in the callback function handle_answer. You can do that, but what is it you're trying to accomplish? if it's handle_answer that's supposed to communicate something then it already does: the return value of handle_answer is returned by diameter:call. > However, I don't want pack the extra arguments into the diameter packet > since the diameter server doesn't know what they are and the extra ones are > not part of standard diameter packet. Not sure what you mean here. There's no way for your client to send the server anything other than a Diameter message. > Now I changed the pick_peer callback signatures to allow extra arguments, > pick_peer([Peer | _], _, _SvcName, _State, A) > > But I got encoding error in the callback prepare_request > > =ERROR REPORT==== 14-Apr-2013::11:40:41 === > Error in process <0.175.0> with exit value: > {undef,[{client_cb_ccra,prepare_request,[{diameter_packet, The arity of your callback probably doesn't agree with the number of extra arguments you've specified: prepare_request will also get your extra arguments. >{diameter_header,1,undefined,undefined,undefined,3958849953,3958849953,undefined,undefined,undefined,undefined},undefined,{diameter_rfc4006_cc_CCR,["who",";","142745567... > > {error,encode} > > In overall, I read the online documents, which have limited information on > how to use the extra arguments in the library and don't quite get how to > utilize the extra arguments to do something tricky, > > Are there any suggestions on how to deal with multiple client processes? Not sure I understand what problem it is you're trying to solve. Multiple processes invoking diameter:call is nothing strange. You typically just return something useful (eg. the answer message from the peer) from handle_answer, which diameter:call then returns for the caller to deal with. Anders > Thanks a lot! > > Samuel > From jwilliston@REDACTED Tue Apr 16 14:52:11 2013 From: jwilliston@REDACTED (John Williston) Date: Tue, 16 Apr 2013 05:52:11 -0700 Subject: [erlang-questions] Issue with "Programming Erlang" sample Message-ID: <07749302400D04498FF156E66C2815C410B3F15F@DRS-EX.drsystems.local> Good morning, I'm learning Erlang using the Programming Erlang (2d ed) text from Pragmatic Bookshelf. Chapter two, "A Whirlwind Tour of Erlang" includes a file server sample that's failing to work as the book suggests, and I'm afraid I'm not getting anywhere figuring it out. First, here's the server code: -module(afile_server). -export([start/1]). start(Dir) -> spawn(afile_server, loop, [Dir]). loop(Dir) -> receive {Client, list_dir} -> Client ! {self(), file:list_dir(Dir)}; {Client, {get_file, File}} -> Full = filename:join(Dir, File), Client ! {self(), file:read_file(Full)} end, loop(Dir). That compiles (albeit with a warning saying "Warning: function loop/1 is unused") but when I try to execute it from within the shell I get an error. For sake of reference the shell commands I'm issuing are: Eshell V5.10.1 (abort with ^G) 1> c(afile_server). afile_server.erl:7: Warning: function loop/1 is unused {ok,afile_server} 2> FileServer = afile_server:start("."). <0.39.0> 3> =ERROR REPORT==== 10-Apr-2013::10:31:17 === Error in process <0.39.0> with exit value: {undef,[{afile_server,loop,["."],[]}]} Can anyone suggest what might be wrong? I'm running under Windows 7 Professional x64 for sake of reference. Thanks in advance! John -------------- next part -------------- An HTML attachment was scrubbed... URL: From czinkos@REDACTED Tue Apr 16 15:07:24 2013 From: czinkos@REDACTED (Zsolt Czinkos) Date: Tue, 16 Apr 2013 15:07:24 +0200 Subject: [erlang-questions] Issue with "Programming Erlang" sample In-Reply-To: <07749302400D04498FF156E66C2815C410B3F15F@DRS-EX.drsystems.local> References: <07749302400D04498FF156E66C2815C410B3F15F@DRS-EX.drsystems.local> Message-ID: Hi, You should export loop function, too. -export([start/1, loop/1]). Best, Zsolt On Tue, Apr 16, 2013 at 2:52 PM, John Williston wrote: > Good morning,**** > > ** ** > > I?m learning Erlang using the *Programming Erlang* (2d ed) text from > Pragmatic Bookshelf. Chapter two, ?A Whirlwind Tour of Erlang? includes a > file server sample that?s failing to work as the book suggests, and I?m > afraid I?m not getting anywhere figuring it out. First, here?s the server > code:**** > > ** ** > > -module(afile_server).**** > > -export([start/1]).**** > > ** ** > > start(Dir) -> spawn(afile_server, loop, [Dir]).**** > > ** ** > > loop(Dir) ->**** > > receive**** > > {Client, list_dir} ->**** > > Client ! {self(), file:list_dir(Dir)};**** > > {Client, {get_file, File}} ->**** > > Full = filename:join(Dir, File),**** > > Client ! {self(), file:read_file(Full)}**** > > end,**** > > loop(Dir).**** > > ** ** > > That compiles (albeit with a warning saying ?Warning: function loop/1 is > unused?) but when I try to execute it from within the shell I get an error. > For sake of reference the shell commands I?m issuing are:**** > > ** ** > > Eshell V5.10.1 (abort with ^G)**** > > 1> c(afile_server).**** > > afile_server.erl:7: Warning: function loop/1 is unused**** > > {ok,afile_server}**** > > 2> FileServer = afile_server:start(".").**** > > <0.39.0>**** > > 3>**** > > =ERROR REPORT==== 10-Apr-2013::10:31:17 ===**** > > Error in process <0.39.0> with exit value: > {undef,[{afile_server,loop,["."],[]}]}**** > > ** ** > > Can anyone suggest what might be wrong? I?m running under Windows 7 > Professional x64 for sake of reference. Thanks in advance!**** > > ** ** > > John**** > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Tue Apr 16 15:08:20 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 16 Apr 2013 15:08:20 +0200 Subject: [erlang-questions] Issue with "Programming Erlang" sample In-Reply-To: <07749302400D04498FF156E66C2815C410B3F15F@DRS-EX.drsystems.local> References: <07749302400D04498FF156E66C2815C410B3F15F@DRS-EX.drsystems.local> Message-ID: <1366117700.4765.51.camel@sekic1152.rnd.ki.sw.ericsson.se> Greetings, When using the old erlang:spawn/3 you must export the function (the second argument). You may consider erlang:spawn/1 that takes a fun. In your case: start(Dir) -> erlang:spawn( fun() -> loop(Dir) end ). Here no export of loop/1 is needed. bengt On Tue, 2013-04-16 at 05:52 -0700, John Williston wrote: > Good morning, > > > > I?m learning Erlang using the Programming Erlang (2d ed) text from > Pragmatic Bookshelf. Chapter two, ?A Whirlwind Tour of Erlang? > includes a file server sample that?s failing to work as the book > suggests, and I?m afraid I?m not getting anywhere figuring it out. > First, here?s the server code: > > > > -module(afile_server). > > -export([start/1]). > > > > start(Dir) -> spawn(afile_server, loop, [Dir]). > > > > loop(Dir) -> > > receive > > {Client, list_dir} -> > > Client ! {self(), file:list_dir(Dir)}; > > {Client, {get_file, File}} -> > > Full = filename:join(Dir, File), > > Client ! {self(), file:read_file(Full)} > > end, > > loop(Dir). > > > > That compiles (albeit with a warning saying ?Warning: function loop/1 > is unused?) but when I try to execute it from within the shell I get > an error. For sake of reference the shell commands I?m issuing are: > > > > Eshell V5.10.1 (abort with ^G) > > 1> c(afile_server). > > afile_server.erl:7: Warning: function loop/1 is unused > > {ok,afile_server} > > 2> FileServer = afile_server:start("."). > > <0.39.0> > > 3> > > =ERROR REPORT==== 10-Apr-2013::10:31:17 === > > Error in process <0.39.0> with exit value: > {undef,[{afile_server,loop,["."],[]}]} > > > > Can anyone suggest what might be wrong? I?m running under Windows 7 > Professional x64 for sake of reference. Thanks in advance! > > > > John > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From carlo.bertoldi@REDACTED Tue Apr 16 15:08:38 2013 From: carlo.bertoldi@REDACTED (Carlo Bertoldi) Date: Tue, 16 Apr 2013 15:08:38 +0200 Subject: [erlang-questions] Issue with "Programming Erlang" sample In-Reply-To: <07749302400D04498FF156E66C2815C410B3F15F@DRS-EX.drsystems.local> References: <07749302400D04498FF156E66C2815C410B3F15F@DRS-EX.drsystems.local> Message-ID: <516D4D56.5020208@ubiquity.it> On 16/04/13 14:52, John Williston wrote: > Good morning, Hi, you should change the export line to this -export([start/1, loop/1]). Since start is using spwan to call the loop function, it must be exported. Otherwise, as the error says, the function can't be used from outside the module. > =ERROR REPORT==== 10-Apr-2013::10:31:17 === > > Error in process <0.39.0> with exit value: > {undef,[{afile_server,loop,["."],[]}]} > > Can anyone suggest what might be wrong? I?m running under Windows 7 > Professional x64 for sake of reference. Thanks in advance! > > John > > Carlo From jdevivo@REDACTED Tue Apr 16 15:58:42 2013 From: jdevivo@REDACTED (Joe DeVivo) Date: Tue, 16 Apr 2013 06:58:42 -0700 Subject: [erlang-questions] Unexpected behavior with -remsh via open_port Message-ID: Hello! I've recently seen some unexpected behavior when trying to open_port to `erl -remsh`. You can see a more detailed version here (https://gist.github.com/joedevivo/5389301) but the interesting bit is below. Assume that 'node@REDACTED' is already running with the same cookie. 1> P = open_port({spawn, "erl -name local@REDACTED -hidden -remsh node@REDACTED -setcookie joe"}, 1> [stream, use_stdio, exit_status, binary, stderr_to_stdout]). #Port<0.517> 2> flush(). Shell got {#Port<0.517>,{data,<<"Eshell V5.9.3.1 (abort with ^G)\n">>}} Shell got {#Port<0.517>,{data,<<"(local@REDACTED)1> ">>}} As you can see, opening the port prompts with 'local@REDACTED'. If I had just run that command from my shell, it would prompt me with 'node@REDACTED'. Does anyone know of a way I can get -remsh to work correctly with open_port? Thanks, --joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmercer@REDACTED Tue Apr 16 17:05:11 2013 From: dmercer@REDACTED (David Mercer) Date: Tue, 16 Apr 2013 10:05:11 -0500 Subject: [erlang-questions] Erlang for Big Data Message-ID: <01d301ce3ab3$cb30e7e0$6192b7a0$@gmail.com> I was at the Big Data Innovation Summit last week, learning about approaches to Big Data. I?m sure there are many more knowledgeable people than I on this list, but if I understand things correctly, Big Data is distinguished from just plain data by its need to be processed on a distributed platform. Hadoop seems to be the distributed platform of choice in managing Big Data, and I was wondering if anyone knows whether Erlang plays at all into the Big Data universe. On the face of it, it seems Erlang could be most useful in managing distributed clusters of data processing. Anyone have any thoughts or experience in this area? Cheers, DBM -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwertymaniac@REDACTED Tue Apr 16 17:12:22 2013 From: qwertymaniac@REDACTED (Harsh J) Date: Tue, 16 Apr 2013 20:42:22 +0530 Subject: [erlang-questions] Erlang for Big Data In-Reply-To: <01d301ce3ab3$cb30e7e0$6192b7a0$@gmail.com> References: <01d301ce3ab3$cb30e7e0$6192b7a0$@gmail.com> Message-ID: There is Disco from Nokia for an MR plus DFS implemented with Erlang. IIRC it also has a BigTable equivalent. Riak is a popular one as well. On Apr 16, 2013 8:33 PM, "David Mercer" wrote: > I was at the Big Data Innovation Summit last week, learning about > approaches to Big Data. I?m sure there are many more knowledgeable people > than I on this list, but if I understand things correctly, Big Data is > distinguished from just plain data by its need to be processed on a > distributed platform. Hadoop seems to be the distributed platform of > choice in managing Big Data, and I was wondering if anyone knows whether > Erlang plays at all into the Big Data universe. On the face of it, it > seems Erlang could be most useful in managing distributed clusters of data > processing. Anyone have any thoughts or experience in this area?**** > > ** ** > > Cheers,**** > > ** ** > > DBM**** > > ** ** > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon@REDACTED Tue Apr 16 20:57:08 2013 From: jon@REDACTED (Jonathan Schneider) Date: Tue, 16 Apr 2013 19:57:08 +0100 Subject: [erlang-questions] file:read_file_info and fstat Message-ID: <09DE4B39-3F52-4E3E-86D0-8268B9E11A62@axismilton.ltd.uk> I've knocked up a file monitor thing. To cope with rotating logs it would be really good if I could do read_file_info on an IoDevice rather like fstat() in the C library. Is there such a function or could read_file_info be enhanced ? Jon From pablo.polvorin@REDACTED Tue Apr 16 22:07:43 2013 From: pablo.polvorin@REDACTED (Pablo Polvorin) Date: Tue, 16 Apr 2013 17:07:43 -0300 Subject: [erlang-questions] MySQL driver In-Reply-To: <36183CD0-CC60-4547-BDA1-B56693307668@erlang-solutions.com> References: <51E1C638-0C5F-4C7F-AB32-7A51123378E8@eonblast.com> <5638AEFC-E7D7-4CB2-ABCF-250CE2882FE8@eonblast.com> <36183CD0-CC60-4547-BDA1-B56693307668@erlang-solutions.com> Message-ID: Is there a driver that supports the binary mysql protocol (not the textual, sql one) for prepared statements?. I come in a situation recently on a query where we have to bind lots of parameters into a prepared statement, and the time to escape all of them by the driver and send the corresponding SET @N = .. was substantial, actually larger than the time it takes mysql to run the query. Using the binary protocol in this situation would help. I found http://erlmysql.sourceforge.net/ that do support the binary protocol, and although it was much faster on this particular query (order of magnitudes faster) it sometimes crashed and I found the code more complex than other mysql drivers I've worked with, both reason to make me nervous about using it. Is anyone using it on production?, do you know alternatives? * * On 16 April 2013 07:06, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > I may be interested in helping out on this thing as well. I have some work > to do changing driver infrastructure to Emysql and when that comes around > (in a couple of days/weeks), i'll probably also try to look into some > merging of patches if possible. > > Please keep me in the loop :) > > Jesper Louis Andersen > Erlang Solutions Ltd., Copenhagen > > > > On Apr 11, 2013, at 12:41 AM, Henning Diedrich > wrote: > > > Hi Garrett, > > > > great, I'll check out your connection time outs, thanks a lot! > > > > It was quite fun to find Jacob, Nick and Fredrik to write this [1] up. > It certainly felt more like historical forensics from some point on > although I was throughout only trying to find out which driver version made > most sense. Which in the end came to light in a way I could not read out of > the source. > > > > If someone would like to share the story of his fork I'll be happy to > include it, for the fun of it. Somehow, this all fits MySQL. > > > > Henning > > > > [1] https://github.com/eonblast/Emysql#History > > > > > > On Apr 11, 2013, at 12:25 AM, Garrett Smith wrote: > > > >> Someday someone will write a book on the history of the Erlang MySQL > >> libraries. Not a great read, but it'd have to be scholarly. > >> > >> I've modified the Eonblast library with some connection timeout tweaks > >> (as in, supporting them): > >> > >> https://github.com/gar1t/Emysql > >> > >> > >> > >> On Wed, Apr 10, 2013 at 4:59 PM, Dave Smith wrote: > >>> Hi Henning, > >>> > >>> Thanks for the pointers. I must admit that I find the level of snark > in the > >>> emysql history delightful, considering that it's at least as forked as > >>> erlang-mysql-driver now. :) > >>> > >>> There are a number of good fixes for the emysql driver, but they are > on 20 > >>> different forks across Github. Looking over csrl's, fixes, I'm pretty > >>> concerned those aren't in mainline since there are some non-trivial > pooling > >>> problems he's addressed (would you like some irony with that irony? > :)). > >>> > >>> Do you know if eonblast is planning to incorporate all the work from > those > >>> different forks? I think we need to build some momentum behind a > driver and > >>> I'm certainly happy to direct people towards a maintained one. > >>> > >>> IMO, it would also be handy if the driver wasn't pooled, but used > poolboy > >>> since getting the pooling right is rather difficult. (I know that > poolboy > >>> has been tested with some of Quviq's concurrent checking stuff and a > number > >>> of race conditions addressed). > >>> > >>> Thanks! > >>> > >>> D. > >>> > >>> > >>> On Wed, Apr 10, 2013 at 3:00 PM, Henning Diedrich > > >>> wrote: > >>>> > >>>> Hi Dave, > >>>> > >>>> if I remember right, you were using the driver that came from Yxa via > >>>> Yarif's erlang-emysql-driver > https://github.com/dizzyd/erlang-mysql-driver ? > >>>> (Here is the history of that. > https://github.com/eonblast/Emysql#History) > >>>> > >>>> It was found less than perfect when Nick and Jacob gave it something > to > >>>> chew building Rupture. I never spotted the problems myself I should > add. > >>>> That's purely from Nick Gerakine's and Jacob Vorreuter's words. > >>>> > >>>> But they report that with much effort they could not get the races > out of > >>>> it. That's why they started emysql, which I later started > maintaining, and > >>>> which Jeremy just mentioned, https://github.com/eonblast/Emysql > >>>> > >>>> Considerable work went into getting the pooling system of Emysql > right. > >>>> (What I don't know is when exactly it got buggy, it may have been > after > >>>> Jacob's Emysql fork that now lies dormant.) But I worked a while, > with help > >>>> from others to get all races out of the queuing. > >>>> > >>>> However, the best fork of Emysql, I *think* is currently that of Chris > >>>> Rempel at https://github.com/csrl/emysql as I got behind while making > >>>> Erlvolt, the VoltDB driver, modeled in parts on Emysql. Chris did some > >>>> impressive clean up. Now that Erlvolt is out, I'll turn around and > catch up > >>>> asap. But that's not a New York asap. > >>>> > >>>> An alternative may be to go back to a more original Yxa version, like > the > >>>> one possibly still to be found in ejabberd. That may be more stable > than > >>>> Yariv's branch that you started out with. The "history" I linked to > above > >>>> has the reasons, why that branch may be not quite clean and hard to > debug. > >>>> > >>>> Best, > >>>> > >>>> Henning > >>>> > >>>> On Apr 10, 2013, at 10:43 PM, Dave Smith wrote: > >>>> > >>>>> Hello list, > >>>>> > >>>>> I'm curious what libraries people are currently using for MySQL > access > >>>>> in Erlang. I have an old library on my github that I've kept > basically > >>>>> working, but am wondering if there are any preferred/production > libraries > >>>>> out there that people like better. > >>>>> > >>>>> I ask because I'm considering retrofitting the pooling system in the > >>>>> driver I maintain to use poolboy, but figure it was worth checking > if there > >>>>> are better alternatives out there. :) > >>>>> > >>>>> After all, the best code to maintain is the code you don't maintain. > >>>>> > >>>>> D. > >>>>> _______________________________________________ > >>>>> erlang-questions mailing list > >>>>> erlang-questions@REDACTED > >>>>> http://erlang.org/mailman/listinfo/erlang-questions > >>>> > >>> > >>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >>> > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Pablo Polvorin ProcessOne -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue Apr 16 22:43:29 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 16 Apr 2013 22:43:29 +0200 Subject: [erlang-questions] MySQL driver In-Reply-To: References: <51E1C638-0C5F-4C7F-AB32-7A51123378E8@eonblast.com> <5638AEFC-E7D7-4CB2-ABCF-250CE2882FE8@eonblast.com> <36183CD0-CC60-4547-BDA1-B56693307668@erlang-solutions.com> Message-ID: <516DB7F1.4080702@ninenines.eu> bank_mysql has binary prepared statements: https://github.com/extend/bank_mysql You can use it with the bank pool (https://github.com/extend/bank) or without, whichever you prefer. No user guide at this time but it should be pretty easy to read. On 04/16/2013 10:07 PM, Pablo Polvorin wrote: > Is there a driver that supports the binary mysql protocol (not the > textual, sql one) for prepared statements?. > > I come in a situation recently on a query where we have to bind lots of > parameters into a prepared statement, and the time to escape all of > them by the driver and send the corresponding SET @N = .. was > substantial, actually larger than the time it takes mysql to run the > query. Using the binary protocol in this situation would help. > > I found http://erlmysql.sourceforge.net/ that do support the binary > protocol, and although it was much faster on this particular query > (order of magnitudes faster) it sometimes crashed and I found the code > more complex than other mysql drivers I've worked with, both reason to > make me nervous about using it. Is anyone using it on production?, do > you know alternatives? > > > *| > |* > > > > > On 16 April 2013 07:06, Jesper Louis Andersen > > wrote: > > I may be interested in helping out on this thing as well. I have > some work to do changing driver infrastructure to Emysql and when > that comes around (in a couple of days/weeks), i'll probably also > try to look into some merging of patches if possible. > > Please keep me in the loop :) > > Jesper Louis Andersen > Erlang Solutions Ltd., Copenhagen > > > > On Apr 11, 2013, at 12:41 AM, Henning Diedrich > wrote: > > > Hi Garrett, > > > > great, I'll check out your connection time outs, thanks a lot! > > > > It was quite fun to find Jacob, Nick and Fredrik to write this > [1] up. It certainly felt more like historical forensics from some > point on although I was throughout only trying to find out which > driver version made most sense. Which in the end came to light in a > way I could not read out of the source. > > > > If someone would like to share the story of his fork I'll be > happy to include it, for the fun of it. Somehow, this all fits MySQL. > > > > Henning > > > > [1] https://github.com/eonblast/Emysql#History > > > > > > On Apr 11, 2013, at 12:25 AM, Garrett Smith > wrote: > > > >> Someday someone will write a book on the history of the Erlang MySQL > >> libraries. Not a great read, but it'd have to be scholarly. > >> > >> I've modified the Eonblast library with some connection timeout > tweaks > >> (as in, supporting them): > >> > >> https://github.com/gar1t/Emysql > >> > >> > >> > >> On Wed, Apr 10, 2013 at 4:59 PM, Dave Smith > wrote: > >>> Hi Henning, > >>> > >>> Thanks for the pointers. I must admit that I find the level of > snark in the > >>> emysql history delightful, considering that it's at least as > forked as > >>> erlang-mysql-driver now. :) > >>> > >>> There are a number of good fixes for the emysql driver, but > they are on 20 > >>> different forks across Github. Looking over csrl's, fixes, I'm > pretty > >>> concerned those aren't in mainline since there are some > non-trivial pooling > >>> problems he's addressed (would you like some irony with that > irony? :)). > >>> > >>> Do you know if eonblast is planning to incorporate all the work > from those > >>> different forks? I think we need to build some momentum behind > a driver and > >>> I'm certainly happy to direct people towards a maintained one. > >>> > >>> IMO, it would also be handy if the driver wasn't pooled, but > used poolboy > >>> since getting the pooling right is rather difficult. (I know > that poolboy > >>> has been tested with some of Quviq's concurrent checking stuff > and a number > >>> of race conditions addressed). > >>> > >>> Thanks! > >>> > >>> D. > >>> > >>> > >>> On Wed, Apr 10, 2013 at 3:00 PM, Henning Diedrich > > > >>> wrote: > >>>> > >>>> Hi Dave, > >>>> > >>>> if I remember right, you were using the driver that came from > Yxa via > >>>> Yarif's erlang-emysql-driver > https://github.com/dizzyd/erlang-mysql-driver ? > >>>> (Here is the history of that. > https://github.com/eonblast/Emysql#History) > >>>> > >>>> It was found less than perfect when Nick and Jacob gave it > something to > >>>> chew building Rupture. I never spotted the problems myself I > should add. > >>>> That's purely from Nick Gerakine's and Jacob Vorreuter's words. > >>>> > >>>> But they report that with much effort they could not get the > races out of > >>>> it. That's why they started emysql, which I later started > maintaining, and > >>>> which Jeremy just mentioned, https://github.com/eonblast/Emysql > >>>> > >>>> Considerable work went into getting the pooling system of > Emysql right. > >>>> (What I don't know is when exactly it got buggy, it may have > been after > >>>> Jacob's Emysql fork that now lies dormant.) But I worked a > while, with help > >>>> from others to get all races out of the queuing. > >>>> > >>>> However, the best fork of Emysql, I *think* is currently that > of Chris > >>>> Rempel at https://github.com/csrl/emysql as I got behind while > making > >>>> Erlvolt, the VoltDB driver, modeled in parts on Emysql. Chris > did some > >>>> impressive clean up. Now that Erlvolt is out, I'll turn around > and catch up > >>>> asap. But that's not a New York asap. > >>>> > >>>> An alternative may be to go back to a more original Yxa > version, like the > >>>> one possibly still to be found in ejabberd. That may be more > stable than > >>>> Yariv's branch that you started out with. The "history" I > linked to above > >>>> has the reasons, why that branch may be not quite clean and > hard to debug. > >>>> > >>>> Best, > >>>> > >>>> Henning > >>>> > >>>> On Apr 10, 2013, at 10:43 PM, Dave Smith > wrote: > >>>> > >>>>> Hello list, > >>>>> > >>>>> I'm curious what libraries people are currently using for > MySQL access > >>>>> in Erlang. I have an old library on my github that I've kept > basically > >>>>> working, but am wondering if there are any > preferred/production libraries > >>>>> out there that people like better. > >>>>> > >>>>> I ask because I'm considering retrofitting the pooling system > in the > >>>>> driver I maintain to use poolboy, but figure it was worth > checking if there > >>>>> are better alternatives out there. :) > >>>>> > >>>>> After all, the best code to maintain is the code you don't > maintain. > >>>>> > >>>>> D. > >>>>> _______________________________________________ > >>>>> erlang-questions mailing list > >>>>> erlang-questions@REDACTED > >>>>> http://erlang.org/mailman/listinfo/erlang-questions > >>>> > >>> > >>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >>> > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Pablo Polvorin > ProcessOne > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From robert.virding@REDACTED Tue Apr 16 23:26:15 2013 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 16 Apr 2013 22:26:15 +0100 (BST) Subject: [erlang-questions] Erlounge in/around Tallahassee next week? In-Reply-To: <1940409300.65191980.1366039578523.JavaMail.root@erlang-solutions.com> Message-ID: <1798427933.67452191.1366147574990.JavaMail.root@erlang-solutions.com> Things are looking up, I have got some feedback on this so hopefully we can arrange something. Any more interested? Robert ----- Original Message ----- > From: "Robert Virding" > To: "Erlang Questions" > Sent: Monday, 15 April, 2013 5:26:18 PM > Subject: Erlounge in/around Tallahassee next week? > > I will be in Tallahassee, Fl next week, 21-26/4, to hold a course. > Are there any Erlangers in the area that feel up to an informal > Erlounge in a suitable pub? Please let me know. > > Robert > From pablo.platt@REDACTED Wed Apr 17 01:38:21 2013 From: pablo.platt@REDACTED (pablo platt) Date: Wed, 17 Apr 2013 02:38:21 +0300 Subject: [erlang-questions] Listen to SSL and normal TCP connections on the same port Message-ID: Hi, Is it possible to accept SSL connections and normal TCP connections on the same port? Maybe accept normal TCP connections. If the connection is SSL connection pass it to the ssl module and if it is a normal TCP connection just handle the socket. If it is possible, how can I distinct SSL from non SSL connections? How can I pass the socket to the ssl module? In my case I control the server but I can't change the client. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablo.polvorin@REDACTED Wed Apr 17 04:31:23 2013 From: pablo.polvorin@REDACTED (Pablo Polvorin) Date: Tue, 16 Apr 2013 23:31:23 -0300 Subject: [erlang-questions] MySQL driver In-Reply-To: <516DB7F1.4080702@ninenines.eu> References: <51E1C638-0C5F-4C7F-AB32-7A51123378E8@eonblast.com> <5638AEFC-E7D7-4CB2-ABCF-250CE2882FE8@eonblast.com> <36183CD0-CC60-4547-BDA1-B56693307668@erlang-solutions.com> <516DB7F1.4080702@ninenines.eu> Message-ID: looks good, I'll try it, thanks! On 16 April 2013 17:43, Lo?c Hoguin wrote: > bank_mysql has binary prepared statements: https://github.com/extend/** > bank_mysql > > You can use it with the bank pool (https://github.com/extend/**bank) > or without, whichever you prefer. > > No user guide at this time but it should be pretty easy to read. > > > On 04/16/2013 10:07 PM, Pablo Polvorin wrote: > >> Is there a driver that supports the binary mysql protocol (not the >> textual, sql one) for prepared statements?. >> >> I come in a situation recently on a query where we have to bind lots of >> parameters into a prepared statement, and the time to escape all of >> them by the driver and send the corresponding SET @N = .. was >> substantial, actually larger than the time it takes mysql to run the >> query. Using the binary protocol in this situation would help. >> >> I found http://erlmysql.sourceforge.**net/ that do support the binary >> protocol, and although it was much faster on this particular query >> (order of magnitudes faster) it sometimes crashed and I found the code >> more complex than other mysql drivers I've worked with, both reason to >> make me nervous about using it. Is anyone using it on production?, do >> you know alternatives? >> >> >> *| >> >> |* >> >> >> >> >> On 16 April 2013 07:06, Jesper Louis Andersen >> >> >> >> wrote: >> >> I may be interested in helping out on this thing as well. I have >> some work to do changing driver infrastructure to Emysql and when >> that comes around (in a couple of days/weeks), i'll probably also >> try to look into some merging of patches if possible. >> >> Please keep me in the loop :) >> >> Jesper Louis Andersen >> Erlang Solutions Ltd., Copenhagen >> >> >> >> On Apr 11, 2013, at 12:41 AM, Henning Diedrich > > wrote: >> >> > Hi Garrett, >> > >> > great, I'll check out your connection time outs, thanks a lot! >> > >> > It was quite fun to find Jacob, Nick and Fredrik to write this >> [1] up. It certainly felt more like historical forensics from some >> point on although I was throughout only trying to find out which >> driver version made most sense. Which in the end came to light in a >> way I could not read out of the source. >> > >> > If someone would like to share the story of his fork I'll be >> happy to include it, for the fun of it. Somehow, this all fits MySQL. >> > >> > Henning >> > >> > [1] https://github.com/eonblast/**Emysql#History >> > >> > >> > On Apr 11, 2013, at 12:25 AM, Garrett Smith > > wrote: >> > >> >> Someday someone will write a book on the history of the Erlang >> MySQL >> >> libraries. Not a great read, but it'd have to be scholarly. >> >> >> >> I've modified the Eonblast library with some connection timeout >> tweaks >> >> (as in, supporting them): >> >> >> >> https://github.com/gar1t/**Emysql >> >> >> >> >> >> >> >> On Wed, Apr 10, 2013 at 4:59 PM, Dave Smith > > wrote: >> >>> Hi Henning, >> >>> >> >>> Thanks for the pointers. I must admit that I find the level of >> snark in the >> >>> emysql history delightful, considering that it's at least as >> forked as >> >>> erlang-mysql-driver now. :) >> >>> >> >>> There are a number of good fixes for the emysql driver, but >> they are on 20 >> >>> different forks across Github. Looking over csrl's, fixes, I'm >> pretty >> >>> concerned those aren't in mainline since there are some >> non-trivial pooling >> >>> problems he's addressed (would you like some irony with that >> irony? :)). >> >>> >> >>> Do you know if eonblast is planning to incorporate all the work >> from those >> >>> different forks? I think we need to build some momentum behind >> a driver and >> >>> I'm certainly happy to direct people towards a maintained one. >> >>> >> >>> IMO, it would also be handy if the driver wasn't pooled, but >> used poolboy >> >>> since getting the pooling right is rather difficult. (I know >> that poolboy >> >>> has been tested with some of Quviq's concurrent checking stuff >> and a number >> >>> of race conditions addressed). >> >>> >> >>> Thanks! >> >>> >> >>> D. >> >>> >> >>> >> >>> On Wed, Apr 10, 2013 at 3:00 PM, Henning Diedrich >> > >> >> >>> wrote: >> >>>> >> >>>> Hi Dave, >> >>>> >> >>>> if I remember right, you were using the driver that came from >> Yxa via >> >>>> Yarif's erlang-emysql-driver >> https://github.com/dizzyd/**erlang-mysql-driver? >> >>>> (Here is the history of that. >> https://github.com/eonblast/**Emysql#History >> ) >> >>>> >> >>>> It was found less than perfect when Nick and Jacob gave it >> something to >> >>>> chew building Rupture. I never spotted the problems myself I >> should add. >> >>>> That's purely from Nick Gerakine's and Jacob Vorreuter's words. >> >>>> >> >>>> But they report that with much effort they could not get the >> races out of >> >>>> it. That's why they started emysql, which I later started >> maintaining, and >> >>>> which Jeremy just mentioned, https://github.com/eonblast/** >> Emysql >> >>>> >> >>>> Considerable work went into getting the pooling system of >> Emysql right. >> >>>> (What I don't know is when exactly it got buggy, it may have >> been after >> >>>> Jacob's Emysql fork that now lies dormant.) But I worked a >> while, with help >> >>>> from others to get all races out of the queuing. >> >>>> >> >>>> However, the best fork of Emysql, I *think* is currently that >> of Chris >> >>>> Rempel at https://github.com/csrl/emysql as I got behind while >> making >> >>>> Erlvolt, the VoltDB driver, modeled in parts on Emysql. Chris >> did some >> >>>> impressive clean up. Now that Erlvolt is out, I'll turn around >> and catch up >> >>>> asap. But that's not a New York asap. >> >>>> >> >>>> An alternative may be to go back to a more original Yxa >> version, like the >> >>>> one possibly still to be found in ejabberd. That may be more >> stable than >> >>>> Yariv's branch that you started out with. The "history" I >> linked to above >> >>>> has the reasons, why that branch may be not quite clean and >> hard to debug. >> >>>> >> >>>> Best, >> >>>> >> >>>> Henning >> >>>> >> >>>> On Apr 10, 2013, at 10:43 PM, Dave Smith > > wrote: >> >>>> >> >>>>> Hello list, >> >>>>> >> >>>>> I'm curious what libraries people are currently using for >> MySQL access >> >>>>> in Erlang. I have an old library on my github that I've kept >> basically >> >>>>> working, but am wondering if there are any >> preferred/production libraries >> >>>>> out there that people like better. >> >>>>> >> >>>>> I ask because I'm considering retrofitting the pooling system >> in the >> >>>>> driver I maintain to use poolboy, but figure it was worth >> checking if there >> >>>>> are better alternatives out there. :) >> >>>>> >> >>>>> After all, the best code to maintain is the code you don't >> maintain. >> >>>>> >> >>>>> D. >> >>>>> ______________________________**_________________ >> >>>>> erlang-questions mailing list >> >>>>> erlang-questions@REDACTED > erlang.org > >> >> >>>>> http://erlang.org/mailman/**listinfo/erlang-questions >> >>>> >> >>> >> >>> >> >>> ______________________________**_________________ >> >>> erlang-questions mailing list >> >>> erlang-questions@REDACTED > erlang.org > >> >> >>> http://erlang.org/mailman/**listinfo/erlang-questions >> >>> >> > >> > ______________________________**_________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > >> >> > http://erlang.org/mailman/**listinfo/erlang-questions >> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > >> >> http://erlang.org/mailman/**listinfo/erlang-questions >> >> >> >> >> -- >> Pablo Polvorin >> ProcessOne >> >> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> >> > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > -- Pablo Polvorin ProcessOne -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Wed Apr 17 09:08:24 2013 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 17 Apr 2013 09:08:24 +0200 Subject: [erlang-questions] Missing module create_certs in ssl-5.2.1 In-Reply-To: <6CA60E3A41E34D4A851816F9BA44F76A3419B1D4@EXMB13TSTRZ2.tcad.telia.se> References: <6CA60E3A41E34D4A851816F9BA44F76A3419B1D4@EXMB13TSTRZ2.tcad.telia.se> Message-ID: Hello! The documentation for ssl-3.10.8 can not be considered to be the current documentation. The current version of ssl is ssl-5.2.1. The code for generating certificates for testing purposes was long ago moved to the test directory where it still resides. Left in the example directory are some example certificates. Please read the latest version of the documentation http://www.erlang.org/doc/apps/ssl/index.html Regards Ingela Erlang/OTP team - Ericsson AB 2013/4/11 > Hi!**** > > ** ** > > According to the current SSL library documentation there should exist an > example module for generating certificates:**** > > ** ** > > ?The following module create_certs is used by the Erlang/OTP SSL > application for generating certificates to be used in tests. The source > code is also found in ssl-X.Y.Z/examples/certs/src.?**** > > ** ** > > There?s no module there (and not in previous version ssl-4.1.6 either).*** > * > > ** ** > > Can I find the module somewhere else?**** > > ** ** > > > http://www.erlang.org/documentation/doc-5.7.5/lib/ssl-3.10.8/doc/html/create_certs.html > **** > > ** ** > > /Andreas**** > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Andreas.Hellstrom@REDACTED Wed Apr 17 09:30:51 2013 From: Andreas.Hellstrom@REDACTED (Andreas.Hellstrom@REDACTED) Date: Wed, 17 Apr 2013 07:30:51 +0000 Subject: [erlang-questions] Missing module create_certs in ssl-5.2.1 In-Reply-To: References: <6CA60E3A41E34D4A851816F9BA44F76A3419B1D4@EXMB13TSTRZ2.tcad.telia.se> Message-ID: <6CA60E3A41E34D4A851816F9BA44F76A3419BC78@EXMB13TSTRZ2.tcad.telia.se> Thanks for helping out! I can't find a test directory in the Windows distribution, do you mean the source test directory <.\otp_src_R16B\erts\test>? Even so, I can't find any module for creating certificates there... Regards, Andreas From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Ingela Andin Sent: den 17 april 2013 9:08 To: erlang-questions@REDACTED Subject: Re: [erlang-questions] Missing module create_certs in ssl-5.2.1 Hello! The documentation for ssl-3.10.8 can not be considered to be the current documentation. The current version of ssl is ssl-5.2.1. The code for generating certificates for testing purposes was long ago moved to the test directory where it still resides. Left in the example directory are some example certificates. Please read the latest version of the documentation http://www.erlang.org/doc/apps/ssl/index.html Regards Ingela Erlang/OTP team - Ericsson AB 2013/4/11 > Hi! According to the current SSL library documentation there should exist an example module for generating certificates: "The following module create_certs is used by the Erlang/OTP SSL application for generating certificates to be used in tests. The source code is also found in ssl-X.Y.Z/examples/certs/src." There's no module there (and not in previous version ssl-4.1.6 either). Can I find the module somewhere else? http://www.erlang.org/documentation/doc-5.7.5/lib/ssl-3.10.8/doc/html/create_certs.html /Andreas _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon@REDACTED Wed Apr 17 09:30:58 2013 From: jon@REDACTED (Jonathan Schneider) Date: Wed, 17 Apr 2013 08:30:58 +0100 Subject: [erlang-questions] Listen to SSL and normal TCP connections on the same port In-Reply-To: References: Message-ID: The STARTTLS command is used for this in SMTP and friends. Jon On 17 Apr 2013, at 00:38, pablo platt wrote: > Hi, > > Is it possible to accept SSL connections and normal TCP connections on the same port? > Maybe accept normal TCP connections. If the connection is SSL connection pass it to the ssl module and if it is a normal TCP connection just handle the socket. > > If it is possible, how can I distinct SSL from non SSL connections? > How can I pass the socket to the ssl module? > > In my case I control the server but I can't change the client. > > Thanks > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ingela.andin@REDACTED Wed Apr 17 09:41:18 2013 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 17 Apr 2013 09:41:18 +0200 Subject: [erlang-questions] Listen to SSL and normal TCP connections on the same port In-Reply-To: References: Message-ID: Hi! 2013/4/17 pablo platt > Hi, > > Is it possible to accept SSL connections and normal TCP connections on the > same port? > Maybe accept normal TCP connections. If the connection is SSL connection > pass it to the ssl module and if it is a normal TCP connection just handle > the socket. > It is possible to upgrade a tcp socket to an ssl socket. If it is possible, how can I distinct SSL from non SSL connections? > That is the hard part. You must have some scheme to negotiate the upgrade with the client over plain tcp. (Like STARTTLS, HTTP Connect etc) > How can I pass the socket to the ssl module? > ssl:ssl_accept(TcpSocket, SslOptions) or ssl:connect(TcpSocket, SslOptions) make sure the socket is passive ({active, false}) before you make the call. Regards Ingela Erlang/OTP team Ericsson AB -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Wed Apr 17 08:08:16 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Wed, 17 Apr 2013 08:08:16 +0200 Subject: [erlang-questions] Listen to SSL and normal TCP connections on the same port In-Reply-To: References: Message-ID: <913DBE26-FE0A-45F4-99D4-4B1FB557FEBF@gmail.com> Hello Pablo, If you can't change the client, surely the client as a way to advertise whether it is intending to use plain old TCP or SSL? Otherwise, to upgrade a plain old TCP socket to SSL, you should use ssl:ssl_accept/{1,3}, which probably suffers from a small documentation problem. Regards, -- Anthony Ramine Le 17 avr. 2013 ? 01:38, pablo platt a ?crit : > Hi, > > Is it possible to accept SSL connections and normal TCP connections on the same port? > Maybe accept normal TCP connections. If the connection is SSL connection pass it to the ssl module and if it is a normal TCP connection just handle the socket. > > If it is possible, how can I distinct SSL from non SSL connections? > How can I pass the socket to the ssl module? > > In my case I control the server but I can't change the client. > > Thanks > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ingela.andin@REDACTED Wed Apr 17 09:46:34 2013 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 17 Apr 2013 09:46:34 +0200 Subject: [erlang-questions] Missing module create_certs in ssl-5.2.1 In-Reply-To: <6CA60E3A41E34D4A851816F9BA44F76A3419BC78@EXMB13TSTRZ2.tcad.telia.se> References: <6CA60E3A41E34D4A851816F9BA44F76A3419B1D4@EXMB13TSTRZ2.tcad.telia.se> <6CA60E3A41E34D4A851816F9BA44F76A3419BC78@EXMB13TSTRZ2.tcad.telia.se> Message-ID: Hi! 2013/4/17 > Thanks for helping out!**** > > ** ** > > I can?t find a test directory in the Windows distribution, do you mean the > source test directory <.\otp_src_R16B\erts\test>? Even so, I can?t find any > module for creating certificates there?**** > > ** > No I mean \otp_src_R16B\lib\ssl-\test (All applications have their own test directory) Regards Ingela Erlang/OTP team - Ericsson AB -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Wed Apr 17 10:15:26 2013 From: tony@REDACTED (Tony Rogvall) Date: Wed, 17 Apr 2013 10:15:26 +0200 Subject: [erlang-questions] Listen to SSL and normal TCP connections on the same port In-Reply-To: References: Message-ID: <56E6F178-2D17-4A6C-A77B-382AE2BB617A@rogvall.se> Hi! In general it is not possible to detect and upgrade a socket to ssl, BUT if the client initiate with some known message data that can be distinguished from the SSL hello message, like HTTP requests then you can do something like (I have, and it works :-): The socket should already be in {packet,0} packet mode, set {active,false}, {packet,0} in the options to listen. Then either read in passive mode: {ok,Header} = gen_tcp:recv(Socket, MinHeaderBlockSize), %% say 8 bytes ? Or something like this in active once mode inet:setopts(Socket, [{active, once}]), %% not easy to control the number of bytes received here yet (there is an EEP!) Header = receive {tcp, Socket, Header} -> ok end Then do a match on the data that the client normally send with some known SSL intro messages: Type = detect(Header), detect(<<"GET", _/binary>>) -> plain; detect(<<"POST", _/binary>>) -> plain; detect(<<"OPTIONS", _/binary>>) -> plain; detect(<<"TRACE", _/binary>>) -> plain; ... detect(<<1:1,_Len:15,1:8,_Version:16, _/binary>>) -> ssl; detect(<>) -> if ContentType == 22 -> %% HANDSHAKE ssl; true -> undefined end; detect(_) -> undefined. Then switch to ssl if that was detected: if Type =:= ssl -> %% Socket MUST be in passive mode at this point gen_tcp:unrecv(Socket, Header), %% push back SSL data {ok, SSLSocket} = ssl:ssl_accept(Socket, SSLOptions, SSLAcceptTimeout), do_ssl_request(SSLSocket); true -> do_plain_request(Socket) end /Tony On 17 apr 2013, at 09:41, Ingela Andin wrote: > Hi! > > 2013/4/17 pablo platt > Hi, > > Is it possible to accept SSL connections and normal TCP connections on the same port? > Maybe accept normal TCP connections. If the connection is SSL connection pass it to the ssl module and if it is a normal TCP connection just handle the socket. > > It is possible to upgrade a tcp socket to an ssl socket. > > If it is possible, how can I distinct SSL from non SSL connections? > > That is the hard part. You must have some scheme to negotiate the upgrade with the client over plain tcp. (Like STARTTLS, HTTP Connect etc) > > How can I pass the socket to the ssl module? > > ssl:ssl_accept(TcpSocket, SslOptions) or ssl:connect(TcpSocket, SslOptions) > make sure the socket is passive ({active, false}) before you make the call. > > Regards Ingela Erlang/OTP team Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions "Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix" -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidw@REDACTED Wed Apr 17 10:42:50 2013 From: davidw@REDACTED (David Welton) Date: Wed, 17 Apr 2013 10:42:50 +0200 Subject: [erlang-questions] Rebar, erlexec and c_src Message-ID: Hi, I'm trying to integrate erlexec as a dependency in my project, as it provides much more advanced handling of OS processes than what Erlang does. However, when I do ./rebar compile, it fails to do anything with the C++ code: Is there anything I should be doing differently? Thanks! INFO: sh info: cwd: "/home/davidw/clients/myclient/myapp/deps/myapp_interface" cmd: git describe --always --tags `git log -n 1 --pretty=format:%h .` DEBUG: opts: [{cd,"/home/davidw/clients/myclient/myapp/deps/exec"}, {use_stdout,false}] INFO: Looking for exec-.* ; found exec-v1.0-5-gff9c373 at /home/davidw/clients/myclient/myapp/deps/exec DEBUG: Available deps: [{dep,"/home/davidw/clients/myclient/myapp/deps/exec", exec,".*", {git,"git://github.com/saleyn/erlexec.git", {branch,"master"}}}] DEBUG: Missing deps : [] DEBUG: is_app_available, looking for App exec with Path "/home/davidw/clients/myclient/myapp/deps/exec" INFO: Looking for exec-.* ; found exec-v1.0-5-gff9c373 at /home/davidw/clients/myclient/myapp/deps/exec DEBUG: Adding /home/davidw/clients/myclient/myapp/deps/exec/ebin to code path DEBUG: Associate sub_dir /home/davidw/clients/myclient/myapp/deps/exec with /home/davidw/clients/myclient/myapp/deps/myapp_interface DEBUG: Associate sub_dir /home/davidw/clients/myclient/myapp/deps/myapp_interface/rel with /home/davidw/clients/myclient/myapp/deps/myapp_interface DEBUG: Predirs: ["/home/davidw/clients/myclient/myapp/deps/exec", "/home/davidw/clients/myclient/myapp/deps/myapp_interface/rel"] DEBUG: Entering /home/davidw/clients/myclient/myapp/deps/exec DEBUG: Consult config file "/home/davidw/clients/myclient/myapp/deps/exec/rebar.config" DEBUG: Available deps: [] DEBUG: Missing deps : [] DEBUG: Predirs: [] ==> exec (pre_compile) ==> exec (compile) DEBUG: Matched required ERTS version: 5.10.1 -> .* DEBUG: Matched required OTP release: R16B -> .* DEBUG: erl_opts [debug_info,warnings_as_errors,warn_export_all] DEBUG: Starting 3 compile worker(s) DEBUG: Worker exited cleanly Compiled src/exec_app.erl DEBUG: Worker exited cleanly Compiled src/exec.erl DEBUG: Worker exited cleanly INFO: No app_vars_file defined. DEBUG: Postdirs: [] -- David N. Welton http://www.dedasys.com/ From magnus@REDACTED Wed Apr 17 11:36:49 2013 From: magnus@REDACTED (Magnus Henoch) Date: Wed, 17 Apr 2013 10:36:49 +0100 Subject: [erlang-questions] Rebar, erlexec and c_src In-Reply-To: (David Welton's message of "Wed, 17 Apr 2013 10:42:50 +0200") References: Message-ID: David Welton writes: > I'm trying to integrate erlexec as a dependency in my project, as it > provides much more advanced handling of OS processes than what Erlang > does. > > However, when I do ./rebar compile, it fails to do anything with the C++ code: > > Is there anything I should be doing differently? The erlexec project has a rebar.config.script file, that is used alongside rebar.config to provide the correct configuration for compilation: https://github.com/saleyn/erlexec/blob/master/rebar.config.script I would expect Rebar to print "DEBUG: Evaluating config file", but instead I see "DEBUG: Consult config file" in the output. (See rebar_config:consult_file and rebar_config:consult_and_eval if you're curious.) That suggests that your rebar is too old. What does "rebar --version" say? Regards, Magnus From davidw@REDACTED Wed Apr 17 13:25:03 2013 From: davidw@REDACTED (David Welton) Date: Wed, 17 Apr 2013 13:25:03 +0200 Subject: [erlang-questions] Rebar, erlexec and c_src In-Reply-To: References: Message-ID: > That suggests that your rebar is too old. What does "rebar > --version" say? That was it. rebar version: 2 date: 20120209_044624 vcs: git 1c1a32b fails rebar 2.1.0-pre R14B04 20130309_213845 git 2.1.0-pre-51-g8377a50-dirty works Thanks! -- David N. Welton http://www.dedasys.com/ From andra.dinu@REDACTED Wed Apr 17 16:58:42 2013 From: andra.dinu@REDACTED (Andra Dinu) Date: Wed, 17 Apr 2013 15:58:42 +0100 (BST) Subject: [erlang-questions] Budapest Erlang Factory Lite 30 May: Early Bird ends 9 May Message-ID: <103004276.68672371.1366210722515.JavaMail.root@erlang-solutions.com> The first Budapest Erlang Factory Lite will take place on 30 May. Early Bird ticket is 20 euro (excluding VAT). Talks include: 1000 actors, one chaos monkey and... everything ok - Pavlo Baron Zotonic, the Erlang Web Framework and CMS - Atilla Erd?di Taming the Rabbit: Writing RabbitMQ Plugins - Alvaro Videla Find out more here: https://www.erlang-factory.com/conference/Budapest2013 -- Andra Dinu Community & Social ERLANG SOLUTIONS LTD New Loom House 101 Back Church Lane London, E1 1LU United Kingdom Tel +44(0)2076550344 Mob +44(0)7983484387 www.erlang-solutions.com From andra.dinu@REDACTED Wed Apr 17 17:00:13 2013 From: andra.dinu@REDACTED (Andra Dinu) Date: Wed, 17 Apr 2013 16:00:13 +0100 (BST) Subject: [erlang-questions] Dublin Erlang Factory Lite 23 May: Free event Message-ID: <1466196358.68674830.1366210813864.JavaMail.root@erlang-solutions.com> The Erlang Factory Lite comes to Dublin on the 23rd of May! The event is co-organised by AOL, and will take place at their headquarters in Dublin: The Brunel Building in Heuston South Quarter, Ireland. This event is a FREE event. For more info and details go here: http://www.erlang-factory.com/conference/Dublin2013 -- Andra Dinu Community & Social ERLANG SOLUTIONS LTD New Loom House 101 Back Church Lane London, E1 1LU United Kingdom Tel +44(0)2076550344 Mob +44(0)7983484387 www.erlang-solutions.com From andra.dinu@REDACTED Wed Apr 17 16:56:29 2013 From: andra.dinu@REDACTED (Andra Dinu) Date: Wed, 17 Apr 2013 15:56:29 +0100 (BST) Subject: [erlang-questions] Erlang User Conference Stockholm 13-14 June: Early Bird registration ends on 15 May Message-ID: <1680380255.68670527.1366210589968.JavaMail.root@erlang-solutions.com> This year, we are proud to have Mike Williams,Robert Virding and Joe Armstrong on the same stage in a joint keynote. The second keynote will be delivered by Claes Wikstr?m - author of Yaws and founder of Tail-F; other speakers include Bruce Tate, Pavlo Baron, Mahesh Paolini- Subramanya, Steve Vinoski and more. The list of speakers can be seen here: https://www.erlang-factory.com/conference/ErlangUserConference2013/speakers The conference will be preceded by three days of training courses: Erlang Express, OTP Express, Building Distributed Clusters with Riak and Test Driven Development in Erlang ? starting on the 10th of June. We are also preparing a day of tutorials on the 12th of June, so keep an eye on the website for updates. To register, go here: https://www.erlang-factory.com/conference/ErlangUserConference2013/register -- Andra Dinu Community & Social ERLANG SOLUTIONS LTD New Loom House 101 Back Church Lane London, E1 1LU United Kingdom Tel +44(0)2076550344 Mob +44(0)7983484387 www.erlang-solutions.com From anotherworldofworld@REDACTED Wed Apr 17 19:17:41 2013 From: anotherworldofworld@REDACTED (Alex toyer) Date: Wed, 17 Apr 2013 23:17:41 +0600 Subject: [erlang-questions] [ANN] Ybot-0.3.3 released Message-ID: Hello, Let me announce new version of Ybot chat robot. Here is shortlist of ChangeLog: * HipChat config moved to separte config from XMPP transport. * XMPP 'is_hipchat' option was removed. * XMPP 'hipchat_nick' option was removed. * Mnesia backend storage added. * Erlang/otp application as plugin support added. * New core plugin - memory. * history_command_limit_count aren't required parametes now. * Web interface added. * New internal command 'name?'. * all api from ybot.erl beside start/stop were removed. * ybot_shell added for testing Ybot from erlang shell. * Improved skype support. * Fixed long message sending for campfire transport. and many more... Full changelog you can find at https://github.com/0xAX/Ybot/blob/master/Changelog.md Ybot plugins catalog - http://0xax.github.io/ybot-plugins.html --------- best regards, twitter: @0xAX github: 0xAX -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Wed Apr 17 19:33:04 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 17 Apr 2013 18:33:04 +0100 Subject: [erlang-questions] Message Loop in Gen_Server Message-ID: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> Hey guys, So, I've hit a "best practice" conundrum in OTP; I have a server utilising gen_server for a RabbitMQ consumer. In the init of that gen_server, I'm setting up a RabbitMQ connection, but I also need to start a loop. My guess was that I shouldn't call this before init exits, as I was passing the Connection and Channel objects to state for handling elsewhere. If I handle the loop in init, surely it will never return? To simplify what I'm saying (as I'm confusing myself here), here's my code: init([]) -> {ok, Connection} = amqp_connection:start(#amqp_params_network{ host="localhost" }), {ok, Channel} = amqp_connection:open_channel(Connection), amqp_channel:call(Channel, #'exchange.declare'{exchange = <<"user_msgs">>, type = <<"direct">>}), #'queue.declare_ok'{queue = Queue} = amqp_channel:call(Channel, #'queue.declare'{exclusive = true}), State = {Channel, Connection}, amqp_channel:call(Channel, #'queue.bind'{exchange = <<"user_msgs">>, routing_key = term_to_binary(node(self())), queue = Queue}), amqp_channel:subscribe(Channel, #'basic.consume'{queue = Queue, no_ack = true}, self()), receive #'basic.consume_ok'{} -> ok end, loop(Channel), {ok, State}. Now, if I don't put the loop in my init, then how can I be sure that the loop is called every time the gen_server restarts? Can someone please suggest the "right" way to call the loop in my gen_server? Thanks loads, Lee From g@REDACTED Wed Apr 17 19:41:59 2013 From: g@REDACTED (Garrett Smith) Date: Wed, 17 Apr 2013 12:41:59 -0500 Subject: [erlang-questions] Message Loop in Gen_Server In-Reply-To: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> References: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> Message-ID: On Wed, Apr 17, 2013 at 12:33 PM, Lee Sylvester wrote: > Hey guys, > > So, I've hit a "best practice" conundrum in OTP; I have a server utilising gen_server for a RabbitMQ consumer. In the init of that gen_server, I'm setting up a RabbitMQ connection, but I also need to start a loop. My guess was that I shouldn't call this before init exits, as I was passing the Connection and Channel objects to state for handling elsewhere. If I handle the loop in init, surely it will never return? > > To simplify what I'm saying (as I'm confusing myself here), here's my code: > > init([]) -> > {ok, Connection} = amqp_connection:start(#amqp_params_network{ host="localhost" }), > {ok, Channel} = amqp_connection:open_channel(Connection), > amqp_channel:call(Channel, #'exchange.declare'{exchange = <<"user_msgs">>, > type = <<"direct">>}), > #'queue.declare_ok'{queue = Queue} = > amqp_channel:call(Channel, #'queue.declare'{exclusive = true}), > State = {Channel, Connection}, > amqp_channel:call(Channel, #'queue.bind'{exchange = <<"user_msgs">>, > routing_key = term_to_binary(node(self())), > queue = Queue}), > amqp_channel:subscribe(Channel, #'basic.consume'{queue = Queue, > no_ack = true}, self()), > receive > #'basic.consume_ok'{} -> ok > end, > loop(Channel), > {ok, State}. Definitely not in init/1 -- that blocks the caller to start_link. > Now, if I don't put the loop in my init, then how can I be sure that the loop is called every time the gen_server restarts? Can someone please suggest the "right" way to call the loop in my gen_server? It's awkward, which is the point of e2_task -- see http://e2project.org if you're curious -- but this is how: init(Args) -> % init here {ok, State, 0}. handle_info(timeout, State) -> loop(). The 0 element in the init result will cause an immediate 'timeout' message to be sent to the process, resulting in a handle_info/2 callback. Garrett From john@REDACTED Wed Apr 17 19:44:31 2013 From: john@REDACTED (John Kemp) Date: Wed, 17 Apr 2013 13:44:31 -0400 Subject: [erlang-questions] Message Loop in Gen_Server In-Reply-To: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> References: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> Message-ID: Perhaps something like this?: init(...) -> spawn_link( module_name, loop, [options] ). loop(...) -> receive exit -> ok ; message -> (your code here), loop(...) end. You can then send your gen_server an exit message to make it stop whatever it's doing. JohnK On Apr 17, 2013, at 1:33 PM, Lee Sylvester wrote: > Hey guys, > > So, I've hit a "best practice" conundrum in OTP; I have a server utilising gen_server for a RabbitMQ consumer. In the init of that gen_server, I'm setting up a RabbitMQ connection, but I also need to start a loop. My guess was that I shouldn't call this before init exits, as I was passing the Connection and Channel objects to state for handling elsewhere. If I handle the loop in init, surely it will never return? > > To simplify what I'm saying (as I'm confusing myself here), here's my code: > > init([]) -> > {ok, Connection} = amqp_connection:start(#amqp_params_network{ host="localhost" }), > {ok, Channel} = amqp_connection:open_channel(Connection), > amqp_channel:call(Channel, #'exchange.declare'{exchange = <<"user_msgs">>, > type = <<"direct">>}), > #'queue.declare_ok'{queue = Queue} = > amqp_channel:call(Channel, #'queue.declare'{exclusive = true}), > State = {Channel, Connection}, > amqp_channel:call(Channel, #'queue.bind'{exchange = <<"user_msgs">>, > routing_key = term_to_binary(node(self())), > queue = Queue}), > amqp_channel:subscribe(Channel, #'basic.consume'{queue = Queue, > no_ack = true}, self()), > receive > #'basic.consume_ok'{} -> ok > end, > loop(Channel), > {ok, State}. > > Now, if I don't put the loop in my init, then how can I be sure that the loop is called every time the gen_server restarts? Can someone please suggest the "right" way to call the loop in my gen_server? > > Thanks loads, > Lee > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From g@REDACTED Wed Apr 17 19:45:29 2013 From: g@REDACTED (Garrett Smith) Date: Wed, 17 Apr 2013 12:45:29 -0500 Subject: [erlang-questions] Message Loop in Gen_Server In-Reply-To: References: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> Message-ID: On Wed, Apr 17, 2013 at 12:41 PM, Garrett Smith wrote: > On Wed, Apr 17, 2013 at 12:33 PM, Lee Sylvester wrote: >> Hey guys, >> >> So, I've hit a "best practice" conundrum in OTP; I have a server utilising gen_server for a RabbitMQ consumer. In the init of that gen_server, I'm setting up a RabbitMQ connection, but I also need to start a loop. My guess was that I shouldn't call this before init exits, as I was passing the Connection and Channel objects to state for handling elsewhere. If I handle the loop in init, surely it will never return? >> >> To simplify what I'm saying (as I'm confusing myself here), here's my code: >> >> init([]) -> >> {ok, Connection} = amqp_connection:start(#amqp_params_network{ host="localhost" }), >> {ok, Channel} = amqp_connection:open_channel(Connection), >> amqp_channel:call(Channel, #'exchange.declare'{exchange = <<"user_msgs">>, >> type = <<"direct">>}), >> #'queue.declare_ok'{queue = Queue} = >> amqp_channel:call(Channel, #'queue.declare'{exclusive = true}), >> State = {Channel, Connection}, >> amqp_channel:call(Channel, #'queue.bind'{exchange = <<"user_msgs">>, >> routing_key = term_to_binary(node(self())), >> queue = Queue}), >> amqp_channel:subscribe(Channel, #'basic.consume'{queue = Queue, >> no_ack = true}, self()), >> receive >> #'basic.consume_ok'{} -> ok >> end, >> loop(Channel), >> {ok, State}. > > Definitely not in init/1 -- that blocks the caller to start_link. > >> Now, if I don't put the loop in my init, then how can I be sure that the loop is called every time the gen_server restarts? Can someone please suggest the "right" way to call the loop in my gen_server? > > It's awkward, which is the point of e2_task -- see > http://e2project.org if you're curious -- but this is how: > > init(Args) -> > % init here > {ok, State, 0}. > > handle_info(timeout, State) -> > loop(). > > The 0 element in the init result will cause an immediate 'timeout' > message to be sent to the process, resulting in a handle_info/2 > callback. Incidentally, if this is a loop that runs for the life of the process, it's easier to just run a function using proc_lib: proc_lib:start_link(M, F, A) See http://erlang.org/doc/man/proc_lib.html for details. You can setup your rabbit client at the start of the loop and just loop thereafter. Garrett From lee.sylvester@REDACTED Wed Apr 17 19:46:27 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 17 Apr 2013 18:46:27 +0100 Subject: [erlang-questions] Message Loop in Gen_Server In-Reply-To: References: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> Message-ID: <8A2A4785-7A46-45E3-B28B-55ECE0A9DF4E@gmail.com> Thank you. I've actually seen that done in the Erlang in Action book. Didn't think of it myself :-) You're a star! Regards, Lee On 17 Apr 2013, at 18:41, Garrett Smith wrote: > On Wed, Apr 17, 2013 at 12:33 PM, Lee Sylvester wrote: >> Hey guys, >> >> So, I've hit a "best practice" conundrum in OTP; I have a server utilising gen_server for a RabbitMQ consumer. In the init of that gen_server, I'm setting up a RabbitMQ connection, but I also need to start a loop. My guess was that I shouldn't call this before init exits, as I was passing the Connection and Channel objects to state for handling elsewhere. If I handle the loop in init, surely it will never return? >> >> To simplify what I'm saying (as I'm confusing myself here), here's my code: >> >> init([]) -> >> {ok, Connection} = amqp_connection:start(#amqp_params_network{ host="localhost" }), >> {ok, Channel} = amqp_connection:open_channel(Connection), >> amqp_channel:call(Channel, #'exchange.declare'{exchange = <<"user_msgs">>, >> type = <<"direct">>}), >> #'queue.declare_ok'{queue = Queue} = >> amqp_channel:call(Channel, #'queue.declare'{exclusive = true}), >> State = {Channel, Connection}, >> amqp_channel:call(Channel, #'queue.bind'{exchange = <<"user_msgs">>, >> routing_key = term_to_binary(node(self())), >> queue = Queue}), >> amqp_channel:subscribe(Channel, #'basic.consume'{queue = Queue, >> no_ack = true}, self()), >> receive >> #'basic.consume_ok'{} -> ok >> end, >> loop(Channel), >> {ok, State}. > > Definitely not in init/1 -- that blocks the caller to start_link. > >> Now, if I don't put the loop in my init, then how can I be sure that the loop is called every time the gen_server restarts? Can someone please suggest the "right" way to call the loop in my gen_server? > > It's awkward, which is the point of e2_task -- see > http://e2project.org if you're curious -- but this is how: > > init(Args) -> > % init here > {ok, State, 0}. > > handle_info(timeout, State) -> > loop(). > > The 0 element in the init result will cause an immediate 'timeout' > message to be sent to the process, resulting in a handle_info/2 > callback. > > Garrett From jbothma@REDACTED Wed Apr 17 19:47:38 2013 From: jbothma@REDACTED (JD Bothma) Date: Wed, 17 Apr 2013 19:47:38 +0200 Subject: [erlang-questions] Message Loop in Gen_Server In-Reply-To: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> References: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> Message-ID: Can I ask what your never-exiting loop is doing? what is the purpose of the gen_server if the process will forever loop in loop and not deal with gen_server messages? On 17 April 2013 19:33, Lee Sylvester wrote: > Hey guys, > > So, I've hit a "best practice" conundrum in OTP; I have a server utilising > gen_server for a RabbitMQ consumer. In the init of that gen_server, I'm > setting up a RabbitMQ connection, but I also need to start a loop. My > guess was that I shouldn't call this before init exits, as I was passing > the Connection and Channel objects to state for handling elsewhere. If I > handle the loop in init, surely it will never return? > > To simplify what I'm saying (as I'm confusing myself here), here's my code: > > init([]) -> > {ok, Connection} = amqp_connection:start(#amqp_params_network{ > host="localhost" }), > {ok, Channel} = amqp_connection:open_channel(Connection), > amqp_channel:call(Channel, #'exchange.declare'{exchange = > <<"user_msgs">>, > type = <<"direct">>}), > #'queue.declare_ok'{queue = Queue} = > amqp_channel:call(Channel, #'queue.declare'{exclusive = true}), > State = {Channel, Connection}, > amqp_channel:call(Channel, #'queue.bind'{exchange = <<"user_msgs">>, > routing_key = > term_to_binary(node(self())), > queue = Queue}), > amqp_channel:subscribe(Channel, #'basic.consume'{queue = Queue, > no_ack = true}, self()), > receive > #'basic.consume_ok'{} -> ok > end, > loop(Channel), > {ok, State}. > > Now, if I don't put the loop in my init, then how can I be sure that the > loop is called every time the gen_server restarts? Can someone please > suggest the "right" way to call the loop in my gen_server? > > Thanks loads, > Lee > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Wed Apr 17 19:57:35 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 17 Apr 2013 18:57:35 +0100 Subject: [erlang-questions] Message Loop in Gen_Server In-Reply-To: References: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> Message-ID: It's checking for messages from RabbitMQ. Cheers, Lee On 17 Apr 2013, at 18:47, JD Bothma wrote: > Can I ask what your never-exiting loop is doing? > > what is the purpose of the gen_server if the process will forever loop in loop and not deal with gen_server messages? > > > On 17 April 2013 19:33, Lee Sylvester wrote: > Hey guys, > > So, I've hit a "best practice" conundrum in OTP; I have a server utilising gen_server for a RabbitMQ consumer. In the init of that gen_server, I'm setting up a RabbitMQ connection, but I also need to start a loop. My guess was that I shouldn't call this before init exits, as I was passing the Connection and Channel objects to state for handling elsewhere. If I handle the loop in init, surely it will never return? > > To simplify what I'm saying (as I'm confusing myself here), here's my code: > > init([]) -> > {ok, Connection} = amqp_connection:start(#amqp_params_network{ host="localhost" }), > {ok, Channel} = amqp_connection:open_channel(Connection), > amqp_channel:call(Channel, #'exchange.declare'{exchange = <<"user_msgs">>, > type = <<"direct">>}), > #'queue.declare_ok'{queue = Queue} = > amqp_channel:call(Channel, #'queue.declare'{exclusive = true}), > State = {Channel, Connection}, > amqp_channel:call(Channel, #'queue.bind'{exchange = <<"user_msgs">>, > routing_key = term_to_binary(node(self())), > queue = Queue}), > amqp_channel:subscribe(Channel, #'basic.consume'{queue = Queue, > no_ack = true}, self()), > receive > #'basic.consume_ok'{} -> ok > end, > loop(Channel), > {ok, State}. > > Now, if I don't put the loop in my init, then how can I be sure that the loop is called every time the gen_server restarts? Can someone please suggest the "right" way to call the loop in my gen_server? > > Thanks loads, > Lee > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Wed Apr 17 20:01:00 2013 From: g@REDACTED (Garrett Smith) Date: Wed, 17 Apr 2013 13:01:00 -0500 Subject: [erlang-questions] Message Loop in Gen_Server In-Reply-To: References: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> Message-ID: If you're receiving messages in the loop, you're working too hard :) Once you start the gen_server, any messages sent to that process (i.e. anything that you'd receive in your loop) can be handled via handle_info/2. If you're polling something, see if you can configure that something to send you a message directly instead. I know you can do this with the rabbit client. On Wed, Apr 17, 2013 at 12:57 PM, Lee Sylvester wrote: > It's checking for messages from RabbitMQ. > > Cheers, > Lee > > > On 17 Apr 2013, at 18:47, JD Bothma wrote: > > Can I ask what your never-exiting loop is doing? > > what is the purpose of the gen_server if the process will forever loop in > loop and not deal with gen_server messages? > > > On 17 April 2013 19:33, Lee Sylvester wrote: >> >> Hey guys, >> >> So, I've hit a "best practice" conundrum in OTP; I have a server utilising >> gen_server for a RabbitMQ consumer. In the init of that gen_server, I'm >> setting up a RabbitMQ connection, but I also need to start a loop. My guess >> was that I shouldn't call this before init exits, as I was passing the >> Connection and Channel objects to state for handling elsewhere. If I handle >> the loop in init, surely it will never return? >> >> To simplify what I'm saying (as I'm confusing myself here), here's my >> code: >> >> init([]) -> >> {ok, Connection} = amqp_connection:start(#amqp_params_network{ >> host="localhost" }), >> {ok, Channel} = amqp_connection:open_channel(Connection), >> amqp_channel:call(Channel, #'exchange.declare'{exchange = >> <<"user_msgs">>, >> type = <<"direct">>}), >> #'queue.declare_ok'{queue = Queue} = >> amqp_channel:call(Channel, #'queue.declare'{exclusive = true}), >> State = {Channel, Connection}, >> amqp_channel:call(Channel, #'queue.bind'{exchange = <<"user_msgs">>, >> routing_key = >> term_to_binary(node(self())), >> queue = Queue}), >> amqp_channel:subscribe(Channel, #'basic.consume'{queue = Queue, >> no_ack = true}, self()), >> receive >> #'basic.consume_ok'{} -> ok >> end, >> loop(Channel), >> {ok, State}. >> >> Now, if I don't put the loop in my init, then how can I be sure that the >> loop is called every time the gen_server restarts? Can someone please >> suggest the "right" way to call the loop in my gen_server? >> >> Thanks loads, >> Lee >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From lee.sylvester@REDACTED Wed Apr 17 20:27:52 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 17 Apr 2013 19:27:52 +0100 Subject: [erlang-questions] Message Loop in Gen_Server In-Reply-To: References: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> Message-ID: <0021AFCA-892C-489A-8104-A71BDE2FFEC4@gmail.com> Ahh, thank you. I have tried to find this using Google, but was unable to find an example. Any idea what kind of function signature I'd use? Thanks, Lee On 17 Apr 2013, at 19:01, Garrett Smith wrote: > If you're receiving messages in the loop, you're working too hard :) > > Once you start the gen_server, any messages sent to that process (i.e. > anything that you'd receive in your loop) can be handled via > handle_info/2. > > If you're polling something, see if you can configure that something > to send you a message directly instead. I know you can do this with > the rabbit client. > > On Wed, Apr 17, 2013 at 12:57 PM, Lee Sylvester wrote: >> It's checking for messages from RabbitMQ. >> >> Cheers, >> Lee >> >> >> On 17 Apr 2013, at 18:47, JD Bothma wrote: >> >> Can I ask what your never-exiting loop is doing? >> >> what is the purpose of the gen_server if the process will forever loop in >> loop and not deal with gen_server messages? >> >> >> On 17 April 2013 19:33, Lee Sylvester wrote: >>> >>> Hey guys, >>> >>> So, I've hit a "best practice" conundrum in OTP; I have a server utilising >>> gen_server for a RabbitMQ consumer. In the init of that gen_server, I'm >>> setting up a RabbitMQ connection, but I also need to start a loop. My guess >>> was that I shouldn't call this before init exits, as I was passing the >>> Connection and Channel objects to state for handling elsewhere. If I handle >>> the loop in init, surely it will never return? >>> >>> To simplify what I'm saying (as I'm confusing myself here), here's my >>> code: >>> >>> init([]) -> >>> {ok, Connection} = amqp_connection:start(#amqp_params_network{ >>> host="localhost" }), >>> {ok, Channel} = amqp_connection:open_channel(Connection), >>> amqp_channel:call(Channel, #'exchange.declare'{exchange = >>> <<"user_msgs">>, >>> type = <<"direct">>}), >>> #'queue.declare_ok'{queue = Queue} = >>> amqp_channel:call(Channel, #'queue.declare'{exclusive = true}), >>> State = {Channel, Connection}, >>> amqp_channel:call(Channel, #'queue.bind'{exchange = <<"user_msgs">>, >>> routing_key = >>> term_to_binary(node(self())), >>> queue = Queue}), >>> amqp_channel:subscribe(Channel, #'basic.consume'{queue = Queue, >>> no_ack = true}, self()), >>> receive >>> #'basic.consume_ok'{} -> ok >>> end, >>> loop(Channel), >>> {ok, State}. >>> >>> Now, if I don't put the loop in my init, then how can I be sure that the >>> loop is called every time the gen_server restarts? Can someone please >>> suggest the "right" way to call the loop in my gen_server? >>> >>> Thanks loads, >>> Lee >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From g@REDACTED Wed Apr 17 20:38:20 2013 From: g@REDACTED (Garrett Smith) Date: Wed, 17 Apr 2013 13:38:20 -0500 Subject: [erlang-questions] Message Loop in Gen_Server In-Reply-To: <0021AFCA-892C-489A-8104-A71BDE2FFEC4@gmail.com> References: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> <0021AFCA-892C-489A-8104-A71BDE2FFEC4@gmail.com> Message-ID: Something like this: {ok, Connection} = amqp_connection:start(Params), {ok, Channel} = amqp_connection:open_channel(Connection), QDecl = #'queue.declare'{queue=Name, auto_delete=true, exclusive=true}, #'queue.declare_ok'{queue=Queue} = amqp_channel:call(Channel, QDecl), QBind = #'queue.bind'{queue=Queue, exchange=Exchange, routing_key=Binding}, amqp_channel:call(Channel, QBind) At this point, messages delivered to the queue should be sent to your process (handle via handle_info/2) as this: {#'basic.deliver'{exchange=Exchange, routing_key=Key}, Msg} Rabbit plugins are implemented as Erlang apps, so I would imagine there are some good examples out there to look at as well. Garrett On Wed, Apr 17, 2013 at 1:27 PM, Lee Sylvester wrote: > Ahh, thank you. I have tried to find this using Google, but was unable to find an example. Any idea what kind of function signature I'd use? > > Thanks, > Lee > > > > On 17 Apr 2013, at 19:01, Garrett Smith wrote: > >> If you're receiving messages in the loop, you're working too hard :) >> >> Once you start the gen_server, any messages sent to that process (i.e. >> anything that you'd receive in your loop) can be handled via >> handle_info/2. >> >> If you're polling something, see if you can configure that something >> to send you a message directly instead. I know you can do this with >> the rabbit client. >> >> On Wed, Apr 17, 2013 at 12:57 PM, Lee Sylvester wrote: >>> It's checking for messages from RabbitMQ. >>> >>> Cheers, >>> Lee >>> >>> >>> On 17 Apr 2013, at 18:47, JD Bothma wrote: >>> >>> Can I ask what your never-exiting loop is doing? >>> >>> what is the purpose of the gen_server if the process will forever loop in >>> loop and not deal with gen_server messages? >>> >>> >>> On 17 April 2013 19:33, Lee Sylvester wrote: >>>> >>>> Hey guys, >>>> >>>> So, I've hit a "best practice" conundrum in OTP; I have a server utilising >>>> gen_server for a RabbitMQ consumer. In the init of that gen_server, I'm >>>> setting up a RabbitMQ connection, but I also need to start a loop. My guess >>>> was that I shouldn't call this before init exits, as I was passing the >>>> Connection and Channel objects to state for handling elsewhere. If I handle >>>> the loop in init, surely it will never return? >>>> >>>> To simplify what I'm saying (as I'm confusing myself here), here's my >>>> code: >>>> >>>> init([]) -> >>>> {ok, Connection} = amqp_connection:start(#amqp_params_network{ >>>> host="localhost" }), >>>> {ok, Channel} = amqp_connection:open_channel(Connection), >>>> amqp_channel:call(Channel, #'exchange.declare'{exchange = >>>> <<"user_msgs">>, >>>> type = <<"direct">>}), >>>> #'queue.declare_ok'{queue = Queue} = >>>> amqp_channel:call(Channel, #'queue.declare'{exclusive = true}), >>>> State = {Channel, Connection}, >>>> amqp_channel:call(Channel, #'queue.bind'{exchange = <<"user_msgs">>, >>>> routing_key = >>>> term_to_binary(node(self())), >>>> queue = Queue}), >>>> amqp_channel:subscribe(Channel, #'basic.consume'{queue = Queue, >>>> no_ack = true}, self()), >>>> receive >>>> #'basic.consume_ok'{} -> ok >>>> end, >>>> loop(Channel), >>>> {ok, State}. >>>> >>>> Now, if I don't put the loop in my init, then how can I be sure that the >>>> loop is called every time the gen_server restarts? Can someone please >>>> suggest the "right" way to call the loop in my gen_server? >>>> >>>> Thanks loads, >>>> Lee >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> > From watson.timothy@REDACTED Wed Apr 17 21:05:01 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 17 Apr 2013 20:05:01 +0100 Subject: [erlang-questions] Message Loop in Gen_Server In-Reply-To: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> References: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> Message-ID: <8EA6C330-0ECC-405A-A770-1F53FB7A0AC7@gmail.com> Hi, When you subscribe, channel messages will be sent to the supplied Pid whenever something happens, e.g., basic.deliver etc. So you don't need to go into a loop, because you've passed self() which in init/1 is the server's process id. What you do instead is to code your handle_info/2 clauses to deal with the AMQP messages. You might want to take a look at amqp_gen_consumer as an alternative to gen_server for this as well. Cheers, Tim On 17 Apr 2013, at 18:33, Lee Sylvester wrote: > Hey guys, > > So, I've hit a "best practice" conundrum in OTP; I have a server utilising gen_server for a RabbitMQ consumer. In the init of that gen_server, I'm setting up a RabbitMQ connection, but I also need to start a loop. My guess was that I shouldn't call this before init exits, as I was passing the Connection and Channel objects to state for handling elsewhere. If I handle the loop in init, surely it will never return? > > To simplify what I'm saying (as I'm confusing myself here), here's my code: > > init([]) -> > {ok, Connection} = amqp_connection:start(#amqp_params_network{ host="localhost" }), > {ok, Channel} = amqp_connection:open_channel(Connection), > amqp_channel:call(Channel, #'exchange.declare'{exchange = <<"user_msgs">>, > type = <<"direct">>}), > #'queue.declare_ok'{queue = Queue} = > amqp_channel:call(Channel, #'queue.declare'{exclusive = true}), > State = {Channel, Connection}, > amqp_channel:call(Channel, #'queue.bind'{exchange = <<"user_msgs">>, > routing_key = term_to_binary(node(self())), > queue = Queue}), > amqp_channel:subscribe(Channel, #'basic.consume'{queue = Queue, > no_ack = true}, self()), > receive > #'basic.consume_ok'{} -> ok > end, > loop(Channel), > {ok, State}. > > Now, if I don't put the loop in my init, then how can I be sure that the loop is called every time the gen_server restarts? Can someone please suggest the "right" way to call the loop in my gen_server? > > Thanks loads, > Lee > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From lee.sylvester@REDACTED Wed Apr 17 21:07:56 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 17 Apr 2013 20:07:56 +0100 Subject: [erlang-questions] Message Loop in Gen_Server In-Reply-To: References: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> <0021AFCA-892C-489A-8104-A71BDE2FFEC4@gmail.com> Message-ID: <564CFCDA-3EF3-4FEA-89F1-E447B6BE5C37@gmail.com> Thanks, Garrett. That give me some direction. Regards, Lee On 17 Apr 2013, at 19:38, Garrett Smith wrote: > Something like this: > > {ok, Connection} = amqp_connection:start(Params), > {ok, Channel} = amqp_connection:open_channel(Connection), > QDecl = #'queue.declare'{queue=Name, auto_delete=true, exclusive=true}, > #'queue.declare_ok'{queue=Queue} = amqp_channel:call(Channel, QDecl), > QBind = #'queue.bind'{queue=Queue, exchange=Exchange, routing_key=Binding}, > amqp_channel:call(Channel, QBind) > > At this point, messages delivered to the queue should be sent to your > process (handle via handle_info/2) as this: > > {#'basic.deliver'{exchange=Exchange, routing_key=Key}, Msg} > > Rabbit plugins are implemented as Erlang apps, so I would imagine > there are some good examples out there to look at as well. > > Garrett > > On Wed, Apr 17, 2013 at 1:27 PM, Lee Sylvester wrote: >> Ahh, thank you. I have tried to find this using Google, but was unable to find an example. Any idea what kind of function signature I'd use? >> >> Thanks, >> Lee >> >> >> >> On 17 Apr 2013, at 19:01, Garrett Smith wrote: >> >>> If you're receiving messages in the loop, you're working too hard :) >>> >>> Once you start the gen_server, any messages sent to that process (i.e. >>> anything that you'd receive in your loop) can be handled via >>> handle_info/2. >>> >>> If you're polling something, see if you can configure that something >>> to send you a message directly instead. I know you can do this with >>> the rabbit client. >>> >>> On Wed, Apr 17, 2013 at 12:57 PM, Lee Sylvester wrote: >>>> It's checking for messages from RabbitMQ. >>>> >>>> Cheers, >>>> Lee >>>> >>>> >>>> On 17 Apr 2013, at 18:47, JD Bothma wrote: >>>> >>>> Can I ask what your never-exiting loop is doing? >>>> >>>> what is the purpose of the gen_server if the process will forever loop in >>>> loop and not deal with gen_server messages? >>>> >>>> >>>> On 17 April 2013 19:33, Lee Sylvester wrote: >>>>> >>>>> Hey guys, >>>>> >>>>> So, I've hit a "best practice" conundrum in OTP; I have a server utilising >>>>> gen_server for a RabbitMQ consumer. In the init of that gen_server, I'm >>>>> setting up a RabbitMQ connection, but I also need to start a loop. My guess >>>>> was that I shouldn't call this before init exits, as I was passing the >>>>> Connection and Channel objects to state for handling elsewhere. If I handle >>>>> the loop in init, surely it will never return? >>>>> >>>>> To simplify what I'm saying (as I'm confusing myself here), here's my >>>>> code: >>>>> >>>>> init([]) -> >>>>> {ok, Connection} = amqp_connection:start(#amqp_params_network{ >>>>> host="localhost" }), >>>>> {ok, Channel} = amqp_connection:open_channel(Connection), >>>>> amqp_channel:call(Channel, #'exchange.declare'{exchange = >>>>> <<"user_msgs">>, >>>>> type = <<"direct">>}), >>>>> #'queue.declare_ok'{queue = Queue} = >>>>> amqp_channel:call(Channel, #'queue.declare'{exclusive = true}), >>>>> State = {Channel, Connection}, >>>>> amqp_channel:call(Channel, #'queue.bind'{exchange = <<"user_msgs">>, >>>>> routing_key = >>>>> term_to_binary(node(self())), >>>>> queue = Queue}), >>>>> amqp_channel:subscribe(Channel, #'basic.consume'{queue = Queue, >>>>> no_ack = true}, self()), >>>>> receive >>>>> #'basic.consume_ok'{} -> ok >>>>> end, >>>>> loop(Channel), >>>>> {ok, State}. >>>>> >>>>> Now, if I don't put the loop in my init, then how can I be sure that the >>>>> loop is called every time the gen_server restarts? Can someone please >>>>> suggest the "right" way to call the loop in my gen_server? >>>>> >>>>> Thanks loads, >>>>> Lee >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >> From lee.sylvester@REDACTED Wed Apr 17 21:08:39 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Wed, 17 Apr 2013 20:08:39 +0100 Subject: [erlang-questions] Message Loop in Gen_Server In-Reply-To: <8EA6C330-0ECC-405A-A770-1F53FB7A0AC7@gmail.com> References: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> <8EA6C330-0ECC-405A-A770-1F53FB7A0AC7@gmail.com> Message-ID: Thanks, Tim. I'll take a look. I've been searching for examples, but they're not that common. I'll have a go at searching on amqp_gen_consumer? I'm sure that'll throw something up :-) Regards, Lee On 17 Apr 2013, at 20:05, Tim Watson wrote: > Hi, > > When you subscribe, channel messages will be sent to the supplied Pid whenever something happens, e.g., basic.deliver etc. > > So you don't need to go into a loop, because you've passed self() which in init/1 is the server's process id. What you do instead is to code your handle_info/2 clauses to deal with the AMQP messages. > > You might want to take a look at amqp_gen_consumer as an alternative to gen_server for this as well. > > Cheers, > Tim > > On 17 Apr 2013, at 18:33, Lee Sylvester wrote: > >> Hey guys, >> >> So, I've hit a "best practice" conundrum in OTP; I have a server utilising gen_server for a RabbitMQ consumer. In the init of that gen_server, I'm setting up a RabbitMQ connection, but I also need to start a loop. My guess was that I shouldn't call this before init exits, as I was passing the Connection and Channel objects to state for handling elsewhere. If I handle the loop in init, surely it will never return? >> >> To simplify what I'm saying (as I'm confusing myself here), here's my code: >> >> init([]) -> >> {ok, Connection} = amqp_connection:start(#amqp_params_network{ host="localhost" }), >> {ok, Channel} = amqp_connection:open_channel(Connection), >> amqp_channel:call(Channel, #'exchange.declare'{exchange = <<"user_msgs">>, >> type = <<"direct">>}), >> #'queue.declare_ok'{queue = Queue} = >> amqp_channel:call(Channel, #'queue.declare'{exclusive = true}), >> State = {Channel, Connection}, >> amqp_channel:call(Channel, #'queue.bind'{exchange = <<"user_msgs">>, >> routing_key = term_to_binary(node(self())), >> queue = Queue}), >> amqp_channel:subscribe(Channel, #'basic.consume'{queue = Queue, >> no_ack = true}, self()), >> receive >> #'basic.consume_ok'{} -> ok >> end, >> loop(Channel), >> {ok, State}. >> >> Now, if I don't put the loop in my init, then how can I be sure that the loop is called every time the gen_server restarts? Can someone please suggest the "right" way to call the loop in my gen_server? >> >> Thanks loads, >> Lee >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From jbothma@REDACTED Wed Apr 17 21:13:29 2013 From: jbothma@REDACTED (JD Bothma) Date: Wed, 17 Apr 2013 21:13:29 +0200 Subject: [erlang-questions] Message Loop in Gen_Server In-Reply-To: References: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> Message-ID: Cool. Like a few replies have said now, handle_info in the gen_server callback module is where you can handle those. If all you need is to handle messages from rabbitmq, you could get away with a plain process, perhaps being supervised. In the long run it's worth the added benefits of proc_lib as Garret suggested, which you get for free in gen_server, but gen_server adds the call and cast options. If you don't need gen_server call and cast, it's worth looking into "OTP-compliant special processes" http://www.erlang.org/doc/design_principles/spec_proc.html - often you don't need a gen_server, or forging a gen_server into what you want isn't the cleanest solution. The easiest way to get started with something good enough, though, is gen_server :) On 17 April 2013 19:57, Lee Sylvester wrote: > It's checking for messages from RabbitMQ. > > Cheers, > Lee > > > On 17 Apr 2013, at 18:47, JD Bothma wrote: > > Can I ask what your never-exiting loop is doing? > > what is the purpose of the gen_server if the process will forever loop in > loop and not deal with gen_server messages? > > > On 17 April 2013 19:33, Lee Sylvester wrote: > >> Hey guys, >> >> So, I've hit a "best practice" conundrum in OTP; I have a server >> utilising gen_server for a RabbitMQ consumer. In the init of that >> gen_server, I'm setting up a RabbitMQ connection, but I also need to start >> a loop. My guess was that I shouldn't call this before init exits, as I >> was passing the Connection and Channel objects to state for handling >> elsewhere. If I handle the loop in init, surely it will never return? >> >> To simplify what I'm saying (as I'm confusing myself here), here's my >> code: >> >> init([]) -> >> {ok, Connection} = amqp_connection:start(#amqp_params_network{ >> host="localhost" }), >> {ok, Channel} = amqp_connection:open_channel(Connection), >> amqp_channel:call(Channel, #'exchange.declare'{exchange = >> <<"user_msgs">>, >> type = <<"direct">>}), >> #'queue.declare_ok'{queue = Queue} = >> amqp_channel:call(Channel, #'queue.declare'{exclusive = true}), >> State = {Channel, Connection}, >> amqp_channel:call(Channel, #'queue.bind'{exchange = <<"user_msgs">>, >> routing_key = >> term_to_binary(node(self())), >> queue = Queue}), >> amqp_channel:subscribe(Channel, #'basic.consume'{queue = Queue, >> no_ack = true}, self()), >> receive >> #'basic.consume_ok'{} -> ok >> end, >> loop(Channel), >> {ok, State}. >> >> Now, if I don't put the loop in my init, then how can I be sure that the >> loop is called every time the gen_server restarts? Can someone please >> suggest the "right" way to call the loop in my gen_server? >> >> Thanks loads, >> Lee >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Wed Apr 17 21:50:11 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 17 Apr 2013 20:50:11 +0100 Subject: [erlang-questions] Message Loop in Gen_Server In-Reply-To: References: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> Message-ID: <55B621A6-77C9-49A2-BA34-8FA46F6CDF34@gmail.com> On 17 Apr 2013, at 20:13, JD Bothma wrote: > > The easiest way to get started with something good enough, though, is gen_server :) > That's usually true, but if you're consuming RabbitMQ messages then amqp_selective_consumer or amqp_gen_consumer would probably be the simplest choice. :) Cheers, Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: From olav@REDACTED Wed Apr 17 22:26:31 2013 From: olav@REDACTED (Olav Frengstad) Date: Wed, 17 Apr 2013 22:26:31 +0200 Subject: [erlang-questions] Wildcard matching a list of strings vs atoms/binaries Message-ID: Hey, I'm getting some unexpected behavior when trying to pattern match the head of a list. When matching against a list of integers I can successfully do a wildcard match: 27> ("ab ++ _) = "abcd". But when trying to do the same with a list of atoms i get the error: * 1: illegal pattern: 28> ([a,b,c] ++ _) = [a,b,c,d]. * 1: illegal pattern So just making sure there's no magic in string matching i also try with an actual list of integers: 29> ([0,1,2] ++ _) = [0,1,2,3,4]. [0,1,2,3,4] Why can't I pattern match a lists with other values integers? What i'm looking to achieve is to provide wildcard matching on a gen_event handler so I can do matches like this: handle_event((Match ++ _), State#state{match = Match}) -> % handle Any other alternatives is appreciated, current implementation uses lists:suffix/2 -- Med Vennlig Hilsen Olav Frengstad Systemutvikler // FWT +47 920 42 090 -------------- next part -------------- An HTML attachment was scrubbed... URL: From co7eb@REDACTED Wed Apr 17 22:51:14 2013 From: co7eb@REDACTED (=?iso-8859-1?Q?Ivan_Carmenates_Garc=EDa?=) Date: Wed, 17 Apr 2013 16:51:14 -0400 Subject: [erlang-questions] Wildcard matching a list of strings vs atoms/binaries In-Reply-To: References: Message-ID: <000c01ce3bad$523662c0$f6a32840$@frcuba.co.cu> Greetings Olav, I don?t understand why you try to math using that kind pattern, you can use the common way. Instead of using ([0,1,2] ++ _) = [0,1,2,3,4] you can use [0,1,2| _] = [0,1,2,3,4] pattern matching. Best, Ivan. De: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] En nombre de Olav Frengstad Enviado el: mi?rcoles, 17 de abril de 2013 16:27 Para: erlang-questions@REDACTED Questions Asunto: [erlang-questions] Wildcard matching a list of strings vs atoms/binaries Hey, I'm getting some unexpected behavior when trying to pattern match the head of a list. When matching against a list of integers I can successfully do a wildcard match: 27> ("ab ++ _) = "abcd". But when trying to do the same with a list of atoms i get the error: * 1: illegal pattern: 28> ([a,b,c] ++ _) = [a,b,c,d]. * 1: illegal pattern So just making sure there's no magic in string matching i also try with an actual list of integers: 29> ([0,1,2] ++ _) = [0,1,2,3,4]. [0,1,2,3,4] Why can't I pattern match a lists with other values integers? What i'm looking to achieve is to provide wildcard matching on a gen_event handler so I can do matches like this: handle_event((Match ++ _), State#state{match = Match}) -> % handle Any other alternatives is appreciated, current implementation uses lists:suffix/2 -- Med Vennlig Hilsen Olav Frengstad Systemutvikler // FWT +47 920 42 090 -------------- next part -------------- An HTML attachment was scrubbed... URL: From olav@REDACTED Wed Apr 17 23:08:52 2013 From: olav@REDACTED (Olav Frengstad) Date: Wed, 17 Apr 2013 23:08:52 +0200 Subject: [erlang-questions] Wildcard matching a list of strings vs atoms/binaries In-Reply-To: <000c01ce3bad$523662c0$f6a32840$@frcuba.co.cu> References: <000c01ce3bad$523662c0$f6a32840$@frcuba.co.cu> Message-ID: Hey Ivan, Please correct me if I'm wrong, but when composing the match from an existing variable AFAIK there is no other way: 1> X = [a, b, c]. [a,b,c] 2> [X | _] = [a, b, c, d]. ** exception error: no match of right hand side value [a,b,c,d] 2013/4/17 Ivan Carmenates Garc?a > Greetings Olav,**** > > ** ** > > I don?t understand why you try to math using that kind pattern, you can > use the common way.**** > > ** ** > > Instead of using ([0,1,2] ++ _) = [0,1,2,3,4] you can use [0,1,2| _] = > [0,1,2,3,4] pattern matching.**** > > ** ** > > ** ** > > Best,**** > > Ivan.**** > > ** ** > > ** ** > > ** ** > > ** ** > > ** ** > > ** ** > > ** ** > > ** ** > > *De:* erlang-questions-bounces@REDACTED [mailto: > erlang-questions-bounces@REDACTED] *En nombre de *Olav Frengstad > *Enviado el:* mi?rcoles, 17 de abril de 2013 16:27 > *Para:* erlang-questions@REDACTED Questions > *Asunto:* [erlang-questions] Wildcard matching a list of strings vs > atoms/binaries**** > > ** ** > > Hey,**** > > ** ** > > I'm getting some unexpected behavior when trying to pattern match the head > of a list.**** > > ** ** > > When matching against a list of integers I can successfully do a wildcard > match:**** > > 27> ("ab ++ _) = "abcd".**** > > ** ** > > But when trying to do the same with a list of atoms i get the error: * 1: > illegal pattern:**** > > 28> ([a,b,c] ++ _) = [a,b,c,d].**** > > * 1: illegal pattern**** > > ** ** > > So just making sure there's no magic in string matching i also try with an > actual list of integers:**** > > 29> ([0,1,2] ++ _) = [0,1,2,3,4]. **** > > [0,1,2,3,4]**** > > ** ** > > Why can't I pattern match a lists with other values integers?**** > > ** ** > > What i'm looking to achieve is to provide wildcard matching on a gen_event > handler so I can do matches like this:**** > > ** ** > > handle_event((Match ++ _), State#state{match = Match}) ->**** > > % handle**** > > ** ** > > Any other alternatives is appreciated, current implementation uses > lists:suffix/2**** > > ** ** > > -- > Med Vennlig Hilsen**** > > Olav Frengstad**** > > ** ** > > Systemutvikler // FWT**** > > +47 920 42 090**** > -- Med Vennlig Hilsen Olav Frengstad Systemutvikler // FWT +47 920 42 090 -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablo.platt@REDACTED Wed Apr 17 23:27:21 2013 From: pablo.platt@REDACTED (pablo platt) Date: Thu, 18 Apr 2013 00:27:21 +0300 Subject: [erlang-questions] Listen to SSL and normal TCP connections on the same port In-Reply-To: <56E6F178-2D17-4A6C-A77B-382AE2BB617A@rogvall.se> References: <56E6F178-2D17-4A6C-A77B-382AE2BB617A@rogvall.se> Message-ID: Thank you for the suggestions. What Tony suggests looks like what I was trying to do. What I'm trying to do is proxy HTTPS and RTMP/S on the same port. Corporate firewalls sometimes allow SSL or even normal HTTP or TCP pass on port 443 because they assume it is secure. I couldn't find a proxy server like SQUID or haproxy that can separate RTMP from HTTP so I thought I'll use erlang for that. Currently I have nginx infront of cowboy but if I'll put cowboy in the front I won't be able to use nginx for serving static files. Is there a light proxy I can put infront of nginx to separate RTMP from HTTP so I'll be able to pass RTMP requests to the media server (oms) and the HTTP requests to nginx? nginx will serve static files and cowboy the dynamic requests. Thanks On Wed, Apr 17, 2013 at 11:15 AM, Tony Rogvall wrote: > Hi! > > In general it is not possible to detect and upgrade a socket to ssl, BUT > if the client initiate with some > known message data that can be distinguished from the SSL hello message, > like HTTP requests then > you can do something like (I have, and it works :-): > > The socket should already be in {packet,0} packet mode, set > {active,false}, {packet,0} in the options to listen. > Then either read in passive mode: > > {ok,Header} = gen_tcp:recv(Socket, MinHeaderBlockSize), %% say 8 bytes ? > > Or something like this in active once mode > inet:setopts(Socket, [{active, once}]), %% not easy to control the number > of bytes received here yet (there is an EEP!) > Header = > receive > {tcp, Socket, Header} -> ok > end > Then do a match on the data that the client normally send with some known > SSL intro messages: > > Type = detect(Header), > > detect(<<"GET", _/binary>>) -> plain; > detect(<<"POST", _/binary>>) -> plain; > detect(<<"OPTIONS", _/binary>>) -> plain; > detect(<<"TRACE", _/binary>>) -> plain; > ... > detect(<<1:1,_Len:15,1:8,_Version:16, _/binary>>) -> ssl; > detect(<>) -> > if ContentType == 22 -> %% HANDSHAKE > ssl; > true -> > undefined > end; > detect(_) -> > undefined. > > Then switch to ssl if that was detected: > > if Type =:= ssl -> > %% Socket MUST be in passive mode at this point > gen_tcp:unrecv(Socket, Header), %% push back SSL data > {ok, SSLSocket} = ssl:ssl_accept(Socket, SSLOptions, SSLAcceptTimeout), > do_ssl_request(SSLSocket); > true -> > do_plain_request(Socket) > end > > /Tony > > On 17 apr 2013, at 09:41, Ingela Andin wrote: > > Hi! > > 2013/4/17 pablo platt > >> Hi, >> >> Is it possible to accept SSL connections and normal TCP connections on >> the same port? >> Maybe accept normal TCP connections. If the connection is SSL connection >> pass it to the ssl module and if it is a normal TCP connection just handle >> the socket. >> > > It is possible to upgrade a tcp socket to an ssl socket. > > If it is possible, how can I distinct SSL from non SSL connections? >> > > That is the hard part. You must have some scheme to negotiate the upgrade > with the client over plain tcp. (Like STARTTLS, HTTP Connect etc) > > >> How can I pass the socket to the ssl module? >> > > ssl:ssl_accept(TcpSocket, SslOptions) or ssl:connect(TcpSocket, > SslOptions) > make sure the socket is passive ({active, false}) before you make the call. > > Regards Ingela Erlang/OTP team Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > "Installing applications can lead to corruption over time. Applications > gradually write over each other's libraries, partial upgrades occur, user > and system errors happen, and minute changes may be unnoticeable and > difficult to fix" > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yogishb@REDACTED Wed Apr 17 23:29:34 2013 From: yogishb@REDACTED (Yogish Baliga) Date: Wed, 17 Apr 2013 14:29:34 -0700 Subject: [erlang-questions] Wildcard matching a list of strings vs atoms/binaries In-Reply-To: References: <000c01ce3bad$523662c0$f6a32840$@frcuba.co.cu> Message-ID: This will do instead 1> X = [a, b, c]. [a,b,c] 2> Y = [a, b, c, d]. [a,b,c,d] 3> X == lists:sublist(Y, length(X)). true On Wed, Apr 17, 2013 at 2:08 PM, Olav Frengstad wrote: > Hey Ivan, > > Please correct me if I'm wrong, but when composing the match from an > existing variable AFAIK there is no other way: > > 1> X = [a, b, c]. > [a,b,c] > 2> [X | _] = [a, b, c, d]. > ** exception error: no match of right hand side value [a,b,c,d] > > > > 2013/4/17 Ivan Carmenates Garc?a > >> Greetings Olav,**** >> >> ** ** >> >> I don?t understand why you try to math using that kind pattern, you can >> use the common way.**** >> >> ** ** >> >> Instead of using ([0,1,2] ++ _) = [0,1,2,3,4] you can use [0,1,2| _] = >> [0,1,2,3,4] pattern matching.**** >> >> ** ** >> >> ** ** >> >> Best,**** >> >> Ivan.**** >> >> ** ** >> >> ** ** >> >> ** ** >> >> ** ** >> >> ** ** >> >> ** ** >> >> ** ** >> >> ** ** >> >> *De:* erlang-questions-bounces@REDACTED [mailto: >> erlang-questions-bounces@REDACTED] *En nombre de *Olav Frengstad >> *Enviado el:* mi?rcoles, 17 de abril de 2013 16:27 >> *Para:* erlang-questions@REDACTED Questions >> *Asunto:* [erlang-questions] Wildcard matching a list of strings vs >> atoms/binaries**** >> >> ** ** >> >> Hey,**** >> >> ** ** >> >> I'm getting some unexpected behavior when trying to pattern match the >> head of a list.**** >> >> ** ** >> >> When matching against a list of integers I can successfully do a wildcard >> match:**** >> >> 27> ("ab ++ _) = "abcd".**** >> >> ** ** >> >> But when trying to do the same with a list of atoms i get the error: * 1: >> illegal pattern:**** >> >> 28> ([a,b,c] ++ _) = [a,b,c,d].**** >> >> * 1: illegal pattern**** >> >> ** ** >> >> So just making sure there's no magic in string matching i also try with >> an actual list of integers:**** >> >> 29> ([0,1,2] ++ _) = [0,1,2,3,4]. **** >> >> [0,1,2,3,4]**** >> >> ** ** >> >> Why can't I pattern match a lists with other values integers?**** >> >> ** ** >> >> What i'm looking to achieve is to provide wildcard matching on a >> gen_event handler so I can do matches like this:**** >> >> ** ** >> >> handle_event((Match ++ _), State#state{match = Match}) ->**** >> >> % handle**** >> >> ** ** >> >> Any other alternatives is appreciated, current implementation uses >> lists:suffix/2**** >> >> ** ** >> >> -- >> Med Vennlig Hilsen**** >> >> Olav Frengstad**** >> >> ** ** >> >> Systemutvikler // FWT**** >> >> +47 920 42 090**** >> > > > > -- > Med Vennlig Hilsen > Olav Frengstad > > Systemutvikler // FWT > +47 920 42 090 > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Wed Apr 17 23:40:53 2013 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Wed, 17 Apr 2013 23:40:53 +0200 Subject: [erlang-questions] Wildcard matching a list of strings vs atoms/binaries In-Reply-To: References: Message-ID: <516F16E5.30405@gmail.com> On 2013-04-17 22:26, Olav Frengstad wrote: > Hey, > > I'm getting some unexpected behavior when trying to pattern match the > head of a list. > > When matching against a list of integers I can successfully do a > wildcard match: > 27> ("ab ++ _) = "abcd". > > But when trying to do the same with a list of atoms i get the error: * > 1: illegal pattern: > 28> ([a,b,c] ++ _) = [a,b,c,d]. > * 1: illegal pattern > > So just making sure there's no magic in string matching i also try with > an actual list of integers: > 29> ([0,1,2] ++ _) = [0,1,2,3,4]. > [0,1,2,3,4] > > Why can't I pattern match a lists with other values integers? The Erlang Reference Manual (http://www.erlang.org/doc/reference_manual/expressions.html#id76648) says the following: When matching strings, the following is a valid pattern: f("prefix" ++ Str) -> ... This is syntactic sugar for the equivalent, but harder to read f([$p,$r,$e,$f,$i,$x | Str]) -> ... Since strings are really just lists of integers, and the double-quoted string syntax has been expanded to actual lists by the time the compiler gets the code, it actually works on lists of any integers, as in your example: ([0,1,2] ++ _) = [0,1,2,3,4]. There's no particular reason that this syntax couldn't be allowed for lists of arbitrary elements, but apparently, that hasn't been done. In fact, it seems that the compiler (in the module sys_pre_expand) doesn't care about what the elements are, but the erl_lint module that checks the code after parsing explicitly disallows this syntax for other things than strings/lists of integers. For now, you'll just have to use the normal list pattern syntax for matching prefixes: [a,b,c | _] = [a,b,c,d]. In any case, the (Prefix++Tail) pattern can only be used when Prefix is a fixed-length list literal (so it can be rewritten to the last form at compile time). Prefix can never be a variable. /Richard From olav@REDACTED Wed Apr 17 23:43:32 2013 From: olav@REDACTED (Olav Frengstad) Date: Wed, 17 Apr 2013 23:43:32 +0200 Subject: [erlang-questions] Wildcard matching a list of strings vs atoms/binaries In-Reply-To: <516F16E5.30405@gmail.com> References: <516F16E5.30405@gmail.com> Message-ID: Thanks for the explanation Richard, that answers my question. lists:prefix/2 seems the obvious way then. 2013/4/17 Richard Carlsson > On 2013-04-17 22:26, Olav Frengstad wrote: > >> Hey, >> >> I'm getting some unexpected behavior when trying to pattern match the >> head of a list. >> >> When matching against a list of integers I can successfully do a >> wildcard match: >> 27> ("ab ++ _) = "abcd". >> >> But when trying to do the same with a list of atoms i get the error: * >> 1: illegal pattern: >> 28> ([a,b,c] ++ _) = [a,b,c,d]. >> * 1: illegal pattern >> >> So just making sure there's no magic in string matching i also try with >> an actual list of integers: >> 29> ([0,1,2] ++ _) = [0,1,2,3,4]. >> [0,1,2,3,4] >> >> Why can't I pattern match a lists with other values integers? >> > > The Erlang Reference Manual (http://www.erlang.org/doc/** > reference_manual/expressions.**html#id76648) > says the following: > > When matching strings, the following is a valid pattern: > > f("prefix" ++ Str) -> ... > > This is syntactic sugar for the equivalent, but harder to read > > f([$p,$r,$e,$f,$i,$x | Str]) -> ... > > Since strings are really just lists of integers, and the double-quoted > string syntax has been expanded to actual lists by the time the compiler > gets the code, it actually works on lists of any integers, as in your > example: > > > ([0,1,2] ++ _) = [0,1,2,3,4]. > > There's no particular reason that this syntax couldn't be allowed for > lists of arbitrary elements, but apparently, that hasn't been done. In > fact, it seems that the compiler (in the module sys_pre_expand) doesn't > care about what the elements are, but the erl_lint module that checks the > code after parsing explicitly disallows this syntax for other things than > strings/lists of integers. > > For now, you'll just have to use the normal list pattern syntax for > matching prefixes: > > [a,b,c | _] = [a,b,c,d]. > > In any case, the (Prefix++Tail) pattern can only be used when Prefix is a > fixed-length list literal (so it can be rewritten to the last form at > compile time). Prefix can never be a variable. > > /Richard > > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -- Med Vennlig Hilsen Olav Frengstad Systemutvikler // FWT +47 920 42 090 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrtndimitrov@REDACTED Thu Apr 18 09:38:28 2013 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Thu, 18 Apr 2013 10:38:28 +0300 Subject: [erlang-questions] Embed a pid into string to be dynamically compiled Message-ID: <516FA2F4.6070206@gmail.com> Hello, I need to build dynamically a module. Inside one of its functions, it has to send a message to Pid. How to compile it? The easiest solution I can think of is to do pid_to_list(Pid) and inside the function Pid = list_to_pid("<0.22.0>") but I really would like to avoid these extra operations. Thanks, Martin From bengt.kleberg@REDACTED Thu Apr 18 09:44:47 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 18 Apr 2013 09:44:47 +0200 Subject: [erlang-questions] Embed a pid into string to be dynamically compiled In-Reply-To: <516FA2F4.6070206@gmail.com> References: <516FA2F4.6070206@gmail.com> Message-ID: <1366271087.4827.13.camel@sekic1152.rnd.ki.sw.ericsson.se> Greetings, Would it be possible for you to build the module with an interface function that accepts the pid as its argument? bengt On Thu, 2013-04-18 at 10:38 +0300, Martin Dimitrov wrote: > Hello, > > I need to build dynamically a module. Inside one of its functions, it > has to send a message to Pid. How to compile it? The easiest solution I > can think of is to do pid_to_list(Pid) and inside the function Pid = > list_to_pid("<0.22.0>") but I really would like to avoid these extra > operations. > > Thanks, > > Martin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mrtndimitrov@REDACTED Thu Apr 18 09:48:14 2013 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Thu, 18 Apr 2013 10:48:14 +0300 Subject: [erlang-questions] Embed a pid into string to be dynamically compiled In-Reply-To: <1366271087.4827.13.camel@sekic1152.rnd.ki.sw.ericsson.se> References: <516FA2F4.6070206@gmail.com> <1366271087.4827.13.camel@sekic1152.rnd.ki.sw.ericsson.se> Message-ID: <516FA53E.8040504@gmail.com> Unfortunately not. The function is called from some macros and the function is either a do-nothing one or compiled to use the Pid. Martin On 4/18/2013 10:44 AM, Bengt Kleberg wrote: > Greetings, > > Would it be possible for you to build the module with an interface > function that accepts the pid as its argument? > > > bengt > > On Thu, 2013-04-18 at 10:38 +0300, Martin Dimitrov wrote: >> Hello, >> >> I need to build dynamically a module. Inside one of its functions, it >> has to send a message to Pid. How to compile it? The easiest solution I >> can think of is to do pid_to_list(Pid) and inside the function Pid = >> list_to_pid("<0.22.0>") but I really would like to avoid these extra >> operations. >> >> Thanks, >> >> Martin >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From bengt.kleberg@REDACTED Thu Apr 18 09:56:19 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 18 Apr 2013 09:56:19 +0200 Subject: [erlang-questions] Embed a pid into string to be dynamically compiled In-Reply-To: <516FA53E.8040504@gmail.com> References: <516FA2F4.6070206@gmail.com> <1366271087.4827.13.camel@sekic1152.rnd.ki.sw.ericsson.se> <516FA53E.8040504@gmail.com> Message-ID: <1366271779.4827.17.camel@sekic1152.rnd.ki.sw.ericsson.se> Depending on how the module is used it might be possible to build and load it, then use the aforementioned interface function to start a process that remembers the pid. The the call from the macro will ask the process for the pid... bengt On Thu, 2013-04-18 at 10:48 +0300, Martin Dimitrov wrote: > Unfortunately not. The function is called from some macros and the > function is either a do-nothing one or compiled to use the Pid. > > Martin > > On 4/18/2013 10:44 AM, Bengt Kleberg wrote: > > Greetings, > > > > Would it be possible for you to build the module with an interface > > function that accepts the pid as its argument? > > > > > > bengt > > > > On Thu, 2013-04-18 at 10:38 +0300, Martin Dimitrov wrote: > >> Hello, > >> > >> I need to build dynamically a module. Inside one of its functions, it > >> has to send a message to Pid. How to compile it? The easiest solution I > >> can think of is to do pid_to_list(Pid) and inside the function Pid = > >> list_to_pid("<0.22.0>") but I really would like to avoid these extra > >> operations. > >> > >> Thanks, > >> > >> Martin > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > From vladdu55@REDACTED Thu Apr 18 09:56:03 2013 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 18 Apr 2013 09:56:03 +0200 Subject: [erlang-questions] Embed a pid into string to be dynamically compiled In-Reply-To: <516FA53E.8040504@gmail.com> References: <516FA2F4.6070206@gmail.com> <1366271087.4827.13.camel@sekic1152.rnd.ki.sw.ericsson.se> <516FA53E.8040504@gmail.com> Message-ID: Hi, If it's a globally known process that you want to send to, then you should probably register a name for it and use that instead of the pid. Usually there are simpler ways to solve problems than dynamically compiling code, but it depends a lot on your use case. regards, Vlad On Thu, Apr 18, 2013 at 9:48 AM, Martin Dimitrov wrote: > Unfortunately not. The function is called from some macros and the > function is either a do-nothing one or compiled to use the Pid. > > Martin > > On 4/18/2013 10:44 AM, Bengt Kleberg wrote: > > Greetings, > > > > Would it be possible for you to build the module with an interface > > function that accepts the pid as its argument? > > > > > > bengt > > > > On Thu, 2013-04-18 at 10:38 +0300, Martin Dimitrov wrote: > >> Hello, > >> > >> I need to build dynamically a module. Inside one of its functions, it > >> has to send a message to Pid. How to compile it? The easiest solution I > >> can think of is to do pid_to_list(Pid) and inside the function Pid = > >> list_to_pid("<0.22.0>") but I really would like to avoid these extra > >> operations. > >> > >> Thanks, > >> > >> Martin > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrtndimitrov@REDACTED Thu Apr 18 10:06:11 2013 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Thu, 18 Apr 2013 11:06:11 +0300 Subject: [erlang-questions] Embed a pid into string to be dynamically compiled In-Reply-To: References: <516FA2F4.6070206@gmail.com> <1366271087.4827.13.camel@sekic1152.rnd.ki.sw.ericsson.se> <516FA53E.8040504@gmail.com> Message-ID: <516FA973.5050208@gmail.com> Yes, I can register a name for the process! Thank you. Martin On 4/18/2013 10:56 AM, Vlad Dumitrescu wrote: > Hi, > > If it's a globally known process that you want to send to, then you should > probably register a name for it and use that instead of the pid. > > Usually there are simpler ways to solve problems than dynamically compiling > code, but it depends a lot on your use case. > > regards, > Vlad > > > > On Thu, Apr 18, 2013 at 9:48 AM, Martin Dimitrov wrote: > >> Unfortunately not. The function is called from some macros and the >> function is either a do-nothing one or compiled to use the Pid. >> >> Martin >> >> On 4/18/2013 10:44 AM, Bengt Kleberg wrote: >>> Greetings, >>> >>> Would it be possible for you to build the module with an interface >>> function that accepts the pid as its argument? >>> >>> >>> bengt >>> >>> On Thu, 2013-04-18 at 10:38 +0300, Martin Dimitrov wrote: >>>> Hello, >>>> >>>> I need to build dynamically a module. Inside one of its functions, it >>>> has to send a message to Pid. How to compile it? The easiest solution I >>>> can think of is to do pid_to_list(Pid) and inside the function Pid = >>>> list_to_pid("<0.22.0>") but I really would like to avoid these extra >>>> operations. >>>> >>>> Thanks, >>>> >>>> Martin >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > From ali.sabil@REDACTED Thu Apr 18 10:15:44 2013 From: ali.sabil@REDACTED (Ali Sabil) Date: Thu, 18 Apr 2013 10:15:44 +0200 Subject: [erlang-questions] Message Loop in Gen_Server In-Reply-To: <55B621A6-77C9-49A2-BA34-8FA46F6CDF34@gmail.com> References: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> <55B621A6-77C9-49A2-BA34-8FA46F6CDF34@gmail.com> Message-ID: I case you need something that is OTP compliant while allowing you to drive the receive loop yourself, you can take a look at gen_process: https://github.com/asabil/gen_process which is a lower level alternative to gen_server and gen_fsm Cheers, Ali On Wed, Apr 17, 2013 at 9:50 PM, Tim Watson wrote: > On 17 Apr 2013, at 20:13, JD Bothma wrote: > > > The easiest way to get started with something good enough, though, is > gen_server :) > > > That's usually true, but if you're consuming RabbitMQ messages > then amqp_selective_consumer or amqp_gen_consumer would probably be the > simplest choice. :) > > Cheers, > Tim > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Thu Apr 18 10:42:26 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 18 Apr 2013 09:42:26 +0100 Subject: [erlang-questions] Message Loop in Gen_Server In-Reply-To: References: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> <55B621A6-77C9-49A2-BA34-8FA46F6CDF34@gmail.com> Message-ID: <2901E7E1-CABF-4446-9D2D-60FE3DE1362C@gmail.com> Just to clarify - if you're consuming RabbitMQ messages then amqp_selective_consumer or amqp_gen_consumer will be the simplest choice. All the other advice is fine and good, but the OP is clearly relatively new to Erlang/OTP and those simplifying behaviours are clearly more appropriate to his needs. On 18 Apr 2013, at 09:15, Ali Sabil wrote: > I case you need something that is OTP compliant while allowing you to drive the receive loop yourself, you can take a look at gen_process: https://github.com/asabil/gen_process which is a lower level alternative to gen_server and gen_fsm > I've not seen this before - it sounds interesting so I'll take a look. :) Cheers, Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: From ali.sabil@REDACTED Thu Apr 18 11:09:34 2013 From: ali.sabil@REDACTED (Ali Sabil) Date: Thu, 18 Apr 2013 11:09:34 +0200 Subject: [erlang-questions] Message Loop in Gen_Server In-Reply-To: <2901E7E1-CABF-4446-9D2D-60FE3DE1362C@gmail.com> References: <3F5D1F37-6989-41F4-B443-A5705BE5477F@gmail.com> <55B621A6-77C9-49A2-BA34-8FA46F6CDF34@gmail.com> <2901E7E1-CABF-4446-9D2D-60FE3DE1362C@gmail.com> Message-ID: I have been using it in production under heavy load for quite some time so it's very stable. I haven't had time to document it though, so patches are very welcome :) On Thu, Apr 18, 2013 at 10:42 AM, Tim Watson wrote: > Just to clarify - if you're consuming RabbitMQ messages > then amqp_selective_consumer or amqp_gen_consumer will be the simplest > choice. All the other advice is fine and good, but the OP is clearly > relatively new to Erlang/OTP and those simplifying behaviours are clearly > more appropriate to his needs. > > On 18 Apr 2013, at 09:15, Ali Sabil wrote: > > I case you need something that is OTP compliant while allowing you to > drive the receive loop yourself, you can take a look at gen_process: > https://github.com/asabil/gen_process which is a lower level alternative > to gen_server and gen_fsm > > > I've not seen this before - it sounds interesting so I'll take a look. :) > > Cheers, > Tim > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yashgt@REDACTED Thu Apr 18 13:26:18 2013 From: yashgt@REDACTED (yashgt@REDACTED) Date: Thu, 18 Apr 2013 11:26:18 +0000 Subject: [erlang-questions] Programmatically replicating Mnesia to DB2 Message-ID: <516fd861.03c5e00a.1261.ffffe458@mx.google.com> An HTML attachment was scrubbed... URL: From mail@REDACTED Thu Apr 18 20:08:43 2013 From: mail@REDACTED (Dominic Williams) Date: Thu, 18 Apr 2013 20:08:43 +0200 Subject: [erlang-questions] Programmatically replicating Mnesia to DB2 In-Reply-To: <516fd861.03c5e00a.1261.ffffe458@mx.google.com> References: <516fd861.03c5e00a.1261.ffffe458@mx.google.com> Message-ID: <21CA8EBC-8931-4B64-A6B9-1039338EA18E@dominicwilliams.net> Hi Yash, Le 18 avr. 2013 ? 13:26, yashgt@REDACTED a ?crit : > Hi, > > My erlang application makes thousands of entries into an Mnesia table. I would like the same records to be also inserted into an equivalent table in a DB2 database without blocking the call that inserts into Mnesia. Is there a hook that Mnesia provides where all insertions can be trapped and where I can put my code that inserts the same record into DB2? > > Thanks, > > Yash You could have a process subscribed to mnesia table events (see mnesia:subscribe and mnesia user guide). It would receive a message for each write and could duplicate the write to DB2. Regards, Dominic Williams http://dominicwilliams.net ---- From alex.arnon@REDACTED Thu Apr 18 23:25:40 2013 From: alex.arnon@REDACTED (Alex Arnon) Date: Fri, 19 Apr 2013 00:25:40 +0300 Subject: [erlang-questions] [ANN] Helperl 0.2.0 (Alpha) Message-ID: Hi All, I am releasing the first version of Helperl, a convenience wrapper for JInterface. Project home page (at Bit Bucket): https://bitbucket.org/alexarnon/helperl The idea was to try and make JInterface usage almost pleasant, because really guys, that API is still rocking all the hotness of Java 1.4. Helperl has been successfully used in integration experiments. However, it is the first released version, and should be considered Alpha quality. Examples: * Construction of simple types, lists and tuples: -------------------- 12345 <- erl(12345) <<"hello">> <- erl("hello") 'hello' <- eAtom("hello") [12345, 'qwerty'] <- erl(12345, eAtom("qwerty")) {ping, [123, 456]} <- eTuple(eAtom("ping"), erl(123, 456)) -------------------- * Pattern matching: Match {ping, From, Message} where is_pid(From) and is_list(Message), and extract the values. -------------------- CompositePattern p = pAnd( pTuple( eAtom("ping"), p_("From"), p_("Message") ), pIsPid("From"), pIsList("Message") ); Map result = p.match(someInput); if (result == null) { // No match. } else { // Extract! OtpErlangObject from = result.get("From"); OtpErlangObject message = result.get("Message"); // ... } -------------------- To download the source code and jars, please look at the Bit Bucket repo, or check out Maven Central, using: net.shambolica.helperl helperl 0.2.0 Note that the Maven artifact has a dependency on JInterface 1.5.6. Please feel free to provide feedback! Cheers, Alex. -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul@REDACTED Fri Apr 19 00:15:53 2013 From: paul@REDACTED (Paul Rubin) Date: Thu, 18 Apr 2013 15:15:53 -0700 Subject: [erlang-questions] distributed-erlang listener config and ssh tunnelling Message-ID: Hi, I'm running an erlang server (with -noshell and -detached) on a remote box, and would like to control it from a remote shell, plus run the process observer, debugger, and so on from my workstation. For security reasons I don't want to open any ports on the remote box. Instead I'd like for the distribution stuff to listen on local ports, that I can then forward to my workstation through ssh. Right now the epmd and other distribution ports listen on all interfaces (0.0.0.0) which is not disastrous because I can block those ports with iptables, but ideally I'd like to configure them to listen only on 127.0.0.1. Anyone know if there is a way to do that? The other thing I'd hope to find is a recipe for tunnelling distributed erlang through ssh. Various docs and books mention this in passing as if it's routine, but none say exactly which ports have to be forwarded. I may be a little confused but there may be a missing piece of the puzzle, in getting epmd to hand off the forwarded port number instead of the port that the node actually listens to. Or does epmd actually proxy all the traffic to all local nodes? Is there a better way to do this stuff in general? All advice is appreciated. Thanks --Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Fri Apr 19 10:28:04 2013 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 19 Apr 2013 10:28:04 +0200 Subject: [erlang-questions] [ANN] Helperl 0.2.0 (Alpha) In-Reply-To: References: Message-ID: Hi Alex! Nice work! I like the API. We have some similar extensions to jinterfaces in erlide, which I didn't got around to publishing as a standalone library. If you feel like checking it out, it's at https://github.com/erlide/erlide/tree/pu/org.erlide.util/src/org/erlide/util/erlang. If you think that these are useful or interesting, we can look at how to integrate them. Some differences are that I implemented pattern matching with something like Bindings b = match("{ping, From:p, Message:l}", someInput); // ':p' means is_pid and ':l' means is_list OtpErlangPid pid = b.getPid("From"); Collection msg = b.getList("Message"); which I find easier to read, but is a bit difficult to extend (for example with more advanced guards, which you can add easily). I also chose to convert terms to Java objects by using a type signature, ? la printf. It is used for doing nicer looking RPCs to Erlang, for example rpc.call("mod", "fun", "ssolxi", ...args) will convert the args to (in order) two strings, a list of whatever ('x') and an integer. There is also a format function that looks like printf or io:format, but produces OtpErlangObjects instead: format("{hello, ~s, ~a, ~b}", "foo", "bar", "baz") --> {hello, "foo", bar, <<"baz">>} regards, Vlad On Thu, Apr 18, 2013 at 11:25 PM, Alex Arnon wrote: > Hi All, > > I am releasing the first version of Helperl, a convenience wrapper for > JInterface. > > Project home page (at Bit Bucket): https://bitbucket.org/alexarnon/helperl > > The idea was to try and make JInterface usage almost pleasant, because > really guys, that API is still rocking all the hotness of Java 1.4. > > Helperl has been successfully used in integration experiments. However, it > is the first released version, and should be considered Alpha quality. > > Examples: > > * Construction of simple types, lists and tuples: > > -------------------- > > 12345 <- erl(12345) > > <<"hello">> <- erl("hello") > > 'hello' <- eAtom("hello") > > [12345, 'qwerty'] <- erl(12345, eAtom("qwerty")) > > {ping, [123, 456]} <- eTuple(eAtom("ping"), erl(123, 456)) > > -------------------- > > > * Pattern matching: > > Match {ping, From, Message} where is_pid(From) and is_list(Message), and > extract the values. > > > -------------------- > > CompositePattern p = > pAnd( > pTuple( eAtom("ping"), p_("From"), p_("Message") ), > pIsPid("From"), > pIsList("Message") ); > > Map result = p.match(someInput); > > if (result == null) { > // No match. > } else { > // Extract! > OtpErlangObject from = result.get("From"); > OtpErlangObject message = result.get("Message"); > > // ... > } > > -------------------- > > To download the source code and jars, please look at the Bit Bucket repo, > or check out Maven Central, using: > > > net.shambolica.helperl > helperl > 0.2.0 > > > Note that the Maven artifact has a dependency on JInterface 1.5.6. > > Please feel free to provide feedback! > > > Cheers, > Alex. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From heinz@REDACTED Fri Apr 19 15:52:58 2013 From: heinz@REDACTED (Heinz Nikolaus Gies) Date: Fri, 19 Apr 2013 15:52:58 +0200 Subject: [erlang-questions] [ANN] Project FiFo webinar Message-ID: <7D480D7E-3166-49EF-8487-6395585CA82C@licenser.net> Hi everyone, I'm working on a (by now) quite big open source Erlang project for cloud management called Project FiFo and am going to hold an webinar regarding the architecture of the whole thing. I'll not deep dive into the erlang guts since it would blow up the timeframe but it still might be interesting and there is half a hour reserved for questions. The webinar is free so if this sounds interesting drop by! http://project-fifo.eventbrite.com/ Cheers, Heinz -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 841 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mediocregopher@REDACTED Fri Apr 19 21:24:12 2013 From: mediocregopher@REDACTED (Brian Picciano) Date: Fri, 19 Apr 2013 15:24:12 -0400 Subject: [erlang-questions] High lock contention on dist_tables Message-ID: We have a pool of 3 erlang nodes, all on different servers. Every afternoon, without fail, we start seeing lots of messages between the nodes start having really high latency, on the order of tens of seconds. Today we ran lcnt on them to see if there's anything there, and found that on one of the nodes dist_tables had a significantly higher lock percentage then anything else, and definitely higher then on the other boxes: (node@REDACTED)8> lcnt:conflicts(). lock id #tries #collisions collisions [%] time [us] duration [%] ----- --- ------- ------------ --------------- ---------- ------------- dist_table 1 3468191 1242055 35.8128 153712413 255.2521 run_queue 24 76969638 4088578 5.3119 14468656 24.0264 process_table 1 2015686 147148 7.3001 3208529 5.3280 timer_wheel 1 12214948 834737 6.8337 3076638 5.1090 timeofday 1 18231600 594487 3.2608 1491633 2.4770 ... while on the other boxes it had closer to 3. On the box with the high lock contention we also saw much higher load then on the other boxes. My question is: what is this lock? We couldn't find much online except that it appears to have to do with communication between nodes, but we're not sure what. Also, what, if anything, could we do to mitigate this problem? (We're running erlang 16B) -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.arnon@REDACTED Fri Apr 19 21:53:02 2013 From: alex.arnon@REDACTED (Alex Arnon) Date: Fri, 19 Apr 2013 22:53:02 +0300 Subject: [erlang-questions] [ANN] Helperl 0.2.0 (Alpha) In-Reply-To: References: Message-ID: Hi Vlad! Thanks for the reply, glad you like the API :) >From a quick look at your utils package, there are certainly some things I see Helperl is missing: - The Bindings interface as the result type of CompositePattern.match(...) - an easy addition. - The format(...) function's approach looks very useful. Particularly if it would be possible to compile it to a generator. - I like the the pattern matching format, and would like to be able to compile it to a CompositePattern - which can then be mixed with guards, kept statically etc. This would be more natural than Helperl's current verbose mapping. Do you have any wrapper for OtpMbox's, e.g. Helperl's ESimpleService, or any gen_server-alike? Cheers, Alex. On Fri, Apr 19, 2013 at 11:28 AM, Vlad Dumitrescu wrote: > Hi Alex! > > Nice work! I like the API. > > We have some similar extensions to jinterfaces in erlide, which I didn't > got around to publishing as a standalone library. If you feel like checking > it out, it's at > https://github.com/erlide/erlide/tree/pu/org.erlide.util/src/org/erlide/util/erlang. > If you think that these are useful or interesting, we can look at how to > integrate them. > > Some differences are that I implemented pattern matching with something > like > Bindings b = match("{ping, From:p, Message:l}", someInput); > // ':p' means is_pid and ':l' means is_list > OtpErlangPid pid = b.getPid("From"); > Collection msg = b.getList("Message"); > which I find easier to read, but is a bit difficult to extend (for example > with more advanced guards, which you can add easily). > > I also chose to convert terms to Java objects by using a type signature, ? > la printf. It is used for doing nicer looking RPCs to Erlang, for example rpc.call("mod", > "fun", "ssolxi", ...args) will convert the args to (in order) two > strings, a list of whatever ('x') and an integer. > > There is also a format function that looks like printf or io:format, but > produces OtpErlangObjects instead: > format("{hello, ~s, ~a, ~b}", "foo", "bar", "baz") --> {hello, "foo", bar, > <<"baz">>} > > regards, > Vlad > > > > On Thu, Apr 18, 2013 at 11:25 PM, Alex Arnon wrote: > >> Hi All, >> >> I am releasing the first version of Helperl, a convenience wrapper for >> JInterface. >> >> Project home page (at Bit Bucket): >> https://bitbucket.org/alexarnon/helperl >> >> The idea was to try and make JInterface usage almost pleasant, because >> really guys, that API is still rocking all the hotness of Java 1.4. >> >> Helperl has been successfully used in integration experiments. However, >> it is the first released version, and should be considered Alpha quality. >> >> Examples: >> >> * Construction of simple types, lists and tuples: >> >> -------------------- >> >> 12345 <- erl(12345) >> >> <<"hello">> <- erl("hello") >> >> 'hello' <- eAtom("hello") >> >> [12345, 'qwerty'] <- erl(12345, eAtom("qwerty")) >> >> {ping, [123, 456]} <- eTuple(eAtom("ping"), erl(123, 456)) >> >> -------------------- >> >> >> * Pattern matching: >> >> Match {ping, From, Message} where is_pid(From) and is_list(Message), and >> extract the values. >> >> >> -------------------- >> >> CompositePattern p = >> pAnd( >> pTuple( eAtom("ping"), p_("From"), p_("Message") ), >> pIsPid("From"), >> pIsList("Message") ); >> >> Map result = p.match(someInput); >> >> if (result == null) { >> // No match. >> } else { >> // Extract! >> OtpErlangObject from = result.get("From"); >> OtpErlangObject message = result.get("Message"); >> >> // ... >> } >> >> -------------------- >> >> To download the source code and jars, please look at the Bit Bucket repo, >> or check out Maven Central, using: >> >> >> net.shambolica.helperl >> helperl >> 0.2.0 >> >> >> Note that the Maven artifact has a dependency on JInterface 1.5.6. >> >> Please feel free to provide feedback! >> >> >> Cheers, >> Alex. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew@REDACTED Sat Apr 20 00:18:26 2013 From: andrew@REDACTED (Andrew Thompson) Date: Fri, 19 Apr 2013 18:18:26 -0400 Subject: [erlang-questions] [ANN] Lager 2.0.0RC2 Message-ID: <20130419221826.GD15637@hijacked.us> So, I announced lager 2.0.0rc1 during my Erlang factory talk ( http://www.youtube.com/watch?v=8BNpOHFvg_Q ), but forgot to announce it to the relevant mailing lists. So I'm announcing RC2 here. What is lager? Lager (as in the beer) is a logging framework for Erlang. Its purpose is to provide a more traditional way to perform logging in an erlang application that plays nicely with traditional UNIX logging tools like logrotate and syslog. https://github.com/basho/lager Changes since lager 1.2.2: * Metadata is passed to lager backends (breaks compatability with old * backends) * Switch from Mochiglobal to ETS for global 'config' storage * Add 'application' to default trace metadata, captured at compile time * Support for 'pretty printing' Cowboy (0.8+) and WebMachine errors * Record pretty printing with lager:pr/2 * Colored console support (requires R16, thanks DeadZen) * Overload protection from error_logger storms * Many performance improvements (often more than 2x as fast) * Many minor bug fixes Changes since lager 2.0.0rc1 * Support for logging unicode to files * New tracing engine, based on goldrush (thanks DeadZen) * Updated documentation for 2.0 Please give it a try, and let me know if you have any problems. Lager 2.0.0 final will ship with Riak 1.4. Andrew From comptekki@REDACTED Sat Apr 20 00:51:11 2013 From: comptekki@REDACTED (Wes James) Date: Fri, 19 Apr 2013 16:51:11 -0600 Subject: [erlang-questions] Cowboy at Erlang Factory SF 2013 Message-ID: Lo?c, I didn't get to go to Erlang SF 2013 this year, but I wanted to see your presentation. Are there any slides or video of your presentation? Thanks, Wes -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Sat Apr 20 00:56:10 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Sat, 20 Apr 2013 00:56:10 +0200 Subject: [erlang-questions] [ANN] Lager 2.0.0RC2 In-Reply-To: <20130419221826.GD15637@hijacked.us> References: <20130419221826.GD15637@hijacked.us> Message-ID: <5171CB8A.9020305@ninenines.eu> On 04/20/2013 12:18 AM, Andrew Thompson wrote: > So, I announced lager 2.0.0rc1 during my Erlang factory talk > ( http://www.youtube.com/watch?v=8BNpOHFvg_Q ), but forgot to announce > it to the relevant mailing lists. So I'm announcing RC2 here. Congratulations! > * New tracing engine, based on goldrush (thanks DeadZen) Glad to see this being made useful. The initial goldrush work was made by someone that I'm sure we all miss. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From essen@REDACTED Sat Apr 20 00:59:09 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Sat, 20 Apr 2013 00:59:09 +0200 Subject: [erlang-questions] Cowboy at Erlang Factory SF 2013 In-Reply-To: References: Message-ID: <5171CC3D.20308@ninenines.eu> On 04/20/2013 12:51 AM, Wes James wrote: > I didn't get to go to Erlang SF 2013 this year, but I wanted to see your > presentation. Are there any slides or video of your presentation? All slides of my talks can be found on the 99s website. Here's the link to this one: http://ninenines.eu/talks/cowboy-0.8/cowboy-0.8.html Guessing it wasn't added to the EF website because I didn't give a PDF. I'm sure the video will be posted soon. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From max.lapshin@REDACTED Sat Apr 20 05:23:15 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 20 Apr 2013 07:23:15 +0400 Subject: [erlang-questions] [ANN] Lager 2.0.0RC2 In-Reply-To: <5171CB8A.9020305@ninenines.eu> References: <20130419221826.GD15637@hijacked.us> <5171CB8A.9020305@ninenines.eu> Message-ID: Thanks! I had severe problems with tracing separate tag into file, will update and file an issues if it remains. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dujinfang@REDACTED Sat Apr 20 10:34:23 2013 From: dujinfang@REDACTED (Seven Du) Date: Sat, 20 Apr 2013 16:34:23 +0800 Subject: [erlang-questions] otp supervisor terminate callback Message-ID: Hi, I'd like to know is it possible to have a terminate callback that clean up things when the supervisor stops? Specifically I want to write an application which starts a supervisor, and add a handler to some existing gen_event by calling gen_event:add_handler() on init(), and I want to call gen_event:remove_handler() when the application is stop. Is there an easy way to do it? Thanks -- Seven Du http://www.freeswitch.org.cn http://about.me/dujinfang http://www.dujinfang.com Sent with Sparrow (http://www.sparrowmailapp.com/?sig) -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Sat Apr 20 11:34:05 2013 From: vances@REDACTED (Vance Shipley) Date: Sat, 20 Apr 2013 15:04:05 +0530 Subject: [erlang-questions] otp supervisor terminate callback In-Reply-To: References: Message-ID: To get the terminate/1 callback in a gen_server or gen_fsm behaviour module what you would do is to set the process flag to trap exits in your init/1 callback. On Apr 20, 2013 4:34 PM, "Seven Du" wrote: > Hi, > > I'd like to know is it possible to have a terminate callback that clean up > things when the supervisor stops? > > Specifically I want to write an application which starts a supervisor, and > add a handler to some existing gen_event by calling gen_event:add_handler() > on init(), and I want to call gen_event:remove_handler() when the > application is stop. Is there an easy way to do it? > > Thanks > > -- > Seven Du > http://www.freeswitch.org.cn > http://about.me/dujinfang > http://www.dujinfang.com > > Sent with Sparrow > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlo.bertoldi@REDACTED Sat Apr 20 14:45:24 2013 From: carlo.bertoldi@REDACTED (Carlo Bertoldi) Date: Sat, 20 Apr 2013 14:45:24 +0200 Subject: [erlang-questions] Webtool Message-ID: <51728DE4.8070400@ubiquity.it> Hi, a simple question: where can I find the HTML displayed by webtool? For example, relatively to appmon, I looked into otp_src_R16B/lib/appmon and lib/webtool/ but I could not find the appmon specific HTML. Thanks, Carlo From comptekki@REDACTED Sat Apr 20 18:53:11 2013 From: comptekki@REDACTED (Wes James) Date: Sat, 20 Apr 2013 10:53:11 -0600 Subject: [erlang-questions] Webtool In-Reply-To: <51728DE4.8070400@ubiquity.it> References: <51728DE4.8070400@ubiquity.it> Message-ID: What platform. On a mac and linux you can cd to the otp source folder and run this: find . -name "*webtool*" -print I got this: ./lib/webtool ./lib/webtool/doc/src/start_webtool.xml ./lib/webtool/doc/src/webtool.xml ./lib/webtool/doc/src/webtool_chapter.xml ./lib/webtool/ebin/webtool.app ./lib/webtool/ebin/webtool.appup ./lib/webtool/ebin/webtool.beam ./lib/webtool/ebin/webtool_sup.beam ./lib/webtool/priv/bin/start_webtool ./lib/webtool/priv/bin/start_webtool.bat ./lib/webtool/src/webtool.app.src ./lib/webtool/src/webtool.appup.src ./lib/webtool/src/webtool.erl ./lib/webtool/src/webtool_sup.erl If you want to find some text in a .erl file, run this: find . -name "*.erl" -exec grep "some-text" {} \; -print It might help in finding what you want. On Sat, Apr 20, 2013 at 6:45 AM, Carlo Bertoldi wrote: > Hi, a simple question: where can I find the HTML displayed by webtool? > For example, relatively to appmon, I looked into otp_src_R16B/lib/appmon > and lib/webtool/ but I could not find the appmon specific HTML. > Thanks, > Carlo > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From prof3ta@REDACTED Sat Apr 20 22:36:18 2013 From: prof3ta@REDACTED (Roberto Aloi) Date: Sat, 20 Apr 2013 22:36:18 +0200 Subject: [erlang-questions] Webtool In-Reply-To: <51728DE4.8070400@ubiquity.it> References: <51728DE4.8070400@ubiquity.it> Message-ID: Hi, the HTML is contained *inside* the webtool.erl file - belonging to the webtool application -. The module, 1207 lines long, alternates beautiful macros such as: -define(HTML_HEADER,"\r\n\r\nWebTool\r\n\r\n\r\n"). To functions returning HTML strings: show_started_apps(State)-> "
Started Tools
"++ list_started_apps(State)++"
 
 Stop a Tool:
  • Select the checkbox for each tool to stop.
  • Click on the button marked Stop.
 
". Oh, yeah! Roberto On Sat, Apr 20, 2013 at 2:45 PM, Carlo Bertoldi wrote: > Hi, a simple question: where can I find the HTML displayed by webtool? > For example, relatively to appmon, I looked into otp_src_R16B/lib/appmon > and lib/webtool/ but I could not find the appmon specific HTML. > Thanks, > Carlo > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -- Roberto Aloi --- Website: http://roberto-aloi.com Twitter: @robertoaloi -------------- next part -------------- An HTML attachment was scrubbed... URL: From prof3ta@REDACTED Sat Apr 20 22:42:29 2013 From: prof3ta@REDACTED (Roberto Aloi) Date: Sat, 20 Apr 2013 22:42:29 +0200 Subject: [erlang-questions] Webtool In-Reply-To: <51728DE4.8070400@ubiquity.it> References: <51728DE4.8070400@ubiquity.it> Message-ID: Also, you can find the appmon pages as part of the appmon application, split between appmon/priv/*.html files and src/appmon_web.erl. Hope this helps, R On Sat, Apr 20, 2013 at 2:45 PM, Carlo Bertoldi wrote: > Hi, a simple question: where can I find the HTML displayed by webtool? > For example, relatively to appmon, I looked into otp_src_R16B/lib/appmon > and lib/webtool/ but I could not find the appmon specific HTML. > Thanks, > Carlo > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -- Roberto Aloi --- Website: http://roberto-aloi.com Twitter: @robertoaloi -------------- next part -------------- An HTML attachment was scrubbed... URL: From t6502ml@REDACTED Sun Apr 21 05:22:09 2013 From: t6502ml@REDACTED (Tatsuya Kawano) Date: Sun, 21 Apr 2013 12:22:09 +0900 Subject: [erlang-questions] [ANN] Project FiFo webinar In-Reply-To: <7D480D7E-3166-49EF-8487-6395585CA82C@licenser.net> References: <7D480D7E-3166-49EF-8487-6395585CA82C@licenser.net> Message-ID: Hi Heinz, > I'm working on a (by now) quite big open source Erlang project for cloud management called Project FiFo and am going to hold an webinar regarding the architecture of the whole thing. Your project looks very interesting. Is there any way to make escripts to communicate FiFo's API server so that I can create, monitor, stop and delete SmartOS zones from these escripts? I'm developing an Erlang-based system and using SmartOS zones to test it. I usually create six zones by vmadm command and provision them with Chef, then deploy the system and run test suite, and finally delete these zones by vmadm. I'm currently doing vmadm part by hands but it will be great if I can escriptize them. Thanks, -- Tatsuya Kawano (Mr.) Tokyo, japan 2013/4/19 Heinz Nikolaus Gies > > Hi everyone, > I'm working on a (by now) quite big open source Erlang project for cloud management called Project FiFo and am going to hold an webinar regarding the architecture of the whole thing. I'll not deep dive into the erlang guts since it would blow up the timeframe but it still might be interesting and there is half a hour reserved for questions. > > The webinar is free so if this sounds interesting drop by! > > http://project-fifo.eventbrite.com/ > > Cheers, > Heinz > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From carlo.bertoldi@REDACTED Mon Apr 22 08:53:54 2013 From: carlo.bertoldi@REDACTED (Carlo Bertoldi) Date: Mon, 22 Apr 2013 08:53:54 +0200 Subject: [erlang-questions] Webtool In-Reply-To: References: <51728DE4.8070400@ubiquity.it> Message-ID: <5174DE82.4000305@ubiquity.it> Thanks to both of you. I've found what I was looking for. :) On 20/04/13 22:42, Roberto Aloi wrote: > Also, you can find the appmon pages as part of the appmon application, > split between appmon/priv/*.html files and src/appmon_web.erl. > > Hope this helps, > > R > > > > On Sat, Apr 20, 2013 at 2:45 PM, Carlo Bertoldi > > wrote: > > Hi, a simple question: where can I find the HTML displayed by webtool? > For example, relatively to appmon, I looked into otp_src_R16B/lib/appmon > and lib/webtool/ but I could not find the appmon specific HTML. > Thanks, > Carlo > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/__listinfo/erlang-questions > > > > > > -- > Roberto Aloi > --- > Website: http://roberto-aloi.com > Twitter: @robertoaloi From s.j.thompson@REDACTED Mon Apr 22 14:52:08 2013 From: s.j.thompson@REDACTED (Simon Thompson) Date: Mon, 22 Apr 2013 13:52:08 +0100 Subject: [erlang-questions] 17th BRAZILIAN SYMPOSIUM ON PROGRAMMING LANGUAGES Message-ID: <2DB5F43E-662D-4B8A-BA5F-AEBB8AC1B221@kent.ac.uk> [Apologies if you receive multiple copies of this CFP] ======================================================= CALL FOR PAPERS 17th BRAZILIAN SYMPOSIUM ON PROGRAMMING LANGUAGES Bras?lia, Distrito Federal, Brazil September 29th to October 4th, 2013 http://cbsoft2013.unb.br/en/sblp-en ======================================================== IMPORTANT DATES Paper abstract submission (15 lines): May 6th, 2013 (extended deadline) Full paper submission: May 10th, 2013 (extended deadline) Notification of acceptance: June 7, 2013 Final papers due: June 28th, 2013 INVITED SPEAKERS * Tim Harris, Oracle Labs * More TBA INTRODUCTION The 17th Brazilian Symposium on Programming Languages, SBLP 2013, will be held in Bras?lia, Brazil, on September 29th to October 4th, 2013. SBLP provides a venue for researchers and practitioners interested in the fundamental principles and innovations in the design and implementation of programming languages and systems. The symposium will be part of the 4th Brazilian Conference on Software: Theory and Practice, CBSoft 2013,http://cbsoft2013.cic.unb.br/, which will host four traditional, well-established symposia: * XXVII Brazilian Symposium on Software Engineering (SBES) * XVII Brazilian Symposium on Programming Languages (SBLP) * XVI Brazilian Symposium on Formal Methods (SBMF) * VII Brazilian Symposium on Components, Software Architecture and Software Reuse (SBCARS) SBLP 2013 invites authors to contribute with technical papers related (but not limited) to: * Program generation and transformation, including domain-specific languages and model-driven development in the context of programming languages. * Programming paradigms and styles, including functional, object-oriented, aspect-oriented, scripting languages, real-time, service-oriented, multithreaded, parallel, and distributed programming. * Formal semantics and theoretical foundations, including denotational, operational, algebraic and categorical. * Program analysis and verification, including type systems, static analysis and abstract interpretation. * Programming language design and implementation, including new programming models, programming language environments, compilation and interpretation techniques. SUBMISSIONS Contributions should be written in Portuguese or English. We solicit papers that should fall into one of two different categories: full papers, with at most 15 pages, or short papers, with at most 5 pages. Full papers submitted in English will be published in a volume of Lecture Notes in Computer Science (LNCS), by Springer. For this reason, all papers must be prepared using the LNCS template, available at http://www.springer.com/computer/lncs?SGWID=0-164-6-793341-0. We encourage the submission of short papers reporting on partial results of on-going master dissertations or doctoral theses. All accepted papers will be published in the conference proceedings distributed in a digital media by the CBSOFT organizers. Submissions should be done using SBLP 2013 installation of the EasyChair conference mangement system athttp://www.easychair.org/conferences/?conf=sblp2013. As in previous editions, a journal special issue, with selected papers from accepted contributions, is anticipated. Selected papers from 2003 to 2008 editions of SBLP were published in special issues of the Journal of Universal Computer Science, by Springer. The post-proceedings of SBLP from 2009 to 2012, also with selected papers from the conference proceedings, are being edited as special issues of Science of Computer Programming, published by Elsevier. CBSOFT CHAIRS Genaina Nunes Rodrigues, UnB Rodrigo Bonif?cio, UnB Diego Aranha, UnB PROGRAMME CHAIRS Andr? Rauber Du Bois, UFPel Phil Trinder, Glasgow University PROGRAMME COMMITTEE Alberto Pardo, Univ. de La Republica Alex Garcia, IME Alvaro Freitas Moreira, UFRGS Andr? Rauber Du Bois, UFPel (co-chair) Andre Santos, UFPE Carlos Camarao, UFMG Christiano Braga, UFF Edwin Brady, University of St. Andrews Fernando Castor Filho, UFPE Fernando Quint?o Pereira, UFMG Francisco Heron de Carvalho Junior, UFC Hans-Wofgang Loidl, Heriot-Watt University Jeremy Singer, Glasgow University Joao Saraiva, Universidade do Minho Jo?o F. Ferreira, Teesside University Lucilia Figueiredo, UFOP Luis Soares Barbosa, Univ. do Minho Manuel Ant?nio Martins, Univ. de Aveiro Marcelo A. Maia, UFU Marcello Bonsangue, Leiden Univ/CWI Marcelo d'Amorim, UFPE Marco Tulio Valente, UFMG Mariza A. S. Bigonha, UFMG Martin A. Musicante, UFRN Noemi Rodriguez, PUC-Rio Peter Mosses, Swansea University Phil Trinder, Glasgow University (co-chair) Qiu Zongyang, Beijing University Rafael Dueire Lins, UFPE Renato Cerqueira, PUC-Rio Ricardo Massa, UFPE Roberto S. Bigonha, UFMG Roberto Ierusalimschy, PUC-Rio Sandro Rigo, UNICAMP Sergio Soares, UFPE Simon Thompson, Univ. of Kent Varmo Vene, Univ. de Tartu Simon Thompson | Professor of Logic and Computation School of Computing | University of Kent | Canterbury, CT2 7NF, UK s.j.thompson@REDACTED | M +44 7986 085754 | W www.cs.kent.ac.uk/~sjt -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Mon Apr 22 16:00:17 2013 From: vances@REDACTED (Vance Shipley) Date: Mon, 22 Apr 2013 19:30:17 +0530 Subject: [erlang-questions] AMD Opteron 6300 (Piledriver) Profiling Message-ID: <20130422140017.GE21384@aluminium> I'm interested to know what performance profiling anyone may have done with Erlang/OTP on AMD Opteron 6300 series processors. Specifically a quad socket with 64 cores. I'd like to think that R16B would do an excellent job of keeping those 64 cores busy with the right work load. Has anyone any data to report? -- -Vance From jared.kofron@REDACTED Mon Apr 22 17:00:07 2013 From: jared.kofron@REDACTED (Jared Kofron) Date: Mon, 22 Apr 2013 08:00:07 -0700 Subject: [erlang-questions] AMD Opteron 6300 (Piledriver) Profiling In-Reply-To: <20130422140017.GE21384@aluminium> References: <20130422140017.GE21384@aluminium> Message-ID: Hi Vance, Alex Gounares gave a pretty cool talk at Erlang Factory this year about scaling Erlang out to many-core systems. His slides are here: http://www.erlang-factory.com/conference/SFBay2013/speakers/AlexanderGounares JK On Mon, Apr 22, 2013 at 7:00 AM, Vance Shipley wrote: > I'm interested to know what performance profiling anyone may > have done with Erlang/OTP on AMD Opteron 6300 series processors. > Specifically a quad socket with 64 cores. I'd like to think that > R16B would do an excellent job of keeping those 64 cores busy with > the right work load. Has anyone any data to report? > > -- > -Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Mon Apr 22 17:18:18 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Mon, 22 Apr 2013 16:18:18 +0100 Subject: [erlang-questions] ICE ICE Baby Message-ID: <9C3EAC53-F4B0-4A12-8E77-31B7FAD1EAC3@gmail.com> Hey guys, Given that Erlang has descended from telecoms roots; does anyone know of a solid STUN/TURN/ICE implementation that is battle hardened? Thanks, Lee From kaiduanx@REDACTED Mon Apr 22 17:49:41 2013 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Mon, 22 Apr 2013 11:49:41 -0400 Subject: [erlang-questions] ICE ICE Baby In-Reply-To: <9C3EAC53-F4B0-4A12-8E77-31B7FAD1EAC3@gmail.com> References: <9C3EAC53-F4B0-4A12-8E77-31B7FAD1EAC3@gmail.com> Message-ID: I have a STUN/TURN implementation in Erlang. /Kaiduan On Mon, Apr 22, 2013 at 11:18 AM, Lee Sylvester wrote: > Hey guys, > > Given that Erlang has descended from telecoms roots; does anyone know of a > solid STUN/TURN/ICE implementation that is battle hardened? > > Thanks, > Lee > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Mon Apr 22 18:06:43 2013 From: lukas@REDACTED (Lukas Larsson) Date: Mon, 22 Apr 2013 18:06:43 +0200 Subject: [erlang-questions] High lock contention on dist_tables In-Reply-To: References: Message-ID: The dist_table mutex refers to the rwmutex which is defined here[1]. There is a bunch of different places where it is used, so saying exactly what is causing the contentions is hard without knowing the code. Generally it should indicate that you are trying to send many messages over distribution while information about remote nodes is changing frequently. One thing I noticed is that the nodes() bif call takes a rwlock on the mutex. Are you using that bif alot? Lukas [1]: https://github.com/erlang/otp/blob/maint/erts/emulator/beam/erl_node_tables.c#L802 On Fri, Apr 19, 2013 at 9:24 PM, Brian Picciano wrote: > We have a pool of 3 erlang nodes, all on different servers. Every > afternoon, without fail, we start seeing lots of messages between the nodes > start having really high latency, on the order of tens of seconds. Today we > ran lcnt on them to see if there's anything there, and found that on one of > the nodes dist_tables had a significantly higher lock percentage then > anything else, and definitely higher then on the other boxes: > > (node@REDACTED)8> lcnt:conflicts(). > > lock id #tries #collisions collisions [%] time > [us] duration [%] > ----- --- ------- ------------ --------------- > ---------- ------------- > dist_table 1 3468191 1242055 35.8128 > 153712413 255.2521 > run_queue 24 76969638 4088578 5.3119 > 14468656 24.0264 > process_table 1 2015686 147148 7.3001 > 3208529 5.3280 > timer_wheel 1 12214948 834737 6.8337 > 3076638 5.1090 > timeofday 1 18231600 594487 3.2608 > 1491633 2.4770 > ... > > while on the other boxes it had closer to 3. On the box with the high lock > contention we also saw much higher load then on the other boxes. > > My question is: what is this lock? We couldn't find much online except > that it appears to have to do with communication between nodes, but we're > not sure what. Also, what, if anything, could we do to mitigate this > problem? > > (We're running erlang 16B) > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Mon Apr 22 19:36:56 2013 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Mon, 22 Apr 2013 18:36:56 +0100 Subject: [erlang-questions] ICE ICE Baby In-Reply-To: References: <9C3EAC53-F4B0-4A12-8E77-31B7FAD1EAC3@gmail.com> Message-ID: Hi Kaiduan, Excellent. Is this publicly hosted? Thanks, Lee On 22 Apr 2013, at 16:49, Kaiduan Xie wrote: > Kaiduan -------------- next part -------------- An HTML attachment was scrubbed... URL: From mediocregopher@REDACTED Mon Apr 22 22:23:48 2013 From: mediocregopher@REDACTED (Brian Picciano) Date: Mon, 22 Apr 2013 16:23:48 -0400 Subject: [erlang-questions] High lock contention on dist_tables In-Reply-To: References: Message-ID: We are actually. Is there an alternative way of easily retrieving which nodes are currently connected? On Mon, Apr 22, 2013 at 12:06 PM, Lukas Larsson wrote: > The dist_table mutex refers to the rwmutex which is defined here[1]. There > is a bunch of different places where it is used, so saying exactly what is > causing the contentions is hard without knowing the code. Generally it > should indicate that you are trying to send many messages over distribution > while information about remote nodes is changing frequently. > > One thing I noticed is that the nodes() bif call takes a rwlock on the > mutex. Are you using that bif alot? > > Lukas > > [1]: > https://github.com/erlang/otp/blob/maint/erts/emulator/beam/erl_node_tables.c#L802 > > > On Fri, Apr 19, 2013 at 9:24 PM, Brian Picciano wrote: > >> We have a pool of 3 erlang nodes, all on different servers. Every >> afternoon, without fail, we start seeing lots of messages between the nodes >> start having really high latency, on the order of tens of seconds. Today we >> ran lcnt on them to see if there's anything there, and found that on one of >> the nodes dist_tables had a significantly higher lock percentage then >> anything else, and definitely higher then on the other boxes: >> >> (node@REDACTED)8> lcnt:conflicts(). >> >> lock id #tries #collisions collisions [%] time >> [us] duration [%] >> ----- --- ------- ------------ --------------- >> ---------- ------------- >> dist_table 1 3468191 1242055 35.8128 >> 153712413 255.2521 >> run_queue 24 76969638 4088578 5.3119 >> 14468656 24.0264 >> process_table 1 2015686 147148 7.3001 >> 3208529 5.3280 >> timer_wheel 1 12214948 834737 6.8337 >> 3076638 5.1090 >> timeofday 1 18231600 594487 3.2608 >> 1491633 2.4770 >> ... >> >> while on the other boxes it had closer to 3. On the box with the high >> lock contention we also saw much higher load then on the other boxes. >> >> My question is: what is this lock? We couldn't find much online except >> that it appears to have to do with communication between nodes, but we're >> not sure what. Also, what, if anything, could we do to mitigate this >> problem? >> >> (We're running erlang 16B) >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lcastro@REDACTED Mon Apr 22 23:34:04 2013 From: lcastro@REDACTED (Laura M. Castro) Date: Mon, 22 Apr 2013 23:34:04 +0200 Subject: [erlang-questions] ACM SIGPLAN Erlang Workshop 2013 Call For Papers Message-ID: Hello, Please find below the First Call for Papers for the Twelfth ACM SIGPLAN Erlang Workshop. Apologies for any duplicates you may receive. CALL FOR PAPERS ================= Twelfth ACM SIGPLAN Erlang Workshop ----------------------------------------------------------- Boston, Massachusetts, September 28, 2013 (tentative date, subject to change) Satellite event of the 18th ACM SIGPLAN International Conference on Functional Programming (ICFP 2013) September 25-27, 2013 Erlang is a concurrent, distributed functional programming language aimed at systems with requirements of massive concurrency, soft real time response, fault tolerance, and high availability. It has been available as open source for 15 years, creating a community that actively contributes to its already existing rich set of libraries and applications. Originally created for telecom applications, its usage has spread to other domains including e-commerce, banking, databases, and computer telephony and messaging. Erlang programs are today among the largest applications written in any functional programming language. These applications offer new opportunities to evaluate functional programming and functional programming methods on a very large scale and suggest new problems for the research community to solve. This workshop will bring together the open source, academic, and industrial programming communities of Erlang. It will enable participants to familiarize themselves with recent developments on new techniques and tools tailored to Erlang, novel applications, draw lessons from users' experiences and identify research problems and common areas relevant to the practice of Erlang and functional programming. We invite three types of submissions. 1. Technical papers describing language extensions, critical discussions of the status quo, formal semantics of language constructs, program analysis and transformation, virtual machine extensions and compilation techniques, implementations and interfaces of Erlang in/with other languages, and new tools (profilers, tracers, debuggers, testing frameworks, etc.). The maximum length for technical papers is restricted to 12 pages. 2. Practice and application papers describing uses of Erlang in the "real-world", Erlang libraries for specific tasks, experiences from using Erlang in specific application domains, reusable programming idioms and elegant new ways of using Erlang to approach or solve a particular problem. The maximum length for the practice and application papers is restricted to 12 pages. Note that this is a maximum length; we welcome shorter papers also, and the program committee will evaluate all papers on an equal basis independent of their lengths. 3. Poster presentations describing topics related to the workshop goals. Each includes a maximum of 2 pages of the abstract and summary. Presentations in this category will be given an hour of shared simultaneous demonstration time. Workshop Chair ----------------------- Steve Vinoski, Basho Technologies, USA Program Chair ------------------- -- Laura M. Castro Souto Departamento de Computaci?n Universidade da Coru?a http://www.madsgroup.org/staff/laura M. Castro, University of A Coru?a, Spain Program Committee ----------------------------- (Note: the Workshop and Program Chairs are also committee members) Lars-Ake Fredlund, Universidad Polit?cnica de Madrid, Spain Kevin Hammond, University of St. Andrews, UK Torben Hoffman, Erlang Solutions Limited, UK Zolt?n Horv?th, E?tv?s Lor?nd University, Hungary Kenneth Lundin, Ericsson AB, Sweden Micka?l R?mond, ProcessOne, France Kenji Rikitake, Basho Japan KK, Japan Simon Thompson, University of Kent, UK Important Dates ----------------------- Submission deadline: Fri June 14, 2013 Author notification: Thu July 11, 2013 Final submission for the publisher: Thu July 25, 2013 Workshop date (tentative, subject to change): September 28, 2013 Instructions to authors -------------------------------- Papers must be submitted online via EasyChair (via the "Erlang2013" event). The submission page is https://www.easychair.org/conferences/?conf=erlang2013 Submitted papers should be in portable document format (PDF), formatted using the ACM SIGPLAN style guidelines. Each submission must adhere to SIGPLAN's republication policy. Violation risks summary rejection of the offending submission. Accepted papers will be published by the ACM and will appear in the ACM Digital Library. Paper submissions will be considered for poster submission in the case they are not accepted as full papers. Venue & Registration Details ------------------------------------------ For registration, please see the ICFP 2013 web site at: http://icfpconference.org/icfp2013/ Related Links -------------------- ICFP 2013 web site: http://www.icfpconference.org/icfp2013/ Past ACM SIGPLAN Erlang workshops: http://www.erlang.org/workshop/ Open Source Erlang: http://www.erlang.org/ EasyChair submission site: https://www.easychair.org/account/signin.cgi?conf=erlang2013 Author Information for SIGPLAN Conferences: http://www.sigplan.org/authorInformation.htm From bryan@REDACTED Tue Apr 23 00:32:25 2013 From: bryan@REDACTED (Bryan Hughes) Date: Mon, 22 Apr 2013 15:32:25 -0700 Subject: [erlang-questions] process_info/2 returning badarg with message_que_len Message-ID: <5175BA79.9010608@go-factory.net> Howdy, I am having some weirdness. Running on the latest build of R16B (erts-5.10.1). Per the documentation (http://erlang.org/doc/man/erlang.html#process_info-2), when I call it: Info = erlang:process_info(self(), [memory, heap_size, message_que_len]), I get the following error: * Reason for termination == ** {badarg,[{erlang,process_info, [<0.127.0>,[memory,heap_size,message_que_len]], []}, When I call it without message_que_len, or instead use messages: Info = erlang:process_info(self(), [memory, heap_size, messages]), Life is good. According to the docs, message_que_len is a valid process_info_item(). Is the documentation wrong, or am I doing something wrong? Note that this also happens in the shell. Thanks again! Cheers, Bryan -- Bryan Hughes *Go Factory* http://www.go-factory.net /"Art is never finished, only abandoned. - Leonardo da Vinci"/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Tue Apr 23 00:38:16 2013 From: bob@REDACTED (Bob Ippolito) Date: Mon, 22 Apr 2013 15:38:16 -0700 Subject: [erlang-questions] process_info/2 returning badarg with message_que_len In-Reply-To: <5175BA79.9010608@go-factory.net> References: <5175BA79.9010608@go-factory.net> Message-ID: Is that a typo in the docs? Should be message_queue_len. On Monday, April 22, 2013, Bryan Hughes wrote: > Howdy, > > I am having some weirdness. Running on the latest build of R16B > (erts-5.10.1). Per the documentation ( > http://erlang.org/doc/man/erlang.html#process_info-2), > when I call it: > > Info = erlang:process_info(self(), [memory, heap_size, message_que_len]), > > I get the following error: > > * Reason for termination == > ** {badarg,[{erlang,process_info, > [<0.127.0>,[memory,heap_size,message_que_len]], > []}, > > When I call it without message_que_len, or instead use messages: > > Info = erlang:process_info(self(), [memory, heap_size, messages]), > > Life is good. According to the docs, message_que_len is a valid > process_info_item(). Is the documentation wrong, or am I doing something > wrong? > > Note that this also happens in the shell. > > Thanks again! > > Cheers, > Bryan > > -- > > Bryan Hughes > *Go Factory* > http://www.go-factory.net > > *"Art is never finished, only abandoned. - Leonardo da Vinci"* > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bryan@REDACTED Tue Apr 23 00:40:56 2013 From: bryan@REDACTED (Bryan Hughes) Date: Mon, 22 Apr 2013 15:40:56 -0700 Subject: [erlang-questions] process_info/2 returning badarg with message_que_len In-Reply-To: References: <5175BA79.9010608@go-factory.net> Message-ID: <5175BC78.5060908@go-factory.net> Thanks Bob! It is a typo in the docs. http://erlang.org/doc/man/erlang.html#process_info-2 process_info_item() = backtrace | binary | catchlevel | current_function | current_location | current_stacktrace | dictionary | error_handler | garbage_collection | group_leader | heap_size | initial_call | links | last_calls | memory | message_que_len <======== TYPO | messages | min_heap_size | min_bin_vheap_size | monitored_by | monitors | priority | reductions | registered_name | sequential_trace_token | stack_size | status | suspending | total_heap_size | trace | trap_exit Cheers, B On 4/22/13 3:38 PM, Bob Ippolito wrote: > Is that a typo in the docs? Should be message_queue_len. > > On Monday, April 22, 2013, Bryan Hughes wrote: > > Howdy, > > I am having some weirdness. Running on the latest build of R16B > (erts-5.10.1). Per the documentation > (http://erlang.org/doc/man/erlang.html#process_info-2), > when I call it: > > Info = erlang:process_info(self(), [memory, heap_size, > message_que_len]), > > I get the following error: > > * Reason for termination == > ** {badarg,[{erlang,process_info, > [<0.127.0>,[memory,heap_size,message_que_len]], > []}, > > When I call it without message_que_len, or instead use messages: > > Info = erlang:process_info(self(), [memory, heap_size, messages]), > > Life is good. According to the docs, message_que_len is a valid > process_info_item(). Is the documentation wrong, or am I doing > something wrong? > > Note that this also happens in the shell. > > Thanks again! > > Cheers, > Bryan > > -- > > Bryan Hughes > *Go Factory* > http://www.go-factory.net > > /"Art is never finished, only abandoned. - Leonardo da Vinci"/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From t6502ml@REDACTED Tue Apr 23 01:46:43 2013 From: t6502ml@REDACTED (Tatsuya Kawano) Date: Tue, 23 Apr 2013 08:46:43 +0900 Subject: [erlang-questions] [ANN] Project FiFo webinar In-Reply-To: <2E7E06FE-D636-4F63-8D4A-F49419AE394B@licenser.net> References: <7D480D7E-3166-49EF-8487-6395585CA82C@licenser.net> <2E7E06FE-D636-4F63-8D4A-F49419AE394B@licenser.net> Message-ID: Hi Heinz, Thank you. I just installed FiFo 0.3.4 and pyfi and played with them. They look good. > that is totally possible depending on how exactly your setup is there are > multiple ways which all hold different advantages and disadvantages: I'll probably choose 2) libsniffle for now, but try to move 3) HTTP/REST API in the future. Thanks, Tatsuya 2013/4/21 Heinz Nikolaus Gies : > Hi! > > that is totally possible depending on how exactly your setup is there are > multiple ways which all hold different advantages and disadvantages: > > 1) Use rpc / node tool from within the fifo zone just as sniffle-admin / > fifoadm does > * Pros: no login, full access to all functions, no layer between your script > and fifo > * Cons: no layer between your script and fifo, needs to talk directly to > fifo, no security, not well documented > 2) Use libsniffle[1] to call the create etc functions > * Pros: standard approach for internal applications to talk to sniffle, well > tested, documented[2], no login required > * Cons: requires mDNS discovery period, no security > 3) User the HTTP/REST API > * Pros: clean separation of your code and fifo, authentication, very well > tested and documented[3], most canonical way to talk to FiFo from an outside > app. > * Cons: No erlang implementation of a API client (you'd have to roll your > own), authentication required (not sure that is a cons) > 4) User pyfi[4][5](python HTTP API client library & console client) > * Pros: implements most of the API calls, working condition, clean > separation of client and server code > * Cons: Not erlang, no implementation of the zone console yet > > Cheers, > Heinz > > [1] https://github.com/project-fifo/libsniffle > [2] http://project-fifo.github.io/libsniffle > [3] http://project-fifo.net/display/PF/API > [4] https://github.com/project-fifo/pyfo > [5] https://pypi.python.org/pypi/PyFi > > On Apr 21, 2013, at 5:22, Tatsuya Kawano wrote: > > Hi Heinz, > > I'm working on a (by now) quite big open source Erlang project for cloud > management called Project FiFo and am going to hold an webinar regarding the > architecture of the whole thing. > > > Your project looks very interesting. > > Is there any way to make escripts to communicate FiFo's API server so > that I can create, monitor, stop and delete SmartOS zones from these > escripts? I'm developing an Erlang-based system and using SmartOS > zones to test it. I usually create six zones by vmadm command and > provision them with Chef, then deploy the system and run test suite, > and finally delete these zones by vmadm. I'm currently doing vmadm > part by hands but it will be great if I can escriptize them. > > Thanks, > > -- > Tatsuya Kawano (Mr.) > Tokyo, japan > > > 2013/4/19 Heinz Nikolaus Gies > > > Hi everyone, > I'm working on a (by now) quite big open source Erlang project for cloud > management called Project FiFo and am going to hold an webinar regarding the > architecture of the whole thing. I'll not deep dive into the erlang guts > since it would blow up the timeframe but it still might be interesting and > there is half a hour reserved for questions. > > The webinar is free so if this sounds interesting drop by! > > http://project-fifo.eventbrite.com/ > > Cheers, > Heinz From heinz@REDACTED Tue Apr 23 02:10:31 2013 From: heinz@REDACTED (Heinz Nikolaus Gies) Date: Tue, 23 Apr 2013 02:10:31 +0200 Subject: [erlang-questions] [ANN] Project FiFo webinar In-Reply-To: References: <7D480D7E-3166-49EF-8487-6395585CA82C@licenser.net> <2E7E06FE-D636-4F63-8D4A-F49419AE394B@licenser.net> Message-ID: Awesome! If you run into any trouble don't hesitate to poke me :) also feel free to drop by in our IRC channel #Project-FiFo (on free node) Cheers, Heinz On Apr 23, 2013, at 1:46, Tatsuya Kawano wrote: > Hi Heinz, > > Thank you. I just installed FiFo 0.3.4 and pyfi and played with them. > They look good. > >> that is totally possible depending on how exactly your setup is there are >> multiple ways which all hold different advantages and disadvantages: > > I'll probably choose 2) libsniffle for now, but try to move 3) > HTTP/REST API in the future. > > Thanks, > Tatsuya > > > 2013/4/21 Heinz Nikolaus Gies : >> Hi! >> >> that is totally possible depending on how exactly your setup is there are >> multiple ways which all hold different advantages and disadvantages: >> >> 1) Use rpc / node tool from within the fifo zone just as sniffle-admin / >> fifoadm does >> * Pros: no login, full access to all functions, no layer between your script >> and fifo >> * Cons: no layer between your script and fifo, needs to talk directly to >> fifo, no security, not well documented >> 2) Use libsniffle[1] to call the create etc functions >> * Pros: standard approach for internal applications to talk to sniffle, well >> tested, documented[2], no login required >> * Cons: requires mDNS discovery period, no security >> 3) User the HTTP/REST API >> * Pros: clean separation of your code and fifo, authentication, very well >> tested and documented[3], most canonical way to talk to FiFo from an outside >> app. >> * Cons: No erlang implementation of a API client (you'd have to roll your >> own), authentication required (not sure that is a cons) >> 4) User pyfi[4][5](python HTTP API client library & console client) >> * Pros: implements most of the API calls, working condition, clean >> separation of client and server code >> * Cons: Not erlang, no implementation of the zone console yet >> >> Cheers, >> Heinz >> >> [1] https://github.com/project-fifo/libsniffle >> [2] http://project-fifo.github.io/libsniffle >> [3] http://project-fifo.net/display/PF/API >> [4] https://github.com/project-fifo/pyfo >> [5] https://pypi.python.org/pypi/PyFi >> >> On Apr 21, 2013, at 5:22, Tatsuya Kawano wrote: >> >> Hi Heinz, >> >> I'm working on a (by now) quite big open source Erlang project for cloud >> management called Project FiFo and am going to hold an webinar regarding the >> architecture of the whole thing. >> >> >> Your project looks very interesting. >> >> Is there any way to make escripts to communicate FiFo's API server so >> that I can create, monitor, stop and delete SmartOS zones from these >> escripts? I'm developing an Erlang-based system and using SmartOS >> zones to test it. I usually create six zones by vmadm command and >> provision them with Chef, then deploy the system and run test suite, >> and finally delete these zones by vmadm. I'm currently doing vmadm >> part by hands but it will be great if I can escriptize them. >> >> Thanks, >> >> -- >> Tatsuya Kawano (Mr.) >> Tokyo, japan >> >> >> 2013/4/19 Heinz Nikolaus Gies >> >> >> Hi everyone, >> I'm working on a (by now) quite big open source Erlang project for cloud >> management called Project FiFo and am going to hold an webinar regarding the >> architecture of the whole thing. I'll not deep dive into the erlang guts >> since it would blow up the timeframe but it still might be interesting and >> there is half a hour reserved for questions. >> >> The webinar is free so if this sounds interesting drop by! >> >> http://project-fifo.eventbrite.com/ >> >> Cheers, >> Heinz From ok@REDACTED Tue Apr 23 02:12:38 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 23 Apr 2013 12:12:38 +1200 Subject: [erlang-questions] Why I love Erlang (part 57) Message-ID: "Part 57" is a joke, but it feels serious. I'm teaching some students who have to use Objective C for a project, although I'm not actually teaching them Objective C. Objective C has a @try/@catch/@finally block which is practically identical to Java's. Except for one thing, well, two things that are one. https://developer.apple.com/library/mac/#documentation/Cocoa/Conceptual/Exceptions/Articles/ExceptionsAndCocoaFrameworks.html#//apple_ref/doc/uid/TP40009045-SW1 (1) The Cocoa frameworks require that all exceptions be instances of NSException or its subclasses. Do not throw objects of other types. So the language and compiler will allow you to @throw other things, it's just that that won't work if you're using any of the things that would be reasons for using Objective C in the first place. (2) The Cocoa frameworks are generally not exception-safe. The general pattern is that exceptions are reserved for programmer error only, and the program catching such an exception should quit soon afterwards. So if the user interface calls back into your code, and your code @throws something, when you catch it, you had better regard the user interface as totally off limits -- in an undefined state -- from then on. I just read that last night, and I am still shocked. Now I find myself wondering if Java really does so much better, or if Objective C is just more honestly documented. Erlang never looked so good! From raould@REDACTED Tue Apr 23 02:38:12 2013 From: raould@REDACTED (Raoul Duke) Date: Mon, 22 Apr 2013 17:38:12 -0700 Subject: [erlang-questions] Why I love Erlang (part 57) In-Reply-To: References: Message-ID: as far as i know, how you interpret it is correct. while it is somewhat shocking from some perspectives, there are other perspectives where you already have the habit of not using exceptions beause nobody /really/ knows how to use exceptions, and instead you use return codes, old skuul style, because really those are just fine; it is your own damn fault if you don't check them :-) (now some systems are trying to move *ahead* of those ideas, rather than falling back ;-) From lukas@REDACTED Tue Apr 23 09:30:32 2013 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 23 Apr 2013 09:30:32 +0200 Subject: [erlang-questions] High lock contention on dist_tables In-Reply-To: References: Message-ID: I don't know of an alternative way. Before trying to come up with a solution I would verify that it is nodes() which is causing the contention. If you do lcnt:inspect(dist_table,[{locations,true}]) you should get a list of which actual locks in the source code it is that the contentions are happening. On Mon, Apr 22, 2013 at 10:23 PM, Brian Picciano wrote: > We are actually. Is there an alternative way of easily retrieving which > nodes are currently connected? > > > On Mon, Apr 22, 2013 at 12:06 PM, Lukas Larsson < > lukas@REDACTED> wrote: > >> The dist_table mutex refers to the rwmutex which is defined here[1]. >> There is a bunch of different places where it is used, so saying exactly >> what is causing the contentions is hard without knowing the code. Generally >> it should indicate that you are trying to send many messages over >> distribution while information about remote nodes is changing frequently. >> >> One thing I noticed is that the nodes() bif call takes a rwlock on the >> mutex. Are you using that bif alot? >> >> Lukas >> >> [1]: >> https://github.com/erlang/otp/blob/maint/erts/emulator/beam/erl_node_tables.c#L802 >> >> >> On Fri, Apr 19, 2013 at 9:24 PM, Brian Picciano > > wrote: >> >>> We have a pool of 3 erlang nodes, all on different servers. Every >>> afternoon, without fail, we start seeing lots of messages between the nodes >>> start having really high latency, on the order of tens of seconds. Today we >>> ran lcnt on them to see if there's anything there, and found that on one of >>> the nodes dist_tables had a significantly higher lock percentage then >>> anything else, and definitely higher then on the other boxes: >>> >>> (node@REDACTED)8> lcnt:conflicts(). >>> >>> lock id #tries #collisions collisions [%] time >>> [us] duration [%] >>> ----- --- ------- ------------ --------------- >>> ---------- ------------- >>> dist_table 1 3468191 1242055 35.8128 >>> 153712413 255.2521 >>> run_queue 24 76969638 4088578 5.3119 >>> 14468656 24.0264 >>> process_table 1 2015686 147148 7.3001 >>> 3208529 5.3280 >>> timer_wheel 1 12214948 834737 6.8337 >>> 3076638 5.1090 >>> timeofday 1 18231600 594487 3.2608 >>> 1491633 2.4770 >>> ... >>> >>> while on the other boxes it had closer to 3. On the box with the high >>> lock contention we also saw much higher load then on the other boxes. >>> >>> My question is: what is this lock? We couldn't find much online except >>> that it appears to have to do with communication between nodes, but we're >>> not sure what. Also, what, if anything, could we do to mitigate this >>> problem? >>> >>> (We're running erlang 16B) >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dronnikov@REDACTED Tue Apr 23 09:49:24 2013 From: dronnikov@REDACTED (Vladimir Dronnikov) Date: Tue, 23 Apr 2013 11:49:24 +0400 Subject: [erlang-questions] Secure Tokens In-Reply-To: References: <57A08917-A253-44AF-9865-440651D8C131@gmail.com> <639C85DA-F075-455E-ACD0-930EE2F9A7BD@gmail.com> Message-ID: https://github.com/dvv/termit is now fixed, thanks to invaluable Bob's assistance. On Tue, Apr 2, 2013 at 9:50 AM, Bob Ippolito wrote: > No reason to celebrate just yet, termit has a broken cryptosystem. Here's > two things I noticed after a quick glance: > > * The IV is derived from the secret key. The IV must be unpredictable at > encryption time in CBC mode. This is VERY VERY bad. > * Verification of the signature isn't constant-time, so it's susceptible > to timing attacks. This is still bad, but probably harder to exploit. > > It would be unwise to use this implementation. I don't claim that the > mochiweb code is perfect, and I'm not a cryptograph expert, but I have > audited it and I didn't find any obvious flaws (other than the timing > attack that I fixed). > > > On Mon, Apr 1, 2013 at 10:37 PM, Lee Sylvester wrote: > >> Wow, a fernet like impl for Erlang!!! Perfect!!! Thank you very much. >> This will make my life so much easier :-) >> >> Regards, >> Lee >> >> >> >> On 2 Apr 2013, at 06:09, Vladimir Dronnikov wrote: >> >> I drive https://github.com/dvv/termit for this. Feel free to >> feedback/blame :) >> >> >> On Tue, Apr 2, 2013 at 12:15 AM, Bob Ippolito wrote: >> >>> There's something similar to your requirements in here: >>> https://github.com/mochi/mochiweb/blob/master/src/mochiweb_session.erl >>> >>> >>> On Mon, Apr 1, 2013 at 1:05 PM, Lee Sylvester wrote: >>> >>>> Hey guys, >>>> >>>> So, I'd like to create secure tokens in Erlang. This can either be a >>>> simple UUID generator which I then store with user credentials or a way to >>>> encode a string, such as JSON, as an encrypted token. In Golang, I would >>>> do this with fernet, but I need an Erlang solution :-) >>>> >>>> I know Erlang isn't best used for such tasks, but does anyone out there >>>> know of something usable for this purpose? >>>> >>>> Thanks loads, >>>> Lee >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Tue Apr 23 11:38:59 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Tue, 23 Apr 2013 11:38:59 +0200 Subject: [erlang-questions] process_info/2 returning badarg with message_que_len In-Reply-To: <5175BC78.5060908@go-factory.net> References: <5175BA79.9010608@go-factory.net> <5175BC78.5060908@go-factory.net> Message-ID: <6DB6261C-7779-4721-8205-8C5D7F5C0C8E@gmail.com> Hello Bryan, This will be fixed in R16B01, a patch has been submitted by Andrew Tunnell-Jones. Regards, -- Anthony Ramine Le 23 avr. 2013 ? 00:40, Bryan Hughes a ?crit : > Thanks Bob! > > It is a typo in the docs. > > http://erlang.org/doc/man/erlang.html#process_info-2 > > process_info_item() = backtrace > | binary > | catchlevel > | current_function > | current_location > | current_stacktrace > | dictionary > | error_handler > | garbage_collection > | group_leader > | heap_size > | initial_call > | links > | last_calls > | memory > | message_que_len <======== TYPO > | messages > | min_heap_size > | min_bin_vheap_size > | monitored_by > | monitors > | priority > | reductions > | registered_name > | sequential_trace_token > | stack_size > | status > | suspending > | total_heap_size > | trace > | trap_exit > > > Cheers, > B > > On 4/22/13 3:38 PM, Bob Ippolito wrote: >> Is that a typo in the docs? Should be message_queue_len. >> >> On Monday, April 22, 2013, Bryan Hughes wrote: >> Howdy, >> >> I am having some weirdness. Running on the latest build of R16B (erts-5.10.1). Per the documentation (http://erlang.org/doc/man/erlang.html#process_info-2), >> when I call it: >> >> Info = erlang:process_info(self(), [memory, heap_size, message_que_len]), >> >> I get the following error: >> >> * Reason for termination == >> ** {badarg,[{erlang,process_info, >> [<0.127.0>,[memory,heap_size,message_que_len]], >> []}, >> >> When I call it without message_que_len, or instead use messages: >> >> Info = erlang:process_info(self(), [memory, heap_size, messages]), >> >> Life is good. According to the docs, message_que_len is a valid process_info_item(). Is the documentation wrong, or am I doing something wrong? >> >> Note that this also happens in the shell. >> >> Thanks again! >> >> Cheers, >> Bryan >> >> -- >> Bryan Hughes >> Go Factory >> http://www.go-factory.net >> >> "Art is never finished, only abandoned. - Leonardo da Vinci" >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From paul@REDACTED Tue Apr 23 14:34:24 2013 From: paul@REDACTED (Paul Rubin) Date: Tue, 23 Apr 2013 05:34:24 -0700 Subject: [erlang-questions] AMD Opteron 6300 (Piledriver) Profiling Message-ID: There was a good presentation at Erlang Factory SF by the guy who runs concurix.com about running on very high numbers of cores. Basically the current BEAM stops scaling well after some medium number, like 10 or 20? The company made some patches that help smooth this out, some of which may make it into future Erlang releases. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fritchie@REDACTED Tue Apr 23 21:01:27 2013 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Tue, 23 Apr 2013 14:01:27 -0500 Subject: [erlang-questions] High lock contention on dist_tables In-Reply-To: Message of "Fri, 19 Apr 2013 15:24:12 EDT." Message-ID: <43187.1366743687@snookles.snookles.com> Brian Picciano wrote: bp> We have a pool of 3 erlang nodes, all on different servers. Every bp> afternoon, without fail, we start seeing lots of messages between bp> the nodes start having really high latency, on the order of tens of bp> seconds. [...] Brian, it's probably worthwhile to continue chasing the 'lcnt' avenue as you've been corresponding with Lukas... ... but at the same time, I also wonder about "tens of seconds". My gut says that such delays would require some amazingly high lock contention rates. Something that can cause such messaging delays much more easily is network congestion/packet loss that triggers TCP slow start. Many Linux kernels have the RTO_min value at one second, which is the amount of time to wait before entering slow start state. If network packet loss is a problem, this blog posting can explain one reason why it's happening: http://www.snookles.com/slf-blog/2012/01/05/tcp-incast-what-is-it/ -Scott From fritchie@REDACTED Tue Apr 23 21:36:54 2013 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Tue, 23 Apr 2013 14:36:54 -0500 Subject: [erlang-questions] "New" vs. "old" console behavior: bug or feature? Message-ID: <45099.1366745814@snookles.snookles.com> Hi, all. I can't figure out if this message should be sent to the erlang-bugs list or the erlang-questions list ... so I'll go for the more general audience. Summary: Starting Erlang with a tty/pseudo-tty can get you a different console shell ("new" and "old", respectively) without you realizing it.(*) If you don't know that you're using the old shell, and if a process tries to send output to the 'user' registered process(**), e.g. io:format(user, "Some message with ~p extra\n", [Extra]), then it is possible that the io:format() call will not return for seconds/minutes/hours/ever. My question: Is the kind of indefinite blocking on I/O described below a bug or a feature? I have a test case that can reproduce this behavior. An automated version (using Expect) can be found at: https://gist.github.com/slfritchie/ad8e5cf1603cbe326be7 The basics of the reproducing the hang are: SSH session #1 SSH session #2 -------------- -------------- Start an Erlang daemon using "run_erl". Attach to the daemon's console using "to_erl". Start another Erlang VM and connect to the first VM via "-remsh". At the console, type the following and press ENTER: {term1, Run this command: io:format(user, "Hey!\n", []). The io:format/3 call in session #2 will behave differently if session #1's "run_erl" command runs with a tty/pseudo-tty or without. A. With a tty/pty: The io:format() call returns immediately. B. Without a tty/pty: The io:format() call will hang indefinitely. It will remain blocked until the Erlang term parser in session #1 has returned. For example, finishing the term with "term2}." and then pressing ENTER. The same effect can be seen by forcing the use of the old shell, without using SSH, by simply running "erl -oldshell" for session #1 (in an Xterm or other terminal window, or at the machine's hardware console) instead of using SSH + "run_erl" + "to_erl". Riak was the application that triggered this bug hunt (in conjunction with the Lager app)(***). Finding it has taken much longer than anyone guessed. The reason is that the necessary precondition, starting Erlang via 'run_erl' via SSH without an associated tty/pseudo-tty, is not common. (Riak's packaging uses "sudo", which refuses to run if there isn't a tty/pty available.) All attempts to duplicate the behavior failed because we didn't understand that the root cause of the bad behavior was the old console being silently chosen at VM startup when not tty/pty is available. -Scott (*) See https://github.com/erlang/otp/blob/maint/lib/kernel/src/user_drv.erl#L103 for how the choice is made. (**) From the 'io' man page: There is always a process registered under the name of user. This can be used for sending output to the user. ... where "output to the user" really means "output to the Erlang virtual machine console." (***) For source code of Riak and Lager, respectively, see: https://github.com/basho/riak https://github.com/basho/lager From john_re@REDACTED Wed Apr 24 07:50:25 2013 From: john_re@REDACTED (giovanni_re) Date: Tue, 23 Apr 2013 22:50:25 -0700 Subject: [erlang-questions] Go adding more Erlang style features - Go Circuit. yc Message-ID: <1366782625.10964.140661221981410.33FBA97C@webmail.messagingengine.com> Go Circuit ? Distributed Go Programming Framework (gocircuit.org) 105 points by Rooki 19 hours ago | 23 comments https://news.ycombinator.com/item?id=5594279 - davidw 16 hours ago | link Sounds like Go adding more Erlang style features. Off the top of my head, some nice things that Erlang lacks in that department might include: * A bit more security by default. Erlang was built for something where they presumed they were walled off from bad stuff. * Some kind of routing mechanism, so that you could have A connected to B, and B connected to C, and the system would be able to send messages from A to C. I have no idea if this is actually a good idea, as this is not the sort of thing I use Erlang for myself. But it seems like it might be cool. reply dualogy 13 hours ago | link > Sounds like Go adding more Erlang style features. Nitpick: this is simply an independent 3rd -party package / external lib for Go developers. Not a language or runtime feature or addition. reply elviejo 3 hours ago | link So maybe like erlang's OTP ? === Paradigm for developing and sustaining Big Data apps http://www.gocircuit.org/ The circuit reduces the human development and sustenance costs of complex massively-scaled systems nearly to the level of their single-process counterparts. It is a combination of proven ideas from the Erlang ecosystem of distributed embedded devices and Go's ecosystem of Internet application development. The circuit extends the reach of Go's linguistic environment to multi-host/multi-process applications. In simple terms, the Go Circuit was born from the desire to be able to write: feedback := make(chan int) circuit.Spawn("host25.datacenter.net", func() { feedback <- 1 }) <-feedback println("Roundtrip complete.") TheSpawn operator will start its argument function on a desired remote host in a new goroutine, while making it possible to communicate between the parent and child goroutines using the same Go code that you would use to communicate between traditional goroutines. Above, the channelfeedback is transparently ?stretched? between the parent goroutine, executing locally, and the child goroutine, executing remotely and hosting the anonymous function execution. Using the circuit one is able to write complex distributed applications ? involving multiple types of collaborating processes ? within a single circuit program. The circuit language used therein is syntactically identical to Go while also: Providing facilities for spawning goroutines on remote hardware, and Treating local and remote goroutines in the same manner, both syntactically and semantically. As a result, distributed application code becomes orders of magnitude shorter, as compared to using traditional alternatives. For isntance, we have been able to write large real-world cloud applications ? e.g. streaming multi-stage MapReduce pipelines ? in as many as 200 lines of code from the ground up. For lifecycle maintenance, the circuit provides a powerful toolkit that can introspect into, control and modify various dynamic aspects of a live circuit application. Robust networking protocols allow for complex runtime maneuvers like, for instance, surgically replacing components of running cloud applications with binaries from different versions of the source tree, without causing service interruption. --- Join the BerkeleyTIP-Global mail list - http://groups.google.com/group/BerkTIPGlobal. All Freedom SW, HW & Culture. From dujinfang@REDACTED Wed Apr 24 12:49:45 2013 From: dujinfang@REDACTED (Seven Du) Date: Wed, 24 Apr 2013 18:49:45 +0800 Subject: [erlang-questions] otp supervisor terminate callback In-Reply-To: References: Message-ID: <0878237C6DC24E81B4CE82C476E18B42@gmail.com> Thanks Vance. Maybe I didn't made clear, I want a terminate callback in the -behaviour(supervisor). -- Seven Du http://www.freeswitch.org.cn http://about.me/dujinfang http://www.dujinfang.com Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Saturday, April 20, 2013 at 5:34 PM, Vance Shipley wrote: > To get the terminate/1 callback in a gen_server or gen_fsm behaviour module what you would do is to set the process flag to trap exits in your init/1 callback. > On Apr 20, 2013 4:34 PM, "Seven Du" wrote: > > Hi, > > > > I'd like to know is it possible to have a terminate callback that clean up things when the supervisor stops? > > > > Specifically I want to write an application which starts a supervisor, and add a handler to some existing gen_event by calling gen_event:add_handler() on init(), and I want to call gen_event:remove_handler() when the application is stop. Is there an easy way to do it? > > > > Thanks > > > > -- > > Seven Du > > http://www.freeswitch.org.cn > > http://about.me/dujinfang > > http://www.dujinfang.com > > > > > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Wed Apr 24 14:08:00 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 24 Apr 2013 13:08:00 +0100 Subject: [erlang-questions] otp supervisor terminate callback In-Reply-To: <0878237C6DC24E81B4CE82C476E18B42@gmail.com> References: <0878237C6DC24E81B4CE82C476E18B42@gmail.com> Message-ID: Steven, That isn't supported by the supervisor API. You'll need to create a child process beneath the supervisor (using gen_server for example) and handle the init/1 and terminate/2 behaviour there. On 24 April 2013 11:49, Seven Du wrote: > Thanks Vance. Maybe I didn't made clear, I want a terminate callback in > the -behaviour(supervisor). > > -- > Seven Du > http://www.freeswitch.org.cn > http://about.me/dujinfang > http://www.dujinfang.com > > Sent with Sparrow > > On Saturday, April 20, 2013 at 5:34 PM, Vance Shipley wrote: > > To get the terminate/1 callback in a gen_server or gen_fsm behaviour > module what you would do is to set the process flag to trap exits in your > init/1 callback. > On Apr 20, 2013 4:34 PM, "Seven Du" wrote: > > Hi, > > I'd like to know is it possible to have a terminate callback that clean up > things when the supervisor stops? > > Specifically I want to write an application which starts a supervisor, and > add a handler to some existing gen_event by calling gen_event:add_handler() > on init(), and I want to call gen_event:remove_handler() when the > application is stop. Is there an easy way to do it? > > Thanks > > -- > Seven Du > http://www.freeswitch.org.cn > http://about.me/dujinfang > http://www.dujinfang.com > > Sent with Sparrow > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From z@REDACTED Wed Apr 24 14:23:48 2013 From: z@REDACTED (Danil Zagoskin) Date: Wed, 24 Apr 2013 16:23:48 +0400 Subject: [erlang-questions] otp supervisor terminate callback In-Reply-To: References: Message-ID: If you need just to remove gen_event handler, use gen_event:add_sup_handler, and handler will be removed automatically when your process dies. 2013/4/20 Seven Du > Hi, > > I'd like to know is it possible to have a terminate callback that clean up > things when the supervisor stops? > > Specifically I want to write an application which starts a supervisor, and > add a handler to some existing gen_event by calling gen_event:add_handler() > on init(), and I want to call gen_event:remove_handler() when the > application is stop. Is there an easy way to do it? > > Thanks > > -- > Seven Du > http://www.freeswitch.org.cn > http://about.me/dujinfang > http://www.dujinfang.com > > Sent with Sparrow > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- --------------------------------------------- ????? ???????? | +7 906 064 20 47 | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Wed Apr 24 14:56:53 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 24 Apr 2013 13:56:53 +0100 Subject: [erlang-questions] otp supervisor terminate callback In-Reply-To: References: Message-ID: Good point, although that might cause unexpected info messages to be delivered to the supervisor's mailbox if the handler is removed. AFAIR that'll only cause some noise in the logs via error_logger though, so maybe it's not a big deal. On 24 April 2013 13:23, Danil Zagoskin wrote: > If you need just to remove gen_event handler, use > gen_event:add_sup_handler, and handler will be removed automatically when > your process dies. > > > 2013/4/20 Seven Du > >> Hi, >> >> I'd like to know is it possible to have a terminate callback that clean >> up things when the supervisor stops? >> >> Specifically I want to write an application which starts a supervisor, >> and add a handler to some existing gen_event by calling >> gen_event:add_handler() on init(), and I want to call >> gen_event:remove_handler() when the application is stop. Is there an easy >> way to do it? >> >> Thanks >> >> -- >> Seven Du >> http://www.freeswitch.org.cn >> http://about.me/dujinfang >> http://www.dujinfang.com >> >> Sent with Sparrow >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > --------------------------------------------- > ????? ???????? | +7 906 064 20 47 | z@REDACTED > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Wed Apr 24 15:06:55 2013 From: vances@REDACTED (Vance Shipley) Date: Wed, 24 Apr 2013 18:36:55 +0530 Subject: [erlang-questions] otp supervisor terminate callback In-Reply-To: <0878237C6DC24E81B4CE82C476E18B42@gmail.com> References: <0878237C6DC24E81B4CE82C476E18B42@gmail.com> Message-ID: <20130424130655.GK21384@aluminium> On Wed, Apr 24, 2013 at 06:49:45PM +0800, Seven Du wrote: } I want a terminate callback in the -behaviour(supervisor). Sometimes it's easiest to change what you want. :) On Apr 20, 2013 4:34 PM, "Seven Du" wrote: } Specifically I want to write an application which starts a supervisor, } and add a handler to some existing gen_event by calling } gen_event:add_handler() on init(), and I want to call } gen_event:remove_handler() when the application is stop. If your application really only installs an event handler you might not need a supervisor at all. You can install the handler in your application behaviour start/1,2 callback and uninstall it in the stop/1 callback. For a robust application you would probably want to have a supervisor with a child worker, say a gen_server behaviour, which can supervise the event handler. Have your application behaviour start/1,2 callback start the supervisor. The supervisor starts the gen_server. The init/1 callback of the gen_server behaviour would use event_handler:add_sup_handler/3 to install the event handler. Add a clause in your handle_info/2 callback to handle the case where the event handler gets removed: handle_info({gen_event_EXIT, Handler, Reason}, State) -> error_logger:error_report([gen_event_EXIT, {event_handler, Handler}, {reason, Reason}]), case gen_event:add_sup_handler(State#state.ref, Handler, []) of ok -> {noreply, State} Other -> {stop, Other, State} end. -- -Vance From mononcqc@REDACTED Wed Apr 24 15:46:02 2013 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 24 Apr 2013 09:46:02 -0400 Subject: [erlang-questions] "New" vs. "old" console behavior: bug or feature? In-Reply-To: <45099.1366745814@snookles.snookles.com> References: <45099.1366745814@snookles.snookles.com> Message-ID: <20130424134601.GA78050@ferdair.local> Hi Scott, The IO world of Erlang is a fun crazy thing :) I've spent time trying to document how the shell works back at http://ferd.ca/repl-a-bit-more-and-less-than-that.html. I'll do a quick roundup of things just to be clear on everything. Before going into the difference between 'new' and 'old' shells, there is a 'user' process, which you mentioned, part of the IO system. The 'user' process acts as a default top-level group leader for all the output coming from a process. All group leaders are inherited from the process' parent. They can also be modified, so that you may have different group leaders across a VM: they are local processes, middle-men (like application_controller), or remote processes (this is how RPC calls get printed to everyone any time). By default, every OTP app will put its controller as a group leader for all sub-processes. This group leader will redirect output, but overload the feature to kill rogue processes on shutdown (it makes a list of all processes, inspects their group leader, and if it's the current app's pid, kills said process). Other tools like eunit and Common Test will have the possibility of injecting themselves above test cases and pick what to print or not. By sending IO directly to 'user', we bypass that hierarchy and go straight to the node's main IO process. Other special cases can be used, such as 'standard_error', which will redirect output to the error channel. That being said, there are two default implementations of a process that registers itself as 'user' on a node: the new (current) shell, and the 'old' shell. The choice of which one to pick is determined at boot time by the user_sup.erl module (part of kernel) through system flags: - If the node is a slave node, the 'user' module will point to a remote process. - If the node is started with no special flag, the new shell is started through 'user_drv'. This 'user' proc will act as a middle-man between input and output with a tty program and the different Erlang groups (see group.erl in kernel) to allow multiple jobs and concurrent shells without messed up output. Evaluation is handled by shell.erl (stdlib) - If the node is started with the -oldshell flag, the process in charge is 'user.erl', which uses special IO devices ({fd,0,1} for IO) to deal with the input and output channels for the node directly. It will send the evaluation to shell.erl also. - If the node is started with -noshell, the 'user.erl' module is still booted, but will not evaluate any input nor forward it. - If the node is started in -noinput mode, the 'user.erl' module is still booted, but it will not forward any input, only output from the node. It's a superset of -noshell and a bit safer because it opens the IO port in a way that only has the 'out' channel open. - There is an undocumented -nouser flag. Such a flag makes sure that neither user.erl nor user_drv.erl are started. The node will crash unless you specifically decide to start a process that registers itself as 'user' and decides to handle IO for your node. This is what you should use were you planning to provide your own Erlang shell and boot it as 'erl -nouser -s custom_shell'. - If it's not possible to boot the tty used by 'user_drv', it should fall-back to 'user.erl' as an IO leader. Alright. That covers most of it for the basics. To figure out why it blocks, we need to figure out the evaluation. The evaluation itself happens in a shell.erl process, which does an io request to the 'user' process (technically, its own group_leader, so that anyone may use the evaluator where they want. It just happens to be the 'user' process in this case). Input --> user.erl <---> shell.erl The shell does an io-request to user, which asks to read characters. The user.erl process forwards that data to the shell. The shell attempts to evaluate it, and if there's not enough data, it asks for more. user.erl then blocks until it can get more data to respond to the io request. When output is sent to 'user' it's sent as an additional io request, as a message. This message will not be read until the shell can answer the previous request. This is where you block. Input --> user.erl <---> shell.erl ^----> other proc The new shell does things differently by using a 'group.erl' process for each IO group. Now each group.erl process has the same potential to block, with the exception that user_drv.erl will start one very specific 'group.erl' process to be 'user', and will not return it as a potential shell.erl input source (it would be 0 in '^G -> j', and it is not possible to select it). user_drv will also consider it to be a special group that can *always* output to tty, wheras other groups will only have their output dumped by default if they're not the currently active one (hence you do not get other shells' output by default when you switch tasks). This means that while you could block things by finding the specific 'group.erl' you're currently sending IO requests to by default, it's unlikely to happen by accident, and 'user' is now a safe process to send IO requests to. I hope this explains things. I would find it difficult to call it a bug given a solution exists to the problem already, but I do see why the fallback to the old shell when no tty is available could be problematic. I'm guessing it would be possible to make a 'raw shell', which does tasks similar to user_drv, but using a user.erl-like adapter instead of a tty program to communicate with and starting it with 'erl -nouser -s rawshell' or something, or eventually making it the default user_drv falls back to instead of 'user:start()'. I'm guessing this would be a very low priority for the OTP team, though. I hope this lengthy response answers your questions! Regards, Fred. On 04/23, Scott Lystig Fritchie wrote: > Hi, all. I can't figure out if this message should be sent to the > erlang-bugs list or the erlang-questions list ... so I'll go for the > more general audience. > > Summary: Starting Erlang with a tty/pseudo-tty can get you a different > console shell ("new" and "old", respectively) without you realizing > it.(*) If you don't know that you're using the old shell, and if a > process tries to send output to the 'user' registered process(**), > e.g. io:format(user, "Some message with ~p extra\n", [Extra]), then it > is possible that the io:format() call will not return for > seconds/minutes/hours/ever. > > My question: Is the kind of indefinite blocking on I/O described below a > bug or a feature? > > I have a test case that can reproduce this behavior. An automated > version (using Expect) can be found at: > > https://gist.github.com/slfritchie/ad8e5cf1603cbe326be7 > > The basics of the reproducing the hang are: > > SSH session #1 SSH session #2 > -------------- -------------- > Start an Erlang daemon > using "run_erl". > > Attach to the daemon's console > using "to_erl". > > Start another Erlang VM > and connect to the first > VM via "-remsh". > > At the console, type the > following and press ENTER: > {term1, > > Run this command: > io:format(user, "Hey!\n", []). > > The io:format/3 call in session #2 will behave differently if session > #1's "run_erl" command runs with a tty/pseudo-tty or without. > > A. With a tty/pty: The io:format() call returns immediately. > B. Without a tty/pty: The io:format() call will hang indefinitely. > It will remain blocked until the Erlang term parser in session #1 > has returned. For example, finishing the term with "term2}." and > then pressing ENTER. > > The same effect can be seen by forcing the use of the old shell, without > using SSH, by simply running "erl -oldshell" for session #1 (in an Xterm > or other terminal window, or at the machine's hardware console) instead > of using SSH + "run_erl" + "to_erl". > > Riak was the application that triggered this bug hunt (in conjunction > with the Lager app)(***). Finding it has taken much longer than anyone > guessed. The reason is that the necessary precondition, starting Erlang > via 'run_erl' via SSH without an associated tty/pseudo-tty, is not > common. (Riak's packaging uses "sudo", which refuses to run if there > isn't a tty/pty available.) > > All attempts to duplicate the behavior failed because we didn't > understand that the root cause of the bad behavior was the old console > being silently chosen at VM startup when not tty/pty is available. > > -Scott > > (*) See > https://github.com/erlang/otp/blob/maint/lib/kernel/src/user_drv.erl#L103 > for how the choice is made. > > (**) From the 'io' man page: > > There is always a process registered under the name of user. This > can be used for sending output to the user. > > ... where "output to the user" really means "output to the Erlang > virtual machine console." > > (***) For source code of Riak and Lager, respectively, see: > https://github.com/basho/riak > https://github.com/basho/lager > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlangprogram@REDACTED Wed Apr 24 17:23:11 2013 From: erlangprogram@REDACTED (S X) Date: Wed, 24 Apr 2013 11:23:11 -0400 Subject: [erlang-questions] diameter callback module In-Reply-To: References: Message-ID: Thanks a lot for your comments. The diameter:call function allows to send extra arguments. It works fine after changing the callback function signatures accordingly. This is very useful when I want to notify the other process to do the next thing. Before sending a diameter message, we call diameter:add_transport to connect to a peer and peform CER/CEA capability information exchange. On the caller side, let's say the client callback module, the peer_up will be invoked when CER/CEA is completed. Why it doesn't have the similar mechanism like diameter:call to allow insert additional arguments so we can utilize them, for example, notify the others to send diameter messages? diameter:add_transport is a sync call, but it doesn't mean you can send diameter messages successfully when the function returns, i.e. it usually gets {error, no_connection} or { error, timeout }(this might because of the server side) if you call diameter:call right after diameter:add_transport. So is there any reason why not allowing to add additional arguments and use them in peer_up callback function? Since it means capability exchange is done and the peer is ready, at this point it should be safe to send diameter messages to the peer. Any suggestions? Thanks! Samuel On Tue, Apr 16, 2013 at 7:22 AM, Anders Svensson wrote: > On Sun, Apr 14, 2013 at 7:01 PM, S X wrote: > > Hello, > > > > Based on the erlang diameter library and the sample code, I want to start > > multiple diameter client processes in one erlang node(one client IP), and > > the client needs to define a diameter_app callback module for certain > > application, for example: > > > > -define(SERVICE(Name), [{'Origin-Host', ?L(Name) ++ ".example.com"}, > > {'Origin-Realm', "example.com"}, > > {'Vendor-Id', 193}, > > {'Product-Name', "Client"}, > > {'Auth-Application-Id', [?DIAMETER_APP_ID_COMMON, > > ?DIAMETER_APP_ID_CCRA]}, > > {application, [{alias, ?APP_CCR_ALIAS}, > > {dictionary, ?DIAMETER_DICT_CCRA}, > > {module, client_cb_ccra}]}]). > > > > > > First question: > > how the diameter library handles this situation? Will all diameter client > > processes share one single diameter_app callback module "client_cb_ccra" > or > > it will automatically attach different instance of the callback module > > (process) to the different client process by using spawn_monitor? So from > > the callback handle_answer in "client_cb_ccra" I can notify the proper > > client proce by just calling client:notify() something. > > diameter doesn't know anything about your client process. If you want > a callback to be able to contact the client process associated with > the service in question (ie. the service whose name the callback gets > as an argument) then you need to give the callback the means to do so. > One way would be to map the service name to your process (eg. it's the > registered name of your client process), another would be to pass some > identification as extra arguments to the callbacks. (Eg. {module, > [client_cb_ccra, X]} in the config above.) > > > Second question: > > Notice that diameter:call allows to set extra arguments, so I was > wondering > > I could set some data like: > > diameter:call(Name, ?APP_CCR_ALIAS, CCR, [{extra, [self()]}]). > > which sets the client process ID and I hope to deliver to the callback > > module "client_cb_ccra" and when the callback module knows which client > > process sends request and response accordingly, for example using the > client > > process ID in the callback function handle_answer. > > You can do that, but what is it you're trying to accomplish? if it's > handle_answer that's supposed to communicate something then it already > does: the return value of handle_answer is returned by diameter:call. > > > However, I don't want pack the extra arguments into the diameter packet > > since the diameter server doesn't know what they are and the extra ones > are > > not part of standard diameter packet. > > Not sure what you mean here. There's no way for your client to send > the server anything other than a Diameter message. > > > Now I changed the pick_peer callback signatures to allow extra arguments, > > pick_peer([Peer | _], _, _SvcName, _State, A) > > > > But I got encoding error in the callback prepare_request > > > > =ERROR REPORT==== 14-Apr-2013::11:40:41 === > > Error in process <0.175.0> with exit value: > > {undef,[{client_cb_ccra,prepare_request,[{diameter_packet, > > The arity of your callback probably doesn't agree with the number of > extra arguments you've specified: prepare_request will also get your > extra arguments. > > > >{diameter_header,1,undefined,undefined,undefined,3958849953,3958849953,undefined,undefined,undefined,undefined},undefined,{diameter_rfc4006_cc_CCR,["who",";","142745567... > > > > {error,encode} > > > > In overall, I read the online documents, which have limited information > on > > how to use the extra arguments in the library and don't quite get how to > > utilize the extra arguments to do something tricky, > > > > Are there any suggestions on how to deal with multiple client processes? > > Not sure I understand what problem it is you're trying to solve. > Multiple processes invoking diameter:call is nothing strange. You > typically just return something useful (eg. the answer message from > the peer) from handle_answer, which diameter:call then returns for the > caller to deal with. > > Anders > > > > Thanks a lot! > > > > Samuel > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Wed Apr 24 18:44:51 2013 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Wed, 24 Apr 2013 11:44:51 -0500 Subject: [erlang-questions] NIF native process updates? Message-ID: It has been a few months since I've begged for native processes, so it is time to do it again. ;) Has there been any progress on NIF native processes? R17? R18? It is the simple event() and io_event() callback mechanisms that really appeal to me, not so much the dirty schedulers. Is it constructive to separate the two and do event()/io_event() first and dirty schedulers later? Thanks, Dan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Wed Apr 24 20:05:38 2013 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 24 Apr 2013 19:05:38 +0100 (BST) Subject: [erlang-questions] "New" vs. "old" console behavior: bug or feature? In-Reply-To: <20130424134601.GA78050@ferdair.local> Message-ID: <1677147773.80683172.1366826738926.JavaMail.root@erlang-solutions.com> Strange because both user.erl and group.erl "should" be able to handle output requests in the middle of getting input. But it is a little difficult to see in group as there is all this tricky search code. :-) Robert ----- Original Message ----- > From: "Fred Hebert" > To: "Scott Lystig Fritchie" > Cc: erlang-questions@REDACTED > Sent: Wednesday, 24 April, 2013 9:46:02 AM > Subject: Re: [erlang-questions] "New" vs. "old" console behavior: bug or feature? > > Hi Scott, > > The IO world of Erlang is a fun crazy thing :) > > I've spent time trying to document how the shell works back at > http://ferd.ca/repl-a-bit-more-and-less-than-that.html. I'll do a > quick > roundup of things just to be clear on everything. > > Before going into the difference between 'new' and 'old' shells, > there > is a 'user' process, which you mentioned, part of the IO system. The > 'user' process acts as a default top-level group leader for all the > output coming from a process. All group leaders are inherited from > the > process' parent. They can also be modified, so that you may have > different group leaders across a VM: they are local processes, > middle-men (like application_controller), or remote processes (this > is > how RPC calls get printed to everyone any time). > > By default, every OTP app will put its controller as a group leader > for > all sub-processes. This group leader will redirect output, but > overload > the feature to kill rogue processes on shutdown (it makes a list of > all > processes, inspects their group leader, and if it's the current app's > pid, kills said process). Other tools like eunit and Common Test will > have the possibility of injecting themselves above test cases and > pick > what to print or not. By sending IO directly to 'user', we bypass > that > hierarchy and go straight to the node's main IO process. Other > special > cases can be used, such as 'standard_error', which will redirect > output > to the error channel. > > That being said, there are two default implementations of a process > that > registers itself as 'user' on a node: the new (current) shell, and > the > 'old' shell. The choice of which one to pick is determined at boot > time > by the user_sup.erl module (part of kernel) through system flags: > > - If the node is a slave node, the 'user' module will point to a > remote > process. > - If the node is started with no special flag, the new shell is > started > through 'user_drv'. This 'user' proc will act as a middle-man > between > input and output with a tty program and the different Erlang groups > (see group.erl in kernel) to allow multiple jobs and concurrent > shells > without messed up output. Evaluation is handled by shell.erl > (stdlib) > - If the node is started with the -oldshell flag, the process in > charge > is 'user.erl', which uses special IO devices ({fd,0,1} for IO) to > deal > with the input and output channels for the node directly. It will > send > the evaluation to shell.erl also. > - If the node is started with -noshell, the 'user.erl' module is > still > booted, but will not evaluate any input nor forward it. > - If the node is started in -noinput mode, the 'user.erl' module is > still booted, but it will not forward any input, only output from > the > node. It's a superset of -noshell and a bit safer because it opens > the > IO port in a way that only has the 'out' channel open. > - There is an undocumented -nouser flag. Such a flag makes sure that > neither user.erl nor user_drv.erl are started. The node will crash > unless you specifically decide to start a process that registers > itself as 'user' and decides to handle IO for your node. This is > what > you should use were you planning to provide your own Erlang shell > and > boot it as 'erl -nouser -s custom_shell'. > - If it's not possible to boot the tty used by 'user_drv', it should > fall-back to 'user.erl' as an IO leader. > > Alright. That covers most of it for the basics. > > To figure out why it blocks, we need to figure out the evaluation. > The > evaluation itself happens in a shell.erl process, which does an io > request to the 'user' process (technically, its own group_leader, so > that anyone may use the evaluator where they want. It just happens to > be > the 'user' process in this case). > > Input --> user.erl <---> shell.erl > > The shell does an io-request to user, which asks to read characters. > The user.erl process forwards that data to the shell. The shell > attempts to evaluate it, and if there's not enough data, it asks for > more. user.erl then blocks until it can get more data to respond to > the > io request. > > When output is sent to 'user' it's sent as an additional io request, > as > a message. This message will not be read until the shell can answer > the > previous request. This is where you block. > > Input --> user.erl <---> shell.erl > ^----> other proc > > The new shell does things differently by using a 'group.erl' process > for > each IO group. Now each group.erl process has the same potential to > block, with the exception that user_drv.erl will start one very > specific > 'group.erl' process to be 'user', and will not return it as a > potential > shell.erl input source (it would be 0 in '^G -> j', and it is not > possible to select it). user_drv will also consider it to be a > special > group that can *always* output to tty, wheras other groups will only > have their output dumped by default if they're not the currently > active > one (hence you do not get other shells' output by default when you > switch tasks). This means that while you could block things by > finding > the specific 'group.erl' you're currently sending IO requests to by > default, it's unlikely to happen by accident, and 'user' is now a > safe > process to send IO requests to. > > I hope this explains things. I would find it difficult to call it a > bug > given a solution exists to the problem already, but I do see why the > fallback to the old shell when no tty is available could be > problematic. > I'm guessing it would be possible to make a 'raw shell', which does > tasks similar to user_drv, but using a user.erl-like adapter instead > of > a tty program to communicate with and starting it with 'erl -nouser > -s > rawshell' or something, or eventually making it the default user_drv > falls back to instead of 'user:start()'. I'm guessing this would be a > very low priority for the OTP team, though. > > I hope this lengthy response answers your questions! > > Regards, > Fred. > > On 04/23, Scott Lystig Fritchie wrote: > > Hi, all. I can't figure out if this message should be sent to the > > erlang-bugs list or the erlang-questions list ... so I'll go for > > the > > more general audience. > > > > Summary: Starting Erlang with a tty/pseudo-tty can get you a > > different > > console shell ("new" and "old", respectively) without you realizing > > it.(*) If you don't know that you're using the old shell, and if a > > process tries to send output to the 'user' registered process(**), > > e.g. io:format(user, "Some message with ~p extra\n", [Extra]), then > > it > > is possible that the io:format() call will not return for > > seconds/minutes/hours/ever. > > > > My question: Is the kind of indefinite blocking on I/O described > > below a > > bug or a feature? > > > > I have a test case that can reproduce this behavior. An automated > > version (using Expect) can be found at: > > > > https://gist.github.com/slfritchie/ad8e5cf1603cbe326be7 > > > > The basics of the reproducing the hang are: > > > > SSH session #1 SSH session #2 > > -------------- -------------- > > Start an Erlang daemon > > using "run_erl". > > > > Attach to the daemon's console > > using "to_erl". > > > > Start another Erlang VM > > and connect to the first > > VM via "-remsh". > > > > At the console, type the > > following and press ENTER: > > {term1, > > > > Run this command: > > io:format(user, > > "Hey!\n", []). > > > > The io:format/3 call in session #2 will behave differently if > > session > > #1's "run_erl" command runs with a tty/pseudo-tty or without. > > > > A. With a tty/pty: The io:format() call returns immediately. > > B. Without a tty/pty: The io:format() call will hang > > indefinitely. > > It will remain blocked until the Erlang term parser in > > session #1 > > has returned. For example, finishing the term with > > "term2}." and > > then pressing ENTER. > > > > The same effect can be seen by forcing the use of the old shell, > > without > > using SSH, by simply running "erl -oldshell" for session #1 (in an > > Xterm > > or other terminal window, or at the machine's hardware console) > > instead > > of using SSH + "run_erl" + "to_erl". > > > > Riak was the application that triggered this bug hunt (in > > conjunction > > with the Lager app)(***). Finding it has taken much longer than > > anyone > > guessed. The reason is that the necessary precondition, starting > > Erlang > > via 'run_erl' via SSH without an associated tty/pseudo-tty, is not > > common. (Riak's packaging uses "sudo", which refuses to run if > > there > > isn't a tty/pty available.) > > > > All attempts to duplicate the behavior failed because we didn't > > understand that the root cause of the bad behavior was the old > > console > > being silently chosen at VM startup when not tty/pty is available. > > > > -Scott > > > > (*) See > > https://github.com/erlang/otp/blob/maint/lib/kernel/src/user_drv.erl#L103 > > for how the choice is made. > > > > (**) From the 'io' man page: > > > > There is always a process registered under the name of user. > > This > > can be used for sending output to the user. > > > > ... where "output to the user" really means "output to the Erlang > > virtual machine console." > > > > (***) For source code of Riak and Lager, respectively, see: > > https://github.com/basho/riak > > https://github.com/basho/lager > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mononcqc@REDACTED Wed Apr 24 21:01:38 2013 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 24 Apr 2013 15:01:38 -0400 Subject: [erlang-questions] "New" vs. "old" console behavior: bug or feature? In-Reply-To: <1677147773.80683172.1366826738926.JavaMail.root@erlang-solutions.com> References: <20130424134601.GA78050@ferdair.local> <1677147773.80683172.1366826738926.JavaMail.root@erlang-solutions.com> Message-ID: <20130424190137.GB78050@ferdair.local> On 04/24, Robert Virding wrote: > Strange because both user.erl and group.erl "should" be able to handle output requests in the middle of getting input. But it is a little difficult to see in group as there is all this tricky search code. :-) > > Robert > The big difference I see is really about an io_request that requires more input coming directly to user. In most cases, things will be fine. When we have shell.erl requesting data to user.erl, we enter an io_request to fetch for content through io:scan_erl_exprs/3. This one has a get_until io request, which makes the user.erl process enter get_chars/7. This one, until it gets the 'eof' value it set itself internally, will keep looping in get_chars/8. In this spot, I believe it should still be able to serve IO reqs, except when it starts getting data from the port, at which point, it enters get_chars_bytes/8, which may end up calling get_chars_apply/7. Once you enter this one, there's a possibility of having incomplete data, prompting for a new state, which calls get_chars_more/7, which selectively receives and ignores io requests and can go recursively. I'm guessing (at a glance, haven't checked if it actually enters these paths with the actual code) that this is where the blocking happens. In fact, a quick try with process_info/1 shows me that it's where the 'user' proc gets stuck: (hi@REDACTED)10> process_info(whereis(user)). [{registered_name,user}, {current_function,{user,get_chars_more,7}}, {initial_call,{erlang,apply,2}}, {status,waiting}, {message_queue_len,1}, {messages,[{io_request,<0.63.0>,<0.29.0>, {put_chars,unicode,io_lib,format,["hey",[]]}}]}, {links,[<0.6.0>,<0.28.0>,<0.33.0>,#Port<0.425>]}, {dictionary,[{encoding,latin1}, {read_mode,list}, {shell,<0.33.0>}]}, ... {suspending,[]}] Looking at this, I'm guessing it would be possible to have a workaround where get_chars_more/7 could listen to io_requests, but only if they are for output -- otherwise you could get into nasty cases where you start accepting other io requests that pull for data and things would get even more confusing really fast and turn the handling of io events inside out. I'm thinking I recall seeing group.erl having a similar function that could get locked up, but I haven't checked or tested to see if it was indeed possible to lock it up in a similar way -- the fact that 'user' is a distinct standalone group process makes the issue unlikely to happen in the first place. Regards, Fred. From fritchie@REDACTED Wed Apr 24 22:15:55 2013 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 24 Apr 2013 15:15:55 -0500 Subject: [erlang-questions] "New" vs. "old" console behavior: bug or feature? In-Reply-To: Message of "Wed, 24 Apr 2013 19:05:38 BST." <1677147773.80683172.1366826738926.JavaMail.root@erlang-solutions.com> Message-ID: <23038.1366834555@snookles.snookles.com> Robert Virding wrote: rv> Strange because both user.erl and group.erl "should" be able to rv> handle output requests in the middle of getting input. But it is a rv> little difficult to see in group as there is all this tricky search rv> code. :-) "should" /= "reality", alas. See the backtrace below when the problem hits. -Scott --- snip --- snip --- snip --- snip --- snip --- snip --- =proc:<0.30.0> State: Waiting Name: user Spawned as: erlang:apply/2 Current call: user:get_chars_more/7 Spawned by: <0.29.0> Started: Tue Apr 9 16:16:46 2013 Message queue length: 1 Message queue: [{io_request,<0.186.0>,<0.179.0>,{put_chars,unicode,<<821 bytes>>}}] Number of heap fragments: 0 Heap fragment data: 0 Link list: [] Dictionary: [{shell,<0.31.0>},{read_mode,list},{unicode,false}] Reductions: 357542 Stack+heap: 610 OldHeap: 0 Heap unused: 152 OldHeap unused: 0 Stack dump: Program counter: 0x00007f0c51d92930 (user:get_chars_more/7 + 232) CP: 0x0000000000000000 (invalid) arity = 0 0x00007f0c51931710 Return addr 0x00007f0c51d8c498 (user:do_io_request/5 + 88) y(0) unicode y(1) {[],[]} y(2) #Port<0.630> y(3) {erl_scan,tokens,[1]} y(4) get_until y(5) io_lib y(6) {erl_scan_continuation,[],no_col,[],2,{erl_scan,#Fun,false,false,false},0,#Fun} 0x00007f0c51931750 Return addr 0x00007f0c51d8c308 (user:server_loop/2 + 1408) y(0) #Port<0.630> y(1) <0.30.0> y(2) <0.25333.1728> 0x00007f0c51931770 Return addr 0x00007f0c51d8b968 (user:catch_loop/3 + 112) y(0) #Port<0.630> 0x00007f0c51931780 Return addr 0x0000000000836c78 () y(0) <0.31.0> y(1) #Port<0.630> y(2) Catch 0x00007f0c51d8b968 (user:catch_loop/3 + 112) From fritchie@REDACTED Wed Apr 24 22:39:36 2013 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Wed, 24 Apr 2013 15:39:36 -0500 Subject: [erlang-questions] "New" vs. "old" console behavior: bug or feature? In-Reply-To: Message of "Wed, 24 Apr 2013 09:46:02 EDT." <20130424134601.GA78050@ferdair.local> Message-ID: <24197.1366835976@snookles.snookles.com> Fred Hebert wrote: fh> Input --> user.erl <---> shell.erl fh> [...] The shell attempts to evaluate it, and if there's not enough fh> data, it asks for more. user.erl then blocks until it can get more fh> data to respond to the io request. fh> When output is sent to 'user' it's sent as an additional io request, fh> as a message. This message will not be read until the shell can fh> answer the previous request. This is where you block. Yup. The combination of using "run_erl" + no tty/pty + lager prior to this work(*) meant that any Erlang process that attempted to log a message would be blocked arbitrarily until user.erl would return from user:get_chars_more() et al. If someone had used the Riak console recently, attaching to the VM's console via "riak attach" (which in turn uses "to_erl"), and if the last thing that they typed was not the end of an expression and then detached from the console(**), every process that attempts to log a message afterward will hang. Forever. Or until someone re-attaches to the console and types "." and ENTER so that user.erl can finish its simple-minded parsing. The completely silent nature of the switch to the old shell was baffling. Not to mention causing us & our customers serious grief. I finally starting finding the root cause when I realized that all cases of the all-lager-events-blocked-arbirarily mystery involved systems where there was no 'user_drv' registered process. -Scott (*) https://github.com/basho/lager/pull/139 (**) Or if that person merely pressed ENTER before detaching, an exceptionally easy thing to do. From maruthavanan_s@REDACTED Thu Apr 25 08:02:19 2013 From: maruthavanan_s@REDACTED (Maruthavanan Subbarayan) Date: Thu, 25 Apr 2013 02:02:19 -0400 Subject: [erlang-questions] lists:member/2 and erlang:function_exported/3 conflict Message-ID: Hi, Why "erlang:function_exported(lists,member,2)" gives me false where as the function is exported? 15> erlang:function_exported(lists,member,2).false16> proplists:lookup(member, element(2,proplists:lookup(exports,lists:module_info()))).{member,2} Tried with some other functions randomly but got true. I tried with R14B and R16B. Is there any hidden reason behind it? Thanks,marutha -------------- next part -------------- An HTML attachment was scrubbed... URL: From maruthavanan_s@REDACTED Thu Apr 25 08:06:33 2013 From: maruthavanan_s@REDACTED (Maruthavanan Subbarayan) Date: Thu, 25 Apr 2013 02:06:33 -0400 Subject: [erlang-questions] lists:member/2 and erlang:function_exported/3 conflict In-Reply-To: References: Message-ID: Hi, Guess I got the answer as "erlang:function_exported" would Returns false for any BIF (functions implemented in C rather than in Erlang). So lists:member/2 would have been implemented in C. Sorry for the noise. Regards,MaruthaFrom: maruthavanan_s@REDACTED To: erlang-questions@REDACTED Date: Thu, 25 Apr 2013 02:02:19 -0400 Subject: [erlang-questions] lists:member/2 and erlang:function_exported/3 conflict Hi, Why "erlang:function_exported(lists,member,2)" gives me false where as the function is exported? 15> erlang:function_exported(lists,member,2).false16> proplists:lookup(member, element(2,proplists:lookup(exports,lists:module_info()))).{member,2} Tried with some other functions randomly but got true. I tried with R14B and R16B. Is there any hidden reason behind it? Thanks,marutha _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjarvstrand@REDACTED Thu Apr 25 09:21:04 2013 From: tjarvstrand@REDACTED (=?ISO-8859-1?Q?Thomas_J=E4rvstrand?=) Date: Thu, 25 Apr 2013 09:21:04 +0200 Subject: [erlang-questions] Request for comment: A proposal to introduce the concept of function domains Message-ID: Hi all, I write this email to get the community's opinion on an idea to extend Erlang with the concept of function domains. The intention is for the end result to be submitted as an EEP. The rationale is that encapsulation is a good thing and that code should not be allowed to depend on library functions that the library's author did not intend to expose to the outside world. Because of this I would like to introduce the idea of exporting functions into different domains. There will be three predefined function domains: one will allow the function to be called from anywhere, one will allow the function to be called from within its own application and one to disallow the function from being explicitly referenced anywhere outside its module (this is for behaviour callbacks, functions passed/returned as funs etc.) To allow for extension in the future, compilation will allow any atom to be given as the domain name (warnings could be added for non-predefined domains), but xref will only be extended with processing of the predefined domains. Suggestions for what to call the predefined domains are: *public, restricted, private* The rationale is that due to how the language works, a domain-declaration will only specify where we allow the function to be referenced with its fully qualified name we can't detect other any other references anyway. * external, internal, restricted* The rationale is that functions in the private domain are not really private at all since they can be called from anywhere (eg. the handle_call of a gen_server will be called from the gen_server-module). Private the becomes restricted, because that's what it really is and we use the duality of external/internal for allowing calls from outside/inside the application. The main issue with this suggestion is that many people associate internal with non-exported functions. I have two suggestions for how the domains should be declared in a module, they are both attributes that take two arguments: the domain and a list of /: *The domain/2 attribute: * Pros: Backwards compatibilty Cons: "Clutters" the attribute namespace. Requires information to be duplicated in the module (need both export and domain) *The export/2 attribute: * Pros: Avoids cluttering the attribute namespace and avoids duplicated information in the code. Cons: Breaks backwards compatibility with earlier OTP releases for code written using the new attribute. I would appreciate some input on this, both from the community and the OTP -team. Regards Thomas J?rvstrand -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Thu Apr 25 09:49:28 2013 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 25 Apr 2013 00:49:28 -0700 Subject: [erlang-questions] Request for comment: A proposal to introduce the concept of function domains In-Reply-To: References: Message-ID: <5178E008.9090604@gmail.com> On 04/25/2013 12:21 AM, Thomas J?rvstrand wrote: > Hi all, > > I write this email to get the community's opinion on an idea to extend Erlang with the concept of function domains. The intention is for the end result to be submitted as an EEP. > > The rationale is that encapsulation is a good thing and that code should not be allowed to depend on library functions that the library's author did not intend to expose to the outside world. Because of this I would like to introduce the idea of exporting functions into different domains. > > There will be three predefined function domains: one will allow the function to be called from anywhere, one will allow the function to be called from within its own application and one to disallow the function from being explicitly referenced anywhere outside its module (this is for behaviour callbacks, functions passed/returned as funs etc.) > > To allow for extension in the future, compilation will allow any atom to be given as the domain name (warnings could be added for non-predefined domains), but xref will only be extended with processing of the predefined domains. > > Suggestions for what to call the predefined domains are: > > *public, restricted, private* > The rationale is that due to how the language works, a domain-declaration will only specify where we allow the function to be referenced with its fully qualified name we can't detect other any other references anyway. > * > external, internal, restricted* > The rationale is that functions in the private domain are not really private at all since they can be called from anywhere (eg. the handle_call of a gen_server will be called from the gen_server-module). Private the becomes restricted, because that's what it really is and we use the duality of > external/internal for allowing calls from outside/inside the application. The main issue with this suggestion is that many people associate internal with non-exported functions. > > I have two suggestions for how the domains should be declared in a module, they are both attributes that take two arguments: the domain and a list of /: > > *The domain/2 attribute: > * > Pros: Backwards compatibilty > Cons: "Clutters" the attribute namespace. Requires information to be duplicated in the module (need both export and domain) > > *The export/2 attribute: > * > Pros: Avoids cluttering the attribute namespace and avoids duplicated information in the code. > Cons: Breaks backwards compatibility with earlier OTP releases for code written using the new attribute. > > I would appreciate some input on this, both from the community and the OTP-team. > > Regards > Thomas J?rvstrand > I think that function domains touches upon a related issue, which could hopefully be combined into one EEP. The issue is a lack of module namespaces, which currently impacts application dependencies. Ignoring any potential use by the developer of a namespace concept, and definitely ignoring the past packages implementation that has since been removed from Erlang, there exists an application dependency problem which limits the growth and flexibility of Erlang systems. The problem is that if application A and application B both depend on C, but on different versions of C, it is difficult to include both A and B without modifying one of them. The idea of function domains can help provide some access control of a module within the context of a single application, however, access control is a larger problem which impacts how applications can be combined. To make Erlang more flexible for integrating with various open source dependencies, and various legacy code dependencies, it is often not realistic to build the release with only a single version of a specific application dependency. I think having the version optionally be added to all of the module names (as a suffix) within an application, which is a problematic dependency, is a way of coping with the issue (it could be based on a flag within the application file of the dependency), but I am sure there are other solutions which may be better. Either way, both emails deal with the concept of access control for Erlang source code. - Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Thu Apr 25 09:53:15 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 25 Apr 2013 09:53:15 +0200 Subject: [erlang-questions] clarify: gen_server:start_link/3 return values? Message-ID: <1366876395.4803.14.camel@sekic1152.rnd.ki.sw.ericsson.se> Greetings, Should gen_server:start_link/3 return {error, Reason} if my init/1 returns {stop, Reason}? When I return {stop, Reason} from init/1, then gen_server:start/3 returns {error, Reason}. I think the documentation promises the same for gen_server:start_link/3. Is my understanding correct? Test program: -module(t). -export([a/0, al/0, init/1]). a() -> gen_server:start(t, [], []). al() -> gen_server:start_link(t, [], []). init([]) -> {stop, t}. bengt From baliulia@REDACTED Thu Apr 25 10:02:41 2013 From: baliulia@REDACTED (=?UTF-8?B?SWduYXMgVnnFoW5pYXVza2Fz?=) Date: Thu, 25 Apr 2013 10:02:41 +0200 Subject: [erlang-questions] "New" vs. "old" console behavior: bug or feature? In-Reply-To: <45099.1366745814@snookles.snookles.com> References: <45099.1366745814@snookles.snookles.com> Message-ID: <5178E321.3070406@gmail.com> Hi Scott, Fred, On 04/23/2013 09:36 PM, Scott Lystig Fritchie wrote: > > The io:format/3 call in session #2 will behave differently if session > #1's "run_erl" command runs with a tty/pseudo-tty or without. > > A. With a tty/pty: The io:format() call returns immediately. B. > Without a tty/pty: The io:format() call will hang indefinitely. It > will remain blocked until the Erlang term parser in session #1 has > returned. For example, finishing the term with "term2}." and then > pressing ENTER. Thank you for pointing this out! I've been seeing this fairly often (specifically in the combination with lager that you mentioned) and never figured out what caused it. On 04/24/2013 03:46 PM, Fred Hebert wrote: > - If it's not possible to boot the tty used by 'user_drv', it should > fall-back to 'user.erl' as an IO leader. So how does Erlang determine whether TTY is available? Currently I've worked around this by forcing a `SHELL=screen` variable in the boot script and it seems to do the trick, but I don't really like this approach. Any suggestions? Regards, Ignas From aggelgian@REDACTED Thu Apr 25 10:03:19 2013 From: aggelgian@REDACTED (Aggelos Giantsios) Date: Thu, 25 Apr 2013 11:03:19 +0300 Subject: [erlang-questions] clarify: gen_server:start_link/3 return values? In-Reply-To: <1366876395.4803.14.camel@sekic1152.rnd.ki.sw.ericsson.se> References: <1366876395.4803.14.camel@sekic1152.rnd.ki.sw.ericsson.se> Message-ID: On Thu, Apr 25, 2013 at 10:53 AM, Bengt Kleberg wrote: > Greetings, > > Should gen_server:start_link/3 return {error, Reason} if my init/1 > returns {stop, Reason}? > Hey there! gen_server:start_link/3 should return {error, Reason} only if init/1 fails with Reason. In the case where init/1 returns {error, Reason} or ignore, the process that calls gen_server:start_link/3 should fail with Reason or ignore respectively. Aggelos -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Thu Apr 25 10:19:12 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 25 Apr 2013 10:19:12 +0200 Subject: [erlang-questions] clarify: gen_server:start_link/3 return values? In-Reply-To: References: <1366876395.4803.14.camel@sekic1152.rnd.ki.sw.ericsson.se> Message-ID: <1366877952.4803.25.camel@sekic1152.rnd.ki.sw.ericsson.se> What about the documentation for gen_server:start_link/3 that says: "If Module:init/1 returns {stop,Reason} or ignore, the process is terminated and the function returns {error,Reason} or ignore, respectively." Not a native English speaker I might be seeing things that are not there but does that not promise a return value? Also, "fail with Reason" usually means crashes/exits. However, even if init/1 crashes/exists I do not get a return value from gen_server:start_link/3. Under what circumstances will gen_server:start_link/3 return {error, Reason}? bengt On Thu, 2013-04-25 at 11:03 +0300, Aggelos Giantsios wrote: > > On Thu, Apr 25, 2013 at 10:53 AM, Bengt Kleberg > wrote: > Greetings, > > Should gen_server:start_link/3 return {error, Reason} if my > init/1 > returns {stop, Reason}? > > Hey there! > > gen_server:start_link/3 should return {error, Reason} only if init/1 > fails with Reason. > In the case where init/1 returns {error, Reason} or ignore, the > process that calls gen_server:start_link/3 should fail with Reason or > ignore respectively. > > Aggelos From pan@REDACTED Thu Apr 25 10:47:16 2013 From: pan@REDACTED (Patrik Nyblom) Date: Thu, 25 Apr 2013 10:47:16 +0200 Subject: [erlang-questions] NIF native process updates? In-Reply-To: References: Message-ID: <5178ED94.1060800@erlang.org> On 04/24/2013 06:44 PM, Daniel Goertzen wrote: > It has been a few months since I've begged for native processes, so it > is time to do it again. ;) > > Has there been any progress on NIF native processes? R17? R18? Native processes *are* in the roadmap, but not likely to be included in R17. My educated guess would be R18 or later :( > > It is the simple event() and io_event() callback mechanisms that > really appeal to me, not so much the dirty schedulers. Is it > constructive to separate the two and do event()/io_event() first and > dirty schedulers later? > > Thanks, > Dan. Best regards, /Patrik, OTP > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From aggelgian@REDACTED Thu Apr 25 11:04:20 2013 From: aggelgian@REDACTED (Aggelos Giantsios) Date: Thu, 25 Apr 2013 12:04:20 +0300 Subject: [erlang-questions] clarify: gen_server:start_link/3 return values? In-Reply-To: <1366877952.4803.25.camel@sekic1152.rnd.ki.sw.ericsson.se> References: <1366876395.4803.14.camel@sekic1152.rnd.ki.sw.ericsson.se> <1366877952.4803.25.camel@sekic1152.rnd.ki.sw.ericsson.se> Message-ID: On Thu, Apr 25, 2013 at 11:19 AM, Bengt Kleberg wrote: > > Under what circumstances will gen_server:start_link/3 return {error, > Reason}? > You are right, there is an inconsistency in the documentation. So let's have a look at the code of gen_server:start_link/3. In gen_server.erl ------------------------ start_link(Mod, Args, Options) -> gen:start(?MODULE, link, Mod, Args, Options). In gen.erl -------------- start(GenMod, LinkP, Mod, Args, Options) -> do_spawn(GenMod, LinkP, Mod, Args, Options). %% We only need this clause as LinkP =:= link in our case do_spawn(GenMod, link, Mod, Args, Options) -> Time = timeout(Options), proc_lib:start_link(?MODULE, init_it, [GenMod, self(), self(), Mod, Args, Options], Time, spawn_opts(Options)); init_it(GenMod, Starter, Parent, Mod, Args, Options) -> init_it2(GenMod, Starter, Parent, self(), Mod, Args, Options). init_it2(GenMod, Starter, Parent, Name, Mod, Args, Options) -> %% So in our case will call gen_server:init_it/6 where Parent will be the a PID GenMod:init_it(Starter, Parent, Name, Mod, Args, Options). In gen_server.erl ------------------------ init_it(Starter, Parent, Name0, Mod, Args, Options) -> Name = name(Name0), Debug = debug_options(Name, Options), %% The actual call of our module's init function case catch Mod:init(Args) of {ok, State} -> proc_lib:init_ack(Starter, {ok, self()}), loop(Parent, Name, State, Mod, infinity, Debug); {ok, State, Timeout} -> proc_lib:init_ack(Starter, {ok, self()}), loop(Parent, Name, State, Mod, Timeout, Debug); {stop, Reason} -> unregister_name(Name0), proc_lib:init_ack(Starter, {error, Reason}), exit(Reason); ignore -> unregister_name(Name0), proc_lib:init_ack(Starter, ignore), exit(normal); {'EXIT', Reason} -> unregister_name(Name0), proc_lib:init_ack(Starter, {error, Reason}), exit(Reason); Else -> Error = {bad_return_value, Else}, proc_lib:init_ack(Starter, {error, Error}), exit(Error) end. So we see that if our module's init function returns {stop, Reason} or exits with Reason then proc_lib:init_ack(Starter, {error, Reason}) will make proc_lib:start_link return {error, Reason}, but then exit(Reason) will actually cause the spawned server process to crash and subsequently the process that called gen_server:start_link. The behaviour you notice is thus completely justified and the only way that the behaviour the documentation describes is accurate is when your process traps exits and not allows exit(Reason) to crash it. Sorry for the long post! I hope this cleared things up! Aggelos -------------- next part -------------- An HTML attachment was scrubbed... URL: From anders.otp@REDACTED Thu Apr 25 11:40:03 2013 From: anders.otp@REDACTED (Anders Svensson) Date: Thu, 25 Apr 2013 11:40:03 +0200 Subject: [erlang-questions] diameter callback module In-Reply-To: References: Message-ID: On Wed, Apr 24, 2013 at 5:23 PM, S X wrote: > Thanks a lot for your comments. > > The diameter:call function allows to send extra arguments. It works fine > after changing the callback function signatures accordingly. This is very > useful when I want to notify the other process to do the next thing. > > Before sending a diameter message, we call diameter:add_transport to connect > to a peer and peform CER/CEA capability information exchange. On the caller > side, let's say the client callback module, the peer_up will be invoked when > CER/CEA is completed. Why it doesn't have the similar mechanism like > diameter:call to allow insert additional arguments so we can utilize them, > for example, notify the others to send diameter messages? No particular reason aside from history and that the need hasn't come up. As it is, you can pass arbitrary options to diameter:add_transport/2 (history again) and retrieve these in a callback with diameter:service_info(PeerRef), so that can be used a substitute for extra arguments in this case. > diameter:add_transport is a sync call, but it doesn't mean you can send add_transport *doesn't* wait for the config it's passed to result in a connection before returning. (It might never happen for one.) > diameter messages successfully when the function returns, i.e. it usually > gets {error, no_connection} or { error, timeout }(this might because of the > server side) if you call diameter:call right after diameter:add_transport. This is because the peer won't be ready to respond to requests until capabilities exchange has completed (at least). If you call before the relevant peer_up callback then the result is {error, no_connection}. After peer_up the result will be {error, timeout} if the peer doesn't answer. One case in which this is expected is after a connection has been reestablished following an exchange of 3 x DWR/DWA. (ie. RFC 3539 DOWN -> REOPEN -> OKAY.) Since both ends of the connection do this, the client can consider the connection to be reestablished before the server. If the client sends a request before the server is done with it's exchange (ie. reached OKAY) then RFC 6733 says it should discard the request, resulting in a timeout on the client end. > So is there any reason why not allowing to add additional arguments and use > them in peer_up callback function? Since it means capability exchange is > done and the peer is ready, at this point it should be safe to send diameter > messages to the peer. See above. Anders > > Any suggestions? > > Thanks! > > Samuel > > > On Tue, Apr 16, 2013 at 7:22 AM, Anders Svensson > wrote: >> >> On Sun, Apr 14, 2013 at 7:01 PM, S X wrote: >> > Hello, >> > >> > Based on the erlang diameter library and the sample code, I want to >> > start >> > multiple diameter client processes in one erlang node(one client IP), >> > and >> > the client needs to define a diameter_app callback module for certain >> > application, for example: >> > >> > -define(SERVICE(Name), [{'Origin-Host', ?L(Name) ++ ".example.com"}, >> > {'Origin-Realm', "example.com"}, >> > {'Vendor-Id', 193}, >> > {'Product-Name', "Client"}, >> > {'Auth-Application-Id', >> > [?DIAMETER_APP_ID_COMMON, >> > ?DIAMETER_APP_ID_CCRA]}, >> > {application, [{alias, ?APP_CCR_ALIAS}, >> > {dictionary, >> > ?DIAMETER_DICT_CCRA}, >> > {module, client_cb_ccra}]}]). >> > >> > >> > First question: >> > how the diameter library handles this situation? Will all diameter >> > client >> > processes share one single diameter_app callback module "client_cb_ccra" >> > or >> > it will automatically attach different instance of the callback module >> > (process) to the different client process by using spawn_monitor? So >> > from >> > the callback handle_answer in "client_cb_ccra" I can notify the proper >> > client proce by just calling client:notify() something. >> >> diameter doesn't know anything about your client process. If you want >> a callback to be able to contact the client process associated with >> the service in question (ie. the service whose name the callback gets >> as an argument) then you need to give the callback the means to do so. >> One way would be to map the service name to your process (eg. it's the >> registered name of your client process), another would be to pass some >> identification as extra arguments to the callbacks. (Eg. {module, >> [client_cb_ccra, X]} in the config above.) >> >> > Second question: >> > Notice that diameter:call allows to set extra arguments, so I was >> > wondering >> > I could set some data like: >> > diameter:call(Name, ?APP_CCR_ALIAS, CCR, [{extra, [self()]}]). >> > which sets the client process ID and I hope to deliver to the callback >> > module "client_cb_ccra" and when the callback module knows which client >> > process sends request and response accordingly, for example using the >> > client >> > process ID in the callback function handle_answer. >> >> You can do that, but what is it you're trying to accomplish? if it's >> handle_answer that's supposed to communicate something then it already >> does: the return value of handle_answer is returned by diameter:call. >> >> > However, I don't want pack the extra arguments into the diameter packet >> > since the diameter server doesn't know what they are and the extra ones >> > are >> > not part of standard diameter packet. >> >> Not sure what you mean here. There's no way for your client to send >> the server anything other than a Diameter message. >> >> > Now I changed the pick_peer callback signatures to allow extra >> > arguments, >> > pick_peer([Peer | _], _, _SvcName, _State, A) >> > >> > But I got encoding error in the callback prepare_request >> > >> > =ERROR REPORT==== 14-Apr-2013::11:40:41 === >> > Error in process <0.175.0> with exit value: >> > {undef,[{client_cb_ccra,prepare_request,[{diameter_packet, >> >> The arity of your callback probably doesn't agree with the number of >> extra arguments you've specified: prepare_request will also get your >> extra arguments. >> >> >> > >{diameter_header,1,undefined,undefined,undefined,3958849953,3958849953,undefined,undefined,undefined,undefined},undefined,{diameter_rfc4006_cc_CCR,["who",";","142745567... >> > >> > {error,encode} >> > >> > In overall, I read the online documents, which have limited information >> > on >> > how to use the extra arguments in the library and don't quite get how >> > to >> > utilize the extra arguments to do something tricky, >> > >> > Are there any suggestions on how to deal with multiple client processes? >> >> Not sure I understand what problem it is you're trying to solve. >> Multiple processes invoking diameter:call is nothing strange. You >> typically just return something useful (eg. the answer message from >> the peer) from handle_answer, which diameter:call then returns for the >> caller to deal with. >> >> Anders >> >> >> > Thanks a lot! >> > >> > Samuel >> > > > From mononcqc@REDACTED Thu Apr 25 13:51:35 2013 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 25 Apr 2013 07:51:35 -0400 Subject: [erlang-questions] "New" vs. "old" console behavior: bug or feature? In-Reply-To: <5178E321.3070406@gmail.com> References: <45099.1366745814@snookles.snookles.com> <5178E321.3070406@gmail.com> Message-ID: <20130425115134.GA37738@ferdmbp.local> On 04/25, Ignas Vy?niauskas wrote: > Hi Scott, Fred, > > On 04/24/2013 03:46 PM, Fred Hebert wrote: > > - If it's not possible to boot the tty used by 'user_drv', it should > > fall-back to 'user.erl' as an IO leader. > > So how does Erlang determine whether TTY is available? Currently I've > worked around this by forcing a `SHELL=screen` variable in the boot > script and it seems to do the trick, but I don't really like this > approach. Any suggestions? > There's a code snippet in user_drv.erl (lines 92-109) that does the detection of whether it's possible to spawn things: server(Pname, Shell) -> process_flag(trap_exit, true), case catch open_port({spawn,Pname}, [eof]) of {'EXIT', _} -> %% Let's try a dumb user instead user:start(); Port -> server1(Port, Port, Shell) end. server(Iname, Oname, Shell) -> process_flag(trap_exit, true), case catch open_port({spawn,Iname}, [eof]) of {'EXIT', _} -> %% It might be a dumb terminal lets start dumb user user:start(); Iport -> Oport = open_port({spawn,Oname}, [eof]), server1(Iport, Oport, Shell) end. The interesting clauses are the 'user:start()' fallbacks. Basically, whenever the user_drv cannot spawn the desired port program (Erlang's tty driver started as 'tty_sl -c -e'), it will fall back to the old shell by default. I'm thinking the two valid approaches are those I discussed with Robert Virding and Andrew Thompson off the mailing list yesterday -- either have lager not produce output to 'user' when it detects the old shell (which is what Andrew and Scott ended up going with iirc), or fixing the old shell the way I mentioned in another post. This would be the solution where the function that currently blocks the shell is able to forward output anyway -- something Robert told me he believes it did in the past. If the old behaviour was to forward the output and now it's gone (I haven't checked, but I'm ready to trust Robert on that), then this would be a regression bug that needs to be fixed by the OTP team (or any contributor with the time to do it) within user.erl. Regards, Fred. From bengt.kleberg@REDACTED Thu Apr 25 14:45:11 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 25 Apr 2013 14:45:11 +0200 Subject: [erlang-questions] clarify: gen_server:start_link/3 return values? In-Reply-To: References: <1366876395.4803.14.camel@sekic1152.rnd.ki.sw.ericsson.se> <1366877952.4803.25.camel@sekic1152.rnd.ki.sw.ericsson.se> Message-ID: <1366893911.4803.35.camel@sekic1152.rnd.ki.sw.ericsson.se> Would it not be consistent with the documentation if the return values {stop, Reason} or ignore, from init/1, would not cause a call to erlang:exit/1 ? bengt On Thu, 2013-04-25 at 12:04 +0300, Aggelos Giantsios wrote: > > On Thu, Apr 25, 2013 at 11:19 AM, Bengt Kleberg > wrote: > > Under what circumstances will gen_server:start_link/3 return > {error, > Reason}? > > You are right, there is an inconsistency in the documentation. > So let's have a look at the code of gen_server:start_link/3. > > In gen_server.erl > ------------------------ > start_link(Mod, Args, Options) -> > gen:start(?MODULE, link, Mod, Args, Options). > > In gen.erl > -------------- > start(GenMod, LinkP, Mod, Args, Options) -> > do_spawn(GenMod, LinkP, Mod, Args, Options). > > %% We only need this clause as LinkP =:= link in our case > do_spawn(GenMod, link, Mod, Args, Options) -> > Time = timeout(Options), > proc_lib:start_link(?MODULE, init_it, > [GenMod, self(), self(), Mod, Args, > Options], > Time, > spawn_opts(Options)); > > init_it(GenMod, Starter, Parent, Mod, Args, Options) -> > init_it2(GenMod, Starter, Parent, self(), Mod, Args, Options). > > init_it2(GenMod, Starter, Parent, Name, Mod, Args, Options) -> > %% So in our case will call gen_server:init_it/6 where Parent will > be the a PID > GenMod:init_it(Starter, Parent, Name, Mod, Args, Options). > > In gen_server.erl > ------------------------ > init_it(Starter, Parent, Name0, Mod, Args, Options) -> > Name = name(Name0), > Debug = debug_options(Name, Options), > %% The actual call of our module's init function > case catch Mod:init(Args) of > {ok, State} -> > proc_lib:init_ack(Starter, {ok, self()}), > loop(Parent, Name, State, Mod, infinity, Debug); > {ok, State, Timeout} -> > proc_lib:init_ack(Starter, {ok, self()}), > loop(Parent, Name, State, Mod, Timeout, Debug); > {stop, Reason} -> > unregister_name(Name0), > proc_lib:init_ack(Starter, {error, Reason}), > exit(Reason); > ignore -> > unregister_name(Name0), > proc_lib:init_ack(Starter, ignore), > exit(normal); > {'EXIT', Reason} -> > unregister_name(Name0), > proc_lib:init_ack(Starter, {error, Reason}), > exit(Reason); > Else -> > Error = {bad_return_value, Else}, > proc_lib:init_ack(Starter, {error, Error}), > exit(Error) > end. > > So we see that if our module's init function returns {stop, Reason} or > exits with Reason then proc_lib:init_ack(Starter, {error, Reason}) > will make proc_lib:start_link return {error, Reason}, but then > exit(Reason) will actually cause the spawned server process to crash > and subsequently the process that called gen_server:start_link. > The behaviour you notice is thus completely justified and the only way > that the behaviour the documentation describes is accurate is when > your process traps exits and not allows exit(Reason) to crash it. > > Sorry for the long post! I hope this cleared things up! > > Aggelos > From aggelgian@REDACTED Thu Apr 25 15:12:48 2013 From: aggelgian@REDACTED (Aggelos Giantsios) Date: Thu, 25 Apr 2013 16:12:48 +0300 Subject: [erlang-questions] clarify: gen_server:start_link/3 return values? In-Reply-To: <1366893911.4803.35.camel@sekic1152.rnd.ki.sw.ericsson.se> References: <1366876395.4803.14.camel@sekic1152.rnd.ki.sw.ericsson.se> <1366877952.4803.25.camel@sekic1152.rnd.ki.sw.ericsson.se> <1366893911.4803.35.camel@sekic1152.rnd.ki.sw.ericsson.se> Message-ID: On Thu, Apr 25, 2013 at 3:45 PM, Bengt Kleberg wrote: > Would it not be consistent with the documentation if the return values > {stop, Reason} or ignore, from init/1, would not cause a call to > erlang:exit/1 ? > > > bengt > The call to erlang:exit/1 is necessary as the spawned process must be terminated. The problem is that the link between the parent and the child process still exists after proc_lib:init_ack/2 returns. I suppose that if you want the behaviour to match the documentation, there should be a call to erlang:unlink/1 before raising the exception and crashing the spawned process. However, I do not know if the present behaviour is by design or a bug. Maybe someone from the OTP team can shed some light on this. Aggelos -------------- next part -------------- An HTML attachment was scrubbed... URL: From colanderman@REDACTED Thu Apr 25 17:37:40 2013 From: colanderman@REDACTED (Chris King) Date: Thu, 25 Apr 2013 11:37:40 -0400 Subject: [erlang-questions] Request for comment: A proposal to introduce the concept of function domains In-Reply-To: References: Message-ID: On Thu, Apr 25, 2013 at 3:21 AM, Thomas J?rvstrand wrote: > I write this email to get the community's opinion on an idea to extend > Erlang with the concept of function domains. The intention is for the end > result to be submitted as an EEP. "Function domain" already has a well-known mathematical definition: http://en.wikipedia.org/wiki/Domain_of_a_function I would instead call this "visibility" or "access level" or some such. From elbrujohalcon@REDACTED Thu Apr 25 19:04:27 2013 From: elbrujohalcon@REDACTED (Fernando Benavides) Date: Thu, 25 Apr 2013 14:04:27 -0300 Subject: [erlang-questions] clarify: gen_server:start_link/3 return values? In-Reply-To: References: <1366876395.4803.14.camel@sekic1152.rnd.ki.sw.ericsson.se> <1366877952.4803.25.camel@sekic1152.rnd.ki.sw.ericsson.se> <1366893911.4803.35.camel@sekic1152.rnd.ki.sw.ericsson.se> Message-ID: Marcelo (from Inaka) wrote a blog post about this particular subject not too long ago: http://inaka.net/blog/2012/11/29/every-day-erlang/ *Fernando Benavides * On Thu, Apr 25, 2013 at 10:12 AM, Aggelos Giantsios wrote: > > On Thu, Apr 25, 2013 at 3:45 PM, Bengt Kleberg > wrote: > >> Would it not be consistent with the documentation if the return values >> {stop, Reason} or ignore, from init/1, would not cause a call to >> erlang:exit/1 ? >> >> >> bengt >> > > The call to erlang:exit/1 is necessary as the spawned process must be > terminated. The problem is that the link between the parent and the child > process still exists after proc_lib:init_ack/2 returns. > I suppose that if you want the behaviour to match the documentation, there > should be a call to erlang:unlink/1 before raising the exception and > crashing the spawned process. > However, I do not know if the present behaviour is by design or a bug. > Maybe someone from the OTP team can shed some light on this. > > Aggelos > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fritchie@REDACTED Thu Apr 25 19:20:32 2013 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Thu, 25 Apr 2013 12:20:32 -0500 Subject: [erlang-questions] "New" vs. "old" console behavior: bug or feature? In-Reply-To: Message of "Thu\, 25 Apr 2013 10\:02\:41 +0200." <5178E321.3070406@gmail.com> Message-ID: <88938.1366910432@snookles.snookles.com> Ignas Vy?niauskas wrote: iv> Currently iv> I've worked around this by forcing a `SHELL=screen` variable in the iv> boot script and it seems to do the trick, but I don't really like iv> this approach. Any suggestions? Ignas, the Expect script that I put in my original/long message to this list contains a magic workaround. In the case where run_erl (on box A) is started via SSH from a remote box (call it box B). If box B uses "ssh -t" to force the allocation of a pseudo-tty on box A before executing "run_erl", then the problem goes away because the VM can use the new shell. If box B uses the ssh defaults, which on (all/most??) platforms does not force allocation a pty, the problem exists. Using 'screen' to force allocation of a pty is another hack. Using "script" would be the same kind of hack. I agree, there's all a bit disagreeable. -Scott From chandu.gokul138@REDACTED Thu Apr 25 22:10:35 2013 From: chandu.gokul138@REDACTED (Gokul Evuri) Date: Thu, 25 Apr 2013 22:10:35 +0200 Subject: [erlang-questions] USB driver Interface Message-ID: Hello I am trying to get data to our application through USB interface. My initial plan to deal with this is through using ports. Due to some reasons, now we are looking for developed opensource libraries. If any one have/working on Erlang interface to access USB devices, to share, that would be really helpful. Thanks in Advance, Regards, Gokul S Evuri opensource -------------- next part -------------- An HTML attachment was scrubbed... URL: From baliulia@REDACTED Thu Apr 25 22:48:09 2013 From: baliulia@REDACTED (=?UTF-8?B?SWduYXMgVnnFoW5pYXVza2Fz?=) Date: Thu, 25 Apr 2013 22:48:09 +0200 Subject: [erlang-questions] "New" vs. "old" console behavior: bug or feature? In-Reply-To: <88938.1366910432@snookles.snookles.com> References: <88938.1366910432@snookles.snookles.com> Message-ID: <51799689.30800@gmail.com> Hi Scott, On 04/25/2013 07:20 PM, Scott Lystig Fritchie wrote: >> Ignas Vy?niauskas wrote: >> >> Currently I've worked around this by forcing a `SHELL=screen` >> variable in the boot script and it seems to do the trick, but I >> don't really like this approach. Any suggestions? [I obviously meant `TERM=screen`.] > Ignas, the Expect script that I put in my original/long message to > this list contains a magic workaround. In the case where run_erl > (on box A) is started via SSH from a remote box (call it box B). Well, in fact you don't need the whole "ssh and several boxes" setup to reproduce the problem, I think this works too: 1. Generate a release of some project 2. Start the release pretending to have no tty capabilities: `TERM= ./rel/node/bin/node start` 3. Attach, input something without termination 4. Trigger some logging via `io:format(user, <..>)` (e.g. by lager) 5. Observe processes hanging, rejoice. > If box B uses "ssh -t" to force the allocation of a pseudo-tty on > box A before executing "run_erl", then the problem goes away because > the VM can use the new shell. Yes, but the problem with that is that it leaves it up to the user/admin to ensure that the release is started in an environment which claims to have decent term capabilities, which is something non-obvious and annoying. After reading a bit, I realise setting TERM *is* essentially the only way to advertise term capabilities, so I might settle for something like if [[ -z $TERM || $TERM == "dumb" ]]; then export TERM=screen; done inside of the release start script, which is hackish, but I think it prevents the problem from happening regardless of how the node is started and also has the nice side-effect of proper tab completion and etc everywhere. -- Ignas From omer.kilic@REDACTED Fri Apr 26 00:32:00 2013 From: omer.kilic@REDACTED (Omer Kilic) Date: Thu, 25 Apr 2013 23:32:00 +0100 Subject: [erlang-questions] USB driver Interface In-Reply-To: References: Message-ID: <5179AEE0.40301@erlang-solutions.com> Hi Gokul, On (25/04/2013 21:10), Gokul Evuri wrote: > Due to some reasons, now we are looking for developed opensource libraries. > If any one have/working on Erlang interface to access USB devices, to > share, that would be really helpful. You don't mention what kind of USB devices you are interfacing with so not entirely sure what sort of library you are looking for but there are two cases: - Devices that use raw USB protocol or generic USB classes such HID: You probably want to use libusb to talk to these devices. A quick search on "libusb erlang" returns [1] from Tony's Github account. - Devices that expose themselves via virtual serial ports: This is pretty common these days and you can use any UART library to talk to these devices. Again, Tony has an Erlang UART library[2] you should look at. HTH, Omer. [1] https://github.com/tonyrog/libusb [2] https://github.com/tonyrog/uart From max.lapshin@REDACTED Fri Apr 26 07:25:16 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 26 Apr 2013 09:25:16 +0400 Subject: [erlang-questions] USB driver Interface In-Reply-To: <5179AEE0.40301@erlang-solutions.com> References: <5179AEE0.40301@erlang-solutions.com> Message-ID: For video I've written uvc: https://github.com/erlyvideo/uvc On Friday, April 26, 2013, Omer Kilic wrote: > > Hi Gokul, > > On (25/04/2013 21:10), Gokul Evuri wrote: > >> Due to some reasons, now we are looking for developed opensource >> libraries. >> If any one have/working on Erlang interface to access USB devices, to >> share, that would be really helpful. >> > > You don't mention what kind of USB devices you are interfacing with so not > entirely sure what sort of library you are looking for but there are two > cases: > > - Devices that use raw USB protocol or generic USB classes such HID: You > probably want to use libusb to talk to these devices. A quick search on > "libusb erlang" returns [1] from Tony's Github account. > > - Devices that expose themselves via virtual serial ports: This is > pretty common these days and you can use any UART library to talk to these > devices. Again, Tony has an Erlang UART library[2] you should look at. > > > HTH, > Omer. > > > [1] https://github.com/tonyrog/**libusb > [2] https://github.com/tonyrog/**uart > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjarvstrand@REDACTED Fri Apr 26 14:34:25 2013 From: tjarvstrand@REDACTED (=?ISO-8859-1?Q?Thomas_J=E4rvstrand?=) Date: Fri, 26 Apr 2013 14:34:25 +0200 Subject: [erlang-questions] Request for comment: A proposal to introduce the concept of function domains In-Reply-To: References: Message-ID: Fair point. How about calling it visibility and using global, local and restricted as the predefined levels? T 2013/4/25 Chris King > On Thu, Apr 25, 2013 at 3:21 AM, Thomas J?rvstrand > wrote: > > I write this email to get the community's opinion on an idea to extend > > Erlang with the concept of function domains. The intention is for the end > > result to be submitted as an EEP. > > "Function domain" already has a well-known mathematical definition: > http://en.wikipedia.org/wiki/Domain_of_a_function > > I would instead call this "visibility" or "access level" or some such. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Fri Apr 26 14:57:03 2013 From: lukas@REDACTED (Lukas Larsson) Date: Fri, 26 Apr 2013 14:57:03 +0200 Subject: [erlang-questions] Request for comment: A proposal to introduce the concept of function domains In-Reply-To: References: Message-ID: Hello Thomas, I agree that there is a need for better encapsulating possibilities in Erlang. However in order to allow calls only within the current application you have to introduce the concept of an application into the language and not only in OTP. Personally I think having language support for a collection of modules (like packages in Java, or namespace in C++) is a necessity for the language to grow and mature. I do however not know what that would look like and what properties that would bring. Have you thought anything about this? I also do not really understand the purpose of private. How does it differ from not-exported? Is it an exported function which can only be called from a specific behaviour module? Lukas On Thu, Apr 25, 2013 at 9:21 AM, Thomas J?rvstrand wrote: > Hi all, > > I write this email to get the community's opinion on an idea to extend > Erlang with the concept of function domains. The intention is for the end > result to be submitted as an EEP. > > The rationale is that encapsulation is a good thing and that code should > not be allowed to depend on library functions that the library's author did > not intend to expose to the outside world. Because of this I would like to > introduce the idea of exporting functions into different domains. > > There will be three predefined function domains: one will allow the > function to be called from anywhere, one will allow the function to be > called from within its own application and one to disallow the function > from being explicitly referenced anywhere outside its module (this is for > behaviour callbacks, functions passed/returned as funs etc.) > > To allow for extension in the future, compilation will allow any atom to > be given as the domain name (warnings could be added for non-predefined > domains), but xref will only be extended with processing of the predefined > domains. > > Suggestions for what to call the predefined domains are: > > *public, restricted, private* > The rationale is that due to how the language works, a domain-declaration > will only specify where we allow the function to be referenced with its > fully qualified name we can't detect other any other references anyway. > * > external, internal, restricted* > The rationale is that functions in the private domain are not really > private at all since they can be called from anywhere (eg. the handle_call > of a gen_server will be called from the gen_server-module). Private the > becomes restricted, because that's what it really is and we use the duality > of > external/internal for allowing calls from outside/inside the application. > The main issue with this suggestion is that many people associate internal > with non-exported functions. > > I have two suggestions for how the domains should be declared in a module, > they are both attributes that take two arguments: the domain and a list of > /: > > *The domain/2 attribute: > * > Pros: Backwards compatibilty > Cons: "Clutters" the attribute namespace. Requires information to be duplicated > in the module (need both export and domain) > > *The export/2 attribute: > * > Pros: Avoids cluttering the attribute namespace and avoids duplicated > information in the code. > Cons: Breaks backwards compatibility with earlier OTP releases for code written > using the new attribute. > > I would appreciate some input on this, both from the community and the OTP > -team. > > Regards > Thomas J?rvstrand > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From scott@REDACTED Fri Apr 26 19:00:35 2013 From: scott@REDACTED (Scott Thoman) Date: Fri, 26 Apr 2013 13:00:35 -0400 Subject: [erlang-questions] Automatically reconnecting nodes when they come back online Message-ID: To all who know more about this than I do: First, I'm just beginning to learn about Erlang/OTP so I figured I'd use to implement something useful. Part of what I'd like to build will involve a "conductor" controller node that directs some other "player" nodes to all do something at approximately the same time - ultimately to actually test the operation of another piece of distributed software. As part of those operations, I expect the player nodes may sometimes crash (actually cause a Windows BSOD in some cases) and then eventually come back to life. What I'm wondering about is what some folks have found to be good ways of getting nodes to rejoin the cluster when they come back to life. They way I'm thinking about it now, is that the player nodes will be passive in the sense that they won't actively connect to any other nodes - they'll only get connected when the conductor node invites them in. I'm also not looking for fault tolerance on the conductor node at this point; if that one fails badly I'll just get some coffee and rerun the scenario again. My first two thoughts were: 1. When the conductor node connects up the player nodes it would also spawn a process whose sole job is to periodically ping the other nodes to ensure they're connected. Then when one goes down, those pings will just fail during that time but when the node comes back a ping will reconnect it to the other nodes. All this time, I'd be monitoring the node up/down messages. 2. I'd start by monitoring all the nodes as the conductor connects them and when receiving a node down message, spawn a process whose job it is to periodically ping only that node only until it comes back. Are there some good practices out there for systems that want to behave like this? Thanks in advance, /stt From norton@REDACTED Fri Apr 26 19:13:11 2013 From: norton@REDACTED (Joseph Wayne Norton) Date: Sat, 27 Apr 2013 02:13:11 +0900 Subject: [erlang-questions] Automatically reconnecting nodes when they come back online In-Reply-To: References: Message-ID: I don't have a direct answer to your question. However, are you aware of the slave module? Some of the recipe(s) in this module might be of use to you. https://github.com/norton/qc/blob/master/src/qc_slave.erl On 2013/04/27, at 2:00, Scott Thoman wrote: > To all who know more about this than I do: > > First, I'm just beginning to learn about Erlang/OTP so I figured I'd > use to implement something useful. > > Part of what I'd like to build will involve a "conductor" controller > node that directs some other "player" nodes to all do something at > approximately the same time - ultimately to actually test the > operation of another piece of distributed software. As part of those > operations, I expect the player nodes may sometimes crash (actually > cause a Windows BSOD in some cases) and then eventually come back to > life. > > What I'm wondering about is what some folks have found to be good ways > of getting nodes to rejoin the cluster when they come back to life. > They way I'm thinking about it now, is that the player nodes will be > passive in the sense that they won't actively connect to any other > nodes - they'll only get connected when the conductor node invites > them in. I'm also not looking for fault tolerance on the conductor > node at this point; if that one fails badly I'll just get some coffee > and rerun the scenario again. > > My first two thoughts were: > 1. When the conductor node connects up the player nodes it would also > spawn a process whose sole job is to periodically ping the other nodes > to ensure they're connected. Then when one goes down, those pings > will just fail during that time but when the node comes back a ping > will reconnect it to the other nodes. All this time, I'd be > monitoring the node up/down messages. > 2. I'd start by monitoring all the nodes as the conductor connects > them and when receiving a node down message, spawn a process whose job > it is to periodically ping only that node only until it comes back. > > Are there some good practices out there for systems that want to > behave like this? > > Thanks in advance, > > /stt > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From scott@REDACTED Fri Apr 26 19:35:47 2013 From: scott@REDACTED (Scott Thoman) Date: Fri, 26 Apr 2013 13:35:47 -0400 Subject: [erlang-questions] Automatically reconnecting nodes when they come back online In-Reply-To: References: Message-ID: On Fri, Apr 26, 2013 at 1:13 PM, Joseph Wayne Norton wrote: > I don't have a direct answer to your question. > > However, are you aware of the slave module? > > Some of the recipe(s) in this module might be of use to you. > > https://github.com/norton/qc/blob/master/src/qc_slave.erl > > On 2013/04/27, at 2:00, Scott Thoman wrote: > > To all who know more about this than I do: > > First, I'm just beginning to learn about Erlang/OTP so I figured I'd > use to implement something useful. > > Part of what I'd like to build will involve a "conductor" controller > node that directs some other "player" nodes to all do something at > approximately the same time - ultimately to actually test the > operation of another piece of distributed software. As part of those > operations, I expect the player nodes may sometimes crash (actually > cause a Windows BSOD in some cases) and then eventually come back to > life. > > What I'm wondering about is what some folks have found to be good ways > of getting nodes to rejoin the cluster when they come back to life. > They way I'm thinking about it now, is that the player nodes will be > passive in the sense that they won't actively connect to any other > nodes - they'll only get connected when the conductor node invites > them in. I'm also not looking for fault tolerance on the conductor > node at this point; if that one fails badly I'll just get some coffee > and rerun the scenario again. > > My first two thoughts were: > 1. When the conductor node connects up the player nodes it would also > spawn a process whose sole job is to periodically ping the other nodes > to ensure they're connected. Then when one goes down, those pings > will just fail during that time but when the node comes back a ping > will reconnect it to the other nodes. All this time, I'd be > monitoring the node up/down messages. > 2. I'd start by monitoring all the nodes as the conductor connects > them and when receiving a node down message, spawn a process whose job > it is to periodically ping only that node only until it comes back. > > Are there some good practices out there for systems that want to > behave like this? > > Thanks in advance, > > /stt > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions I'm not aware of it yet but I'll take a look... Thanks, /stt From robert.virding@REDACTED Fri Apr 26 19:38:25 2013 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 26 Apr 2013 18:38:25 +0100 (BST) Subject: [erlang-questions] Request for comment: A proposal to introduce the concept of function domains In-Reply-To: Message-ID: <1035009944.84210439.1366997905645.JavaMail.root@erlang-solutions.com> There is already an EEP which touches on this, http://www.erlang.org/eeps/eep-0005.html Robert ----- Original Message ----- > From: "Lukas Larsson" > To: "Thomas J?rvstrand" > Cc: "Erlang" > Sent: Friday, 26 April, 2013 8:57:03 AM > Subject: Re: [erlang-questions] Request for comment: A proposal to > introduce the concept of function domains > Hello Thomas, > I agree that there is a need for better encapsulating possibilities > in Erlang. However in order to allow calls only within the current > application you have to introduce the concept of an application into > the language and not only in OTP. Personally I think having language > support for a collection of modules (like packages in Java, or > namespace in C++) is a necessity for the language to grow and > mature. I do however not know what that would look like and what > properties that would bring. Have you thought anything about this? > I also do not really understand the purpose of private. How does it > differ from not-exported? Is it an exported function which can only > be called from a specific behaviour module? > Lukas > On Thu, Apr 25, 2013 at 9:21 AM, Thomas J?rvstrand < > tjarvstrand@REDACTED > wrote: > > Hi all, > > > I write this email to get the community's opinion on an idea to > > extend Erlang with the concept of function domains. The intention > > is > > for the end result to be submitted as an EEP. > > > The rationale is that encapsulation is a good thing and that code > > should not be allowed to depend on library functions that the > > library's author did not intend to expose to the outside world. > > Because of this I would like to introduce the idea of exporting > > functions into different domains. > > > There will be three predefined function domains: one will allow the > > function to be called from anywhere, one will allow the function to > > be called from within its own application and one to disallow the > > function from being explicitly referenced anywhere outside its > > module (this is for behaviour callbacks, functions passed/returned > > as funs etc.) > > > To allow for extension in the future, compilation will allow any > > atom > > to be given as the domain name (warnings could be added for > > non-predefined domains), but xref will only be extended with > > processing of the predefined domains. > > > Suggestions for what to call the predefined domains are: > > > public, restricted, private > > > The rationale is that due to how the language works, a > > domain-declaration will only specify where we allow the function to > > be referenced with its fully qualified name we can't detect other > > any other references anyway. > > > external, internal, restricted > > > The rationale is that functions in the private domain are not > > really > > private at all since they can be called from anywhere (eg. the > > handle_call of a gen_server will be called from the > > gen_server-module). Private the becomes restricted, because that's > > what it really is and we use the duality of > > > external/internal for allowing calls from outside/inside the > > application. The main issue with this suggestion is that many > > people > > associate internal with non-exported functions. > > > I have two suggestions for how the domains should be declared in a > > module, they are both attributes that take two arguments: the > > domain > > and a list of /: > > > The d omain/2 attribute : > > > Pros : Backwards comp atibilty > > > Cons: "Clutters" the attribute namespace. Requires information t o > > be > > duplicated in the module ( need both export and domain ) > > > Th e export/2 attribute: > > > Pros: Avoids cluttering the attribute namespace and avo ids d upl > > icated information in the code. > > > Cons: Brea ks b ackwards compatibility with earl ier OTP releases > > for > > code w ritten using the new attribute. > > > I would app reciate some input on this, both from the comm unity > > and > > the OTP -team. > > > Re gards > > > Thomas J?rvstrand > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From scott@REDACTED Fri Apr 26 21:55:34 2013 From: scott@REDACTED (Scott Thoman) Date: Fri, 26 Apr 2013 15:55:34 -0400 Subject: [erlang-questions] Automatically reconnecting nodes when they come back online In-Reply-To: References: Message-ID: It looks like the slave thing won't quite work in my case since I'll likely be in a heterogeneous environment where the controller is linux but, unfortunately, the machines-under-test will be Windows. I will keep that in mind, though, if I need that functionality now that I know it exists. :) /stt From maruthavanan_s@REDACTED Fri Apr 26 21:58:27 2013 From: maruthavanan_s@REDACTED (Maruthavanan Subbarayan) Date: Fri, 26 Apr 2013 15:58:27 -0400 Subject: [erlang-questions] Comparing tuple records in MatchSpec Message-ID: Hi, i have mnesia table which stores date in tuple format {Y,M,D}, To fetch data that falls on given date frames I tried to use this record MatchSpec = [{#appointment{id='$1', starttime = '$2', endtime = '$3', _='_'}, [{'orelse',{'>=','$2',StartTime},{'orelse',{'=<','$3',EndTime}}}], ['*']}] where Startime and Endtime are records. This throws and bad arg during transaction. But when I change the Startime and End time as integer or list it executes without any problem. Is this the normal behaviour, or am I missing out anything, Any work around to achieve this? Thanks,Marutha -------------- next part -------------- An HTML attachment was scrubbed... URL: From anders.nygren@REDACTED Fri Apr 26 23:22:34 2013 From: anders.nygren@REDACTED (Anders Nygren) Date: Fri, 26 Apr 2013 22:22:34 +0100 Subject: [erlang-questions] Comparing tuple records in MatchSpec In-Reply-To: References: Message-ID: Hi, I don't do this often enough to remember the details so I normally cheat and use ets:fun2ms. There are some strange things with tuples having to be in a tuple in the match spec that always trips me up. /Anders On Fri, Apr 26, 2013 at 8:58 PM, Maruthavanan Subbarayan < maruthavanan_s@REDACTED> wrote: > Hi, > > i have mnesia table which stores date in tuple format {Y,M,D}, > > To fetch data that falls on given date frames I tried to use this record > > MatchSpec = [{#appointment{id='$1', starttime = '$2', endtime = '$3', > _='_'}, > > [{'orelse',{'>=','$2',StartTime},{'orelse',{'=<','$3',EndTime}}}], > ['*']}] > > where Startime and Endtime are records. This throws and bad arg during > transaction. > > But when I change the Startime and End time as integer or list it executes > without any problem. > > Is this the normal behaviour, or am I missing out anything, Any work > around to achieve this? > > Thanks, > Marutha > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tilman.holschuh@REDACTED Sat Apr 27 00:02:31 2013 From: tilman.holschuh@REDACTED (Tilman Holschuh) Date: Fri, 26 Apr 2013 15:02:31 -0700 Subject: [erlang-questions] [ANN] jerg - JSON Schema to Erlang Records Generator Message-ID: <517AF977.6050204@gmail.com> Dear list, On behalf of David Dossot I'd like to announce: jerg, a JSON Schema to Erlang Records Generator You can find jerg on github: https://github.com/ddossot/jerg David also wrote a blog on jerg: http://blog.dossot.net/2013/04/meet-jerg-json-schema-to-erlang-records.html Feedback is more than welcome. Cheers - Tilman From yogishb@REDACTED Sat Apr 27 00:35:59 2013 From: yogishb@REDACTED (Yogish Baliga) Date: Fri, 26 Apr 2013 15:35:59 -0700 Subject: [erlang-questions] Comparing tuple records in MatchSpec In-Reply-To: References: Message-ID: <1EB0740B-B970-4C44-8734-32678659D09B@yahoo.com> Why not store unix time stamp instead? On Apr 26, 2013, at 12:58, Maruthavanan Subbarayan wrote: > Hi, > > i have mnesia table which stores date in tuple format {Y,M,D}, > > To fetch data that falls on given date frames I tried to use this record > > MatchSpec = [{#appointment{id='$1', starttime = '$2', endtime = '$3', _='_'}, > [{'orelse',{'>=','$2',StartTime},{'orelse',{'=<','$3',EndTime}}}], > ['*']}] > > where Startime and Endtime are records. This throws and bad arg during transaction. > > But when I change the Startime and End time as integer or list it executes without any problem. > > Is this the normal behaviour, or am I missing out anything, Any work around to achieve this? > > Thanks, > Marutha > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From fritchie@REDACTED Sat Apr 27 04:20:32 2013 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Fri, 26 Apr 2013 21:20:32 -0500 Subject: [erlang-questions] Schedulers getting "stuck", part II Message-ID: <96629.1367029232@snookles.snookles.com> Howdy. This is a followup to the discussion that took place on this list in October 2012, see: http://erlang.org/pipermail/erlang-questions/2012-October/069503.html (first message only, I dunno why) http://erlang.org/pipermail/erlang-questions/2012-October/069585.html (the rest of the thread) I've been trying to figure out how to introduce the stuff that I've written at: https://github.com/slfritchie/nifwait/tree/md5#readme ... but I still can't decide. So I'll try for something short and un-Scott-like. For the long story, please read the README in the URL above. As for the short story, I believe a couple of things: * R15B0x's schedulers are broken: Basho seen "stuck" schedulers in one of our apps with no custom NIF code. And it's possible to get them stuck using only the 'crypto' module's MD5 functions. * R16B's schedulers appear to be even more broken: I have a mostly-deterministic case that demonstrates schedulers that go to sleep and do not wake for minutes (or hours) when there is plenty of work to do. This also is using only the 'crypto' module and does not require custom NIF code. Discuss. :-) -Scott From mjtruog@REDACTED Sat Apr 27 05:09:33 2013 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 26 Apr 2013 20:09:33 -0700 Subject: [erlang-questions] Schedulers getting "stuck", part II In-Reply-To: <96629.1367029232@snookles.snookles.com> References: <96629.1367029232@snookles.snookles.com> Message-ID: <517B416D.1030902@gmail.com> On 04/26/2013 07:20 PM, Scott Lystig Fritchie wrote: > Howdy. This is a followup to the discussion that took place on this > list in October 2012, see: > > http://erlang.org/pipermail/erlang-questions/2012-October/069503.html > (first message only, I dunno why) > http://erlang.org/pipermail/erlang-questions/2012-October/069585.html > (the rest of the thread) > > I've been trying to figure out how to introduce the stuff that I've > written at: > > https://github.com/slfritchie/nifwait/tree/md5#readme > > ... but I still can't decide. So I'll try for something short and > un-Scott-like. For the long story, please read the README in the URL > above. > > As for the short story, I believe a couple of things: > > * R15B0x's schedulers are broken: Basho seen "stuck" schedulers in one of > our apps with no custom NIF code. And it's possible to get them stuck > using only the 'crypto' module's MD5 functions. > > * R16B's schedulers appear to be even more broken: I have a > mostly-deterministic case that demonstrates schedulers that go to > sleep and do not wake for minutes (or hours) when there is plenty of > work to do. This also is using only the 'crypto' module and does not > require custom NIF code. > > Discuss. :-) > > -Scott > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > I would expect that some useful numbers to understand the problem of crypto's impact on the scheduler would be the min, max, mean, and stddev of the time spent computing the MD5 within the NIF. Based on previous complaints about long-running NIF code, this problem should be more likely with higher latency finding the MD5 (so slower computers). I have not looked at the scheduling criteria in the recent source code, but I would assume that this behaviour is caused by some abnormality in the latency within the MD5 NIF function (where either the mean or stddev is high enough to cause this behaviour). With the min, max, mean and stddev (found with a custom NIF) you could probably model the behaviour with a function that does a quick sleep (which would provide the OTP team with test cases for development). I don't have data or code to back this up, but I think the approach would be helpful and the data would help determine the proper use of NIFs. I believe that an older version of the crypto driver from R12 or R13 had code for using the async thread pool (it may not have been turned on, I remember it being conditional), such that this behaviour should not occur. So, testing erlang md5 usage in R12 or R13 should confirm that this is a problem specific to the scheduling of NIF code. - Michael From max.lapshin@REDACTED Sat Apr 27 10:58:33 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 27 Apr 2013 12:58:33 +0400 Subject: [erlang-questions] disksup told me to delete all files =) Message-ID: Beware of documented disksup behaviour. There is a file cleaner in flussonic which asks disksup disk usage. If it is bigger than threshold, oldest file is deleted. But disksup doesn't refresh its cache! So flussonic has deleted all files on disk. I have two options: 1) send timeout message to disksup before asking disk usage This option is very easy, but I don't understand why disksup is so complicated. Perhaps I don't know some problems under it? 2) don't use disksup, launch os:cmd("df -h") and parse it. I need to launch it about 100 times per hour, so this expensive fork/exec is not very expensive. Which way is better? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rapsey@REDACTED Sat Apr 27 11:16:58 2013 From: rapsey@REDACTED (Rapsey) Date: Sat, 27 Apr 2013 11:16:58 +0200 Subject: [erlang-questions] Schedulers getting "stuck", part II In-Reply-To: <517B416D.1030902@gmail.com> References: <96629.1367029232@snookles.snookles.com> <517B416D.1030902@gmail.com> Message-ID: Our code uses a lot of small NIF calls. The code basically does some memcpy and ends with a write to a non blocking file descriptor. It can get called thousands of times per second. We moved a few servers to R16. The CPU usage is noticeably lower than R14, but worryingly so. At first we had a lot of problems with schedulers just stop working for up to a minute. Even the console was unresponsive. I saw basho added erlang:bump_reductions calls to their NIF calls and we added that. It did improve the situation but small blocks still seem to happen. Sergej On Sat, Apr 27, 2013 at 5:09 AM, Michael Truog wrote: > > On 04/26/2013 07:20 PM, Scott Lystig Fritchie wrote: > > Howdy. This is a followup to the discussion that took place on this > > list in October 2012, see: > > > > http://erlang.org/pipermail/erlang-questions/2012-October/069503.html > > (first message only, I dunno why) > > http://erlang.org/pipermail/erlang-questions/2012-October/069585.html > > (the rest of the thread) > > > > I've been trying to figure out how to introduce the stuff that I've > > written at: > > > > https://github.com/slfritchie/nifwait/tree/md5#readme > > > > ... but I still can't decide. So I'll try for something short and > > un-Scott-like. For the long story, please read the README in the URL > > above. > > > > As for the short story, I believe a couple of things: > > > > * R15B0x's schedulers are broken: Basho seen "stuck" schedulers in one of > > our apps with no custom NIF code. And it's possible to get them stuck > > using only the 'crypto' module's MD5 functions. > > > > * R16B's schedulers appear to be even more broken: I have a > > mostly-deterministic case that demonstrates schedulers that go to > > sleep and do not wake for minutes (or hours) when there is plenty of > > work to do. This also is using only the 'crypto' module and does not > > require custom NIF code. > > > > Discuss. :-) > > > > -Scott > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > I would expect that some useful numbers to understand the problem of crypto's impact on the scheduler would be the min, max, mean, and stddev of the time spent computing the MD5 within the NIF. Based on previous complaints about long-running NIF code, this problem should be more likely with higher latency finding the MD5 (so slower computers). I have not looked at the scheduling criteria in the recent source code, but I would assume that this behaviour is caused by some abnormality in the latency within the MD5 NIF function (where either the mean or stddev is high enough to cause this behaviour). With the min, max, mean and stddev (found with a custom NIF) you could probably model the behaviour with a function that does a quick sleep (which would provide the OTP team with test cases for development). I don't have data or code to back this up, but I think the approach would be helpful and the data would help determine the proper use of NIFs. I believe that an older > version of the crypto driver from R12 or R13 had code for using the async thread pool (it may not have been turned on, I remember it being conditional), such that this behaviour should not occur. So, testing erlang md5 usage in R12 or R13 should confirm that this is a problem specific to the scheduling of NIF code. > > - Michael > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From rapsey@REDACTED Sat Apr 27 11:29:50 2013 From: rapsey@REDACTED (Rapsey) Date: Sat, 27 Apr 2013 11:29:50 +0200 Subject: [erlang-questions] [ANN] jerg - JSON Schema to Erlang Records Generator In-Reply-To: <517AF977.6050204@gmail.com> References: <517AF977.6050204@gmail.com> Message-ID: When it comes to web development, my experience with records as the data structure behind an API was not positive. We used records a lot at the beginning, but their use declined with every new project. Erlangs biggest advantage in web development is flexibility. Records are inherently inflexible. If you use records across multiple modules, you make code updates that change your data structure way more problematic. One module knows a record to be one thing, another knows it to be another and everything crashes until all modules are updated. In web development those code updates are quite frequent, since most of what a website backend does is move data around. Sergej On Sat, Apr 27, 2013 at 12:02 AM, Tilman Holschuh wrote: > Dear list, > > On behalf of David Dossot I'd like to announce: > > jerg, a JSON Schema to Erlang Records Generator > > You can find jerg on github: > > https://github.com/ddossot/jerg > > David also wrote a blog on jerg: > > http://blog.dossot.net/2013/04/meet-jerg-json-schema-to-erlang-records.html > > Feedback is more than welcome. > > Cheers > - Tilman > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ivan@REDACTED Sat Apr 27 12:10:50 2013 From: ivan@REDACTED (Ivan Uemlianin) Date: Sat, 27 Apr 2013 11:10:50 +0100 Subject: [erlang-questions] [ANN] jerg - JSON Schema to Erlang Records Generator In-Reply-To: References: <517AF977.6050204@gmail.com> Message-ID: <84936DA3-07D2-4A8F-8B4D-CB1701FE6642@llaisdy.com> I'm hanging around on twitter too much. I tried to favourite this. -- festina lente On 27 Apr 2013, at 10:29, Rapsey wrote: > When it comes to web development, my experience with records as the > data structure behind an API was not positive. We used records a lot > at the beginning, but their use declined with every new project. > Erlangs biggest advantage in web development is flexibility. Records > are inherently inflexible. If you use records across multiple modules, > you make code updates that change your data structure way more > problematic. One module knows a record to be one thing, another knows > it to be another and everything crashes until all modules are updated. > In web development those code updates are quite frequent, since most > of what a website backend does is move data around. > > > Sergej > > On Sat, Apr 27, 2013 at 12:02 AM, Tilman Holschuh > wrote: >> Dear list, >> >> On behalf of David Dossot I'd like to announce: >> >> jerg, a JSON Schema to Erlang Records Generator >> >> You can find jerg on github: >> >> https://github.com/ddossot/jerg >> >> David also wrote a blog on jerg: >> >> http://blog.dossot.net/2013/04/meet-jerg-json-schema-to-erlang-records.html >> >> Feedback is more than welcome. >> >> Cheers >> - Tilman >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From tjarvstrand@REDACTED Sat Apr 27 13:07:43 2013 From: tjarvstrand@REDACTED (=?ISO-8859-1?Q?Thomas_J=E4rvstrand?=) Date: Sat, 27 Apr 2013 13:07:43 +0200 Subject: [erlang-questions] Request for comment: A proposal to introduce the concept of function domains In-Reply-To: <1035009944.84210439.1366997905645.JavaMail.root@erlang-solutions.com> References: <1035009944.84210439.1366997905645.JavaMail.root@erlang-solutions.com> Message-ID: @Lukas: There is actually no need need to introduce the concept of applications into the language itself, merely support for the notion of different types of exports. Interpreting the meaning of these exports can be left to xref, which already has support for applications. The point of the private type of export is for functions that have to be externally callable, but that should never be explicitly referenced. Examples of this are behaviour callback functions (such as the handle_call etc. of a gen_server) and functions that are passed to or returned from functions calls (such as spawn). Ie, they are functions that may only be implicitly referenced (ie. in the calling code, either the module, the function name or both would have to be a variable). Basically this means that any calls to private functions that xref is able to detect are illegal. @Robert Yes, I'm aware of that EEP. It has similar aims and looks cosmetically similar but the suggestion to limit access on a module level would make things awfully messy IMHO. With EEP 5, you have to clutter your code with individual exports for every single caller/callee-pair inside your application. If you want to call a new function in your module, you would now have to edit two files instead of one. A more serious flaw though is that if you're writing a utility library that is to be used by others, you have no idea who will be calling your module, and so you cannot export your functions to them. Thomas 2013/4/26 Robert Virding > There is already an EEP which touches on this, > http://www.erlang.org/eeps/eep-0005.html > > Robert > > ------------------------------ > > *From: *"Lukas Larsson" > *To: *"Thomas J?rvstrand" > *Cc: *"Erlang" > *Sent: *Friday, 26 April, 2013 8:57:03 AM > *Subject: *Re: [erlang-questions] Request for comment: A proposal to > introduce the concept of function domains > > > Hello Thomas, > > I agree that there is a need for better encapsulating possibilities in > Erlang. However in order to allow calls only within the current application > you have to introduce the concept of an application into the language and > not only in OTP. Personally I think having language support for a > collection of modules (like packages in Java, or namespace in C++) is a > necessity for the language to grow and mature. I do however not know what > that would look like and what properties that would bring. Have you thought > anything about this? > > I also do not really understand the purpose of private. How does it differ > from not-exported? Is it an exported function which can only be called from > a specific behaviour module? > > Lukas > > > On Thu, Apr 25, 2013 at 9:21 AM, Thomas J?rvstrand wrote: > >> Hi all, >> >> I write this email to get the community's opinion on an idea to extend >> Erlang with the concept of function domains. The intention is for the end >> result to be submitted as an EEP. >> >> The rationale is that encapsulation is a good thing and that code should >> not be allowed to depend on library functions that the library's author did >> not intend to expose to the outside world. Because of this I would like to >> introduce the idea of exporting functions into different domains. >> >> There will be three predefined function domains: one will allow the >> function to be called from anywhere, one will allow the function to be >> called from within its own application and one to disallow the function >> from being explicitly referenced anywhere outside its module (this is for >> behaviour callbacks, functions passed/returned as funs etc.) >> >> To allow for extension in the future, compilation will allow any atom to >> be given as the domain name (warnings could be added for non-predefined >> domains), but xref will only be extended with processing of the predefined >> domains. >> >> Suggestions for what to call the predefined domains are: >> >> *public, restricted, private* >> The rationale is that due to how the language works, a domain-declaration >> will only specify where we allow the function to be referenced with its >> fully qualified name we can't detect other any other references anyway. >> * >> external, internal, restricted* >> The rationale is that functions in the private domain are not really >> private at all since they can be called from anywhere (eg. the handle_call >> of a gen_server will be called from the gen_server-module). Private the >> becomes restricted, because that's what it really is and we use the duality >> of >> external/internal for allowing calls from outside/inside the application. >> The main issue with this suggestion is that many people associate internal >> with non-exported functions. >> >> I have two suggestions for how the domains should be declared in a >> module, they are both attributes that take two arguments: the domain and a >> list of /: >> >> *The domain/2 attribute: >> * >> Pros: Backwards compatibilty >> Cons: "Clutters" the attribute namespace. Requires information to be duplicated >> in the module (need both export and domain) >> >> *The export/2 attribute: >> * >> Pros: Avoids cluttering the attribute namespace and avoids duplicated >> information in the code. >> Cons: Breaks backwards compatibility with earlier OTP releases for code written >> using the new attribute. >> >> I would appreciate some input on this, both from the community and the >> OTP-team. >> >> Regards >> Thomas J?rvstrand >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjarvstrand@REDACTED Sat Apr 27 13:11:34 2013 From: tjarvstrand@REDACTED (=?ISO-8859-1?Q?Thomas_J=E4rvstrand?=) Date: Sat, 27 Apr 2013 13:11:34 +0200 Subject: [erlang-questions] Request for comment: A proposal to introduce the concept of function domains In-Reply-To: References: <1035009944.84210439.1366997905645.JavaMail.root@erlang-solutions.com> Message-ID: Correction: private functions may be of course be referenced from inside their own module. Thomas 2013/4/27 Thomas J?rvstrand > @Lukas: > There is actually no need need to introduce the concept of applications > into the language itself, merely support for the notion of different types > of exports. Interpreting the meaning of these exports can be left to xref, > which already has support for applications. > > The point of the private type of export is for functions that have to be > externally callable, but that should never be explicitly referenced. > Examples of this are behaviour callback functions (such as the handle_call > etc. of a gen_server) and functions that are passed to or returned from > functions calls (such as spawn). Ie, they are functions that may only be > implicitly referenced (ie. in the calling code, either the module, the > function name or both would have to be a variable). Basically this means > that any calls to private functions that xref is able to detect are illegal. > > @Robert > Yes, I'm aware of that EEP. It has similar aims and looks cosmetically > similar but the suggestion to limit access on a module level would make > things awfully messy IMHO. With EEP 5, you have to clutter your code with > individual exports for every single caller/callee-pair inside your > application. If you want to call a new function in your module, you would > now have to edit two files instead of one. A more serious flaw though is > that if you're writing a utility library that is to be used by others, you > have no idea who will be calling your module, and so you cannot export your > functions to them. > > Thomas > > > 2013/4/26 Robert Virding > >> There is already an EEP which touches on this, >> http://www.erlang.org/eeps/eep-0005.html >> >> Robert >> >> ------------------------------ >> >> *From: *"Lukas Larsson" >> *To: *"Thomas J?rvstrand" >> *Cc: *"Erlang" >> *Sent: *Friday, 26 April, 2013 8:57:03 AM >> *Subject: *Re: [erlang-questions] Request for comment: A proposal to >> introduce the concept of function domains >> >> >> Hello Thomas, >> >> I agree that there is a need for better encapsulating possibilities in >> Erlang. However in order to allow calls only within the current application >> you have to introduce the concept of an application into the language and >> not only in OTP. Personally I think having language support for a >> collection of modules (like packages in Java, or namespace in C++) is a >> necessity for the language to grow and mature. I do however not know what >> that would look like and what properties that would bring. Have you thought >> anything about this? >> >> I also do not really understand the purpose of private. How does it >> differ from not-exported? Is it an exported function which can only be >> called from a specific behaviour module? >> >> Lukas >> >> >> On Thu, Apr 25, 2013 at 9:21 AM, Thomas J?rvstrand > > wrote: >> >>> Hi all, >>> >>> I write this email to get the community's opinion on an idea to extend >>> Erlang with the concept of function domains. The intention is for the end >>> result to be submitted as an EEP. >>> >>> The rationale is that encapsulation is a good thing and that code should >>> not be allowed to depend on library functions that the library's author did >>> not intend to expose to the outside world. Because of this I would like to >>> introduce the idea of exporting functions into different domains. >>> >>> There will be three predefined function domains: one will allow the >>> function to be called from anywhere, one will allow the function to be >>> called from within its own application and one to disallow the function >>> from being explicitly referenced anywhere outside its module (this is for >>> behaviour callbacks, functions passed/returned as funs etc.) >>> >>> To allow for extension in the future, compilation will allow any atom to >>> be given as the domain name (warnings could be added for non-predefined >>> domains), but xref will only be extended with processing of the predefined >>> domains. >>> >>> Suggestions for what to call the predefined domains are: >>> >>> *public, restricted, private* >>> The rationale is that due to how the language works, a >>> domain-declaration will only specify where we allow the function to be >>> referenced with its fully qualified name we can't detect other any other >>> references anyway. >>> * >>> external, internal, restricted* >>> The rationale is that functions in the private domain are not really >>> private at all since they can be called from anywhere (eg. the handle_call >>> of a gen_server will be called from the gen_server-module). Private the >>> becomes restricted, because that's what it really is and we use the duality >>> of >>> external/internal for allowing calls from outside/inside the >>> application. The main issue with this suggestion is that many people >>> associate internal with non-exported functions. >>> >>> I have two suggestions for how the domains should be declared in a >>> module, they are both attributes that take two arguments: the domain and a >>> list of /: >>> >>> *The domain/2 attribute: >>> * >>> Pros: Backwards compatibilty >>> Cons: "Clutters" the attribute namespace. Requires information to be duplicated >>> in the module (need both export and domain) >>> >>> *The export/2 attribute: >>> * >>> Pros: Avoids cluttering the attribute namespace and avoids duplicated >>> information in the code. >>> Cons: Breaks backwards compatibility with earlier OTP releases for code >>> written using the new attribute. >>> >>> I would appreciate some input on this, both from the community and the >>> OTP-team. >>> >>> Regards >>> Thomas J?rvstrand >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Sat Apr 27 16:14:56 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Sat, 27 Apr 2013 16:14:56 +0200 Subject: [erlang-questions] Comparing tuple records in MatchSpec In-Reply-To: References: Message-ID: Hello Maruthavanan, You need to escape the tuples StartTime and EndTime as tuples have a special meaning in match specifications. The documentation states[1]: > Literals in theMatchCondition/MatchBody can either be written as is, which works for all literals except tuples, or by using the special form {const, T}, where T is any Erlang term. For tuple literals in the match_spec, one can also use double tuple parentheses, i.e., construct them as a tuple of arity one containing a single tuple, which is the one to be constructed. The "double tuple parenthesis" syntax is useful to construct tuples from already bound variables, like in {{'$1', [a,b,'$2']}}. So do either {StartTime} or {const,StartTime}. But as Anders said, using ets:fun2ms/1 would be more readable. Regards, [1] http://www.erlang.org/doc/apps/erts/match_spec.html -- Anthony Ramine Le 26 avr. 2013 ? 21:58, Maruthavanan Subbarayan a ?crit : > Hi, > > i have mnesia table which stores date in tuple format {Y,M,D}, > > To fetch data that falls on given date frames I tried to use this record > > MatchSpec = [{#appointment{id='$1', starttime = '$2', endtime = '$3', _='_'}, > [{'orelse',{'>=','$2',StartTime},{'orelse',{'=<','$3',EndTime}}}], > ['*']}] > > where Startime and Endtime are records. This throws and bad arg during transaction. > > But when I change the Startime and End time as integer or list it executes without any problem. > > Is this the normal behaviour, or am I missing out anything, Any work around to achieve this? > > Thanks, > Marutha > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From robert.virding@REDACTED Sat Apr 27 16:54:25 2013 From: robert.virding@REDACTED (Robert Virding) Date: Sat, 27 Apr 2013 15:54:25 +0100 (BST) Subject: [erlang-questions] Erlounge in/around Tallahassee next week? In-Reply-To: <1798427933.67452191.1366147574990.JavaMail.root@erlang-solutions.com> Message-ID: <1767901044.85767696.1367074465636.JavaMail.root@erlang-solutions.com> A big thanks to those who could make it I enjoyed myself. Off to Washington today. Robert ----- Original Message ----- > From: "Robert Virding" > To: "Erlang Questions" > Sent: Tuesday, 16 April, 2013 5:26:15 PM > Subject: Re: [erlang-questions] Erlounge in/around Tallahassee next week? > > Things are looking up, I have got some feedback on this so hopefully > we can arrange something. Any more interested? > > Robert > > ----- Original Message ----- > > From: "Robert Virding" > > To: "Erlang Questions" > > Sent: Monday, 15 April, 2013 5:26:18 PM > > Subject: Erlounge in/around Tallahassee next week? > > > > I will be in Tallahassee, Fl next week, 21-26/4, to hold a course. > > Are there any Erlangers in the area that feel up to an informal > > Erlounge in a suitable pub? Please let me know. > > > > Robert > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From robert.virding@REDACTED Sat Apr 27 16:57:13 2013 From: robert.virding@REDACTED (Robert Virding) Date: Sat, 27 Apr 2013 15:57:13 +0100 (BST) Subject: [erlang-questions] Erlounge in/around Washington DC next week In-Reply-To: <1312588249.85768141.1367074510127.JavaMail.root@erlang-solutions.com> Message-ID: <953851489.85771072.1367074633550.JavaMail.root@erlang-solutions.com> I will be in Washington DC next week, 28/4 - 3/5, to hold a course. Are there any Erlangers in the area that feel up to an informal Erlounge in a suitable pub? Please let me know. Sorry for sending out last minute request, I am JIT person. :-) Robert From tilman.holschuh@REDACTED Sat Apr 27 23:58:26 2013 From: tilman.holschuh@REDACTED (Tilman Holschuh) Date: Sat, 27 Apr 2013 14:58:26 -0700 Subject: [erlang-questions] [ANN] jerg - JSON Schema to Erlang Records Generator In-Reply-To: References: <517AF977.6050204@gmail.com> Message-ID: <517C4A02.3000604@gmail.com> On 13-04-27 02:29 AM, Rapsey wrote: > If you use records across multiple modules, ... Which you shouldn't. - Tilman From heinz@REDACTED Sun Apr 28 04:43:07 2013 From: heinz@REDACTED (Heinz Nikolaus Gies) Date: Sun, 28 Apr 2013 04:43:07 +0200 Subject: [erlang-questions] [ANN] Project FiFo webinar In-Reply-To: References: <7D480D7E-3166-49EF-8487-6395585CA82C@licenser.net> <2E7E06FE-D636-4F63-8D4A-F49419AE394B@licenser.net> Message-ID: As promised the recording of the webinar is online: http://www.youtube.com/watch?v=VHdrN4YrmoI Cheers, Heinz On Apr 23, 2013, at 2:10, Heinz Nikolaus Gies wrote: > Awesome! > > If you run into any trouble don't hesitate to poke me :) also feel free to drop by in our IRC channel #Project-FiFo (on free node) > > Cheers, > Heinz > On Apr 23, 2013, at 1:46, Tatsuya Kawano wrote: > >> Hi Heinz, >> >> Thank you. I just installed FiFo 0.3.4 and pyfi and played with them. >> They look good. >> >>> that is totally possible depending on how exactly your setup is there are >>> multiple ways which all hold different advantages and disadvantages: >> >> I'll probably choose 2) libsniffle for now, but try to move 3) >> HTTP/REST API in the future. >> >> Thanks, >> Tatsuya >> >> >> 2013/4/21 Heinz Nikolaus Gies : >>> Hi! >>> >>> that is totally possible depending on how exactly your setup is there are >>> multiple ways which all hold different advantages and disadvantages: >>> >>> 1) Use rpc / node tool from within the fifo zone just as sniffle-admin / >>> fifoadm does >>> * Pros: no login, full access to all functions, no layer between your script >>> and fifo >>> * Cons: no layer between your script and fifo, needs to talk directly to >>> fifo, no security, not well documented >>> 2) Use libsniffle[1] to call the create etc functions >>> * Pros: standard approach for internal applications to talk to sniffle, well >>> tested, documented[2], no login required >>> * Cons: requires mDNS discovery period, no security >>> 3) User the HTTP/REST API >>> * Pros: clean separation of your code and fifo, authentication, very well >>> tested and documented[3], most canonical way to talk to FiFo from an outside >>> app. >>> * Cons: No erlang implementation of a API client (you'd have to roll your >>> own), authentication required (not sure that is a cons) >>> 4) User pyfi[4][5](python HTTP API client library & console client) >>> * Pros: implements most of the API calls, working condition, clean >>> separation of client and server code >>> * Cons: Not erlang, no implementation of the zone console yet >>> >>> Cheers, >>> Heinz >>> >>> [1] https://github.com/project-fifo/libsniffle >>> [2] http://project-fifo.github.io/libsniffle >>> [3] http://project-fifo.net/display/PF/API >>> [4] https://github.com/project-fifo/pyfo >>> [5] https://pypi.python.org/pypi/PyFi >>> >>> On Apr 21, 2013, at 5:22, Tatsuya Kawano wrote: >>> >>> Hi Heinz, >>> >>> I'm working on a (by now) quite big open source Erlang project for cloud >>> management called Project FiFo and am going to hold an webinar regarding the >>> architecture of the whole thing. >>> >>> >>> Your project looks very interesting. >>> >>> Is there any way to make escripts to communicate FiFo's API server so >>> that I can create, monitor, stop and delete SmartOS zones from these >>> escripts? I'm developing an Erlang-based system and using SmartOS >>> zones to test it. I usually create six zones by vmadm command and >>> provision them with Chef, then deploy the system and run test suite, >>> and finally delete these zones by vmadm. I'm currently doing vmadm >>> part by hands but it will be great if I can escriptize them. >>> >>> Thanks, >>> >>> -- >>> Tatsuya Kawano (Mr.) >>> Tokyo, japan >>> >>> >>> 2013/4/19 Heinz Nikolaus Gies >>> >>> >>> Hi everyone, >>> I'm working on a (by now) quite big open source Erlang project for cloud >>> management called Project FiFo and am going to hold an webinar regarding the >>> architecture of the whole thing. I'll not deep dive into the erlang guts >>> since it would blow up the timeframe but it still might be interesting and >>> there is half a hour reserved for questions. >>> >>> The webinar is free so if this sounds interesting drop by! >>> >>> http://project-fifo.eventbrite.com/ >>> >>> Cheers, >>> Heinz > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From anishparameswaran@REDACTED Sun Apr 28 10:08:54 2013 From: anishparameswaran@REDACTED (Anish Parameswaran) Date: Sun, 28 Apr 2013 13:38:54 +0530 Subject: [erlang-questions] Undefined type ''Gauge32''? Is SNMPv2-SMI prebuilt into Erlang SNMP MIB Compiler ? Message-ID: Hi everyone, I am trying to compile an MIB file using erlang SNMP MIB compiler. However the following error is preventing the compilation MY-TRAP-MIB.mib: 29: Undefined type ''Gauge32'' The compilation command is erlc +'{group_check,false}' -I . MY-TRAP-MIB.mib The files in current folder MY-TRAP-MIB.mib OTP-MIB.bin OTP-REG.bin OTP-TC.bin The MY-TRAP-MIB.mib is a modified version from SNMP_Quick_Start The file MY-TRAP-MIB.mib is atatched. If i remove the testG32 (Guage32) object from the file , it compiles successfully. R16 Documentation tells that SNMPv2-SMI is prebuilt in Erlang SNMP compiler. See Importing MIBs section at http://www.erlang.org/doc/apps/snmp/snmp_mib_compiler.html#id76266 I tried this with Erlang releases R14B and R16A . Any help is highly appreciated. Anish -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: MY-TRAP-MIB.mib Type: application/octet-stream Size: 1491 bytes Desc: not available URL: From gomoripeti@REDACTED Sun Apr 28 11:42:08 2013 From: gomoripeti@REDACTED (=?ISO-8859-1?Q?Peti_G=F6m=F6ri?=) Date: Sun, 28 Apr 2013 11:42:08 +0200 Subject: [erlang-questions] Undefined type ''Gauge32''? Is SNMPv2-SMI prebuilt into Erlang SNMP MIB Compiler ? In-Reply-To: References: Message-ID: Hi Anish, (Maybe you've already figured that out since) You need to import Gauge32 type also from SNMPv2-SMI. (like OTP-MIB does for example) br Peter On Sun, Apr 28, 2013 at 10:08 AM, Anish Parameswaran < anishparameswaran@REDACTED> wrote: > Hi everyone, > > I am trying to compile an MIB file using erlang SNMP MIB compiler. > However the following error is preventing the compilation > > MY-TRAP-MIB.mib: 29: Undefined type ''Gauge32'' > > The compilation command is > erlc +'{group_check,false}' -I . MY-TRAP-MIB.mib > > The files in current folder > MY-TRAP-MIB.mib OTP-MIB.bin OTP-REG.bin OTP-TC.bin > > The MY-TRAP-MIB.mib is a modified version from SNMP_Quick_Start > > The file MY-TRAP-MIB.mib is atatched. > > If i remove the testG32 (Guage32) object from the file , it compiles > successfully. > > R16 Documentation tells that SNMPv2-SMI is prebuilt in Erlang SNMP > compiler. See Importing MIBs section at > http://www.erlang.org/doc/apps/snmp/snmp_mib_compiler.html#id76266 > > I tried this with Erlang releases R14B and R16A . > > Any help is highly appreciated. > > Anish > > > > > > > > > > > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From anishparameswaran@REDACTED Sun Apr 28 12:35:16 2013 From: anishparameswaran@REDACTED (Anish Parameswaran) Date: Sun, 28 Apr 2013 16:05:16 +0530 Subject: [erlang-questions] Undefined type ''Gauge32''? Is SNMPv2-SMI prebuilt into Erlang SNMP MIB Compiler ? In-Reply-To: References: Message-ID: Hi Peter, As you said , I imported Gauge32 from SNMPv2-SMI. Now the file is compiling fine. I have a set of MIBs from one telecom vendor. The MIBs use Gauge32 but don't import Gauge32, Thanks for the help Peter. Thanks all Anish On Sun, Apr 28, 2013 at 3:12 PM, Peti G?m?ri wrote: > Hi Anish, > > (Maybe you've already figured that out since) > You need to import Gauge32 type also from SNMPv2-SMI. > (like OTP-MIB does for example) > > br > Peter > > On Sun, Apr 28, 2013 at 10:08 AM, Anish Parameswaran < > anishparameswaran@REDACTED> wrote: > >> Hi everyone, >> >> I am trying to compile an MIB file using erlang SNMP MIB compiler. >> However the following error is preventing the compilation >> >> MY-TRAP-MIB.mib: 29: Undefined type ''Gauge32'' >> >> The compilation command is >> erlc +'{group_check,false}' -I . MY-TRAP-MIB.mib >> >> The files in current folder >> MY-TRAP-MIB.mib OTP-MIB.bin OTP-REG.bin OTP-TC.bin >> >> The MY-TRAP-MIB.mib is a modified version from SNMP_Quick_Start >> >> The file MY-TRAP-MIB.mib is atatched. >> >> If i remove the testG32 (Guage32) object from the file , it compiles >> successfully. >> >> R16 Documentation tells that SNMPv2-SMI is prebuilt in Erlang SNMP >> compiler. See Importing MIBs section at >> http://www.erlang.org/doc/apps/snmp/snmp_mib_compiler.html#id76266 >> >> I tried this with Erlang releases R14B and R16A . >> >> Any help is highly appreciated. >> >> Anish >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tyron.zerafa@REDACTED Sun Apr 28 16:07:15 2013 From: tyron.zerafa@REDACTED (Tyron Zerafa) Date: Sun, 28 Apr 2013 16:07:15 +0200 Subject: [erlang-questions] XREF - fun objects dependencies Message-ID: Is it possible to get the function dependencies of a fun object using xref. For instance given the following function: -module(somemod). f(File1) -> Ans = file:read(File1), F = fun() -> file:write(File1, "Done") end. Is it possible to get [ *{{somemod,'-f/1-fun-0-',1},{file,read,1}},* {{somemod,f,1},{file,read,1}}, {{somemod,f,1},{file,write,2}} ] instead of just [ {{somemod,f,1},{file,read,1}}, {{somemod,f,1},{file,write,2}} ] Thanks :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierrefenoll@REDACTED Sun Apr 28 22:58:49 2013 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Sun, 28 Apr 2013 22:58:49 +0200 Subject: [erlang-questions] Simple supervisor failed_to_start_child Message-ID: Hi all, I am trying start a very simple one_for_one supervisor architecture over a working project; but all I get when I start_link/0 my supervisor is: {shutdown,{failed_to_start_child,brocessing,brocessing}} I don't know why it mentions "brocessing" twice; well I don't understand the message much. brocessing is a module (containing the API to start a gen_server), but also it is the registered name of my gen_server. Here is my setup, and the whole code not far away: [1] So far Internet hasn't been helpful. Can you share your insights? [1]: https://bitbucket.org/fenollp/brocessing/src/5d7169f4600a05be42942ce890b2bee42c3fddda/src/brocessing_sup.erl?at=master Cheers, Pierre -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon Apr 29 01:28:10 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 29 Apr 2013 11:28:10 +1200 Subject: [erlang-questions] Request for comment: A proposal to introduce the concept of function domains In-Reply-To: References: Message-ID: On 25/04/2013, at 7:21 PM, Thomas J?rvstrand wrote: > The rationale is that encapsulation is a good thing and that code should not be allowed to depend on library functions that the library's author did not intend to expose to the outside world. Because of this I would like to introduce the idea of exporting functions into different domains. "Domain" here appears to mean "scope". There has been a proposal for many years now, inspired by Eiffel, to have -export_to(Module, [F1/N1,...,Fk/Nk]). You'll find it written up as EEP 5 http://www.erlang.org/eeps/eep-0005.html but it is at least 10 years older than that. The original proposal came with a variant that said "these are callback functions for such-and-such a behaviour", with the intention that it would be checked at compile and/or load time that such a behaviour existed and wanted those functions. The Eiffel approach is organised around what is to be exported: feature {} The EEP5 approach is organised around the destination of exports: -export_to(, []). If EEP5 is extended slightly to allow -export_to([], []). and to allow [X] to be simplified to X when there is a single X, then either way of slicing up the export matrix could be used, whichever the programmer thought clearer. > There will be three predefined function domains: one will allow the function to be called from anywhere, I take it this has the same effect as -export does now. > one will allow the function to be called from within its own application The trouble is that "application" is an OTP concept, not an Erlang concept, in that the Erlang compiler otherwise has no knowledge of "applications". Now, suppose we take the slightly extended EEP5. Can that do this job? Well, suppose we set up a convention, that an application foo has a foo_modules.hrl file containing -define(FOO_MODULES, []). Then you will be able to do -include('foo_modules.hrl'). -export_to(?FOO_MODULES, []). Some good things about this are * no requirement for a run-time test 'are these two modules part of the same application', which I am not sure is implementable in principle * you can export to *less* than a whole application but more than a single module * you can export to more than one application but less than the entire universe > and one to disallow the function from being explicitly referenced anywhere outside its module (this is for behaviour callbacks, functions passed/returned as funs etc.) Ah, this is a tricky one. You're talking about allowing something to be called elsewhere, but not explicitly. But that's rather a fragile concept. m:f(X1, X2) % explicit M = m, M:f(X1, X2) % not explicit F = f, m:F(X1, X2) % not explicit M = m, F = f, M:F(X1, X2) % not explicit But if the compiler front end does constant propagation (and why shouldn't it?) all of these would look identical to the back end. Do we really have a problem here that needs solving? In any case, doesn't this conflict with export_all? > Suggestions for what to call the predefined domains are: > > public, restricted, private > The rationale is that due to how the language works, a domain-declaration will only specify where we allow the function to be referenced with its fully qualified name we can't detect other any other references anyway. But a private function under your scheme would in no sense be private; it is after all a kind of _export_. What's more since you say that "private" is for "behaviour callbacks ... etc" and since existing behaviour modules are outside your application, "private" actually makes a function available to the *same* modules that "public" does. It seems to me that you are mixing up two very different things here: - WHICH other modules may call a function - HOW may they call it. I think a coherent design needs to separate these issues. From sean@REDACTED Mon Apr 29 08:00:02 2013 From: sean@REDACTED (Functional Jobs) Date: Mon, 29 Apr 2013 02:00:02 -0400 Subject: [erlang-questions] New Functional Programming Job Opportunities Message-ID: <517e0c6e1c219@functionaljobs.com> Here are some functional programming job opportunities that were posted recently:Erlang Engineer working w/ billions of daily, real-time auctions at AdRollhttp://functionaljobs.com/jobs/135-erlang-engineer-working-w-billions-of-daily-real-time-auctions-at-adrollCheers,Sean MurphyFunctionalJobs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Mon Apr 29 08:20:20 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 29 Apr 2013 08:20:20 +0200 Subject: [erlang-questions] Simple supervisor failed_to_start_child In-Reply-To: References: Message-ID: <1367216420.5171.1.camel@sekic1152.rnd.ki.sw.ericsson.se> Greetings, How small can you make an example, that still fails with the error message in question? bengt On Sun, 2013-04-28 at 22:58 +0200, Pierre Fenoll wrote: > Hi all, > > > I am trying start a very simple one_for_one supervisor architecture > over > a working project; but all I get when I start_link/0 my supervisor is: > > > {shutdown,{failed_to_start_child,brocessing,brocessing}} > > > > > > I don't know why it mentions "brocessing" twice; well I don't > understand the message > much. brocessing is a module (containing the API to start a > gen_server), but also it is > the registered name of my gen_server. > > > Here is my setup, and the whole code not far away: [1] > > > So far Internet hasn't been helpful. Can you share your insights? > > > [1]: > https://bitbucket.org/fenollp/brocessing/src/5d7169f4600a05be42942ce890b2bee42c3fddda/src/brocessing_sup.erl?at=master > > > Cheers, > Pierre > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From baliulia@REDACTED Mon Apr 29 08:41:50 2013 From: baliulia@REDACTED (=?UTF-8?B?SWduYXMgVnnFoW5pYXVza2Fz?=) Date: Mon, 29 Apr 2013 08:41:50 +0200 Subject: [erlang-questions] Automatically reconnecting nodes when they come back online In-Reply-To: References: Message-ID: <517E162E.7050609@gmail.com> On 04/26/2013 07:00 PM, Scott Thoman wrote: > My first two thoughts were: 1. When the conductor node connects up > the player nodes it would also spawn a process whose sole job is to > periodically ping the other nodes to ensure they're connected. Then > when one goes down, those pings will just fail during that time but > when the node comes back a ping will reconnect it to the other > nodes. All this time, I'd be monitoring the node up/down messages. 2. > I'd start by monitoring all the nodes as the conductor connects them > and when receiving a node down message, spawn a process whose job it > is to periodically ping only that node only until it comes back. * you don't need a pinging mechanism, just use the existing `net_kernel:monitor(true)` and handle the events. * if you can afford a fixed node name for at least the "conductor" node, then you can do something along the lines you described yourself -- should be trivial. * otherwise you can try to hack things using `net_adm:world()` or something like that for "dynamic" node discovery Also, take a look at the `{sync_nodes_optional, NodeList}` parameter of `kernel`. -- Ignas From fredrik@REDACTED Mon Apr 29 08:42:07 2013 From: fredrik@REDACTED (fredrik@REDACTED) Date: Sun, 28 Apr 2013 23:42:07 -0700 (PDT) Subject: [erlang-questions] Segfault in NIF call Message-ID: <3da9e2e4-e1fc-4187-9573-faed5e1c249d@googlegroups.com> Hello folks, I'm having difficulties locating the cause of a segfault I'm getting when running tests with a NIF implementation I have. Anything that would shed light on what's wrong is appreciated. The backtrace does not seem to have anything to do with my NIF. The NIF:ed function is called from a few different processes, and it forwards the call to a worker thread which sends a message back to the caller process. > strings -a ../../otp/R16B/pre-5.10.1-mz-0.2/lib/erlang/erts-5.10.1/bin/beam.smp|fgrep GCC|sort -u GCC: (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3 > USE_GDB=1 rebar eunit [cut] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x2aaaef381700 (LWP 19162)] unlink_free_block (allctr=0x85f240, block=0x0, flags=0) at beam/erl_goodfit_alloc.c:458 458 Uint sz = MBC_FBLK_SZ(&blk->block_head); (gdb) backtrace #0 unlink_free_block (allctr=0x85f240, block=0x0, flags=0) at beam/erl_goodfit_alloc.c:458 #1 0x0000000000444ee1 in get_free_block (allctr=0x85f240, size=, cand_blk=0x0, cand_size=0, flags=0) at beam/erl_goodfit_alloc.c:426 #2 0x000000000043b6da in mbc_alloc_block (alcu_flgsp=, blk_szp=, size=, allctr=0x85f240) at beam/erl_alloc_util.c:1309 #3 mbc_alloc (allctr=0x85f240, size=) at beam/erl_alloc_util.c:1451 #4 0x0000000000440d7b in do_erts_alcu_alloc (size=32, extra=0x85f240, type=148) at beam/erl_alloc_util.c:3530 #5 erts_alcu_alloc_thr_pref (type=148, extra=, size=) at beam/erl_alloc_util.c:3607 #6 0x0000000000490595 in erts_alloc (size=32, type=18967) at beam/erl_alloc.h:208 #7 new_message_buffer (size=0) at beam/erl_message.c:72 #8 erts_alloc_message_heap_state (statep=0x2aaaef380c4c, receiver_locks=0x2aaaef380cb0, receiver=0x2aaaac9ccac8, ohpp=, bpp=, size=0) at beam/global.h:1017 #9 erts_send_message (sender=0x2aaaac9cd118, receiver=0x2aaaac9ccac8, receiver_locks=0x2aaaef380cb0, message=513931, flags=) at beam/erl_message.c:1039 #10 0x0000000000476350 in do_send (p=0x2aaaac9cd118, to=550027, msg=513931, suspend=1, refp=) at beam/bif.c:2025 #11 0x0000000000476bf0 in erl_send (p=0x2aaaac9cd118, to=550027, msg=513931) at beam/bif.c:2138 #12 0x000000000052d1d8 in process_main () at beam/beam_emu.c:2558 #13 0x0000000000491463 in sched_thread_func (vesdp=0x2aaaac34de40) at beam/erl_process.c:5632 #14 0x0000000000590440 in thr_wrapper (vtwd=0x7fffffffd7e0) at pthread/ethread.c:106 #15 0x00002aaaab3ffe9a in start_thread (arg=0x2aaaef381700) at pthread_create.c:308 #16 0x00002aaaab911cbd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #17 0x0000000000000000 in ?? () (gdb) -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Mon Apr 29 09:24:10 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Mon, 29 Apr 2013 10:24:10 +0300 Subject: [erlang-questions] Automatically reconnecting nodes when they come back online In-Reply-To: <517E162E.7050609@gmail.com> References: <517E162E.7050609@gmail.com> Message-ID: <863C6AB0-9ACC-477B-A2DF-06FE30541CE6@gmail.com> Hello, I am using the following approach for similar issue. - net_kernel:monitor allow your process to receive nodeup/nodedown events. - the player nodes requires a list of 'seed' nodes at config file. It should connect those seed nodes at boot time. If none of seeds is connected then player node has to die with alarm. - Dmitry On Apr 29, 2013, at 9:41 AM, Ignas Vy?niauskas wrote: > On 04/26/2013 07:00 PM, Scott Thoman wrote: >> My first two thoughts were: 1. When the conductor node connects up >> the player nodes it would also spawn a process whose sole job is to >> periodically ping the other nodes to ensure they're connected. Then >> when one goes down, those pings will just fail during that time but >> when the node comes back a ping will reconnect it to the other >> nodes. All this time, I'd be monitoring the node up/down messages. 2. >> I'd start by monitoring all the nodes as the conductor connects them >> and when receiving a node down message, spawn a process whose job it >> is to periodically ping only that node only until it comes back. > > * you don't need a pinging mechanism, just use the existing > `net_kernel:monitor(true)` and handle the events. > * if you can afford a fixed node name for at least the "conductor" node, > then you can do something along the lines you described yourself -- > should be trivial. > * otherwise you can try to hack things using `net_adm:world()` or > something like that for "dynamic" node discovery > > Also, take a look at the `{sync_nodes_optional, NodeList}` parameter of > `kernel`. > > -- > Ignas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mrtndimitrov@REDACTED Mon Apr 29 09:23:20 2013 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Mon, 29 Apr 2013 10:23:20 +0300 Subject: [erlang-questions] Supervision of temporary workers Message-ID: <517E1FE8.403@gmail.com> Hello all, I choose for restart strategy *one_for_all". Then for child specification, the restart to be *temporary*. As I understand, a temporary child process will never be restarted, but will it trigger the death of the supervision tree? My test show no but then how should I accomplish this - kill the tree as in one_for_all but don't restart the workers? Thanks for your time, Regards, Martin From sverker.eriksson@REDACTED Mon Apr 29 11:58:25 2013 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Mon, 29 Apr 2013 11:58:25 +0200 Subject: [erlang-questions] Segfault in NIF call In-Reply-To: <3da9e2e4-e1fc-4187-9573-faed5e1c249d@googlegroups.com> References: <3da9e2e4-e1fc-4187-9573-faed5e1c249d@googlegroups.com> Message-ID: <517E4441.2040202@erix.ericsson.se> Looks like some sort of memory corruption. Run on debug emulator and hope for a better (earlier) crash. # cd $ERL_TOP/erts/emulator # make TYPE=debug smp plain # $ERL_TOP/bin/cerl -debug /Sverker, Erlang/OTP Ericsson fredrik@REDACTED wrote: > Hello folks, > > I'm having difficulties locating the cause of a segfault I'm getting when > running tests with a NIF implementation I have. > > Anything that would shed light on what's wrong is appreciated. > > The backtrace does not seem to have anything to do with my NIF. The NIF:ed > function is called from a few different processes, and it forwards the call > to a worker thread which sends a message back to the caller process. > > >> strings -a >> > ../../otp/R16B/pre-5.10.1-mz-0.2/lib/erlang/erts-5.10.1/bin/beam.smp|fgrep > GCC|sort -u > GCC: (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3 > > >> USE_GDB=1 rebar eunit >> > [cut] > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x2aaaef381700 (LWP 19162)] > unlink_free_block (allctr=0x85f240, block=0x0, flags=0) at > beam/erl_goodfit_alloc.c:458 > 458 Uint sz = MBC_FBLK_SZ(&blk->block_head); > (gdb) backtrace > #0 unlink_free_block (allctr=0x85f240, block=0x0, flags=0) at > beam/erl_goodfit_alloc.c:458 > #1 0x0000000000444ee1 in get_free_block (allctr=0x85f240, size= out>, cand_blk=0x0, cand_size=0, flags=0) at beam/erl_goodfit_alloc.c:426 > #2 0x000000000043b6da in mbc_alloc_block (alcu_flgsp=, > blk_szp=, size=, allctr=0x85f240) at > beam/erl_alloc_util.c:1309 > #3 mbc_alloc (allctr=0x85f240, size=) at > beam/erl_alloc_util.c:1451 > #4 0x0000000000440d7b in do_erts_alcu_alloc (size=32, extra=0x85f240, > type=148) at beam/erl_alloc_util.c:3530 > #5 erts_alcu_alloc_thr_pref (type=148, extra=, > size=) at beam/erl_alloc_util.c:3607 > #6 0x0000000000490595 in erts_alloc (size=32, type=18967) at > beam/erl_alloc.h:208 > #7 new_message_buffer (size=0) at beam/erl_message.c:72 > #8 erts_alloc_message_heap_state (statep=0x2aaaef380c4c, > receiver_locks=0x2aaaef380cb0, receiver=0x2aaaac9ccac8, ohpp= pointer>, bpp=, size=0) at beam/global.h:1017 > #9 erts_send_message (sender=0x2aaaac9cd118, receiver=0x2aaaac9ccac8, > receiver_locks=0x2aaaef380cb0, message=513931, flags=) at > beam/erl_message.c:1039 > #10 0x0000000000476350 in do_send (p=0x2aaaac9cd118, to=550027, msg=513931, > suspend=1, refp=) at beam/bif.c:2025 > #11 0x0000000000476bf0 in erl_send (p=0x2aaaac9cd118, to=550027, > msg=513931) at beam/bif.c:2138 > #12 0x000000000052d1d8 in process_main () at beam/beam_emu.c:2558 > #13 0x0000000000491463 in sched_thread_func (vesdp=0x2aaaac34de40) at > beam/erl_process.c:5632 > #14 0x0000000000590440 in thr_wrapper (vtwd=0x7fffffffd7e0) at > pthread/ethread.c:106 > #15 0x00002aaaab3ffe9a in start_thread (arg=0x2aaaef381700) at > pthread_create.c:308 > #16 0x00002aaaab911cbd in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 > #17 0x0000000000000000 in ?? () > (gdb) > > > > ------------------------------------------------------------------------ > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From attila.r.nohl@REDACTED Mon Apr 29 14:17:15 2013 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Mon, 29 Apr 2013 14:17:15 +0200 Subject: [erlang-questions] [erlang-patches] Scan "=<<" as something useful In-Reply-To: <517E608F.3030606@ninenines.eu> References: <262226227.88551570.1367209751108.JavaMail.root@erlang-solutions.com> <517E608F.3030606@ninenines.eu> Message-ID: In source code I usually put spaces around =, so it didn't bite me that often, but it is really inconvenient in the shell, in one-liners when I want to shovel as much as possible into the terminal width and there every character counts, so I avoid spaces. 2013/4/29 Lo?c Hoguin : > I don't encounter it often anymore because I got used to it being broken, > and put an extra space, but that doesn't mean it shouldn't be fixed, it > would relieve the many new developers from making the mistake and then > losing time asking about it. > > > On 04/29/2013 06:29 AM, Robert Virding wrote: >> >> Well, I can honestly say I never encounter it so that doesn't prove >> anything either way. Just put spaces around = and you will get easier to >> read code as well. IMAO >> >> Robert >> >> ----- Original Message ----- >>> >>> From: "Anthony Ramine" >>> To: mc@REDACTED >>> Cc: "Robert Virding" , "Erlang >>> Patches" >>> Sent: Sunday, 28 April, 2013 4:59:06 AM >>> Subject: Re: [erlang-patches] Scan "=<<" as something useful >>> >>> Hello Pedram, >>> >>> I will also add that most often I get hit by that bug in the shell, >>> not in compiled modules. It is quite annoying to encounter an >>> avoidable syntax error in the REPL while testing new things and >>> doing "Bin=<<>>". >>> >>> Regards >>> >>> -- >>> Anthony Ramine >>> >>> Le 28 avr. 2013 ? 05:36, Pedram Nimreezi a ?crit : >>> >>>> While I completely agree with this sentiment- I don't agree the >>>> good outweighs the bad here.. >>>> >>>> - I think it is a real language wart. >>>> - I think it is inconsistent with other types. >>>> - I think time spent looking for this error is wasted time. >>>> - I believe that at best this should warn and not error. >>>> >>>> Lastly, I consider this a byproduct of incomplete scanning and >>>> should be corrected not celebrated.. >>>> >>>> >>>> On Sat, Apr 27, 2013 at 11:09 PM, Robert Virding >>>> wrote: >>>> I don't want to be a wet blanket but I feel that in general having >>>> blanks between operators is a Good Thing which improves >>>> legibility. So I don't really like a syntax change which supports >>>> not having blanks between operators. >>>> >>>> Robert >>>> >>>> From: "Pedram Nimreezi" >>>> To: "Sean Cribbs" >>>> Cc: "Erlang Patches" >>>> Sent: Monday, 22 April, 2013 11:06:24 AM >>>> Subject: Re: [erlang-patches] Scan "=<<" as something useful >>>> >>>> >>>> Good erlang wart to remove... +1 >>>> >>>> >>>> On Mon, Apr 22, 2013 at 12:21 AM, Sean Cribbs >>>> wrote: >>>> I can't count how many times this has bitten me, kudos. >>>> >>>> >>>> On Sun, Apr 21, 2013 at 11:15 PM, Anthony Ramine >>>> wrote: >>>> Hello, >>>> >>>> I've noticed that Erlang/OTP own code tend to be space-free, e.g. >>>> {a, >>>> b} is rather written as {a,b}. Then there is the occasional Bin= >>>> <<42>> with a space because erl_scan will scans "=<<" as '=<','<' >>>> instead of '=','<<'. This patch teaches erl_scan how to not do >>>> that. >>>> >>>> There is no backwards compatibility problem as '=<','<' can't >>>> appear >>>> in valid code. >>>> >>>> git fetch https://github.com/nox/otp.git >>>> disambiguate-equal-binary >>>> >>>> https://github.com/nox/otp/compare/disambiguate-equal-binary >>>> >>>> https://github.com/nox/otp/compare/disambiguate-equal-binary.patch >>>> >>>> Regards, >>>> >>>> -- >>>> Anthony Ramine >>>> _______________________________________________ >>>> erlang-patches mailing list >>>> erlang-patches@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-patches >>>> >>>> >>>> >>>> -- >>>> Sean Cribbs >>>> Software Engineer >>>> Basho Technologies, Inc. >>>> http://basho.com/ >>>> >>>> _______________________________________________ >>>> erlang-patches mailing list >>>> erlang-patches@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-patches >>>> >>>> >>>> >>>> >>>> -- >>>> /* Sincerely >>>> -------------------------------------------------------------- >>>> Pedram Nimreezi - Chief Technology Officer */ >>>> >>>> // The hardest part of design ? is keeping features out. - Donald >>>> Norman >>>> >>>> >>>> >>>> _______________________________________________ >>>> erlang-patches mailing list >>>> erlang-patches@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-patches >>>> >>>> >>>> >>>> >>>> -- >>>> /* Sincerely >>>> -------------------------------------------------------------- >>>> Pedram Nimreezi - Chief Technology Officer */ >>>> >>>> // The hardest part of design ? is keeping features out. - Donald >>>> Norman >>>> >>>> >>>> _______________________________________________ >>>> erlang-patches mailing list >>>> erlang-patches@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-patches >>> >>> >>> >> _______________________________________________ >> erlang-patches mailing list >> erlang-patches@REDACTED >> http://erlang.org/mailman/listinfo/erlang-patches >> > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > > _______________________________________________ > erlang-patches mailing list > erlang-patches@REDACTED > http://erlang.org/mailman/listinfo/erlang-patches From g@REDACTED Mon Apr 29 14:23:06 2013 From: g@REDACTED (Garrett Smith) Date: Mon, 29 Apr 2013 07:23:06 -0500 Subject: [erlang-questions] Supervision of temporary workers In-Reply-To: <517E1FE8.403@gmail.com> References: <517E1FE8.403@gmail.com> Message-ID: Hi Martin, On Mon, Apr 29, 2013 at 2:23 AM, Martin Dimitrov wrote: > Hello all, > > I choose for restart strategy *one_for_all". Then for child > specification, the restart to be *temporary*. > > As I understand, a temporary child process will never be restarted, but > will it trigger the death of the supervision tree? My test show no but > then how should I accomplish this - kill the tree as in one_for_all but > don't restart the workers? I don't quite understand what you're looking to do. But if you want a worker failure to propagate up to the supervisor, use a permanent strategy and set restart frequency (MaxR) to 0. Garrett From mrtndimitrov@REDACTED Mon Apr 29 14:45:56 2013 From: mrtndimitrov@REDACTED (Martin Dimitrov) Date: Mon, 29 Apr 2013 15:45:56 +0300 Subject: [erlang-questions] Supervision of temporary workers In-Reply-To: References: <517E1FE8.403@gmail.com> Message-ID: <517E6B84.6060100@gmail.com> Thank you. I will do as you suggest. On 4/29/2013 3:23 PM, Garrett Smith wrote: > Hi Martin, > > On Mon, Apr 29, 2013 at 2:23 AM, Martin Dimitrov wrote: >> Hello all, >> >> I choose for restart strategy *one_for_all". Then for child >> specification, the restart to be *temporary*. >> >> As I understand, a temporary child process will never be restarted, but >> will it trigger the death of the supervision tree? My test show no but >> then how should I accomplish this - kill the tree as in one_for_all but >> don't restart the workers? > > I don't quite understand what you're looking to do. > > But if you want a worker failure to propagate up to the supervisor, > use a permanent strategy and set restart frequency (MaxR) to 0. > > Garrett > From pierrefenoll@REDACTED Mon Apr 29 16:59:30 2013 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Mon, 29 Apr 2013 16:59:30 +0200 Subject: [erlang-questions] Simple supervisor failed_to_start_child In-Reply-To: <2742011367217866@web12d.yandex.ru> References: <2742011367217866@web12d.yandex.ru> Message-ID: Thanks to Slava Yurin, that nailed it! Indeed, start_link/0 should return {ok,Pid} | ignore | {error,Error}. On 29 April 2013 08:44, Slava Yurin wrote: > Your stateful module return incorect value from start_link(). > https://bitbucket.org/fenollp/brocessing/src/5d7169f4600a/src/stateful.erl?at=master#cl-113 > . > See http://www.erlang.org/doc/man/gen_server.html#start_link-3. > > 29.04.2013, 03:59, "Pierre Fenoll" : > > Hi all, > > I am trying start a very simple one_for_one supervisor architecture over > a working project; but all I get when I start_link/0 my supervisor is: > > {shutdown,{failed_to_start_child,brocessing,brocessing}} > > > I don't know why it mentions "brocessing" twice; well I don't understand > the message > much. brocessing is a module (containing the API to start a gen_server), > but also it is > the registered name of my gen_server. > > Here is my setup, and the whole code not far away: [1] > > So far Internet hasn't been helpful. Can you share your insights? > > [1]: > https://bitbucket.org/fenollp/brocessing/src/5d7169f4600a05be42942ce890b2bee42c3fddda/src/brocessing_sup.erl?at=master > > Cheers, > Pierre > , > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rr@REDACTED Mon Apr 29 17:42:49 2013 From: rr@REDACTED (Rick Reed) Date: Mon, 29 Apr 2013 08:42:49 -0700 Subject: [erlang-questions] Schedulers getting "stuck", part II In-Reply-To: <96629.1367029232@snookles.snookles.com> References: <96629.1367029232@snookles.snookles.com> Message-ID: <517E94F9.8090407@whatsapp.com> We are still seeing stuck schedulers on R16B although now that I've looked at it, we've been running with +swt medium (vs. +swt low on R15B) -- I had forgotten I'd made that change when we first started deploying R16B because I wanted to gauge whether there had been an improvement. This is what a sample %cpu distribution looks like in this condition: 97.0 96.3 95.6 95.3 93.6 93.2 92.6 92.1 91.5 90.5 90.3 87.9 87.6 87.5 84.2 83.9 83.6 83.0 82.7 82.2 80.8 79.4 77.9 2.7 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 When I dug into this a while back on R15B, I think I reached the conclusion that at least with our workload, the run queues on a scheduler were being drained to <2 often enough (even with very significant overall load) that wakeup_other_check was keeping wakeup_other below the limit for wakeup. Even with an overall scheduler utilization over 75%, our run queues are quite short. Rr On 4/26/13 7:20 PM, Scott Lystig Fritchie wrote: > Howdy. This is a followup to the discussion that took place on this > list in October 2012, see: > > http://erlang.org/pipermail/erlang-questions/2012-October/069503.html > (first message only, I dunno why) > http://erlang.org/pipermail/erlang-questions/2012-October/069585.html > (the rest of the thread) > > I've been trying to figure out how to introduce the stuff that I've > written at: > > https://github.com/slfritchie/nifwait/tree/md5#readme > > ... but I still can't decide. So I'll try for something short and > un-Scott-like. For the long story, please read the README in the URL > above. > > As for the short story, I believe a couple of things: > > * R15B0x's schedulers are broken: Basho seen "stuck" schedulers in one of > our apps with no custom NIF code. And it's possible to get them stuck > using only the 'crypto' module's MD5 functions. > > * R16B's schedulers appear to be even more broken: I have a > mostly-deterministic case that demonstrates schedulers that go to > sleep and do not wake for minutes (or hours) when there is plenty of > work to do. This also is using only the 'crypto' module and does not > require custom NIF code. > > Discuss. :-) > > -Scott > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From fredrik@REDACTED Mon Apr 29 20:57:14 2013 From: fredrik@REDACTED (Fredrik Linder) Date: Mon, 29 Apr 2013 11:57:14 -0700 Subject: [erlang-questions] Segfault in NIF call In-Reply-To: <517E4441.2040202@erix.ericsson.se> References: <3da9e2e4-e1fc-4187-9573-faed5e1c249d@googlegroups.com> <517E4441.2040202@erix.ericsson.se> Message-ID: Found one error (yay!) -- enif_open_resource_type should have NULL as the module argument (missed this one in the docs) Running with cerl does not generate a segfault, nor any (extra) printouts Running with erl still generate segfault Now I get either of the following (using erl), does it reveal what my error is? > USE_GDB=1 rebar eunit [cut] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x2aaaad6c0700 (LWP 6384)] 0x000000000052b28a in process_main () at x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:979 979 MoveDeallocateReturn(xb(tmp_packed1&BEAM_LOOSE_MASK), r(0), Qb((tmp_packed1>>BEAM_LOOSE_SHIFT))); (gdb) backtrace #0 0x000000000052b28a in process_main () at x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:979 #1 0x0000000000491463 in sched_thread_func (vesdp=0x2aaaac343ac0) at beam/erl_process.c:5632 #2 0x0000000000590440 in thr_wrapper (vtwd=0x7fffffffd850) at pthread/ethread.c:106 #3 0x00002aaaab3ffe9a in start_thread (arg=0x2aaaad6c0700) at pthread_create.c:308 #4 0x00002aaaab911cbd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #5 0x0000000000000000 in ?? () or Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x2aaaad6c0700 (LWP 6914)] unlink_free_block (allctr=0x85e5c0, block=0x0, flags=0) at beam/erl_goodfit_alloc.c:458 458 Uint sz = MBC_FBLK_SZ(&blk->block_head); (gdb) backtrace #0 unlink_free_block (allctr=0x85e5c0, block=0x0, flags=0) at beam/erl_goodfit_alloc.c:458 #1 0x0000000000444ee1 in get_free_block (allctr=0x85e5c0, size=, cand_blk=0x0, cand_size=0, flags=0) at beam/erl_goodfit_alloc.c:426 #2 0x000000000043b6da in mbc_alloc_block (alcu_flgsp=, blk_szp=, size=, allctr=0x85e5c0) at beam/erl_alloc_util.c:1309 #3 mbc_alloc (allctr=0x85e5c0, size=) at beam/erl_alloc_util.c:1451 #4 0x0000000000440d7b in do_erts_alcu_alloc (size=32, extra=0x85e5c0, type=148) at beam/erl_alloc_util.c:3530 #5 erts_alcu_alloc_thr_pref (type=148, extra=, size=) at beam/erl_alloc_util.c:3607 #6 0x0000000000490595 in erts_alloc (size=32, type=18967) at beam/erl_alloc.h:208 #7 new_message_buffer (size=0) at beam/erl_message.c:72 #8 erts_alloc_message_heap_state (statep=0x2aaaad6bfc4c, receiver_locks=0x2aaaad6bfcb0, receiver=0x2aaaac9c1db8, ohpp=, bpp=, size=0) at beam/global.h:1017 #9 erts_send_message (sender=0x2aaaac9c2470, receiver=0x2aaaac9c1db8, receiver_locks=0x2aaaad6bfcb0, message=564171, flags=) at beam/erl_message.c:1039 #10 0x0000000000476350 in do_send (p=0x2aaaac9c2470, to=793099, msg=564171, suspend=1, refp=) at beam/bif.c:2025 #11 0x0000000000476bf0 in erl_send (p=0x2aaaac9c2470, to=793099, msg=564171) at beam/bif.c:2138 #12 0x000000000052d1d8 in process_main () at beam/beam_emu.c:2558 #13 0x0000000000491463 in sched_thread_func (vesdp=0x2aaaac343ac0) at beam/erl_process.c:5632 #14 0x0000000000590440 in thr_wrapper (vtwd=0x7fffffffd850) at pthread/ethread.c:106 #15 0x00002aaaab3ffe9a in start_thread (arg=0x2aaaad6c0700) at pthread_create.c:308 #16 0x00002aaaab911cbd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #17 0x0000000000000000 in ?? () /Fredrik On Mon, Apr 29, 2013 at 2:58 AM, Sverker Eriksson < sverker.eriksson@REDACTED> wrote: > Looks like some sort of memory corruption. > > Run on debug emulator and hope for a better (earlier) crash. > > # cd $ERL_TOP/erts/emulator > # make TYPE=debug smp plain > # $ERL_TOP/bin/cerl -debug > > /Sverker, Erlang/OTP Ericsson > > fredrik@REDACTED wrote: > >> Hello folks, >> >> I'm having difficulties locating the cause of a segfault I'm getting when >> running tests with a NIF implementation I have. >> >> Anything that would shed light on what's wrong is appreciated. >> >> The backtrace does not seem to have anything to do with my NIF. The >> NIF:ed function is called from a few different processes, and it forwards >> the call to a worker thread which sends a message back to the caller >> process. >> >> >> >>> strings -a >>> >> ../../otp/R16B/pre-5.10.1-mz-**0.2/lib/erlang/erts-5.10.1/**bin/beam.smp|fgrep >> GCC|sort -u >> GCC: (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3 >> >> >> >>> USE_GDB=1 rebar eunit >>> >>> >> [cut] >> Program received signal SIGSEGV, Segmentation fault. >> [Switching to Thread 0x2aaaef381700 (LWP 19162)] >> unlink_free_block (allctr=0x85f240, block=0x0, flags=0) at >> beam/erl_goodfit_alloc.c:458 >> 458 Uint sz = MBC_FBLK_SZ(&blk->block_head); >> (gdb) backtrace >> #0 unlink_free_block (allctr=0x85f240, block=0x0, flags=0) at >> beam/erl_goodfit_alloc.c:458 >> #1 0x0000000000444ee1 in get_free_block (allctr=0x85f240, >> size=, cand_blk=0x0, cand_size=0, flags=0) at >> beam/erl_goodfit_alloc.c:426 >> #2 0x000000000043b6da in mbc_alloc_block (alcu_flgsp=> pointer>, blk_szp=, size=, >> allctr=0x85f240) at beam/erl_alloc_util.c:1309 >> #3 mbc_alloc (allctr=0x85f240, size=) at >> beam/erl_alloc_util.c:1451 >> #4 0x0000000000440d7b in do_erts_alcu_alloc (size=32, extra=0x85f240, >> type=148) at beam/erl_alloc_util.c:3530 >> #5 erts_alcu_alloc_thr_pref (type=148, extra=, >> size=) at beam/erl_alloc_util.c:3607 >> #6 0x0000000000490595 in erts_alloc (size=32, type=18967) at >> beam/erl_alloc.h:208 >> #7 new_message_buffer (size=0) at beam/erl_message.c:72 >> #8 erts_alloc_message_heap_state (statep=0x2aaaef380c4c, >> receiver_locks=0x2aaaef380cb0, receiver=0x2aaaac9ccac8, ohpp=> pointer>, bpp=, size=0) at beam/global.h:1017 >> #9 erts_send_message (sender=0x2aaaac9cd118, receiver=0x2aaaac9ccac8, >> receiver_locks=0x2aaaef380cb0, message=513931, flags=) at >> beam/erl_message.c:1039 >> #10 0x0000000000476350 in do_send (p=0x2aaaac9cd118, to=550027, >> msg=513931, suspend=1, refp=) at beam/bif.c:2025 >> #11 0x0000000000476bf0 in erl_send (p=0x2aaaac9cd118, to=550027, >> msg=513931) at beam/bif.c:2138 >> #12 0x000000000052d1d8 in process_main () at beam/beam_emu.c:2558 >> #13 0x0000000000491463 in sched_thread_func (vesdp=0x2aaaac34de40) at >> beam/erl_process.c:5632 >> #14 0x0000000000590440 in thr_wrapper (vtwd=0x7fffffffd7e0) at >> pthread/ethread.c:106 >> #15 0x00002aaaab3ffe9a in start_thread (arg=0x2aaaef381700) at >> pthread_create.c:308 >> #16 0x00002aaaab911cbd in clone () at ../sysdeps/unix/sysv/linux/** >> x86_64/clone.S:112 >> #17 0x0000000000000000 in ?? () >> (gdb) >> >> ------------------------------**------------------------------** >> ------------ >> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Mon Apr 29 22:47:56 2013 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 29 Apr 2013 13:47:56 -0700 Subject: [erlang-questions] [ANN] reltool_util Released Message-ID: <517EDC7C.9020907@gmail.com> https://github.com/okeuday/reltool_util This is an application/module which provides functions for dynamically starting or stopping applications as a hierarchy (with application_start and application_stop). The functionality is a pretty common desire when you are testing source code before making a release (with ct or eunit). For an example, you can see https://github.com/okeuday/cpg/blob/master/test/cpg_test.erl (n.b., using the file's eunit function definition order only works when inparallel is not being used). There also is an experimental function (script_start) for loading a whole script file in the running VM, which may be considered bad, since it doesn't use the normal process of generating a boot file and firing up an Erlang VM. The release escript provided in the project is a way of generating a release using the same basic sequence provided by "rebar generate" on unix, but without the template functionality and the windows support. The release escript was created after having problems with some of the validation rebar does before the release generation, which was a false negative (seems to be dependencies that aren't built with rebar can anger rebar's release generation). From dparfitt@REDACTED Mon Apr 29 22:59:10 2013 From: dparfitt@REDACTED (Dave Parfitt) Date: Mon, 29 Apr 2013 16:59:10 -0400 Subject: [erlang-questions] [ANN] ETP - An Erlang Term Parser for Java Message-ID: Hello - I'd like to announce the release of ETP v0.3. ETP is a parser for Java built with Antlr 4 that can parse a string representation of an Erlang term. The parsed model can be manipulated in Java, and rendered back to a string by calling toString(). The source can be found online at: http://metadave.github.io/etp/ The Maven artifact is available from the Sonatype OSS repository. com.github.metadave etp 0.3 Here's a simple example: ETPTuple tuple = (ETPTuple)ETP.parse("{mylist, [1,2,3,4], \n" + "my_string, \"Hello world\"}"); // All ETP objects subclass ETPTerm // you can use "instanceof" to see what the result of the parse is ETPAtom atom = (ETPAtom)tuple.getValue(0); System.out.println("Atom = " + atom); ETPList list = (ETPList)tuple.getValue(1); // use getValue() to access the list of terms for(ETPTerm v : list.getValue()) { System.out.println(" List item " + v); } ETPAtom my_string_atom = (ETPAtom)tuple.getValue(2); System.out.println(my_string_atom); ETPString s = (ETPString)tuple.getValue(3); System.out.println(s.getValue()); // getValue() gets the "raw" value System.out.println(s.toString()); // toString() gets the string representation // you can set values on the ETP structure and call toString() to render // a new Erlang term atom.setValue("new_atom_value"); list.getValue().add(new ETPInteger(100)); System.out.println(tuple.toString()); which yields the following output: Atom = mylist List item 1 List item 2 List item 3 List item 4 my_string Hello world "Hello world" {new_atom_value,[1,2,3,4,100],my_string,"Hello world"} Cheers - Dave https://github.com/metadave -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Tue Apr 30 02:40:53 2013 From: steven.charles.davis@REDACTED (Steve Davis) Date: Mon, 29 Apr 2013 19:40:53 -0500 Subject: [erlang-questions] Static type checking... Message-ID: <6AB510F1-B600-47E3-98B6-108230D507D2@gmail.com> It's my experience that attempts at static type checks ends up in "turtles all the way down" and enormously onerous code. So I'm thinking that Joe A had a very big deal point (beyond the design of UBF) with regard to UBF contracts? Is this because, despite the limitations of records, what really lacks is some kind of way of expressing formal boundary checking? I do know that RV expresses a liking for, and recommends, boundary checks (which work for me). Is this validation "layer" really the missing link? Is it possible to find an enforcement for that (and not engage in strict typing)? Throwing this out as my 2c, /s From drew.varner@REDACTED Tue Apr 30 02:56:57 2013 From: drew.varner@REDACTED (Drew Varner) Date: Tue, 30 Apr 2013 00:56:57 -0000 Subject: [erlang-questions] FW: Message-ID: http://www.jaok.sakura.ne.jp/92p4ad.php From stamarit@REDACTED Tue Apr 30 03:23:18 2013 From: stamarit@REDACTED (Salvador Tamarit) Date: Tue, 30 Apr 2013 03:23:18 +0200 Subject: [erlang-questions] Managing a huge binary file Message-ID: hi all, I have a very interesting problem for us :) Suppose that we have a huge binary file (8GB or so) containing all the integers from 0 to a given (very big) number. The numbers are unsorted and some of them (70 numbers for instance) has been removed. The goal is to find the the Nth removed number (in order). For instance, if the file contains the numbers from 0 to 99, and then number 101, the 1st removed would be 100. I have been trying to solve this problem with different solutions, but I always get stucked in reading the file. I'm reading it sequentally. I open the file with: file:open(FileName,[binary]) then I read each number with: file:read(FDevice, 4) this works fine, but too slow. Anyway, supposing that I would be able to read it, then I don't know how to manage the data (I think that it is impossible to load all this data in memory). How would you solve a problem like this? Thanks, Salva From ok@REDACTED Tue Apr 30 04:00:05 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 30 Apr 2013 14:00:05 +1200 Subject: [erlang-questions] Managing a huge binary file In-Reply-To: References: Message-ID: <52ADF12A-AF3E-43E6-BCC5-B02CB891F0C4@cs.otago.ac.nz> On 30/04/2013, at 1:23 PM, Salvador Tamarit wrote: > Suppose that we have a huge binary file (8GB or so) containing all the > integers from 0 to a given (very big) number. The numbers are unsorted > and some of them (70 numbers for instance) has been removed. This sounds like a well known problem with an answer in http://books.google.co.nz/books?id=415loiMd_c0C&lpg=PP1&dq=muthukrishnan+data+stream+algorithms&hl=el&pg=PA1&redir_esc=y with the twist that you are asked to find a specific missing number (in value order) rather than all of them. When you talk about reading the numbers 4 bytes at a time, that tells me the numbers are not >that< "huge". One very simple step would be to set up number-present : map (0..6535) -> nat := (all -> 0)_ for each number in the input number-present(high 16 bits of that number) +:= 1 Now you can tell which block of numbers must contain the number you are looking for, and you can make a second pass through the data counting numbers in that block, and then it's a simple scan through the block. So if the numbers are 0..N-1, the problem can be solved in O(N**1/2) space in two passes, or in O(N**1/3) space in three passes, &c. This is _not_ mathematically optimal, but it's quite easy to program, and if you really had this problem, I'd rather have the data read twice by a simple program I trusted than once by a clever program that was almost certainly wrong. From erlangprogram@REDACTED Tue Apr 30 04:35:33 2013 From: erlangprogram@REDACTED (S X) Date: Mon, 29 Apr 2013 22:35:33 -0400 Subject: [erlang-questions] diameter callback module In-Reply-To: References: Message-ID: Hi Anders, Thanks for your explanation. Setting additional options in diameter:add_transport and retrieving them with diameter:service_info from the peer_up callback work well. And the descriptions about the peer states according to RFC3539 and 6733 make a lot of sense. The case I am trying to resolve is: Start multiple processes on one pc, each process tries to connect to one same server (obviously I will get already_connected warning, but it can keep going), in each process, then I try to send a diameter message to the server, I used to insert a certain amount of sleep time before I send the message after connecting to the server and it works ok. But I wanted to use the peer up callback to behave like a handshaking procedure among the client and the server, so the server is supposed to be ready when the client process sends the message. Now, the fact is that only one peer up callback is invoked for multiple processes. Then I tried to notify all the processes in the peer up callback when invoked and the processes send out the diameter messages. However, the processes don't receive any responses. Or sometimes only part of the processes get responses. Why inserting an amount of sleep time always works. This is a little bit confusing me, how peer up should be used properly to get the notification done right for multiple processes. Thanks for any suggestions! Samuel On Thu, Apr 25, 2013 at 5:40 AM, Anders Svensson wrote: > On Wed, Apr 24, 2013 at 5:23 PM, S X wrote: > > Thanks a lot for your comments. > > > > The diameter:call function allows to send extra arguments. It works fine > > after changing the callback function signatures accordingly. This is very > > useful when I want to notify the other process to do the next thing. > > > > Before sending a diameter message, we call diameter:add_transport to > connect > > to a peer and peform CER/CEA capability information exchange. On the > caller > > side, let's say the client callback module, the peer_up will be invoked > when > > CER/CEA is completed. Why it doesn't have the similar mechanism like > > diameter:call to allow insert additional arguments so we can utilize > them, > > for example, notify the others to send diameter messages? > > No particular reason aside from history and that the need hasn't come > up. As it is, you can pass arbitrary options to > diameter:add_transport/2 (history again) and retrieve these in a > callback with diameter:service_info(PeerRef), so that can be used a > substitute for extra arguments in this case. > > > diameter:add_transport is a sync call, but it doesn't mean you can send > > add_transport *doesn't* wait for the config it's passed to result in a > connection before returning. (It might never happen for one.) > > > diameter messages successfully when the function returns, i.e. it usually > > gets {error, no_connection} or { error, timeout }(this might because of > the > > server side) if you call diameter:call right after > diameter:add_transport. > > This is because the peer won't be ready to respond to requests until > capabilities exchange has completed (at least). If you call before the > relevant peer_up callback then the result is {error, no_connection}. > After peer_up the result will be {error, timeout} if the peer doesn't > answer. One case in which this is expected is after a connection has > been reestablished following an exchange of 3 x DWR/DWA. (ie. RFC 3539 > DOWN -> REOPEN -> OKAY.) Since both ends of the connection do this, > the client can consider the connection to be reestablished before the > server. If the client sends a request before the server is done with > it's exchange (ie. reached OKAY) then RFC 6733 says it should discard > the request, resulting in a timeout on the client end. > > > So is there any reason why not allowing to add additional arguments and > use > > them in peer_up callback function? Since it means capability exchange is > > done and the peer is ready, at this point it should be safe to send > diameter > > messages to the peer. > > See above. > > Anders > > > > > > Any suggestions? > > > > Thanks! > > > > Samuel > > > > > > On Tue, Apr 16, 2013 at 7:22 AM, Anders Svensson > > wrote: > >> > >> On Sun, Apr 14, 2013 at 7:01 PM, S X wrote: > >> > Hello, > >> > > >> > Based on the erlang diameter library and the sample code, I want to > >> > start > >> > multiple diameter client processes in one erlang node(one client IP), > >> > and > >> > the client needs to define a diameter_app callback module for certain > >> > application, for example: > >> > > >> > -define(SERVICE(Name), [{'Origin-Host', ?L(Name) ++ ".example.com"}, > >> > {'Origin-Realm', "example.com"}, > >> > {'Vendor-Id', 193}, > >> > {'Product-Name', "Client"}, > >> > {'Auth-Application-Id', > >> > [?DIAMETER_APP_ID_COMMON, > >> > ?DIAMETER_APP_ID_CCRA]}, > >> > {application, [{alias, ?APP_CCR_ALIAS}, > >> > {dictionary, > >> > ?DIAMETER_DICT_CCRA}, > >> > {module, client_cb_ccra}]}]). > >> > > >> > > >> > First question: > >> > how the diameter library handles this situation? Will all diameter > >> > client > >> > processes share one single diameter_app callback module > "client_cb_ccra" > >> > or > >> > it will automatically attach different instance of the callback module > >> > (process) to the different client process by using spawn_monitor? So > >> > from > >> > the callback handle_answer in "client_cb_ccra" I can notify the proper > >> > client proce by just calling client:notify() something. > >> > >> diameter doesn't know anything about your client process. If you want > >> a callback to be able to contact the client process associated with > >> the service in question (ie. the service whose name the callback gets > >> as an argument) then you need to give the callback the means to do so. > >> One way would be to map the service name to your process (eg. it's the > >> registered name of your client process), another would be to pass some > >> identification as extra arguments to the callbacks. (Eg. {module, > >> [client_cb_ccra, X]} in the config above.) > >> > >> > Second question: > >> > Notice that diameter:call allows to set extra arguments, so I was > >> > wondering > >> > I could set some data like: > >> > diameter:call(Name, ?APP_CCR_ALIAS, CCR, [{extra, [self()]}]). > >> > which sets the client process ID and I hope to deliver to the callback > >> > module "client_cb_ccra" and when the callback module knows which > client > >> > process sends request and response accordingly, for example using the > >> > client > >> > process ID in the callback function handle_answer. > >> > >> You can do that, but what is it you're trying to accomplish? if it's > >> handle_answer that's supposed to communicate something then it already > >> does: the return value of handle_answer is returned by diameter:call. > >> > >> > However, I don't want pack the extra arguments into the diameter > packet > >> > since the diameter server doesn't know what they are and the extra > ones > >> > are > >> > not part of standard diameter packet. > >> > >> Not sure what you mean here. There's no way for your client to send > >> the server anything other than a Diameter message. > >> > >> > Now I changed the pick_peer callback signatures to allow extra > >> > arguments, > >> > pick_peer([Peer | _], _, _SvcName, _State, A) > >> > > >> > But I got encoding error in the callback prepare_request > >> > > >> > =ERROR REPORT==== 14-Apr-2013::11:40:41 === > >> > Error in process <0.175.0> with exit value: > >> > {undef,[{client_cb_ccra,prepare_request,[{diameter_packet, > >> > >> The arity of your callback probably doesn't agree with the number of > >> extra arguments you've specified: prepare_request will also get your > >> extra arguments. > >> > >> > >> > > >{diameter_header,1,undefined,undefined,undefined,3958849953,3958849953,undefined,undefined,undefined,undefined},undefined,{diameter_rfc4006_cc_CCR,["who",";","142745567... > >> > > >> > {error,encode} > >> > > >> > In overall, I read the online documents, which have limited > information > >> > on > >> > how to use the extra arguments in the library and don't quite get how > >> > to > >> > utilize the extra arguments to do something tricky, > >> > > >> > Are there any suggestions on how to deal with multiple client > processes? > >> > >> Not sure I understand what problem it is you're trying to solve. > >> Multiple processes invoking diameter:call is nothing strange. You > >> typically just return something useful (eg. the answer message from > >> the peer) from handle_answer, which diameter:call then returns for the > >> caller to deal with. > >> > >> Anders > >> > >> > >> > Thanks a lot! > >> > > >> > Samuel > >> > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gumm@REDACTED Tue Apr 30 07:09:50 2013 From: gumm@REDACTED (Jesse Gumm) Date: Tue, 30 Apr 2013 00:09:50 -0500 Subject: [erlang-questions] [ANN] qdate - Unified Date Formatting/Parsing and Timezone Management Message-ID: Hello Erlangers, I'm proud to announce the beta release (v0.1.0) of "qdate" - a unified date parsing/formatting and timezone management utility for Erlang. Blog post describing my thoughts that went into its development http://sigma-star.com/blog/post/qdate Source code on GitHub (includes a README that covers all the functionality) https://github.com/choptastic/qdate At its core, qdate is basically a wrapper around ec_date (fork of dh_date and part of erlware_commons) and erlang_localtime. But it's much more than just a wrapper. It's the complete date and time utility I feel is greatly needed in the Erlang world (since I found myself needing it). Briefly, here's what it does: * Easily convert from any date format to any other date format: datetime(), now(), unix timestamp, and formatted date strings (list and binaries) are all acceptable: > qdate:to_string("Y-m-d g:ia", Date). "2012-12-21 5:51pm" > qdate:to_string(<<"Y-m-d g:ia">>, Date). <<"2012-12-21 5:51pm">> > qdate:to_unixtime(Date). 1356112260 > qdate:to_date(Date). {{2012,12,21},{17,51,0}} > qdate:to_now(Date). {1356,112260,0} % Note, that the argument `Date` can be anything % that's a Date/Time Format (datetime Tuple, now tuple, % Integer (unixtime), or string ("2012-12-21 5:51pm") * Extends ec_date's formatting functions to include PHP's timezone-related characters: > qdate:to_string("Y-m-d g:ia T", Date) "2012-12-21 5:51pm CST" * Able to pre-parse timezones from text strings, and convert from one timezone to another: % Read Timezone from string, convert to datetime in EST > qdate:to_date("2012-12-21 5:51pm CST","EST"). {{2012,12,21},{18,51,0}} % Read timezone from string, convert to a string in EST > qdate:to_string("n/j/Y g:ia T","2012-12-21 5:51pm CST", "EST"). "12/21/2012 6:51pm EST" * Register timezones with the qdate server by a key (to make a timezone alias) or by pid(): %% Set timezone for the current process to GMT > qdate:set_timezone("GMT"). %% Set timezone for some_key to HKT > qdate:set_timezone(some_key, "HKT"). * Register custom formatting strings with the qdate server, for quick reference. > qdate:register_format(long_date, "l, F jS, Y g:i A T"). ok > qdate:to_string(long_date,1356133860). "Friday, December 21st, 2012 11:51 PM GMT" * Register custom parsing functions with the qdate server, to be transparently used for conversion. > qdate:register_parser(weird_format, fun parse_esoteric_format/1). ok > qdate:to_date("20121221.115100"). {{2012,12,21},{11,51,00}} * Attempt to maintain API compatibility with ec_date by supporting qdate:parse, qdate:nparse, and qdate:format. I hope some of you find this useful in your apps. Any issues, comments, complaints, bug reports, whatever, feel free to post in this thread, make a github issue, shoot me a message on twitter (@jessegumm) or email me directly. Thanks, and I hope everyone has a glorious day or evening or whatever timezone you happen to be in. -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitrii@REDACTED Tue Apr 30 08:50:01 2013 From: dmitrii@REDACTED (Dmitrii Dimandt) Date: Tue, 30 Apr 2013 08:50:01 +0200 Subject: [erlang-questions] [ANN] qdate - Unified Date Formatting/Parsing and Timezone Management In-Reply-To: References: Message-ID: <0E6ACAB0-6A5B-4C85-84E2-CC7B590317B6@dmitriid.com> Have you also looked at tempo, https://github.com/selectel/tempo ? Your library and tempo could probably be combined On Apr 30, 2013, at 07:09 , Jesse Gumm wrote: > Hello Erlangers, > > I'm proud to announce the beta release (v0.1.0) of "qdate" - a unified date parsing/formatting and timezone management utility for Erlang. > > Blog post describing my thoughts that went into its development > http://sigma-star.com/blog/post/qdate > > Source code on GitHub (includes a README that covers all the functionality) > https://github.com/choptastic/qdate > > > At its core, qdate is basically a wrapper around ec_date (fork of dh_date and part of erlware_commons) and erlang_localtime. > > But it's much more than just a wrapper. It's the complete date and time utility I feel is greatly needed in the Erlang world (since I found myself needing it). > > Briefly, here's what it does: > > * Easily convert from any date format to any other date format: datetime(), now(), unix timestamp, and formatted date strings (list and binaries) are all acceptable: > > qdate:to_string("Y-m-d g:ia", Date). > "2012-12-21 5:51pm" > > qdate:to_string(<<"Y-m-d g:ia">>, Date). > <<"2012-12-21 5:51pm">> > > qdate:to_unixtime(Date). > 1356112260 > > qdate:to_date(Date). > {{2012,12,21},{17,51,0}} > > qdate:to_now(Date). > {1356,112260,0} > % Note, that the argument `Date` can be anything > % that's a Date/Time Format (datetime Tuple, now tuple, > % Integer (unixtime), or string ("2012-12-21 5:51pm") > > * Extends ec_date's formatting functions to include PHP's timezone-related characters: > > qdate:to_string("Y-m-d g:ia T", Date) > "2012-12-21 5:51pm CST" > > * Able to pre-parse timezones from text strings, and convert from one timezone to another: > % Read Timezone from string, convert to datetime in EST > > qdate:to_date("2012-12-21 5:51pm CST","EST"). > {{2012,12,21},{18,51,0}} > % Read timezone from string, convert to a string in EST > > qdate:to_string("n/j/Y g:ia T","2012-12-21 5:51pm CST", "EST"). > "12/21/2012 6:51pm EST" > > * Register timezones with the qdate server by a key (to make a timezone alias) or by pid(): > %% Set timezone for the current process to GMT > > qdate:set_timezone("GMT"). > %% Set timezone for some_key to HKT > > qdate:set_timezone(some_key, "HKT"). > > * Register custom formatting strings with the qdate server, for quick reference. > > qdate:register_format(long_date, "l, F jS, Y g:i A T"). > ok > > qdate:to_string(long_date,1356133860). > "Friday, December 21st, 2012 11:51 PM GMT" > > * Register custom parsing functions with the qdate server, to be transparently used for conversion. > > qdate:register_parser(weird_format, fun parse_esoteric_format/1). > ok > > qdate:to_date("20121221.115100"). > {{2012,12,21},{11,51,00}} > > * Attempt to maintain API compatibility with ec_date by supporting qdate:parse, qdate:nparse, and qdate:format. > > > I hope some of you find this useful in your apps. > > Any issues, comments, complaints, bug reports, whatever, feel free to post in this thread, make a github issue, shoot me a message on twitter (@jessegumm) or email me directly. > > Thanks, and I hope everyone has a glorious day or evening or whatever timezone you happen to be in. > > -- > Jesse Gumm > Owner, Sigma Star Systems > 414.940.4866 || sigma-star.com || @jessegumm > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From anders.otp@REDACTED Tue Apr 30 11:04:57 2013 From: anders.otp@REDACTED (Anders Svensson) Date: Tue, 30 Apr 2013 11:04:57 +0200 Subject: [erlang-questions] diameter callback module In-Reply-To: References: Message-ID: Hi Samuel. On Tue, Apr 30, 2013 at 4:35 AM, S X wrote: > Hi Anders, > > Thanks for your explanation. Setting additional options in > diameter:add_transport and retrieving them with diameter:service_info from > the peer_up callback work well. And the descriptions about the peer states > according to RFC3539 and 6733 make a lot of sense. > > The case I am trying to resolve is: > Start multiple processes on one pc, each process tries to connect to one > same server (obviously I will get already_connected warning, but it can keep > going), in each process, then I try to send a diameter message to the > server, I used to insert a certain amount of sleep time before I send the > message after connecting to the server and it works ok. But I wanted to use > the peer up callback to behave like a handshaking procedure among the client > and the server, so the server is supposed to be ready when the client > process sends the message. > > Now, the fact is that only one peer up callback is invoked for multiple > processes. Then I tried to notify all the processes in the peer up callback > when invoked and the processes send out the diameter messages. However, the > processes don't receive any responses. Or sometimes only part of the > processes get responses. Why inserting an amount of sleep time always works. > > This is a little bit confusing me, how peer up should be used properly to > get the notification done right for multiple processes. It's difficult to say what your problem is without seeing code but here are a few points. It's not your client processes that connect to the server. A transport connection results from someone calling diameter:add_transport/2. Once diameter establishes the connection, successfully performs capabilities exchange, and makes peer_up callbacks, that transport connection is available to any process that wants to send to the peer in question. In your case, you have multiple processes that want to do so, but if you're calling diameter:add_transport/2 with the same config from each of those processes then you're effectively telling diameter to establish multiple transport connections, only one of which will succeed by default. If you want to allow multiple transport connections per peer then you need to configure your service with {restrict_connections, false}, but the peer need not support this (see 2.1 in RFC 6733) so may reject all but one CER. Without restrict_connections it's diameter that disallows multiple connections. Given that you want multiple client processes to send from (not wrong, but why?), I would treat these as workers that have nothing to do with configuration. That is, call diameter:start_service/2 and diameter:add_transport/2 from elsewhere and have your workers kick into action after capabilities exchange. Like you said, your workers want to know when the peer becomes available. There are two ways to get notification: a peer_up callback or a diameter event. The latter are subscribed to with diameter:subscribe/1 so one way is for each worker to subscribe to events and react to them. If you want to use peer_up as a trigger then you need to code some way for the callback to notify your worker processes. There are plenty of ways to do this, none of which are diameter-specific. For example: maintain your worker pids in an ets table that the callback examines; have workers register with a registered gen_server that finds out about callbacks; spawn workers as a result of peer_up. Sleep is never a reliable solution. Anders > > Thanks for any suggestions! > > Samuel > > > > > On Thu, Apr 25, 2013 at 5:40 AM, Anders Svensson > wrote: >> >> On Wed, Apr 24, 2013 at 5:23 PM, S X wrote: >> > Thanks a lot for your comments. >> > >> > The diameter:call function allows to send extra arguments. It works fine >> > after changing the callback function signatures accordingly. This is >> > very >> > useful when I want to notify the other process to do the next thing. >> > >> > Before sending a diameter message, we call diameter:add_transport to >> > connect >> > to a peer and peform CER/CEA capability information exchange. On the >> > caller >> > side, let's say the client callback module, the peer_up will be invoked >> > when >> > CER/CEA is completed. Why it doesn't have the similar mechanism like >> > diameter:call to allow insert additional arguments so we can utilize >> > them, >> > for example, notify the others to send diameter messages? >> >> No particular reason aside from history and that the need hasn't come >> up. As it is, you can pass arbitrary options to >> diameter:add_transport/2 (history again) and retrieve these in a >> callback with diameter:service_info(PeerRef), so that can be used a >> substitute for extra arguments in this case. >> >> > diameter:add_transport is a sync call, but it doesn't mean you can send >> >> add_transport *doesn't* wait for the config it's passed to result in a >> connection before returning. (It might never happen for one.) >> >> > diameter messages successfully when the function returns, i.e. it >> > usually >> > gets {error, no_connection} or { error, timeout }(this might because of >> > the >> > server side) if you call diameter:call right after >> > diameter:add_transport. >> >> This is because the peer won't be ready to respond to requests until >> capabilities exchange has completed (at least). If you call before the >> relevant peer_up callback then the result is {error, no_connection}. >> After peer_up the result will be {error, timeout} if the peer doesn't >> answer. One case in which this is expected is after a connection has >> been reestablished following an exchange of 3 x DWR/DWA. (ie. RFC 3539 >> DOWN -> REOPEN -> OKAY.) Since both ends of the connection do this, >> the client can consider the connection to be reestablished before the >> server. If the client sends a request before the server is done with >> it's exchange (ie. reached OKAY) then RFC 6733 says it should discard >> the request, resulting in a timeout on the client end. >> >> > So is there any reason why not allowing to add additional arguments and >> > use >> > them in peer_up callback function? Since it means capability exchange is >> > done and the peer is ready, at this point it should be safe to send >> > diameter >> > messages to the peer. >> >> See above. >> >> Anders >> >> >> > >> > Any suggestions? >> > >> > Thanks! >> > >> > Samuel >> > >> > >> > On Tue, Apr 16, 2013 at 7:22 AM, Anders Svensson >> > wrote: >> >> >> >> On Sun, Apr 14, 2013 at 7:01 PM, S X wrote: >> >> > Hello, >> >> > >> >> > Based on the erlang diameter library and the sample code, I want to >> >> > start >> >> > multiple diameter client processes in one erlang node(one client IP), >> >> > and >> >> > the client needs to define a diameter_app callback module for certain >> >> > application, for example: >> >> > >> >> > -define(SERVICE(Name), [{'Origin-Host', ?L(Name) ++ ".example.com"}, >> >> > {'Origin-Realm', "example.com"}, >> >> > {'Vendor-Id', 193}, >> >> > {'Product-Name', "Client"}, >> >> > {'Auth-Application-Id', >> >> > [?DIAMETER_APP_ID_COMMON, >> >> > ?DIAMETER_APP_ID_CCRA]}, >> >> > {application, [{alias, ?APP_CCR_ALIAS}, >> >> > {dictionary, >> >> > ?DIAMETER_DICT_CCRA}, >> >> > {module, client_cb_ccra}]}]). >> >> > >> >> > >> >> > First question: >> >> > how the diameter library handles this situation? Will all diameter >> >> > client >> >> > processes share one single diameter_app callback module >> >> > "client_cb_ccra" >> >> > or >> >> > it will automatically attach different instance of the callback >> >> > module >> >> > (process) to the different client process by using spawn_monitor? So >> >> > from >> >> > the callback handle_answer in "client_cb_ccra" I can notify the >> >> > proper >> >> > client proce by just calling client:notify() something. >> >> >> >> diameter doesn't know anything about your client process. If you want >> >> a callback to be able to contact the client process associated with >> >> the service in question (ie. the service whose name the callback gets >> >> as an argument) then you need to give the callback the means to do so. >> >> One way would be to map the service name to your process (eg. it's the >> >> registered name of your client process), another would be to pass some >> >> identification as extra arguments to the callbacks. (Eg. {module, >> >> [client_cb_ccra, X]} in the config above.) >> >> >> >> > Second question: >> >> > Notice that diameter:call allows to set extra arguments, so I was >> >> > wondering >> >> > I could set some data like: >> >> > diameter:call(Name, ?APP_CCR_ALIAS, CCR, [{extra, [self()]}]). >> >> > which sets the client process ID and I hope to deliver to the >> >> > callback >> >> > module "client_cb_ccra" and when the callback module knows which >> >> > client >> >> > process sends request and response accordingly, for example using the >> >> > client >> >> > process ID in the callback function handle_answer. >> >> >> >> You can do that, but what is it you're trying to accomplish? if it's >> >> handle_answer that's supposed to communicate something then it already >> >> does: the return value of handle_answer is returned by diameter:call. >> >> >> >> > However, I don't want pack the extra arguments into the diameter >> >> > packet >> >> > since the diameter server doesn't know what they are and the extra >> >> > ones >> >> > are >> >> > not part of standard diameter packet. >> >> >> >> Not sure what you mean here. There's no way for your client to send >> >> the server anything other than a Diameter message. >> >> >> >> > Now I changed the pick_peer callback signatures to allow extra >> >> > arguments, >> >> > pick_peer([Peer | _], _, _SvcName, _State, A) >> >> > >> >> > But I got encoding error in the callback prepare_request >> >> > >> >> > =ERROR REPORT==== 14-Apr-2013::11:40:41 === >> >> > Error in process <0.175.0> with exit value: >> >> > {undef,[{client_cb_ccra,prepare_request,[{diameter_packet, >> >> >> >> The arity of your callback probably doesn't agree with the number of >> >> extra arguments you've specified: prepare_request will also get your >> >> extra arguments. >> >> >> >> >> >> > >> >> > > >{diameter_header,1,undefined,undefined,undefined,3958849953,3958849953,undefined,undefined,undefined,undefined},undefined,{diameter_rfc4006_cc_CCR,["who",";","142745567... >> >> > >> >> > {error,encode} >> >> > >> >> > In overall, I read the online documents, which have limited >> >> > information >> >> > on >> >> > how to use the extra arguments in the library and don't quite get >> >> > how >> >> > to >> >> > utilize the extra arguments to do something tricky, >> >> > >> >> > Are there any suggestions on how to deal with multiple client >> >> > processes? >> >> >> >> Not sure I understand what problem it is you're trying to solve. >> >> Multiple processes invoking diameter:call is nothing strange. You >> >> typically just return something useful (eg. the answer message from >> >> the peer) from handle_answer, which diameter:call then returns for the >> >> caller to deal with. >> >> >> >> Anders >> >> >> >> >> >> > Thanks a lot! >> >> > >> >> > Samuel >> >> > >> > >> > > > From jesper.louis.andersen@REDACTED Tue Apr 30 12:38:20 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 30 Apr 2013 12:38:20 +0200 Subject: [erlang-questions] Static type checking... In-Reply-To: <6AB510F1-B600-47E3-98B6-108230D507D2@gmail.com> References: <6AB510F1-B600-47E3-98B6-108230D507D2@gmail.com> Message-ID: On Apr 30, 2013, at 2:40 AM, Steve Davis wrote: > It's my experience that attempts at static type checks ends up in "turtles all the way down" and enormously onerous code. Static typing, preferably with inference of the types, is a really nice tool for finding and killing bugs early. But interestingly, there is not that much formality when it comes to checking the *protocol* between two processes. Since Erlang processes are often simple internally, it becomes way more interesting to have a formal protocol checker to verify adherence. I don't care what you are doing internally in your process. You could write it in any language you desired, as long as you adhere to the protocol we laid out on top. There is a similarity to having a signature on a module specifying a skeleton of how that code will execute type-wise. There are two things needed with a validation layer: It has to be effective. It needs to catch bugs. It needs to remove bugs before they wreak havoc on the system. It has to be simple. We can't afford the layer to take up considerable human resources to specify a protocol contract. Otherwise, we end up wasting time. If you are going to pay time up-front, you better know that you find errors later on in the process. Otherwise, that time was just wasted. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen From stamarit@REDACTED Tue Apr 30 13:40:06 2013 From: stamarit@REDACTED (Salvador Tamarit) Date: Tue, 30 Apr 2013 13:40:06 +0200 Subject: [erlang-questions] Managing a huge binary file In-Reply-To: <52ADF12A-AF3E-43E6-BCC5-B02CB891F0C4@cs.otago.ac.nz> References: <52ADF12A-AF3E-43E6-BCC5-B02CB891F0C4@cs.otago.ac.nz> Message-ID: 2013/4/30 Richard A. O'Keefe : > > On 30/04/2013, at 1:23 PM, Salvador Tamarit wrote: > >> Suppose that we have a huge binary file (8GB or so) containing all the >> integers from 0 to a given (very big) number. The numbers are unsorted >> and some of them (70 numbers for instance) has been removed. > > This sounds like a well known problem with an answer in > http://books.google.co.nz/books?id=415loiMd_c0C&lpg=PP1&dq=muthukrishnan+data+stream+algorithms&hl=el&pg=PA1&redir_esc=y > That's great. I was suspicious that it existed, but I didn't know how to find it :P Thank you very much Richard. > with the twist that you are asked to find a specific missing number > (in value order) rather than all of them. > > When you talk about reading the numbers 4 bytes at a time, that tells > me the numbers are not >that< "huge". > > One very simple step would be to set up > number-present : map (0..6535) -> nat := (all -> 0)_ > for each number in the input > number-present(high 16 bits of that number) +:= 1 > Now you can tell which block of numbers must contain the > number you are looking for, and you can make a second pass > through the data counting numbers in that block, and then > it's a simple scan through the block. It would solve the problem of being able to load the data into memory, but I still having problems reading the whole file. It takes too much time. I know my function is very naive, however I'm not sure how could I improve it. Only reading integers from file, without further processing, takes a lot of time (minutes) in a fast laptop. read_integers(FDevice) -> case file:read(FDevice, 4) of {ok,<<_:2/little-unit:8,Readed:2/little-unit:8>>} -> read_integers(FDevice); eof -> ok end. > > So if the numbers are 0..N-1, the problem can be solved in > O(N**1/2) space in two passes, or in O(N**1/3) space in > three passes, &c. > > This is _not_ mathematically optimal, but it's quite easy to > program, and if you really had this problem, I'd rather have > the data read twice by a simple program I trusted than once > by a clever program that was almost certainly wrong. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From hobson42@REDACTED Tue Apr 30 13:41:10 2013 From: hobson42@REDACTED (Ian) Date: Tue, 30 Apr 2013 12:41:10 +0100 Subject: [erlang-questions] Managing a huge binary file In-Reply-To: References: Message-ID: <517FADD6.8030006@gmail.com> Hi Salvador, On 30/04/2013 02:23, Salvador Tamarit wrote: > hi all, > > I have a very interesting problem for us :) > > Suppose that we have a huge binary file (8GB or so) containing all the > integers from 0 to a given (very big) number. The numbers are unsorted > and some of them (70 numbers for instance) has been removed. > > The goal is to find the the Nth removed number (in order). For > instance, if the file contains the numbers from 0 to 99, and then > number 101, the 1st removed would be 100. And the 100th number would be 101, so you can calculate that 1 number is missing. (101 - 100) I would approach this with a binary chop. See http://en.wikipedia.org/wiki/Binary_search_algorithm Lets say you want the Nth missing number. Your key is in two parts - most significant is the number of missing numbers, next in significance is the number itself. Binary search for (N,0) - and when your range shrinks to 1, it will point to the smallest number with N or more missing Lets call this Q. Now read the previous number (P). The range of values from P+1 to Q-1 will include the Nth value. Simple calculation will determine which you want. (The breaks may not always be single numbers). Regards Ian > > I have been trying to solve this problem with different solutions, but > I always get stucked in reading the file. I'm reading it sequentally. > I open the file with: > > file:open(FileName,[binary]) > > then I read each number with: > > file:read(FDevice, 4) > > this works fine, but too slow. > > Anyway, supposing that I would be able to read it, then I don't know > how to manage the data (I think that it is impossible to load all this > data in memory). > > How would you solve a problem like this? > > Thanks, > Salva > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > . > From jesper.louis.andersen@REDACTED Tue Apr 30 13:42:22 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 30 Apr 2013 13:42:22 +0200 Subject: [erlang-questions] Managing a huge binary file In-Reply-To: References: Message-ID: <35BB45F5-469D-4090-9133-ADB98EF1CAE3@erlang-solutions.com> On Apr 30, 2013, at 3:23 AM, Salvador Tamarit wrote: > Suppose that we have a huge binary file (8GB or so) containing all the > integers from 0 to a given (very big) number. The numbers are unsorted > and some of them (70 numbers for instance) has been removed. > > The goal is to find the the Nth removed number (in order). For > instance, if the file contains the numbers from 0 to 99, and then > number 101, the 1st removed would be 100. Quick shot at a possible solution: Variant of a Fenwick-tree. Essentially, the idea is to count bit-patterns in the numbers and let each bit correspond to a slot in an array. Now you can "count" how many times a given pattern is there. With a bit of mangling, it may be possible to walk over all the data building up the counts in the array and then finally scrutinize it for 0'es and then work backwards what missing pattern you have. Another variant: Radix tree or trie structure on bit-patterns since this greatly compresses the in-memory representation. Hash Array Mapped Tries comes to mind here. You can easily walk it in order to determine "holes" in the structure afterwards. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen From magnus@REDACTED Tue Apr 30 13:56:38 2013 From: magnus@REDACTED (Magnus Henoch) Date: Tue, 30 Apr 2013 12:56:38 +0100 Subject: [erlang-questions] Managing a huge binary file In-Reply-To: (Salvador Tamarit's message of "Tue, 30 Apr 2013 03:23:18 +0200") References: Message-ID: Salvador Tamarit writes: > I have been trying to solve this problem with different solutions, but > I always get stucked in reading the file. I'm reading it sequentally. > I open the file with: > > file:open(FileName,[binary]) > > then I read each number with: > > file:read(FDevice, 4) > > this works fine, but too slow. Try opening the file with: file:open(FileName,[binary, raw, read_ahead]) I just wrote a little escript that reads four bytes at a time, and adding 'raw' and 'read_ahead' seems to make it twice as fast (1.5 s vs 3 s for a 2 MB file). Interestingly, read_ahead alone didn't make a difference, and raw alone actually made it slower. Regards, Magnus From stamarit@REDACTED Tue Apr 30 14:06:21 2013 From: stamarit@REDACTED (Salvador Tamarit) Date: Tue, 30 Apr 2013 14:06:21 +0200 Subject: [erlang-questions] Managing a huge binary file In-Reply-To: <35BB45F5-469D-4090-9133-ADB98EF1CAE3@erlang-solutions.com> References: <35BB45F5-469D-4090-9133-ADB98EF1CAE3@erlang-solutions.com> Message-ID: 2013/4/30 Jesper Louis Andersen : > > On Apr 30, 2013, at 3:23 AM, Salvador Tamarit wrote: > >> Suppose that we have a huge binary file (8GB or so) containing all the >> integers from 0 to a given (very big) number. The numbers are unsorted >> and some of them (70 numbers for instance) has been removed. >> >> The goal is to find the the Nth removed number (in order). For >> instance, if the file contains the numbers from 0 to 99, and then >> number 101, the 1st removed would be 100. > > Quick shot at a possible solution: Variant of a Fenwick-tree. > > Essentially, the idea is to count bit-patterns in the numbers and let each bit correspond to a slot in an array. Now you can "count" how many times a given pattern is there. With a bit of mangling, it may be possible to walk over all the data building up the counts in the array and then finally scrutinize it for 0'es and then work backwards what missing pattern you have. > > Another variant: Radix tree or trie structure on bit-patterns since this greatly compresses the in-memory representation. Hash Array Mapped Tries comes to mind here. You can easily walk it in order to determine "holes" in the structure afterwards. > Both solutions seems very interesting. Thanks Jesper and Ian for your proposals. However, I can't load the file (in a reasonable time) because it's too big, so your solutions will be useful for my second step :) > Jesper Louis Andersen > Erlang Solutions Ltd., Copenhagen > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From stamarit@REDACTED Tue Apr 30 14:08:55 2013 From: stamarit@REDACTED (Salvador Tamarit) Date: Tue, 30 Apr 2013 14:08:55 +0200 Subject: [erlang-questions] Managing a huge binary file In-Reply-To: References: Message-ID: 2013/4/30 Magnus Henoch : > Salvador Tamarit writes: > >> I have been trying to solve this problem with different solutions, but >> I always get stucked in reading the file. I'm reading it sequentally. >> I open the file with: >> >> file:open(FileName,[binary]) >> >> then I read each number with: >> >> file:read(FDevice, 4) >> >> this works fine, but too slow. > > Try opening the file with: > > file:open(FileName,[binary, raw, read_ahead]) > > I just wrote a little escript that reads four bytes at a time, and > adding 'raw' and 'read_ahead' seems to make it twice as fast (1.5 s vs 3 > s for a 2 MB file). Interestingly, read_ahead alone didn't make a > difference, and raw alone actually made it slower. Twice fast only changing these options... :o Unfortunately, It still take a lot of time to read the file :P > > Regards, > Magnus > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dmkolesnikov@REDACTED Tue Apr 30 14:16:20 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Tue, 30 Apr 2013 15:16:20 +0300 Subject: [erlang-questions] Managing a huge binary file In-Reply-To: References: <35BB45F5-469D-4090-9133-ADB98EF1CAE3@erlang-solutions.com> Message-ID: <6D83C68E-F9A5-4C9D-8267-FC4257970510@gmail.com> Hello, Performance of you solution is bound on disk I/O. Just to validate here, are you able to consume data with desirable speed from the storage using other run-time e.g. C? Open the file in raw mode with read ahead flag should be sufficient for many use-cases (e.g. I am using that method for late CSV intake and the speed is acceptable). What is an average line per second you are getting in you application? Best Reagrads, Dmitry On Apr 30, 2013, at 3:06 PM, Salvador Tamarit wrote: > 2013/4/30 Jesper Louis Andersen : >> >> On Apr 30, 2013, at 3:23 AM, Salvador Tamarit wrote: >> >>> Suppose that we have a huge binary file (8GB or so) containing all the >>> integers from 0 to a given (very big) number. The numbers are unsorted >>> and some of them (70 numbers for instance) has been removed. >>> >>> The goal is to find the the Nth removed number (in order). For >>> instance, if the file contains the numbers from 0 to 99, and then >>> number 101, the 1st removed would be 100. >> >> Quick shot at a possible solution: Variant of a Fenwick-tree. >> >> Essentially, the idea is to count bit-patterns in the numbers and let each bit correspond to a slot in an array. Now you can "count" how many times a given pattern is there. With a bit of mangling, it may be possible to walk over all the data building up the counts in the array and then finally scrutinize it for 0'es and then work backwards what missing pattern you have. >> >> Another variant: Radix tree or trie structure on bit-patterns since this greatly compresses the in-memory representation. Hash Array Mapped Tries comes to mind here. You can easily walk it in order to determine "holes" in the structure afterwards. >> > > Both solutions seems very interesting. Thanks Jesper and Ian for your > proposals. However, I can't load the file (in a reasonable time) > because it's too big, so your solutions will be useful for my second > step :) > >> Jesper Louis Andersen >> Erlang Solutions Ltd., Copenhagen >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jon@REDACTED Tue Apr 30 14:21:59 2013 From: jon@REDACTED (Jon Schneider) Date: Tue, 30 Apr 2013 13:21:59 +0100 Subject: [erlang-questions] Managing a huge binary file In-Reply-To: <6D83C68E-F9A5-4C9D-8267-FC4257970510@gmail.com> References: <35BB45F5-469D-4090-9133-ADB98EF1CAE3@erlang-solutions.com> <6D83C68E-F9A5-4C9D-8267-FC4257970510@gmail.com> Message-ID: <984543097eb1806736dc15891accf728.squirrel@mail.jschneider.net> You shouldn't read it in as such anyway. If it were me and C and I had stacks of RAM C I'd simply mmap it. However you _might_ get better performance if the readahead done by the kernel and the device is less than what you might read() using plan A. A modernish disk should give you 100MB/s sustained (assuming no significant fragmentation and contrary to popular belief even on SSDs fragmentation does have a cost). Jon > Hello, > > Performance of you solution is bound on disk I/O. > Just to validate here, are you able to consume data with desirable speed > from the storage using other run-time e.g. C? From max.lapshin@REDACTED Tue Apr 30 14:32:12 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 30 Apr 2013 16:32:12 +0400 Subject: [erlang-questions] Managing a huge binary file In-Reply-To: <984543097eb1806736dc15891accf728.squirrel@mail.jschneider.net> References: <35BB45F5-469D-4090-9133-ADB98EF1CAE3@erlang-solutions.com> <6D83C68E-F9A5-4C9D-8267-FC4257970510@gmail.com> <984543097eb1806736dc15891accf728.squirrel@mail.jschneider.net> Message-ID: It is a myth that mmap helps you to read file faster. I've believed in this myth and packed mmap.erl with erlyvideo. But later it happened that 3 Gbps are served from disk with plain file:read and mmap is MUCH worser because it breaks scheduling. So, file:read can give you several gbps. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon@REDACTED Tue Apr 30 14:37:43 2013 From: jon@REDACTED (Jon Schneider) Date: Tue, 30 Apr 2013 13:37:43 +0100 Subject: [erlang-questions] Managing a huge binary file In-Reply-To: References: <35BB45F5-469D-4090-9133-ADB98EF1CAE3@erlang-solutions.com> <6D83C68E-F9A5-4C9D-8267-FC4257970510@gmail.com> <984543097eb1806736dc15891accf728.squirrel@mail.jschneider.net> Message-ID: <394127255a7fc56053f80ba7ad90dd8d.squirrel@mail.jschneider.net> > It is a myth that mmap helps you to read file faster. I can quite believe that but it does save you wasting memory by duplicating data. Interestingly last time I looked SunOS's implementation of cat used mmap (for a file obviously). Jon From sverker.eriksson@REDACTED Tue Apr 30 14:38:20 2013 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Tue, 30 Apr 2013 14:38:20 +0200 Subject: [erlang-questions] Segfault in NIF call In-Reply-To: References: <3da9e2e4-e1fc-4187-9573-faed5e1c249d@googlegroups.com> <517E4441.2040202@erix.ericsson.se> Message-ID: <517FBB3C.8060700@erix.ericsson.se> You wrote: "The NIF:ed function is called from a few different processes, and it forwards the call to a worker thread." Have you done proper thread synchronization for this inter thread communication (with enif_mutex_lock() for example)? NIFs are only thread safe as long as they access their own environment. Shared memory structures such as a queue for a worker thread must be protected with locks (or implemented with a lock-free algorithm). Another approach is to see if valgrind can detect any memory corruptions: # cd $ERL_TOP/erts/emulator # make TYPE=valgrind smp plain # export VALGRIND_LOG_DIR=/write/valgrind/logs/here # export VALGRIND_MISC_FLAGS="--suppressions=$ERL_TOP/erts/emulator/valgrind/suppress.standard --show-possibly-lost=no" # $ERL_TOP/bin/cerl -valgrind /Sverker Fredrik Linder wrote: > Found one error (yay!) -- enif_open_resource_type should have NULL as the > module argument (missed this one in the docs) > > Running with cerl does not generate a segfault, nor any (extra) printouts > Running with erl still generate segfault > > Now I get either of the following (using erl), does it reveal what my error > is? > > >> USE_GDB=1 rebar eunit >> > [cut] > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x2aaaad6c0700 (LWP 6384)] > 0x000000000052b28a in process_main () at > x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:979 > 979 MoveDeallocateReturn(xb(tmp_packed1&BEAM_LOOSE_MASK), r(0), > Qb((tmp_packed1>>BEAM_LOOSE_SHIFT))); > (gdb) backtrace > #0 0x000000000052b28a in process_main () at > x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:979 > #1 0x0000000000491463 in sched_thread_func (vesdp=0x2aaaac343ac0) at > beam/erl_process.c:5632 > #2 0x0000000000590440 in thr_wrapper (vtwd=0x7fffffffd850) at > pthread/ethread.c:106 > #3 0x00002aaaab3ffe9a in start_thread (arg=0x2aaaad6c0700) at > pthread_create.c:308 > #4 0x00002aaaab911cbd in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 > #5 0x0000000000000000 in ?? () > > or > > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x2aaaad6c0700 (LWP 6914)] > unlink_free_block (allctr=0x85e5c0, block=0x0, flags=0) at > beam/erl_goodfit_alloc.c:458 > 458 Uint sz = MBC_FBLK_SZ(&blk->block_head); > (gdb) backtrace > #0 unlink_free_block (allctr=0x85e5c0, block=0x0, flags=0) at > beam/erl_goodfit_alloc.c:458 > #1 0x0000000000444ee1 in get_free_block (allctr=0x85e5c0, size= out>, cand_blk=0x0, cand_size=0, flags=0) at beam/erl_goodfit_alloc.c:426 > #2 0x000000000043b6da in mbc_alloc_block (alcu_flgsp=, > blk_szp=, size=, allctr=0x85e5c0) at > beam/erl_alloc_util.c:1309 > #3 mbc_alloc (allctr=0x85e5c0, size=) at > beam/erl_alloc_util.c:1451 > #4 0x0000000000440d7b in do_erts_alcu_alloc (size=32, extra=0x85e5c0, > type=148) at beam/erl_alloc_util.c:3530 > #5 erts_alcu_alloc_thr_pref (type=148, extra=, > size=) at beam/erl_alloc_util.c:3607 > #6 0x0000000000490595 in erts_alloc (size=32, type=18967) at > beam/erl_alloc.h:208 > #7 new_message_buffer (size=0) at beam/erl_message.c:72 > #8 erts_alloc_message_heap_state (statep=0x2aaaad6bfc4c, > receiver_locks=0x2aaaad6bfcb0, receiver=0x2aaaac9c1db8, ohpp= pointer>, bpp=, size=0) at beam/global.h:1017 > #9 erts_send_message (sender=0x2aaaac9c2470, receiver=0x2aaaac9c1db8, > receiver_locks=0x2aaaad6bfcb0, message=564171, flags=) at > beam/erl_message.c:1039 > #10 0x0000000000476350 in do_send (p=0x2aaaac9c2470, to=793099, msg=564171, > suspend=1, refp=) at beam/bif.c:2025 > #11 0x0000000000476bf0 in erl_send (p=0x2aaaac9c2470, to=793099, > msg=564171) at beam/bif.c:2138 > #12 0x000000000052d1d8 in process_main () at beam/beam_emu.c:2558 > #13 0x0000000000491463 in sched_thread_func (vesdp=0x2aaaac343ac0) at > beam/erl_process.c:5632 > #14 0x0000000000590440 in thr_wrapper (vtwd=0x7fffffffd850) at > pthread/ethread.c:106 > #15 0x00002aaaab3ffe9a in start_thread (arg=0x2aaaad6c0700) at > pthread_create.c:308 > #16 0x00002aaaab911cbd in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 > #17 0x0000000000000000 in ?? () > > > /Fredrik > > > > > > > On Mon, Apr 29, 2013 at 2:58 AM, Sverker Eriksson < > sverker.eriksson@REDACTED> wrote: > > >> Looks like some sort of memory corruption. >> >> Run on debug emulator and hope for a better (earlier) crash. >> >> # cd $ERL_TOP/erts/emulator >> # make TYPE=debug smp plain >> # $ERL_TOP/bin/cerl -debug >> >> /Sverker, Erlang/OTP Ericsson >> >> fredrik@REDACTED wrote: >> >> >>> Hello folks, >>> >>> I'm having difficulties locating the cause of a segfault I'm getting when >>> running tests with a NIF implementation I have. >>> >>> Anything that would shed light on what's wrong is appreciated. >>> >>> The backtrace does not seem to have anything to do with my NIF. The >>> NIF:ed function is called from a few different processes, and it forwards >>> the call to a worker thread which sends a message back to the caller >>> process. >>> >>> >>> >>> >>>> strings -a >>>> >>>> >>> ../../otp/R16B/pre-5.10.1-mz-**0.2/lib/erlang/erts-5.10.1/**bin/beam.smp|fgrep >>> GCC|sort -u >>> GCC: (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3 >>> >>> >>> >>> >>>> USE_GDB=1 rebar eunit >>>> >>>> >>>> >>> [cut] >>> Program received signal SIGSEGV, Segmentation fault. >>> [Switching to Thread 0x2aaaef381700 (LWP 19162)] >>> unlink_free_block (allctr=0x85f240, block=0x0, flags=0) at >>> beam/erl_goodfit_alloc.c:458 >>> 458 Uint sz = MBC_FBLK_SZ(&blk->block_head); >>> (gdb) backtrace >>> #0 unlink_free_block (allctr=0x85f240, block=0x0, flags=0) at >>> beam/erl_goodfit_alloc.c:458 >>> #1 0x0000000000444ee1 in get_free_block (allctr=0x85f240, >>> size=, cand_blk=0x0, cand_size=0, flags=0) at >>> beam/erl_goodfit_alloc.c:426 >>> #2 0x000000000043b6da in mbc_alloc_block (alcu_flgsp=>> pointer>, blk_szp=, size=, >>> allctr=0x85f240) at beam/erl_alloc_util.c:1309 >>> #3 mbc_alloc (allctr=0x85f240, size=) at >>> beam/erl_alloc_util.c:1451 >>> #4 0x0000000000440d7b in do_erts_alcu_alloc (size=32, extra=0x85f240, >>> type=148) at beam/erl_alloc_util.c:3530 >>> #5 erts_alcu_alloc_thr_pref (type=148, extra=, >>> size=) at beam/erl_alloc_util.c:3607 >>> #6 0x0000000000490595 in erts_alloc (size=32, type=18967) at >>> beam/erl_alloc.h:208 >>> #7 new_message_buffer (size=0) at beam/erl_message.c:72 >>> #8 erts_alloc_message_heap_state (statep=0x2aaaef380c4c, >>> receiver_locks=0x2aaaef380cb0, receiver=0x2aaaac9ccac8, ohpp=>> pointer>, bpp=, size=0) at beam/global.h:1017 >>> #9 erts_send_message (sender=0x2aaaac9cd118, receiver=0x2aaaac9ccac8, >>> receiver_locks=0x2aaaef380cb0, message=513931, flags=) at >>> beam/erl_message.c:1039 >>> #10 0x0000000000476350 in do_send (p=0x2aaaac9cd118, to=550027, >>> msg=513931, suspend=1, refp=) at beam/bif.c:2025 >>> #11 0x0000000000476bf0 in erl_send (p=0x2aaaac9cd118, to=550027, >>> msg=513931) at beam/bif.c:2138 >>> #12 0x000000000052d1d8 in process_main () at beam/beam_emu.c:2558 >>> #13 0x0000000000491463 in sched_thread_func (vesdp=0x2aaaac34de40) at >>> beam/erl_process.c:5632 >>> #14 0x0000000000590440 in thr_wrapper (vtwd=0x7fffffffd7e0) at >>> pthread/ethread.c:106 >>> #15 0x00002aaaab3ffe9a in start_thread (arg=0x2aaaef381700) at >>> pthread_create.c:308 >>> #16 0x00002aaaab911cbd in clone () at ../sysdeps/unix/sysv/linux/** >>> x86_64/clone.S:112 >>> #17 0x0000000000000000 in ?? () >>> (gdb) >>> >>> ------------------------------**------------------------------** >>> ------------ >>> >>> ______________________________**_________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/**listinfo/erlang-questions >>> >>> >>> >> > > From fredrik@REDACTED Tue Apr 30 15:11:23 2013 From: fredrik@REDACTED (Fredrik Linder) Date: Tue, 30 Apr 2013 06:11:23 -0700 Subject: [erlang-questions] Segfault in NIF call In-Reply-To: <517FBB3C.8060700@erix.ericsson.se> References: <3da9e2e4-e1fc-4187-9573-faed5e1c249d@googlegroups.com> <517E4441.2040202@erix.ericsson.se> <517FBB3C.8060700@erix.ericsson.se> Message-ID: Thx, I'll try Valgrind. Each function returns only terms created in its given process env. Each thread has two envs, one mutex-protected area for inter-thread-communication and one for sending. The mutex env is used to create an in-queue and is written once and read once only. These terms are only written/read inside mutex-protection. Q: How are the terms cleaned up in this env? - I cannot use enif_clear_env as it would clear too much. The send env is cleared after each enif_send, and only terms belonging to that env is used (made or copied there). Side question: enif_self() was unusable for me (the embedded term had value -13 when printed with %d), the pid returned could not be sent to. So now I'm having the .erl provide self() which I do enif_get_local_pid() on, and this works fine. This is on R16B. My latest version segfaults on R16B but works on R15B03-1. /Fredrik On 30 apr 2013, at 05:38, Sverker Eriksson wrote: > You wrote: > > "The NIF:ed function is called from a few different processes, and it forwards > the call to a worker thread." > > Have you done proper thread synchronization for this inter thread communication (with enif_mutex_lock() for example)? NIFs are only thread safe as long as they access their own environment. Shared memory structures such as a queue for a worker thread must be protected with locks (or implemented with a lock-free algorithm). > > > Another approach is to see if valgrind can detect any memory corruptions: > > # cd $ERL_TOP/erts/emulator > # make TYPE=valgrind smp plain > # export VALGRIND_LOG_DIR=/write/valgrind/logs/here > # export VALGRIND_MISC_FLAGS="--suppressions=$ERL_TOP/erts/emulator/valgrind/suppress.standard --show-possibly-lost=no" > # $ERL_TOP/bin/cerl -valgrind > > /Sverker > > > Fredrik Linder wrote: >> Found one error (yay!) -- enif_open_resource_type should have NULL as the >> module argument (missed this one in the docs) >> >> Running with cerl does not generate a segfault, nor any (extra) printouts >> Running with erl still generate segfault >> >> Now I get either of the following (using erl), does it reveal what my error >> is? >> >> >>> USE_GDB=1 rebar eunit >> [cut] >> Program received signal SIGSEGV, Segmentation fault. >> [Switching to Thread 0x2aaaad6c0700 (LWP 6384)] >> 0x000000000052b28a in process_main () at >> x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:979 >> 979 MoveDeallocateReturn(xb(tmp_packed1&BEAM_LOOSE_MASK), r(0), >> Qb((tmp_packed1>>BEAM_LOOSE_SHIFT))); >> (gdb) backtrace >> #0 0x000000000052b28a in process_main () at >> x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:979 >> #1 0x0000000000491463 in sched_thread_func (vesdp=0x2aaaac343ac0) at >> beam/erl_process.c:5632 >> #2 0x0000000000590440 in thr_wrapper (vtwd=0x7fffffffd850) at >> pthread/ethread.c:106 >> #3 0x00002aaaab3ffe9a in start_thread (arg=0x2aaaad6c0700) at >> pthread_create.c:308 >> #4 0x00002aaaab911cbd in clone () at >> ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 >> #5 0x0000000000000000 in ?? () >> >> or >> >> Program received signal SIGSEGV, Segmentation fault. >> [Switching to Thread 0x2aaaad6c0700 (LWP 6914)] >> unlink_free_block (allctr=0x85e5c0, block=0x0, flags=0) at >> beam/erl_goodfit_alloc.c:458 >> 458 Uint sz = MBC_FBLK_SZ(&blk->block_head); >> (gdb) backtrace >> #0 unlink_free_block (allctr=0x85e5c0, block=0x0, flags=0) at >> beam/erl_goodfit_alloc.c:458 >> #1 0x0000000000444ee1 in get_free_block (allctr=0x85e5c0, size=> out>, cand_blk=0x0, cand_size=0, flags=0) at beam/erl_goodfit_alloc.c:426 >> #2 0x000000000043b6da in mbc_alloc_block (alcu_flgsp=, >> blk_szp=, size=, allctr=0x85e5c0) at >> beam/erl_alloc_util.c:1309 >> #3 mbc_alloc (allctr=0x85e5c0, size=) at >> beam/erl_alloc_util.c:1451 >> #4 0x0000000000440d7b in do_erts_alcu_alloc (size=32, extra=0x85e5c0, >> type=148) at beam/erl_alloc_util.c:3530 >> #5 erts_alcu_alloc_thr_pref (type=148, extra=, >> size=) at beam/erl_alloc_util.c:3607 >> #6 0x0000000000490595 in erts_alloc (size=32, type=18967) at >> beam/erl_alloc.h:208 >> #7 new_message_buffer (size=0) at beam/erl_message.c:72 >> #8 erts_alloc_message_heap_state (statep=0x2aaaad6bfc4c, >> receiver_locks=0x2aaaad6bfcb0, receiver=0x2aaaac9c1db8, ohpp=> pointer>, bpp=, size=0) at beam/global.h:1017 >> #9 erts_send_message (sender=0x2aaaac9c2470, receiver=0x2aaaac9c1db8, >> receiver_locks=0x2aaaad6bfcb0, message=564171, flags=) at >> beam/erl_message.c:1039 >> #10 0x0000000000476350 in do_send (p=0x2aaaac9c2470, to=793099, msg=564171, >> suspend=1, refp=) at beam/bif.c:2025 >> #11 0x0000000000476bf0 in erl_send (p=0x2aaaac9c2470, to=793099, >> msg=564171) at beam/bif.c:2138 >> #12 0x000000000052d1d8 in process_main () at beam/beam_emu.c:2558 >> #13 0x0000000000491463 in sched_thread_func (vesdp=0x2aaaac343ac0) at >> beam/erl_process.c:5632 >> #14 0x0000000000590440 in thr_wrapper (vtwd=0x7fffffffd850) at >> pthread/ethread.c:106 >> #15 0x00002aaaab3ffe9a in start_thread (arg=0x2aaaad6c0700) at >> pthread_create.c:308 >> #16 0x00002aaaab911cbd in clone () at >> ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 >> #17 0x0000000000000000 in ?? () >> >> >> /Fredrik >> >> >> >> >> >> >> On Mon, Apr 29, 2013 at 2:58 AM, Sverker Eriksson < >> sverker.eriksson@REDACTED> wrote: >> >> >>> Looks like some sort of memory corruption. >>> >>> Run on debug emulator and hope for a better (earlier) crash. >>> >>> # cd $ERL_TOP/erts/emulator >>> # make TYPE=debug smp plain >>> # $ERL_TOP/bin/cerl -debug >>> >>> /Sverker, Erlang/OTP Ericsson >>> >>> fredrik@REDACTED wrote: >>> >>> >>>> Hello folks, >>>> >>>> I'm having difficulties locating the cause of a segfault I'm getting when >>>> running tests with a NIF implementation I have. >>>> >>>> Anything that would shed light on what's wrong is appreciated. >>>> >>>> The backtrace does not seem to have anything to do with my NIF. The >>>> NIF:ed function is called from a few different processes, and it forwards >>>> the call to a worker thread which sends a message back to the caller >>>> process. >>>> >>>> >>>> >>>> >>>>> strings -a >>>> ../../otp/R16B/pre-5.10.1-mz-**0.2/lib/erlang/erts-5.10.1/**bin/beam.smp|fgrep >>>> GCC|sort -u >>>> GCC: (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3 >>>> >>>> >>>> >>>> >>>>> USE_GDB=1 rebar eunit >>>> [cut] >>>> Program received signal SIGSEGV, Segmentation fault. >>>> [Switching to Thread 0x2aaaef381700 (LWP 19162)] >>>> unlink_free_block (allctr=0x85f240, block=0x0, flags=0) at >>>> beam/erl_goodfit_alloc.c:458 >>>> 458 Uint sz = MBC_FBLK_SZ(&blk->block_head); >>>> (gdb) backtrace >>>> #0 unlink_free_block (allctr=0x85f240, block=0x0, flags=0) at >>>> beam/erl_goodfit_alloc.c:458 >>>> #1 0x0000000000444ee1 in get_free_block (allctr=0x85f240, >>>> size=, cand_blk=0x0, cand_size=0, flags=0) at >>>> beam/erl_goodfit_alloc.c:426 >>>> #2 0x000000000043b6da in mbc_alloc_block (alcu_flgsp=>>> pointer>, blk_szp=, size=, >>>> allctr=0x85f240) at beam/erl_alloc_util.c:1309 >>>> #3 mbc_alloc (allctr=0x85f240, size=) at >>>> beam/erl_alloc_util.c:1451 >>>> #4 0x0000000000440d7b in do_erts_alcu_alloc (size=32, extra=0x85f240, >>>> type=148) at beam/erl_alloc_util.c:3530 >>>> #5 erts_alcu_alloc_thr_pref (type=148, extra=, >>>> size=) at beam/erl_alloc_util.c:3607 >>>> #6 0x0000000000490595 in erts_alloc (size=32, type=18967) at >>>> beam/erl_alloc.h:208 >>>> #7 new_message_buffer (size=0) at beam/erl_message.c:72 >>>> #8 erts_alloc_message_heap_state (statep=0x2aaaef380c4c, >>>> receiver_locks=0x2aaaef380cb0, receiver=0x2aaaac9ccac8, ohpp=>>> pointer>, bpp=, size=0) at beam/global.h:1017 >>>> #9 erts_send_message (sender=0x2aaaac9cd118, receiver=0x2aaaac9ccac8, >>>> receiver_locks=0x2aaaef380cb0, message=513931, flags=) at >>>> beam/erl_message.c:1039 >>>> #10 0x0000000000476350 in do_send (p=0x2aaaac9cd118, to=550027, >>>> msg=513931, suspend=1, refp=) at beam/bif.c:2025 >>>> #11 0x0000000000476bf0 in erl_send (p=0x2aaaac9cd118, to=550027, >>>> msg=513931) at beam/bif.c:2138 >>>> #12 0x000000000052d1d8 in process_main () at beam/beam_emu.c:2558 >>>> #13 0x0000000000491463 in sched_thread_func (vesdp=0x2aaaac34de40) at >>>> beam/erl_process.c:5632 >>>> #14 0x0000000000590440 in thr_wrapper (vtwd=0x7fffffffd7e0) at >>>> pthread/ethread.c:106 >>>> #15 0x00002aaaab3ffe9a in start_thread (arg=0x2aaaef381700) at >>>> pthread_create.c:308 >>>> #16 0x00002aaaab911cbd in clone () at ../sysdeps/unix/sysv/linux/** >>>> x86_64/clone.S:112 >>>> #17 0x0000000000000000 in ?? () >>>> (gdb) >>>> >>>> ------------------------------**------------------------------** >>>> ------------ >>>> >>>> ______________________________**_________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/**listinfo/erlang-questions > From greg@REDACTED Tue Apr 30 15:13:30 2013 From: greg@REDACTED (Greg Burd) Date: Tue, 30 Apr 2013 09:13:30 -0400 Subject: [erlang-questions] Segfault in NIF call In-Reply-To: References: <3da9e2e4-e1fc-4187-9573-faed5e1c249d@googlegroups.com> <517E4441.2040202@erix.ericsson.se> Message-ID: Are you trying to build a NIF where the function is run on a non-Erlang scheduler thread (async, via some thread pool)? If so, here's a method for doing it. https://gist.github.com/gburd/4121795 It took a while to get this just right, reuse and enjoy. I'll update the nifwait (https://github.com/basho/nifwait) repo with example code. The best example of how to use it is in the wterl (https://github.com/basho-labs/wterl) repo. -greg @gregburd | Basho Technologies | Riak | http://basho.com | @basho On Mon, Apr 29, 2013 at 2:57 PM, Fredrik Linder wrote: > Found one error (yay!) -- enif_open_resource_type should have NULL as the > module argument (missed this one in the docs) > > Running with cerl does not generate a segfault, nor any (extra) printouts > Running with erl still generate segfault > > Now I get either of the following (using erl), does it reveal what my error > is? > >> USE_GDB=1 rebar eunit > [cut] > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x2aaaad6c0700 (LWP 6384)] > 0x000000000052b28a in process_main () at > x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:979 > 979 MoveDeallocateReturn(xb(tmp_packed1&BEAM_LOOSE_MASK), r(0), > Qb((tmp_packed1>>BEAM_LOOSE_SHIFT))); > (gdb) backtrace > #0 0x000000000052b28a in process_main () at > x86_64-unknown-linux-gnu/opt/smp/beam_hot.h:979 > #1 0x0000000000491463 in sched_thread_func (vesdp=0x2aaaac343ac0) at > beam/erl_process.c:5632 > #2 0x0000000000590440 in thr_wrapper (vtwd=0x7fffffffd850) at > pthread/ethread.c:106 > #3 0x00002aaaab3ffe9a in start_thread (arg=0x2aaaad6c0700) at > pthread_create.c:308 > #4 0x00002aaaab911cbd in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 > #5 0x0000000000000000 in ?? () > > or > > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x2aaaad6c0700 (LWP 6914)] > unlink_free_block (allctr=0x85e5c0, block=0x0, flags=0) at > beam/erl_goodfit_alloc.c:458 > 458 Uint sz = MBC_FBLK_SZ(&blk->block_head); > (gdb) backtrace > #0 unlink_free_block (allctr=0x85e5c0, block=0x0, flags=0) at > beam/erl_goodfit_alloc.c:458 > #1 0x0000000000444ee1 in get_free_block (allctr=0x85e5c0, size= out>, cand_blk=0x0, cand_size=0, flags=0) at beam/erl_goodfit_alloc.c:426 > #2 0x000000000043b6da in mbc_alloc_block (alcu_flgsp=, > blk_szp=, size=, allctr=0x85e5c0) at > beam/erl_alloc_util.c:1309 > #3 mbc_alloc (allctr=0x85e5c0, size=) at > beam/erl_alloc_util.c:1451 > #4 0x0000000000440d7b in do_erts_alcu_alloc (size=32, extra=0x85e5c0, > type=148) at beam/erl_alloc_util.c:3530 > #5 erts_alcu_alloc_thr_pref (type=148, extra=, > size=) at beam/erl_alloc_util.c:3607 > #6 0x0000000000490595 in erts_alloc (size=32, type=18967) at > beam/erl_alloc.h:208 > #7 new_message_buffer (size=0) at beam/erl_message.c:72 > #8 erts_alloc_message_heap_state (statep=0x2aaaad6bfc4c, > receiver_locks=0x2aaaad6bfcb0, receiver=0x2aaaac9c1db8, ohpp= pointer>, bpp=, size=0) at beam/global.h:1017 > #9 erts_send_message (sender=0x2aaaac9c2470, receiver=0x2aaaac9c1db8, > receiver_locks=0x2aaaad6bfcb0, message=564171, flags=) at > beam/erl_message.c:1039 > #10 0x0000000000476350 in do_send (p=0x2aaaac9c2470, to=793099, msg=564171, > suspend=1, refp=) at beam/bif.c:2025 > #11 0x0000000000476bf0 in erl_send (p=0x2aaaac9c2470, to=793099, msg=564171) > at beam/bif.c:2138 > #12 0x000000000052d1d8 in process_main () at beam/beam_emu.c:2558 > #13 0x0000000000491463 in sched_thread_func (vesdp=0x2aaaac343ac0) at > beam/erl_process.c:5632 > #14 0x0000000000590440 in thr_wrapper (vtwd=0x7fffffffd850) at > pthread/ethread.c:106 > #15 0x00002aaaab3ffe9a in start_thread (arg=0x2aaaad6c0700) at > pthread_create.c:308 > #16 0x00002aaaab911cbd in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 > #17 0x0000000000000000 in ?? () > > > /Fredrik > > > > > > > On Mon, Apr 29, 2013 at 2:58 AM, Sverker Eriksson > wrote: >> >> Looks like some sort of memory corruption. >> >> Run on debug emulator and hope for a better (earlier) crash. >> >> # cd $ERL_TOP/erts/emulator >> # make TYPE=debug smp plain >> # $ERL_TOP/bin/cerl -debug >> >> /Sverker, Erlang/OTP Ericsson >> >> fredrik@REDACTED wrote: >>> >>> Hello folks, >>> >>> I'm having difficulties locating the cause of a segfault I'm getting when >>> running tests with a NIF implementation I have. >>> >>> Anything that would shed light on what's wrong is appreciated. >>> >>> The backtrace does not seem to have anything to do with my NIF. The >>> NIF:ed function is called from a few different processes, and it forwards >>> the call to a worker thread which sends a message back to the caller >>> process. >>> >>> >>>> >>>> strings -a >>> >>> >>> ../../otp/R16B/pre-5.10.1-mz-0.2/lib/erlang/erts-5.10.1/bin/beam.smp|fgrep >>> GCC|sort -u >>> GCC: (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3 >>> >>> >>>> >>>> USE_GDB=1 rebar eunit >>>> >>> >>> [cut] >>> Program received signal SIGSEGV, Segmentation fault. >>> [Switching to Thread 0x2aaaef381700 (LWP 19162)] >>> unlink_free_block (allctr=0x85f240, block=0x0, flags=0) at >>> beam/erl_goodfit_alloc.c:458 >>> 458 Uint sz = MBC_FBLK_SZ(&blk->block_head); >>> (gdb) backtrace >>> #0 unlink_free_block (allctr=0x85f240, block=0x0, flags=0) at >>> beam/erl_goodfit_alloc.c:458 >>> #1 0x0000000000444ee1 in get_free_block (allctr=0x85f240, >>> size=, cand_blk=0x0, cand_size=0, flags=0) at >>> beam/erl_goodfit_alloc.c:426 >>> #2 0x000000000043b6da in mbc_alloc_block (alcu_flgsp=>> pointer>, blk_szp=, size=, >>> allctr=0x85f240) at beam/erl_alloc_util.c:1309 >>> #3 mbc_alloc (allctr=0x85f240, size=) at >>> beam/erl_alloc_util.c:1451 >>> #4 0x0000000000440d7b in do_erts_alcu_alloc (size=32, extra=0x85f240, >>> type=148) at beam/erl_alloc_util.c:3530 >>> #5 erts_alcu_alloc_thr_pref (type=148, extra=, >>> size=) at beam/erl_alloc_util.c:3607 >>> #6 0x0000000000490595 in erts_alloc (size=32, type=18967) at >>> beam/erl_alloc.h:208 >>> #7 new_message_buffer (size=0) at beam/erl_message.c:72 >>> #8 erts_alloc_message_heap_state (statep=0x2aaaef380c4c, >>> receiver_locks=0x2aaaef380cb0, receiver=0x2aaaac9ccac8, ohpp=>> pointer>, bpp=, size=0) at beam/global.h:1017 >>> #9 erts_send_message (sender=0x2aaaac9cd118, receiver=0x2aaaac9ccac8, >>> receiver_locks=0x2aaaef380cb0, message=513931, flags=) at >>> beam/erl_message.c:1039 >>> #10 0x0000000000476350 in do_send (p=0x2aaaac9cd118, to=550027, >>> msg=513931, suspend=1, refp=) at beam/bif.c:2025 >>> #11 0x0000000000476bf0 in erl_send (p=0x2aaaac9cd118, to=550027, >>> msg=513931) at beam/bif.c:2138 >>> #12 0x000000000052d1d8 in process_main () at beam/beam_emu.c:2558 >>> #13 0x0000000000491463 in sched_thread_func (vesdp=0x2aaaac34de40) at >>> beam/erl_process.c:5632 >>> #14 0x0000000000590440 in thr_wrapper (vtwd=0x7fffffffd7e0) at >>> pthread/ethread.c:106 >>> #15 0x00002aaaab3ffe9a in start_thread (arg=0x2aaaef381700) at >>> pthread_create.c:308 >>> #16 0x00002aaaab911cbd in clone () at >>> ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 >>> #17 0x0000000000000000 in ?? () >>> (gdb) >>> >>> >>> ------------------------------------------------------------------------ >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From g@REDACTED Tue Apr 30 15:44:48 2013 From: g@REDACTED (Garrett Smith) Date: Tue, 30 Apr 2013 08:44:48 -0500 Subject: [erlang-questions] [ANN] Silly benchmarking Message-ID: This is not an announcement of anything -- but [ANN] seems to flag "something I can maybe use" which does apply in this case :) Occasionally I wonder, "what's faster"? It's not often, but it happens. I've found the best way to answer this is to measure things. So I have this silly project: https://github.com/gar1t/erlang-bench It's not rigorous but it's simple and I can experiment quickly with different implementations. My goal is just to get a sense of things -- not to formally prove anything. It's so trivial it's almost not worth sharing/reusing -- *however* it may provide value as a distributed repository for what people are interested in. As it's in github there's no ownership -- please feel free to fork and use for your own concerns! Garrett From mjtruog@REDACTED Tue Apr 30 16:35:28 2013 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 30 Apr 2013 07:35:28 -0700 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: References: Message-ID: <517FD6B0.3070505@gmail.com> On 04/30/2013 06:44 AM, Garrett Smith wrote: > This is not an announcement of anything -- but [ANN] seems to flag > "something I can maybe use" which does apply in this case :) > > Occasionally I wonder, "what's faster"? It's not often, but it happens. > > I've found the best way to answer this is to measure things. > > So I have this silly project: > > https://github.com/gar1t/erlang-bench > > It's not rigorous but it's simple and I can experiment quickly with > different implementations. My goal is just to get a sense of things -- > not to formally prove anything. > > It's so trivial it's almost not worth sharing/reusing -- *however* it > may provide value as a distributed repository for what people are > interested in. As it's in github there's no ownership -- please feel > free to fork and use for your own concerns! > > Garrett > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > You might want to look at erlbench here https://github.com/okeuday/erlbench since it has the same basic purpose, and allows you to use different compilation methods now (through the makefile specifying an optimization level). The erlbench project is also ad-hoc, but it has been enough to produce results in the past. The other option is trying to use basho_bench here https://github.com/basho/basho_bench, if you are testing key/value storage. - Michael From flw@REDACTED Tue Apr 30 18:35:42 2013 From: flw@REDACTED (Florian Waas) Date: Tue, 30 Apr 2013 09:35:42 -0700 Subject: [erlang-questions] How to get proper code coverage with rebar/eunit for yecc/leex files Message-ID: Rebar's reporting of coverage during eunit runs computes coverage numbers for the generated lexer/parser code not the original xrl/yrl files. The generated files contain a ton of boiler plate code from leexinc.hrl and yeccpre.hrl which is not exercised. As a result, the coverage number is rather meaningless. Anybody have ideas for workarounds? Elegant ways to exclude these files from the report? Thanks, -fl. From g@REDACTED Tue Apr 30 18:42:37 2013 From: g@REDACTED (Garrett Smith) Date: Tue, 30 Apr 2013 11:42:37 -0500 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: <517FD6B0.3070505@gmail.com> References: <517FD6B0.3070505@gmail.com> Message-ID: On Tue, Apr 30, 2013 at 9:35 AM, Michael Truog wrote: > On 04/30/2013 06:44 AM, Garrett Smith wrote: >> This is not an announcement of anything -- but [ANN] seems to flag >> "something I can maybe use" which does apply in this case :) >> >> Occasionally I wonder, "what's faster"? It's not often, but it happens. >> >> I've found the best way to answer this is to measure things. >> >> So I have this silly project: >> >> https://github.com/gar1t/erlang-bench >> >> It's not rigorous but it's simple and I can experiment quickly with >> different implementations. My goal is just to get a sense of things -- >> not to formally prove anything. >> >> It's so trivial it's almost not worth sharing/reusing -- *however* it >> may provide value as a distributed repository for what people are >> interested in. As it's in github there's no ownership -- please feel >> free to fork and use for your own concerns! >> > You might want to look at erlbench here https://github.com/okeuday/erlbench since it has the same basic purpose, and allows you to use different compilation methods now (through the makefile specifying an optimization level). The erlbench project is also ad-hoc, but it has been enough to produce results in the past. > > The other option is trying to use basho_bench here https://github.com/basho/basho_bench, if you are testing key/value storage. Yes, but you'll notice how *easy* it is to use erlang-bench, which is nothing more than escript files with a 10 line include file. I'm an extraordinarily lazy person :) Though seriously, thanks for the references. If I was more concerned about benchmark integrity, those might be good options -- but this is just a sniff test approach to satisfy my curiosity about various topics. Garrett From gumm@REDACTED Tue Apr 30 18:45:08 2013 From: gumm@REDACTED (Jesse Gumm) Date: Tue, 30 Apr 2013 11:45:08 -0500 Subject: [erlang-questions] [ANN] qdate - Unified Date Formatting/Parsing and Timezone Management In-Reply-To: <0E6ACAB0-6A5B-4C85-84E2-CC7B590317B6@dmitriid.com> References: <0E6ACAB0-6A5B-4C85-84E2-CC7B590317B6@dmitriid.com> Message-ID: Hi Dmitrii, I had not looked into tempo (I was not aware of its existence). Given that qdate is fundamentally a wrapper, I could see adding an option to change formatting backend to tempo (qdate currently uses ec_date as the backend for both parsing and formatting). Thank you, I appreciate the suggestion, -Jesse On Tue, Apr 30, 2013 at 1:50 AM, Dmitrii Dimandt wrote: > Have you also looked at tempo, https://github.com/selectel/tempo ? > > Your library and tempo could probably be combined > > On Apr 30, 2013, at 07:09 , Jesse Gumm wrote: > > Hello Erlangers, > > I'm proud to announce the beta release (v0.1.0) of "qdate" - a unified > date parsing/formatting and timezone management utility for Erlang. > > Blog post describing my thoughts that went into its development > http://sigma-star.com/blog/post/qdate > > Source code on GitHub (includes a README that covers all the functionality) > https://github.com/choptastic/qdate > > > At its core, qdate is basically a wrapper around ec_date (fork of dh_date > and part of erlware_commons) and erlang_localtime. > > But it's much more than just a wrapper. It's the complete date and time > utility I feel is greatly needed in the Erlang world (since I found myself > needing it). > > Briefly, here's what it does: > > * Easily convert from any date format to any other date format: > datetime(), now(), unix timestamp, and formatted date strings (list and > binaries) are all acceptable: > > qdate:to_string("Y-m-d g:ia", Date). > "2012-12-21 5:51pm" > > qdate:to_string(<<"Y-m-d g:ia">>, Date). > <<"2012-12-21 5:51pm">> > > qdate:to_unixtime(Date). > 1356112260 > > qdate:to_date(Date). > {{2012,12,21},{17,51,0}} > > qdate:to_now(Date). > {1356,112260,0} > % Note, that the argument `Date` can be anything > % that's a Date/Time Format (datetime Tuple, now tuple, > % Integer (unixtime), or string ("2012-12-21 5:51pm") > > * Extends ec_date's formatting functions to include PHP's timezone-related > characters: > > qdate:to_string("Y-m-d g:ia T", Date) > "2012-12-21 5:51pm CST" > > * Able to pre-parse timezones from text strings, and convert from one > timezone to another: > % Read Timezone from string, convert to datetime in EST > > qdate:to_date("2012-12-21 5:51pm CST","EST"). > {{2012,12,21},{18,51,0}} > % Read timezone from string, convert to a string in EST > > qdate:to_string("n/j/Y g:ia T","2012-12-21 5:51pm CST", "EST"). > "12/21/2012 6:51pm EST" > > * Register timezones with the qdate server by a key (to make a timezone > alias) or by pid(): > %% Set timezone for the current process to GMT > > qdate:set_timezone("GMT"). > %% Set timezone for some_key to HKT > > qdate:set_timezone(some_key, "HKT"). > > * Register custom formatting strings with the qdate server, for quick > reference. > > qdate:register_format(long_date, "l, F jS, Y g:i A T"). > ok > > qdate:to_string(long_date,1356133860). > "Friday, December 21st, 2012 11:51 PM GMT" > > * Register custom parsing functions with the qdate server, to be > transparently used for conversion. > > qdate:register_parser(weird_format, fun parse_esoteric_format/1). > ok > > qdate:to_date("20121221.115100"). > {{2012,12,21},{11,51,00}} > > * Attempt to maintain API compatibility with ec_date by supporting > qdate:parse, qdate:nparse, and qdate:format. > > > I hope some of you find this useful in your apps. > > Any issues, comments, complaints, bug reports, whatever, feel free to post > in this thread, make a github issue, shoot me a message on twitter > (@jessegumm) or email me directly. > > Thanks, and I hope everyone has a glorious day or evening or whatever > timezone you happen to be in. > > -- > Jesse Gumm > Owner, Sigma Star Systems > 414.940.4866 || sigma-star.com || @jessegumm > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremy@REDACTED Tue Apr 30 18:57:08 2013 From: jeremy@REDACTED (Jeremy Ong) Date: Tue, 30 Apr 2013 09:57:08 -0700 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: References: <517FD6B0.3070505@gmail.com> Message-ID: I'd be very interested if we got a wiki going on one of these projects with community updated numbers of various benchmark runs. For example: keylists vs orddicts vs dicts fold vs recursion fibonacci all the other various standard benchmarks On Tue, Apr 30, 2013 at 9:42 AM, Garrett Smith wrote: > On Tue, Apr 30, 2013 at 9:35 AM, Michael Truog wrote: >> On 04/30/2013 06:44 AM, Garrett Smith wrote: >>> This is not an announcement of anything -- but [ANN] seems to flag >>> "something I can maybe use" which does apply in this case :) >>> >>> Occasionally I wonder, "what's faster"? It's not often, but it happens. >>> >>> I've found the best way to answer this is to measure things. >>> >>> So I have this silly project: >>> >>> https://github.com/gar1t/erlang-bench >>> >>> It's not rigorous but it's simple and I can experiment quickly with >>> different implementations. My goal is just to get a sense of things -- >>> not to formally prove anything. >>> >>> It's so trivial it's almost not worth sharing/reusing -- *however* it >>> may provide value as a distributed repository for what people are >>> interested in. As it's in github there's no ownership -- please feel >>> free to fork and use for your own concerns! >>> >> You might want to look at erlbench here https://github.com/okeuday/erlbench since it has the same basic purpose, and allows you to use different compilation methods now (through the makefile specifying an optimization level). The erlbench project is also ad-hoc, but it has been enough to produce results in the past. >> >> The other option is trying to use basho_bench here https://github.com/basho/basho_bench, if you are testing key/value storage. > > Yes, but you'll notice how *easy* it is to use erlang-bench, which is > nothing more than escript files with a 10 line include file. > > I'm an extraordinarily lazy person :) > > Though seriously, thanks for the references. If I was more concerned > about benchmark integrity, those might be good options -- but this is > just a sniff test approach to satisfy my curiosity about various > topics. > > Garrett > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Tue Apr 30 18:59:20 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 30 Apr 2013 18:59:20 +0200 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: References: <517FD6B0.3070505@gmail.com> Message-ID: <517FF868.7030709@ninenines.eu> It's not very interesting unless the numbers are also available for systems under load. On 04/30/2013 06:57 PM, Jeremy Ong wrote: > I'd be very interested if we got a wiki going on one of these projects > with community updated numbers of various benchmark runs. > > For example: > > keylists vs orddicts vs dicts > fold vs recursion > fibonacci > all the other various standard benchmarks > > On Tue, Apr 30, 2013 at 9:42 AM, Garrett Smith wrote: >> On Tue, Apr 30, 2013 at 9:35 AM, Michael Truog wrote: >>> On 04/30/2013 06:44 AM, Garrett Smith wrote: >>>> This is not an announcement of anything -- but [ANN] seems to flag >>>> "something I can maybe use" which does apply in this case :) >>>> >>>> Occasionally I wonder, "what's faster"? It's not often, but it happens. >>>> >>>> I've found the best way to answer this is to measure things. >>>> >>>> So I have this silly project: >>>> >>>> https://github.com/gar1t/erlang-bench >>>> >>>> It's not rigorous but it's simple and I can experiment quickly with >>>> different implementations. My goal is just to get a sense of things -- >>>> not to formally prove anything. >>>> >>>> It's so trivial it's almost not worth sharing/reusing -- *however* it >>>> may provide value as a distributed repository for what people are >>>> interested in. As it's in github there's no ownership -- please feel >>>> free to fork and use for your own concerns! >>>> >>> You might want to look at erlbench here https://github.com/okeuday/erlbench since it has the same basic purpose, and allows you to use different compilation methods now (through the makefile specifying an optimization level). The erlbench project is also ad-hoc, but it has been enough to produce results in the past. >>> >>> The other option is trying to use basho_bench here https://github.com/basho/basho_bench, if you are testing key/value storage. >> >> Yes, but you'll notice how *easy* it is to use erlang-bench, which is >> nothing more than escript files with a 10 line include file. >> >> I'm an extraordinarily lazy person :) >> >> Though seriously, thanks for the references. If I was more concerned >> about benchmark integrity, those might be good options -- but this is >> just a sniff test approach to satisfy my curiosity about various >> topics. >> >> Garrett >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From g@REDACTED Tue Apr 30 19:02:04 2013 From: g@REDACTED (Garrett Smith) Date: Tue, 30 Apr 2013 12:02:04 -0500 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: References: <517FD6B0.3070505@gmail.com> Message-ID: This is sort of my thinking with the github project: - Better than a wiki because you can run the files - Easily forked and modified -- so people can share and modify code trivially - Distributed is better than centralized! There's probably a temptation to view anything like this as authoritative -- it of course is not and cannot be (e.g. the three stupid files I have are totally arbitrary). I think distributed source like this is a good push-back for that temptation. On Tue, Apr 30, 2013 at 11:57 AM, Jeremy Ong wrote: > I'd be very interested if we got a wiki going on one of these projects > with community updated numbers of various benchmark runs. > > For example: > > keylists vs orddicts vs dicts > fold vs recursion > fibonacci > all the other various standard benchmarks > > On Tue, Apr 30, 2013 at 9:42 AM, Garrett Smith wrote: >> On Tue, Apr 30, 2013 at 9:35 AM, Michael Truog wrote: >>> On 04/30/2013 06:44 AM, Garrett Smith wrote: >>>> This is not an announcement of anything -- but [ANN] seems to flag >>>> "something I can maybe use" which does apply in this case :) >>>> >>>> Occasionally I wonder, "what's faster"? It's not often, but it happens. >>>> >>>> I've found the best way to answer this is to measure things. >>>> >>>> So I have this silly project: >>>> >>>> https://github.com/gar1t/erlang-bench >>>> >>>> It's not rigorous but it's simple and I can experiment quickly with >>>> different implementations. My goal is just to get a sense of things -- >>>> not to formally prove anything. >>>> >>>> It's so trivial it's almost not worth sharing/reusing -- *however* it >>>> may provide value as a distributed repository for what people are >>>> interested in. As it's in github there's no ownership -- please feel >>>> free to fork and use for your own concerns! >>>> >>> You might want to look at erlbench here https://github.com/okeuday/erlbench since it has the same basic purpose, and allows you to use different compilation methods now (through the makefile specifying an optimization level). The erlbench project is also ad-hoc, but it has been enough to produce results in the past. >>> >>> The other option is trying to use basho_bench here https://github.com/basho/basho_bench, if you are testing key/value storage. >> >> Yes, but you'll notice how *easy* it is to use erlang-bench, which is >> nothing more than escript files with a 10 line include file. >> >> I'm an extraordinarily lazy person :) >> >> Though seriously, thanks for the references. If I was more concerned >> about benchmark integrity, those might be good options -- but this is >> just a sniff test approach to satisfy my curiosity about various >> topics. >> >> Garrett >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From g@REDACTED Tue Apr 30 19:03:18 2013 From: g@REDACTED (Garrett Smith) Date: Tue, 30 Apr 2013 12:03:18 -0500 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: <517FF868.7030709@ninenines.eu> References: <517FD6B0.3070505@gmail.com> <517FF868.7030709@ninenines.eu> Message-ID: Not very interesting to you. Of course you can write whatever you like. For *me* I was curious about some relative performance characteristics. No religion here. On Tue, Apr 30, 2013 at 11:59 AM, Lo?c Hoguin wrote: > It's not very interesting unless the numbers are also available for systems > under load. > > > On 04/30/2013 06:57 PM, Jeremy Ong wrote: >> >> I'd be very interested if we got a wiki going on one of these projects >> with community updated numbers of various benchmark runs. >> >> For example: >> >> keylists vs orddicts vs dicts >> fold vs recursion >> fibonacci >> all the other various standard benchmarks >> >> On Tue, Apr 30, 2013 at 9:42 AM, Garrett Smith wrote: >>> >>> On Tue, Apr 30, 2013 at 9:35 AM, Michael Truog wrote: >>>> >>>> On 04/30/2013 06:44 AM, Garrett Smith wrote: >>>>> >>>>> This is not an announcement of anything -- but [ANN] seems to flag >>>>> "something I can maybe use" which does apply in this case :) >>>>> >>>>> Occasionally I wonder, "what's faster"? It's not often, but it happens. >>>>> >>>>> I've found the best way to answer this is to measure things. >>>>> >>>>> So I have this silly project: >>>>> >>>>> https://github.com/gar1t/erlang-bench >>>>> >>>>> It's not rigorous but it's simple and I can experiment quickly with >>>>> different implementations. My goal is just to get a sense of things -- >>>>> not to formally prove anything. >>>>> >>>>> It's so trivial it's almost not worth sharing/reusing -- *however* it >>>>> may provide value as a distributed repository for what people are >>>>> interested in. As it's in github there's no ownership -- please feel >>>>> free to fork and use for your own concerns! >>>>> >>>> You might want to look at erlbench here >>>> https://github.com/okeuday/erlbench since it has the same basic purpose, and >>>> allows you to use different compilation methods now (through the makefile >>>> specifying an optimization level). The erlbench project is also ad-hoc, but >>>> it has been enough to produce results in the past. >>>> >>>> The other option is trying to use basho_bench here >>>> https://github.com/basho/basho_bench, if you are testing key/value storage. >>> >>> >>> Yes, but you'll notice how *easy* it is to use erlang-bench, which is >>> nothing more than escript files with a 10 line include file. >>> >>> I'm an extraordinarily lazy person :) >>> >>> Though seriously, thanks for the references. If I was more concerned >>> about benchmark integrity, those might be good options -- but this is >>> just a sniff test approach to satisfy my curiosity about various >>> topics. >>> >>> Garrett >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu From essen@REDACTED Tue Apr 30 19:06:04 2013 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 30 Apr 2013 19:06:04 +0200 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: References: <517FD6B0.3070505@gmail.com> <517FF868.7030709@ninenines.eu> Message-ID: <517FF9FC.3040204@ninenines.eu> I'm sure *you* know these results might not be true for systems under load, but others might not. Not having both could cause more harm than good, and would not be interesting in that sense. On 04/30/2013 07:03 PM, Garrett Smith wrote: > Not very interesting to you. Of course you can write whatever you > like. For *me* I was curious about some relative performance > characteristics. No religion here. > > On Tue, Apr 30, 2013 at 11:59 AM, Lo?c Hoguin wrote: >> It's not very interesting unless the numbers are also available for systems >> under load. >> >> >> On 04/30/2013 06:57 PM, Jeremy Ong wrote: >>> >>> I'd be very interested if we got a wiki going on one of these projects >>> with community updated numbers of various benchmark runs. >>> >>> For example: >>> >>> keylists vs orddicts vs dicts >>> fold vs recursion >>> fibonacci >>> all the other various standard benchmarks >>> >>> On Tue, Apr 30, 2013 at 9:42 AM, Garrett Smith wrote: >>>> >>>> On Tue, Apr 30, 2013 at 9:35 AM, Michael Truog wrote: >>>>> >>>>> On 04/30/2013 06:44 AM, Garrett Smith wrote: >>>>>> >>>>>> This is not an announcement of anything -- but [ANN] seems to flag >>>>>> "something I can maybe use" which does apply in this case :) >>>>>> >>>>>> Occasionally I wonder, "what's faster"? It's not often, but it happens. >>>>>> >>>>>> I've found the best way to answer this is to measure things. >>>>>> >>>>>> So I have this silly project: >>>>>> >>>>>> https://github.com/gar1t/erlang-bench >>>>>> >>>>>> It's not rigorous but it's simple and I can experiment quickly with >>>>>> different implementations. My goal is just to get a sense of things -- >>>>>> not to formally prove anything. >>>>>> >>>>>> It's so trivial it's almost not worth sharing/reusing -- *however* it >>>>>> may provide value as a distributed repository for what people are >>>>>> interested in. As it's in github there's no ownership -- please feel >>>>>> free to fork and use for your own concerns! >>>>>> >>>>> You might want to look at erlbench here >>>>> https://github.com/okeuday/erlbench since it has the same basic purpose, and >>>>> allows you to use different compilation methods now (through the makefile >>>>> specifying an optimization level). The erlbench project is also ad-hoc, but >>>>> it has been enough to produce results in the past. >>>>> >>>>> The other option is trying to use basho_bench here >>>>> https://github.com/basho/basho_bench, if you are testing key/value storage. >>>> >>>> >>>> Yes, but you'll notice how *easy* it is to use erlang-bench, which is >>>> nothing more than escript files with a 10 line include file. >>>> >>>> I'm an extraordinarily lazy person :) >>>> >>>> Though seriously, thanks for the references. If I was more concerned >>>> about benchmark integrity, those might be good options -- but this is >>>> just a sniff test approach to satisfy my curiosity about various >>>> topics. >>>> >>>> Garrett >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> http://ninenines.eu -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From jeremy@REDACTED Tue Apr 30 19:15:49 2013 From: jeremy@REDACTED (Jeremy Ong) Date: Tue, 30 Apr 2013 10:15:49 -0700 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: <517FF9FC.3040204@ninenines.eu> References: <517FD6B0.3070505@gmail.com> <517FF868.7030709@ninenines.eu> <517FF9FC.3040204@ninenines.eu> Message-ID: I think comparison benchmarks are still useful. Even against a system with no other load, knowing that X does the equivalent thing that Y does but faster could provide initial direction in the implementation of something new at least. Agreed that benchmarks can be harmful if they aren't viewed objectively though. We just need a caveat that the user measures his or her own application as well! On Tue, Apr 30, 2013 at 10:06 AM, Lo?c Hoguin wrote: > I'm sure *you* know these results might not be true for systems under load, > but others might not. Not having both could cause more harm than good, and > would not be interesting in that sense. > > > On 04/30/2013 07:03 PM, Garrett Smith wrote: >> >> Not very interesting to you. Of course you can write whatever you >> like. For *me* I was curious about some relative performance >> characteristics. No religion here. >> >> On Tue, Apr 30, 2013 at 11:59 AM, Lo?c Hoguin wrote: >>> >>> It's not very interesting unless the numbers are also available for >>> systems >>> under load. >>> >>> >>> On 04/30/2013 06:57 PM, Jeremy Ong wrote: >>>> >>>> >>>> I'd be very interested if we got a wiki going on one of these projects >>>> with community updated numbers of various benchmark runs. >>>> >>>> For example: >>>> >>>> keylists vs orddicts vs dicts >>>> fold vs recursion >>>> fibonacci >>>> all the other various standard benchmarks >>>> >>>> On Tue, Apr 30, 2013 at 9:42 AM, Garrett Smith wrote: >>>>> >>>>> >>>>> On Tue, Apr 30, 2013 at 9:35 AM, Michael Truog >>>>> wrote: >>>>>> >>>>>> >>>>>> On 04/30/2013 06:44 AM, Garrett Smith wrote: >>>>>>> >>>>>>> >>>>>>> This is not an announcement of anything -- but [ANN] seems to flag >>>>>>> "something I can maybe use" which does apply in this case :) >>>>>>> >>>>>>> Occasionally I wonder, "what's faster"? It's not often, but it >>>>>>> happens. >>>>>>> >>>>>>> I've found the best way to answer this is to measure things. >>>>>>> >>>>>>> So I have this silly project: >>>>>>> >>>>>>> https://github.com/gar1t/erlang-bench >>>>>>> >>>>>>> It's not rigorous but it's simple and I can experiment quickly with >>>>>>> different implementations. My goal is just to get a sense of things >>>>>>> -- >>>>>>> not to formally prove anything. >>>>>>> >>>>>>> It's so trivial it's almost not worth sharing/reusing -- *however* it >>>>>>> may provide value as a distributed repository for what people are >>>>>>> interested in. As it's in github there's no ownership -- please feel >>>>>>> free to fork and use for your own concerns! >>>>>>> >>>>>> You might want to look at erlbench here >>>>>> https://github.com/okeuday/erlbench since it has the same basic >>>>>> purpose, and >>>>>> allows you to use different compilation methods now (through the >>>>>> makefile >>>>>> specifying an optimization level). The erlbench project is also >>>>>> ad-hoc, but >>>>>> it has been enough to produce results in the past. >>>>>> >>>>>> The other option is trying to use basho_bench here >>>>>> https://github.com/basho/basho_bench, if you are testing key/value >>>>>> storage. >>>>> >>>>> >>>>> >>>>> Yes, but you'll notice how *easy* it is to use erlang-bench, which is >>>>> nothing more than escript files with a 10 line include file. >>>>> >>>>> I'm an extraordinarily lazy person :) >>>>> >>>>> Though seriously, thanks for the references. If I was more concerned >>>>> about benchmark integrity, those might be good options -- but this is >>>>> just a sniff test approach to satisfy my curiosity about various >>>>> topics. >>>>> >>>>> Garrett >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> >>> -- >>> Lo?c Hoguin >>> Erlang Cowboy >>> Nine Nines >>> http://ninenines.eu > > > > -- > Lo?c Hoguin > > Erlang Cowboy > Nine Nines > http://ninenines.eu From g@REDACTED Tue Apr 30 19:20:16 2013 From: g@REDACTED (Garrett Smith) Date: Tue, 30 Apr 2013 12:20:16 -0500 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: <517FF9FC.3040204@ninenines.eu> References: <517FD6B0.3070505@gmail.com> <517FF868.7030709@ninenines.eu> <517FF9FC.3040204@ninenines.eu> Message-ID: You have a point. Though "load" would be pretty arbitrary -- I could see some load-xxx.escript files that did terrible things while the other scripts were run. But it would depend on what else is running on your system. And your system architecture. And your system hardware. Waiiit a minute, this just got real complicated! So, don't make decisions unless you have reasonably complete information, whatever that means :) On Tue, Apr 30, 2013 at 12:06 PM, Lo?c Hoguin wrote: > I'm sure *you* know these results might not be true for systems under load, > but others might not. Not having both could cause more harm than good, and > would not be interesting in that sense. > > > On 04/30/2013 07:03 PM, Garrett Smith wrote: >> >> Not very interesting to you. Of course you can write whatever you >> like. For *me* I was curious about some relative performance >> characteristics. No religion here. >> >> On Tue, Apr 30, 2013 at 11:59 AM, Lo?c Hoguin wrote: >>> >>> It's not very interesting unless the numbers are also available for >>> systems >>> under load. >>> >>> >>> On 04/30/2013 06:57 PM, Jeremy Ong wrote: >>>> >>>> >>>> I'd be very interested if we got a wiki going on one of these projects >>>> with community updated numbers of various benchmark runs. >>>> >>>> For example: >>>> >>>> keylists vs orddicts vs dicts >>>> fold vs recursion >>>> fibonacci >>>> all the other various standard benchmarks >>>> >>>> On Tue, Apr 30, 2013 at 9:42 AM, Garrett Smith wrote: >>>>> >>>>> >>>>> On Tue, Apr 30, 2013 at 9:35 AM, Michael Truog >>>>> wrote: >>>>>> >>>>>> >>>>>> On 04/30/2013 06:44 AM, Garrett Smith wrote: >>>>>>> >>>>>>> >>>>>>> This is not an announcement of anything -- but [ANN] seems to flag >>>>>>> "something I can maybe use" which does apply in this case :) >>>>>>> >>>>>>> Occasionally I wonder, "what's faster"? It's not often, but it >>>>>>> happens. >>>>>>> >>>>>>> I've found the best way to answer this is to measure things. >>>>>>> >>>>>>> So I have this silly project: >>>>>>> >>>>>>> https://github.com/gar1t/erlang-bench >>>>>>> >>>>>>> It's not rigorous but it's simple and I can experiment quickly with >>>>>>> different implementations. My goal is just to get a sense of things >>>>>>> -- >>>>>>> not to formally prove anything. >>>>>>> >>>>>>> It's so trivial it's almost not worth sharing/reusing -- *however* it >>>>>>> may provide value as a distributed repository for what people are >>>>>>> interested in. As it's in github there's no ownership -- please feel >>>>>>> free to fork and use for your own concerns! >>>>>>> >>>>>> You might want to look at erlbench here >>>>>> https://github.com/okeuday/erlbench since it has the same basic >>>>>> purpose, and >>>>>> allows you to use different compilation methods now (through the >>>>>> makefile >>>>>> specifying an optimization level). The erlbench project is also >>>>>> ad-hoc, but >>>>>> it has been enough to produce results in the past. >>>>>> >>>>>> The other option is trying to use basho_bench here >>>>>> https://github.com/basho/basho_bench, if you are testing key/value >>>>>> storage. >>>>> >>>>> >>>>> >>>>> Yes, but you'll notice how *easy* it is to use erlang-bench, which is >>>>> nothing more than escript files with a 10 line include file. >>>>> >>>>> I'm an extraordinarily lazy person :) >>>>> >>>>> Though seriously, thanks for the references. If I was more concerned >>>>> about benchmark integrity, those might be good options -- but this is >>>>> just a sniff test approach to satisfy my curiosity about various >>>>> topics. >>>>> >>>>> Garrett >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> >>> -- >>> Lo?c Hoguin >>> Erlang Cowboy >>> Nine Nines >>> http://ninenines.eu > > > > -- > Lo?c Hoguin > > Erlang Cowboy > Nine Nines > http://ninenines.eu From mononcqc@REDACTED Tue Apr 30 19:35:09 2013 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 30 Apr 2013 13:35:09 -0400 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: References: <517FD6B0.3070505@gmail.com> <517FF868.7030709@ninenines.eu> <517FF9FC.3040204@ninenines.eu> Message-ID: <20130430173507.GA23239@ferdair.local> A rather simple trick to simulate load is to have a few busy-looping processes on the VM, and give it a higher priority than the rest. It should eat time more than anything else and make it look like there's a lot of contention for schedulers, with reduced execution time for what you want to measure. I've used it a few times, although I am not 100% sure it's super representative o a real system being under load. On 04/30, Garrett Smith wrote: > You have a point. Though "load" would be pretty arbitrary -- I could > see some load-xxx.escript files that did terrible things while the > other scripts were run. But it would depend on what else is running on > your system. And your system architecture. And your system hardware. > Waiiit a minute, this just got real complicated! > > So, don't make decisions unless you have reasonably complete > information, whatever that means :) > > On Tue, Apr 30, 2013 at 12:06 PM, Lo?c Hoguin wrote: > > I'm sure *you* know these results might not be true for systems under load, > > but others might not. Not having both could cause more harm than good, and > > would not be interesting in that sense. > > > > > > On 04/30/2013 07:03 PM, Garrett Smith wrote: > >> > >> Not very interesting to you. Of course you can write whatever you > >> like. For *me* I was curious about some relative performance > >> characteristics. No religion here. > >> > >> On Tue, Apr 30, 2013 at 11:59 AM, Lo?c Hoguin wrote: > >>> > >>> It's not very interesting unless the numbers are also available for > >>> systems > >>> under load. > >>> > >>> > >>> On 04/30/2013 06:57 PM, Jeremy Ong wrote: > >>>> > >>>> > >>>> I'd be very interested if we got a wiki going on one of these projects > >>>> with community updated numbers of various benchmark runs. > >>>> > >>>> For example: > >>>> > >>>> keylists vs orddicts vs dicts > >>>> fold vs recursion > >>>> fibonacci > >>>> all the other various standard benchmarks > >>>> > >>>> On Tue, Apr 30, 2013 at 9:42 AM, Garrett Smith wrote: > >>>>> > >>>>> > >>>>> On Tue, Apr 30, 2013 at 9:35 AM, Michael Truog > >>>>> wrote: > >>>>>> > >>>>>> > >>>>>> On 04/30/2013 06:44 AM, Garrett Smith wrote: > >>>>>>> > >>>>>>> > >>>>>>> This is not an announcement of anything -- but [ANN] seems to flag > >>>>>>> "something I can maybe use" which does apply in this case :) > >>>>>>> > >>>>>>> Occasionally I wonder, "what's faster"? It's not often, but it > >>>>>>> happens. > >>>>>>> > >>>>>>> I've found the best way to answer this is to measure things. > >>>>>>> > >>>>>>> So I have this silly project: > >>>>>>> > >>>>>>> https://github.com/gar1t/erlang-bench > >>>>>>> > >>>>>>> It's not rigorous but it's simple and I can experiment quickly with > >>>>>>> different implementations. My goal is just to get a sense of things > >>>>>>> -- > >>>>>>> not to formally prove anything. > >>>>>>> > >>>>>>> It's so trivial it's almost not worth sharing/reusing -- *however* it > >>>>>>> may provide value as a distributed repository for what people are > >>>>>>> interested in. As it's in github there's no ownership -- please feel > >>>>>>> free to fork and use for your own concerns! > >>>>>>> > >>>>>> You might want to look at erlbench here > >>>>>> https://github.com/okeuday/erlbench since it has the same basic > >>>>>> purpose, and > >>>>>> allows you to use different compilation methods now (through the > >>>>>> makefile > >>>>>> specifying an optimization level). The erlbench project is also > >>>>>> ad-hoc, but > >>>>>> it has been enough to produce results in the past. > >>>>>> > >>>>>> The other option is trying to use basho_bench here > >>>>>> https://github.com/basho/basho_bench, if you are testing key/value > >>>>>> storage. > >>>>> > >>>>> > >>>>> > >>>>> Yes, but you'll notice how *easy* it is to use erlang-bench, which is > >>>>> nothing more than escript files with a 10 line include file. > >>>>> > >>>>> I'm an extraordinarily lazy person :) > >>>>> > >>>>> Though seriously, thanks for the references. If I was more concerned > >>>>> about benchmark integrity, those might be good options -- but this is > >>>>> just a sniff test approach to satisfy my curiosity about various > >>>>> topics. > >>>>> > >>>>> Garrett > >>>>> _______________________________________________ > >>>>> erlang-questions mailing list > >>>>> erlang-questions@REDACTED > >>>>> http://erlang.org/mailman/listinfo/erlang-questions > >>>> > >>>> > >>>> _______________________________________________ > >>>> erlang-questions mailing list > >>>> erlang-questions@REDACTED > >>>> http://erlang.org/mailman/listinfo/erlang-questions > >>>> > >>> > >>> > >>> -- > >>> Lo?c Hoguin > >>> Erlang Cowboy > >>> Nine Nines > >>> http://ninenines.eu > > > > > > > > -- > > Lo?c Hoguin > > > > Erlang Cowboy > > Nine Nines > > http://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From g@REDACTED Tue Apr 30 19:36:30 2013 From: g@REDACTED (Garrett Smith) Date: Tue, 30 Apr 2013 12:36:30 -0500 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: <20130430173507.GA23239@ferdair.local> References: <517FD6B0.3070505@gmail.com> <517FF868.7030709@ninenines.eu> <517FF9FC.3040204@ninenines.eu> <20130430173507.GA23239@ferdair.local> Message-ID: I'm curious what the OTP team does (in the lab) to vet various implementation options. On Tue, Apr 30, 2013 at 12:35 PM, Fred Hebert wrote: > A rather simple trick to simulate load is to have a few busy-looping > processes on the VM, and give it a higher priority than the rest. It > should eat time more than anything else and make it look like there's a > lot of contention for schedulers, with reduced execution time for what > you want to measure. > > I've used it a few times, although I am not 100% sure it's super > representative o a real system being under load. > > On 04/30, Garrett Smith wrote: >> You have a point. Though "load" would be pretty arbitrary -- I could >> see some load-xxx.escript files that did terrible things while the >> other scripts were run. But it would depend on what else is running on >> your system. And your system architecture. And your system hardware. >> Waiiit a minute, this just got real complicated! >> >> So, don't make decisions unless you have reasonably complete >> information, whatever that means :) >> >> On Tue, Apr 30, 2013 at 12:06 PM, Lo?c Hoguin wrote: >> > I'm sure *you* know these results might not be true for systems under load, >> > but others might not. Not having both could cause more harm than good, and >> > would not be interesting in that sense. >> > >> > >> > On 04/30/2013 07:03 PM, Garrett Smith wrote: >> >> >> >> Not very interesting to you. Of course you can write whatever you >> >> like. For *me* I was curious about some relative performance >> >> characteristics. No religion here. >> >> >> >> On Tue, Apr 30, 2013 at 11:59 AM, Lo?c Hoguin wrote: >> >>> >> >>> It's not very interesting unless the numbers are also available for >> >>> systems >> >>> under load. >> >>> >> >>> >> >>> On 04/30/2013 06:57 PM, Jeremy Ong wrote: >> >>>> >> >>>> >> >>>> I'd be very interested if we got a wiki going on one of these projects >> >>>> with community updated numbers of various benchmark runs. >> >>>> >> >>>> For example: >> >>>> >> >>>> keylists vs orddicts vs dicts >> >>>> fold vs recursion >> >>>> fibonacci >> >>>> all the other various standard benchmarks >> >>>> >> >>>> On Tue, Apr 30, 2013 at 9:42 AM, Garrett Smith wrote: >> >>>>> >> >>>>> >> >>>>> On Tue, Apr 30, 2013 at 9:35 AM, Michael Truog >> >>>>> wrote: >> >>>>>> >> >>>>>> >> >>>>>> On 04/30/2013 06:44 AM, Garrett Smith wrote: >> >>>>>>> >> >>>>>>> >> >>>>>>> This is not an announcement of anything -- but [ANN] seems to flag >> >>>>>>> "something I can maybe use" which does apply in this case :) >> >>>>>>> >> >>>>>>> Occasionally I wonder, "what's faster"? It's not often, but it >> >>>>>>> happens. >> >>>>>>> >> >>>>>>> I've found the best way to answer this is to measure things. >> >>>>>>> >> >>>>>>> So I have this silly project: >> >>>>>>> >> >>>>>>> https://github.com/gar1t/erlang-bench >> >>>>>>> >> >>>>>>> It's not rigorous but it's simple and I can experiment quickly with >> >>>>>>> different implementations. My goal is just to get a sense of things >> >>>>>>> -- >> >>>>>>> not to formally prove anything. >> >>>>>>> >> >>>>>>> It's so trivial it's almost not worth sharing/reusing -- *however* it >> >>>>>>> may provide value as a distributed repository for what people are >> >>>>>>> interested in. As it's in github there's no ownership -- please feel >> >>>>>>> free to fork and use for your own concerns! >> >>>>>>> >> >>>>>> You might want to look at erlbench here >> >>>>>> https://github.com/okeuday/erlbench since it has the same basic >> >>>>>> purpose, and >> >>>>>> allows you to use different compilation methods now (through the >> >>>>>> makefile >> >>>>>> specifying an optimization level). The erlbench project is also >> >>>>>> ad-hoc, but >> >>>>>> it has been enough to produce results in the past. >> >>>>>> >> >>>>>> The other option is trying to use basho_bench here >> >>>>>> https://github.com/basho/basho_bench, if you are testing key/value >> >>>>>> storage. >> >>>>> >> >>>>> >> >>>>> >> >>>>> Yes, but you'll notice how *easy* it is to use erlang-bench, which is >> >>>>> nothing more than escript files with a 10 line include file. >> >>>>> >> >>>>> I'm an extraordinarily lazy person :) >> >>>>> >> >>>>> Though seriously, thanks for the references. If I was more concerned >> >>>>> about benchmark integrity, those might be good options -- but this is >> >>>>> just a sniff test approach to satisfy my curiosity about various >> >>>>> topics. >> >>>>> >> >>>>> Garrett >> >>>>> _______________________________________________ >> >>>>> erlang-questions mailing list >> >>>>> erlang-questions@REDACTED >> >>>>> http://erlang.org/mailman/listinfo/erlang-questions >> >>>> >> >>>> >> >>>> _______________________________________________ >> >>>> erlang-questions mailing list >> >>>> erlang-questions@REDACTED >> >>>> http://erlang.org/mailman/listinfo/erlang-questions >> >>>> >> >>> >> >>> >> >>> -- >> >>> Lo?c Hoguin >> >>> Erlang Cowboy >> >>> Nine Nines >> >>> http://ninenines.eu >> > >> > >> > >> > -- >> > Lo?c Hoguin >> > >> > Erlang Cowboy >> > Nine Nines >> > http://ninenines.eu >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From mjtruog@REDACTED Tue Apr 30 19:52:26 2013 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 30 Apr 2013 10:52:26 -0700 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: References: <517FD6B0.3070505@gmail.com> Message-ID: <518004DA.50402@gmail.com> I like the idea of having a site like http://benchmarksgame.alioth.debian.org/ that is more specific to Erlang. Ideally we could test fun stuff, like comparing actors in Scala/Akka and JActor (https://github.com/laforge49/JActor) with Erlang. However, to keep stuff transparent and useful we would need to document hardware and avoid short runs that skew results. To have a serious site requires a monetary commitment and I am not sure who would be willing to pursue that (Ericsson might be the best organization to pursue this and show how Erlang shines). Either way, having the source code available is a requirement for making sure the tests are repeatable, so I think we can pursue that direction in the meantime. One other benchmarking project I am aware of is here: http://www.softlab.ntua.gr/release/bencherl/index.html So there is probably a lot of individual testing efforts that could be combined, to provide definitive data for decision making. That probably requires discussion and coordination at a higher-level. On 04/30/2013 10:02 AM, Garrett Smith wrote: > This is sort of my thinking with the github project: > > - Better than a wiki because you can run the files > - Easily forked and modified -- so people can share and modify code trivially > - Distributed is better than centralized! > > There's probably a temptation to view anything like this as > authoritative -- it of course is not and cannot be (e.g. the three > stupid files I have are totally arbitrary). I think distributed source > like this is a good push-back for that temptation. > > On Tue, Apr 30, 2013 at 11:57 AM, Jeremy Ong wrote: >> I'd be very interested if we got a wiki going on one of these projects >> with community updated numbers of various benchmark runs. >> >> For example: >> >> keylists vs orddicts vs dicts >> fold vs recursion >> fibonacci >> all the other various standard benchmarks >> >> On Tue, Apr 30, 2013 at 9:42 AM, Garrett Smith wrote: >>> On Tue, Apr 30, 2013 at 9:35 AM, Michael Truog wrote: >>>> On 04/30/2013 06:44 AM, Garrett Smith wrote: >>>>> This is not an announcement of anything -- but [ANN] seems to flag >>>>> "something I can maybe use" which does apply in this case :) >>>>> >>>>> Occasionally I wonder, "what's faster"? It's not often, but it happens. >>>>> >>>>> I've found the best way to answer this is to measure things. >>>>> >>>>> So I have this silly project: >>>>> >>>>> https://github.com/gar1t/erlang-bench >>>>> >>>>> It's not rigorous but it's simple and I can experiment quickly with >>>>> different implementations. My goal is just to get a sense of things -- >>>>> not to formally prove anything. >>>>> >>>>> It's so trivial it's almost not worth sharing/reusing -- *however* it >>>>> may provide value as a distributed repository for what people are >>>>> interested in. As it's in github there's no ownership -- please feel >>>>> free to fork and use for your own concerns! >>>>> >>>> You might want to look at erlbench here https://github.com/okeuday/erlbench since it has the same basic purpose, and allows you to use different compilation methods now (through the makefile specifying an optimization level). The erlbench project is also ad-hoc, but it has been enough to produce results in the past. >>>> >>>> The other option is trying to use basho_bench here https://github.com/basho/basho_bench, if you are testing key/value storage. >>> Yes, but you'll notice how *easy* it is to use erlang-bench, which is >>> nothing more than escript files with a 10 line include file. >>> >>> I'm an extraordinarily lazy person :) >>> >>> Though seriously, thanks for the references. If I was more concerned >>> about benchmark integrity, those might be good options -- but this is >>> just a sniff test approach to satisfy my curiosity about various >>> topics. >>> >>> Garrett >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > . > From eriksoe@REDACTED Tue Apr 30 20:11:57 2013 From: eriksoe@REDACTED (=?ISO-8859-1?Q?Erik_S=F8e_S=F8rensen?=) Date: Tue, 30 Apr 2013 20:11:57 +0200 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: <517FD6B0.3070505@gmail.com> References: <517FD6B0.3070505@gmail.com> Message-ID: I think also ecriterion should be mentioned - it does an honest attempt at verifying the reliability of the results. Den 30/04/2013 16.35 skrev "Michael Truog" : > On 04/30/2013 06:44 AM, Garrett Smith wrote: > > This is not an announcement of anything -- but [ANN] seems to flag > > "something I can maybe use" which does apply in this case :) > > > > Occasionally I wonder, "what's faster"? It's not often, but it happens. > > > > I've found the best way to answer this is to measure things. > > > > So I have this silly project: > > > > https://github.com/gar1t/erlang-bench > > > > It's not rigorous but it's simple and I can experiment quickly with > > different implementations. My goal is just to get a sense of things -- > > not to formally prove anything. > > > > It's so trivial it's almost not worth sharing/reusing -- *however* it > > may provide value as a distributed repository for what people are > > interested in. As it's in github there's no ownership -- please feel > > free to fork and use for your own concerns! > > > > Garrett > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > You might want to look at erlbench here > https://github.com/okeuday/erlbench since it has the same basic purpose, > and allows you to use different compilation methods now (through the > makefile specifying an optimization level). The erlbench project is also > ad-hoc, but it has been enough to produce results in the past. > > The other option is trying to use basho_bench here > https://github.com/basho/basho_bench, if you are testing key/value > storage. > > - Michael > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Tue Apr 30 20:32:56 2013 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 30 Apr 2013 11:32:56 -0700 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: References: <517FD6B0.3070505@gmail.com> Message-ID: <51800E58.60208@gmail.com> Thank you for mentioning ecriterion (https://github.com/jlouis/ecriterion/), I hadn't seen that previously. It doesn't seem to have any tests or analysis yet, but I think it would be great if we could get a common Erlang testing project going. On 04/30/2013 11:11 AM, Erik S?e S?rensen wrote: > > I think also ecriterion should be mentioned - it does an honest attempt at verifying the reliability of the results. > > Den 30/04/2013 16.35 skrev "Michael Truog" >: > > On 04/30/2013 06:44 AM, Garrett Smith wrote: > > This is not an announcement of anything -- but [ANN] seems to flag > > "something I can maybe use" which does apply in this case :) > > > > Occasionally I wonder, "what's faster"? It's not often, but it happens. > > > > I've found the best way to answer this is to measure things. > > > > So I have this silly project: > > > > https://github.com/gar1t/erlang-bench > > > > It's not rigorous but it's simple and I can experiment quickly with > > different implementations. My goal is just to get a sense of things -- > > not to formally prove anything. > > > > It's so trivial it's almost not worth sharing/reusing -- *however* it > > may provide value as a distributed repository for what people are > > interested in. As it's in github there's no ownership -- please feel > > free to fork and use for your own concerns! > > > > Garrett > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > You might want to look at erlbench here https://github.com/okeuday/erlbench since it has the same basic purpose, and allows you to use different compilation methods now (through the makefile specifying an optimization level). The erlbench project is also ad-hoc, but it has been enough to produce results in the past. > > The other option is trying to use basho_bench here https://github.com/basho/basho_bench, if you are testing key/value storage. > > - Michael > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Tue Apr 30 21:17:22 2013 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 30 Apr 2013 21:17:22 +0200 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: <51800E58.60208@gmail.com> References: <517FD6B0.3070505@gmail.com> <51800E58.60208@gmail.com> Message-ID: <518018C2.4000308@gmail.com> And here's my unfinished benchmarking project: https://github.com/richcarl/berk /Richard On 2013-04-30 20:32, Michael Truog wrote: > Thank you for mentioning ecriterion > (https://github.com/jlouis/ecriterion/), I hadn't seen that previously. > It doesn't seem to have any tests or analysis yet, but I think it would > be great if we could get a common Erlang testing project going. > > On 04/30/2013 11:11 AM, Erik S?e S?rensen wrote: >> >> I think also ecriterion should be mentioned - it does an honest >> attempt at verifying the reliability of the results. >> >> Den 30/04/2013 16.35 skrev "Michael Truog" > >: >> >> On 04/30/2013 06:44 AM, Garrett Smith wrote: >> > This is not an announcement of anything -- but [ANN] seems to flag >> > "something I can maybe use" which does apply in this case :) >> > >> > Occasionally I wonder, "what's faster"? It's not often, but it >> happens. >> > >> > I've found the best way to answer this is to measure things. >> > >> > So I have this silly project: >> > >> > https://github.com/gar1t/erlang-bench >> > >> > It's not rigorous but it's simple and I can experiment quickly with >> > different implementations. My goal is just to get a sense of >> things -- >> > not to formally prove anything. >> > >> > It's so trivial it's almost not worth sharing/reusing -- >> *however* it >> > may provide value as a distributed repository for what people are >> > interested in. As it's in github there's no ownership -- please feel >> > free to fork and use for your own concerns! >> > >> > Garrett >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> You might want to look at erlbench here >> https://github.com/okeuday/erlbench since it has the same basic >> purpose, and allows you to use different compilation methods now >> (through the makefile specifying an optimization level). The >> erlbench project is also ad-hoc, but it has been enough to produce >> results in the past. >> >> The other option is trying to use basho_bench here >> https://github.com/basho/basho_bench, if you are testing key/value >> storage. >> >> - Michael >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From desired.mta@REDACTED Tue Apr 30 21:42:35 2013 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Tue, 30 Apr 2013 20:42:35 +0100 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: <518018C2.4000308@gmail.com> References: <517FD6B0.3070505@gmail.com> <51800E58.60208@gmail.com> <518018C2.4000308@gmail.com> Message-ID: On Tue, Apr 30, 2013 at 8:17 PM, Richard Carlsson wrote: > And here's my unfinished benchmarking project: > > https://github.com/richcarl/berk > This looks quite big and unusual compared to others. Can you explain what you calibrate, and what is the purpose of sending a message many times to a dead process? Does it have some well-defined behavior in the virtual machine? It was not obvious from a 5-minute peek at the source, so an answer might save time for others. Thanks, Motiejus From noah@REDACTED Tue Apr 30 21:54:34 2013 From: noah@REDACTED (Noah Diewald) Date: Tue, 30 Apr 2013 14:54:34 -0500 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: <518004DA.50402@gmail.com> References: <517FD6B0.3070505@gmail.com> <518004DA.50402@gmail.com> Message-ID: <5180217A.8080106@diewald.me> I've always wished that they had a network of computers as one of the options instead of just single computers. On 04/30/2013 12:52 PM, Michael Truog wrote: > I like the idea of having a site like http://benchmarksgame.alioth.debian.org/ that is more specific to Erlang. Ideally we could test fun stuff, like comparing actors in Scala/Akka and JActor (https://github.com/laforge49/JActor) with Erlang. However, to keep stuff transparent and useful we would need to document hardware and avoid short runs that skew results. To have a serious site requires a monetary commitment and I am not sure who would be willing to pursue that (Ericsson might be the best organization to pursue this and show how Erlang shines). Either way, having the source code available is a requirement for making sure the tests are repeatable, so I think we can pursue that direction in the meantime. > > One other benchmarking project I am aware of is here: > http://www.softlab.ntua.gr/release/bencherl/index.html > > So there is probably a lot of individual testing efforts that could be combined, to provide definitive data for decision making. That probably requires discussion and coordination at a higher-level. > > On 04/30/2013 10:02 AM, Garrett Smith wrote: >> This is sort of my thinking with the github project: >> >> - Better than a wiki because you can run the files >> - Easily forked and modified -- so people can share and modify code trivially >> - Distributed is better than centralized! >> >> There's probably a temptation to view anything like this as >> authoritative -- it of course is not and cannot be (e.g. the three >> stupid files I have are totally arbitrary). I think distributed source >> like this is a good push-back for that temptation. >> >> On Tue, Apr 30, 2013 at 11:57 AM, Jeremy Ong wrote: >>> I'd be very interested if we got a wiki going on one of these projects >>> with community updated numbers of various benchmark runs. >>> >>> For example: >>> >>> keylists vs orddicts vs dicts >>> fold vs recursion >>> fibonacci >>> all the other various standard benchmarks >>> >>> On Tue, Apr 30, 2013 at 9:42 AM, Garrett Smith wrote: >>>> On Tue, Apr 30, 2013 at 9:35 AM, Michael Truog wrote: >>>>> On 04/30/2013 06:44 AM, Garrett Smith wrote: >>>>>> This is not an announcement of anything -- but [ANN] seems to flag >>>>>> "something I can maybe use" which does apply in this case :) >>>>>> >>>>>> Occasionally I wonder, "what's faster"? It's not often, but it happens. >>>>>> >>>>>> I've found the best way to answer this is to measure things. >>>>>> >>>>>> So I have this silly project: >>>>>> >>>>>> https://github.com/gar1t/erlang-bench >>>>>> >>>>>> It's not rigorous but it's simple and I can experiment quickly with >>>>>> different implementations. My goal is just to get a sense of things -- >>>>>> not to formally prove anything. >>>>>> >>>>>> It's so trivial it's almost not worth sharing/reusing -- *however* it >>>>>> may provide value as a distributed repository for what people are >>>>>> interested in. As it's in github there's no ownership -- please feel >>>>>> free to fork and use for your own concerns! >>>>>> >>>>> You might want to look at erlbench here https://github.com/okeuday/erlbench since it has the same basic purpose, and allows you to use different compilation methods now (through the makefile specifying an optimization level). The erlbench project is also ad-hoc, but it has been enough to produce results in the past. >>>>> >>>>> The other option is trying to use basho_bench here https://github.com/basho/basho_bench, if you are testing key/value storage. >>>> Yes, but you'll notice how *easy* it is to use erlang-bench, which is >>>> nothing more than escript files with a 10 line include file. >>>> >>>> I'm an extraordinarily lazy person :) >>>> >>>> Though seriously, thanks for the references. If I was more concerned >>>> about benchmark integrity, those might be good options -- but this is >>>> just a sniff test approach to satisfy my curiosity about various >>>> topics. >>>> >>>> Garrett >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >> . >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 555 bytes Desc: OpenPGP digital signature URL: From dmercer@REDACTED Tue Apr 30 21:57:48 2013 From: dmercer@REDACTED (David Mercer) Date: Tue, 30 Apr 2013 14:57:48 -0500 Subject: [erlang-questions] [ANN] qdate - Unified Date Formatting/Parsing and Timezone Management In-Reply-To: <0E6ACAB0-6A5B-4C85-84E2-CC7B590317B6@dmitriid.com> References: <0E6ACAB0-6A5B-4C85-84E2-CC7B590317B6@dmitriid.com> Message-ID: <012101ce45dc$fda95de0$f8fc19a0$@gmail.com> Enh, I would tend to prefer Jesse?s qdate because it is pure Erlang. Cheers, DBM From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Dmitrii Dimandt Sent: Tuesday, April 30, 2013 01:50 To: Jesse Gumm Cc: Erlang Questions Subject: Re: [erlang-questions] [ANN] qdate - Unified Date Formatting/Parsing and Timezone Management Have you also looked at tempo, https://github.com/selectel/tempo ? Your library and tempo could probably be combined On Apr 30, 2013, at 07:09 , Jesse Gumm wrote: Hello Erlangers, I'm proud to announce the beta release (v0.1.0) of "qdate" - a unified date parsing/formatting and timezone management utility for Erlang. Blog post describing my thoughts that went into its development http://sigma-star.com/blog/post/qdate Source code on GitHub (includes a README that covers all the functionality) https://github.com/choptastic/qdate At its core, qdate is basically a wrapper around ec_date (fork of dh_date and part of erlware_commons) and erlang_localtime. But it's much more than just a wrapper. It's the complete date and time utility I feel is greatly needed in the Erlang world (since I found myself needing it). Briefly, here's what it does: * Easily convert from any date format to any other date format: datetime(), now(), unix timestamp, and formatted date strings (list and binaries) are all acceptable: > qdate:to_string("Y-m-d g:ia", Date). "2012-12-21 5:51pm" > qdate:to_string(<<"Y-m-d g:ia">>, Date). <<"2012-12-21 5:51pm">> > qdate:to_unixtime(Date). 1356112260 > qdate:to_date(Date). {{2012,12,21},{17,51,0}} > qdate:to_now(Date). {1356,112260,0} % Note, that the argument `Date` can be anything % that's a Date/Time Format (datetime Tuple, now tuple, % Integer (unixtime), or string ("2012-12-21 5:51pm") * Extends ec_date's formatting functions to include PHP's timezone-related characters: > qdate:to_string("Y-m-d g:ia T", Date) "2012-12-21 5:51pm CST" * Able to pre-parse timezones from text strings, and convert from one timezone to another: % Read Timezone from string, convert to datetime in EST > qdate:to_date("2012-12-21 5:51pm CST","EST"). {{2012,12,21},{18,51,0}} % Read timezone from string, convert to a string in EST > qdate:to_string("n/j/Y g:ia T","2012-12-21 5:51pm CST", "EST"). "12/21/2012 6:51pm EST" * Register timezones with the qdate server by a key (to make a timezone alias) or by pid(): %% Set timezone for the current process to GMT > qdate:set_timezone("GMT"). %% Set timezone for some_key to HKT > qdate:set_timezone(some_key, "HKT"). * Register custom formatting strings with the qdate server, for quick reference. > qdate:register_format(long_date, "l, F jS, Y g:i A T"). ok > qdate:to_string(long_date,1356133860). "Friday, December 21st, 2012 11:51 PM GMT" * Register custom parsing functions with the qdate server, to be transparently used for conversion. > qdate:register_parser(weird_format, fun parse_esoteric_format/1). ok > qdate:to_date("20121221.115100"). {{2012,12,21},{11,51,00}} * Attempt to maintain API compatibility with ec_date by supporting qdate:parse, qdate:nparse, and qdate:format. I hope some of you find this useful in your apps. Any issues, comments, complaints, bug reports, whatever, feel free to post in this thread, make a github issue, shoot me a message on twitter (@jessegumm) or email me directly. Thanks, and I hope everyone has a glorious day or evening or whatever timezone you happen to be in. -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmercer@REDACTED Tue Apr 30 22:01:19 2013 From: dmercer@REDACTED (David Mercer) Date: Tue, 30 Apr 2013 15:01:19 -0500 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: <517FF868.7030709@ninenines.eu> References: <517FD6B0.3070505@gmail.com> <517FF868.7030709@ninenines.eu> Message-ID: <013101ce45dd$7b3ff0c0$71bfd240$@gmail.com> On Tuesday, April 30, 2013, Lo?c Hoguin wrote: > It's not very interesting unless the numbers are also available for > systems under load. Wouldn?t you want benchmarks to eliminate confounding variables rather than add them? If you wanted to benchmark under load, however, why not just put the system under the load you want to test under and then run your benchmarks. I would not expect the benchmarking framework to provide that load for you. Cheers, DBM From carlsson.richard@REDACTED Tue Apr 30 22:02:22 2013 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 30 Apr 2013 22:02:22 +0200 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: References: <517FD6B0.3070505@gmail.com> <51800E58.60208@gmail.com> <518018C2.4000308@gmail.com> Message-ID: <5180234E.3050108@gmail.com> On 2013-04-30 21:42, Motiejus Jak?tys wrote: > On Tue, Apr 30, 2013 at 8:17 PM, Richard Carlsson > wrote: >> And here's my unfinished benchmarking project: >> >> https://github.com/richcarl/berk >> > > This looks quite big and unusual compared to others. > > Can you explain what you calibrate, I started out wanting a really stable core measurement loop, and a way to calibrate the system under test so that it could automatically set good defaults for number of iterations, and be aware of the clock precision and typical variance. As I went on, I started abstracting more and more, and some ideas (like the "berps" - bogus erlang reductions per second) got left as rough sketches, and there's nothing yet that actually saves the result of a calibration and reuses it. The parts that are working well are the runner/1, gather/1, and stats/1 functions, and the functions run/2, run_for/2, and calibrate/0, which you can study to understand how the runner and gather functions work. There's still some tweaking to be done - the calibration doesn't always converge as well as I'd like it. > and what is the purpose of sending > a message many times to a dead process? Does it have some well-defined > behavior in the virtual machine? No, it was just a way to set up a calibration loop that did something very cheap but that the compiler couldn't optimize away. But that part isn't very exciting - I'm not using it for anything yet. I just dropped all of the code in a single module to keep it simple at this stage. /Richard From max.lapshin@REDACTED Tue Apr 30 22:54:01 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 1 May 2013 00:54:01 +0400 Subject: [erlang-questions] Packaging commercial software with erlang Message-ID: Are there any caveats with Erlang licensing (EPL) if I want to distribute commercial application with binary erlang in the same package? -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Tue Apr 30 23:26:33 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 30 Apr 2013 22:26:33 +0100 Subject: [erlang-questions] [ANN] Silly benchmarking In-Reply-To: <5180217A.8080106@diewald.me> References: <517FD6B0.3070505@gmail.com> <518004DA.50402@gmail.com> <5180217A.8080106@diewald.me> Message-ID: <9CF4EC7E-BAB3-4C03-8ED2-8249C06C756F@gmail.com> On 30 Apr 2013, at 20:54, Noah Diewald wrote: > I've always wished that they had a network of computers as one of the options > instead of just single computers. > https://github.com/nebularis/systest exists to address the issue of running tests against multiple nodes, although it's still in quite an early stage (no remote nodes via ssh support yet) and already in need of some refactoring. It currently supports running in standalone (or shell) mode and integrating with common_test (via the ct_hooks mechanism) and its goal is to manage test resources (e.g., a node, a cluster of nodes, etc) during a test run. There's nothing to prevent this from being used for benchmarks instead of integration tests. You can see example (test) usage in https://github.com/rabbitmq/rabbitmq-test/tree/bug25421/multi-node, common_test integration via https://github.com/rabbitmq/rabbitmq-test/tree/bug25421/multi-node/test and SysTest API calls in https://github.com/rabbitmq/rabbitmq-test/blob/bug25421/multi-node/src/rabbit_ha_test_utils.erl There's plenty of things wrong with the current implementation though - put together in a bit of a rush - if you're interested in the dirty laundry and/or state of usefulness, https://github.com/nebularis/systest/wiki/The-Big-Refactor outlines some of the existing (structural) things that I'm in the process of fixing. Next big ticket feature is an ssh based layer to manage remote nodes during test runs. Nonetheless, we've found and fixed plenty of bugs using it to test clustering/HA scenarios. Suggestions, comments and/or pull requests are welcome, though do bear in mind I'm going to rip up large swathes of it and rewrite them very soon. :) Cheers, Tim -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 235 bytes Desc: Message signed with OpenPGP using GPGMail URL: